Exporting data out of WL, deleting old files
Author: ruediger78
Creation Date: 8/1/2013 11:00 PM
profile picture

ruediger78

#1
I want to export data out of Wealth lab. I fould the Wiki article "Exporting data out of WL6 to ASCII and binary files" already. Now before exporting the data, I want to check if there is an old file still there. If so I want to delete. I dont know how to program it. If I put the File.Delete in the execute bracket it deletes the file every time it opens a new stock in the Dataset. But the file should contain the data of the whole dataset.
profile picture

Eugene

#2
Please clarify or rephrase your requirements.

1. If you added a File.Delete, why wonder that it deletes the file every time it opens a new stock in the Dataset?
2. What does it mean, that "the file should contain the data of the whole dataset"?
profile picture

ruediger78

#3
Ok, I will try to explain it better.
I want to export the symbol, the close of the last bar and the date of the last bar of any symbol in a Dataset. I want to do this every trading day. On a new trading the old file (data.csv) should be deleted and a new one should be created. It should not take the old file and write more lines below the old lines. My code right now looks like this: With that code, I guess it deletes any time the file when a new symbol is called. So my file only contains the data of the last symbol in the dataset. But it should contain the data of every symbol in the data set.

CODE:
Please log in to see this code.
profile picture

Eugene

#4
This can be approached by 1) looping through DataSetSymbols and 2) clicking on just a single (any given) symbol.

Here you go. Let me stress it: this code should be run in single-symbol mode, not as a mutli-symbol backtest:

CODE:
Please log in to see this code.


Note: to write a file in the root (C: \) you should run Wealth-Lab with elevated privileges (as admin, by right-clicking the shortcut and selecting 'Run as admin').
profile picture

ruediger78

#5
Thank you Eugene. I have managed it.
profile picture

Carova

#6
Eugene,

Tried to use the examples above as a guide, but I have some error that prevents the numerical values from being written to the file.

CODE:
Please log in to see this code.


I am guessing the mistake is simple, but I cannot see what I am doing wrong. Help!

Vince
profile picture

KGo

#7
Try:
CODE:
Please log in to see this code.
profile picture

Carova

#8
Thanks KGo, that seems to have solved the problem. Much appreciated!

Vince
profile picture

Carova

#9
Ok, next issue... I want to skip those symbols that do not have complete overlap with the symbol with the complete history over the Date Range.

So I tried this approach (out of complete ignorance!) and it does not exclude symbols with incomplete data:

CODE:
Please log in to see this code.


but I get Runtime Synchronization error. How can I solve this problem?

Vince
profile picture

superticker

#10
QUOTE:
I get Runtime Synchronization error.

So you need to add a try{...} catch{...} construct to handle this error because the
CODE:
Please log in to see this code.

line will not catch this error. I don't think Synchronize(b) will ever return a null result.

Since you'll be replacing the Synchronize(b) step with a try...catch construct, you can now change "false" to "true" in the GetExternalSymbol(symbol, ...) line.

Some other thoughts:

1) Using StreamWriter is significantly more efficient on execution and memory management than assembling a List<string> collection.
2) If you're concatenating a string in a FOR loop, I would use StringBuilder ...Append() for this; otherwise, each iteration of the loop creates new garbage the garbage collector needs to collect. Read "The String and StringBuilder types" section in StringBuilder class for an explanation.

In practice, your file is probably small enough that items 1 and 2 may not make much difference in execution, but you should still appreciate the importance of these steps if your file was much larger.

CODE:
Please log in to see this code.
profile picture

Carova

#11
Thanks superticker for creating a script to address my feeble attempt!

My use of "b = Synchronize(b);" was my attempt to eliminate any symbol that did not have data over the total Date Range. I guess that I can manually edit out those cases. Is there anyway to add that?

I need to learn more about using the "try"/"catch" construct. This never came up in my years of using C. ;) I also will dig into the StringBuilder Class to improve my knowledge of string handling. I have not had much need for it in WL over the years, but may in the future, so it will definitely be helpful.

I appreciate your help!

Vince
profile picture

Eugene

#12
QUOTE:
I also will dig into the StringBuilder Class to improve my knowledge of string handling

+1
Like superticker points out, StringBuilder.Append is a performance win over string.Concat for massive string operations in a loop.
profile picture

Carova

#13
Good to know. And, yes, I have a LOT to learn!

Any suggestion about how to eliminate symbols that do not have complete data?

Vonce

profile picture

superticker

#14
I added a StringBuilder(length) parameter to create the right initial storage allocation in its declaration; otherwise, StringBuilder will have to guess, and it will repeatably guess wrong creating extra garbage collection.

QUOTE:
Any suggestion about how to eliminate symbols that do not have complete data?

So what criteria do you want to use to detect that? You forgot to say. In the solution below, I'm using anything more than 4 consecutive bars without changes to declare missing data, but that may not be the best way.

CODE:
Please log in to see this code.

I haven't tried the above code, so you'll need to debug it. I don't know if the "continue;" statement is functional in a foreach loop. But if it is, it should prevent missing symbols from getting into the output file. If the continue; statement doesn't work inside a foreach loop, then you'll have to convert it into a traditional FOR loop.

UPDATE: I just tried this code, and it doesn't detect missing data. You'll need to debug it and post your solution.
profile picture

Carova

#15
And "Thanks!" again superticker! You have given me a great framework to play with to get what I need.

Again, MUCH APPRECIATED!

Vince
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).