I have written a form of bad data checking that will scan a complete data set and generate a list of symbols with bad data. I have then generated a new dataset with just the bad symbols (> 300). Now rather than loading each symbol and RightClick->ReloadSymbolData, I would like to write a simple script to reload the data. Is it possible to do the data reload via an API?

yes. you can do it via API. official method is to use

CODE:
Please log in to see this code.

I have a different way, but it only works if your strategy is framed in a library (dll)

I was under the impression that GetExternalSymbol() only read the data that was already on my computer and that it did not force a new download of the data. What am I missing?

QUOTE:
I was under the impression that GetExternalSymbol() only read the data that was already on my computer and that it did not force a new download of the data. What am I missing?

I think, If you check "UpdateDataOnDemand" then data will be updated every time :).

Bryce,

It's hard to see how this little code that I once wrote for QuickRef or On Demand Update could be useful for your task. I think RA81 confuses updating a DataSet with refreshing the data.

To refresh means to find the symbol(s), delete their files, download the data from scratch and save them as Bars. I probably can think of a way to make it in a strategy using the static API.

Thank you Eugene. I will wait for your update regarding this API.

QUOTE:
I think RA81 confuses updating a DataSet with refreshing the data.

ops... if bghood want to reload all data (not only update) then Eugene is right.
But (I don't check this) don't WLD code reload data from dataSource every time when data is requested?? If "cache data" is on then WLD try to load it from cache, otherwise from source. Thomething like this i think. (Metastock and ASCII adapters work so).

I believe that Bryce is talking about Fidelity data. Neither Metastock/ASCII adapters have the possibility to Reload Symbol Data, nor it makes sense in this context.

QUOTE:
Neither Metastock/ASCII adapters have the possibility to Reload Symbol Data

If i click on the symbol in metastock dataset then all data are reloaded for this symbol, right? So if he uses not online data but something like Metastock static provider, then refresh and reload are equal, I think. or I don't understand what is it - "Reload Symbol Data"

Refer to the Wealth-Lab User Guide, Charting > Chart Context Menu > Reload Chart History.

Hi Eugene. You are correct that I am talking about Fidelity data.

I think the question is if you can connect to the Fidelity Server API yourself. No. Even if you want to take a hack at it and succeed, you'll have to decrypt it next.

I suspect that everyone really understands what I am trying to do, but to be sure I will elaborate a little. I have written a scanner to sort through a large number of stock looking for an interesting "event". During that I discovered a large portion of the symbols stored on my system contained lots of data errors (some symbols have hundreds of errors). I used the utilities->Bad History Data Checker and soon became frustrated because of checking each symbol individually. I then modified that code to scan a DataSet to find all symbols in the DataSet that contained bad data and output the list in the debug window. I have built a new DataSet that contians a list of symbols (> 300 daily) that have bad data, and would now like to automate the "Reloading Symbol Data" to my system. Going through each symbol by hand is tedious (~75% of the 5 min data has errors in the last year). This is why we write programs to do our work:-) If there was a menu pick to refresh all symbols in the DataSet, I would be done! I assume that there could be an API that would allow me to programatically do what I can manually do (Reload Chart History for a single symbol). I would assume that all customers have large numbers of data errors and ignore them. For some of us, those errors are important to be fixed. Please let me know if more information is required.

If you've got the list of symbols, then you need only to write a little code to go through the Provider folders to kill those data files. Then, the next time you update, they'll be refreshed. Sounds like you'll be able to do that yourself, but if you need help, let us know.

Keep in mind, however, bad data may get replaced with bad data unless you tell your provider about it first.

Going through the list of symbols in BarDataStore is what I had on my mind to try, no Fidelity API :)

Just thought of another way without using code - use Fidelity's Quarantined symbol list feature:

1. Close WLP

2. Add the symbols to kill to the ..\Data\FidelityStaticProvider\SymbolQuarantineList.txt file. Each symbol should be on a new line, and, you need to add an "=" sign to the end of each symbol. See the file for examples.

3. Start WLP, open the Data Manager > Fidelity Data tab, highlight the symbols in the Quarantine list to delete, and click the button to delete them.

Note! This actions deletes data from all time frames for each symbol, not just Daily.

----
Here's a way to programmatically add all the symbols of a DataSet to that file (just an example) -

CODE:
Please log in to see this code.

Hi Cone and Eugene,

From Cone's 6:55 post, I infer that if I delete the files in .../roaming/Fidelity Investments/WealthLabPro/1.0.0.0/Data/FidelityStaticProvider/Daily/... that have bad data, then at the next update, I will "Reload Chart History" automatically at the next update. Is my understanding correct?

From Eugines 6:58 post, I am not sure what you are thinking?

From Cone's 7:04 post, if I understand, this will cause Wealthlab to quarantine all of the symbols and delete the data. Then be removing the symbols from the quarantine list, I could update and get the data updated. Do I have it correct?

There seems to be a broader issue here that I would like to understand. How can I avoid getting bad data in the first place. I can make many assumptions regarding why bad data occurs, but I clearly lack enough details to fully understand. However, for many customers, having bad data will alter backtesting results and could cause bad decisions to be made.

QUOTE:
...if I delete the files ... Is my understanding correct?

Yes.

QUOTE:
From Eugines 6:58 post, I am not sure what you are thinking?

I was think of a somewhat more complex scripting solution that would delete the bad symbol's files using some static provider API calls.

QUOTE:
Then be removing the symbols from the quarantine list, I could update and get the data updated.

Correct.

QUOTE:
How can I avoid getting bad data in the first place. ... However, for many customers, having bad data will alter backtesting results and could cause bad decisions to be made.

As data isn't our product, please call Fidelity when you see bad data coming in.

Hi Bryce,

Here's another solution to wipe out your bad symbols' data. Some notes:

* Put your tickers in an array. It's ideal if you already have a line of bad symbols delimited by comma or space: then you just have to paste that line in BadSymbols.
* Designed to work with Fidelity data.
* If you're concerned about dirty daily data for a number of symbols, the code will not affect the data in other bar scales at all (e.g. 30-minute). Therefore, you need to run the code in single symbol mode on a daily chart of any Fidelity symbol. For another bar scale, paste another bunch of symbols and repeat the procedure on a chart in that time frame.

CODE:
Please log in to see this code.

Running this strategy will zero your bad symbol files. One step left is to manually update your DataSet(s) to reload data from scratch.

I think you should search google, You will definitely found solution for your problem.


Thanks and Regards
Rachel

And I think that what you have done by attaching the address was a marketing trick to promote your site -- which has completely nothing to do with trading, Wealth-Lab or this question.

Please no illusions: this has never worked here and won't do.

Post 19 still works ? Somehow, my yahoo data for a symbol contains 40 days of data but after doing symbol reload, I get years of data... I figure that I may have to refresh my entire DataSet or somehow DataTool may be a better choice ?

Hey, how about giving both post #19 and Data Tool a try and telling the forum what works best for you?

I am doing this for yahoo data... it is still running (2000 symbols)... somehow I check one of the symbols in data file YahooStaticProvider/Daily... symbol file is still there... Does it remove after completion only ?

I tried the DataTool (remove all) but it was going pop up exception.

CODE:
Please log in to see this code.

Why won't you simply delete the YahooStaticProvider\Daily folder and update your DataSets (or all data).

because I have 9000 symbols total... somehow only symbols in this one set seems to be missing data... I split my 9000 symbols into like 3 sets. Removing the entire folder would take a long time to refresh and not sure if it will fetch all data next time around either.

ok, I just put the symbols in and did something like this in a script rm ./Z/ZYME.WL
and got rid of all my datafile