Index-Lab DataSet isn't up-to-date after updating
Author: streak
Creation Date: 4/14/2012 4:42 AM
profile picture

streak

#1
Maybe I'm doing something wrong here... Data Manager/Update Data/select Custom Indexes/click "Update all data for selected Providers now".

The Log shows; "Symbols already up to date", lists each index, and "Provider update complete for Custom Indexes".

However, while my eod data is up to date for 13th April, the most recent index data is the 12th March. Any suggestions as to what I may have missed appreciated, thanks.
profile picture

Eugene

#2
QUOTE:
what I may have missed

1. The data provider, native bar scale, symbols.
2. Does the problem occur with some particular data provider, or is it general.
profile picture

streak

#3
The Data Provider is Aronow Software Watchlist, Daily. This appears to be functioning normally with strategies etc.

The 17 daily indicies are all from a 124 symbol watchlist all updated to 13th April (3 symbols did not trade; last date being 12th April). There is one other index from a different watchlist of 25 symbols all updated to 13th April.

The only thing I can think of that has changed is ascii caching has been switched on and off (currently on).

Could it be anything to do me not having updated them since first setting them up a month or so back? (I had updated them a couple of times after making them but not since).
profile picture

Eugene

#4
I think I know what happened, because it can be easily reproduced with the help of additional details provided. I don't think that ASCII Cache has to do with it.

When creating the DataSet with Aronow Software Watchlist provider, you checked the "Cache data" option. With this option activated in the DataSet's properties at the time of its creation, the Index Manager can't update the data because the Watchlist provider is returning outdated bars. Fortunately, here's a workaround: open the Data Manager, "Watchlists", select your watchlist, uncheck "Cache Data", and finally click "Save".

Another (but more cumbersome option) is to keep the cache but right click on a chart of the index and choose "Reload chart data".
profile picture

streak

#5
Eugene

Thanks for that. While typing my last here I tried testing the cache; unchecking it etc, but didn't see anything change.

Consequently (?) your first workaround didn't seem to work, however, the latter one did. Turned cache back on and now all is fine. It wasn't too cumbersome - only a few to step through. When I created the watchlists I cannot remember whether or not I had cache turned on (it wasn't when just checking). Appreciate your speedy reply late on a Saturday night!
profile picture

Eugene

#6
The 1st workaround works perfectly if you click on "Save".
profile picture

streak

#7
I did save the watchlists with cache unchecked (in previous part of thread I mentioned "Turned cache back on...". This was on the Ascii Data tab, not on the Watchlists tab).

From that point doing an update for Custom Indicies did not work.

Maybe if it persists I should delete the indices and rebuild them?
profile picture

Eugene

#8
You're right, the 1st workaround doesn't seem to work with ASCII DataSets. Even if the WatchList provider was left out of the equation. Maybe it has something to do with that ASCII does not support on demand data updates. Previously, I was testing other combinations which support on demand update, such as Yahoo+Watchlist. Consequently, I don't think that rebuilding the indices makes sense; just reload them manually.
profile picture

streak

#9
Thanks Eugene

Its no problem reloading them, and only recently started looking at them and their possibilities. Guess only a few would be used if/when incorporating them in trading.
profile picture

Ben_Zurich

#10
We had the same problem as described above, thanks for the resolution.

We are very grateful to Mr Aronow that he has written the Watchlist tool, but there are a couple of minor problems that over time should be adressed.

This is one of them; another one is that it is not possible any more (as it was in WL4) to bulk-process large numbers of watchlist symbols (e.g. for custom index members etc.)

profile picture

Eugene

#11
I don't think that Dave necessarily follows this Index Manager thread. Further, if by the "bulk-process of watchlist symbols" you mean the legacy WatchList* methods missing in WL6, it's certainly outside the scope of any data provider developer - that's a Wealth-Lab enhancement request for Dynamic Watchlist support (Fidelity QC 757, deferred).
profile picture

Swissi

#12
I have an updating problem with Index-Lab DataSets. They are not updating.

My DataProvider is Aronow Software Watchlist, Daily. What is somewhat strange, that some indices are updating, some are not, even from the same Watchlist.
In which folder can I find the Aronow WatchList definitions?

profile picture

Eugene

#13
The Aronow Software Watchlist is abandoned by its author and has naturally never been supported by our company. To get started let's try to reproduce the Index-Lab issue you're experiencing using a MS123 supported data provider. tia.
profile picture

Swissi

#14
We are using MS SQL server. The data import is working very good, but Index-Lab is still not updating the data sets!

If I run the Data Manager I get the message: Symbols already up to date:

Updating Index-Lab Daily data ...
Symbols already up to date:
%ABSMA25_XB_FX_CROSS_AUD,%ABSMA25_XB_FX_CROSS_CHF,%ABSMA25_XB_FX_CROSS_EUR,
%ABSMA25_XB_FX_CROSS_USD,%AD_XB_FX_CROSS_AUD,%AD_XB_FX_CROSS_CHF,
%AD_XB_FX_CROSS_EUR,%AD_XB_FX_CROSS_USD,%IDX_XB_FX_CROSS_AUD,%IDX_XB_FX_CROSS_CHF,
%IDX_XB_FX_CROSS_EUR,%IDX_XB_FX_CROSS_USD


Any suggestions?

profile picture

Eugene

#15
If what you're trying to do is force update the indices you might delete IL cache i.e. the *.WL files and repeat update:

c:\Users\Windows username\AppData\Roaming\Fidelity Investments\WealthLabDev\1.0.0.0\Data\IndexStaticProvider\Daily
profile picture

Swissi

#16
Thank you for the hint. It is working perfectly! All files are update now.
Do I have to do the "force update" whenever I use the data sets?
profile picture

Eugene

#17
Good to know. Well, yes if you're updating an index which is based on data which has today's incomplete bar factored in. If you're using the Database provider to poll your MSSQL DB then it requests everything there is. The Index-Lab reasonably considers that the data is up to date, there's nothing to do, retry another day.
profile picture

Swissi

#18
Not quite clear to me, because I use only data up to the previous day. I do not work with intra-day data.

I found out something else: If I right-click the mouse and select "Reload Chart History", the Index-Lab file is updated! Is there something similar available in the code which is doing the same?
profile picture

Eugene

#19
It may only matter which data does the SQL DB contain rather than what last bar you request. Like I said if the DB contains today's partial bar you should be able to update I-L tomorrow.

"Reload..." does force data update from scratch, ignoring any check re: data being already up-to-date the I-L makes. Sure there's unsupported way to reload data programmatically which you can find in my reply @ 8/29/2009 5:00 AM to this thread:

Is there a way to reload chart history via an API, post #19

Obviously you'd need to change this at the very least:

CODE:
Please log in to see this code.



However, I'd vote for a simple batch/powershell script to delete the *.WL files by the location in post #15 above.
profile picture

Swissi

#20
Ok, I wrote a batch file to delete all *.WL files.

But now, how can I restore the Index-Lab files? At least with "Update", no data index files are restored! In the Index-Lab Daily folder, all data are now gone.

profile picture

Eugene

#21
I confirm this. Guess I was wrong. Restore the I-L *.WL files from backup. In the impossible scenario that you didn't keep a backup, clean up the remnants in your CustomIndices subfolder, delete the I-L Daily DataSet, and start I-L DataSet creation from scratch. Then proceed with the other unsupported suggestion in my post #19. Modify the code as required (i.e. change "Fidelity" to "IndexStaticProvider"). No guarantees that it will work - you're on your own (did I say this is unsupported?)
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).