Hi!
Please help!
I use Finam data provider. I use Data Manager then Update Dataset to download market data. It takes about five minutes to download 60-minutes history per one future contract. It is about 1000 bars. Why does it take too long?
After downloading via Data Manager, it takes practically the same time to open chart of already downloaded security. It is looks like WLD downloads all data again and I have to wait several minutes again just to open chart. Why is it so? Does not WLD store all market data after downloading on my hard disk?
If I try to download for example 50 or 60 contracts in takes several hours :(
Size:
Color:
A picture
Size:
Color:
Hi,
As stated in post #74 dated April 2018 in
this thread, "
Data updates will now become much slower due to an extra delay introduced to avoid the new anti-bot protection."
At Finam, initial download of intraday data (backfill) per se has never been a piece of cake. You aren't allowed to download entire intraday history in one shot, multiple web requests are prohibited, there's anti-bot protection now etc. To avoid serious download restrictions the provider has to make multiple requests in steps and combine the data. Since April 2018 there's a 1 second delay in between those requests, adding further to the wait.
Add inability to make parallel downloads so each stock must be downloaded one by one and your intraday backfill takes a very long time. As there's nothing we can do to speed up the process, schedule your initial backfill for the night using the Data Manager. On the bright side, daily updates are are fast enough.
Size:
Color:
OK.
Thanks.
But why does it it take several minutes to open already downloaded chart?
Size:
Color:
Because "On Demand Data Update" is enabled in the Data Manager. This causes a series of extra requests to attempt update the symbol. Disabling it should make the chart open up instantly.
Size:
Color:
Thank you! !!!!
I will use your advise this evening.
You are a Great Time Saver :)
Size:
Color:
It works!
Eugene, thanks!!
Size:
Color:
Sergey,
Great to hear it. Glad to have helped!
Size:
Color:
As a followup to explanation from my post #3, I think there's potential for a considerable speed improvement of intraday history backfills. In my testing, the download time was reduced twofold by employing some tweak!
There may be a couple downsides (which essentially are just borderline cases):
1. On aborted data download (e.g. network timeout or server issue) you'll be left with incomplete history which can not be fixed by "Update Data" but only by right click + "Reload Chart History".
2. For an illiquid Futures symbol, a crippled history near its inception may be delivered (but usually should be okay)
Look for v2019.03 (late February) with this improvement.
Size:
Color:
To anybody interested: we've released a new data provider for the Russian markets (mainly) - MFD.ru. Although its intraday backfill isn't as powerful as Finam's (MFD returns just 1 year of intraday data), speed is its advantage. As there are no delays, backfill and daily updates are multi-threaded and very fast.
Check it out here:
Direct installation linkOnline user guideWe hope it becomes a good backup solution to the Finam data feed.
Size:
Color: