I download end-of-day Fidelity data in the evenings. Due to some issue, I was not able to d/l it on an evening earlier this week so d/l it the next morning instead. When I tried to d/l the data that evening (after a fresh trading day had closed), no price data was d/l and I could only d/l it the next morning. Same issue today; my evening price data did not d/l as I had d/l it in the morning.

It seems to me that there is a log (or a switch) somewhere that documents the date when the data is d/l and then doesn't allow it to be d/l again, oblivious of the fact that a new trading day has elapsed and there's now fresh data available. Am I correct? Where is this log and can I manually edit it as this is so frustrating?

Have you tried my tip in this thread?

Fidelity Static Provider Data Corrections

The Data Tool option is cumbersome; why should one have to delete last bar's data just to d/l today's data? Anyway, its not even an option for me as I use v6.1, not 6.2, due to several issues w 6.2 I had previously documented and which remain unfixed.

Bottom line is: Why does the Fidelity Static Provider think that the data is up-to-date? Clearly, its not good programming if the 'thinking' is based solely on a date record (where?) w/o regard to time.

Amen: Spoke with Fidelity Monday and it seems this is a know issue that is being worked on,but as of today, same issue about 30 percent of the days data is a day old. This is a new problem and for eod trades where open for West Coast market opening is 6:30, so I have ( for the past 2 years) updated and placed my trades for the am.
If there is an issue Fidelity would best have notices and some form of text or a page devoted to wlab on the Fidelity site so that Cone and Eugene can do the programing and related programing issues and not need worry about the data. Now it seems this forum is all we seem to have to bring these things up.
OK, now I use Y! for 9-10PM when the symbols get updated, Fidelity data will not be of value to me as I would not like to get up before the dawn to think of these things.

I can understand your pain, Mike. I am thankful for small mercies - like being on the east coast!

But the bigger point is: I run many scripts, and there are days when I have > 100 alerts, and I have to enter them manually (not just at Fidelity, but other brokers, too). I have no time for all this in the morning; that's why I d/l the EOD daily data in the evening. And now even that is getting to be a mess.

Called Fidelity about 6:00pm. I was handed off from active trader to WL division. On hold for over 45 minutes with the canned music, then decided to call it quits. I'll call tomorrow. WTF!!

QUOTE:
why should one have to delete last bar's data just to d/l today's data?

Because it's an instantly available, simple and effortless workaround for a bigger problem that will take some time to resolve.

Mike:
My experience has been similar. On the 1-2 occasions I have had to call after 8pm EST, I have been put on an indefinite hold, too. Though I must say I don't have your degree of patience! ;)
-------------------------------------------

Eugene:
[Can you change the thread title from the generic 'Data Download issues' to something more meaningful, like 'Fidelity Static Data Provider doesn't update in PM if data already already downloaded in AM'? Thx.]

My question was rhetorical; of course I knew the reasoning behind it.

I really think that the BEST solution is to remove the once-per-day constraint on the Fidelity data download. I can understand why it may have been implemented - to reduce the load on Fidelity's servers - but in so doing it has also blocked those users who have a legitimate reason to do so. Do you really know anyone who d/l the data more than once just for fun? I mean, really?? In my many years as a trader, I have come across just one data download provider that imposed a once-per-day limit, and it was a small company; not sure if they are still around. Even the free data providers don't have such limits.

I hope I have given you something to chew upon. And the solution should be one that's for the greater good, not one aimed at a few bad apples (if any).

No need to call in. In this case, the people who need to know about this issue already know and agree that there are good reasons to change the behavior for the Fidelity provider in 6.4. All this cropped up too late to make changes for 6.3.

I am not sure I understand. I am actually using the earlier version of WL 6.2 and prior to this week had none of these issues. What happened, and why hasn't some announcement been made regarding this restriction? I actually was told from the Fidelity team this would be a short term issue, with apology ( days) and was related to data vendor. Now it seems such explanation is inconsistent with Eugene's

QUOTE:
take some time to resolve

Cone or Eugene, can you please provide a more detailed summery including eta and reasons for this issue? Why do I and others that use the end of day data, who have become dependent on it need to find these things out through personal trial and error, guessing and assuming such a glitch would soon be resolved?

Thanks


BTW I only download one time. Typically 5 or 6 pm , but lately( last few days) a bit latter only to get some symbols updated and roughly 30 percent remaining stuck the day before. After that, no change. Lately, I wait until Yahoo is fully updated, ( 11:00pm- 12:00am)so I can actually trade.

1. Everyone is using 6.2 still; 6.3 isn´t done.
2. There is no general problem with downloads. (I haven't seen any problems.)
3. It seems that some customers are hitting a server that may not have the latest session's data.
4. The issue that we're discussing is that WLP has caching logic so as not to continually request EOD data once it has been updated - or attempted to have been updated. So when #3 happens, WLP doesn't even try to get the latest data even if you want it. This is only coming to light now because of a temporary condition (#3).

Also, there's another good reason to for EOD updates if you have updated the data before corrections were applied. That's in another thread.

Consequently, there's no reason to complain to someone about changing the Fidelity update behavior, because it's practically agreed to be completed for 6.4. However, if you are consistently hitting a server that isn't giving you the latest data, by all means call it in.

QUOTE:
Typically 5 or 6 pm

I do not recommend updating data even after the market close on the same day. See the dicussion on Fidelity Provider Static Data Corrections

I just called Fidelity WL department 9:20 AM Pacific. I am being told that they know about the data issue, it's being investigated and he was unsure of time of fix, but that it is being worked on. I asked for a ballpark, e.g. days, weeks months and he could not say.

I mentioned Eugene and Cones comments here, from which he could not explain.

I recommended that they need to have a way to communicate through notice on the Fidelity website so that Eugene and Cone don't become bogged down with data issues or Fidelity specific issues(this one) related to WL Pro. He ( Shawn said he would send up my request.)

Seems Fidelity should be accountable,and remain transparent as it could effect users. Users need a way to field concerns through them and not here. This seems far from what I seem to be finding.

Correction: I am using 6.1 and up until Thursday the 19th had no problems with eod updates at just after market close.

Thanks

First, read my previous message. Always call in data problems.

Regarding the Fidelity provider's update behavior for EOD data in WLP, it's understandable that the group supporting the production product isn't going to know about something that management decided on yesterday.

Let's be clear. There is no new problem with the WLP client. The Fidelity Provider has always worked the way it does now. It seems clear to me that there's a Fidelity server out there with latent data for some symbols.

The change we're discussing would allow WLP to update the last bar of data (due to an update before corrections or due to missing the data altogether) without resorting to workarounds.

Cone: Do you have an idea how many errors one might find prior to corrections? Are we talking 1 percent or 20 percent errors?

So far your above explanation is the most comprehensive. Thank you. Why couldn't this information be given to me from Shawn at Fidelity.

Now as far as 3

QUOTE:
Consequently, there's no reason to complain to someone about changing the Fidelity update behavior, because it's practically agreed to be completed for 6.4. However, if you are consistently hitting a server that isn't giving you the latest data, by all means call it in.

This confuses me. Please help me understand the last sentence.

We are typing at the same time. I have been reading your response after I ask thew questions.

Just a quick note: Thank you for having the issue documented here. I have been struggling trying to figure out why my data is appearing 1 day behind. This has happened before, then went away. Now it is back again. While very annoying, at least there is a tool that might help me with this problem (after I can download the extension - support ticket created).

Bryce,
The inability to download extensions was fixed so you can install the Data Tool to work around the issue.

Eugene,
As an interim workaround for WLP v6.1 users, do you mind making the Data Tool extension work with v6.1 instead of the minimum v6.2 required? Atleast, that way I can get some work done the same evening.

WLP 6.3 is almost at the door with fixes to some of 6.2 bugs. However, next release of the Data Tool is today. It will be possible to install v2012.02 in WLP/D 6.1.

The Data Tool has been updated with new features, grab it.

My data download problems also started after the weekend. When the updater gets to Fidelity data download, it starts cycling forever through the symbols without updating them. I cannot stop the cycling by closing either the data update or WLPro itself. The only way I can stop it is to shut it down via the windows task manager. When I unchecked Fidelity as data provider, the Data updater still tried to get Fidelity data and started cycling again. Any suggestion how to fix this is appreciated. BTW, tried data tool - deleting the last bar doesn't help.

Thank you for making the Data Tool v6.1 compatible. :) However, as Ruschem noted, deleting the last bar doesn't help. :( The problem seems to be at Fidelity server end and appears to primarily involve Nasdaq/OTC symbols as most NYSE/ARCA symbols are getting updated EOD.
*/

QUOTE:
appears to primarily involve Nasdaq/OTC symbols as most NYSE/ARCA symbols

Precisely correct and here's how to avoid it:

For at least the next 2 to 3 weeks, DO NOT UPDATE Fidelity EOD data after the market close until after 2:00 am the following day.

Even this problem aside, as I mentioned in Fidelity Static Provider Data Corrections, I recommend waiting until the corrections have been applied, and scheduling updates for 6:00 am the next day.

To look at daily charts after the market close without updating, make sure that Data on Demand is off and turn on Streaming.

In summary, there will be server changes in a few weeks to alleviate the problem with the Nasdaq symbols after the market close, but also changes will be planned for 6.4 for WLP's Fidelity Provider update behavior to allow a way to update/refresh the most recent bars.

Waiting to update ALL data till the next day is just not a practical option for many users, including myself; sorry. I will continue to update EOD data the same evening, atleast to get those symbols' data whose data is available and, more importantly, to get the fundamental data for all symbols. This step alone takes about an hour and by doing this in the evening I save that much time the next day as I'm only using Update Prices the next morning, not Update Dataset.

The only reason to update data next morning is b/c Fidelity's data provider sometimes includes after-market data when data is d/l the same evening but cleans it overnight. If Fidelity were to get a clean data provider, or insist on the current one to exercise quality control, the issue of when to d/l would become moot. Problem solved.

Why not let the data update with EOD corrections while you're sleeping? Look, I think it's important to get those corrections one way or another for the reasons expressed in the link I've posted 3 or 4 times. If that's not going to work for you, find another EOD provider.. we have many of them integrated in Extensions.

It's not about getting "clean" data, which are from Reuters. The issue is that the daily bar data contain the post-market activity until the corrections are applied, which for some reason Fidelity can't get done as quickly as other providers.

I had considered doing updates while sleeping but then discarded the idea for 2 reasons mainly - PC security + wasting energy while running the PC all night. Its not a cost issue but really a 'green planet' issue (note: that is not meant to be a political statement, just my belief; I do want a healthier planet). :)

Is Reuters the supplier for both price and fundamental data, or just price data?
Surprised, b/c Reuters EOD data - via Metastock Downloader - has generally been clean, more or less, when I used to d/l at 6pm EST. Can't see why they can't do the same for Fidelity. After all, Fidelity is the 800 lb gorilla and I would imagine if they put their weight behind something they will get their suppliers to readily comply.

Set your BIOS to turn on the machine at 5:50 AM ET and use the Windows Task Scheduler to launch Wealth-Lab when the computer starts. Schedule the update for 6:00AM. Voila, a green solution.

btw, Windows Scheduler can wake the machine up to run the task so no need to deal with BIOS.

Thanks for the suggestions, guys - but I'll still have to wake up just to type in my password. I really have no desire to make life this complicated - such maneuverings should be quite unnecessary!

By far the best and simplest solution is for Fidelity to provide its users with clean data - and the ability to d/l it whenever they want. And in this day and age that's really not too much to ask for.
*/

Why is this issue primarily affecting the Nasdaq symbols and not the NYSE/ARCA symbols?

Eugene- Cone, Do you ever forward these discussions to Fidelity. Seems there should be a way to raise these issues directly with Fidelity, but not just through private phone conversations as is now the case.

I suggested that Fidelity develop a web page to have announcements and an open forum such as this for Fidelity types of issues. Considering the roles each of you all play or don't, e.g. software development (you guys) vs data vendor decisions which is Fidelity , confusion might be somewhat alleviated. Hopefully.

Eugene, Cone
I have a EOD strategy that runs a few hours after market close and relies on the day's closing prices. I noticed that when I update my dataset not all symbols receive the update. When I rerun the update, DM realizes that some symbols are out of date and re-requests updates, but no new bars are downloaded. I even ran the DM at 9:00 pm EST and still no updated data. I believe about 1/2 of my symbols were not updated. Any ideas to fix this?

-thx, J

If the provider doesn't return the data, there's nothing that can be fixed locally. You can turn to a back up provider.

Nonetheless, I wouldn't run an EOD Strategy with Fidelity data before corrections are applied the next morning. Here's why.