Updating data using either the "Update Data" tab from the Data Manager or click on "Update DataSet" on individual dataset does not update dataset even when there are symbols that have not been updated.

When I tried to run data update again using the data manager, I got "Invalid Symbol !NO_DATA" error as shown in the attached picture below. When I tried to update individual dataset I got "Update Completed" message as shown in the picture attached - even though that are still many daily symbols that have not been updated.

It seems the Update Data from the Data Manager does not update data correctly on the 2nd or later runs, I recalled to have experienced this in the past. Thanks.

I am using IQFeed. Typicall,y I ran data update few hours or later after market close and the both the daily and intraday data are available and updated correctly. Today, I ran the data update about 20 minutes after the market close and there are a number daily symbols that are not updated probably due to the data being updated too soon right after market close. However, why the data update when run again now, NOT update those symbols that still have not being updated?

The message is that the symbols are invalid. I'm not familiar with all the symbols, but I've checked a handful of them - all are currently invalid tickers. I changed the title of the post to reflect this.

Is it possible that you're trying to get data for symbols that are no longer trading?
If that's the case, I recommend Norgate Data or Premium Data for that. With the Norgate Data Platinum subscription, you can do historical backtesting for many of the US and Aussie major indices. All data is updated automatically, without any intervention. But if you just want the data for delisted symbols in an old Metastock format, then you can buy that for a low one-time fee at the Premium Data link.

Hi Cone,

Thanks for your reply. I am not using those symbols listed in the "Invalid Symbols" error message in post #1 and they are not in any datasets that I am using or trying to update. I am not sure how they got here, that is a mystery to me.

The problem is that why does the Update Data or Update DataSet do not update symbols (please see a sample of the symbols that I am using listed below) that are not yet been updated when I repeatedly ran them late after the market has closed. It seems like it does not detect these out dated symbols. From my recollections, it seems like the Update DataSet from Data Manager works fine in the first time you run it and any subsequence (later) runs using the tool does not work correctly - namely it just passes through the symbols whether the symbols have or have not been updated. This statement is probably a stretch but that what it happened yesterday.

In the past when I ran into problem with the Update Data tool from the Data Manager, I can clicked on a given dataset and update it from there and that worked, but yesterday that didn't work. Even after restart WL, the data update still not work. Thanks.

Here is a sample of the symbols that I am using. Again, I am not using those delisted symbols in the error message. One of the symbols yesterday that is not yet updated but couldn't be updated is AAPL.

A AA AAN AAON AAP AAPL AAWW AAXN AB ABB ABCB ABEV ABG ABM ABR ACC ACCO ACGL ACIW ACLS ACN ADBE ADC ADDYY ADES ADI ADM ADP ADS ADSK ADTN ADX AEE AEG AEGN AEIS AEL AEM AEO AEP AES AFG AFL AGCO AGG AGM AGO AGX AGYS AHT AI AIG AIN AINV AIQUY AIR AIT AIV AIZ AJG AJRD AKAM AKR AKZOY ALB ALE ALG ALK ALL ALV ALX AMAT AMBC AMD AME AMG AMKR AMNB AMOT AMP AMSC AMSF AMSWA AMT AMTD AMWD

QUOTE:
I am not sure how they got here, that is a mystery to me.

If you configure Data Manager > Update Data tab as below, their data shouldn't be requested:

(o) Update only symbols contained in DataSets
( ) Also update symbols that you accessed but not contained in DataSets

QUOTE:
From my recollections, it seems like the Update DataSet from Data Manager works fine in the first time you run it and any subsequence (later) runs using the tool does not work correctly - namely it just passes through the symbols whether the symbols have or have not been updated. This statement is probably a stretch but that what it happened yesterday.

After the initial post-market update, I force subsequent updates by temporarily turning off the IQFeed Smart Update option. This allows me to get late or follow-on updates (corrections), if these are available at that time. However (the disadvantage) it will go through all the symbols. I also have observed that almost all of my IQFeed symbol’s updates and corrections are available several hours after market close.

Thank you to Eugene and RoDenSwain, I will apply your suggestions. RoDenSwain, it seems our observations are consistent with regard to the fully availability of data and corrections for IQFeed's symbols at few hours or later after market close. Thanks.

Data Manager > IQFeed > Smart Update
If you're having a problem updating valid symbols, then turn off Smart Update as suggested by RoDen.

The way this works, when a data update is requested..
1. Convert local time to EST. If it's during market hours, the request goes through (skip the rest)
2. Outside of market hours, find the time of the last update as follows:
3a. For intraday, use the date/time of the last bar in the cached data file.
3b. For daily bars, convert the data file's last write date/time to EST (all IQFeed data are in EST)
4. Obtain the date of the expected last update and add the market's closing time to it.
5. Compare it 4 to 3.

If #4 is after #3, Smart Update prevents requesting data again, because presumably, the data are already up-to-date.

Consequently, if IQFeed doesn't have all the daily data ready at 4pm and you perform an update just after that time, Smart Update would prevent requesting further updates until the next session. This seems to be what RoDen suggests: "After the initial post-market update, I force subsequent updates..."

If that's not it, I'm wondering if there's a problem this week after the end of DST in the U.S.
Is anyone in one of those 'weird' time zones that doesn't observe DST?

Thanks Cone. I think the problem is more likely what RoDen is suggested and the solution is easy enough to just turn off Smart Update on subsequence runs if any symbols still not updated. A little drawback is that it will go through all symbols again as RoDen is suggested.