Data import from Yahoo or MSN
Author: hankt
Creation Date: 5/25/2010 10:23 AM
profile picture

hankt

#1
I'm using Wealth-Lab Developer 5.6 from Hong Kong and I am having trouble updating my end of day data imports to create tomorrows trade alerts. It seems as if there is a problem getting the most recent day's data as I'm always one day lagged.
When my day is over, the US trading day hasn't begun and I'm wondering if there is something in the code that is forcing data retrieval from the previous trading day (my t-1) rather than today's close for Hong Kong, Australia, Japan, Korea, Singapore and Taiwan markets.

I've just updated my data sets and Australia is showing Friday (5/21) and Hong Kong is showing Thursday (5/20 as 5/21 was a holiday) as most recent data. It is now my Tuesday at 11pm and I should be seeing data from 5/25 close. The data is certainly available on Yahoo and MSN.

Thanks,
Hank
profile picture

Eugene

#2
Hi Hank,

Try enabling "Always return data with partial bar" on the Yahoo! Data tab in the Data Manager.
profile picture

hankt

#3
There is definitely a bug in this process. I now have data from the 25th, but not the 24th - for all Asian market data.
profile picture

Eugene

#4
Let's take look at Yahoo! data (Asia):

YOJ.F - Japan, last date: 05/21
0013.HK - HK, last date: 05/20

And here's QQQQ for comparison - last date: 05/24

So it's Yahoo! - or their vendor - having some temporary issues. You may disable partial bar and wait until the data becomes available.
profile picture

Cone

#5
I don't know what to tell you with the small amount of information you've given us. I just did a quick check on a small DataSet with some ".HK" symbols and the most-recent data were returned. Here's the log -

Updating DataSet Hong Kong ...
Preparing requests ...
Requests are ready to go.
Updating Security Names for 10 symbols...
Security Names updated.
[1] 1142.HK 20 bars 05.25.2010 20 bars added
[6] 0689.HK 939 bars 05.25.2010 939 bars added
[2] 1041.HK 1899 bars 05.25.2010 1899 bars added
[9] 0093.HK 1899 bars 05.25.2010 1899 bars added
[8] 0340.HK 1899 bars 05.25.2010 1899 bars added
[0] 8153.HK 2314 bars 05.25.2010 2314 bars added
[4] 0993.HK 1899 bars 05.25.2010 1899 bars added
[5] 0893.HK 154 bars 05.25.2010 154 bars added
[3] 0998.HK 755 bars 05.25.2010 755 bars added
[7] 0663.HK 1739 bars 05.25.2010 1739 bars added
Update completed (7.453 sec)
----------------

1. Is your computer clock correct for the time zone you've specified?
2. What is your computer's time zone?
3. Copy the update log, like above to provide us with more data.
profile picture

hankt

#6
My computer clock is set to GMT +8.

My update data set looks just like this:

[3] 0001.HK 1900 bars 05.26.2010 1 bars added
[5] 0005.HK 1900 bars 05.26.2010 1 bars added
[2] 0010.HK 1900 bars 05.26.2010 1 bars added
[7] 0012.HK 1900 bars 05.26.2010 1 bars added
[1] 0014.HK 1900 bars 05.26.2010 1 bars added
[9] 0016.HK 1900 bars 05.26.2010 1 bars added
[8] 0083.HK 1900 bars 05.26.2010 1 bars added
[6] 023530.KS 1062 bars 05.26.2010 1 bars added
[4] 0101.HK 1900 bars 05.26.2010 1 bars added
[0] 0272.HK 893 bars 05.26.2010 1 bars added
[3] 036570.KS 1772 bars 05.26.2010 1 bars added
[5] 0688.HK 1900 bars 05.26.2010 1 bars added
[2] 0750.HK 338 bars 05.26.2010 1 bars added
[7] 0754.HK 1900 bars 05.26.2010 1 bars added
[9] 0813.HK 958 bars 05.26.2010 1 bars added
[8] 1101.TW 2686 bars 05.26.2010 1 bars added
[1] 0766.HK 2038 bars 05.26.2010 1 bars added
[6] 1109.HK 1900 bars 05.26.2010 1 bars added
[4] 1138.HK 1900 bars 05.26.2010 1 bars added
[0] 1207.HK 1900 bars 05.26.2010 1 bars added
[5] 2777.HK 886 bars 05.26.2010 1 bars added
[2] 3336.HK 1132 bars 05.26.2010 1 bars added
[7] 3383.HK 1102 bars 05.26.2010 1 bars added
[8] AAPL 6486 bars 05.25.2010 1 bars added

Which was just run - for some reason data is now updated for the 25th and 26th, but not the 24th.

Eugene, I checked the tickers on Yahoo manually and saw that the data was available in a 5 day chart before my first post. However, I now have HSBC (0005.HK) data in Wealth-lab for all but Monday the 24th but the manual pull shows nothing for this week (from the link:http://finance.yahoo.com/q/hp?s=0005.HK&a=00&b=1&c=2003&d=04&e=26&f=2010&g=d)

PRICES

Date Open High Low Close Volume Adj Close*
May 20, 2010 73.00 73.05 71.50 72.55 32,470,100 72.55
May 19, 2010 71.90 72.70 71.70 72.25 34,289,400 72.25
May 19, 2010 $ 0.08 Dividend
May 18, 2010 73.80 74.40 73.35 73.95 23,883,500 73.87



Thanks for your help.

Hank
profile picture

Eugene

#7
QUOTE:
Eugene, I checked the tickers on Yahoo manually and saw that the data was available in a 5 day chart before my first post.

FYI: forget the charts altogether when checking Yahoo data errors - what you need is exclusively the "Historical Prices" link above, since it uses the same data source as the Yahoo! provider. Yahoo! charts are powered by a different subsystem.

Using the symbol names posted by Cone yesterday, I can only download their data up to 05/20/2010 or 05/21/2010:

[0] 0013.HK 94 bars 05.20.2010 0 bars added
[9] 1142.HK 19 bars 05.20.2010 19 bars added
[3] 0663.HK 94 bars 05.20.2010 94 bars added
[9] YOJ.F 98 bars 05.21.2010 0 bars added
[0] 8153.HK 78 bars 05.20.2010 78 bars added
[5] 0893.HK 94 bars 05.20.2010 94 bars added
[8] 1041.HK 94 bars 05.20.2010 94 bars added
[4] 0689.HK 94 bars 05.20.2010 94 bars added
[1] 0093.HK 94 bars 05.20.2010 94 bars added
[6] 0993.HK 94 bars 05.20.2010 94 bars added
[7] 0998.HK 94 bars 05.20.2010 94 bars added
[2] 0340.HK 94 bars 05.20.2010 94 bars added

And if I go to the web interface to check for historical prices, I'm getting exact same dates. What we have here looks like a Yahoo server issue. We're hitting different Yahoo! servers, and mine supposedly contains delayed/incomplete data.

If you have a missing bar in between, it can be fixed:

1) for a few symbols - by opening a chart and selecting "Reload chart history" in the right click menu, or
2) for many symbols - by enabling "Delete data for symbols not present in DataSets" in the Data Manager tool, deleting the symbols from all DataSets, updating the data by provider using "Update all data..." button (making sure that a checkbox next to "Yahoo! Finance" is checked). This will effectively refresh a DataSet/DataSource.
profile picture

hankt

#8
I tried that for a single security: 0005.HK I LOST THE DATA FROM THE 25th! The data set for HSBC now containst the 20th and the 26th with 24th and 25th missing.

It may well be a Yahoo issue.

Hank
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).