Unable to load ASCII Intraday Data
Author: wbeckcom
Creation Date: 8/27/2010 6:33 AM
profile picture

wbeckcom

#1
Unable to create an intraday ASCII Data Source that loads the following 5 minute data:
20050630,955,36.549999,36.549999,36.41,36.41
20050630,1000,36.41,36.490002,36.310001,36.389999
Format I am trying to use is
.TXT extension
yyyyMMdd
Hmm
Able to create a daily ASCII Data Source with data in this format but without the time field.
Is there a utility to convert ASCII to the .WL file structure?

Help appreciated,

wbeckcom
profile picture

Eugene

#2
Unfortunately, you won't be able to load intraday data with the Hmm time format due to a bug in Microsoft .NET.

Suggestion: somehow add a "0" before the Hmm in your data (write a script in any language) to turn it into HHmm and get rid of the issue.
profile picture

Cone

#3
QUOTE:
Is there a utility to convert ASCII to the .WL file structure?
The ASCII provider does this automatically if you turn on its caching feature. But of course, it needs to be able to read the data.
profile picture

Eugene

#4
Follow-up. Some say that it's not actually a 100% bug in .NET, rather a poor handling of an ambiguous time string. For example, with "Hmm" it's impossible to tell if 105 is 1:05 AM or 10:5 (10:05). I find this argument not making sense as the "mm" in "Hmm" is making it very clear what number of digits should be used for minutes of an hour, so "10:5" is a no-no.
profile picture

Eugene

#5
Here's a simple WealthScript strategy to try. It should convert all Hmm times like "955" in a specified CSV file to HHmm e.g. "0955".

Specify the source and destination files, and the position of Time in 'timeFieldNum' (in your data, it's 2):
CODE:
Please log in to see this code.
profile picture

wbeckcom

#6
Per Eugene's suggestion, I converted the intraday ASCII data to the following format:

20040412,0955,13.875,13.88,13.84,13.86
20040412,1000,13.855,13.875,13.785,13.81

and attempted to create an ASCII data source in WL6 with
.txt extension yyyyMMdd Hmm settings. Does not work!

Is there any setting and any intraday data format that will work?

wbeckcom
profile picture

Eugene

#7
Did you read my reply # 8/27/2010 6:37 AM carefully? It suggested that you would use HHmm after conversion.
profile picture

wbeckcom

#8
I do not see HHmm as a setting in WL6. The closest is 'hhmm tt' but this does not work.
profile picture

Eugene

#9
Simply type HHmm in the input box.

This way you can define any custom DateTime format (reference: WL User Guide > Data > Data Manager > Create New DataSet > Data Providers > ASCII).
profile picture

wbeckcom

#10
Inserting HHmm results in an error message for the line where time is 1300. If lines below 1255 are deleted WL6 creates the Data Source. Any suggestions? I can structure the data in any format. Is the tt format an answer and what is it?

20040412,0935,13.75,13.795,13.745,13.7945
20040412,0940,13.7905,13.90,13.7905,13.86
20040412,0945,13.865,13.875,13.86,13.875
20040412,0950,13.875,13.89,13.865,13.875
20040412,0955,13.875,13.88,13.84,13.86
20040412,1000,13.855,13.875,13.785,13.81
20040412,1005,13.805,13.835,13.80,13.825
20040412,1010,13.83,13.89,13.83,13.86
20040412,1015,13.855,13.875,13.855,13.87
20040412,1020,13.865,13.895,13.855,13.895
20040412,1025,13.895,13.945,13.895,13.945
20040412,1030,13.935,13.94,13.905,13.91
20040412,1035,13.915,13.925,13.91,13.925
20040412,1040,13.925,13.945,13.895,13.915
20040412,1045,13.91,13.94,13.89,13.905
20040412,1050,13.905,13.925,13.90,13.91
20040412,1055,13.915,13.919,13.89,13.91
20040412,1100,13.91,13.94,13.905,13.935
20040412,1105,13.93,13.935,13.925,13.93
20040412,1110,13.925,13.93,13.91,13.92
20040412,1115,13.92,13.935,13.905,13.91
20040412,1120,13.91,13.915,13.875,13.895
20040412,1125,13.89,13.94,13.89,13.94
20040412,1130,13.93,13.94,13.925,13.935
20040412,1135,13.935,13.935,13.915,13.915
20040412,1140,13.912,13.94,13.912,13.915
20040412,1145,13.9195,13.925,13.91,13.925
20040412,1150,13.925,13.939,13.92,13.931
20040412,1155,13.93,13.935,13.92,13.93
20040412,1200,13.93,13.97,13.91,13.96
20040412,1205,13.965,13.98,13.96,13.98
20040412,1210,13.975,13.98,13.97,13.975
20040412,1215,13.98,14.00,13.97,13.98
20040412,1220,13.98,13.98,13.97,13.975
20040412,1225,13.97,13.98,13.97,13.98
20040412,1230,13.975,13.98,13.97,13.975
20040412,1235,13.975,13.98,13.97,13.97
20040412,1240,13.97,13.975,13.965,13.97
20040412,1245,13.965,13.975,13.965,13.975
20040412,1250,13.975,13.975,13.965,13.965
20040412,1255,13.965,13.975,13.965,13.965
20040412,1300,13.97,13.97,13.965,13.9655
20040412,1305,13.97,13.97,13.965,13.9655
profile picture

Eugene

#11
Double-check your input carefully. You're inserting an extra character or some symbol was wrong. Undoubtedly HHmm is correct, but I even created a new DataSet using the lines above and of course, there was no error message.
QUOTE:
Is the tt format an answer and what is it?

No. However, it's always a good idea to see the reference: WL User Guide > Data > Data Manager > Create New DataSet > Data Providers > ASCII. There you'll find a link to MSDN with date/time format descriptions.
profile picture

Eugene

#12
Added a new method ConvertHmm to manually convert data files with this unsupported time format.

In a forthcoming build, the ASCII provider will make this conversion on-the-fly.
profile picture

Eugene

#13
The ASCII Provider will support the "Hmm" and "Hmmss" time formats, starting from version 1.3.
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).