Hi.

I am trying to import and use a one minute intra-day data file I obtained from quantquote.com for QQQ. (I am doing this because, while WL-Pro provides intraday data it only goes back to 2005 or so rather than the beginning date of 3-10-99.

I am having difficulty, however, with WL accepting the format of the Time. Time in the data is in the following example format:

400 = 4 am. Hours go 1-23. I used this format setting in WL:

Hmm

This appears correct to me. However, I get an error message: "string was not recognized as a valid DateTime". I am using the ASCII file import button and settings with file format .csv.

Can anyone provide any help?

Thanks.

Michael

Hi Michael,

Since you didn't answer to my reply in your ticket 23547 an hour ago, I closed it so let's continue here.

I managed to find a sample data file (1-minute) on the quantquote.com website: https://quantquote.com/sample/SPY_MINUTE_TRADE.csv. To set it up properly, choose 1-Minute, and then configure the dialog like shown below:



It's not a time format problem. You did almost everything right but the mistake was in not setting Ignore First Lines In File to 1.

Thanks for the response. Sorry I missed it in the ticket.

OK, I replicated what you have above. I was indeed close to that. Why do you set ignore first lines in file to 1?

However, I am still getting the same error message.

Maybe this has something with the setup of the data file. The files are in a folder names QQQ Data. Within the folder are a series of folders by date. They start in 1998 I think so something like 19980102. However, QQQ did not start trading until 19990310. The folders before this were blank. I had to click on the overall file and set the target file in the box to 19990310 for WL-P to let me go to the next step, eg. the screen above. Should I delete the day folders with no data and start with 199903030? Or is something else going on?

Thanks much.

Michael

QUOTE:
Why do you set ignore first lines in file to 1?

Because it contains headers:

CODE:
Please log in to see this code.

QUOTE:
However, I am still getting the same error message.

As you can see on my screenshot, it's perfectly OK with their demo data. Perhaps you overlooked the 3 Filler fields as on my screenshot that should be added. If their real files are any different from the sample, click "View Data File in Notepad" to carefully inspect the actual data file.

Hmmm. So the screen shot was produced with their demo data. No, I put the filler fields in there just as you had them. I also had already carefully looked at the data file via the notepad and all looked fine. Let me see if I can upload the data file as it is on my computer if you don't mind. Note sure if it's going to upload. I got an error message and there is a red - sign beside the file name. Let me know.

Maybe that will help if you can look at it.

Thanks again.

Michael

CSV isn't on the list of allowed extensions (*.jpg, *.gif, *.png, *.zip, *.rar, *.pdf, *.txt). If you zip the problematic data file before uploading, it will work out.

I just tried to send you the zipped file that I had downloaded. Did you get it?

Michael

No. Make sure it's under 2 Megabytes.

Hi. Not sure if you are still around. I had to start the evening. The file I tried to send you was 23 mb. I have deleted a lot of the folders later in the file and will try again. This should work, only 1 mb. I tried this file out and am still having the same problems as before. So it should be useful to see if you can figure out how to solve my problem. Thanks again.

Michael

You might want to ask your data vendor if downloads are available in a continuous manner. Wealth-Lab employs the "one file per symbol" model. Not only it's unreal to set up thousands of "table_qqq" DataSets, it would be impossible to utilize this collection for backtesting. One could code a script to glue the files together but it's no fun at all when you paid for the data!

Aside from that unfortunate breakup of the data by day, making it hardly usable in Wealth-Lab, there's no problem importing the individual files from subfolders, one by one. The purchased data is just slightly different: it doesn't seem to contain headers. Here's a fragment of the first data file from your collection:

QUOTE:
19990310,931,48.24,48.24,48.24,48.24,181667,1,0,0

I used the same settings (except "Ignore First Lines In File" = 0) and had positively no issues. Please double check everything as it's clear to me that it's an "operator error". Good luck.

So, unless I get what you are calling a continuous data file I can't effectively use this data supply in wealth lab pro? Obviously, processing on day at a time is not useful. I am not sure what you mean by operator error above.

If they cannot provide the data in a continuous file, do you know any vendor for which I could get intraday data at least down to five minute bars for QQQ?

Michael

QUOTE:
So, unless I get what you are calling a continuous data file I can't effectively use this data supply in wealth lab pro?

You got my explanation correctly.

QUOTE:
I am not sure what you mean by operator error above.

"Operator error" means that you (the user) is making some sort of error. I double checked that their data gets imported into WL6 just fine.

QUOTE:
If they cannot provide the data in a continuous file, do you know any vendor for which I could get intraday data at least down to five minute bars for QQQ?

IQFeed, for example.

Imported just fine but not in a way that is practically useable. How exactly did you go about getting this one day in. BTW, they apparently do have a continuous data file method of giving me the data. That was not so clear to me the way it was stated on the website. They have told me that they will provide this format shortly.

Michael

Hope you get the data in a continuous form which will be usable. Glad to have helped.

Hi. About a week ago I contacted you regarding a problem I was having getting WL-Pro to recognize a data file I purchased from a vendor. You figured out that the format of the data was not useful and needed to be provided in a different format. The data came in via a separate spreadsheet for every day. What was needed was a continuous file. I contacted the vendor and they provided me the data as a continuous file in text (txt) format.

However, I am still having difficulty. WL-P says that the datefile is not correct. I have attached a few lines of sample file. (This came from the top of the file). I've not been able to get this work with this short sample or with the total file (which is some 20 MB).

The order of the data is: date, time, open, high, low, close, volume, splits, earnings, dividends.

I provided the following specifications per this data to WL-P:

Date: yyyyMMdd
Time: Hmm
There are no headings so this was not indicated.

Please carefully follow my setup as illustrated in post #2 above. Double check that 1-minute scale has been selected and the dialog looks exactly as shown (with headers being the exception). If your data file hasn't changed after becoming continuous, the settings should work. Otherwise you should inspect the exact error message (since there can't be such error as "the datefile is not correct"), the failing line, and review your data file. Good luck.

Hi and thanks for the response.

Yes, I followed the setup as illustrated above exactly as indicated in my last post. The 1 minute scale was also selected. The error message is as follows:

String was not recognized as a valid DateTime.

File C:\ProgramFile\FidelityInvestments\Wealth-Lb Pro6\DataSets\QQQ.txt
Line:0
Field: 931
Field Name: Time
Field Type: Time

Ignore First Lines in File was not set to 1 as there are no headers in the data file. Clicking on View Data in Notepad the data file looks fine. I am not sure what you mean by the "failing line".

Attached is a short sample of the data file that I also could not get to work. I would appreciate your looking at this file and seeing if you can get it to work.

Thanks.

Michael

Hmm should be the correct format for your data per the msdn documentation, but there's obviously a problem with how it actually interprets 3 digits

It seems that you'll need to export your data using 4 digits for 24 hour time.

This annoying .NET issue has existed (and maybe still applies) but we added a workaround in the ASCII provider starting from version 1.3.0.0 dated 05/11/2011:

QUOTE:
Time formats Hmm, Hmmss are now supported.

Now I see why it has always worked for me (in WLD). As I recall, Fidelity was shunned from including the Market Manager in WLP by default when we added it to the ASCII provider, and so the version you have installed must be from 2009!

Please update the ASCII provider to its latest version using the Extension Manager. You may need to switch to "Other" extensions. After restarting WLP, your data will be imported correctly.

Hi. Thanks for the above. Sorry for lateness of response. Life intervened.

OK, I have done the following. I was promptedby WLP to download a new version 6.5.20 (though it installed on the screen as 6.6). Note that when I click on new dataset that ascii no longer is available as an option - only fidelity, msc, etc. When I go to extension manager ascii is not listed also,

I then went to wealth-lab.com, logged in, searched for ascii. It brought up ascii files static 1.3.4.0. When I clicked download I was given this message: You don't have permission to download this extension.

So . . . I have gone from trying to use ascii but it not working to not having any option for using ascii.

Please help and advise.

Michael

Since it's part of the product, you don't have to go to the website. As I told above, switch to "Other" extensions in the Extension Manager and update the ASCII provider to its latest version. After restarting WLP, your data will be imported correctly.

Hi Eugene. Well, as I said in that last post, there is NO ASCII OPTION now that I "upgraded" to 6.5.20. When I click on other extensions it says 'no items to display'. When I go to new dataset there is no ascii option provided. You told me in the last post to go the wealth-lab.com and download the latest ascii extension version. As I said above, it won't let me do this saying I do not have permissions to download the file.

Michael

Michael,

I well know that there's no ASCII option. Long story short, older versions of the provider disappear in 6.4+ and upgrade is required. Although the ASCII provider is missing from the system, in this case it's not an obstable to the Extension Manager.

It seems that there are no items to display because the first (default) tab is "Addins" but ASCII is a provider. Naturally, you have to choose the corresponding tab. So, click on "Other", and then click on "Providers" (or "Updates"). There you will find the updated ASCII provider.

Hope that helps. In any case, waiting for your reply to ticket 23656.

Hi Eugene. As requested I am picking this conversation up from the original thread. Here is your posting from yesterday:

Hi Michael,

I verified that your sample file works if you follow my suggestions in reply #20: switch to "Other" extensions in the Extension Manager,
update the ASCII provider to v1.3.4.0, then restart WLP for the changes to apply.

If you have any followup questions, please do not split the conversation and consider using the original forum thread where it all began.

I did try going to 'other' extensions before I contacted you. I tried again just now. The following message shows up: "no items to display". In the lower left corner is a clickable message that says: "Open Exensions*.wle file" but if I click on this it takes me to a 'data' folder and I am unclear what to do from there. From the "no items to display" window if I click either updates or providers nothing happens.

Please advise.

Michael

If ASCII provider is not to be found among "Others", it's among Fidelity-supported extensions. In this case, its version is most likely not the latest one (1.3.4.0). This is possible if you have uninstalled ASCII or installed WLP on new/different PC (or having reinstalled Windows). As you should know from past experience, only the actual version can handle your CSV data in Hmm format.

To find out an extension's version, use the Extension Manager (or for ASCII in particular, open the New DataSet Wizard and notice the label in the bottom right corner). As an alternative, switch to "Updates" in the EM and if an update to ASCII is available, update the provider and restart WLP. To close the question of how to install/uninstall/update Wealth-Lab extensions, review this illustrated tutorial and the Wealth-Lab User Guide > Extension Manager.

Finally, when the provider's version is 1.3.4.0 and you still can not import your file, double check your settings and carefully examine the data file in Notepad or other text viewer/editor. Although the sample file attached in your latest ticket works flawlessly, experience suggests that real data files may have differences in format!

Hi Eugene:

In Extensions Manager when I go to Fidelity Supported Extensions several items are visible including: ASCII Files Static 1.2.9.0. I note that this is obviously a lower or earlier version than the one that you mention. However, the update now button is greyed out and nothing happens when you click it. The only options available are "change log" and "uninstall". If I go to Updates nothing shows up except Yahoo.

So I am unclear how to get to the most updated version of ASCII. Do I uninstall and then reinstall or what?

OK, I went to the tutorial and followed instructions to download and install the updated version. It appeared to load in correctly and prompted me to restart the program. However, when the program comes up the updated version does not show up. ASCII Files Static 1.2.9.0 is still there. No reference to the updated program.

????

<Michael

Michael,

This seems to be an illustration of having 32-bit and 64-bit WLP installed on the same PC. For information on how to resolve it, please refer to this FAQ:

Extension installed in Wealth-Lab Developer doesn't show up after restarting application.

Basically, the last installed WLP gets all installed extensions.

Don't forget to confirm the Windows UAC prompt that comes up when WLP restarts right after installing/updating an extension. However if it doesn't, this might be an indication that you're NOT starting the WLP edition where the extension was copied to, OR a conflict with antivirus that prevents the ExtensionAgent from installing/updating.

P.S. Additionally, this reading doesn't seem to apply right now but may become useful later:

Errors installing/updating Extensions

Hi Eugene:

The machine I am running is 32 bit. I understand what you and the link are saying but I don't understand what to do about this. The link says: "Extensions should be installed separately for x86 and x64 versions of Wealth-Lab Developer". But it does not tell me how to do this. Looking at WLP I see no info that tells me what version this is. I see only one version of WLP on my computer unless another is hiding somewhere. However, I have noted that once I started working with trying to update extension that the icon of WLP changed to the previous wavy line icon whereas before it looked different.

Can you please give guidance on exactly what to do at this point?

Thanks.

Michael

If you believe that your OS is really 32-bit, then please disregard the FAQ on 32/64-bit. I also think that an antivirus conflict preventing the updated file from installing properly is pretty unlikely. Assuming that you did confirm the UAC prompt when WLP was being restarted, there are only 2 final things to try:

1. You could have downloaded some other extension, not the ASCII provider, hence it did not get updated. Here's the proper link to retry:

ASCII provider

BTW, the reason why you couldn't find ASCII in the Extension Manager before was that you overlooked the "Providers" (or "Updates") tab after having clicked on "Other".

2. You did not say which WLP version is installed. If it's 6.6, the most likely reason is that you overlooked the specific error message from WealthLab.Extensions.Agent after WLP restart, thinking that it succeeded. Before anything else, uncheck the read-only attribute from WealthLab.DataProviders.AsciiFilesStatic.dll as this KB article suggests:

"Access denied" error when trying to update an Extension in Wealth-Lab Pro 6.6

But if it's 6.5.20, then I have no more suggestions for you.

Hi Eugene:

I have WLP 6.6. Yes, my machine is 32 bit. I am not sure what you mean by a UAC prompt that I am supposed to have gotten. I have tried both (1) dragging the downloaded ASCII file onto the window of the extension manager and (2) clicking the "Open Extension's *.wle file". Both had the same result. A scrolling window appeared in the section in which the label ASCII Files Status 1.2.9.0 is located. A line moved across showing progress and then I get the following message: "Wealth Lab restart required to finish installation".

I close and restart the program. A box comes up with two messages:

ASCII File Static
Action: Install File: WealthLab DataProviders.AsciiFilesStatic.dlll
Result: ! Failed: Access to the path C:\Program Files\Fidelity Investments\Wealth-Lab Pro 6\WealthLab.DataProviders.A. . .

ASCII Files Static
Action: Install File: WealthLab DataProviders.MarketManager.dlll
Result: Succeeded.

Is this what you call the UAC prompt? So what to do?

You mention an anti-virus program possibly preventing the download? I use Avast. It did not give me an alert in the download process.

Please advise.

Michael

Here is your error message: Result: ! Failed: Access to the path...

And here is my penultimate reply #29. I'm sure that you can notice some striking similarities in it and in the link:

QUOTE:
2. You did not say which WLP version is installed. If it's 6.6, the most likely reason is that you overlooked the specific error message from WealthLab.Extensions.Agent after WLP restart, thinking that it succeeded. Before anything else, uncheck the read-only attribute from WealthLab.DataProviders.AsciiFilesStatic.dll as this KB article suggests:

"Access denied" error when trying to update an Extension in Wealth-Lab Pro 6.6

When you noticed the error message that was overlooked at first, finally we got to the bottom of it. Now you already have the solution. I'm so happy to finally say: case closed.

Hi Eugene:

I hope you don't see the case closed because I haven't figured this out yet. The article you reference states the following:

Consequently, a permission error is generated on update. To fix it, either uncheck Read-Only attribute in the Explorer's File Properties
dialog for the DLL file, or close Wealth-Lab Pro and delete the file before updating the extension.

It's hard to read these articles because acronyms are often used based on the assumption that the readers knows what they mean. What is a DLL file? I went to the file for the ASCII extension update that is to be installed, clicked properties (I am assuming we are talking about Windows file explorer). The read-only box is now checked.

BTW, if it matters, the ASCII file is names: ASCII.WMS_1.3.4.0 and is described as a WLE file.

The other option discussed above is to "close Wealth-Lab Pro and delete the file before updating the extension". Delete what file? Wealth-lab pro? That doesn't make sense.

Please advise and stay with me till this is resolved if you would.

Thanks.

Michael

Hi Eugene:

Ok. Time to close the case. I was successful. I decided to delete the outdated ASCII extension from the extension manager. Then I also went and found what I now know is the DLL file in the folder for the program, deleted it, restarted WLP6.6, and then tried, successfully, to create a new data set with the data I have. All appears to be working correctly.

Thanks.

Michael

Michael,

Glad to have helped.

Just in case someone else stumbles onto the same problem (created unintentionally in WLP 6.6), he has to:

1. Open this path in Explorer: C:\PROGRAM FILES\FIDELITY INVESTMENTS\WEALTH-LAB PRO 6\

2. Delete or uncheck the Read Only attribute from this file in Explorer's File Properties dialog: WealthLab.DataProviders.AsciiFilesStatic.dll

3. Update/install the ASCII extension again and restart WLP.