Database provider with OBDC Text Driver
Author: abegy
Creation Date: 5/4/2012 5:28 PM
profile picture

abegy

#1
I try to use Database Provider extension with an ODBC driver linked to a .csv file.
I configure it properly with WLD (I hope) because WLD was able to see columns names and symbols.
But when I try to display a chart, all the times I have "No Data to display".
I don't understand why.

Is it possible to help me ?

File is like that :
CODE:
Please log in to see this code.


and the scheme created by the connector is like that :
CODE:
Please log in to see this code.


For information, when I use the ASCII provider extension, I have no problem to see data. It's that why I don't understand where is the problem with the Database Provider extension

Thanks for your assistance :-)
profile picture

Eugene

#2
It's not the database provider is reading the file, ODBC is (or OLE DB can do the same if you haven't already tried it). The ODBC driver is a legacy technology with limitations. There's no guarantees: it may work if you're lucky, or may not work. (For example, a little known feature of the ODBC driver is parsing HTML tables from websites... unfortunately, it's limited to the simplest, purest, vanilla tables that present-day web developers would hardly ever use.)

Knowing what you're trying to do (replacing the ASCII provider to get on demand updates in the SM), is there a chance that the file is already open for writing by another program? The ODBC driver may not like this (as I recall from my experiments from the development time).

And if the file is not opened for writing, "No Data Available" means that the ODBC Text driver is having problems reading the file. It's quite picky, and may not like that you defined the intraday Date field in schema.ini as Date and not as DateTime, or not specified the DateTime format (M/dd/yyyy H:mm), for example.

If tuning the schema.ini according to these suggestions didn't work out, please enter a new support ticket and attach a sample file, leaving a reference to this thread. I'll try to take a closer look.
profile picture

Eugene

#3
While I don't envision this issue becoming popular (I believe it's pretty unique), here's an explanation and a workaround should anyone else stumble onto the same problem.

Culprit:

CODE:
Please log in to see this code.


Background:

When creating the DataSet, ODBC/OLE DB connector does a forced uppercase and returns "EUROPE" instead of "Europe". The Database provider accepts the input and stores it in the DataSet's XML file. When requesting the data, the DB provider asks for "EUROPE" and ODBC/OLEDB connector willingly returns 0 rows, not finding any data.

Workaround:

Working around this ODBC/OLE DB bug/feature is rather easy:

1. Rename all mixed case symbols to upper case in the data e.g. "Europe" to "EUROPE".

2. If #1 isn't acceptable:
a. find the DataSet's XML file in Wealth-Lab (\Data\DataSets under %AppData%),
b. close Wealth-Lab,
c. open the XML file in your favorite editor,
d. force upper case of all such symbols: "Europe" becomes "EUROPE" etc.
e. restart Wealth-Lab.
profile picture

Eugene

#4
P.S. Also, a couple of related notes that could come helpful to others...

1 - A schema.ini isn't required at all to parse a file like above, if using OLE DB driver.

2 - Like our Metastock provider, both ODBC and OLE DB are not available for use by 64-bit Wealth-Lab due to lack of 64-bit system drivers.

3 - The whole point of this venture (i.e. importing CSV/TXT files with the Database provider) is to obtain support for plain ASCII files in the Strategy Monitor as a data feed.
profile picture

abegy

#5
Thank you Eugene,

I change Europe to EUROPE and now I can have a chart with my data.
But, when I try to use a script in Strategy Monitor with this Data (through the Database provider extension), I have all the time this error message :
CODE:
Please log in to see this code.


Script in working fine outside the Strategy Monitor. So, I don't understand the problem
profile picture

Eugene

#6
As I already noticed in a ticket, currently it's not possible to use ASCII data with the Database provider in the Strategy Monitor, and selecting data ranges other than "All data" or "Fixed bars" will not work either.

Not in the current release, at least.
profile picture

abegy

#7
I confirm you that I put "All Data" and have the message on the WLD 32 Bit version.
profile picture

Eugene

#8
I know, but it will start partially working in the current build if you switch to OLE DB and remove schema.ini.
profile picture

abegy

#9
Thanks a lot. Where I can find OLE DB ?
profile picture

Eugene

#10
In the very same "Select provider:" dropbox. Have you noticed "System.Data.OleDb" next to "System.Data.Odbc"? That's OLE DB. Clicking on the bluish "Connection string Help" label will help you with building a connection string.
profile picture

abegy

#11
Sorry for that but I was not clear. What I would like is to know where I can find OLE DB engine installation and how to configure it for a "Text connector" ?

If I used ODBC by default, it's because you have it by default in the Windows 7 setup.
profile picture

Eugene

#12
Read my previous reply carefully.
profile picture

abegy

#13
I try it with the following command line :
CODE:
Please log in to see this code.
but when I try to display a chart, I have the following error message :
CODE:
Please log in to see this code.


I have removing Monitor.ini file as you have said. What is it wrong now ?
profile picture

Eugene

#14
The data snippet above doesn't indicate the presense of any header in the file, so I think it should rather be "HDR=No;"

In next release I'm going to make a move to partially rely on parameterized SQL queries in the provider. As a side effect, ASCII files through ODBC/OLEDB will start working as expected.

Anyway, your mileage may vary but I don't think that having an ASCII file opened for writing by a 3rd party application and reading it in WL will work. (See my reply above # 5/5/2012 1:05 AM).
profile picture

abegy

#15
I try it with the correction and it is working now correctly with Chart feature but not with the Strategy Monitor.
The error message is :
CODE:
Please log in to see this code.


Do you have an idea when you will release the new version of this Dataprovider extension ?
profile picture

Eugene

#16
Most provider users are working with a flavor of SQL Server. When considering a core change like partial switch to parameterized SQL queries, ODBC/OLEDB is not my primary objective. I'm worried about getting the code tested extensively with the serious database engines - SQL Server for the most part. So the answer is: when I'm sure that the change will not adversely affect the user base.
profile picture

Eugene

#17
Alexandre, this bug has been fixed in the latest release. Update to v2012.06 to use ASCII files in the Strategy Monitor. Any data range will work with OLEDB/ODBC sources (like ASCII files) too, not only "All data".

Note that trying to work with non-database sources like OLEDB/ODBC for plain ASCII files is not guaranteed to work. It's not a fault of the provider but a limitation of the system driver which might not support simultaneous access if the file has already been open for writing by another program. If this doesn't work for you in the SM, switch to a full-featured database. There are many choices of lightweight desktop databases with small footprint like SQLite, SQL Server 2012 Express LocalDB, Firebird, SQL Server Compact etc.
profile picture

abegy

#18
I try with the new version of the DataBase provider extension but without success.
Since the new version, I'm unable to see the chart. I have all the time the following error message :
CODE:
Please log in to see this code.
or
CODE:
Please log in to see this code.
depnding on the selection of data range.

The error is for ODBC and OLEDB source. I have change nothing except the installation of the new database provider extension.
Any idea ?
profile picture

Eugene

#19
You should not use schema.ini.
profile picture

Eugene

#20
But if you must, give the security name field ("SECURITY CODE") a name without a space in it.
profile picture

abegy

#21
I try what you asked me and now I have no error message but in the chart, nothing is displayed.
The configuration is without ini file and with OLEDB command line :
CODE:
Please log in to see this code.

Any new idea ?
profile picture

Eugene

#22
It continues working fine for me with OLEDB/ODBC, using either localized or English Windows. I can not reproduce.

I see you've changed the folder location. Everything might matter, so make sure your setup is identical.
profile picture

Eugene

#23
I think I know what went wrong. You might have forgotten to rename EUROPE into Europe in the DataSet's XML file.
profile picture

abegy

#24
After several try, I found the solution. For your information with the new version, the symbol code must be in uppercase.
profile picture

Eugene

#25
Thanks for letting me know but that's not the case. For me it worked as "Europe" in both the XML and in the data file.
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).