Hi,

this is more of a general question but you guys here are more helpful and more experienced with trader needs.

What would be the easiest approach to manage own data sets for individual stocks, inidices, commodities, and forex. For example, I normally update my end-of-day stock data via WL from Yahoo and calculate my entry signals for the day. But what if Yahoo has a problem? I also collect data from one of my brokers, e.g. IB, for futures and intraday data. But I also purchase different data from data vendors, etc.

My idea would be to create simple data sets (with any consumer firendly Windows SW, like MS Access or whatever, nothing fancy) which then will be my core data sets. Then to have some routines to add data from different sources: E.g. to AAPL EOD data every day from Yahoo (via WL download and scripts). In case it is not working from Yahoo the using broker A (or vice versa). The idea is to only fill in new data if there is none, meaning for EOD data only adding if there is no data for that one perticular day else leave it alone (as it might be cleaned data). Same for e.g. minute based data: if there is data for 9:39:00 on March 24, 2009, then leave it alone, else the new data line.

As I am using different programs to work with this data, from WL to NinjaTrader, Excel and batch files, VB and C#, I would then (automatically) convert that data (if required) for a certain solution like into text CSV files or import data into variables, ...

So the core of my question is: What would be an easy to use database application where I can add data from different sources. Maybe some extra hints would be very welcome too.

Thanks!

A database is extra overhead for data access than reading data from straight binary files; though you may be willing to sacrifice a little extra speed for the reliability.

It sounds like you really want a provider that hits multiple sources, and one is already avaiable as a Provider extension: Multi Quote Server, Real-Time and historical data. Check it out.

QuoteTracker is also pretty nice data feed aggregator (supports IB, Esignal, IQFeed and hundred others).

Multi Quote Server looks interesting. Just don't like that it needs a life connection to them. What if they have a problem (they don't look like a mutli billion dollar company, but heck, those do go down the drain too).

I love QuoteTracker, is my main charting SW. The best is that Jerry and the other guys respond literally instantenously on every question. Never thought of using it for collecting data. Will check.

Maybe it is easier (especially for me) to keep ASCII files and loop thru them? In WL for example I could use the built in Yahoo update and a script which writes each symbol to an individual ASCII file if that date does not exist (I guess writing a new ASCII each time as inserting isn't possible). So I then could use that ASCII also in WL as the data source for my scripts (as I might need to update it from a different source than Yahoo). And I also can use it in my other programs like Excel, NinjaTrader, etc.

ASCII files are a good solution in WL5 -- thanks to the ASCII Cache feature that makes working with them indiscernible from (speedy) binary files. Not in WL4 where ASCII is slow.