Wealth-Lab 6 performance (speed) & SQL database
Author: ImranQ
Creation Date: 9/24/2010 6:46 AM
profile picture

ImranQ

#1
Hi,
I started developing my own backtesting system in C# and it is quite basic but
has been working fine. However I realized that I needed more functionality such
as detailed system performance statistics. So I figured that rather than re-inventing
the wheel, I would maybe try out Wealth Lab 6.0. The problem I have come across
is that I have all my market data in SQL Server and Wealth lab currently doesnt
support this as a data source. Eugene has told me that a SQL Server datasource extension is due out shortly.

Now using my system I can traverse the complete
database of approx 7000 stocks over their the whole history spanning back to even
1960's using my coded portfolio system in about 10 mins.

This leads me to my question which is that if I move to Wealth Lab, what
kind of system performance can I expect? How long would it take for me to
run Wealth lab over my system which is relatively simple. Should I have low expectations?

Thanks,
iq
profile picture

Cone

#2
You'll take a data access performance hit with an SQL database due to overhead (we prefer accessing binary data straight from disk), but Wealth-Lab 6 strategies run very fast. There's a ton of post-processing options (Performance Visualizers), and these also add to test overhead depending on which and how many you use.

No need to guess, just take the free trial and evaluate it.
profile picture

ImranQ

#3
The 10 mins I'm referring to is total time (reading data from SQL Server and applying strategy).
profile picture

Eugene

#4
QUOTE:
You'll take a data access performance hit with an SQL database due to overhead (we prefer accessing binary data straight from disk),

From my limited experience, the very first data request in a Wealth-Lab session has a slight delay as the Database provider is required to establish connection to an instance of SQL Server. Afterwards there's no difference loading several thousand bars (per symbol) from SQL or a binary file.
QUOTE:
The 10 mins I'm referring to is total time (reading data from SQL Server and applying strategy).

It would be interesting to estimate total execution time of a simple strategy using a stock database in the same ballpark that I have. Problem is, it's not in the SQL Server format - first I would have to code an import procedure.
profile picture

Eugene

#5
I came up with a routine to bulk import the data to SQL and sucked in a part of my data. No time for a full test on the whole universe right now, so here are some quick preliminary results.

With SQL Server Express, running a portfolio simulation on 40 years of data for 1000 symbols takes:

1. Load the data from my database + execute a Moving Average Crossover strategy: below 30 sec.
2. Apply (half a dozen) performance visualizers: ~40 sec.

So if we extrapolate the time #1 to the whole universe of 6,000 symbols, we'll get 3 minutes + post-processing overhead created by performance visualizers (depends on which one you use).

Worst case (all those visualizers enabled) would be 7 minutes, and disabling some resource-hungry visualizers would speed the thing up a minute.

P.S. The 2-year old, mid-range Core 2 Duo CPU was set to max out at 50% of its clock in my Windows power plan, so there's some room for an instant speed-up. ;) And if you add a Core i7, it will beat the C2D hands down (2-4 times faster).
profile picture

ImranQ

#6
Thanks Eugene - statistics sound good.

I do want to purchase wealth lab but want to wait until the sql server extension is released.

profile picture

Eugene

#7
With the recently released Database static/streaming provider, connecting to database have become effortless.
profile picture

ImranQ

#8
Thanks Eugene - great news!
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).