Generate DataSources programatically based on Industry

ss161

#1

3/15/2009 8:07 PM

i have a dataset with about 1500 names through which i want to search for good pair trades. i've written programs to do this where for each name i get it's industry, then create a subset list of all the other names in this dataset within the industry, and then loop through each name in the subset searching for suitable pairs. All this works great one name at a time, but when i run it against the entire dataset (in multisymbol backtest mode) i quickly generate system.outofmemory errors.

so, what i want to do, is write a short script that will let me click on an existing watchlist symbol, and loop through that entire watchlist and generate a new datasource for each IndustrySubGroup in the list that would contain all the symbols for that Industry SubGroup. I can then run my Pair script one at a time through each industry subgroup (much more manageable than running it 1500 times manually). But, I can't figure out how to generate DataSources programatically.

Here is the code so far:

CODE:
Please log in to see this code.

Would someone point me to some documentation that might describe how i can create these datasources? thanks for the help.

Steve

Size:

Color:

i have a dataset with about 1500 names through which i want to search for good pair trades.  i've written programs to do this where for each name i get it's industry, then create a subset list of all the other names in this dataset within the industry, and then loop through each name in the subset searching for suitable pairs.  All this works great one name at a time, but when i run it against the entire dataset (in multisymbol backtest mode) i quickly generate system.outofmemory errors.

so, what i want to do, is write a short script that will let me click on an existing watchlist symbol, and loop through that entire watchlist and generate a new datasource for each IndustrySubGroup in the list that would contain all the symbols for that Industry SubGroup.  I can then run my Pair script one at a time through each industry subgroup (much more manageable than running it 1500 times manually).  But, I can't figure out how to generate DataSources programatically.

Here is the code so far:

[CODE]	public class GenerateWatchLists : WealthScript {
 
		protected override void Execute() {
			
			string path = "C:\\Data\\COINT\\EG\\INDSUBGROUP\\";
			
			//wlsp.CreateDataSource();
					
			//  this class does a lot of the work -- it loops through each name in the DataSource
			//  finds it's industry (or group, or subgroup, or sector depending on what's requested
			//  and puts the Symbol and Industry Name in a table
			WLGroup grp = new WLGroup(this, "SUBGROUPNAME");
			List<string> ind = grp.GetGroups();    // This returns a List of all the unique industry groups
			for (int i = 0; i < ind.Count; i++) {  // loop through each industry group
				// -------------------------------------------------------------------------
				//  HERE I NEED TO CREATE THE DATASOURCE WITH THE NAME OF THE INDUSTRY GROUP
				//  I DON'T KNOW HOW TO DO THIS
				// -------------------------------------------------------------------------
				// CREATE DATA SOURCE
				List<string> memb = grp.GetGroupMembers(ind[i]);  // THIS ROUTING RETURNS EACH SYMBOL IN INDUSTRY GROUP
				for (int j = 0; j < memb.Count; j++) {
				      //  HERE I NEED TO ADD THE SYMBOL --> memb[j]  to the datasource I just Created
				}
			}
		}
	}
[/CODE]

Would someone point me to some documentation that might describe how i can create these datasources?  thanks for the help.

Steve

Eugene

#2

3/16/2009 2:12 AM

Steve,

DataSets are created by data providers. The API documentation isn't available, please call Fidelity directly on this one.

Cone

#3

3/16/2009 10:10 AM

CODE:
Please log in to see this code.

This isn't a feature offered by Version 5 presently. What you could do is look at an example xml file in the \Data\DataSets folder and create several yourself as you suggest. You'll have to restart WLP after creating new DataSets in that manner for them to be recognized.

I've got to do something like this anyway for the TeleChart Industry Groups, so probably tomorrow I can post something that you can use in a script.

ss161

#4

3/16/2009 12:06 PM

i appreciate the help. I'm trying to use Dave Aronow's WatchList.dll as it seems to include what i am trying to do, which is reference an existing datasource to create the new datasource. Here's where i've gotten to, but it still is not working. I don't think i can set some of the attributes directly like i am doing.

CODE:
Please log in to see this code.

Steve

Size:

Color:

i appreciate the help.  I'm trying to use Dave Aronow's WatchList.dll as it seems to include what i am trying to do, which is reference an existing datasource to create the new datasource.  Here's where i've gotten to, but it still is not working.  I don't think i can set some of the attributes directly like i am doing.

[CODE]
/*
* Created by SharpDevelop.
* User: Steve
* Date: 3/8/2009
* Time: 7:33 AM
* 
* To change this template use Tools | Options | Coding | Edit Standard Headers.
*/

using System; 
using System.Collections; 
using System.Collections.Generic; 
using System.Text; 
using System.Text.RegularExpressions; 
using System.Drawing; 
using WealthLab; 
using CenterSpace.NMath.Core; 
using CenterSpace.NMath.Stats; 
using AronowSoftware.DataProviders.Watchlist; 
using TimeSeriesObjects; 
namespace WealthLab.Strategies {

public class GenerateWatchLists : WealthScript {

protected override void Execute() {

string DataSourcePath = "C:\\Data\\DataSources\\"; 
			DataSourceManager dm = new DataSourceManager();

//----------------------------------------------------------------------------------------- 
			// this class does a lot of the work -- it loops through each name in the DataSource
			// finds it's industry (or group, or subgroup, or sector depending on what's requested
			// and puts the Symbol and Industry Name in a table
			//-----------------------------------------------------------------------------------------

string GrpType = "SUBGROUPNAME"; 
			WLGroup grp = new WLGroup(this, "SUBGROUPNAME");

List<string> ind = grp.GetGroups(); // This returns a List of all the unique industry groups

for (int i = 0; i < ind.Count; i++) { // loop through each industry group

// -------------------------------------------------------------------------
				// CREATE THE DATASOURCE WITH THE NAME OF THE INDUSTRY GROUP
				// -------------------------------------------------------------------------

WatchlistStaticProvider wlsp = new WatchlistStaticProvider(); 
				DataSource ds = new DataSource( wlsp ); 
				ds.BarDataScale = Bars.DataScale; 
				ds.BarInterval = Bars.BarInterval; 
				ds.Name = TrimChars( ind[i] );

//-------------------------------------------------------------------------- 
				// THIS METHOD RETURNS EACH SYMBOL IN INDUSTRY GROUP 
				//--------------------------------------------------------------------------

List<string> memb = grp.GetGroupMembers(ind[i]);

for (int j = 0; j < memb.Count; j++) {

// HERE I NEED TO ADD THE SYMBOL --> memb[j] to the datasource I just Created 
					ds.Symbols.Add( memb[j] ); 
				}

// ------------------------------------------------------------------------ 
				// finally, once created, save the datasource
				// ------------------------------------------------------------------------

string fn = DataSourcePath + "\\" + GrpType + "\\" + ds.Name + ".XML";
				ds.SaveToFile( fn ); 
			}

}

public string TrimChars ( string ToTrim ) { 
			return Regex.Replace( ToTrim, "[^[:alnum:]]", "", RegexOptions.None ); 
		}
	}
[/CODE]

Steve

traderray

#5

3/16/2009 3:24 PM

I have many scripts where I use the WLP4 Watchlist commands. I have been holding off on implementing these scripts in WLP5 until the Watchlist commands are implemented in WLP5 rather than trying to change the way the scripts work now with the Watchlist commands.

Will the WLP4 Watchlist commands be added to WLP5?

Cone

#6

3/16/2009 5:08 PM

As far as I can tell, these functions are not on the radar for Version 5.

traderray

#7

3/17/2009 4:18 AM

I use the Watchlist commands in WL4 a lot. I would like to see the decision to not include them
in WL5 reconsidered. They would be a significant addition.

Eugene

#8

3/17/2009 4:42 AM

Then you need to let Fidelity know it's still important for you.

Nonetheless, you still can:

Loop by DataSetSymbols
Use Dave Aronow's watchlist provider
WatchListName = GetDataSetName
WatchListSymbol = DataSetSymbols[i]
WatchListCount = DataSetSymbols.Count

WatchListSelect, WatchListRemoveSymbol, WatchListDelete, WatchListClear, WatchListAddSymbol = N/A

ss161

#9

3/17/2009 7:31 AM

I believe Dave Aronow's Watchlist dll can replicate all of the functionality of the WL4 watchlist commands, if / as i figure out how to use it, i'll post.

traderray

#10

4/2/2009 1:30 PM

I make extensive use of the Watchlist commands shown above as N/A. Many of my scripts
cannot be duplicated in WL5 without these commands.

I have called Fidelity and told them that these commands are important. Anyone else
that uses these commands might also call Fidelity to help the cause.