Hi all,
I have created symbol database for all the symbols traded in US using following folders.
1. AMEX - lists all stock traded on Amex
2. NASDAQ - all on nasdaq
3. NYSE - all NYSE
4. ETF - All exchange traded ETF.
5. Major idices foders such as NDX,SPX500, IXIC,DJI SP100 etc.
I would like to group all the stocks in OS univers according to GICS and its sub-catogory etc.
E.g. AA -> Materials->Materials->Metals & Mining->Aluminum
15 = Materials
1510 = Materials
151040 = Metals & Mining
15104010 = Aluminum
Ofcourse, i can manually create each idustry groups by "crreat new dataset" => use industry group etc.
-But is is tedious and also there is no easy way of giving each sector a folder name
-Besides, I have all the stocks under different folder under NASDAQ,AMEX etc.

Ideally, I would like to write a script that does following:
-Goes through all my existing datasets
-Groups them according to industry-sector-subsector etc.
-Then creates a NEW dataset for each group, e.g. for AA the dataset name can be "MMM Aluminium", all the stocks belonging to these "MMM Aluminium" are under it, i.e. "NHYDY ADGR AWC KALU CENX AA.P AA MNSF ACH GBAMF ORMT NOR "
-After that I want to remove NASDAQ,AMEX,NYSE folder.

A separate question:
If I have two different datasets that contain same stocks:
-Are there any drawback
-Such as speed, storage etc.
Thank you.

QUOTE:
If I have two different datasets that contain same stocks:
-Are there any drawback
-Such as speed, storage etc.

DataSets are like Windows shortcuts. It's a 1 Kilobyte XML file with specific properties and a list of symbols. No storage issues, no drawbacks, no speed concerns.

Eugene,
As always, thxs for ur quick response.
Can u show me an example of how can I (or point me to link where it exists) Programmatically:
1. create Dataset, give it "my custom" name based on GICS classification, e.g. case of AA in previous post.
2. Then scan through ALL of my current datasets such as NASDAQ,AMEX etc.
3. Group then according to their GICS classification in their respective datasets with "my custom name"
Thank u in advance. Happy holydays.

Sorry, I can not show you such example. Please count me out as I'm busy with other community ideas, including a few provider requests, and other work later on.

Furthermore, there is no documented method to create a DataSet other than to use the static data provider API. Fortunately for you, Fidelity DataSet files are the simplest of all to create so you can avoid this complication and treat them as a simple XML structure. Take a look at their structure in the %AppData%\Data\DataSets folder. When you have a template, the only thing left is to modify the DSString entity that contains the list of symbols of a DataSet:

QUOTE:
...
<BarInterval>0</BarInterval>
<DSString>AAPL,BA,C,CAT,CSCO,DIS,HPQ,IBM,INTC,IP,JPM,KO,MSFT,MTLQQ,SBUX,T,WMT,</DSString>
...

You can find hundreds examples of working with XML on the web e.g. Dotnetperls.

I almost have a script that does what you are looking for. The difference is that I combine AMEX, NASD and NYSE in one dataset. I also have the 3 exchanges in individual datasets like you do. A lot of my processing is on the big dataset.

My script will scan a dataset, determine the GICS structure for each symbol and build the appropriate datasets by that structure. To use it, you will need to either modify it to use multiple datasets or combine the three exchanges into another single dataset. I believe you will find that the ETF's and indices do not have GICS assignments.

This was really easy in WLP4 where you have the watchlist commands. I keep hoping that WLP5 includes them some day. I use them so much that I have actually been staying on WLP4, primarily because of this hole in the WLP5 commands. I just started to learn C# and WLP5 yesterday and this is my first script. I hope to have it completed in the morning.

Traderray.
I will keep u posted if i make any progress. The problem with Fidelity WLP is that they are so secretive of everything. Usually, one has to search through the community post to figure out some hidden methods etc. so that one can proceed. It would be nice if most of this "undocumented" stuff is "documented" in one place :)
Also WLP is a great product but Fidelity's RT feed sucks. On high volume days only RT feed that works are the major indices..Most of the stocks, however, u have to keep hitting F5 every few minutes to update RT charts.
I would be nice if they supported external data vendors like e-signal or atleast publish the adapter API documentation. U can call and complain, may be they will listen..if many complained.

traderray, as Eugene mentioned..u can manually edit a DataSet.xml file and c&p stocks and change The <Name> string with name of the DataSet u want as a fix. However, i need to write a clean all-weather solution.

...
<BarInterval>0</BarInterval>
<DSString>AAPL,BA,C,CAT,CSCO,DIS,HPQ,IBM,INTC,IP,JPM,KO,MSFT,MTLQQ,SBUX,T,WMT,</DSString>
.....
Thxs Eugene.

Note that no new DataSet will appear in the tree while WL5 is working when creating them programmatically. You'll need to restart WLP. This also applies to mostly any XML config file.

traderay,
u had mentioned " I just started to learn C# and WLP5 yesterday and this is my first script. I hope to have it completed in the morning. ...".
I had also reluctently swithched from WL4 to WLP5 the end of last years. If ur new to c#, belowe are some pointers that may help.
1. Best place to get the big picture for developing c#.
- http://www.homeandlearn.co.uk/csharp/csharp.html
- After this, google and u will find many more advanced c# tutorial sites.
- For advance programming, good idea to read up on COM Interop (basically converts com/activeX .tbl to format that the .NET can understand.)
2. Make sure u compile ur code with .NET 2.0..if ur using VS2008 or C# express edition (free from MS)
3. Somewhere in the Wealth-lab Wiki FAQ.. Eugene or Cone has a detailed explanation on how to debug ur strategy using VS2005,VS2008 or csharp etc. This is a MUST read because WLP does not have a DEBUGGER.
4. Fidelity has just published part of API documentation. Download ASAP and make a local copy. Before the API docs. I had spend countless hours on forum searching for "undocumented" features of the API.
5. BTW ignore my typos, i use only two fingers to type :)
Good Luck.

I have what I suspect is a trivial problem in my code but I need some help with the error message:

The type or namespace name 'GICSHolder' could not be found (are you missing a using directive or an assembly reference?)

I appreciate any help.

CODE:
Please log in to see this code.

Nice work.

It's a visibility issue. The GICSHolder is currently acting as a nested class, so the BuildGICSDataSets class won't find it. Move the GICSHolder class out of BuildGICSDataSets.

Some suggestions and questions:

1. Save some CPU cycles by removing these lines, as making trades is not the purpose of this strategy -

CODE:
Please log in to see this code.

2. What is the IComparer implementation used for? I see no need in sorting?

After fixing some syntax errors and closing XML tags, you'll have this code:

CODE:
Please log in to see this code.

Eugene,

Thanks for the help.

You are right about the sort. I was going to process the list using different logic that needed it sorted. When I changed to this logic I did not think about the sort not being needed any more.

I will go back and reread setcontext.

Is there a way to reset DataSetSymbols to a different dataset to be able to process more than one dataset in the same script?

bodydesi123,

Here is my script. You will note that it only works on one dataset. You will need to combine your three exchange datasets into one in order to build the complete datasets by GICS structure.

You will notice that I do not build the structure by sub-industry. I do build a list for sector, industry-group and industry. It's easy to change if you want different datasets. For my use I also want a prefix on each dataset to keep each level together and easy for me to identify.

CODE:
Please log in to see this code.

QUOTE:
Is there a way to reset DataSetSymbols to a different dataset to be able to process more than one dataset in the same script?

See some of my code examples below, hope they'll bring some insights:

Iterate through datasets
Is there a way to reload chart history via an API (8/29/2009 5:00 AM)
DataSet name (10/12/2008 10:53 AM)

Adding/Removing symbols from Data Manager

Traderray & Eugene,
Thanks for ur help. thats what i like about this community forum...people are helpful!

I'm quizzical about the DSString entity that Database Manager has created, as I wish to create watchlists via code.

Can someone help with what is happening in the following DSString, please, as I cannot see a way around it.

There are 2677 symbols in the following Dataset & Wld seems to handle them all as expected:

CODE:
Please log in to see this code.

QUOTE:
There are 2677 symbols in the following Dataset & Wld seems to handle them all as expected:

This is way offtopic here (it's a thread about GICS) but I'll reply: the ASCII provider doesn't store symbol names, it looks files up by their extension:

CODE:
Please log in to see this code.

Whatever is there that ends with *.txt, be it 2 files or 27000, will appear in your ASCII DataSet as a symbol. This is how the ASCII provider knows how to handle symbols.

QUOTE:
I'm quizzical about the DSString entity that Database Manager has created

This is an API question, but the Data Manager does not create DSStrings; the Provider does.
In other words, when creating a DataSet the Static Provider creates a DSString with the information that it needs to know in order to retrieve
data from the DataSet - like properties of the data and where to find it on the computer. When data is required from that DataSet,
Wealth-Lab passes the DSString back to the Provider, which parses/deserializes it in order to "connect" to the appropriate DataSet.

Thanks Eugene, Cone

I'd better go to another thread.