Creating DataSets on-the-fly isn't supported (at least currently.) Even if you create the XML file for a DataSet in your Strategy, the new DataSet won't be recognized until WLP is restarted. Same for changes to Symbol Info Mgr config and other XML config files made not by the program itself.
Your proposed solution seems like it could work out. A couple of pointers.
QUOTE:
I would also like to test a strategy with a large number of random subsets of the symbols in a dataset.
For this (only), but have you considered rewriting your Strategy in the "Symbol Rotation" manner (i.e. operation in single symbol mode and looping by DataSetSymbols), running it on the whole big DataSet but deriving the actual subset of symbols it would limit itself to by randomizing them on each run?
QUOTE:
Is there any way I can write C# code that will automatically generate new datasets with new bar data using existing data and/or run multiple back tests?
1. Running multiple backtests - for legacy version (WL4), the "Reports Feed" tool of Reports-Lab 2 would do that; it does not work with WL5 and no alternatives exist. However, you could experiment with multiple backtests by using the undocumented TradingSystemExecutor class (see Community.Components
source code > Utility.cs > runDonor/LoadStrategyFromDisk and
HERE.)
2. New bar data - you can randomize/massage the existing data in a bar by bar loop and save the result using Bars.
SaveToFile method. The saved files "belong" to the original data provider and thus can be placed in its BarDataStore (e.g. "Data\FidelityStaticProvider\15 minute\..." for Fidelity 15-min data.)