Exporting values to CSV file
Author: dansmo
Creation Date: 8/14/2009 4:46 AM
profile picture

dansmo

#1
Hello,

using the following piece of code
CODE:
Please log in to see this code.

will result in data being written to the file.

There are two things, I dont understand:

1. when I open the csv file, the data begins in column B instead of column A.
2. when I run the script once again, the data is not written in the next row, but
it seems like that the first line or the whole file is rewritten.
What do I have to change, so that the code writes to the next row in an existing file. (Just like it used to be with FileWrite in v4)

regards,
dansmo
profile picture

Eugene

#2
BinaryWriter is for writing to binary files. For text files, take a look at this KB article in our WL Wiki:

Data | Exporting data out of WL5 to ASCII and WL4 native binary files
profile picture

dansmo

#3
When I change to StreamWriter, I have the same behaviour I described under 2.
profile picture

Eugene

#4
Then your attempt was incorrect. Post the whole code sufficient to identify the problem.
profile picture

dansmo

#5
CODE:
Please log in to see this code.
profile picture

Eugene

#6
QUOTE:
1. when I open the csv file, the data begins in column B instead of column A.

Is this a problem of your spreadsheet app?
CODE:
Please log in to see this code.

Where is the data here? The data is written to the file as specified:
QUOTE:
Output2 = (string)(((((((((((((((((Bars.Symbol + Delimiter) + FormatFloat(FMT, ProfitFactor)) + Delimiter) + FormatFloat(FMT, WinPct)) + Delimiter) + FormatFloat(FMT, AvgWL)) + Delimiter) + Convert.ToString(TotTrds)) + Delimiter) + "Days: ") + Convert.ToString((int) OptVar2.Value)) + Delimiter) + "S/L: ") + Short_Long) + Delimiter) + "O/C: ") + Open_Close);

QUOTE:
2. when I run the script once again, the data is not written in the next row, but it seems like that the first line or the whole file is rewritten.

Yes, because this is what the Strategy is programmed to do (see below).
QUOTE:
What do I have to change, so that the code writes to the next row in an existing file. (Just like it used to be with FileWrite in v4)

Looks like you need to append text instead of creating the file again and again. Here's a possible way. Change your "Export Data" sequence to this one:
CODE:
Please log in to see this code.
profile picture

dansmo

#7
Edit: it is working now!
profile picture

max_velocity

#8
For readability and maintenance you might want to consider changing your
generation of the output to the more eye-friendly string.Format method.

Something like

CODE:
Please log in to see this code.



Should do the trick, and also make it possible to present a clearer picture of
what you want to achieve
profile picture

innertrader

#9
I was making good progress on my project until I got to the export. I incorporated the following code:

CODE:
Please log in to see this code.


Since then I have this bizarre behavior where StreamWriter is duplicating the output multiple times. It turn out the multiple = number of symbols in the folder. (I.e., With 2 symbols it repeats the entire output over again once, and with 4 symbols you get it 4 times. It appears to work fine in the debugger.

Here's the output file after a run with 4 symbols:

QUOTE:
XLI, 9, 18, 0.250555555555557, 0.300016910246341
GE, 8, 7, 0.2305625, 1.95979660076962
CHK, 2, 14, 0.01885, 3.59852619808521
TEVA, 10, 19, 0.27478, 2.40909228238562
XLI, 9, 18, 0.250555555555557, 0.300016910246341
GE, 8, 7, 0.2305625, 1.95979660076962
CHK, 2, 14, 0.01885, 3.59852619808521
TEVA, 10, 19, 0.27478, 2.40909228238562
XLI, 9, 18, 0.250555555555557, 0.300016910246341
GE, 8, 7, 0.2305625, 1.95979660076962
CHK, 2, 14, 0.01885, 3.59852619808521
TEVA, 10, 19, 0.27478, 2.40909228238562
XLI, 9, 18, 0.250555555555557, 0.300016910246341
GE, 8, 7, 0.2305625, 1.95979660076962
CHK, 2, 14, 0.01885, 3.59852619808521
TEVA, 10, 19, 0.27478, 2.40909228238562


Here's the output in the Debug window for the same run:
QUOTE:
Number of Closed Positions is 87
XLI
Winners: 9
Losers: 18
Avg Profit = 0.250555555555557(0.300016910246341%)
Avg Loss = -0.235755555555557(-0.28323341685171%)
1
GE
Winners: 8
Losers: 7
Avg Profit = 0.2305625(1.95979660076962%)
Avg Loss = -0.0864285714285715(-0.678514893752169%)
2
CHK
Winners: 2
Losers: 14
Avg Profit = 0.01885(3.59852619808521%)
Avg Loss = -0.00499285714285713(-0.936601238412572%)
3
TEVA
Winners: 10
Losers: 19
Avg Profit = 0.27478(2.40909228238562%)
Avg Loss = -0.0859473684210525(-0.708833719691789%)
4

The code for the PrintDebug and Streamwriter code are in the same context as you can see below. I put a counter in to see if i was looping more times than I thought, but it was equal to the # of symbols, which it should be. Still I thought I must be executing code within another loop I was not aware of, but I can't see it.

Is it possible a StreamWriter or AppendText bug could be causing this? Could there be some caching from the System variables that need to be flushed?

Code is below. Please let me know if you see something.
CODE:
Please log in to see this code.


profile picture

Eugene

#10
You're not trying to run this in Multi-Symbol Backtest mode (MSB), aren't you? This code must be run in Single-Symbol Backtest mode (SSB).

Note that File.AppendText does append text to the file on subsequent runs.

QUOTE:
Is it possible a StreamWriter or AppendText bug could be causing this?

Sure, bugs can be discovered even in .NET Framework 18 years after its release. ;)
profile picture

innertrader

#11
QUOTE:
You're not trying to run this in Multi-Symbol Backtest mode (MSB), aren't you?

Not clear what your question is (grammatically). Do you mean "...are you?"

Yes, I am trying to run a Dataset (testing 4 symbols). You are saying this can't be done with this code? Is there some other code I should be using as a model?

The program is generating correct values for mutisymbol backtest. I verified that by comparing to donor strategy. It outputs the results properly to PrintDebug, but duplicates data in StreamWriter file. Does that make sense?
QUOTE:
Note that File.AppendText does append text to the file on subsequent runs.

I'm confused by this comment and by the apparent behavior of the Streamwriter call. I am only calling Streamwirter 4 times in the code above (I think). But StreamWriter output looks as if I looped through the entire program 4 times, not just the foreach loop (4x4 = 16 lines of file output for 4 symbols).

How is StreamWriter reproducing the results for the entire set of 4 symbols 4 times? The results of the 4 symbols backtest are not being stored anywhere except in the text file. StreamWriter variables values are being changed each time through the loop. StreamWriter is acting as it it had saved the information in its own array.
profile picture

Eugene

#12
The file export works correctly. Your code above should be run on the first symbol (SSB mode) because this unsupported method generates a portfolio backtest (MSB) automatically:
CODE:
Please log in to see this code.

Let's not discuss the runDonor and related methods in this topic.
profile picture

superticker

#13
You can only call the line below once to open the file. The file automatically closes when exiting the using block.
CODE:
Please log in to see this code.

But you can call writer.WriteLine(...) as many times as you want to append records. So you need to place you data generating code inside the using block for StreamWriter, which is the way you would normally do it. Follow Microsoft's examples for using StreamWriter the "normal way". https://docs.microsoft.com/en-us/dotnet/standard/io/how-to-write-text-to-a-file

Keep in mind, when you exit the StreamWriter using block, the file gets closed on disk (i.e. last record flushes to disk). For record concatenation (employing WriteLine), you want to avoid exiting the using block until you have executed your last WriteLine statement.
profile picture

Eugene

#14
QUOTE:
But you can call writer.WriteLine(...) as many times as you want to append records.

...just not with this (unsupported) code which runs a portfolio backtest internally:
CODE:
Please log in to see this code.

Although a valid advice in general, appending does not make sense in this (specific) context.


UPDATE 02/26/2020

Loosely related discussion from this and other topics has been moved into a new thread:

Export a summary of performance stats by symbol programmatically
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).