I want to go over in more detail comparing models. First, let me tell you a little about creating models when you first start. This is the screen that you start with :
I recommend leaving everything set the way it comes to start with. Leave the simple mode checked. We will get into the advanced mode later. I choose the 'End of Day' field check as opposed to the 'End of Week'. I also leave the start date for downloading history at the preset date, although you could go back more than a years worth of data. The reserved historical period for testing I also leave the way it comes set. The only thing I change is the model type for each of the five models that I am going to run. This is why I rename each model to reflect the model type it represents, thus FORD-1 through FORD-5.
Anyway, when I get all 5 models of the stock run, I then want to do a comparison of all 5 systems. This is easily done by going to the 'Analysis' tab at the top and choosing 'Strategy Performance Comparison'.
This will bring up a box of all models that have been run. In this case I want to compare all 5 of the Ford models I have created:
I click the 'OK' button and let it run. It then will produce a report comparing all 5 models as such:
This way I can compare the performance of each model and select the one I want to go with. You may want to choose more than one model, but I usually choose the best one as far as total net profit, and the winning percentage . But that's just me.
Originally I ran all 5 of these models with a stop-loss of 5 percent. Then I ran all 5 again with no stop-loss just to see the difference in the models. I made myself a little spreadsheet to compare stop-loss vs. no stops. But I could have easily let the software do it for me just by creating models that were differentiated with a different model name, like FORD-1 SL for the type 1 model with a stop-loss or FORD-1 NSL for the type 1 model with no stop-loss. I actually should have done this, but below is the comparison I came up with between the two sets of models :
After adding the lines of net profit up on both sets of models, I saw no clear winner using a stop-loss vs. no stop loss. Although the largest net profit came from FORD-3 without a stop loss, and with the most number of trades (20) , the model FORD-4 with a 5 percent stop showed 100 percent profitability with a total of 14 trades.
At this point I could choose both models to run with, renaming them FORD1 and FORD2, and just deleting the others. But the choice would be up to you.
You might ask "How do I choose the stocks to model?" In my next post, I will tell you how I pick the stocks that I use in the software. One thing I will let you know is that this year alone I have made a total gain of just over 128% on just one stock I have traded!
Have a great day,
Roger
Nice article, do you use insample/ outofsample data testing, or use best in 5 models and start trading live. How often do you relearn the system
ReplyDeleteI just use the best of the 5 models that I create. There is no need to relearn the system unless you want to choose more historical data, or want to change the parameters of the learning process. Once a model is learned, it will stay that way unless you want to change the data set it learns from. Back in 1985 all we had to work with in college was Sperry/Rand main frames and a slew of compilers like Fortran, Cobal, RPG, Assembler and the like. It took hours just to come up with a usable database to compile. It is amazing to look at how far things have progressed over the years. I feel like a dinosaur sometimes, but the concepts are still the same.
ReplyDeleteThe insample data is all that I use, although it might be interesting to see what outofsample would do. I just see it as applying what the program has learned to a data set that it didn't originally compute from. It might actually confirm its algorithm, although it is much more than that. I might try that sometime just to compare results.
THANKS
ReplyDelete