Symptoms
You run an optimization, you double click one of the entries in the Optimization Results tab, run a backtest with that parameter set and get different results.
Cause
This can have several potential causes:
- The first thing you should take a look at is the Spread dropdown in the backtest pane. You should either have it set to Variable or otherwise a fixed number should be configured. If it's set to Current, that's your problem right there, the spread will be different between the optimization and the backtest which will result in different outcomes.
- If the backtest result has the same number of trades as the selected optimization pass, it could be that you have the Slippage setting enabled (in the Tick Data Settings dialog). Since by default optimizations will not have slippage (the Optimization slippage checkbox is disabled in the Slippage tab), your optimization pass was running without slippage and your backtest had slippage enabled. If this is the case, simply disable Slippage to get the same results.
Side note: I recommend not using slippage during optimization because it may have misleading results due to its randomness - some passes might get lucky while others might get unlucky. Instead you can optimize without slippage and once you find a good set of parameters, you can run it through a sort of trial-by-fire by enabling adverse slippage, increased spreads etc. - If the backtest result has the same number of trades as the optimization pass and you are running your backtest and/or optimization during the week, chances are that the difference is caused by the fluctuations in the price of the currency pair used to convert the earnings from the quote currency of your backtest symbol to the backtest currency you configured.
For example, if you are backtesting AUDCAD and you configured USD as backtest currency in the Expert Properties dialog, the MT4 backtesting engine will use the current USDCAD price to convert the backtest earnings from CAD to USD. If you are backtesting during the weekend, there is no problem but if you are backtesting during the week the USDCAD price fluctuates constantly and it will result in slightly different results even between two consecutive backtests with the same settings.
If this describes your problem, an easy solution is to use the quote currency as the backtest currency to avoid conversions. In the case described above, you would have to use CAD as your backtest currency. - I would recommend looking at #2 and #3 above if your backtest has a different number of trades, especially if the EA you are backtesting involves some sort of martingale, grid trading or otherwise is an EA that is almost always in the market.
- If your EA is using any kind of chart objects (e.g. trend lines, support lines etc.) and relying on their use, you need to be aware that chart objects are not working at all during optimizations and your results may differ in very significant ways because of that.
- If you're working on the EA, recompiling the source code after you started the optimization but before you started the backtest will result in the backtest running with the newly compiled source code while the optimization would continue to run with the old code. This may of course result in differences due to the changes in the code. If you suspect this to be the problem, simply run the optimization and then the backtest once more, refraining from recompiling the source code in the meantime.
- Make sure that the optimization is not using an existing read-only FXT - this may cause issues especially if used in conjunction with the Use date checkbox. If you haven't changed any settings in the Misc tab of the Tick Data Settings dialog, this definitely won't be an issue. If you're uncertain, simply click the Defaults button in the Misc tab and then click Ok.
- If none of the above points covers your situation, I would recommend ensuring that you're using the same settings - Use Date if you have it enabled with the same date range as well as the settings in the Tick Data Suite dialog. If you did nothing other than double-clicking the result set and then clicking Start, your backtest is likely using the same settings. Also make sure that you are not running several MT4 instances from the same folder as that can have potential adverse effects when running parallel optimizations.
If your settings are identical, chances are the problem is related to the EA you are backtesting. To check if this is the case, you can run an optimization of the MACD Sample EA using the same settings, double-click one of its passes and run a backtest to ensure that the results are identical to those obtained during the optimization pass. Make sure you right click in the Optimization Results tab and disable the Skip Useless Results option - the MACD Sample EA tends to produce negative results which wouldn't be visible otherwise. If the MACD Sample EA produces identical results, the problem must be located within the EA that you are backtesting.
As a final note, each released version of the Tick Data Suite undergoes a very extensive automated test phase executed on multiple operating systems. In the context of this article it's worth mentioning that among the tests performed there are several tests that ensure that running an optimization will result in identical ticks as running a backtest with the same settings. This should leave no room for error as far as the Tick Data Suite is concerned.