Back Testing is defined well by the glossary on Stockcharts.com:

“Back Testing: A strategy that is optimized on historical data, then applied to current data to see if the results are similar. Rarely done properly and usually resorts to a form of curve fitting.”

The advantage of back testing signals or a complete system is that levels of risk, exits, and money management can be determined that best suites the system. Without testing, we can have no idea of the potential outcome of different entries, exits, and sizing methods. That’s what makes back testing useful.

What makes back testing useless is when it’s overused, curve fitted, and overfitted. Overfitting is when a system targets specific observations rather than general market structure. For example, since 2009 many investment professionals have started to using backtesting to create hypothetical performance that looked good during the October 2009 to March 2009 bear market. This is called “data snooping”.

When you see a silly looking chart that goes straight up with no drops, it’s overfitted… its past performance is not an indication of the future.