What a Control Group Looks Like for a Trading Rule
By the time the first trade of the morning hits the tape, the data set for the day is already partially formed, a reality noted at orb trading review consultoriainnova. A trader looks at an opening range breakout and sees a signal, but the math requires a comparison against the trades that were filtered out. Testing a strategy requires more than just looking at the wins that remain after a filter is applied.
The Ghost Data Problem
A filter works by removing specific setups from a baseline. If a rule excludes every trade that occurs during the first fifteen minutes of the session, the resulting data set is smaller. This smaller set might show a higher win rate, but that figure is incomplete. The performance of the original unfiltered list represents the control group. Without the data from the excluded trades, the true impact of the filter stays hidden. A mistake occurs when the filtered results are treated as the total reality instead of a subset of a larger whole.
Reconstructing the Control Group
Every time a rule is applied to an intraday timeframe, a shadow list of rejected trades is created. If a 15 minute range filter is used to avoid volatility, the trades that would have occurred during that window must be logged. These are the trades that the rule would have stopped. Comparing the win rate of the filtered trades against the win rate of the unfiltered trades reveals if the filter actually adds value. A filter that removes a high number of winning trades to stop a few losers is a net loss for the equity curve. The comparison must stay active throughout the entire backtest.
Measuring the Delta
The difference between the original sample and the filtered sample is the only way to measure effectiveness. If the original orb results show a profit factor of 1.5, and the filtered results show a profit factor of 1.6, the filter has provided a marginal gain. However, if the filter also reduced the total number of trades by fifty percent, the cost of that filter is high. The math shows that the gain in efficiency might not compensate for the loss in opportunity. Every trade removed from the 5 minute data set must be accounted for in the final calculation.
Avoiding Selection Bias
A common error involves looking only at the trades that passed the filter. This creates a false sense of accuracy. If a rule is designed to ignore the market open, the filtered results only show what happened after the volatility subsided. The data must include the performance of the trades that were rejected to ensure the filter is not simply cherry picking favorable conditions. The control group is the baseline from which all deviations are measured.