A CAUTIONARY "TAIL" ABOUT TESTING FOR PARITY
Posted on Tue, Mar 23, 2010 @ 01:47 PM
USE CAUTION IN SETTING ACTION STANDARDS FOR PARITY
Most market research testing is conducted looking for superiority:
 Is this new product better than the Current?
 Is my company's product better than a product made by a competitor?
 Which of several new commercials or concepts is most effective in generating purchase interest, in communicating a key copy point, in conveying the best image for a new product?
In these situations, action standards are traditionally established at the 90% or 95% level of confidence, using either a onetail test or a twotail test. If there is an ingoing hypothesis that the new product is better or if there is a desire to support an advertising claim of superiority, a onetail test may be used.
However, sometimes the objective of a study is to determine if "parity" exists, for example between two products. One of the more common examples of this is a Margin Improvement product (also called a Cost Reduction). The action standard to determine if the Margin Improvement is acceptable is generally set so that the reformulation should achieve parity with the Current product. In this situation, using a twotest at the 90% level of confidence is not appropriate and could represent an unacceptable risk of alienating the franchise.
While it seems contrary to expectations, when testing for parity, a lower level of confidence represents a more stringent test. Historically, action standards for parity are established the 67% level of confidence using onetail test) to ensure that there is no difference between the products.
The following hypothetical example has been created to illustrate the differences in interpretation that could occur if the wrong actions standards are used. A Paired Comparison Home Use Test of a Margin Improvement vs. Current is conducted among 200 users of a brand. Based on overall preference, 55% prefer Current and 45% prefer the Margin Improvement (no preference split equally). If the action standard had been set as parity at the 90% level of confidence (twotail test), the conclusion that would be reached is that the Margin Improvement is at parity with Current. However, if the action standard had been set at the 67% level of confidence using a onetail test, the Margin Improvement would be rejected because it fails to achieve parity with Current. Using the wrong action standard could represent a risk of franchise alienation.


Action Standards 

Total Sample(200)
% 
90% level of confidence
(twotail test) 
67% level of confidence
(onetail test) 
Prefer Current 
55 


Prefer Margin Improvement 
45 
At parity 
Fails to achieve parity 
To put this another way, when two numbers are tested for significance at the 90% level of confidence, if they are not significantly different, the statistical conclusion that is drawn is that the two products are at parity. However, it is unlikely that anyone would want to replace a Current product with a Margin Improvement that loses to the Current product at the 89% level of confidence. However, if an action standard of parity at the 90% level of confidence using a twotail test, a loss at the 89% level of confidence vs. Current would pass the action standards.
Figures 1 and 2 illustrate the difference between using the 90% level of confidence twotail test and the 67% level of confidence onetail test for parity.
Figure 1 shows that at the 90% level of confidence twotail test, any preference scores that fall within the blue shaded area would be considered to be at parity.
Figure 1
90% Level of ConfidenceTwoTail Test
If testing for parity is at the 90% level of confidence (twotail test), anything within these lines is at parity. Based on a sample of 200 respondents, a Margin Improvement would be considered to be at parity if it lost by a margin of 55% to 45%. 
Figure 2 shows the 67% level of confidence. In this example, only preference scores that fall within the red shaded area would be considered to be at parity.
Figure 2
67% Level of ConfidenceOneTail Test
If testing for parity is at the 67% level of confidence (onetail test), only losing preference scores that fall within the red area would be judged at parity. Based on a sample of 200 respondents, a Margin Improvement would have to achieve a preference score of at least 48.5% (vs 51.5% for the Current) in order to be considered at parity. 
Conclusion
When the action standards for a study are established, make sure that they are appropriate and will result in the correct decision for the brand or product.
If you have any questions, please contact:
David Buchler
Executive Vice President
Target Research Group, Inc.
8454261200, ext. 24
david.buchler@targetresearchgroup.com