Subscribe via E-mail

Your email:

Our Blog

Current Articles | RSS Feed RSS Feed


Write a blog article!

Current Articles | RSS Feed RSS Feed

A CAUTIONARY "TAIL" ABOUT TESTING FOR PARITY

  
  
  
  
 

USE CAUTION IN SETTING ACTION STANDARDS FOR PARITY

Most market research testing is conducted looking for superiority: 

  • Is this new product better than the Current?
  • Is my company's product better than a product made by a competitor?
  • Which of several new commercials or concepts is most effective in generating purchase interest, in communicating a key copy point, in conveying the best image for a new product?

In these situations, action standards are traditionally established at the 90% or 95% level of confidence, using either a one-tail test or a two-tail test.  If there is an in-going hypothesis that the new product is better or if there is a desire to support an advertising claim of superiority, a one-tail test may be used.

However, sometimes the objective of a study is to determine if "parity" exists, for example between two products.  One of the more common examples of this is a Margin Improvement product (also called a Cost Reduction). The action standard to determine if the Margin Improvement is acceptable is generally set so that the reformulation should achieve parity with the Current product.  In this situation, using a two-test at the 90% level of confidence is not appropriate and could represent an unacceptable risk of alienating the franchise.     

While it seems contrary to expectations, when testing for parity, a lower level of confidence represents a more stringent test.  Historically, action standards for parity are established the 67% level of confidence using one-tail test) to ensure that there is no difference between the products.

The following hypothetical example has been created to illustrate the differences in interpretation that could occur if the wrong actions standards are used.  A Paired Comparison Home Use Test of a Margin Improvement vs. Current is conducted among 200 users of a brand.  Based on overall preference, 55% prefer Current and 45% prefer the Margin Improvement (no preference split equally).  If the action standard had been set as parity at the 90% level of confidence (two-tail test), the conclusion that would be reached is that the Margin Improvement is at parity with Current.  However, if the action standard had been set at the 67% level of confidence using a one-tail test, the Margin Improvement would be rejected because it fails to achieve parity with Current.  Using the wrong action standard could represent a risk of franchise alienation.

 

 

 

Action Standards

 

Total Sample(200)

%

90% level of confidence

(two-tail test)

67% level of confidence

(one-tail test)

Prefer Current

55

 

 

Prefer Margin Improvement

45

At parity

Fails to achieve parity

 

To put this another way, when two numbers are tested for significance at the 90% level of confidence, if they are not significantly different, the statistical conclusion that is drawn is that the two products are at parity.  However, it is unlikely that anyone would want to replace a Current product with a Margin Improvement that loses to the Current product at the 89% level of confidence. However, if an action standard of parity at the 90% level of confidence using a two-tail test, a loss at the 89% level of confidence vs. Current would pass the action standards.

Figures 1 and 2 illustrate the difference between using the 90% level of confidence two-tail test and the 67% level of confidence one-tail test for parity. 

Figure 1 shows that at the 90% level of confidence two-tail test, any preference scores that fall within the blue shaded area would be considered to be at parity.

 

Figure 1

90% Level of Confidence-Two-Tail Test

90% Level of Confidence-Two-Tail Test

If testing for parity is at the 90% level of confidence (two-tail test), anything within these lines is at parity. Based on a sample of 200 respondents, a Margin Improvement would be considered to be at parity if it lost by a margin of 55% to 45%.

 

Figure 2 shows the 67% level of confidence. In this example, only preference scores that fall within the red shaded area would be considered to be at parity.  

Figure 2

90% Level of Confidence-Two-Tail Test 67% Level of Confidence-One-Tail Test

67% Level of Confidence-One-Tail Test

If testing for parity is at the 67% level of confidence (one-tail test), only losing preference scores that fall within the red area would be judged at parity. Based on a sample of 200 respondents, a Margin Improvement would have to achieve a preference score of at least 48.5% (vs 51.5% for the Current) in order to be considered at parity.

 

Conclusion

When the action standards for a study are established, make sure that they are appropriate and will result in the correct decision for the brand or product. 

 

 

If you have any questions, please contact:

David Buchler

Executive Vice President

Target Research Group, Inc.

845-426-1200, ext. 24

david.buchler@targetresearchgroup.com

 

 

Comments

Currently, there are no comments. Be the first to post one!
Post Comment
Name
 *
Email
 *
Website (optional)
Comment
 *

Allowed tags: <a> link, <b> bold, <i> italics