# Is the difference in your results real — or just noise

Compare two scores or percentages and find out whether the gap between them is statistically significant, in seconds.

### Significance check

- **Concept A:** 72%  
- **Concept B:** 64%  
- **Confidence:** 95%  
- **0.03 p-value — significant**

Built for innovators. Loved by product teams. Trusted by leading brands.

## What is statistical significance?

Statistical significance tells you whether the difference between two results is likely real or just due to random chance. In consumer research, that means knowing whether a gap in purchase intent, liking scores, or preference percentages reflects a genuine difference — not sampling noise.

### Two tests, one answer

1. **Proportions z-test**  
   Compare two percentages — like 72% purchase intent for Concept A versus 64% for Concept B. Built for survey results and concept test scores.

2. **t-test on means**  
   Compare two average scores — like an overall liking of 7.2 versus 6.8 on a 9-point scale. Built for sensory testing and product ratings.

3. **Confidence level**  
   How sure you want to be before calling a difference real. Most researchers use 95%; exploratory work often uses 90%, high-stakes decisions 99%.

## Try it yourself

Choose your test type, enter your two results, and get an instant read on significance.

### Statistical Significance Calculator

- **Proportions (z-test)**  
- **Means (t-test)**

- **Sample Size A:**  
- **Proportion A (%):**  
- **Sample Size B:**  
- **Proportion B (%):**  
- **Confidence Level:**

**Statistically Significant**  
- **Test Statistic:**  
   z = 2.4254  
- **p-value:**  
   0.0114  
- **Confidence Level:**  
   95%

The 8-percentage-point difference (72% vs. 64%) is statistically significant at the 95% confidence level (p = 0.0114). This means we can be confident the difference reflects a real effect, not just random variation in the samples.

## Common significance-testing mistakes to avoid

- **Confusing significant with meaningful:**  
   A statistically significant difference tells you it’s real — not that it’s large enough to matter for your decision.

- **Testing with too few respondents:**  
   Small samples rarely reach significance even when a real difference exists. Plan for at least 100 per group for most proportion tests, 30+ for mean comparisons.

- **Ignoring a non-significant result:**  
   A result that isn’t significant doesn’t mean there’s no difference — just that your current data can’t prove one yet.

## Get the data worth testing

A significance test is only as good as the data behind it. Highlight’s end-to-end testing gives you real consumer scores to put into the calculator.

- Recruit the right respondents for your concept or sensory test  
- Collect clean, verified survey and rating data  
- Get results fast enough to test, learn, and iterate before launch

## Common statistical significance calculator questions

- **What does a p-value of 0.05 mean?**  
   There’s a 5% probability that the observed difference happened by chance alone — the standard threshold for calling a result statistically significant.

- **What’s the difference between a z-test and a t-test?**  
   A z-test compares two percentages, like purchase intent or top-box scores. A t-test compares two average scores, like overall liking on a rating scale.

- **How many respondents do I need?**  
   At least 100 respondents per group for most proportion tests, and 30 or more per group for mean comparisons. Fewer, and even a real difference may not reach significance.

- **Can I use this for A/B testing?**  
   Yes, but it’s optimized for product research scenarios — concept, sensory, and claims testing — where sample sizes are typically smaller than website A/B tests.

- **What if my result is not significant?**  
   It doesn’t mean there’s no difference — just that your current data can’t prove one. A larger sample or a clearer difference may still reach significance.