> For the complete documentation index, see [llms.txt](https://keak.gitbook.io/keak-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://keak.gitbook.io/keak-docs/basics/images-and-media/statistical-significance-criteria.md).

# Statistical Significance Criteria

By default, Keak uses robust statistical calculations on each new visitor to evaluate significance.​

**Statistical significance** is calculated continuously so that each test is exposed to the minimum number of visitors required to get a trustworthy result​. This means Keak won’t unnecessarily keep a test running once it has enough evidence to pick a winner, saving you traffic and time.

Before Keak concludes a test, **two conditions** must be met​:

1. **Confidence level threshold is reached:** The current confidence (statistical certainty) that one variation is better than the other must exceed 95%.
2. **Minimum sample size is met:** The test has accumulated at least the minimum number of impressions (visitors) you specified. This is to ensure the result isn’t called too early on very little data. For example, you might require at least 1000 visitors per variation before trusting the outcome, even if confidence shoots high with fewer views. 

Only when *both* criteria are satisfied will Keak stop the test and declare a winner​. If the confidence level is high but you haven’t hit the minimum sample, the test will continue (and vice versa). These safeguards maintain test integrity and give you reliable results.

***

### Here is how the confidence level is calculated: 

1. **Conversion Rate Comparison** \
   We calculate each variation’s conversion rate by dividing conversions by total visitors. We then use a standard statistical test (a two-proportion z‑test) to see whether any difference in these rates is large enough to be more than random fluctuation.
2. **Confidence & p‑Value**\
   The two-proportion z‑test produces a p‑value, which tells us how likely it is to see a difference as large as we observed if, in reality, there were no true difference. If the p‑value is below 0.05, we conclude that the difference is statistically significant.
3. **Automated Decision-Making**\
   Our platform continuously monitors each test’s p‑value and automatically declares a winner when the difference meets or exceeds the specified confidence level (95% by default).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://keak.gitbook.io/keak-docs/basics/images-and-media/statistical-significance-criteria.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.