Exact A/B test sample size, duration, z-test, chi-square, Fisher-Freeman-Halton and peeking risk.
Maintain this server? Claim it
Call this server's tools from your browser. Nothing to install.
Requests go through mcptry because browsers can't call most MCP servers directly. Header values are sent to the server for that request only and are never stored or logged.
Tool list from our last check. Connect live to refresh it.
Exact sample size per variant for an A/B test on a conversion rate (two independent proportions): baseline rate, minimum detectable effect (relative or absolute), alpha, power, one or two-sided, several variants with optional Bonferroni, and unequal allocation. Uses the pooled/unpooled normal formula (same as R power.prop.test) and also reports the figure Evan Miller's calculator gives, so the two can be reconciled.
Minimum detectable effect. Relative by default (0.10 = a 10% lift); set mde_type "absolute" for percentage points as a proportion (0.01 = 1 point).
Control conversion rate as a proportion, for example 0.042 for 4.2%.
Significance level, default 0.05.
Statistical power, default 0.80.
two-sided (default) or one-sided test.
relative (default) or absolute.
Number of arms including control. 2 = A/B, 3 = A/B/C.
Split alpha across the variant-vs-control comparisons (Bonferroni).
Users in each variant per user in control. 1 = even split.
Pick your client. Values in capitals, like YOUR_TOKEN, are yours to fill in.
claude mcp add --transport http ab-test-sample-size-calculator-mcp https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp
Exact sample size per variant for an A/B test on a conversion rate (two independent proportions): baseline rate, minimum detectable effect (relative or absolute), alpha, power, one or two-sided, several variants with optional Bonferroni, and unequal allocation. Uses the pooled/unpooled normal formula (same as R power.prop.test) and also reports the figure Evan Miller's calculator gives, so the two can be reconciled.
| Argument | Type | Description |
|---|---|---|
| mde* | number | Minimum detectable effect. Relative by default (0.10 = a 10% lift); set mde_type "absolute" for percentage points as a proportion (0.01 = 1 point). |
| alpha | number | Significance level, default 0.05. |
| power | number | Statistical power, default 0.80. |
| sided | string | two-sided (default) or one-sided test. |
| mde_type | string | relative (default) or absolute. |
| variants | integer | Number of arms including control. 2 = A/B, 3 = A/B/C. |
| bonferroni | boolean | Split alpha across the variant-vs-control comparisons (Bonferroni). |
| baseline_rate* | number | Control conversion rate as a proportion, for example 0.042 for 4.2%. |
| allocation_ratio | number | Users in each variant per user in control. 1 = even split. |
{
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"baseline_rate",
"mde"
],
"properties": {
"mde": {
"type": "number",
"description": "Minimum detectable effect. Relative by default (0.10 = a 10% lift); set mde_type \"absolute\" for percentage points as a proportion (0.01 = 1 point).",
"exclusiveMinimum": 0
},
"alpha": {
"type": "number",
"default": 0.05,
"description": "Significance level, default 0.05.",
"exclusiveMaximum": 0.5,
"exclusiveMinimum": 0
},
"power": {
"type": "number",
"default": 0.8,
"description": "Statistical power, default 0.80.",
"exclusiveMaximum": 1,
"exclusiveMinimum": 0.5
},
"sided": {
"enum": [
"two",
"one"
],
"type": "string",
"default": "two",
"description": "two-sided (default) or one-sided test."
},
"mde_type": {
"enum": [
"relative",
"absolute"
],
"type": "string",
"default": "relative",
"description": "relative (default) or absolute."
},
"variants": {
"type": "integer",
"default": 2,
"maximum": 20,
"minimum": 2,
"description": "Number of arms including control. 2 = A/B, 3 = A/B/C."
},
"bonferroni": {
"type": "boolean",
"default": false,
"description": "Split alpha across the variant-vs-control comparisons (Bonferroni)."
},
"baseline_rate": {
"type": "number",
"description": "Control conversion rate as a proportion, for example 0.042 for 4.2%.",
"exclusiveMaximum": 1,
"exclusiveMinimum": 0
},
"allocation_ratio": {
"type": "number",
"default": 1,
"maximum": 10,
"description": "Users in each variant per user in control. 1 = even split.",
"exclusiveMinimum": 0
}
},
"additionalProperties": false
}What peeking does to an A/B test: the real false positive rate if you check a fixed-horizon test K times and stop at the first p < alpha (for example about 14% at 5 looks with alpha 0.05), and the Pocock and O'Brien-Fleming group sequential boundaries (z and nominal p per look) that keep the overall rate at alpha, for up to 10 equally spaced looks.
| Argument | Type | Description |
|---|---|---|
| alpha | number | Significance level, default 0.05. |
| looks* | integer | How many times the result is checked, including the final look. Equally spaced. |
{
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"looks"
],
"properties": {
"alpha": {
"type": "number",
"default": 0.05,
"description": "Significance level, default 0.05.",
"exclusiveMaximum": 0.5,
"exclusiveMinimum": 0
},
"looks": {
"type": "integer",
"maximum": 50,
"minimum": 1,
"description": "How many times the result is checked, including the final look. Equally spaced."
}
},
"additionalProperties": false
}How many days an A/B test must run: from a known sample size per variant, or from baseline rate and MDE (computed as in calculate_sample_size), plus daily visitors and the share of traffic in the test. Returns exact days and the recommended whole weeks (to cover weekday effects).
| Argument | Type | Description |
|---|---|---|
| mde | number | Minimum detectable effect (relative by default). |
| alpha | number | Significance level, default 0.05. |
| power | number | |
| mde_type | string | |
| variants | integer | Number of arms including control. |
| baseline_rate | number | Control conversion rate, for computing the sample size. |
| traffic_share | number | Fraction of those visitors entered into the test, default 1. |
| daily_visitors* | number | Eligible visitors per day across the whole site or page. |
| sample_size_per_variant |
Exact p-value for a contingency table of counts. 2x2: Fisher exact test (two-sided, greater or less, as R fisher.test). Larger r x c tables (for example 3 variants x converted/not, or 4 x 4 survey tables): Fisher-Freeman-Halton exact test by a network algorithm. Also returns Pearson chi-square and how many cells have expected counts below 5, so you can see when the chi-square approximation is unsafe.
| Argument | Type | Description |
|---|---|---|
| table* | array | Rows of counts, for example [[12, 488], [25, 475]] (rows = groups, columns = outcomes). |
| alternative | string | One-sided options apply to 2x2 only; greater = odds ratio above 1. |
{
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"table"
],
"properties": {
"table": {
"type": "array",
"items": {
"type": "array",
"items": {
"type": "integer",
"minimum": 0
},
"maxItems": 12,
"minItems": 2
},
"maxItems": 12,
"minItems": 2,
"description": "Rows of counts, for example [[12, 488], [25, 475]] (rows = groups, columns = outcomes)."
},
"alternative": {
"enum": [
"two-sided",
"greater",
"less"
],
"type": "string",
"default": "two-sided",
"description": "One-sided options apply to 2x2 only; greater = odds ratio above 1."
}
},
"additionalProperties": false
}Is an A/B test result statistically significant? From visitors and conversions per group: two-proportion z-test (pooled), one or two-sided p-value, Wald confidence interval for the difference, relative lift, Pearson chi-square with and without Yates correction (R prop.test), Fisher exact p for the same 2x2 table, and a sample ratio mismatch (SRM) check against the planned split.
| Argument | Type | Description |
|---|---|---|
| alpha | number | Significance level, default 0.05. |
| alternative | string | greater = variant converts better than control. |
| expected_split | number | Planned share of traffic in control, for the SRM check. Default 0.5. |
| control_visitors* | integer | |
| variant_visitors* | integer | |
| control_conversions* | integer | |
| variant_conversions* | integer |
We read every tool's name, description and inputs the way the model will, looking for instructions hidden from you. A clean result means no warning signs, not a guarantee. Scanned Nov 2, 2026.
| integer |
| Known users per arm. If omitted, give baseline_rate and mde. |
{
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"daily_visitors"
],
"properties": {
"mde": {
"type": "number",
"description": "Minimum detectable effect (relative by default).",
"exclusiveMinimum": 0
},
"alpha": {
"type": "number",
"default": 0.05,
"description": "Significance level, default 0.05.",
"exclusiveMaximum": 0.5,
"exclusiveMinimum": 0
},
"power": {
"type": "number",
"default": 0.8,
"exclusiveMaximum": 1,
"exclusiveMinimum": 0.5
},
"mde_type": {
"enum": [
"relative",
"absolute"
],
"type": "string",
"default": "relative"
},
"variants": {
"type": "integer",
"default": 2,
"maximum": 20,
"minimum": 2,
"description": "Number of arms including control."
},
"baseline_rate": {
"type": "number",
"description": "Control conversion rate, for computing the sample size.",
"exclusiveMaximum": 1,
"exclusiveMinimum": 0
},
"traffic_share": {
"type": "number",
"default": 1,
"maximum": 1,
"description": "Fraction of those visitors entered into the test, default 1.",
"exclusiveMinimum": 0
},
"daily_visitors": {
"type": "number",
"description": "Eligible visitors per day across the whole site or page.",
"exclusiveMinimum": 0
},
"sample_size_per_variant": {
"type": "integer",
"description": "Known users per arm. If omitted, give baseline_rate and mde.",
"exclusiveMinimum": 0
}
},
"additionalProperties": false
}{
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"control_visitors",
"control_conversions",
"variant_visitors",
"variant_conversions"
],
"properties": {
"alpha": {
"type": "number",
"default": 0.05,
"description": "Significance level, default 0.05.",
"exclusiveMaximum": 0.5,
"exclusiveMinimum": 0
},
"alternative": {
"enum": [
"two-sided",
"greater",
"less"
],
"type": "string",
"default": "two-sided",
"description": "greater = variant converts better than control."
},
"expected_split": {
"type": "number",
"default": 0.5,
"description": "Planned share of traffic in control, for the SRM check. Default 0.5.",
"exclusiveMaximum": 1,
"exclusiveMinimum": 0
},
"control_visitors": {
"type": "integer",
"exclusiveMinimum": 0
},
"variant_visitors": {
"type": "integer",
"exclusiveMinimum": 0
},
"control_conversions": {
"type": "integer",
"minimum": 0
},
"variant_conversions": {
"type": "integer",
"minimum": 0
}
},
"additionalProperties": false
}