com.llmcutover/auditv1.0.0 Domain verified
Checks if a cheaper model agrees with your GPT, Claude or Gemini outputs, and what routing saves.
Maintain this server? Claim it
Try it
Call this server's tools from your browser. Nothing to install.
https://api.llmcutover.com/mcp
Optional. 'Bearer <full-audit pass>' for paid runs; without it you get the free tier (3 audits/day, 10 samples).
Requests go through mcptry because browsers can't call most MCP servers directly. Header values are sent to the server for that request only and are never stored or logged.
Tool list from our last check. Connect live to refresh it.
list_models Read-only
Current models with $/1M token prices and effort levels, usable as current.model.
This tool takes no arguments.
Install
Pick your client. Values in capitals, like YOUR_TOKEN, are yours to fill in.
Run in a terminal
claude mcp add --transport http audit https://api.llmcutover.com/mcp
Headers
Authorization(optional, not in the config above): Optional. 'Bearer <full-audit pass>' for paid runs; without it you get the free tier (3 audits/day, 10 samples).
Tools (2)
list_modelsCurrent models with $/1M token prices and effort levels, usable as current.model.
read-only
list_models
Current models with $/1M token prices and effort levels, usable as current.model.
No arguments.
Input schema (JSON)
{
"type": "object",
"properties": {}
}run_cutover_auditMeasures how often a cheaper model (Sage 2) agrees with your current LLM on your samples, the confidence threshold for routing, and the monthly cost now vs a hybrid. Returns the full report as JSON.
read-only
run_cutover_audit
Measures how often a cheaper model (Sage 2) agrees with your current LLM on your samples, the confidence threshold for routing, and the monthly cost now vs a hybrid. Returns the full report as JSON.
| Argument | Type | Description |
|---|---|---|
| mode* | string | yesno: a yes/no decision per text. extract: fields pulled out of each text. |
| usage | object | |
| fields | object | extract only. Up to 12 fields: {"name": {"type": "string|number|integer|boolean|date|money", "definition": "..."}} |
| target | number | Agreement you need on the calls the cheap model keeps. |
| current* | object | Your current model: {"model": "<id from GET /api/models>", "effort": "<level>"} or custom prices {"in": USD per 1M input tokens, "out": USD per 1M output tokens}. |
| samples* | any | Real inputs with the answer your CURRENT model gave. Array of objects or a JSONL/CSV string. yesno: {"text","label":"yes|no"}; extract: {"text","expected":{...}}. Each text ≤ 4000 chars. Min 5. Max 10 anonymous, 25 free on the website, 500 yesno / 200 extract with a pass. |
| question | string | yesno only. The yes/no question your current model answers, e.g. "Is this ticket about billing?" |
Input schema (JSON)
{
"type": "object",
"required": [
"mode",
"samples",
"current"
],
"properties": {
"mode": {
"enum": [
"yesno",
"extract"
],
"type": "string",
"description": "yesno: a yes/no decision per text. extract: fields pulled out of each text."
},
"usage": {
"type": "object",
"properties": {
"monthlyCalls": {
"type": "integer",
"default": 100000,
"maximum": 1000000000,
"minimum": 0
},
"outputTokens": {
"type": "integer",
"maximum": 4000,
"minimum": 0
},
"reasoningTokens": {
"type": "integer",
"maximum": 50000,
"minimum": 0,
"description": "Measured reasoning tokens per call, if you know them; otherwise estimated from effort."
},
"promptOverheadTokens": {
"type": "integer",
"default": 400,
"maximum": 20000,
"minimum": 0
}
}
},
"fields": {
"type": "object",
"description": "extract only. Up to 12 fields: {\"name\": {\"type\": \"string|number|integer|boolean|date|money\", \"definition\": \"...\"}}",
"additionalProperties": {
"type": "object",
"properties": {
"type": {
"enum": [
"string",
"number",
"integer",
"boolean",
"date",
"money"
],
"type": "string"
},
"definition": {
"type": "string",
"maxLength": 300
}
}
}
},
"target": {
"enum": [
0.95,
0.98,
0.99
],
"type": "number",
"default": 0.98,
"description": "Agreement you need on the calls the cheap model keeps."
},
"current": {
"type": "object",
"properties": {
"in": {
"type": "number",
"maximum": 1000,
"minimum": 0
},
"out": {
"type": "number",
"maximum": 1000,
"minimum": 0
},
"model": {
"enum": [
"claude-fable-5.1",
"claude-opus-5.5",
"claude-sonnet-5.5",
"claude-haiku-5.5",
"gpt-6-astra",
"gpt-5.6-sol",
"gpt-6.1-sol",
"gpt-6.1-terra",
"gpt-5-mini",
"gpt-6-luna",
"gemini-4-argon",
"gemini-3.8-flash",
"gemini-3.1-flash-lite"
],
"type": "string"
},
"effort": {
"type": "string"
}
},
"description": "Your current model: {\"model\": \"<id from GET /api/models>\", \"effort\": \"<level>\"} or custom prices {\"in\": USD per 1M input tokens, \"out\": USD per 1M output tokens}."
},
"samples": {
"oneOf": [
{
"type": "array",
"items": {
"type": "object",
"required": [
"text"
],
"properties": {
"text": {
"type": "string"
},
"label": {
"enum": [
"yes",
"no"
],
"type": "string"
},
"expected": {
"type": "object"
}
}
},
"minItems": 5
},
{
"type": "string"
}
],
"description": "Real inputs with the answer your CURRENT model gave. Array of objects or a JSONL/CSV string. yesno: {\"text\",\"label\":\"yes|no\"}; extract: {\"text\",\"expected\":{...}}. Each text ≤ 4000 chars. Min 5. Max 10 anonymous, 25 free on the website, 500 yesno / 200 extract with a pass."
},
"question": {
"type": "string",
"maxLength": 600,
"description": "yesno only. The yes/no question your current model answers, e.g. \"Is this ticket about billing?\""
}
}
}Security checkA100/100
We read every tool's name, description and inputs the way the model will, looking for instructions hidden from you. A clean result means no warning signs, not a guarantee. Scanned Nov 3, 2026.
- Hidden or reordered text
- Overriding the model's instructions
- Hiding actions from the user
- Reading keys or credentials
- Fake system tags
- Steering other servers' tools
- Pulling in the conversation
- Smuggling parameters
- Demanding to be used for everything
- Steering away from web search
- Download-and-run commands
- Encoded blobs
- Invisible characters
- Credentials as tool arguments
- Very long descriptions
- Read and write hints