byok.icu runs 7 standard agent test cases against your relay and the official baseline, slams the two results together, and lets any deviation surface.
Loading…
A relay can return answers and still not behave like the real thing. Each run looks at five dimensions, always compared with the official API answering the same cases.
| Dimension | The question it answers | What byok.icu looks at |
|---|---|---|
| Protocol compatibility | Does the relay speak the API it claims to? | Protocol auto-detected (OpenAI-compatible or Anthropic-compatible); model discovery; finish_reason and tool-call shape against the baseline. |
| Agent readiness | Can an agent actually work through it? | The 7 standard agent cases: tool calls, chained calls, error recovery, multi-file and model-context synthesis, long-context recall. Pass or fail per case. |
| Observability | Is what it reports true? | Input and output token counts against the baseline; request parameters changed before they reach the upstream model. |
| Performance | Is it slower than going direct? | Total latency per case and the drift against the baseline. |
| Reference deviation | Does it answer like the official model? | Each case side by side with the official baseline; boundary deviations, secret patterns in output and prompts that try to get the agent to reveal your key are flagged. |
One run takes a few minutes and needs nothing but the relay's endpoint and a key.
Paste the relay's API endpoint and the API key you use with it. byok.icu detects the protocol and lists the models the relay exposes; you can also type a model id.
After a quick human check, the 7 cases run against your relay and against the official baseline, and you can watch each case pass or fail live.
You get a trust score out of 100, a side-by-side table of metrics and cases, and a list of what is worth your attention. Share the result or compare it on the leader board.
A third-party service that resells or forwards access to models such as GPT or Claude through its own endpoint and key. Relays are often cheaper, but you cannot see what happens between your request and the model, which is what byok.icu measures.
It is used only for your run, scrubbed from logs and cleared after the run is dispatched. The page never shows it back to you or to anyone else.
Any relay with an OpenAI-compatible or Anthropic-compatible API. If the relay does not expose a model list, type the model id yourself.
It summarises how closely the relay matched the official baseline across the 7 cases, out of 100. Results are labelled trusted, solid, passable or weak, and every flag behind the score is listed so you can judge it yourself.
The top 20 relay models by trust score from completed evaluations, with endpoint, model, score, latency and pass rate.