An AI product fails quietly. The page loads, the API says 200, and the agent gives a wrong answer, the MCP server lost a tool, or the gateway fell back to another model. OpenPing checks those parts directly, from outside, the way your users meet them.
Real questions on a schedule and after every deploy. Rules check the answer (contains, leaves out, matches a schema, no personal data, no refusal), then an AI judge reads it against your rubric. You see the pass rate over time and hear when it drops.
The handshake on new and older protocol versions, the sign-in, the tool, resource and prompt lists, any change to a tool, and a safe read-only test call. Destructive tools are never called.
Next: the protocol’s own ping, and checks that a server refuses what it should, such as a call without a token. Coming
Every model your app uses, asked with the settings your app really sends: the system prompt shape, tools, reasoning settings, headers. Time to first token, and which model actually answered. A setting a new model refuses fails here before it fails for your users.
Everything a check reads is also read for text written to steer an AI: in pages, API answers, MCP tool lists, agent cards and agent answers. Invisible characters and hidden HTML included. On by default, on every plan.
Most teams test that their app works. Few test what it does when what reaches it is hostile or broken, and that is where AI apps fail. On paid plans, for apps you’ve shown are yours, OpenPing will test on a schedule that:
Every test signs in as a test account that sees only made-up data, never as a person. You get a grade and plain fixes, and an alert only when something that passed starts failing. On the free plan the report lists each test it didn’t run, so you can see what you’d get.
An MCP server, a REST API and plain-Markdown docs (llms.txt), so your coding agent can add the checks for you.
How TaskOS is checked: its chat, its MCP server, its uploads and scanners, its jobs and mail.
The go-live check grades any address in about two minutes. No setup.
Get early access