Your Health Check Is Lying to You

The canonical health check is three lines and looks responsible:

app.get("/health", (c) => c.json({ status: "ok" }));

It answers one question: is the process running and able to serve HTTP. That is a real question. The problem is that everyone points an uptime monitor at it and then believes it is answering a different question, which is whether the service works.

What that gap looks like

Our managed database exhausted a monthly compute quota. Every connection was refused at the connection layer. Every authenticated request returned a 500. Sign-in was broken, the API was broken, inbound mail was deferring.

The health check returned 200 the entire time, because it never touched the database. Nothing alerted for five days.

This is not an unusual failure. It is the default outcome of the default health check, and the more dependencies a service has, the wider the gap gets.

Why the obvious fix is wrong

The instinct is to make the health check query the database. For us that would have caused the outage rather than caught it.

The database in question suspends its compute after five idle minutes, and on that plan the suspend cannot be disabled. An uptime monitor hitting a database-touching endpoint every 60 seconds keeps the instance awake permanently. The quota is consumed by wall-clock time spent awake, not by query count. A health check polling every minute would have burned the entire monthly allowance on its own, then reported the outage it caused.

This is a specific instance of a general problem: an active health check is traffic. It costs connections, it costs quota, and it defeats anything that scales to zero. On a serverless database, on a connection-pool-limited Postgres, or anywhere you pay per unit of activity, an aggressive readiness probe is a real load source.

Observe the traffic you already have

A service under load is already asking the database thousands of questions a minute and getting answers. That is a far better health signal than a synthetic ping, and it costs nothing.

// Wrap the client once, where every caller already goes through.
prisma = new PrismaClient().$extends({
  query: {
    async $allOperations({ query, args }) {
      try {
        const result = await query(args);
        noteOk();
        return result;
      } catch (err) {
        noteFail(err);
        throw err;
      }
    },
  },
});

Then the endpoint reports what it has actually seen:

app.get("/health", (c) => {
  const db = dbHealth();
  return c.json(
    { status: db.status === "down" ? "degraded" : "ok", database: db },
    db.status === "down" ? 503 : 200,
  );
});

Three states, not two

The design detail that makes this work is a third state. Not just up and down, but idle: no traffic recently, so no opinion.

• ok: a query succeeded recently.
• down: the most recent outcome was a failure, within the window. Return 503.
• idle: nothing recent either way. Return 200, but say so.

Refusing to claim health you have not observed is the whole point. The original endpoint's sin was not that it was wrong, it was that it was confidently wrong about something it had never checked.

A single failed query does not trip it, because the next successful one supersedes it. Only a sustained outage keeps a failure as the most recent outcome, which is exactly the condition worth waking someone for.

The rule

A health check should either verify a dependency or refuse to speak for it. Returning 200 for a service you have not tested is worse than having no health check at all, because it converts an outage into an outage plus a false sense of safety.