PR #9640 review: keep live token on double refresh

PR #9640 · Review workspace · head e730e667 · author harshithmullapudi · re-review (earlier version reviewed 2f87bacf)

Summary

Notion's MCP server (workers-oauth-provider) keeps a redeemed refresh token valid until its successor is used. So two API instances can both refresh with one token, and the instance that writes to the DB first can store the token the server already replaced. The user is then signed out at the next refresh, 8 h later. The PR splits refresh into redeem / store / reconcile. When the loser of the compare-and-swap had a successful redemption, it redeems the winner's token to find out which pair is live. It also makes 429 and non-JSON bodies ConnectorUnavailableError, and logs every refused refresh.

Verdict

minor issues The design is sound, and my earlier findings (GitHub/Slack dead-token codes, logging only some refusals) are fixed. But the recursive reconcile in the third-process case still stores a dead token. I reproduced this at head, and the fix is three lines.

Findings

Medium · correctness

packages/trpc/src/lib/connectors/refresh.ts:217: when our check redemption of the winner's token succeeds (check.kind === "issued") but its store loses to a third process, the recursive call still passes ours / ourRedeemedToken. The check just rotated the grant, so ours is dead, and check.tokens is live (unless the third process redeemed after us, and then the recursion finds that on its own). If the third process's token is also superseded, the recursion gets invalid_grant and stores ours. That is a dead token, which is the sign-out this PR is meant to stop. This is CodeRabbit's last open comment, and the head does not address it yet.

Repro (probe test, below): our redemption gives rotated-1. The winner stored winner, which is live. A third process C redeems winner → c and stores it. Then our check redeems winner (now previous) → rotated-2, which supersedes c. Our store loses, so we reconcile against c, which gives invalid_grant, and we store rotated-1. The server's live token is rotated-2.

Fix:

return check.kind === "issued"
	? await reconcile(latest, endpoints, check.tokens, winnerToken)
	: await reconcile(latest, endpoints, ours, ourRedeemedToken);

Keep the probe as the regression test. With the fix, the probe passes and the suite is 234/234.

Low · observability

packages/trpc/src/lib/connectors/refresh.ts:278: the reconcile check uses redeem, so the expected outcome "winner's token superseded, ours is live" logs refresh refused … oauthError: "invalid_grant". That line is the same as the one for a grant that is really dead (see the captured output below). The PR names these logs as the post-deploy check for SUPER-2973, so every lost race that the fix handles will look like a sign-out. Fix: pass a flag (or the call site) into redeem and log the reconcile result under a different message, for example refresh race reconciled, or log only at the call sites that mark needs_reauth.

Low · scope note

packages/trpc/src/lib/connectors/refresh.ts:200-213: reconcile runs for every connector whose loser redeemed successfully, not only for Notion. For non-rotating providers (the same refresh token comes back each time), every lost race now sends one extra token request and does one extra write. This is harmless and is in the PR description. A rejection with an error other than invalid_grant returns the winner unchanged, which is the behaviour on main. No change needed. I note it so the reviewer knows the blast radius.

Evidence

Surface: API library only (packages/trpc/src/lib/connectors/refresh.ts). There is no UI, desktop, or mobile surface, so I took no screenshots. I could not do a live run against a real Notion grant from two instances, because that needs a signed-in Notion account. The author says the same.

Probe at head e730e667 (stores the dead rotated-1, while the server's live token is rotated-2):

328 | 		await ensureFreshConnection(connection());
329 | 		console.log("server live:", grant.current, "| stored:", written.map((w) => w.refreshToken));
330 | 		expect(written).toEqual([
                        ^
error: expect(received).toEqual(expected)
@@ -1,4 +1,8 @@
  [
-   ObjectContaining {
-     "refreshToken": "rotated-2",
+   {
+     "accessToken": "access-1",
+     "disconnectReason": null,
+     "disconnectedAt": null,
+     "refreshToken": "rotated-1",
+     "tokenExpiresAt": 2026-11-12T08:29:32.919Z,
    },
- Expected  - 2
+ Received  + 6
      at <anonymous> (/workspace/packages/trpc/src/lib/connectors/refresh.test.ts:330:19)
(fail) a refresh race the server let both sides win > PROBE: keeps the check's tokens when a third process stores a superseded one [4.80ms]
 0 pass
 22 filtered out
 1 fail
 1 expect() calls
Ran 1 test across 1 file. [330.00ms]

Same probe with the 3-line fix applied locally (whole connectors and plugins suite):

server live: rotated-2 | stored: [ "rotated-2" ]
 234 pass
 0 fail

Reconcile's expected invalid_grant, logged like a dead grant (from the PR's own test "keeps our tokens when the winner stored the one the server superseded"):

[connectors/linear] refresh refused {
  connectionId: "conn-1",
  status: 400,
  oauthError: "invalid_grant",
}

 1 pass
Local fix + probe diff (not pushed)
diff --git a/packages/trpc/src/lib/connectors/refresh.test.ts b/packages/trpc/src/lib/connectors/refresh.test.ts
index fc63aa01..cd8e4771 100644
--- a/packages/trpc/src/lib/connectors/refresh.test.ts
+++ b/packages/trpc/src/lib/connectors/refresh.test.ts
@@ -310,6 +310,27 @@ describe("a refresh race the server let both sides win", () => {
 			expect.objectContaining({ refreshToken: grant.current }),
 		]);
 	});
+	test("PROBE: keeps the check's tokens when a third process stores a superseded one", async () => {
+		grant = { previous: "refresh", current: "rotated-0" };
+		const inner = globalThis.fetch;
+		let call = 0;
+		globalThis.fetch = (async (input: string | URL, init?: RequestInit) => {
+			call += 1;
+			// C reconciled against the winner just before our check did.
+			if (call === 2) grant = { previous: "winner", current: "c" };
+			const response = await inner(input, init);
+			if (call === 1) grant = { previous: "refresh", current: "winner" };
+			return response;
+		}) as typeof fetch;
+		reads = [connection(), winnerRow("winner"), winnerRow("c")];
+		swaps = [false, false];
+
+		await ensureFreshConnection(connection());
+		console.log("server live:", grant.current, "| stored:", written.map((w) => w.refreshToken));
+		expect(written).toEqual([
+			expect.objectContaining({ refreshToken: grant.current }),
+		]);
+	});
 });
 
 describe("a token endpoint that is merely down", () => {
diff --git a/packages/trpc/src/lib/connectors/refresh.ts b/packages/trpc/src/lib/connectors/refresh.ts
index ef59477d..e107c5df 100644
--- a/packages/trpc/src/lib/connectors/refresh.ts
+++ b/packages/trpc/src/lib/connectors/refresh.ts
@@ -214,7 +214,9 @@ async function reconcile(
 	// A third process stored first; its token may be the superseded one too.
 	const latest = await readConnection(winner.id);
 	if (!latest || latest.refreshToken === winner.refreshToken) return winner;
-	return await reconcile(latest, endpoints, ours, ourRedeemedToken);
+	return check.kind === "issued"
+		? await reconcile(latest, endpoints, check.tokens, winnerToken)
+		: await reconcile(latest, endpoints, ours, ourRedeemedToken);
 }
 
 type IssuedTokens = Record<string, unknown> & { access_token: string };

Questions for the author

Test / CI notes