Conversation
…4/401 The runtime proxy token is short-lived and expires well before an assignment is torn down, so an expired token produces the exact same 404/401 that a genuinely terminated runtime would. `exec`/`repl`/ `console`/`run` treated that error as proof the session was lost and deleted the local binding outright -- even though `list_assignments()` keeps listing the assignment (and keeps minting it a fresh proxy token) long after the stored token has stopped working. A live, billing runtime was being destroyed by its own client roughly once per token lifetime on any long-running session. `State.prune_or_recover_session()` asks the server before deleting: if the endpoint is still listed, adopt the fresh token/url and keep the session (this call still reports failure, but the next one succeeds); if the endpoint is confirmed gone, prune as before; if the control plane can't be reached at all, keep the session rather than delete on an inconclusive check. Fixes googlecolab#106.
|
Independent validation of this fix (3-hour A/B run) I backported the Setup: macOS, Python 3.14, google-colab-cli 0.6.0, jupyter-kernel-client 0.14.0,
The refreshed token's lifetime restarts at issuance (it expired again about 3600 s after each refresh), which matches One gap this PR doesn't cover: file operations. After the token expires, Thanks for the fix. It would be great to see it merged and released. |
Summary
The runtime-proxy token is short-lived, roughly 60 minutes (the server reports a 3600-second lifetime via
tokenExpiresInSeconds), and expires well before an assignment is actually torn down. An expired token produces the exact same 404/401 that a genuinely terminated runtime would.exec,repl,console, andruntreated any 404/401 as proof the session was lost and deleted the local binding outright, even thoughlist_assignments()keeps listing the assignment and keeps minting it a fresh proxy token long after the previously-stored token stops working. A live, still-billing runtime could be destroyed by its own client roughly once per hour on any session kept open longer than that.Fixes #106.
What changed
State.prune_or_recover_session()replaces the unconditional prune on this error path. Before deleting anything it asks the server. If the endpoint is still listed, it adopts the fresh token and url and keeps the local session; this invocation still fails, but the next one succeeds with the refreshed credential. If the endpoint is confirmed gone, it prunes as before. If the control plane is unreachable, it preserves the session rather than delete on an inconclusive check. This only runs reactively, in response to an actual 404/401 from the runtime proxy; there's no background or proactive token refresh.Testing
Full test suite passes. Added unit coverage for all three outcomes of
prune_or_recover_session, plus CLI-level tests covering both the pruned and preserved paths forexec.