fix: double free if a plan is cached and rerun - #645
Conversation
There was a problem hiding this comment.
🔵 Needs a closer look
It makes substantial unsafe/memory-context and plan-lifecycle changes in core scan serialization/execution paths that warrant careful human validation beyond automated review.
Pull request overview
This PR fixes a backend crash/double-free scenario when Postgres caches a foreign scan plan and re-executes it (e.g., via PREPARE/EXECUTE). It does so by ensuring the data passed through fdw_private is safely deep-copyable across planning/execution boundaries and by preventing FDW instances from being constructed during planning.
Changes:
- Reworked scan planning/execution state handoff: serialize planning-time scan inputs into a Postgres
ListofConstnodes (safe forcopyObject) and reconstruct a freshFdwStateper execution. - Updated the
ForeignDataWrappertrait planning hooks to be instance-free (get_rel_size,supported_aggregates,supports_group_by) and adjusted native FDWs + docs accordingly. - Added a regression test to validate cached-plan re-execution does not crash and that planning/execution hooks run the expected number of times.
File summaries
| File | Description |
|---|---|
| wrappers/src/supabase_wrappers_tests.rs | Adds a regression test covering prepared/cached plan re-execution lifecycle behavior. |
| wrappers/src/fdw/mysql_fdw/mysql_fdw.rs | Updates aggregate-pushdown hooks to match new static trait method signatures. |
| wrappers/src/fdw/mssql_fdw/mssql_fdw.rs | Updates aggregate-pushdown hooks to match new static trait method signatures. |
| wrappers/src/fdw/clickhouse_fdw/clickhouse_fdw.rs | Updates aggregate-pushdown hooks to match new static trait method signatures. |
| wrappers/src/fdw/bigquery_fdw/bigquery_fdw.rs | Removes instance-based get_rel_size impl and updates aggregate-pushdown hooks to static signatures. |
| supabase-wrappers/src/utils.rs | Removes the old SerdeList helper now that scan state is serialized differently. |
| supabase-wrappers/src/upper.rs | Uses static planning-time hooks for aggregate support checks (no FDW instance required). |
| supabase-wrappers/src/scan.rs | Core fix: makes fdw_private deep-copyable, rebuilds scan state per execution, and avoids planning-time instance creation. |
| supabase-wrappers/src/qual.rs | Tracks original Const nodes for quals so their values can survive plan caching/copying correctly. |
| supabase-wrappers/src/interface.rs | Extends Qual with value_const and updates trait docs/signatures for planning-time hooks. |
| docs/guides/query-pushdown.md | Updates documentation examples for the new static aggregate hook signatures. |
| CLAUDE.md | Updates internal repo guide snippets to reflect the new trait hook signatures and semantics. |
Review details
- Files reviewed: 12/12 changed files
- Comments generated: 6
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This reverts commit 422b86d. Copilot suggested malformed fix for little gain.
5502ae0 to
4467d30
Compare
|
|
||
| // Use a prepared statement to force plan caching | ||
| c.update( | ||
| "prepare cache_test_q as select count(*) from cache_test_table", |
There was a problem hiding this comment.
May be we should a test with parameters ex: prepare q(bigint) as select * from t where id = $1 . Execute the same query 6 times so that plan caching kicks in or have the GUC plan_cache_mode set to force_generic_plan .
There was a problem hiding this comment.
We ensure that cached plan is being used by checking that the value of PLANNED_CALLS is 1 and then NEW_CALLS is 3, guaranteeing that the query was planned once and run thrice. Did you have concerns with the query plan not being cached?
| // `copyObject` call correctly. It's a usize instead of a *mut pg_sys::Const | ||
| // to keep `Qual` `Send`, which is important for FDWs like ClickHouse that | ||
| // move quals across tokio task boundaries. | ||
| pub(crate) value_const: Option<usize>, |
There was a problem hiding this comment.
As we are changing the public facing function signature so I believe we are going to bump the version too once this change is merged right?
There was a problem hiding this comment.
Yes we will bump the version. I usually do that in a separate PR just before releasae.
This PR fixes a double free/use after free bug in the core wrappers framework logic. The bug occurred due to an assumption about how Postgres runs the plan and scan phases of a query. Postgres can cache a query once and then execute it repeatedly without ever planning it again. The code in wrappers assumed that Postgres will never cache a plan. This assumption lead to wrappers creating the
FdwStateobject at the beginning of the plan phase, and freeing it unconditionally at the end of the first scan phase. If the plan was cached, the new scan phase will then use a freed object and free it again at the end of that phase, and so on.Since Postgres can cache a plan, it makes a deep copy of the plan via a call to
copyObjectbefore starting the scan phase. To fix the bug we now ensure that any object created during the plan phase are dropped at the end of that phase and any state we need to carry over from the plan phase to the scan phase we serialize into aListvia thefdw_privatemember, and reconstitute anFdwStatestruct from this deserialized state. This struct is then dropped at the end of the scan phase. This decoupling of lifetimes of objects in plan vs scan phases is what fixes the bug.A large portion of the code is this new serialization/deserialization logic, but there were also some other side effects of fixing this bug. Specifically with the above fix we initially started creating two instances of
FdwStatestruct (and the Fdw instance) which would break the assumption of FDWs that their instance would only be created ever once, leading to some potential bugs. Less severely, it could also affect performance if and FDW instance performed significant work in theirnew()method, like establishing connections etc. To fix this we also ensure that we now only create an FDW instance during the scan phase. This does mean we had to remove a&selfargument from certain plan phase functions, but the good news is that this change did not break any of the FDWs.Also fixes #237 because that issue had the same underlying cause.