Repository navigation
Allow changefeed creation during import/restore on filtered-out tables #6552
Description
Is your feature request related to a problem?
After #6525, changefeed creation is rejected whenever the upstream has an active Lightning physical import, BR restore, or IMPORT INTO task registration, even when the task only affects tables excluded by the changefeed's table filter.
This prevents users from starting replication for one set of tables while importing or restoring an independent set of tables on the same upstream.
Example scenario and current behavior
-
Prepare two independent tables on the upstream:
business.ordersandbusiness.history. -
Start a Lightning physical import, BR restore, or
IMPORT INTOtask targeting onlybusiness.history, and keep the task running with an active registration lease. -
While the task is running, create a changefeed that only replicates
business.orders:[filter] rules = ["business.orders"]
The same issue applies to an explicit exclusion:
[filter]
rules = ["business.*", "!business.history"]Current behavior: changefeed creation returns CDC:ErrUpstreamHasRunningImport, despite there being no overlap between the import/restore target tables and the changefeed's replication scope.
This behavior is identified by code inspection of the #6525 merge commit, be38e502a452333eeed117281938fca051631a95. The scenario above has not been validated against a running cluster.
Relevant code:
- CreateChangefeed calls hasRunningImport without passing the changefeed's filter.
- hasRunningImport scans the entire /tidb/brie/import prefix and rejects creation if any task has a positive lease TTL.
Describe the feature you'd like
Allow changefeed creation while upstream import/restore tasks are running when their target tables do not overlap with the changefeed's replication scope after applying its table filter.
Expected behavior:
- Non-overlapping tables: allow changefeed creation and normal replication of the selected tables.
- Overlapping tables: retain the rejection that protects against incomplete replication.
- Unknown or incomplete task scope: retain conservative rejection unless non-overlap can be established.
- Apply this behavior consistently to Lightning physical import, BR restore, and
IMPORT INTO. - Support both positive filter rules and explicit exclusions.
Please add coverage for overlapping and non-overlapping scopes. For BR restore, the scope check should also cover tables that the restore task will create, rather than relying only on tables currently visible when the changefeed is created.
Describe alternatives you've considered
Wait for the unrelated import/restore task to finish before creating the changefeed.
This unnecessarily couples independent workloads and can delay replication setup for the duration of a large import or restore.
Teachability, Documentation, Adoption, Migration Strategy
No response
Activity