Skip to content

Handle deleted rollover index exclusions - #1319

Open
Pybsama wants to merge 1 commit into
block:mainfrom
Pybsama:codex/allow-missing-search-indices
Open

Handle deleted rollover index exclusions#1319
Pybsama wants to merge 1 commit into
block:mainfrom
Pybsama:codex/allow-missing-search-indices

Conversation

@Pybsama

@Pybsama Pybsama commented Jul 31, 2026

Copy link
Copy Markdown

Why

ElasticGraph caches the concrete indices related to a rollover template. If
those indices are then deleted manually, a later search can still target a
cached concrete name or include it as an exclusion in its _msearch index
expression. OpenSearch rejects either request with
index_not_found_exception.

What

  • Always set ignore_unavailable: true on _msearch metadata headers. Unlike
    allow_no_indices, this handles both a missing concrete target and a deleted
    concrete name in an exclusion expression.
  • Distinguish a zero-shard response caused by cached rollover indices being
    deleted from a genuinely unconfigured index. The router now requires every
    narrowed target to be a rollover definition with known cached indices before
    treating a zero-shard response as deleted-index state.
  • Add live-datastore regressions for partial deletion, an empty time set with an
    aggregation, and deleting every cached rollover index. Unit coverage retains
    the configuration error for genuinely unconfigured rollover, non-rollover,
    narrowed, and mixed-index searches.

Validation

  • Current PR-base patch on OpenSearch 3.6.0: router specs 26 examples and the
    focused Apollo/GraphQL live-datastore suite 53 examples, all passing.
  • The identical patch rebased onto current main (matching stable patch ID):
    full repository suite 5,341 examples, 0 failures, with 100% line and branch
    coverage; focused live-datastore suite 55 examples, 0 failures.
  • GraphQL gem suite: 1,695 examples, 0 failures, 100% line and branch coverage.
  • Apollo gem suite: 123 examples, 0 failures, 100% line and branch coverage.
  • script/lint, script/type_check, script/spellcheck, schema artifact check,
    and git diff --check pass.
  • Site generation and internal-link checks pass. The final external-link check
    is currently blocked by redirects/timeouts and changed anchors on existing
    Apollo documentation URLs; no documentation links are changed by this PR.

Risk assessment

The change sends an officially supported _msearch metadata option on every
search. The router still raises its existing configuration error unless all
narrowed targets have cached rollover state, so a cached rollover definition
cannot hide a genuinely unconfigured non-rollover or second rollover target.
Index-expression construction, request bodies, query grouping, and public APIs
are unchanged.

References

Closes #589.

@CLAassistant

CLAassistant commented Jul 31, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@myronmarston myronmarston self-assigned this Jul 31, 2026
myronmarston

This comment was marked as resolved.

Always ignore unavailable indices in msearch headers, while preserving the configuration error unless every narrowed target is a previously known rollover index. Add real datastore and router coverage for partial, complete, empty-time, mixed-index, and genuinely unconfigured cases.
@Pybsama
Pybsama force-pushed the codex/allow-missing-search-indices branch from bbdb940 to 247eef3 Compare August 28, 2026 17:50
@Pybsama

Pybsama commented Aug 28, 2026

Copy link
Copy Markdown
Author

Implemented the requested changes and resolved all four inline threads on the current single-commit head 247eef3fc1e28335f148fb47cc89beaea7c1fdc6.

  • Every _msearch header now sends ignore_unavailable: true.
  • Zero-shard suppression now requires every narrowed target to have cached rollover state; a mixed cached-rollover/unconfigured-non-rollover regression keeps the boundary conservative.
  • Added the requested empty-time aggregation and all-cached-indices-deleted live-datastore cases, retained genuine unconfigured rollover/non-rollover coverage, and removed :no_vcr.

Local verification on OpenSearch 3.6.0 passes the focused live-datastore suite and router specs. The identical patch on current main passes 5,341 repository examples with 100% line and branch coverage, plus lint, type, spelling, and schema checks. Site generation and internal links pass; only unchanged Apollo documentation external URLs fail the final external-link probe because of redirects/timeouts and changed anchors.

Marking this ready for re-review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use allow_no_indices: true to avoid failures after manually deleted indices

3 participants