[feature](cloud) Add strict instance auth for meta service rpc by deardeng · Pull Request #69981 · apache/doris

deardeng · 2026-11-02T09:09:48Z

Related PR: #69732

What problem does this PR solve?

This PR improves instance-level validation for Meta Service RPCs and enables it by default.

It updates FE request handling and combines first-FE registration into a single RPC. Existing cluster_id deployments remain supported. Temporary compatibility is retained for virtual compute group updates during FE upgrades.

Release note

enable_strict_instance_auth now defaults to true.

FE self-registration in cluster_id mode requires no additional configuration. For mixed-version deployments, enable_strict_instance_auth = false remains available as a temporary compatibility setting.

Check List (For Author)

Test: FE/Cloud builds; unit tests covering strict validation, bootstrap, compatibility handling, and FE request construction.
Manual test: first-FE startup with cluster_id, FE restart, and SQL ADD BACKEND before and after restart.
Behavior changed: Yes. Strict instance validation is enabled by default.
Does this need documentation? Yes. Document the new default and upgrade considerations.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Related PR: apache#69732 Problem Summary: Rpcs naming an instance by `instance_id` alone, e.g. `alter_cluster`, `alter_instance`, `update_ak_sk`, `alter_ram_user`, `list_snapshot`, `compact_snapshot` and `get_cluster_status` by `instance_ids`, are accepted by meta service without any proof that the caller belongs to that instance. They can not simply be rejected, because FE sends `alter_cluster` and `alter_instance` with `instance_id` only, and `AlterInstanceRequest` has no `cloud_unique_id` field. Fix: 1. New meta service config `enable_strict_instance_auth` (default false, not mutable). When enabled, `MetaServiceProxy::call_impl` rejects with INVALID_ARGUMENT an rpc carrying `instance_id` without `cloud_unique_id`, and `get_cluster_status` with `instance_ids`. `alter_instance` REFRESH, which meta service sends to other meta services by instance_id, is still accepted. The http api authorized by token calls the impl directly and is not affected. 2. `AlterInstanceRequest` gets `optional string cloud_unique_id = 6`. 3. FE `MetaServiceClient` fills its `cloud_unique_id` into `alter_cluster` and `alter_instance` requests that do not carry one, as it already does for other rpcs. The cloud_unique_id derived from cluster_id (`1:<id>:fe`) is not filled, since anyone can build it and it is not registered before the first FE adds itself. This only requires `cloud_unique_id` to be present. Whether `instance_id` belongs to the `cloud_unique_id` is checked by apache#69732, and the two are meant to be used together. Rollout: upgrade meta service, then all FE, then enable the config. It is not meant for cluster_id mode, where FE creates and joins its instance by instance_id. Add meta service config `enable_strict_instance_auth` to reject rpcs that name an instance by instance_id without a cloud_unique_id. FE now sends its cloud_unique_id in alter_cluster and alter_instance requests. - Test: Unit Test - meta_service_test StrictInstanceAuthTest covers the config off and on - FE MetaServiceClientTest covers filling, keeping a given and skipping a derived cloud_unique_id - Behavior changed: Yes. FE sends cloud_unique_id in alter_cluster and alter_instance; with the new config enabled, rpcs naming an instance by instance_id only are rejected. - Does this need documentation: Yes, the new meta service config

hello-stephen · 2026-11-02T09:09:54Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

deardeng · 2026-11-02T09:09:57Z

/review

github-actions · 2026-11-02T10:00:36Z

Codex automated review failed and did not complete.

Error: Selected model is at capacity. Please try a different model.
Workflow run: https://github.com/apache/doris/actions/runs/41418429124

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

deardeng · 2026-11-02T11:10:52Z

/review

github-actions · 2026-11-02T12:06:18Z

Codex automated review failed and did not complete.

Error: Selected model is at capacity. Please try a different model.
Workflow run: https://github.com/apache/doris/actions/runs/41434724388

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

### What problem does this PR solve? Issue Number: None Related PR: apache#69981 Problem Summary: Strict instance authentication rejects the legacy RPCs used to create an instance and register its first FE. Replace those calls with a trusted bootstrap_instance RPC that creates the instance and its initial FE cluster in one transaction. Bootstrap leaves existing instances unchanged and uses the existing cluster_id configuration. FE carries its cloud_unique_id on subsequent management RPCs. Temporarily allow legacy ALTER_VCLUSTER_INFO requests without that identity so virtual compute group updates remain compatible during FE upgrades. Scope legacy test settings to the affected fixtures. ### Release note Meta Service enables strict instance authentication by default. First-FE registration uses bootstrap_instance with the existing cluster_id setting; no additional FE or MS bootstrap configuration is needed. Legacy virtual compute group updates remain accepted without cloud_unique_id until all FEs are upgraded. Other legacy instance-ID-only RPCs still require disabling strict mode during the mixed-version upgrade window. ### Check List (For Author) - Test: Cloud unit-test build and six focused Meta Service tests, including strict auth and the legacy virtual compute group exception. Previously verified FE/Cloud builds and cluster_id-only startup/restart with SQL ADD BACKEND before and after restart. - Behavior changed: Yes. Strict instance authentication defaults to enabled and the first FE uses the trusted bootstrap RPC. ALTER_VCLUSTER_INFO has a temporary compatibility exception for legacy FE clients. - Does this need documentation: Yes. The release note above describes the new default and the mixed-version upgrade setting.

deardeng · 2026-11-13T02:34:16Z

run buildall

### What problem does this PR solve? Issue Number: None Related PR: apache#69981 Problem Summary: Strict instance authentication rejects the legacy RPCs used to create an instance and register its first FE. Replace those calls with a trusted bootstrap_instance RPC that creates the instance and its initial FE cluster in one transaction. Bootstrap leaves existing instances unchanged and uses the existing cluster_id configuration. FE carries its cloud_unique_id on subsequent management RPCs. Temporarily allow legacy ALTER_VCLUSTER_INFO requests without that identity so virtual compute group updates remain compatible during FE upgrades. Scope legacy test settings to the affected fixtures. ### Release note Meta Service enables strict instance authentication by default. First-FE registration uses bootstrap_instance with the existing cluster_id setting; no additional FE or MS bootstrap configuration is needed. Legacy virtual compute group updates remain accepted without cloud_unique_id until all FEs are upgraded. Other legacy instance-ID-only RPCs still require disabling strict mode during the mixed-version upgrade window. ### Check List (For Author) - Test: Cloud unit-test build and six focused Meta Service tests, including strict auth and the legacy virtual compute group exception. Previously verified FE/Cloud builds and cluster_id-only startup/restart with SQL ADD BACKEND before and after restart. - Behavior changed: Yes. Strict instance authentication defaults to enabled and the first FE uses the trusted bootstrap RPC. ALTER_VCLUSTER_INFO has a temporary compatibility exception for legacy FE clients. - Does this need documentation: Yes. The release note above describes the new default and the mixed-version upgrade setting.

deardeng · 2026-11-13T03:08:19Z

run buildall

### What problem does this PR solve? Issue Number: None Related PR: apache#69981 Problem Summary: Strict instance authentication rejects the legacy RPCs used to create an instance and register its first FE. Replace those calls with a trusted bootstrap_instance RPC that creates the instance and its initial FE cluster in one transaction. For a precreated instance without an FE cluster, bootstrap preserves its metadata and registers the initial FE. Existing FE clusters are left unchanged; conflicting cluster IDs or names are rejected before registration. Bootstrap uses the existing cluster_id configuration and accepts configured same-instance ID suffixes. FE carries its cloud_unique_id on subsequent management RPCs. Temporarily allow legacy ALTER_VCLUSTER_INFO requests without that identity so virtual compute group updates remain compatible during FE upgrades. Scope legacy test settings to the affected fixtures. ### Release note Meta Service enables strict instance authentication by default. First-FE registration uses bootstrap_instance with the existing cluster_id setting; no additional FE or MS bootstrap configuration is needed. Legacy virtual compute group updates remain accepted without cloud_unique_id until all FEs are upgraded. Other legacy instance-ID-only RPCs still require disabling strict mode during the mixed-version upgrade window. ### Check List (For Author) - Test: Cloud build and seven focused Meta Service tests, including strict auth, the legacy virtual compute group exception, precreated-instance bootstrap with metadata preservation, custom identity suffixes, and cluster ID/name conflicts without metadata changes. Previously verified FE/Cloud builds and cluster_id-only startup/restart with SQL ADD BACKEND before and after restart. - Behavior changed: Yes. Strict instance authentication defaults to enabled and the first FE uses the trusted bootstrap RPC. ALTER_VCLUSTER_INFO has a temporary compatibility exception for legacy FE clients. - Does this need documentation: Yes. The release note above describes the new default and the mixed-version upgrade setting.

deardeng · 2026-11-13T07:15:25Z

Local pipeline review — ✅ PASS

schema: doris-repo-review/v1
status: PASS
pr: apache/doris#69981
commit: 425992aee2f6e17567fa06240966e938ec2a88b5
base: b325dcd122b5f50685881c1399f1eb904ea2efa3
reviewed_at: 2026-11-13T15:15+08:00
reviewer: deardeng
model: gpt-6-astra
effort: xhigh
findings: {blocker: 0, major: 0, minor: 0, nit: 0}
rounds: 1
converged: true

Notes for maintainers

Full Docker end-to-end regression was not run. Implementation validation included the Cloud build, seven focused Meta Service tests, formatting checks, and static analysis.

_{Reviewed locally with the doris-repo-review pipeline. Repository policy may accept this receipt for the matching commit; it is not a human Apache approval.}

deardeng · 2026-11-13T07:16:48Z

run buildall

deardeng requested review from gavinchou, liaoxin01 and luwei16 as code owners November 2, 2026 09:09

deardeng added the dev/4.2.x label Nov 2, 2026

gavinchou reviewed Nov 10, 2026

View reviewed changes

Comment thread cloud/src/meta-service/meta_service.h Resolved

gavinchou previously approved these changes Nov 11, 2026

View reviewed changes

deardeng dismissed gavinchou’s stale review via 44ba948 November 13, 2026 02:33

deardeng force-pushed the fix-brpc-instance-id branch from 44ba948 to 2600827 Compare November 13, 2026 02:52

deardeng force-pushed the fix-brpc-instance-id branch from 2600827 to 425992a Compare November 13, 2026 07:12

apache deleted a comment from ShadowStackDev Nov 13, 2026

Navigation Menu

Repository navigation

[feature](cloud) Add strict instance auth for meta service rpc - #69981

[feature](cloud) Add strict instance auth for meta service rpc#69981
deardeng wants to merge 2 commits into
apache:masterfrom
deardeng:fix-brpc-instance-id

deardeng commented Nov 2, 2026 •

edited

Loading

Uh oh!

hello-stephen commented Nov 2, 2026

Uh oh!

deardeng commented Nov 2, 2026

Uh oh!

github-actions Bot commented Nov 2, 2026

Uh oh!

deardeng commented Nov 2, 2026

Uh oh!

github-actions Bot commented Nov 2, 2026

Uh oh!

Uh oh!

deardeng commented Nov 13, 2026

Uh oh!

deardeng commented Nov 13, 2026

Uh oh!

deardeng commented Nov 13, 2026

Uh oh!

deardeng commented Nov 13, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

deardeng commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Nov 2, 2026

Uh oh!

deardeng commented Nov 2, 2026

Uh oh!

github-actions Bot commented Nov 2, 2026

Uh oh!

deardeng commented Nov 2, 2026

Uh oh!

github-actions Bot commented Nov 2, 2026

Uh oh!

Uh oh!

deardeng commented Nov 13, 2026

Uh oh!

deardeng commented Nov 13, 2026

Uh oh!

deardeng commented Nov 13, 2026

Local pipeline review — ✅ PASS

Uh oh!

deardeng commented Nov 13, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

deardeng commented Nov 2, 2026 •

edited

Loading