fix(runners): pin dedicated host family so mac-m4 (and mac1/c7i) hosts allocate correctly - #1120
Closed
coderbirju wants to merge 1 commit into
Closed
coderbirju wants to merge 1 commit into
coderbirju wants to merge 1 commit into
Conversation
…s allocate correctly arm macOS runners moved to mac-m4.metal in #1117, but deploys failed instantly with "The requested configuration is currently not supported. ... Launching EC2 instance failed." Root cause: the dedicated Host Resource Group in createHostResourceGroup used AWS::EC2::HostManagement with auto-allocate-host=true / auto-release-host=false but did NOT pin an instance family. With no family pin the group auto-allocates / retains a stale family (the old mac2 hosts predating #1117), so EC2 refuses to place a mac-m4.metal instance. Fix: pin the family via the 'allowed-host-families' AWS::EC2::HostManagement parameter, derived per platform/arch so it always matches the launch template instanceType: mac+arm -> mac-m4, mac+x86 -> mac1, windows -> c7i. auto-allocate-host=true, auto-release-host=false, and the existing Generic config are unchanged. instanceType logic (#1117 mac-m4.metal) and the AMI lookup are untouched. Parameter verified against AWS docs (do NOT guess): the hypothesized 'instance-family' is NOT a valid HostManagement parameter. The correct, documented parameter is 'allowed-host-families' (array of family identifiers, e.g. c5/m5), per the AWS Resource Groups User Guide, 'Supported configuration types and parameters' -> AWS::EC2::HostManagement: https://docs.aws.amazon.com/ARG/latest/userguide/about-slg-types.html (also shown in the AWS::ResourceGroups::Group CloudFormation reference). Verified locally: npm ci; npm run build (tsc) pass; npm test 18/18 pass, incl. a new assertion test that the synthesized AWS::ResourceGroups::Group HostManagement config contains allowed-host-families = [mac-m4] (mac/arm), [mac1] (mac/x86), [c7i] (windows). Rendered-template dump confirmed the same values across every dedicated-host runner type. CLI 'cdk synth' of the pipeline stacks needs cross-account creds for the Vpc.fromLookup context provider, so the in-process synth (Template.fromStack) is the local proof. Runtime follow-up (operational, NOT in this code change): existing RETAINED mac2 dedicated hosts in the current host groups must be released so the groups can allocate mac-m4 hosts; auto-release-host=false retained them. Handled separately by the operator. Signed-off-by: Arjun Yogidas <arjunry@amazon.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
arm macOS runners were moved to
mac-m4.metalin #1117 (correct), but deploys fail instantly with:This is not capacity (fails instantly in
us-west-2bwhere mac-m4 is offered) and not the AMI (amzn-ec2-macos-15.7.9sequoia/arm64 is M4-compatible; there is no separate M4 AMI).Root cause
The dedicated Host Resource Group created in
lib/asg-runner-stack.tscreateHostResourceGroupusedAWS::EC2::HostManagementwithauto-allocate-host: true/auto-release-host: falsebut did not pin an instance family. With no family pin, the group auto-allocates/retains a stale family — the old mac2 hosts predating themac-m4.metalmove — so EC2 refuses to place amac-m4.metalinstance → "configuration not supported".Fix
Pin the family the group is allowed to allocate/place hosts for, via the
allowed-host-familiesAWS::EC2::HostManagementparameter, derived per platform/arch so it always matches the launch template'sinstanceType:mac-m4.metalmac-m4mac1.metalmac1c7i.2xlargec7iauto-allocate-host: true,auto-release-host: false, and the existingAWS::ResourceGroups::Genericconfig are unchanged. TheinstanceTypelogic (#1117), AMI lookup, #1113, and the #1119$Latestrevert are all untouched.Parameter verification (the key risk)
The obvious guess
instance-familyis NOT a validAWS::EC2::HostManagementparameter. The correct, documented parameter isallowed-host-families— an array of instance-family identifiers (e.g.c5,m5; heremac-m4, not the fullmac-m4.metal). Verified against:AWS::EC2::HostManagement: https://docs.aws.amazon.com/ARG/latest/userguide/about-slg-types.htmlAWS::ResourceGroups::GroupCloudFormation reference (host-resource-group example): https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-resource-resourcegroups-group.htmlBoth list
allowed-host-familiesas a supported HostManagement parameter and give a worked example (["c5","m5"]). This is deploy-valid, unlike theinstance-familyguess.Verification (real command output, not reasoning)
npm ci✅ ·npm run build(tsc) ✅npm test→ 18/18 pass, 12 suites ✅ — includes a new assertion test that the synthesizedAWS::ResourceGroups::GroupHostManagement config containsallowed-host-families = [mac-m4](mac/arm),[mac1](mac/x86),[c7i](windows), while keeping the existingauto-release-host: falseassertion intact.app.synth()) confirmed the same values across every dedicated-host runner type.cdk synthof the pipeline stacks requires cross-account credentials for theVpc.fromLookupcontext provider (tries to assume a lookup role in the target account), so the in-process synth viaTemplate.fromStackis the authoritative local proof.Operational follow-up (not in this PR)
Existing retained mac2 dedicated hosts in the current host groups must be released so the groups can allocate
mac-m4hosts (auto-release-host: falseretained them). This is an operational step handled separately from this code change.