Fix forwarding ARN sequence lengths from QairtPipelinePass to QairtEncapsulationPass#2526
Open
qti-kromero wants to merge 4 commits into
Open
Conversation
…tion QairtEncapsulation uses sequence_lengths from model_attributes to compute max_length in genai_config.json. QairtGenAIBuilder populates this via its own config, but QairtPipelinePass had no equivalent path. Extract the arn list from the genai_builder.transform_options in the YAML recipe and pass it through model_attributes so QairtEncapsulation can pick it up.
…-pipeline-context-lengths
Covers the case where arn is present in genai_builder.transform_options and forwarded as sequence_lengths in model_attributes, and the case where arn is absent and sequence_lengths is not set.
qti-kromero
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
This PR ensures QAIRT pipeline recipes that specify AR decode lengths (stages.genai_builder.transform_options.arn) correctly forward that information through QairtPipelinePass via model_attributes["sequence_lengths"], enabling QairtEncapsulation to compute genai_config.json max_length consistently with the QairtGenAIBuilder path.
Changes:
- Forward
arnfrom the pipeline recipe intoQairtModelHandler.model_attributes["sequence_lengths"]inQairtPipelinePass. - Add unit tests verifying
arnis forwarded when present and omitted when absent.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
olive/passes/qairt/pipeline.py |
Extracts arn from recipe and forwards it as sequence_lengths via model_attributes for downstream encapsulation. |
test/passes/qairt/test_pipeline_pass.py |
Adds coverage for forwarding behavior and the no-arn behavior. |
…psulation QairtEncapsulation calls min(sequence_lengths) on the forwarded value, so a malformed YAML arn (string, bare int, empty list, mixed types) would fail with an opaque error. Raise a clear ValueError at the pipeline pass boundary instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
QairtEncapsulation uses sequence_lengths from model_attributes to compute max_length in genai_config.json. QairtGenAIBuilder populates this via its own config, but QairtPipelinePass had no equivalent path. Extract the arn list from the genai_builder.transform_options in the YAML recipe and pass it through model_attributes so QairtEncapsulation can pick it up.
Previously caused an issue in execution where context length would be exceeded and some kind of crash would occur
Describe your changes
Checklist before requesting a review
lintrunner -a(Optional) Issue link