feat(mpl): Smithy model, orchestrator, client-java, and test suite for MPL TestServer - #13
Open
josecorella wants to merge 50 commits into
Open
josecorella wants to merge 50 commits into
josecorella wants to merge 50 commits into
Conversation
…es TestServer Defines the rpcv2Cbor wire contract for the Primitives TestServer: AesEncrypt, AesDecrypt, GenerateRandomBytes, Digest, Hmac, Hkdf, KbkdfCtrHmac, EcdsaGenerateKeyPair, EcdsaSign, EcdsaVerify. Configuration_Set has Rust as the first language entry (port 8094), pointing at the aws-crypto-tools-rust prim-test-server branch.
Defines the rpcv2Cbor wire contract for the MPL TestServer: CreateRawAesKeyring, CreateDefaultCmm, GetEncryptionMaterials, DecryptMaterials. Uses a multi-type handle registry (keyringId/cmmId). Configuration_Set has Rust as the first language entry (port 8093), pointing at the aws-crypto-tools-rust mpl-test-server branch. Builds on the primitives model commit.
Complete test infrastructure for the Primitives TestServer: - client-java/: smithy-java codegen-plugin generates a typed Java Test_Client from the Smithy model (rpcv2Cbor, client mode) - tests/: JUnit 5 test suite with AesRoundTripTest and EcdsaRoundTripTest driving the pairwise endpoint matrix via runtime configuration - orchestrator/: pipeline runner that loads configuration-set.json, builds and launches Language_Servers (RustLaunchPlan), runs the tests, and reports results - Makefile: developer convenience targets (validate, build, orchestrate) The orchestrator supports Language_Repository_Run mode (context=language:rust languageRepoRoot=<path>) for CI in the Rust repo.
Complete test infrastructure for the MPL TestServer: - client-java/: smithy-java codegen-plugin generates a typed Java Test_Client from the MPL Smithy model (rpcv2Cbor, client mode) - tests/: JUnit 5 suite with RawAesRoundTripTest — creates keyring + CMM on server A, gets encryption materials, then decrypts on server B using the EDKs from A (pairwise endpoint matrix) - orchestrator/: pipeline runner with RustLaunchPlan, configuration loading, and GradleTestRunner - Makefile: developer convenience targets Builds on the primitives orchestrator commit.
josecorella
changed the title
Runs on PRs touching primitives/test-server/**. Steps: JDK 21, Go, Rust toolchain, build client-java codegen, build tests, build orchestrator. Full orchestrate step commented out until Rust server branch merges.
…l-test-server-model
Runs on PRs touching mpl/test-server/**. Steps: JDK 21, Go, Rust toolchain, build client-java codegen, build tests, build orchestrator. Full orchestrate step commented out until Rust MPL server branch merges and rustc 1.94+ is satisfied.
smithy-java 1.4.0 generates JavaBean-style accessors (getCiphertext(), getAuthTag(), getVerificationKey(), isValid(), etc.) not fluent-style. Fix all test files to match the generated client API.
smithy-java 1.4.0 generates JavaBean-style accessors (getCiphertext(), getAuthTag(), getVerificationKey(), isValid(), etc.) not fluent-style. Fix all test files to match the generated client API.
Process.waitFor takes (long timeout, TimeUnit unit), not the reverse.
Process.waitFor takes (long timeout, TimeUnit unit), not the reverse.
…l-test-server-model
…chestrate in CI The orchestrator now clones the Rust repo at the branch configured in configuration-set.json (aws-crypto-rust/primitives-test-server) when running in Commons_Run mode (context=commons). Uses HTTPS URLs so CI can clone without SSH keys. CI workflow now runs the full orchestrate step: clone → cargo build → launch server → run Java tests.
…l-test-server-model
… CI orchestrate Same pattern as primitives: orchestrator clones the Rust repo at the configured branch (aws-crypto-rust/mpl-test-server) using HTTPS URLs. CI workflow now runs the full orchestrate step.
The aws-crypto-tools-rust repo is private. The orchestrator's git clone needs a PAT (same pattern the ESDK test-server uses). Falls back with a warning if the secret isn't configured.
…l-test-server-model # Conflicts: # .github/workflows/mpl-test-server.yml
This workflow belongs only on the jocorell/mpl-test-server-model branch.
…l-test-server-model
Runs on PRs touching mpl/test-server/**. Authenticates via COMMONS_REPO_PAT, clones the Rust MPL server, builds, launches, and runs the Java test suite.
The existing ESDK workflow uses LANGUAGE_REPO_PAT to clone private repos. Use the same secret name so the already-configured PAT works. Also rewrite both SSH and HTTPS URL patterns.
…l-test-server-model
…ge run The JVM subprocess spawned by Gradle's application plugin does not see the ~/.gitconfig insteadOf rule set in an earlier step. Work around by cloning the Rust repo directly in the shell step (where the PAT is available) and passing the clone path to the orchestrator via context=language:rust languageRepoRoot=<clone>.
…l-test-server-model
System.exit() started JVM shutdown from inside the try block, so the finally block that calls LaunchedServer.stop() never ran and every launched Language_Server subprocess was leaked to the runner's orphan cleanup. Capture the exit code and exit after the finally instead.
Commit the 8.14.1 Gradle wrappers for client-java, tests, and
orchestrator, matching the ESDK TestServer modules, so `make orchestrate`
works locally and CI no longer generates a wrapper on the fly from the
runner's preinstalled Gradle. Add the per-module .gitignore files the
ESDK modules carry so build output stays untracked.
Workflow changes:
* Read the Rust Language_Server url/ref from configuration-set.json
instead of hardcoding the branch, so the workflow and the
Configuration_Set cannot drift, and key the caches on the resolved
tip commit (the old key hashed `**/Cargo.lock`, of which this repo
has none, so it was a constant).
* Cache the cargo downloads plus an sccache compiler cache rather than
target/: the orchestrated run clones the Rust repo fresh each run, so
a restored target/ yields an incorrect incremental build.
* Split cache restore/save with `if: always()` — the combined action's
post step only saves on job success, and the orchestrate step exits
non-zero on a failed Test, so nothing was ever saved.
* Fold the redundant insteadOf auth step into the clone and fail loudly
on a missing LANGUAGE_REPO_PAT instead of warning; scrub the token
from the clone's remote afterwards.
* Upload the server log and JUnit reports for diagnostics.
* Drop the stale header comment claiming the orchestrated run is not
yet enabled.
The ls-remote resolved the Rust branch tip into a step output that no cache key consumed — the cargo/sccache cache keys on the run id and restores by prefix — so it bought nothing while adding a private-repo auth dependency the step had no credential for, which emitted a spurious 'could not resolve' warning every run. Keep the step for the url/ref lookup that removes the hardcoded branch.
hanabanaka
added a commit
that referenced
this pull request
Sep 2, 2026
…_ENCRYPTION_CONTEXT - Add SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT to the test-server CryptoAction enum (matches the DBE library's v2 config). - Baseline schema uses v2, populating the header Encryption Context with aws-crypto-attr.<name> and aws-crypto-legend entries. - Add tests: #7 (legend-action swap - isolates recipient-tag verification from parser rejection), #11/#12/#13 (EDK Key Provider ID / Provider Info / ciphertext, targeting the three distinct rejection paths). - Refactor #9 to fail loudly on empty EC instead of silently reflipping the count byte; add hasContextAttribute precondition to buildContext. - Refactor testContexts() -> testPairs() so per-pair setup failures surface as per-test failures with proper JUnit XML, instead of aborting the whole class's XML output.
…mmon Extract the product-neutral orchestrator, tests-support, and tools into a top-level test-server-common/ directory shared by the ESDK and DB-ESDK test servers, leaving the DB-ESDK-specific tests, config, model, and client under dbesdk/test-server/. Rewire the tests includeBuild, the Makefile orchestrator path, and the CI workflow paths and caches to the new location. Point the server-config.json refs and the workflow at commons main now that the servers live there, rewrite the CI workflow comments to read plainly, and add .jqwik-database to a new root .gitignore.
… into jocorell/primitives-test-server-model
…e-file configuration Consume the shared test-server-common/ orchestrator and tests-support instead of a per-product orchestrator copy, and replace configuration-set.json with the design's three-file configuration (server-config.json + feature-set.json + bug-list.json). Tests move to the shared registry, feature gate, and client cache; the Makefile and CI workflow move to the single 'make orchestrate' entry point with the commons_ref/rust_ref workflow_call contract.
…l-test-server-model
…configuration Consume the shared test-server-common/ orchestrator and tests-support instead of a per-product orchestrator copy, and replace configuration-set.json with the design's three-file configuration (server-config.json + feature-set.json + bug-list.json). Tests move to the shared registry, feature gate, and client cache; the Makefile and CI workflow move to the single 'make orchestrate' entry point with the commons_ref/rust_ref workflow_call contract.
… into jocorell/primitives-test-server-model
…l-test-server-model
Add Configuration_Entries for the java, net, python, go, and rust-dafny servers in aws-cryptographic-material-providers-library, an MPL-repo working-tree overlay, and the default java reference implementation.
… the launch plans A library component that is a smithy-dafny runtimes/<lang> directory is now transpiled with its project's Makefile before the server builds: the Java plan runs make build_java mvn_local_deploy, the .NET plan derives its transpile project from the library path instead of AwsEncryptionSDK, and the Python plan runs make transpile_python and installs the library without the pinned MPL. Dafny builds in one repository are serialized, since servers launch concurrently and each transpile edits shared sources. The Python server module is <product>_test_server, and the Java plan also resolves a JDK 17-class home from setup-java's JAVA_HOME_<major>_X64.
Add JDK 17, Python 3.11, .NET, and Dafny for the servers in aws-cryptographic-material-providers-library, and an mpl_ref workflow_call input.
…l-test-server-model
…l-test-server-model
…Server Add InitializeEncryptionMaterials, InitializeDecryptionMaterials, OnEncrypt and OnDecrypt to the model so a test can drive a keyring directly and carry materials and EDKs between two servers. Handles become a ResourceId (@Length(min: 1)). Narrow the feature catalog to raw-aes and default-cmm, the surface the native Rust MPL implements and the Tests cover. ESDK suites only. Tests: RawAesKeyringTest (round trip per wrapping algorithm with a byte-level EDK check, wrapping an existing data key, wrong key and non-matching name rejected), meta/HandleAndErrorContractTest (unknown and wrong-kind handles are GenericServerError, an MPL rejection is MPLClientError), and a pinned default suite in RawAesRoundTripTest.
Configure the Java server from aws-cryptographic-material-providers- library (test-server/mpl/java-v1-server/, branch jocorell/mpl-test-server-java) alongside the Rust server. It runs against the published MPL, so libraryRepository points at the server directory and the orchestrator skips the Dafny transpile. With java configured, drop the referenceImplementation=rust override and add an MPL_JAVA_REPO working-tree overlay for local runs.
Configure the four new servers in aws-cryptographic-material-providers- library (test-server/mpl/<lang>-v<major>-server/) alongside Java and native Rust. net and python run against the published MPL, so their libraryRepository path is the server directory; go and rust-dafny build the MPL's committed releases/ code.
With the referenceImplementation=rust override gone, a CI run (no working-tree overlays) left ORCH_ARGS empty and passed --args="", which Gradle rejects. Guard it the way the primitives Makefile does.
josecorella
changed the base branch from
jocorell/primitives-test-server-model
to
main
September 28, 2026 22:00
# Conflicts: # .github/workflows/primitives-test-server.yml # primitives/test-server/config/server-config.json
josecorella
commented
Sep 30, 2026
…le workflow, build from source Same shape as the Primitives TestServer (#33). The run moves to mpl-test-server-reusable.yml; every server is prebuilt once per commit on its own runner and shared as an artifact; a focus input tests one language against every server. The java, net, and python servers build on the MPL's own source: their library paths point at AwsCryptographicMaterialProviders/runtimes/, and each prebuild transpiles the MPL and its dependencies from Dafny 4.9.0 (aws-cryptographic-material-providers-library 829e15ab5 switched the servers off the published packages). Carries #33's JavaLaunchPlan Dafny-library stamp (identical change) and the ORCHESTRATE_ARGS Makefile passthrough.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds the full MPL TestServer infrastructure (stacked on #12):
make orchestrate LANGUAGE_REPO_ROOT=<path>Design: multi-type handle registry
Unlike the ESDK server (single clientId → keyring), the MPL server stores:
keyringId → Arc<dyn Keyring>cmmId → Arc<dyn Cmm>Operations chain: CreateRawAesKeyring → keyringId, CreateDefaultCmm(keyringId) → cmmId, GetEncryptionMaterials/DecryptMaterials(cmmId).
Cross-repo wiring
Consumes the Rust Language_Server at
aws-crypto-tools-rustbranchaws-crypto-rust/mpl-test-server(port 8093).Depends on
To make it fully runnable