Skip to content

Commit 7d16229

Browse files
kessplastexastony
kessplas
and
texastony
authored
feat!: Release Amazon S3 Encryption Client for Python (#189)
BREAKING CHANGE: Implements the S3 Encryption Client for Python. Co-authored-by: Tony Knapp <5892063+texastony@users.noreply.github.com>
1 parent abc858c commit 7d16229

312 files changed

Lines changed: 42090 additions & 9 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.duvet/.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
reports/
2+
requirements/
3+
specification/

‎.duvet/config.toml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
'$schema' = "https://awslabs.github.io/duvet/config/v0.4.0.json"
2+
3+
[[source]]
4+
pattern = "src/**/*.py"
5+
type = "implementation"
6+
comment-style = { meta = "##=", content = "##%" }
7+
[[source]]
8+
pattern = "test/**/*.py"
9+
type = "test"
10+
comment-style = { meta = "##=", content = "##%" }
11+
[[source]]
12+
pattern = "compliance_exceptions/**/*.md"
13+
type = "exception"
14+
comment-style = { meta = "##=", content = "##%" }
15+
16+
# Include required specifications here
17+
[[specification]]
18+
source = "specification/s3-encryption/materials/keyrings.md"
19+
[[specification]]
20+
source = "specification/s3-encryption/materials/s3-keyring.md"
21+
[[specification]]
22+
source = "specification/s3-encryption/materials/s3-kms-keyring.md"
23+
[[specification]]
24+
source = "specification/s3-encryption/client.md"
25+
[[specification]]
26+
source = "specification/s3-encryption/decryption.md"
27+
[[specification]]
28+
source = "specification/s3-encryption/encryption.md"
29+
[[specification]]
30+
source = "specification/s3-encryption/key-commitment.md"
31+
[[specification]]
32+
source = "specification/s3-encryption/key-derivation.md"
33+
[[specification]]
34+
source = "specification/s3-encryption/data-format/content-metadata.md"
35+
[[specification]]
36+
source = "specification/s3-encryption/data-format/metadata-strategy.md"
37+
38+
[report.html]
39+
enabled = true
40+
41+
# Enable snapshots to prevent requirement coverage regressions
42+
[report.snapshot]
43+
enabled = false

‎.github/workflows/all-ci.yml‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
name: All CI
2+
3+
on:
4+
push:
5+
branches: [ main, staging ]
6+
pull_request:
7+
workflow_dispatch:
8+
inputs:
9+
python-version:
10+
description: 'Python version to use'
11+
default: '3.11'
12+
required: false
13+
type: string
14+
15+
jobs:
16+
python-lint:
17+
name: Lint
18+
uses: ./.github/workflows/lint.yml
19+
20+
run-test-server:
21+
permissions:
22+
id-token: write
23+
contents: read
24+
name: Run TestServer Tests
25+
uses: ./.github/workflows/test-server.yml
26+
with:
27+
python-version: ${{ inputs.python-version || '3.11' }}
28+
secrets: inherit
29+
30+
python-integ:
31+
permissions:
32+
id-token: write
33+
contents: read
34+
name: Python Integration Tests
35+
uses: ./.github/workflows/python-integ.yml
36+
with:
37+
python-version: ${{ inputs.python-version || '3.11' }}
38+
secrets: inherit
39+
40+
python-perf:
41+
permissions:
42+
id-token: write
43+
contents: read
44+
name: Python Performance Tests
45+
uses: ./.github/workflows/python-perf.yml
46+
with:
47+
python-version: ${{ inputs.python-version || '3.11' }}
48+
secrets: inherit
49+
50+
run-duvet:
51+
permissions:
52+
id-token: write
53+
contents: read
54+
pages: write
55+
name: Run Duvet
56+
uses: ./.github/workflows/duvet.yml
57+
secrets: inherit
58+
59+
run-duvet-test-server:
60+
permissions:
61+
id-token: write
62+
contents: read
63+
pages: write
64+
name: Run Duvet
65+
uses: ./.github/workflows/duvet-test-server.yml
66+
secrets: inherit

‎.github/workflows/daily_ci.yml‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
name: Daily CI
2+
3+
on:
4+
schedule:
5+
# 5 AM PST = 1 PM UTC, Monday–Friday
6+
- cron: "0 13 * * 1-5"
7+
workflow_dispatch:
8+
inputs:
9+
python-version:
10+
description: 'Python version to use'
11+
default: '3.11'
12+
required: false
13+
type: string
14+
15+
jobs:
16+
run-test-server:
17+
permissions:
18+
id-token: write
19+
contents: read
20+
name: Run TestServer Tests
21+
uses: ./.github/workflows/test-server.yml
22+
with:
23+
python-version: ${{ inputs.python-version || '3.11' }}
24+
secrets: inherit
25+
26+
python-integ:
27+
permissions:
28+
id-token: write
29+
contents: read
30+
name: Python Integration Tests
31+
uses: ./.github/workflows/python-integ.yml
32+
with:
33+
python-version: ${{ inputs.python-version || '3.11' }}
34+
secrets: inherit
35+
36+
notify:
37+
needs:
38+
[
39+
run-test-server,
40+
python-integ
41+
]
42+
permissions:
43+
id-token: write
44+
contents: read
45+
if: ${{ failure() }}
46+
uses: aws/aws-cryptographic-material-providers-library/.github/workflows/slack-notification.yml@main
47+
with:
48+
message: "Daily CI failed on `${{ github.repository }}`. View run: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
49+
secrets:
50+
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL_CI }}

‎.github/workflows/duvet-test-server.yml‎

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
name: Generate Duvet Report for TestServer
2+
3+
on:
4+
workflow_call:
5+
# Optional inputs that can be provided when calling this workflow
6+
7+
jobs:
8+
duvet:
9+
runs-on: macos-latest
10+
permissions:
11+
id-token: write
12+
contents: read
13+
pages: write
14+
15+
steps:
16+
- name: Checkout code
17+
uses: actions/checkout@v6
18+
19+
# There are a lot of submodules here
20+
# This initializes the checkouts in parallel (--jobs)
21+
# rather than in series the way actions/checkout@v6 does it.
22+
23+
- name: Get CPU count
24+
id: cpu-count
25+
run: echo "count=$(node -p 'require("os").cpus().length')" >> $GITHUB_OUTPUT
26+
27+
- name: Setup git submodules with PAT
28+
run: |
29+
git config --global url."https://github.com/".insteadOf "git@github.com:"
30+
git config --global credential.helper store
31+
echo "https://x-token-auth:${{ secrets.PAT_FOR_SPEC }}@github.com" > ~/.git-credentials
32+
33+
- name: Optimize git for performance
34+
run: |
35+
git config --global fetch.parallel ${{ steps.cpu-count.outputs.count }}
36+
git config --global submodule.fetchJobs ${{ steps.cpu-count.outputs.count }}
37+
git config --global remote.origin.tagOpt --no-tags
38+
39+
- name: Checkout submodules with --jobs
40+
run: |
41+
git submodule update --init --depth 1 --single-branch --jobs ${{ steps.cpu-count.outputs.count }} test-server/
42+
43+
44+
- name: Checkout CPP code cpp-v3
45+
uses: actions/checkout@v6
46+
with:
47+
submodules: recursive
48+
repository: aws/aws-sdk-cpp
49+
ref: main
50+
path: test-server/cpp-v3-server/aws-sdk-cpp/
51+
52+
- name: Setup Rust toolchain
53+
uses: actions-rust-lang/setup-rust-toolchain@v1
54+
with:
55+
toolchain: stable
56+
57+
- name: Build and install duvet
58+
run: |
59+
cargo install duvet --locked
60+
61+
- name: Run duvet
62+
if: always()
63+
run: cd test-server && make duvet
64+
65+
- name: Upload duvet reports
66+
if: always()
67+
uses: actions/upload-artifact@v7
68+
with:
69+
name: test-server-reports
70+
include-hidden-files: true
71+
path: test-server/*-server/.duvet/reports/report.html
72+
73+
- name: Generate compliance dashboard
74+
if: always()
75+
run: |
76+
cd test-server/spec-compliance-dashboard
77+
python generate_compliance_dashboard.py
78+
79+
- name: Create dashboard redirect index.html
80+
if: always()
81+
run: |
82+
cat > test-server/index.html << 'EOF'
83+
<!DOCTYPE html>
84+
<html lang="en">
85+
<head>
86+
<meta charset="UTF-8">
87+
<meta http-equiv="refresh" content="0; url=spec-compliance-dashboard/compliance_homepage.html">
88+
<title>Redirecting to Compliance Dashboard...</title>
89+
</head>
90+
<body>
91+
<p>Redirecting to <a href="spec-compliance-dashboard/compliance_homepage.html">Compliance Dashboard</a>...</p>
92+
</body>
93+
</html>
94+
EOF
95+
96+
- name: Upload compliance dashboard
97+
if: always()
98+
uses: actions/upload-artifact@v7
99+
with:
100+
name: compliance-dashboard
101+
include-hidden-files: true
102+
path: |
103+
test-server/spec-compliance-dashboard/compliance_homepage.html
104+
test-server/*/compliance_summary_report.html
105+
test-server/*/.duvet/reports/report.html
106+
test-server/spec-compliance-dashboard/templates/*
107+
test-server/index.html
108+
109+
- name: Setup Pages
110+
if: always() && (github.ref == 'refs/heads/staging' || github.ref == 'refs/heads/fireegg-test-servers') && github.event_name == 'push'
111+
uses: actions/configure-pages@v5
112+
113+
- name: Upload Pages artifact
114+
if: always() && (github.ref == 'refs/heads/staging' || github.ref == 'refs/heads/fireegg-test-servers') && github.event_name == 'push'
115+
uses: actions/upload-pages-artifact@v3
116+
with:
117+
path: test-server/
118+
119+
- name: Deploy to GitHub Pages
120+
if: always() && (github.ref == 'refs/heads/staging' || github.ref == 'refs/heads/fireegg-test-servers') && github.event_name == 'push'
121+
uses: actions/deploy-pages@v4

‎.github/workflows/duvet.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: duvet on the local S3EC-Python
2+
3+
on:
4+
workflow_call:
5+
# Optional inputs that can be provided when calling this workflow
6+
7+
jobs:
8+
test:
9+
runs-on: ubuntu-slim
10+
permissions:
11+
id-token: write
12+
contents: read
13+
pages: write
14+
15+
steps:
16+
- name: Checkout code
17+
uses: actions/checkout@v6
18+
19+
- name: Checkout specific specification
20+
run: git submodule update --init --recursive specification
21+
22+
- name: Setup Rust toolchain
23+
uses: actions-rust-lang/setup-rust-toolchain@v1
24+
with:
25+
toolchain: stable
26+
27+
- name: Install duvet
28+
run: |
29+
cargo install duvet --locked
30+
31+
- name: Run duvet
32+
run: make duvet
33+
34+
- name: Upload duvet reports
35+
uses: actions/upload-artifact@v7
36+
with:
37+
name: reports
38+
include-hidden-files: true
39+
path: .duvet/reports/report.html
40+

‎.github/workflows/lint.yml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
name: Lint
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
workflow_call:
7+
workflow_dispatch:
8+
9+
jobs:
10+
lint:
11+
runs-on: macos-15
12+
13+
steps:
14+
- name: Checkout code
15+
uses: actions/checkout@v6
16+
17+
- name: Set up Python
18+
uses: actions/setup-python@v6
19+
with:
20+
python-version: '3.11'
21+
22+
- name: Install Uv
23+
run: pip install uv
24+
25+
- name: Install dependencies and run linting
26+
run: |
27+
make install
28+
make format-check
29+
make lint

0 commit comments

Comments
 (0)