Feat/working branch - #159
Open
simonvanlierde wants to merge 1342 commits into
Open
Conversation
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
simonvanlierde
added a commit
that referenced
this pull request
Jul 12, 2026
- guard the getUser slot by sequence instead of promise identity (CodeQL missing-await) - read api.generated.ts directly instead of existsSync-then-read (CodeQL TOCTOU) - drop redundant location.hash guard and document server-side OAuth enforcement (CodeQL bypass FP) - anchor the Scalar request-blocking regex in docs e2e (CodeQL regex anchor) - catch Exception instead of BaseException in shutdown steps (code-quality) - add stamp-freshness HEALTHCHECK to the backups image (Trivy DS-0026) - use docstring bodies for protocol stubs instead of bare ellipses (code-quality) - remove unused Union import from alembic migration (code-quality)
simonvanlierde
added a commit
that referenced
this pull request
Jul 12, 2026
- break auth-services<->core import cycle: inline redis bool coercion in blocklist_store - break auth models<->user_database cycle: move get_user_db/get_auth_async_session to user_manager - rpi-cam ws auth: hash camera rate-limit bucket via rate_limit_bucket_key like all other callers - rpi-cam models: replace non-exhaustive match with dict lookup in to_http_error - connection_manager: sanitize camera_id in logs; pairing: drop unused _STATUS_WAITING - support_services/test_upload_security: replace del-parameter idiom (noqa B027 / pass) - test_email_providers: match provider URLs with startswith instead of substring - codeql config: enable AlertSuppression packs; exclude py/unused-import (ruff owns it) and informational py/cyclic-import - suppress verified false positives inline (token fingerprint sha256, secret-name logging, masked email logging)
simonvanlierde
force-pushed
the
feat/working-branch
branch
from
July 14, 2026 14:31
b0c5b17 to
11f74b6
Compare
simonvanlierde
added a commit
that referenced
this pull request
Jul 14, 2026
- guard the getUser slot by sequence instead of promise identity (CodeQL missing-await) - read api.generated.ts directly instead of existsSync-then-read (CodeQL TOCTOU) - drop redundant location.hash guard and document server-side OAuth enforcement (CodeQL bypass FP) - anchor the Scalar request-blocking regex in docs e2e (CodeQL regex anchor) - catch Exception instead of BaseException in shutdown steps (code-quality) - add stamp-freshness HEALTHCHECK to the backups image (Trivy DS-0026) - use docstring bodies for protocol stubs instead of bare ellipses (code-quality) - remove unused Union import from alembic migration (code-quality)
simonvanlierde
added a commit
that referenced
this pull request
Jul 14, 2026
- break auth-services<->core import cycle: inline redis bool coercion in blocklist_store - break auth models<->user_database cycle: move get_user_db/get_auth_async_session to user_manager - rpi-cam ws auth: hash camera rate-limit bucket via rate_limit_bucket_key like all other callers - rpi-cam models: replace non-exhaustive match with dict lookup in to_http_error - connection_manager: sanitize camera_id in logs; pairing: drop unused _STATUS_WAITING - support_services/test_upload_security: replace del-parameter idiom (noqa B027 / pass) - test_email_providers: match provider URLs with startswith instead of substring - codeql config: enable AlertSuppression packs; exclude py/unused-import (ruff owns it) and informational py/cyclic-import - suppress verified false positives inline (token fingerprint sha256, secret-name logging, masked email logging)
- describe the file/directory mode enforcement and placeholder rejection that just deploy-secrets-check now performs, in install.md and deployment.mdx - add the first-time chown commands for the backup bind-mount dir in deployment.mdx, matching deploy/CUTOVER.md §10
…aceholder rejection - pg_isready reports healthy on a missing-superuser cluster (observed on staging); the mismatch must be resolved via POSTGRES_SUPERUSER, the stack will not surface it - deploy-secrets-check rejects both replace-me and legacy placeholder- secret generations
…ted clusters Existing tables are owned by the cluster superuser, and DDL needs ownership, not grants; found live in the staging rehearsal (permission denied for alembic_version). The scratch replay could not catch it because a fresh scratch database is migrator-owned.
- Convert the six base/ui/icon.tsx call sites (profile Integrations/ Preferences/shared/SecuritySection, product ComponentRow) to base/Icon's name+color API; delete the now-unused base/ui/icon.tsx wrapper - Drop the vendored ToggleIcon/ToggleGroupIcon helpers, the only other ui/icon.tsx consumers, since neither was called anywhere in the app - Rename Icon.tsx's map keys from MaterialCommunityIcons-shaped strings to Lucide names and collapse duplicate keys pointing at the same glyph - Fix every call site across the app that used a renamed IconName
- drop unused AppTokens.spacing and constants.spacing.{lg,xl,xxl}
- drop unused breakpoints.desktop and radius.{md,lg} aliases
- migrate radius.sm call sites to radius.control, drop the sm alias
- stop exporting SEMANTIC_COLORS, lightTheme, and darkTheme from the
public theme barrel; import lightTheme/darkTheme from theme/themes
directly in the tests that need theme instances
- Extend ErrorState with title/actionLabel/iconColor props - Rewrite ProductPageErrorState on top of ErrorState, dropping the themeColors prop in favor of useAppTheme() internally - Update ProductDetailScreen's caller to drop themeColors - Delete dead centerState/centerText/subtleCenterText styles from States.tsx (slowLoadingContainer kept, still used for positioning) LivePreview.tsx's PreviewErrorOverlay is left as-is: it's an absolute-positioned overlay inside a fixed aspect-[4/3] card (in-card pattern), not the full-height centered pattern ErrorState covers.
- drop ParentMediaCrud.get_by_id, which no caller used, and point its two tests at the get_parent_media function it forwarded to - flatten auth/routers/admin/ into admin_users.py; the package held one module and a one-element router list
- _docker-ci-perf-report and _docker-ci-perf-thresholds only forwarded to backend recipes that CI already calls directly - _docker-ci-build was never invoked; _docker-ci-up already passes --build - _commit was an unreferenced alias for uv run cz commit
- drop the refreshAuthToken web test: it spied on a re-export the flow never consults, ran on ios so the web branch was unreachable, and asserted a value every failure path returns; the real behaviour is covered in the web describe and in authRefresh.test.ts - assert which broadcast the monitor-stream lookup asks for, so fetching the wrong camera's stream now fails the test - make the ProductNotFoundError type-guard assertion reachable instead of skipping it whenever the call does not throw - drop two repeated identical assertions
- rename the muted palette row to mutedForeground and document the real muted surface tone, which the table never named - rename the h2 type step to title, the key both consumers use, and add the micro step that existed only in tokens.json - name tokens.json as a machine source and point at the BRAND_PARITY table that reconciles the prose and JSON vocabularies - document elevationAndroid and the palette keys the table does not restate - repoint every Mirrors anchor, including two that were already stale and one that truncated the System-Font Rule mid-sentence
…or name fastapi-storages is not installed and not a dependency; StorageFile is local code. Renames FastAPIStorageFileNotFoundError to StorageFileNotFoundError, matching its ModelFileNotFoundError sibling.
brand.css ships a single IBM Plex Mono face at weight 400, and the only mono label in www (.stats-subhead) already renders at 400. The weight-500 claim described nothing, so a 500 label would have been browser-synthesised. - correct the type scale and the www label token to 400 - drop www's "IBM Plex Mono 400-500", which had forked from docs' 400
… imports - match a dedicated X-Relab-Telemetry-Key header in the WAF-skip rule instead of the Authorization value, which any zone-read grant could read back; jsonencode the value - filter DNS import lookups to CNAME so an apex TXT/MX record can never be bound - rewrite the split handover check-first; record the Redis fail-open rate-limit gap - fix stale state-encryption and state-path descriptions
- put a read-only docker-socket-proxy in front of Alloy; the agent never holds the socket - pass /dev/kmsg with CAP_SYSLOG so cAdvisor can actually count OOM kills - drop Alloy's own containers from discovery to stop an export-failure feedback loop - explicit export queue and retry window, self-metrics shipped, hostname fallback - mem/pids limits on alloy and the GPU exporter; json-file caps on all deploy services - send the X-Relab-Telemetry-Key header from api and alloy; require it with the endpoint - only truthy GPU_METRICS values include the GPU overlay; empty endpoint includes nothing - compose-config no longer includes the alloy overlay twice - warn when the offsite rclone.conf is missing entirely
- watchdog checks every stack service, all three timers, and empty RELAB_PING_* URLs, with time-bounded docker calls and compose stderr kept out of container ids - run_scheduled.sh traps TERM so a killed job still posts its failure ping - restore-check runs at 06:00, clear of the backup prune, with a deterministic container name the unit reaps on timeout - backup unit retries are bounded (3 per day) instead of forever - installer escapes sed replacements and warns when ping URLs are left empty - deploy env reader strips quotes and comments like the compose reader does - test the traversal escape at the path a real escape would write
… key - install guide: replace the deleted backup-unit recipe with timers-install - drop the backups profile from every up example; backup-run seeds the first snapshot - document TELEMETRY_EDGE_KEY next to the token everywhere the token is set - cutover: host-level checks for log rotation and telemetry leaving the host - DEPLOY-PROD: daemon.json log-opts fallback, what the watchdog now checks, and the rule that its local checks stay until the central alerts exist - security: the log-sanitization guarantee stops at the backend's own logs
- remove MONITORING-DESIGN.md and MONITORING-HANDOVER.md; the architecture is CMLPlatform/monitoring ADR 0002 and the open work is its docs/HANDOVER.md - the one Relab-side constraint, keeping the local watchdog checks until the central alerts exist, already lives in DEPLOY-PROD.md
- add create_password_user to the auth integration shared helpers and use it at the 19 sites that hashed a password inline - add build_mfa_user for the 13 MFA flow tests that stubbed the same user/user_manager pair - parametrize the seven assertion-claim rejection tests behind one fixture, and pin each case to its own rejection message; three of them previously asserted no message at all
- camerasQuery factory for 18 useCamerasQuery stubs - renderAssociations helper per describe in the OAuth associations tests - expectAlert helper for six of the login alert assertions - renderMulti/renderSingle for the filter modal prop lists - one shared twoImages fixture in the gallery lightbox tests - hoist the no-op delete-mutation mock into beforeEach - hlsOpts factory for the web HLS helper options
- read RELAB_PING_* from the unit environment when the seeded 0600 file is unreadable; systemd already loaded it as root, so a filled-in URL no longer alerts as empty forever - seed /etc/relab/relab.env owned by the deploy user and fix ownership on re-install, which also lets the post-install emptiness warning actually run - keep compose stderr out of the config --services capture so a warning cannot word-split into phantom "not running" services - factor the service-state and ping-URL checks into reducers and cover them in test_ops.sh, including the unreadable-file regression - treat a whitespace-only ping URL as empty, matching the install-time check - remove a leftover relab-backup container before backup-run so a host crash cannot wedge the deterministic name - strip quotes before the rclone: prefix test in the offsite-remote warning - give the watchdog timer Persistent=true like its two siblings
- run Alloy with cap_drop ALL plus SYSLOG and DAC_READ_SEARCH and no-new-privileges: root keeps its read-only host reads without the default set's write-side DAC_OVERRIDE - give the GPU exporter a read-only root, an empty capability set and no-new-privileges - feed the Cloudflare API token to curl over stdin instead of argv, where /proc exposes it to every local user
- scope the edge rate-limit claim to the auth endpoints and name both Cloudflare roots; upload and RPi limits are app-layer and fail open - describe Alloy's socket-proxy wiring and dropped capability set instead of a socket mount it never had - list the egress network's real members in the compose comment
The Alloy overlay, its config and the scheduled-job wrapper are now taken byte-identical from CMLPlatform/monitoring at v0.2.0. Everything that differs between projects arrives as an environment variable, so a fix upstream reaches every project host unchanged. - rename compose.logging.alloy.yaml and compose.gpu.yaml to compose.telemetry.yml and compose.telemetry.gpu.yml - parameterise the project identity: PROJECT replaces the hardcoded `relab` in the Alloy config and the API's resource attributes, committed per environment in deploy/env/ because it identifies the project, not the host - rename the WAF-skip header to X-Telemetry-Key and the dead-man's-switch variables to PING_*, both without a compatibility shim; deploy/CUTOVER-PROD.md carries the manual order, which for the zone-global Cloudflare rule is deploy both hosts first and apply second - rename the backup-run and backup-restore-smoke recipes to backup and restore-check so the vendored wrapper resolves a job to a recipe by name - run the root env policy check on `up`: the vendored overlay cannot hard-require TELEMETRY_EDGE_KEY, and an empty key loses the WAF exemption silently - record the vendored files and their upstream tag in deploy/README.md - cover the job-to-recipe resolution in scripts/test_ops.sh, including that every job the systemd units invoke exists as a recipe
- reference `expo/types` from the tracked `uniwind-env.d.ts`, so the app typecheck resolves the `global.css` side-effect import without the gitignored `expo-env.d.ts` that only exists on a developer machine - set `ENVIRONMENT=testing` on the email-template recipes; `CoreSettings` refuses to build without it, so `just check` died in `email-check` - reformat the deploy docs to what mdformat produces, and tell shellcheck the systemd job loop splits on words on purpose - bump the pinned node, python, uv, and postgres base digests, picking up the util-linux fixes behind 39 HIGH findings per image - drop the unused pip from the backend runtime stages and upgrade openssl in the backups image, which the postgres base still ships unpatched - allowlist the caddy Go binary CVEs with the same expiry-and-statement pattern as the existing entries; upstream ships no rebuilt release - allow the two image-size advisories in dependency review: they cover every published version, and metro only ever parses local dev files - fix two asserts that reopened an image inside the assertion, and explain the empty `except FileNotFoundError` in the brand asset sync
GHSA-5p4m-2wfm-xmqj landed after the previous run and dependency review flagged the transitive js-yaml 4.3.0. Scope the override to the 4.x range; the 3.15.1 already in the tree carries the fix.
GHSA-vcc3-ghjq-m6fr reaches the app through query-string@7, which every expo-router release still pins, and the patched release is ESM-only so an override breaks routing at runtime. Record why each allowed advisory is unfixable and what would let us drop it.
The six container scans and the three CodeQL analyses are most of the Security workflow's runtime, and re-running them on every intermediate push to a draft buys nothing. Skip them while `draft` is true and add `ready_for_review` to the trigger types so marking a PR ready fills them in. Schedule, merge queue, dispatch, and pushes to main are untouched.
`filters categories by search query` left waitFor polling for the 300ms debounce, which cost 5-7s locally against 5-20ms for every sibling test in the file, and crossed the 15s per-test budget on slower CI hardware. Advancing the timers directly makes it deterministic and drops it to 8ms.
api and migrator both mount `user_uploads`, and Docker seeds an empty named volume from each container's image as it starts. Started together by the orchestration smoke test, the two seedings raced and one container died on `mkdir .../images: file exists`. Mirror the dependency compose.deploy.yaml already declares, `required: false` so `compose up api` alone is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat/working-branch → main: security hardening, MFA, brand refresh, and repo restructure
TL;DR
A ~2.5-month working branch (728 commits) intended as a single squash-merge. It hardens the auth/account-security stack, adds TOTP MFA with recovery codes, removes dead data-model surface (organizations, newsletter), rebuilds the public homepage around live stats, rebrands toward R9lab / ReLab, and restructures the two frontend subrepos.
Why squash
The branch grew organically over ~2.5 months, so its intermediate history is noisy. Squashing collapses it into one clean, coherent commit on
main. The changes were checked againstmainacross multiple angles — correctness, security, test coverage, performance, and over-engineering — before landing.Highlights by area
🔐 Auth & account security (largest area)
The auth module was substantially reworked — service layer split into focused units (
lifecycle,login_flow,mfa_service,session_flow,token_store,email/*,oauth/*) and hardened end to end.🗄️ Data model & migrations
17 new migrations. Notable removals and reshapes:
email_canonical),has_usable_password, MFA/recovery tables.📁 File storage hardening
Paginated media lists, real zip-size enforcement, malware scanning extended to device thumbnails, quota owner keying fixed, sensitive-key rejection in local storage, JSON-recursion guard, dotted filenames accepted, uploaded filenames validated against MIME type.
🌐 Public site (
www, formerlyfrontend-web)Homepage rebuilt: new
Hero,SiteFooter, privacy page, and aStatsPanelfed by new public stats API (/stats/totals,/stats/categories,/stats/series) — monthly activity chart with per-category part counts. New brand/token CSS, theme handling, 404 page, and a Vitest suite for the new components.📷 RPi camera plugin
WebSocket relay bounded against unresponsive devices, camera-flapping-to-offline fixed, livestream/recording lifecycle corrected, device key kept on the LAN, response ownership verified, device-assertion lifetime capped, circuit breaker made Redis-only and atomic.
📱 App (React Native, formerly
frontend-app)MFA challenge screen + pending-login routing, OAuth callback via URL fragment, new-product drafts can set type/material before first save, "already live" stream dialog, single-flight auth/MFA submits, gallery no longer silently deletes unrenderable images, static-background refactor (parallax scaffolding removed), plus a large sweep of review-driven bug fixes.
🎨 Brand → R9lab / ReLab
R9lab logo/wordmark/flask marks wired into www, docs, and app UIs; email templates rebranded with hosted wordmark; README wordmark. Asset generators live in
assets/logo-src/.Copy pass across the app UI, docs, and www: unified sign-in/sign-out terminology, sentence-case titles and buttons, plainer error and empty-state messages, and less boilerplate — with tests and e2e specs updated to match.
🏗️ Infra / CI / deploy
needs, OpenAPI/app-codegen freshness gates, JUnit → Codecov Test Analytics, per-PR RN a11y lint, Playwright browser caching.X-Forwarded-For.🧭 Repo restructure
frontend-web→wwwfrontend-app→appRenames (with history preserved) touch devcontainers, CI, docs, and configs — the bulk of the file count.
Verification
Each subrepo's loop (
just fix→just check→just test) was run as changes landed. Beyond the test suites, the diff was reviewed across correctness, security, test-coverage, performance, and simplification angles. New migrations include a downgrade/upgrade round-trip test, and new logic (stats, auth flows, storage, MFA) ships with regression tests.