Skip to content

SYSTEM_DESIGN.md

Latest commit

 

History

History
668 lines (554 loc) · 35.9 KB

SYSTEM_DESIGN.md

File metadata and controls

668 lines (554 loc) · 35.9 KB
Raw

TaskSphere - Enterprise AI Project Management Platform

Technical Architecture & System Design Document

This document serves as the comprehensive, production-grade technical specification for TaskSphere. It provides developer guidance on architecture design, data models, security frameworks, background worker operations, AI governance, analytics logic, and day-to-day operations.

🗺️ 1. System Design & Architecture

High-Level Architecture Diagram

TaskSphere routes client requests through an Nginx reverse proxy, distributing ingress across static client asset services, REST API routes, and full-duplex WebSocket channels. Backed by Redis caching and queues, a MongoDB cluster, and external Google Gemini intelligence APIs, the architecture maintains strict tenant isolation and low-latency response times.

graph TD
    %% Clients
    Client[Web Browser / Client UI] -->|Port 80/443| Nginx{Nginx Reverse Proxy}

    %% Reverse Proxy Ingress Distribution
    Nginx -->|Static Assets| WebRoot[Vite Static HTML/JS Dist]
    Nginx -->|Express REST /api/v1| App[Express Backend Servers]
    Nginx -->|WebSockets /socket.io| SocketIO[Socket.io WebSocket Server]
    
    %% Cache & Pub/Sub
    App -->|Rate Limits & Cache| Redis[(Redis Cache & Session Store)]
    SocketIO -->|Adapter Pub/Sub| Redis
    
    %% Datastores & Job Queues
    App -->|Write/Read Operations| DB[(MongoDB Cluster)]
    App -->|Push Jobs| Queue[BullMQ Job Queues]
    
    %% Workers & Processors
    Worker[Background Queue Workers] -->|Process Jobs| Queue
    Worker -->|Fetch Session Context| Redis
    Worker -->|Read/Write Records| DB
    Worker -->|Gemini Inference API| Gemini[Google Gemini AI Models]
Loading

Request Lifecycle

The following diagram and step-by-step trace illustrates a typical client request (e.g., updating a Task's status via the Kanban board):

sequenceDiagram
    autonumber
    actor Client as Browser Client
    participant Proxy as Nginx Proxy
    participant Web as Express Web Server
    participant Auth as Auth Middleware
    participant Controller as Task Controller
    participant Service as Task Service
    participant Repo as Task Repository
    participant MDB as MongoDB Database
    participant EB as Event Bus (EventEmitter)
    participant WS as Socket.io Server
    
    Client->>Proxy: PUT /api/v1/tasks/:id (with JWT & Body)
    Proxy->>Web: Forward Request
    Web->>Auth: Parse and Validate JWT Access Token
    alt Token Invalid
        Auth-->>Client: 401 Unauthorized
    end
    Auth->>Web: Inject req.user & req.organization
    Web->>Web: Sanitize inputs (SQL/NoSQL query injection filters)
    Web->>Controller: Route to updateTask(req, res)
    Controller->>Service: call TaskService.updateTask(id, orgId, body, actorId, clientInfo)
    Service->>Repo: call TaskRepository.update(id, orgId, updateData)
    Repo->>MDB: Execute findOneAndUpdate()
    MDB-->>Repo: Return updated document
    Repo-->>Service: Return populated model
    Service->>EB: eventBus.emit('task.updated', eventPayload)
    Note over Service,EB: Event bus triggers activity feeds, audit logging & webhooks async
    Service-->>Controller: Return task object
    Controller-->>Client: 200 OK (updated task payload)
    
    %% Real-time synchronization
    EB->>WS: Push Socket emit task_updated
    WS->>Client: Broadcast task_updated to Room `task:{id}` & `org:{orgId}`
Loading
  1. Ingress Route: Client browser initiates a PUT request to /api/v1/tasks/:id. Nginx terminates TLS and routes request to the active Express server cluster.
  2. Security & Validation: Express passes the request through security filters:
    • Helmet: Standardizes security headers.
    • Express Rate Limiter: Validates rate-limit quotas against the client IP in Redis.
    • Sanitizer: Strips any input keys containing MongoDB operator prefix flags ($).
  3. Authentication Handshake: authMiddleware.ts extracts the JWT from the Authorization bearer token, decodes the signature, and appends req.user and req.headers['x-organization-id'] as req.organization.
  4. Controller Routing: Express router matches path and calls taskController.updateTask.
  5. Business Logic Execution: TaskService.updateTask verifies existence, checks permissions, and queries the datastore.
  6. Data Mutation: TaskRepository updates Mongoose document. If model validation succeeds, change is written to MongoDB.
  7. Asynchronous Event Broadcasting:
    • eventBus publishes task.updated.
    • activityFeedService catches event and inserts user activity records.
    • webhookService constructs webhook payloads and dispatches them to configured URL endpoints.
    • socketService emits task_updated to the room task:{id} to sync other active developer displays.
  8. Egress Response: Controller returns a formatted JSON payload containing { success: true, data: updatedTask }.

🗄️ 2. Database Design Document

TaskSphere utilizes MongoDB with Mongoose ODM for schematized document-relational models.

Collection Dictionary & Schema Definitions

Collection Model Class Description Key Indexes / Fields
users User User profiles, login hashes, and preferences. email (Unique), isActive, onlineStatus
organizations Organization Tenancy groupings holding users and billing tiers. name
projects Project Workspace project divisions with unique prefix keys. organization (Ref), key (Unique prefix)
tasks Task Core issue logs (tasks, stories, bugs, epics). project (Ref), sprint (Ref), assignees (Ref Array)
sprints Sprint Agile development cadence boundaries. project (Ref), startDate, endDate, status
comments Comment Nested developer message boards on tasks. task (Ref), author (Ref), parentId (Self-ref)
refreshtokens RefreshToken Expiring sessions for automated access token rotation. user (Ref), token (Unique), expiresAt (TTL Index)
invitations Invitation Email workspace member signup tokens. organization (Ref), token (Unique), expiresAt (TTL)
wikipages WikiPage Markdown team wiki documentation records. project (Ref), slug (Unique)
auditlogs AuditLog Comprehensive security records for SOC-2 compliance. organization (Ref), actor (Ref), createdAt
automations Automation Trigger-action rules for board events. project (Ref), triggerType, actionType

Relationships & Cardinality

erDiagram
    ORGANIZATION ||--o{ USER : "has members"
    ORGANIZATION ||--o{ PROJECT : "owns"
    ORGANIZATION ||--o{ TASK : "houses"
    PROJECT ||--o{ TASK : "contains"
    PROJECT ||--o{ SPRINT : "defines"
    TASK ||--o{ COMMENT : "has"
    TASK }o--o{ USER : "assigned to"
    USER ||--o{ REFRESH_TOKEN : "owns"
    ORGANIZATION ||--o{ INVITATION : "issues"
Loading
  1. Organization to Project (1:N): A tenant organization houses multiple independent projects.
  2. Project to Task (1:N): Each task belongs to a single project and gains a prefix ticket key (e.g., TSP-101).
  3. Task Assignees (N:M): A task can be assigned to multiple developers; a developer can be working on multiple tasks concurrently.
  4. Task Self-References (1:1 / 1:N): A task can reference an epic parent or a parentTask (for subtasks), and can point to a list of blocked/blocking task dependencies.
  5. Comment Nesting (1:N): Threaded comments reference a parentId comment.

Indexing Strategies

  • Text Search Indexes: Optimized for global search queries: 
    TaskSchema.index({ title: 'text', description: 'text', labels: 'text' });
ProjectSchema.index({ name: 'text', key: 'text', description: 'text' });
  • Compound Query Indexes: Avoid in-memory sort collections:
    • Task: { organization: 1, project: 1, status: 1 } - Facilitates quick Kanban status load operations.
    • Project: { organization: 1, key: 1 } (Unique) - Ensures project prefix codes never collide within the same company.
  • TTL (Time To Live) Indexes:
    • RefreshToken: Expired token documents are automatically cleared out by MongoDB: 
      RefreshTokenSchema.index({ expiresAt: 1 }, { expireAfterSeconds: 0 });
    • Invitation: Pending workspace registrations expire automatically after the signup window: 
      InvitationSchema.index({ expiresAt: 1 }, { expireAfterSeconds: 0 });

Data Resilience Strategies

  • Soft Delete Pattern: Critical models like Task and Project include a nullable deletedAt: Date | null field. Standard application repository queries filter out soft-deleted documents (deletedAt: null). Permanent deletions are postponed for 30 days, allowing users to restore items from the Trash Bin.
  • Optimistic Locking: Handled using Mongoose version keys (__v). For fields exposed to high concurrency (such as loggedHours or status drag-and-drops), update mutations are executed conditionally (__v match) to detect and resolve edit collisions between collaborative developers.

🔒 3. Security Threat Model

TaskSphere mitigates the OWASP Top 10 web vulnerabilities using defensive middleware, structured schemas, and strict JWT policies.

Threat Area Specific Risk Mitigation Strategy in TaskSphere
XSS (Cross-Site Scripting) Malicious scripts injected via Markdown fields or comments. Content is sanitized on the frontend using DOMPurify. The Express backend utilizes Helmet middleware to enforce strict Content Security Policy (CSP) headers.
CSRF (Cross-Site Request Request) State-changing request forged on behalf of authenticated users. All client credentials (JWT) are sent in headers rather than authorization cookies for API requests, making requests immune to standard CSRF. Refresh flows use a double-submit cookie pattern.
NoSQL Injection Injecting malicious operator keys (e.g., { $ne: "" }) to bypass login queries. Global Express sanitization middleware traverses req.body, req.query, and req.params, systematically deleting any key prefix starting with a dollar sign ($).
JWT Theft & Exposure Session hijacking via captured client access tokens. Access tokens have a short 15-minute lifespan and are stored in client memory. The Refresh token is stored in an HttpOnly, Secure, SameSite=Strict cookie, rotated on every refresh call.
Brute Force & DoS Database overload through automated login trials. express-rate-limit connected to a Redis bucket stores request counts. Limits login paths to 5 requests per minute, and global API calls to 100 requests per 15 minutes per IP.
Session Hijacking Accessing the app through copy-pasted active tokens. The refreshtokens collection stores active session metadata, logging the target client deviceIp and userAgent. Ip changes or token reuse attempts immediately revoke the entire user session.
File Upload Abuse Uploading large files containing malicious scripts or viruses. File service limits sizes to 10MB, verifies magic number signatures to assert MIME type rather than trusting extensions, and passes files through ClamAV containers before writing to S3 buckets.
SSRF (Server-Side Request) Malicious webhooks target localhost internal network services. Webhook queue dispatcher runs checks using the ipaddr.js module. Prohibits request routing to RFC1918 private network spaces (e.g., 127.0.0.1, 10.0.0.0/8, 192.168.0.0/16).

🔄 4. Data Lifecycle

stateDiagram-v2
    [*] --> Created : Task Form Submitted
    Created --> Updated : Field Modified (Title/Assignee)
    Updated --> Updated : In Progress / In Code Review
    Updated --> Completed : Status Moved to 'Done'
    Completed --> Archived : Sprint Completed & Closed (Auto-Archive)
    Archived --> SoftDeleted : User Clicks 'Delete'
    SoftDeleted --> Restored : Restore from Trash Bin (within 30 days)
    Restored --> Archived
    SoftDeleted --> HardDeleted : Cleanup Worker runs (age > 30 days)
    HardDeleted --> [*]
Loading
  1. Created: Task is initialized via Board UI. System allocates database entry, triggers event hooks, and registers target deadlines in the database index.
  2. Updated: Active lifecycle step. Subject to status modifications, comments, time tracking accumulations, and version key increments.
  3. Completed: Status shifts to Done. Satisfies Definition of Done (DoD).
  4. Archived: Sprint close events archive all child tasks. Documents are indexed out of active dashboard fetches but remain queryable for historic charts.
  5. Soft Deleted: Soft deletion marks deletedAt = new Date(). Task is moved to workspace Trash.
  6. Hard Deleted: Background cron workers query the datastore for documents where deletedAt < Date.now() - 30 days. Matching records are deleted from MongoDB, and their static file attachments are purged from AWS S3.

⚙️ 5. Background Jobs (BullMQ Registry)

Background tasks run out-of-process using a dedicated Node worker tier, keeping the REST API responsive.

                    ┌──────────────┐
                    │  REST API    │
                    └──────┬───────┘
                           │ Producer: Queue.add()
                           ▼
                 ┌──────────────────┐
                 │  Redis Cluster   │
                 └─────────┬────────┘
                           │ Consumer: Worker()
                           ▼
                  ┌────────────────┐
                  │ Background Tier│
                  └────────────────┘

Queue Definitions

1. email-queue

  • Responsibility: Sends transactional and system notifications (invitations, registration, password resets).
  • Payload Schema: 
    interface EmailJob {
  to: string;
  subject: string;
  template: 'invite' | 'reset' | 'daily-digest';
  variables: Record<string, any>;
}
  • Concurrency Limit: 10 concurrent operations. Supports automatic exponential backoff retry.

2. notification-queue

  • Responsibility: Processes multi-channel alerts (Web Push notifications, internal alerts, Slack webhooks).
  • Payload Schema: 
    interface NotificationJob {
  userId: string;
  title: string;
  message: string;
  channel: 'websocket' | 'slack' | 'email';
  slackWebhookUrl?: string;
}
  • Execution Profile: Low latency, immediate execution. Integrates with the SocketService to push notifications to active sessions.

3. ai-queue

  • Responsibility: Manages calls to Google Gemini AI models, mitigating rate limits and socket timeouts.
  • Payload Schema: 
    interface AIJob {
  type: 'sprint_planning' | 'story_generation' | 'meeting_summary';
  taskId?: string;
  backlogItems?: string[];
  velocity?: number;
  description?: string;
  rawNotes?: string;
}
  • Failure Strategy: Attempts 3 retries with a 5-second backoff delay in case of transient API failures or rate limits.

4. reminder-queue

  • Responsibility: Triggers alarms, warnings, and calendar alerts for upcoming task deadlines.
  • Payload Schema: 
    interface ReminderJob {
  taskId: string;
  recipientIds: string[];
  dueDate: Date;
}

⏰ 6. Scheduled Jobs (Cron)

Scheduled cron tasks are registered in BullMQ using repeatable job options.

┌─────────────────────────────────┬──────────────┬────────────────────────────────────────────────────────┐
│ Job Name                        │ Expression   │ Responsibility                                         │
├─────────────────────────────────┼──────────────┼────────────────────────────────────────────────────────┤
│ daily-database-backup           │ 0 2 * * *    │ Generates a mongodump and uploads it to an S3 bucket   │
│ archive-completed-sprints       │ 0 0 * * 1    │ Archives sprints and tasks that ended last week        │
│ rebuild-analytics-cache         │ */30 * * * * │ Aggregates project performance data in Redis           │
│ purge-expired-sessions          │ 0 3 * * *    │ Deletes expired RefreshTokens from the database        │
│ clear-stale-invitations         │ 0 4 * * *    │ Removes organization invitations past their expiration  │
│ send-digest-emails              │ 0 8 * * 1-5  │ Compiles daily task summaries for active developers    │
└─────────────────────────────────┴──────────────┴────────────────────────────────────────────────────────┘

Execution Implementation

Scheduled cron workers query database indices to batch cleanups (e.g., removing invitations where expiresAt < Date.now()), avoiding large collection scans.

🌐 7. API Design & Standards

TaskSphere's REST API follows JSON-API guidelines, implementing standardized pagination, sorting, filtering, and field selection.

Endpoint Conventions

  • Version Isolation: All active endpoints prefix /api/v1/.
  • Pagination Model: Uses zero-indexed page numbers via query keys:
    • page: Page index (Default: 0).
    • limit: Records per page (Default: 20, Max: 100).
    • Payload Structure: 
      {
  "success": true,
  "data": [...],
  "meta": {
    "currentPage": 0,
    "totalPages": 5,
    "totalRecords": 98,
    "hasMore": true
  }
}
  • Sorting Standards: Handled via the sort query parameter. Use a hyphen prefix (-) for descending order:
    • GET /api/v1/tasks?sort=-createdAt (Newest first).
    • GET /api/v1/tasks?sort=priority,dueDate (Ascending priority, then ascending due date).
  • Filtering Schema: Filter queries use nested brackets:
    • GET /api/v1/tasks?filter[status]=Done&filter[priority]=high
  • Field Selection: Limits bandwidth by requesting only specific database fields:
    • GET /api/v1/tasks?fields=title,status,assignees

Write and Execution Patterns

  • Bulk Operations: Restricted to specific bulk routes:
    • POST /api/v1/tasks/bulk-create - Allocates up to 50 tasks in a single query.
    • PATCH /api/v1/tasks/bulk-status - Updates multiple tasks at once (e.g., bulk-moving items to a sprint).
  • API Versioning Policy: Non-breaking changes are deployed directly in /api/v1. Breaking adjustments (such as schema overhauls) trigger a new /api/v2 namespace. Deprecated routes return an X-API-Deprecated: true response header.

🔌 8. WebSocket Architecture

Real-time synchronization uses Socket.io, providing cross-tab state updates without client polling.

                 Client (Browser)
                     │  WSS Connection
                     ▼
             Socket.io Namespace "/"
                     │
         ┌───────────┴───────────┐
         ▼                       ▼
    Room: org:{id}        Room: task:{id}
    (Kanban Drag/Drop)    (Collaborative Editor)

Namespace & Room Layouts

TaskSphere utilizes the default namespace /, segmenting events into specific rooms based on tenant scope:

  • org:{organizationId}: Broadcasts team actions, Kanban card movements, and sprint updates.
  • project:{projectId}: Broadcasts project settings updates and wiki changes.
  • task:{taskId}: Synchronizes editor modifications, timers, and comments.

Connection & Lifecycle Security

  1. Authentication Handshake:
    • Connection attempts require a valid JWT token in handshake authentication arguments: 
      const socket = io({ auth: { token: "Bearer <token>" } });
    • The connection middleware validates the token, rejecting connection requests if validation fails.
  2. Reconnection & Resilience:
    • Client socket options are configured for automatic reconnection: 
      reconnection: true,
reconnectionAttempts: 10,
reconnectionDelay: 1000,
reconnectionDelayMax: 5000
    • On reconnection, the client automatically re-joins active rooms using a saved local state.
  3. Presence Tracking:
    • User presence is managed in Redis using hashes (presence:user:<userId>).
    • Sockets publish a heartbeat every 30 seconds. If no heartbeat is received within 45 seconds, the user is marked offline, and an alert is broadcast to their organization's members.
  4. Event Naming Conventions:
    • Socket events use colon-separated active verbs:
      • task:created
      • task:updated
      • comment:added
      • wiki:editing

🤖 9. AI Governance

TaskSphere integrates Google Gemini models to assist agile workflows. This framework enforces rate limits, manages costs, and validates responses.

       Express API
            │
            ▼
┌──────────────────────────────┐
│  AI Governance Framework     │
│  - Token Bucket Rate Limiting│
│  - Prompt Version Control    │
│  - Response Sanitizer / JSON │
└───────────┬──────────────────┘
            │ Gemini SDK Call
            ▼
    Google Gemini API

Prompt Engineering Registry

1. Sprint Forecaster

  • Template Version: sprint-planning-v1.2
  • Target Model: gemini-1.5-flash (Optimized for context processing speed and cost efficiency)
  • Base Prompt: 
    You are an expert Agile Scrum Master. Perform AI Sprint Planning.
Backlog Items:
[Insert Backlog List]
Sprint Velocity Limit: [Insert Target Points] Story Points.
Generate a Sprint Plan. Group the items into Sprints, estimating story points, outlining the Sprint Goal for Sprint 1, and identifying potential bottlenecks or dependencies. Format output in neat Markdown.

2. Acceptance Criteria Generator

  • Template Version: bdd-user-story-v2.0
  • Base Prompt: 
    Create a comprehensive Agile User Story based on: "[Insert Feature Summary]".
Include Description (As a... I want to... So that...), Acceptance Criteria in BDD format (Given/When/Then), and technical considerations.

Policies and Cost Guards

  • Token Constraints: Request payloads are limited to 4,000 tokens, and response sizes are capped at 1,500 tokens.
  • Rate Limiting: Users are limited to 10 AI operations per hour.
  • Cost Monitoring: System admins can view dashboards tracking token usage. If an organization exceeds its billing tier budget, AI endpoints return a 402 Payment Required status.
  • Response Validation: All model outputs are checked by a validation utility. If a response does not match the requested JSON schema or contains prohibited content, it is discarded, and the system retries with a higher temperature or falls back to a template.
  • Caching Policy: Standard prompts and backlog summaries are hashed. If a prompt's hash matches an active cache entry in Redis, the system returns the cached response, avoiding duplicate Gemini API costs.

📊 10. Analytics Calculation Engine

The analytics dashboard calculates project metrics using standardized equations.

Velocity Tracking

Calculates completed story points per sprint: $$\text{Sprint Velocity} = \sum (\text{Story Points of completed tasks in Sprint } S)$$ The Average Velocity is calculated over the last 3 completed sprints: $$\text{Average Velocity} = \frac{V_{S-1} + V_{S-2} + V_{S-3}}{3}$$

Lead Time & Cycle Time

  • Lead Time: Time elapsed from task creation to task completion: $$\text{Lead Time} = t_{\text{completed}} - t_{\text{created}}$$
  • Cycle Time: Time elapsed from when work actively starts (moved to an "In Progress" state) to task completion: $$\text{Cycle Time} = t_{\text{completed}} - t_{\text{in_progress}}$$

Sprint Burndown Progression

Tracks remaining story points against sprint days: $$\text{Remaining SP on Day } d = \text{Total SP} - \sum (\text{Story Points of tasks completed by Day } d)$$

  • Ideal Line Formula: $$\text{Ideal Target SP on Day } d = \text{Total SP} \times \left(1 - \frac{d}{D_{\text{total}}}\right)$$ Where $D_{\text{total}}$ is the total duration of the sprint in days.

Team Throughput & Sprint Health

  • Throughput: Number of tasks completed per week: $$\text{Throughput} = \frac{\text{Count of Completed Tasks}}{\text{Duration in Weeks}}$$
  • Sprint Health Score: $$\text{Health Score} = 0.5 \times \left(\frac{\text{Completed SP}}{\text{Committed SP}}\right) + 0.3 \times \left(\frac{\text{Scope Creep SP}}{\text{Committed SP}}\right) + 0.2 \times \left(\frac{\text{Bugs Resolved}}{\text{Bugs Reported}}\right)$$

🗺️ 11. Feature Roadmap

Beyond the v1.0 release, TaskSphere plans to implement features focused on security, scalability, and DevOps integrations.

   v1.0 (Current)             v2.0 (Planned)               v3.0 (Future)
┌────────────────────┐     ┌─────────────────────┐     ┌─────────────────────┐
│ - Kanban Boards    │     │ - GitHub & GitLab   │     │ - GraphQL Gateway   │
│ - Sprint Planner   │     │ - CI/CD Integration │     │ - Kubernetes Deploy │
│ - Gemini Pro AI    │ ──> │ - API Key Provision │ ──> │ - Offline Mode      │
│ - Markdown Wiki    │     │ - OpenTelemetry     │     │ - ML Risk Engine    │
│ - Socket.io Sync   │     │ - Mobile App (PWA)  │     │ - Multi-Region DB   │
└────────────────────┘     └─────────────────────┘     └─────────────────────┘

Version 2.0 Roadmap Details

  • Version Control Integrations: Automatically link GitHub pull requests to tasks based on branch names or commits (e.g., feature/TSP-101).
  • CI/CD Integrations: Display Jenkins/GitHub Actions build status and deployment metrics directly on the task dashboard.
  • Enterprise API Keys: Allow developers to provision API keys with scoped permissions to automate custom workflows.
  • Observability: Integrate OpenTelemetry to collect distributed traces across the backend API, workers, and databases.
  • Mobile Support: Transition the frontend to a Progressive Web App (PWA) with responsive mobile layouts.

📈 12. Performance Targets (SLAs)

TaskSphere monitors API performance against key Service Level Agreements (SLAs) in production:

┌───────────────────────────┬───────────────┬────────────────────────────────────────────────────────┐
│ Operations Path           │ Target SLA    │ Operational Guardrails                                 │
├───────────────────────────┼───────────────┼────────────────────────────────────────────────────────┤
│ POST /api/v1/auth/login   │ < 500 ms      │ Limits bcrypt rounds to 12; utilizes fast Redis lookup │
│ GET /api/v1/tasks         │ < 200 ms      │ Uses compound indexes; retrieves only selected fields  │
│ GET /api/v1/analytics     │ < 2.0 s       │ Pre-aggregates metrics; reads from Redis cache         │
│ GET /api/v1/search        │ < 300 ms      │ Uses MongoDB text index and limits results to 10       │
│ WebSocket Event Sync      │ < 250 ms      │ Low payload sizes and dedicated Redis Pub/Sub          │
│ POST /api/v1/ai/summary   │ < 10.0 s      │ Uses BullMQ to run requests out-of-process             │
└───────────────────────────┴───────────────┴────────────────────────────────────────────────────────┘

👁️ 13. Observability & Telemetry

TaskSphere implements structured logging and metrics collection for production visibility.

Prometheus Metrics Configuration

The application exposes a scraper endpoint /metrics that exports Prometheus-formatted performance metrics:

  • tasksphere_memory_usage_bytes{type="heapUsed"}: Node heap usage.
  • tasksphere_process_uptime_seconds: Engine process uptime.
  • tasksphere_http_requests_total{method, route, status}: Total HTTP requests.
  • tasksphere_db_query_duration_seconds{collection, operation}: Database query duration.

Structured Logging & Correlation IDs

  • Correlation IDs: Every request is assigned a unique X-Correlation-ID uuid via middleware. This ID is appended to all logs, queue jobs, and socket events, allowing developers to trace issues end-to-end.
  • Structured Logs: Logs are structured as JSON and sent to console or files using Winston: 
    {"timestamp":"2026-07-12T20:56:11Z","level":"info","message":"Task created","correlationId":"890810b9-b14f-4741-b532-69cf7181bff6","orgId":"123","actorId":"abc"}
  • Queue Diagnostics: Monitors queue depth, job latency, and failure rates in BullMQ.
  • Cache Hit Ratios: Tracks Redis cache efficiency (redis_cache_hits_total and redis_cache_misses_total) to optimize performance.

💻 14. Development & Coding Standards

To maintain code quality across developer teams, TaskSphere enforces strict validation checks.

Code Style & Formatting

  • Linter & Formatter: Configured with ESLint and Prettier. Run checks locally: 
    npm run lint
npm run format
  • Git Hooks: Managed via Husky. Prevents commits if linter or tests fail:
    • pre-commit: Runs npm run lint and checks staged changes.
    • commit-msg: Validates commit messages against Conventional Commits patterns.

Git Conventions

  • Commit Messages: Messages must follow the Conventional Commits structure:
    • feat(auth): add MFA verification flow
    • fix(board): fix task drag-and-drop ordering bug
  • Branch Naming:
    • Features: feature/TSP-[TaskId]-summary
    • Hotfixes: hotfix/[TaskId]-summary

Pull Request & Review Checklist

All pull requests require approval from at least one senior developer and must pass the automated review checklist:

  • Code follows project ESLint conventions.
  • New endpoints have corresponding Swagger/OpenAPI annotations.
  • Database schema changes include migration scripts.
  • No credential keys, passwords, or secrets are hardcoded.

💰 15. Infrastructure Cost Estimations

TaskSphere runs on AWS, utilizing the Free Tier for early-stage deployments while structuring migration paths for production workloads.

┌──────────────────────┬──────────────────────┬────────────────────────┬─────────────────────────────┐
│ AWS Service          │ Dev (Free Tier) Cost │ Production Tier Cost   │ Architecture Sizing         │
├──────────────────────┼──────────────────────┼────────────────────────┼─────────────────────────────┤
│ EC2 Nodes            │ $0.00 (t3.micro)     │ $38.00 / mo (t3.medium)│ 2x application containers    │
│ AWS S3 Storage       │ $0.00 (5GB limits)   │ $12.00 / mo            │ 100GB storage + transfers   │
│ Redis Cache/Queue    │ $0.00 (Redis Local)  │ $15.00 / mo (Redis El.)│ 1GB RAM cache node          │
│ MongoDB Atlas        │ $0.00 (Shared M0)    │ $56.00 / mo (M10 tier) │ 10GB storage, replica sets  │
│ CloudFront CDN       │ $0.00 (1TB free)     │ $20.00 / mo            │ Global edge asset caching   │
│ Route 53 DNS         │ $0.50 (1 Hosted Zone)│ $3.00 / mo             │ Active routing rules        │
├──────────────────────┼──────────────────────┼────────────────────────┼─────────────────────────────┤
│ Total Sizing         │ $0.50 / month        │ $144.00 / month        │ Production scaling budget   │
└──────────────────────┴──────────────────────┴────────────────────────┴─────────────────────────────┘

🔄 16. Disaster Recovery Plan

TaskSphere maintains data availability and system resilience through daily backup routines.

  • Backup Schedule: Automated daily logical backups (mongodump) are created at 02:00 UTC and uploaded to secure AWS S3 buckets.
  • Retention Policy: Daily backups are retained for 30 days. Weekly backups are kept for 90 days.
  • RTO & RPO:
    • Recovery Time Objective (RTO): < 1 Hour. System operations must be restored within 1 hour of a critical failure.
    • Recovery Point Objective (RPO): < 15 Minutes. Database changes are synced using journal writes, limiting data loss to 15 minutes.
  • Restoration Drill:
    1. Spin up a new target MongoDB instance.
    2. Download the latest archive bundle from S3: 
      aws s3 cp s3://tasksphere-backups/prod-2026-07-12.archive.gz ./
    3. Restore the database using mongorestore: 
      mongorestore --gzip --archive=prod-2026-07-12.archive.gz --uri="mongodb://localhost:27017"

📘 17. Operational Runbook

This guide describes routine operational tasks for the platform.

App Deployment

Deploy new versions using Docker:

# Pull changes from main repository
git pull origin main

# Build and start services in detached mode
docker compose up -d --build --remove-orphans

Quick Rollback Plan

Rollback a release in case of critical production bugs:

# Rollback codebase to the previous tag
git checkout tags/v1.1.4-working

# Rebuild containers immediately
docker compose up -d --build --remove-orphans

Rotating Secrets

Rotate JWT secrets or database credentials:

  1. Update values in backend .env file: 
    JWT_SECRET="new_high_entropy_secret_key_2026"
  2. Restart backend nodes sequentially to prevent downtime: 
    docker compose restart backend
  3. Rotating secrets invalidates current sessions, requiring users to log back in.

Redis Diagnostics

Clear Redis cache values:

# Connect to local Redis container
docker exec -it tasksphere-redis redis-cli

# Clear cache keys (preserves BullMQ job queues)
redis-cli --scan --pattern "cache:*" | xargs redis-cli del

Managing BullMQ Queues

Restart failed background jobs in the worker tier:

# Check queue status via PM2 or log outputs
docker compose logs backend-worker --tail=50

# Restart the worker process container
docker compose restart worker