RAG Workspace

General-purpose agentic RAG chat app for working with documents, reports, and PDFs, with a Next.js chat UI, a FastAPI backend, and citation highlighting straight from the source files.

Amazon reports are the sample corpus, not a restriction on the subject matter. This release targets a local, single-user workspace, not an authenticated multi-user service. See Production deployment below before exposing the backend beyond your own machine.

Structure

backend/      FastAPI app - LangGraph agentic RAG pipeline, chat + citation-highlight APIs
frontend/     Next.js (App Router, TypeScript, Tailwind) chat UI
notebooks/    Prototyping notebooks the backend logic was developed and validated in
Docs/         Source PDFs - served statically and used for citation highlighting

• notebooks/2024bot.ipynb / notebooks/chatbot.ipynb - earlier retrieval pipeline iterations.
• notebooks/agentic_rag.ipynb - the agentic LangGraph pipeline (self-correcting retrieval + generation, LLM-judge eval harness) that backend/app/graph.py is ported from.

Running the app

Backend (from repo root, using the uv-managed venv):

cd backend
source ../.venv/bin/activate
uvicorn app.main:app --reload --port 8000

Frontend:

cd frontend
npm install
npm run dev

Open http://localhost:3000. It calls http://localhost:8000 by default (frontend/.env.local).

API troubleshooting and acceptance tests

• Use the repo's .venv interpreter for both the backend and tests; a system Python may have different provider adapters installed. Restore declared dependencies with uv sync.
• DeepSeek runs in non-thinking mode because its structured query analysis, grading, calls force a tool choice that V4 thinking mode rejects with HTTP 400.
• Backend logs are written to backend/logs/app.log. Failed runs log their run ID, stage, provider, model, and a safe HTTP error category; credentials and upstream error response bodies are not included. Restart the backend after Python changes unless it is running with --reload.
• Offline regression tests: LANGSMITH_TRACING=false .venv/bin/python -m unittest backend.tests.test_graph backend.tests.test_workspace backend.tests.test_providers
• Paid live acceptance test: LANGSMITH_TRACING=false RUN_LIVE_E2E=1 .venv/bin/python -m unittest backend.tests.test_live_api This exercises upload, reindex, hybrid retrieval, reranking, DeepSeek Flash/Pro and Together chat, citations, usage, previews, and session persistence. It requires the configured DeepSeek, Together, Jina, Cohere, and Qdrant credentials. Only synthetic evidence is sent; local state and a uniquely named Qdrant test collection are isolated and removed afterward. Normal tests never enable paid calls.

Chat controls and appearance

• Sidebar settings include a live message preview, chat font size, spacing, and visibility controls. Appearance preferences are saved locally in this browser; Reset appearance restores defaults without deleting chats.
• Settings, uploads, and source previews open in keyboard-accessible dialogs with Escape-to-close and focus restoration. On small screens, conversation history opens in a drawer; the composer stays visible while messages and suggestions scroll. Minimal mode hides secondary controls, and light/dark themes share the same typography and spacing.
• Stop cancels the browser request and requests cancellation of remaining server work using its run ID. Unconfirmed cancellation is reported. Retry reuses the last unanswered message without duplicating it. Regenerate keeps the existing answer until a replacement succeeds. Switching chats cancels the current browser request; cancellation does not guarantee that an upstream provider stops processing or billing.
• Both model pickers use the same filtered catalog and prefer higher-capability options only when listed. An unlisted saved model is shown explicitly rather than silently replaced, and unavailable providers are disabled.
• The model picker estimates a single call. Answer details and Insights track run usage separately, including retries where reported. Unknown prices are never counted as zero; estimated known costs are not a provider invoice.
• Storage failures are shown in the UI. Invalid saved chat data is not overwritten automatically.

Workspace

The collapsible sidebar separates everyday chat from deeper tools:

• Chat: use Context above the composer to select documents, tags, file types, document dates, or page ranges. Empty document selection means the entire current project, not the entire installation. Filters remain hard constraints when retrieval has no matches. Choose concise, detailed, or evidence-only answers.
• Library: browse and filter sources, inspect their metadata and indexing status, edit tags, preview evidence, select documents for chat, and manage explicit project memories. Upload remains available beside the composer.
• Insights: inspect runs, execution traces, usage and unknown-cost coverage. Saved historical results remain viewable and downloadable.

Research tools (comparison runs, typed extraction, guided workflows, and evaluations) have been removed. Existing chat history, sources, budgets, and historical research/evaluation records are preserved; no new research runs can be submitted. Ordinary document questions, including comparisons, still work in Chat.

Sources stay collapsed until needed. Click a numbered reference in an answer to open its evidence. PDF previews support page navigation; other formats use text and format-specific locations. A missing original or unsuccessful highlight is reported rather than presented as an exact match.

Grounding checks are automated signals, not guarantees. When the system cannot support an answer after bounded retries, it should return supported findings with evidence gaps or abstain, not silently accept an unsupported draft.

Projects, history, and memory

Projects scope sources, conversations, and explicitly saved memory. Use the project selector to switch or create a project. Memory is off by default in chat; enable it in Context when useful. Saved notes are editable and removable and are never treated as cited document evidence.

Conversation history is persisted by the local backend. Existing browser chats are imported idempotently; the pre-import browser copy is retained for recovery. Pending browser changes are retried on reconnect. Storage errors are visible: do not clear browser storage while it contains unsaved changes. Appearance preferences remain browser-local.

The full conversation is saved, but only bounded recent history is sent to the model. Save important longer-term context explicitly as project memory.

Optional web search

Configure BRAVE_SEARCH_API_KEY on the backend to enable public search. Document-only chat remains the default. Enter and approve an exact public query in Context. Never include private document contents, identifiers, or chat history in it. Chat approval is consumed by the next message and is cleared when the query changes or the page reloads.

Public URL imports and search evidence must respect source access restrictions. The app does not support bypassing logins, paywalls, or fetching local/private network destinations. Web evidence includes its URL and fetch timestamp and remains distinguishable from uploaded sources.

Ingestion and costs

The expanded ingestion path covers text PDFs, TXT/Markdown, DOCX, CSV/XLSX, PPTX, and supported public pages. Scanned PDFs require optional local OCR prerequisites; missing parsers/OCR support must produce actionable errors. Spreadsheet evidence uses sheet/cell locations, and slide evidence uses slide numbers rather than pretending every format is a PDF.

Ingestion checks raw-text coverage for every extracted page/section before embedding, requires one dense and sparse embedding per chunk, and reads every written point back from Qdrant to verify its source payload and selected vector slots. Only then is the new version activated and the previous version retired in one SQLite transaction. Failed or interrupted staging remains outside the active-version search scope; the previous working version is preserved. Blank pages are recorded separately, and an OCR page returning no text fails explicitly rather than disappearing from an otherwise successful import.

Library distinguishes verified at ingestion coverage from an available but unverified old index. Legacy catalog discovery counts existing points; it does not prove that the original was fully indexed. For missing sections/tables, use Reindex / retry on the affected source and wait for verified completion. This reprocesses the original, can incur embedding costs, and does not reindex the rest of the library. Verification establishes storage coverage of extracted text, not OCR accuracy, table interpretation, or guaranteed retrieval relevance. Extraction-only uploads are not labeled index-verified. Page/chunk counts and version/run IDs are recorded in ingestion logs without source text.

Chat, indexing, summarization, search, and reranking can consume paid provider calls. No provider work is started when opening Insights. Budgets are optional, with no preset dollar cap. Limits cannot guarantee exact invoice caps when prices or usage are unavailable, and cancelling a request cannot undo an in-flight charge.

Evaluation metrics require appropriate labels. Recall/precision against expected sources and model-judged grounding answer different questions; neither is a measured universal "hallucination rate."

In Insights, expand Pricing estimates & advanced settings to inspect source-dated defaults and configure workspace-wide overrides using provider/model keys. The model calculator and runtime meter use the same effective rates. DeepSeek's published pricing (verified September 8, 2026) supplies V4 Flash, Pro, and Flash Vision estimates: peak Monday-Friday 01:00-04:00 and 06:00-10:00 UTC, half price otherwise. Each operation snapshots the rates at its start. Reported cache-hit tokens use the discounted input rate; absent cache details are explicitly an uncached estimate. Together's successfully retrieved model catalog supplies its available rates. The settings display shows source dates; rates are estimates, not invoices, and published rates may change.

An override must use exactly one basis: per_call, per_unit, or both input_per_million and output_per_million with optional cached_input_per_million. Jina prepaid/account rates and Cohere Rerank v3.5 unit prices are not assumed: enter your account's rates. For Jina use jina/jina-embeddings-v3 (including hybrid retrieval) and an output rate of zero; reported input tokens are captured for queries and uploads. For Cohere use cohere/rerank-v3.5 and per_unit in USD per search unit, not per document. The actual meta.billed_units.search_units determines its estimated cost. When search units cannot be reserved reliably, capped runs require explicit approval for unknown costs before scheduling.

An empty override object restores defaults. Historical calls retain their original pricing snapshots, including unknown costs; setting prices never retroactively invents charges. Mixed usage displays a known subtotal (partial), not a complete total. Missing usage, unconfigured/local operations, and failed calls remain unknown rather than free. Provider billing is never changed.

Local operations and limits

• Local state lives in .workspace/ by default (WORKSPACE_DIR overrides it). It includes SQLite, documents, versions, memories, runs, and artifacts and is excluded from Git. It is not encrypted at rest. Back it up together with the Qdrant collection; stop the backend before copying the local state directory.
• Run one Uvicorn application process, not multiple --workers. The local worker queue uses two threads by default. Restarted work is marked interrupted, not silently replayed or rebilled.
• Upload API calls without an embedding provider extract only; the UI exposes an indexing-provider selector, including an extraction-only option. Reindexing is explicit and can incur costs. Office parsers are included in uv sync.
• PDF OCR uses local Tesseract through PyMuPDF. Install Tesseract and the needed language data, and configure TESSDATA_PREFIX when discovery is unavailable. OCR is opt-in; no remote OCR service is used.
• Default limits: 25 MiB per upload, 500 pages, 10,000 table rows, 100 MiB expanded archives, 24 execution steps, 100 provider operations, 100,000 known/estimated tokens, and 600 seconds per run. The queue accepts at most 12 waiting runs. Environment settings in backend/app/config.py control these bounds; a requested UI limit cannot exceed the server cap.
• Removed sources leave retrieval immediately. If vector/file deletion fails, Library displays pending cleanup with an explicit retry action. Existing sample files in Docs/ are preserved.
• Original files and citations are version-bound. Legacy originals changed outside the app require explicit reindexing rather than silently resolving old citations to new content.

Validation

Offline regression tests use synthetic fixtures and mocked paid providers:

PYTHONPATH=backend .venv/bin/python -m unittest discover -s backend/tests -p 'test_*.py'
cd frontend && npm run build

Targeted ingestion regressions (synthetic 30-page PDF, late tables, dropped chunks, corrupted/partial vector writes, and activation rollback): LANGSMITH_TRACING=false .venv/bin/python -m unittest backend.tests.test_ingest backend.tests.test_workspace.ParserTests

Live model/search quality and actual OCR need separate opt-in checks with the chosen provider keys and local OCR installation.

API surface

• POST /api/chat - send { message, history, config, project_id, session_id, context }; source scope, answer quality, run ID, and usage extend the original { answer, retries, citations } contract.
• GET /api/workspace?project_id=... - projects, document inventory, memory, recent runs, usage summary, and web readiness.
• /api/documents/... - uploads, URL imports, metadata, previews, and indexing jobs.
• /api/sessions and /api/memories - local conversation and explicit memory persistence.
• GET /api/runs/{id} - inspect chat, ingestion, or preserved historical runs; POST /api/runs/{id}/cancel requests cancellation, and GET /api/runs/{id}/export exports saved results. The former /api/research/... endpoints are no longer available.
• GET /api/usage?project_id=... - recorded calls, token counts, estimated known cost, and unknown-cost coverage.
• POST /api/citation/highlight - given a citation's { filename, page_number, passage }, renders that PDF page as a PNG with the passage highlighted (PyMuPDF). Chunks use Anthropic's Contextual Retrieval pattern (an LLM-generated "Context: ..." preamble prepended to each chunk for embedding quality, stripped before searching); a highlight isn't guaranteed even after stripping it, since some source PDFs have non-standard font encodings that make their text layer unsearchable - the page still renders without a highlight box in that case.
• GET /api/config/options - available LLM/embedding/sparse providers for the frontend's model picker.

Model combinations

• LLM provider (deepseek | ollama | together) is fully independent of retrieval and freely swappable - it only affects query analysis, grading, and answer generation. For Together, use the serverless model openai/gpt-oss-20b (verified working with the repo's TOGETHER_API_KEY).
• Embedding provider: the FinalNew52_Chains Qdrant collection declares four named vectors (jina | text_large | text_small | voyage). jina and voyage are indexed and usable. text_large/text_small (OpenAI) are indexed but need OPENAI_API_KEY set to use. GET /api/config/options reports real-time availability per provider (checked via a live probe query, not just API-key presence), and the frontend disables anything unusable; POST /api/chat also rejects unusable providers with a clear 400 error rather than silently returning empty context.
• Any LLM x usable embedding provider x either sparse provider (splade_sparse | bm25_sparse) is valid.
• Using ollama requires a local Ollama server running with the requested model pulled (e.g. ollama pull llama3.1:8b).

Setup

1. uv sync (installs backend/notebook Python dependencies from pyproject.toml).
2. Populate .env at the repo root with QDRANT_API_KEY, COHERE_API_KEY, JINA_API_KEY, DEEPSEEK_API_KEY (or TOGETHER_API_KEY if using the Together LLM provider; and optionally OPENAI_API_KEY/VOYAGE_API_KEY for the other embedding providers, LANGSMITH_API_KEY for tracing).
3. cd frontend && npm install.

Production deployment

Containerized production artifacts live at the repo root and in deploy/:

• backend/Dockerfile / frontend/Dockerfile - multi-stage production images (gunicorn+uvicorn workers for the backend; Next.js output: "standalone" for the frontend).
• docker-compose.yml - wires backend, frontend, and an nginx reverse proxy together, with a persistent volume for .workspace/.
• deploy/nginx/nginx.conf - reverse proxy routing (/api/* and /health to the backend, everything else to the frontend).
• .env.production.example - template for required production environment variables; copy to .env.production (gitignored) and fill in real values.
• deploy/deploy.sh - pulls the latest production branch, rebuilds, and restarts the stack on a server.

See deploy/README.md for full instructions, including running the stack locally with Docker Compose and deploying to an AWS EC2 instance. Qdrant is already cloud-hosted (QDRANT_URL/QDRANT_API_KEY), so no vector database needs to be containerized.

Notes

• All secrets are read from the repo-root .env - none are hardcoded in backend or frontend code.