VibeClip AI

VibeClip AI is a creator-facing long-video understanding and temporal-grounding workflow. It turns a natural-language question into an evidence-aligned video interval, explains why the moment matters, and carries the selection through highlight scoring, platform-specific content generation, safety review, and export planning.

What the interactive demo shows

Choose a video → Analyze → Retrieve a moment → Inspect evidence → Generate a creator pack → Review risk → Export

• Natural-language video Q&A with timestamps and transcript evidence
• Explainable highlight ranking across hook, emotion, context, visual stability, platform fit, and risk
• Human-in-the-loop moment selection
• Creator packs for TikTok, YouTube Shorts, Instagram Reels, Xiaohongshu, and Bilibili
• Pre-publish privacy and safety review
• Responsive, recruiter-friendly English interface

Demo mode vs. model mode

The public experience is deterministic by default so reviewers can always complete the full workflow. The UI calls POST /api/moment-search; that route uses the local demo retriever when no research service is configured.

Set TEMPORAL_GROUNDING_API_URL to connect a model service that exposes POST /moment-search. When the service returns a valid interval, the interface labels the answer Live model and renders the returned timestamps. If the optional service is unavailable, the app falls back to demo mode rather than breaking the portfolio experience.

The model service is maintained separately in the private research repository; it is not bundled into or deployed by this Next.js repository. See .env.example for the only web-app environment variable.

Expected model contract

Request:

{
  "video_id": "lecture-demo",
  "query": "Where does the speaker explain temporal grounding?",
  "transcript": []
}

Supported response fields:

{
  "predictedStart": 322.0,
  "predictedEnd": 358.0,
  "confidence": 0.89,
  "groundingScore": 0.81,
  "reason": "The interval contains the clearest retrieval-versus-grounding comparison.",
  "saliencyScores": [],
  "sampledFrames": []
}

Snake-case interval aliases (predicted_start, predicted_end, grounding_score) are also accepted.

Stack

• Next.js App Router
• React and TypeScript
• Tailwind CSS
• Optional Python/FastAPI temporal-grounding service

Routes

• / — Product overview
• /demo — Demo project selection
• /workspace/[videoId] — Analysis dashboard and grounded timeline
• /search/[videoId] — Video Q&A and moment retrieval
• /clip/[videoId]/[momentId] — Creator studio and safety review
• /api/moment-search — Model adapter with deterministic fallback

Local development

corepack enable
pnpm install --frozen-lockfile
copy .env.example .env.local  # Windows; optional for model mode
pnpm dev

Omit .env.local for the deterministic demo. For a model-backed session, start the separate FastAPI service and set TEMPORAL_GROUNDING_API_URL to its server-reachable base URL.

Deployment

The web app can be deployed on any Node.js host that supports Next.js 15:

1. Install with pnpm install --frozen-lockfile and build with pnpm build.
2. Use pnpm start for a self-hosted production server; managed Next.js platforms can use their standard start command.
3. Leave TEMPORAL_GROUNDING_API_URL unset for the reliable public demo, or configure it as a server-side environment variable for model mode.
4. If model mode is enabled, deploy the GPU-backed FastAPI service separately and ensure the web host can reach it. A browser-accessible URL alone is not sufficient if the web runtime cannot connect to it.

The repository intentionally contains no provider-specific deployment manifest. Run pnpm build before publishing and smoke-test /demo plus POST /api/moment-search in the deployed environment.

Evidence status

Implemented product behavior and research evidence are tracked separately in docs/evidence-and-evaluation.md. The published interface must not present the listed resume metrics as independently reproduced until the corresponding immutable experiment artifacts are available.

Research-to-product architecture

The research service owns query-aware sampling, multimodal inference, and temporal localization. The web application owns product orchestration: result normalization, explainability, human selection, creator-pack generation, safety review, and a resilient public demo fallback. This separation keeps model experimentation independent from the recruiter-facing product experience.