Phase 1 Long-Video Workspace

This repository contains the minimal long-video Phase 1 tuning workspace for the dual-sampling pipeline.

What lives here

• phase1_longvideo/: standalone long-video sampling package
• split/: train/val/test split manifests
• scripts/: metadata build, subset creation, evaluation, sweep, and validation utilities
• colab_phase1_longvideo.ipynb: Colab workflow for the long-video tuning loop

Local dataset expectation

The long-video dataset is not committed to git. Place the extracted dataset shards in a local dataset/ directory before running metadata build.

Baseline

Phase 1 baseline is uniform sampling with the same final_num_frames budget as the dual-stage run. Run separate baselines for 16, 32, and 64 frames.

Suggested workflow

1. Build metadata:

python scripts/build_phase1_longvideo_metadata.py

2. Create a small subset:

python scripts/sample_phase1_subset.py `
  --subset-name smoke12 `
  --target-video-count 12 `
  --source-split train

3. Run a quick synthetic smoke evaluation:

python scripts/eval_phase1_sampling.py `
  --input results/subsets/smoke12/smoke12_train_metadata.jsonl `
  --split train `
  --backend synthetic `
  --limit 20

4. Run a real SigLIP sweep in Colab:

python scripts/sweep_phase1_sampling.py `
  --input results/subsets/sample40/sample40_train_metadata.jsonl `
  --subset-name sample40 `
  --split train `
  --backend siglip `
  --siglip-device cuda `
  --resume

Colab guidance

• Prefer T4 first for this Phase 1 sweep.
• Switch to L4 only if a T4 run is too slow.
• Avoid A100 for this round; it is better reserved for LoRA or end-to-end runs.
• Copy the active video subset from Drive to local Colab disk before large sweeps to reduce Drive random I/O overhead.

Shared end-to-end metrics

For group-level comparisons with Phase 2, standardize on:

• mIoU
• R@1@0.3
• R@1@0.5
• R@1@0.7
• BERTScore
• AvgRuntimeSec

Do not treat BLEU/ROUGE/METEOR from inference_test.py as the main shared table metrics.