Every EEG decoding score, reported with the protocol that produced it.
Website · Explore · Start here · Glossary · 中文 · Dataset · Data use & privacy
A benchmark is only readable with its conditions. BCI Report publishes EEG decoding results together with the cohort, electrode count, evaluation mode, chance level, training budget and known limitations that produced each number — and refuses to collapse them into one ranking, because the protocols do not share a scale.
The core matrix scores 14 methods on eight fixed protocols drawn from seven public datasets, 69 comparisons in all; cells that do not apply to a protocol are marked apart from cells not run. Around it, seventeen questions each read their own reviewed export: deployment conditions, clinical groups, foundation models as frozen probes and adapted, when a decoder should not act, questions asked in words, data quality checked end to end, measurements beyond scalp EEG and small pilots. They are kept apart rather than summed, because they measure different things. Research results, not diagnosis. No raw EEG, no per-participant scores, no model weights.
from datasets import load_dataset
load_dataset("Twu31/bci-report", "results") # the core matrix as first released
load_dataset("Twu31/bci-report", "core_matrix_rows") # rows that entered the core matrix later
load_dataset("Twu31/bci-report", "register") # the open register's dataset ladderOther configurations — topics, contrasts, seed_sensitivity, foundation_models, eegpt_core, core_matrix_contrasts — are listed on the dataset card.
Or straight from the site, no account:
curl -O https://bci.report/data/experiments.json # the core matrix as first released, with the directories
curl -s https://bci.report/api/index.md # every download, what it holds and the release that ships itWhat each file holds, loading examples and how to cite a release: bci.report/api.
| Protocol | Dataset | Cohort | Chance |
|---|---|---|---|
| Motor imagery vs. rest | ds003810 | 10 | 50% |
| SSVEP, 4 and 8 electrodes | BETA | 70 | 2.5% |
| P300 target | ds006593 | 21 | 50% |
| Semantic target | TMNRED | 30 | 50% |
| Mental arithmetic | EEGMAT | 36 | 50% |
| Sleep staging | EESM19 | 20 | 20% |
| Idle false activation | ds005342 | 4 | — |
Seventeen questions sit alongside, each with its own protocol: dry vs. wet electrodes · screen to VR · fewer electrodes · clinical groups · standing, walking, running · an earlier session to a later one · what calibration buys · which part of a pretrained model to update · when not to act · sleep-stage balance · does pretraining help · one model, several questions · questions asked in words · SSVEP frequencies a decoder never trained on · what the files show when checked end to end · beyond scalp EEG · what a small pilot can show
Every dataset and every method also has its own page, gathering each figure measured on it wherever it appears on the site: datasets (BETA, EESM19, the wearable dry/wet SSVEP set, the Mobile BCI set, …) · methods (EEGNet, LaBraM, CBraMod, EEGPT, CCA, FBCCA, eTRCA, …, plus the cross-protocol frozen foundation-encoder overview). So does each core-matrix protocol: protocols gives each one its cohort, split, electrodes, window, training mode, chance level, every method's score, and an inline pick-two comparator. To inspect how methods separate across protocols without collapsing them into an overall score, explore provides a level-removed 2D method profile map, a free protocol-by-protocol interval scatter, a 7×7 Spearman agreement matrix, and a paired pick-two comparator; start here and the glossary introduce EEG paradigms, chance levels, and descriptive intervals for new readers. Those pages add no number of their own; the site's checks re-read every figure on them from the download it came from.
For assistants and agents: llms.txt indexes every
page with its short answer, llms-full.txt
holds every English page in one file, and every page has a Markdown copy at
<page>/index.md.
"Jev-style" here means the interface of a decision model like TypeSafe's Jev, applied to EEG: encode the recording once, then answer several explicit, typed questions about it, each with a probability. The research plan started from a vision paper that applies the same idea to images: Yu & Yao, 2026 · Visual Jev: Accurate and Efficient Decisions from Shared Visual Context (arXiv:2609.25845).
The plan, set out on when not to act, asks whether this holds up on EEG in three routes, all run on public data, and Jev-style questions on EEG: the evidence gathers them: reliable decisions · one model, several questions · questions in language, mostly a negative result for questions the EEG head was never trained on.
What the newest releases changed, one line for every earlier release, and a dated register of what was held back: bci.report/releases. Every reviewed release as released, each download with its SHA-256: bci.report/releases/archive.
- Chance level differs per protocol. 57% on 40-class SSVEP is far above chance; 57% on a binary task is barely above it. Sorting across protocols compares numbers that do not share a scale.
- Intervals are descriptive. Bootstrap spreads over the scoring set, not confidence intervals, and not a basis for significance claims.
- Most comparisons use one seed. Two protocols add a three-seed check.
- Electrode subsets are not headsets. A 4-channel subset of a lab recording does not validate a 4-channel device.
- A small cohort is close to its parts. The idle protocol has four people, and its published mean can be turned back into a count. The arithmetic is written out on the site rather than left for a reader to discover.
Private sources → fixed local experiments → independent audits → source and
model release decisions → reviewed export → site/dist/ → host.
Raw EEG, trial rows, participant identifiers, embeddings, trained heads and pretrained weights never cross the export boundary. Every published number is reproduced from pinned inputs and checked against a review audit before it can ship; the Hugging Face mirror is built from the same bytes the site serves, so there is no second export path.
cd site && npm run build && node scripts/check-workbench.mjs
.venv/bin/python -m unittest discover -s pipeline/publication -p 'test_*.py'
.venv/bin/python -m unittest discover -s pipeline/tests -p 'test_*.py'
.venv/bin/python pipeline/publication/check_site_artifact.pyPassing these is not proof of anonymity. They match known-bad shapes and cannot reason about reconstruction from small denominators — which is why the idle cohort's invertibility is disclosed rather than asserted away.
| Path | What it is |
|---|---|
site/ |
The Astro site. Zero framework JS; the coverage matrix is server-rendered. |
pipeline/publication/ |
The release boundary: reviewed export, artifact inspection, dataset mirror. |
pipeline/ |
Literature catalogue, verified-result store, news ingestion with a manual gate. |
data/ |
Hand-maintained catalogues of models, benchmarks and verified results. |
research/ |
Rights and privacy review evidence, per dataset. |
Scientific corrections, benchmark-method questions, and rights, privacy or attribution concerns are welcome — including ones that would withdraw a published result. Affected results can be withheld while an issue is reviewed.
Open an issue, or write to contact@bci.report for scientific corrections and privacy@bci.report for rights, privacy and withdrawal requests. Please do not send raw EEG, participant names or health records.
Results were computed from public datasets released by other researchers. Credit belongs to them; this project adds only the measurements. Every published row carries its own source, licence and attribution.
| What | Licence |
|---|---|
Aggregate results: the result tables and protocol descriptors in site/public/data/, served under bci.report/data/ (see the data API) and mirrored on Hugging Face |
CC BY 4.0 — see LICENSE-DATA |
| Code: the site, the publication pipeline and their scripts | MIT |
Self-hosted web fonts (Newsreader, Inter, IBM Plex Mono in site/public/fonts/) |
SIL OFL 1.1 |
| The EEG recordings the results were computed from | Their authors' own terms. Not in this repository; each dataset page names the source and its licence |
CC BY 4.0 covers the measurements this project produced — it does not and cannot relicense the underlying recordings, which keep their own terms.
To cite a release, use CITATION.cff (GitHub's "Cite this repository"), and cite the upstream dataset each figure was computed on — every dataset page gives its credit line. Every topic, dataset and method page also ends with a "Cite this page" block naming the releases its figures come from. New releases appear in the Atom feed.
Every GitHub release is archived on Zenodo with its own version DOI; the
concept DOI 10.5281/zenodo.23123296
always resolves to the newest one, and is the one to cite. .zenodo.json
describes the repository for that archive; it is written by
site/scripts/write-zenodo-metadata.mjs from CITATION.cff and the built
dataset pages, and the site checks hold it to them.
Known limits. Most core-matrix cells come from one seed, the smallest cohort is four people, and catalogued models without a score are listed as such in the model directory. Each page states its own limits next to its numbers.