Benchmark catalogue
BCI-Bench is organized into focused tracks that test generalization and deployment behavior under standardized conditions.
Track definitions and dataset selections are under community development.
Can a BCI algorithm work on unseen users without subject-specific training?
Train on one participant group and evaluate on held-out participants excluded from training and validation.
Multi-subject motor imagery and ERP datasets.
Does an algorithm remain reliable across future sessions and neural drift?
Develop on earlier sessions and test chronologically on unseen future recordings.
Longitudinal and repeated-session datasets.
Can performance transfer across acquisition systems, channel layouts, and sites?
Train on one device or center and evaluate on recordings from another.
Compatible multi-device and multi-center cohorts.
How much new-user data is required before a model becomes useful?
Report zero-shot, 1-shot, 5-shot, and calibration-curve performance.
Datasets with sufficient trials per participant.
How gracefully does a model degrade when channels are noisy or unavailable?
Controlled channel dropout, signal corruption, and montage reduction.
High-density datasets with channel metadata.
Do pretrained neural models transfer fairly across BCI tasks and populations?
Frozen, adapted, and fully tuned protocols with controlled data access.
Multi-task datasets spanning common BCI paradigms.
Can an algorithm satisfy practical compute, memory, and real-time constraints?
Standardized runtime profiles, end-to-end latency, memory use, and throughput under declared hardware conditions.
Representative benchmark inputs with repeatable runtime harnesses.
Reporting standard
Track reports will pair central performance with subject-level distributions, calibration cost, robustness, and resource usage where relevant.