Final evaluation should remain separate from model development and tuning.
About the project
Community-oriented benchmark infrastructure.
BCI-Bench aims to provide transparent, reproducible, and deployment-oriented evaluation standards for brain-computer interface algorithms.
Maintaining team
Neural Engineering Lab & Database Group @ Tsinghua
Our mission
Make claims of BCI generalization easier to test and trust.
Current results are often difficult to compare because datasets, splits, preprocessing, calibration, and reporting differ between studies. BCI-Bench is designed to turn these choices into explicit, versioned benchmark protocols.
The platform will focus on generalization beyond familiar data: unseen users, future sessions, different acquisition systems, limited calibration, degraded signals, and practical deployment constraints.
Principles
Standards for a credible benchmark.
Definitions, data roles, metrics, and verification levels should be explicit.
Starter kits, baselines, environments, and reports should support inspection.
Accuracy should be interpreted alongside reliability, calibration, and cost.
Track design should reflect input from researchers, clinicians, and engineers.
Evaluation should reflect real constraints and realistic failure conditions.
Team and contact
Built at Tsinghua, open to the BCI community.
BCI-Bench is maintained by the Neural Engineering Lab & Database Group @ Tsinghua. Formal governance and acknowledgments will be published as the project enters its first benchmark release.
Project discussion
Follow development, propose benchmark ideas, or report website issues through GitHub.
Open GitHubResearch and collaboration
Contact the project regarding datasets, benchmark design, or institutional collaboration.
hello@bcibench.com