Evaluate protein conformational ensembles, temporal evidence and independent event predictions.
Dynbench is an evaluation protocol, not a molecular generator. Summary measurements and geometric checks describe properties of an output; independent event scores test its predictions against common held-out molecular-dynamics (MD) outcomes. Unordered ensembles have no temporal claim and are not assigned a temporal failure.
Start with the V35 reproduction entry. It accompanies Dynamical information does not establish predictive accuracy in protein conformational generators and contains the manuscript-specific analysis software and numerical source data. This dated snapshot does not replace the archived v1.0.1 release or create a new release tag.
The primary comparison evaluates MDGen and ConfRover on 73 ATLAS and 36 Lockbox proteins. It distinguishes information provided by temporal order and starting geometry from accuracy relative to independent MD. Sampling budgets and probability estimators are examined separately; scores alone do not identify an architecture's causal error source.
- MDGen's native frame interval is 0.4 ns, not 1 ns.
- Current two-model forecasts use 50 frames at 2 ns, spanning 0–98 ns.
MDGen selects
0,5,...,245from one continuous 250-frame block; ConfRover and MD select0,2,...,98from their 1-ns records. - No event crosses a concatenated rollout boundary. The original 1-ns MD/ConfRover summary analysis is separate; MDGen has no 1-ns temporal estimate.
- The earlier mixed-clock MDGen/ConfRover ranking and rank-reversal interpretation are withdrawn from current conclusions. Historical files remain records, not a corrected common-time leaderboard.
Read the clock and version precedence before using an older report or script.
Python 3.10+ is required. These commands use local saved inputs: no model generation, MD, GPU use or trajectory download.
git clone https://github.com/kkkniengLiu/dynbench.git
cd dynbench/analysis/v35
python -m pip install numpy pandas
python prepare_data.py
python Supplementary_Software/verify_source_data.py --data Source_Data
python Supplementary_Software/reference_sensitivity_20261014/reproduce.py --source-data Source_Data --out reference_sensitivity_replay_newPreparation checks the archive SHA-256 and every member before installing the data. The archive is approximately 54 MiB; one uncompressed JSON exceeds GitHub's individual-file limit. Existing data directories are verified, not overwritten.
| Analysis | Reproduction entry |
|---|---|
| Matched-time forecasts, order contrasts and seed summaries | Source-data verifier |
| Initial-state support | Conditional support |
| Generation budget and same-protein reference length (A1/B3) | Budget/reference guide |
| Shared starting geometry (A2) | Shared-start replay |
| Complete summary-rule counterexample (JG1) | Known-process replay |
| Correct versus exchanged inputs (CE1) | Condition-exchange replay |
| Held-out-replica L4 and eight-protein omission checks | V35 reproduction map |
The evaluation guide explains task applicability, reference roles, observation budgets and parallel prediction scoring. A temporal-and-quality pass does not certify an identical transition law or application-level predictive accuracy.
V35's reference-only sensitivity accepts 243/246 held-out-replica evaluations, with 79/82 proteins passing all three folds. All failed cases remain included. These are repeated evaluations of 82 proteins, not 246 independent proteins; the original model scores are not replaced.
The root dynbench/ scorer, frozen_board_v*, historical figures and
experiments/lockbox36/ retain their archived versions. The JSON field realdyn
is a historical name for the combined temporal/quality decision, not a fraction
of physical dynamics learned. Full trajectory-level scoring requires separately
obtained coordinates and the appropriate clock, snapshot and dependencies.
The prospective selection of Lockbox-36 does not make later analyses on that examined cohort a new independent confirmation. Keep each module's uncertainty unit and scope. Numerical replay is distinct from rerunning molecular generation or MD and from validating a new biological use.
Use CITATION.cff for the archived software release. For current
analyses also identify the exact Git commit and
manuscript-v35-20261014 snapshot. No new DOI or formal
software release was created by this synchronization.
Dynbench-authored software is MIT licensed; original documentation, figures and source tables are CC BY 4.0. Upstream dataset/model terms still apply to derived inputs; see LICENSES.md and THIRD_PARTY_NOTICES.md. Submission documents, author forms, model weights and private raw MD trajectories are not part of this update.