yammbs-dataset-submission

Input files and scripts for benchmarking OpenFF force fields with yammbs

Usage

Running benchmarks in CI

  1. Create a new branch starting from the master branch.

  2. Create a new entry in the submissions directory, with the general format YYYY-MM-DD-Name. For example:

    mkdir submissions/$(date +%Y-%m-%d)-Sage-2.1.0
  3. Add a YAML input file in this directory specifying the force field and datasets to use for the run. For example: 
    forcefield: openff-2.1.0.offxml
datasets:
    - datasets/cache/industry.json

    All paths should be relative to the root of the repository, and cached datasets must be used. Force fields can also correspond to built-in force fields recognized by the toolkit (as in the example). Currently only single datasets are supported.

  4. Push your branch and open a PR.
  5. Request a review, and get the PR approved.
  6. Make a comment of the form /run-optimization-benchmarks path/to/submission [conda-env.yaml] or /run-torsion-benchmarks path/to-submission [conda-env.yaml] on the PR. The brackets indicate an optional argument. If the path to the conda environment is omitted, the default environment will be used (devtools/env.yaml).
  7. Wait for the benchmarks to finish.
  8. Review the results, and update your submission with a README containing a link to the Zenodo archive created by CI.
    • An OpenFF admin (probably your PR reviewer) will need to manually publish the Zenodo entry before you can do this.
  9. Merge your PR!

Output

This will produce CSV files corresponding to the DDE, RMSD, TFD, and internal-coordinate RMSD (ICRMSD) metrics computed by yammbs.

Generating datasets

See datasets/README.md

Benchmark Results

| Submission | Description | DOI | |————————————————-|————————————————————————————|————————————————————————————————————-| | Parsley-1.3.1-unconstrained | openff_unconstrained-1.3.1.offxml | 10.5281/zenodo.14172472 | | Sage-2.0.0 | openff_unconstrained-2.0.0.offxml | 10.5281/zenodo.14188644 | | Sage-2.1.0 | openff-2.1.0.offxml | 10.5281/zenodo.14053221 | | Sage-2.1.0-unconstrained | openff_unconstrained-2.1.0.offxml | 10.5281/zenodo.14058464 | | Sage-2.2.1 | openff_unconstrained-2.2.1.offxml | 10.5281/zenodo.14200591 | | Null-0.0.3 | Protein parameter fit, null model v0.0.3, unconstrained | DOI | | Null-0.0.3-Looser-Priors | Protein parameter fit, null model v0.0.3, unconstrained with looser torsion priors | DOI |

This site is open source. Improve this page.