Skip to content

Revision/dataset to enable batch post processing - #447

Merged
chrisiacovella merged 9 commits into
choderalab:mainfrom
chrisiacovella:revision/dataset
Oct 20, 2026
Merged

chrisiacovella merged 9 commits into
choderalab:mainfrom
chrisiacovella:revision/dataset

Conversation

@chrisiacovella

@chrisiacovella chrisiacovella commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Pull Request Summary

This primarily focuses on making it easier to examine a trained model after the fact.

Key changes

Notable points that this PR has either accomplished or will accomplish.

  • When loading an hdf5 file, we store the name of the record; this quantity is available within the TorchDataset. To get the name associated with an individual index in the torch_dataset, we can call the _get_datapoint function that will return the name and then the normal output from the iterator.
  • Adds in a helper function to setup an instance of the DataModule directly from a config .toml file, making it easier to exactly replicate splitting and any filtering that was done during training.
  • Adds in a helper function to perform batch evaluation of a subset of the dataset (in the DataModule instance). This will perform some aggregation as well, adding in the name, model output, and associated metadata.
  • Fixes doc string in geom qm9.
  • Adds in a notebook to show how to use the helper functions.

Associated Issue(s)

Pull Request Checklist

  • Issue(s) raised/addressed and linked
  • Includes appropriate unit test(s)
  • Appropriate docstring(s) added/updated
  • Appropriate .rst doc file(s) added/updated
  • PR is ready for review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

evaluate_batch maps batched results to the wrong records for subset="train" because the train dataloader shuffles by default while the code assumes fixed indices order.

6 open issues
What changed in this PR

This PR adds tooling to make it easier to examine a trained model after the fact. It propagates each record's name from the hdf5 file through the TorchDataset (exposed via a new _get_datapoint), adds initialize_datamodule_from_config to rebuild a DataModule from a training .toml, and adds evaluate_batch to run batched inference over a dataset subset while aggregating name/output/metadata per original index. It also hardens hdf5 processing against records with zero valid conformers and corrects the GEOM QM9 energy-level-of-theory docstrings (GFN2-xTB, not r2SCAN-3c DFT) per issue #445.

Changes:

  • Track and expose record names through the dataset; add _get_datapoint, initialize_datamodule_from_config, and evaluate_batch helpers.
  • Guard _from_hdf5 so records with no valid configurations are skipped, and align the names list accordingly; replace stray prints with log.debug.
  • Fix GEOM QM9 docstrings to reflect xtb energies, bump curation version to 1.1, and add an evaluation example notebook/tests.
File Description
modelforge/​potential/​potential.py Adds evaluate_batch for batched subset evaluation; has a train-subset ordering bug and a redundant per-batch append.
modelforge/​dataset/​dataset.py Adds name tracking, _get_datapoint, initialize_datamodule_from_config, zero-conformer guard, and cache-validation update.
modelforge/​tests/​test_potentials.py New test_batch_evaluation covering only the test subset.
modelforge/​tests/​test_dataset.py Tests name access via _get_datapoint; includes a leftover debug print.
modelforge-curate/​.../​geom_qm9_curation.py Corrects energy level-of-theory docstring to xtb.
modelforge-curate/​.../​scripts/​curate_geom_qm9.py Corrects docstring and bumps dataset version to 1.1.
modelforge-curate/​.../​sourcedataset.py Uses OrderedDict for records to preserve order.
notebooks/​evaluating_models.ipynb New example demonstrating the evaluation helpers.
modelforge-curate/​.../​examples/​*.ipynb Notebook metadata version bumps only.

Review effort: Balanced

Comment thread modelforge/dataset/dataset.py Resolved
Comment thread modelforge/dataset/dataset.py Resolved
Comment thread modelforge/potential/potential.py Resolved
Comment thread modelforge/potential/potential.py Resolved
Comment thread modelforge/potential/potential.py Resolved
Comment thread modelforge/tests/test_dataset.py Resolved
@chrisiacovella
chrisiacovella merged commit d7671e6 into choderalab:main Oct 20, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants