Repository navigation
Revision/dataset to enable batch post processing - #447
Conversation
…le and a function to take in a potential instance and batch, to perform evaluation of e.g., the entire test subset.
There was a problem hiding this comment.
🟡 Changes recommended
evaluate_batch maps batched results to the wrong records for subset="train" because the train dataloader shuffles by default while the code assumes fixed indices order.
6 open issues
Typo in comment: "reocrds" should be "records". · New The return type annotation is incorrect:__getitem__returns aBatchData, not atorch.Tensor,… · New Typo in docstring: "torch_datset" should be "torch_dataset". · New Forsubset="train",dm.train_dataloader()is called with its defaultshuffle=True, so the… · Newper_atom_properties.append("atomic_numbers")runs on every batch iteration instead of only once.… · New Leftover debugprintin a test. This PR removes similar stray prints elsewhere (e.g. in… · New
What changed in this PR
This PR adds tooling to make it easier to examine a trained model after the fact. It propagates each record's name from the hdf5 file through the TorchDataset (exposed via a new _get_datapoint), adds initialize_datamodule_from_config to rebuild a DataModule from a training .toml, and adds evaluate_batch to run batched inference over a dataset subset while aggregating name/output/metadata per original index. It also hardens hdf5 processing against records with zero valid conformers and corrects the GEOM QM9 energy-level-of-theory docstrings (GFN2-xTB, not r2SCAN-3c DFT) per issue #445.
Changes:
- Track and expose record
namesthrough the dataset; add_get_datapoint,initialize_datamodule_from_config, andevaluate_batchhelpers. - Guard
_from_hdf5so records with no valid configurations are skipped, and align thenameslist accordingly; replace strayprints withlog.debug. - Fix GEOM QM9 docstrings to reflect xtb energies, bump curation version to 1.1, and add an evaluation example notebook/tests.
| File | Description |
|---|---|
| modelforge/potential/potential.py | Adds evaluate_batch for batched subset evaluation; has a train-subset ordering bug and a redundant per-batch append. |
| modelforge/dataset/dataset.py | Adds name tracking, _get_datapoint, initialize_datamodule_from_config, zero-conformer guard, and cache-validation update. |
| modelforge/tests/test_potentials.py | New test_batch_evaluation covering only the test subset. |
| modelforge/tests/test_dataset.py | Tests name access via _get_datapoint; includes a leftover debug print. |
| modelforge-curate/.../geom_qm9_curation.py | Corrects energy level-of-theory docstring to xtb. |
| modelforge-curate/.../scripts/curate_geom_qm9.py | Corrects docstring and bumps dataset version to 1.1. |
| modelforge-curate/.../sourcedataset.py | Uses OrderedDict for records to preserve order. |
| notebooks/evaluating_models.ipynb | New example demonstrating the evaluation helpers. |
| modelforge-curate/.../examples/*.ipynb | Notebook metadata version bumps only. |
Review effort: Balanced

Pull Request Summary
This primarily focuses on making it easier to examine a trained model after the fact.
Key changes
Notable points that this PR has either accomplished or will accomplish.
nameof the record; this quantity is available within the TorchDataset. To get the name associated with an individual index in the torch_dataset, we can call the_get_datapointfunction that will return the name and then the normal output from the iterator.DataModuledirectly from a config .toml file, making it easier to exactly replicate splitting and any filtering that was done during training.DataModuleinstance). This will perform some aggregation as well, adding in the name, model output, and associated metadata.Associated Issue(s)
Pull Request Checklist