Usage Guide¶
Command Line Interface¶
The quickest way to exercise the validation checks is through the
kim-test-pm-lite command line interface. Provide the path to a KIM model
and the species you want to validate:
kim-test-pm-lite /path/to/kim-model Ar
Use -v to surface informational messages, -vv to enable debug logging
with scalar results, and -vvv to additionally print array data such as
positions and forces. In verbose modes the intermediate quantities reported by
the checks are logged using Rich formatting:
kim-test-pm-lite ./models/Sim_LJ_G Ar Ne -v
Specific checks can be requested with --checks. Multiple checks may be
provided by repeating the option:
kim-test-pm-lite ./models/Sim_LJ_G --checks InversionSymmetryCheck
Add --archive to retain the temporary workspace archive, or specify an
explicit location with --archive-path /path/to/archive.tar.gz.
Use -l logs/run.log (or --log-file) to mirror the log stream to a file
while still rendering via Rich in the terminal.
CLI Option Reference¶
Option |
Description |
Default |
|---|---|---|
|
Path to the model directory or archive (positional argument). |
Required |
|
One or more chemical symbols to exercise (positional argument). |
Optional |
|
Select a subset of checks; repeat the option to list multiple entries. |
All checks |
|
Reserved placeholder for supplying an extxyz structure. |
|
|
Override the model name detected from the workspace. |
Auto-detected |
|
Mirror the log output to the specified file. |
|
|
Archive the temporary workspace to a timestamped |
Disabled |
|
Archive to the given path (implies |
|
|
Increase verbosity: |
|
|
Seed NumPy’s global RNG so randomised checks are reproducible. |
|
|
Apply canned tolerance profiles ( |
|
|
Run property tests such as |
|
Property Tests¶
Property tests emit quantitative results rather than pass/fail verdicts. They
are only executed when listed via -t/--tests; if you provide tests, only
the explicitly requested checks are executed alongside them.
Currently available:
LatticeConstantTest– Builds (or uses a provided) bulk structure, relaxes the cell, and reports the equilibrium lattice constant and total energy.
Example invocations:
# Use an auto-generated bulk structure
kim-test-pm-lite ./models/Sim_LJ_G Ar --tests LatticeConstantTest -vv
# Reuse an external unit cell
kim-test-pm-lite ./models/Sim_LJ_G Ar --tests LatticeConstantTest \
--structure path/to/unitcell.extxyz
The MemoryLeakCheck
relies on Valgrind being available on PATH.
When invoked, it writes the complete Valgrind report to
MemoryLeakTest/valgrind_output.txt for inspection. The check is not
part of the default suite; enable it explicitly with --checks
MemoryLeakCheck when you need the additional coverage.
Example CLI Session¶
Running selected checks¶
$ kim-test-pm-lite ./TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000 -vvv --seed 123 -c NumericalForcesCheck --tolerances light
[12:24:04] DEBUG NumPy RNG seeded with 123 cli.py:132
[12:24:04] DEBUG Using tolerance profile 'light' (tol=0.001) cli.py:136
[12:24:04] DEBUG Using requested checks: NumericalForcesCheck cli.py:147
[12:24:04] DEBUG Starting test run for model at cli.py:149
TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000
[12:24:04] DEBUG Instantiating checks with tolerance 1.000e-03 test_runner.py:50
[12:24:04] DEBUG Registered check NumericalForces test_runner.py:61
[12:24:04] DEBUG Entered temporary workspace at /tmp/kim_test_pm_lite_nkru988t file_system.py:57
[12:24:04] DEBUG Temporary workspace initialised at test_runner.py:81
/tmp/kim_test_pm_lite_nkru988t
[12:24:04] DEBUG Model copied to file_system.py:74
/tmp/kim_test_pm_lite_nkru988t/TorchML_20231203_MACE_MP_0_128_
L1_EP199__MO_000000000000_000
[12:24:04] DEBUG Model copied to workspace. test_runner.py:83
Found local item named: TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000.
In source directory: /tmp/kim_test_pm_lite_nkru988t/TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000.
(If you are trying to install an item from openkim.org
rerun this command from a different working directory,
or rename the source directory mentioned above.)
Found installed driver... TorchML__MD_173118614730_001
[100%] Built target TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000
Install the project...
-- Install configuration: "Debug"
-- Installing: /tmp/kim_test_pm_lite_nkru988t/TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000/libkim-api-portable-model.so
-- Set runtime path of "/tmp/kim_test_pm_lite_nkru988t/TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000/libkim-api-portable-model.so" to ""
Success!
[12:24:15] DEBUG Model installation invoked via kim-api-collections-management. file_system.py:90
[12:24:15] DEBUG Model installation completed. test_runner.py:85
[12:24:15] INFO Running model: test_runner.py:90
TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000
[12:24:17] DEBUG Active species: H, He, Li, Be, B, C, N, O, F, Ne, Na, Mg, Al, test_runner.py:103
Si, P, S, Cl, Ar, K, Ca, Sc, Ti, V, Cr, Mn, Fe, Co, Ni, Cu,
Zn, Ga, Ge, As, Se, Br, Kr, Rb, Sr, Y, Zr, Nb, Mo, Tc, Ru,
Rh, Pd, Ag, Cd, In, Sn, Sb, Te, I, Xe, Cs, Ba, La, Ce, Pr,
Nd, Pm, Sm, Eu, Gd, Tb, Dy, Ho, Er, Tm, Yb, Lu, Hf, Ta, W,
Re, Os, Ir, Pt, Au, Hg, Tl, Pb, Bi, Ac, Th, Pa, U, Np, Pu
[12:24:17] INFO Running check NumericalForces test_runner.py:108
[12:24:17] DEBUG Generated numerical forces structure saved to numerical_forces.py:52
NumericalForcesTest/TorchML_20231203_MACE_MP_0_128_L1_EP1
99__MO_000000000000_000_Base.xyz
[12:24:17] DEBUG Base configuration positions: base_check.py:101
Symbol Component 1 Component 2 Component 3
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Ho 5.884585 4.108978 2.885591
Ar 2.352705 2.059068 4.374298
Ac 2.631433 0.358067 2.388266
Ce 4.427972 1.094950 1.052711
[12:24:17] DEBUG Generated 1 configurations. test_runner.py:110
[12:24:17] DEBUG Analytical forces: base_check.py:101
Symbol Component 1 Component 2 Component 3
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Ho -0.333525 -0.463390 -0.145646
Ar -0.012726 0.095187 0.107238
Ac -4.602713 -1.807174 3.543514
Ce 4.948963 2.175377 -3.505105
[12:24:29] DEBUG Numerical forces (finite difference): base_check.py:101
Symbol Component 1 Component 2 Component 3
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Ho -0.333525 -0.463390 -0.145646
Ar -0.012726 0.095187 0.107238
Ac -4.602713 -1.807174 3.543514
Ce 4.948963 2.175377 -3.505105
[12:24:29] DEBUG Force error log10: base_check.py:101
Symbol Component 1 Component 2 Component 3
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Ho -9.575502 -9.636217 -10.391448
Ar -10.017699 -10.281395 -9.702339
Ac -7.057846 -8.026554 -7.675128
Ce -7.043665 -7.989765 -7.689366
[12:24:29] DEBUG Numerical forces check | max error: 9.043e-08 | rms numerical_forces.py:125
error: 3.753e-08 | status: pass
[12:24:29] DEBUG Raw result payload: {'analytical_forces': test_runner.py:113
[[-0.33352469891292347, -0.463390219227642,
-0.1456458745690321], [-0.012725724232692913,
0.09518723228198898, 0.107237529586577], [-4.602712616024824,
-1.8071736373059473, 3.543513619758104], [4.94896303917044,
2.1753766242516006, -3.505105274775649]], 'numerical_forces':
[[-0.3335246991786889, -0.4633902189965511,
-0.14564587460963452], [-0.012725724136686267,
0.09518723222967651, 0.10723752978503143],
[-4.602712703554168, -1.8071736278990613, 3.543513640886786],
[4.948963129605133, 2.1753766140131248,
-3.5051052952228545]], 'force_errors':
[[2.657654096793749e-10, 2.31090924174282e-10,
4.060241032277645e-11], [9.60066453420394e-11,
5.231247379722248e-11, 1.984544334421301e-10],
[8.752934377298516e-08, 9.406885936869003e-09,
2.1128682359261575e-08], [9.043469262337567e-08,
1.0238475844914774e-08, 2.044720570637537e-08]], 'max_error':
9.043469262337567e-08, 'rms_error': 3.752534921241504e-08,
'finite_diff_step': 0.0001, 'status': 'pass',
'failure_reasons': []}
[12:24:29] DEBUG Cleaned up temporary workspace at file_system.py:63
/tmp/kim_test_pm_lite_nkru988t
[12:24:29] INFO Summary: 1 total | 1 passed | 0 failed | 0 errors test_runner.py:150
KIM-Test-PM-Lite Test Results
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Metric ┃ Value ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Model │ TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000 │
│ Max Cutoff │ 6.000000 │
│ Status │ pass │
│ Total │ 1 │
│ Passed │ 1 │
│ Failed │ 0 │
│ Errors │ 0 │
│ Species │ H, He, Li, Be, B, C, N, O, F, Ne, Na, Mg, Al, Si, P, S, Cl, Ar, K, Ca, Sc, Ti, V, │
│ │ Cr, Mn, Fe, Co, Ni, Cu, Zn, Ga, Ge, As, Se, Br, Kr, Rb, Sr, Y, Zr, Nb, Mo, Tc, Ru, │
│ │ Rh, Pd, Ag, Cd, In, Sn, Sb, Te, I, Xe, Cs, Ba, La, Ce, Pr, Nd, Pm, Sm, Eu, Gd, Tb, │
│ │ Dy, Ho, Er, Tm, Yb, Lu, Hf, Ta, W, Re, Os, Ir, Pt, Au, Hg, Tl, Pb, Bi, Ac, Th, Pa, │
│ │ U, Np, Pu │
└────────────┴─────────────────────────────────────────────────────────────────────────────────────┘
Check Status
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Check ┃ Status ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ NumericalForces │ pass │
└──────────────────────────────────────────────────────────────────┴───────────────────────────────┘
Running tests¶
Generate a structure to run test on (optional, can you default element list too).
python -c "from ase.build import bulk; from ase.io import write; write('ca.xyz', bulk('Ca', cubic=True))"
$ kim-test-pm-lite ./TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000 -vvv --tests LatticeConstantTest --structure ca.xyz
[14:36:09] DEBUG Using tolerance profile 'intermediate' (tol=1e-06) cli.py:213
[14:36:09] DEBUG Using requested tests: LatticeConstantTest cli.py:221
[14:36:09] DEBUG Checks skipped because tests were explicitly requested cli.py:228
[14:36:09] DEBUG Starting test run for model at cli.py:233
TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000
[14:36:09] INFO Species: Ca cli.py:235
[14:36:09] INFO Using 1 provided structure(s) for tests cli.py:237
[14:36:09] DEBUG Instantiating checks with tolerance 1.000e-06 test_runner.py:60
[14:36:09] DEBUG Registered test LatticeConstantTest test_runner.py:83
[14:36:09] DEBUG Entered temporary workspace at /tmp/kim_test_pm_lite_myhdw0p6 file_system.py:57
[14:36:09] DEBUG Temporary workspace initialised at test_runner.py:104
/tmp/kim_test_pm_lite_myhdw0p6
[14:36:09] DEBUG Model copied to file_system.py:74
/tmp/kim_test_pm_lite_myhdw0p6/TorchML_20231203_MACE_MP_0_128_
L1_EP199__MO_000000000000_000
[14:36:09] DEBUG Model copied to workspace. test_runner.py:106
Found local item named: TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000.
In source directory: /tmp/kim_test_pm_lite_myhdw0p6/TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000.
(If you are trying to install an item from openkim.org
rerun this command from a different working directory,
or rename the source directory mentioned above.)
Found installed driver... TorchML__MD_173118614730_001
[100%] Built target TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000
Install the project...
-- Install configuration: "Debug"
-- Installing: /tmp/kim_test_pm_lite_myhdw0p6/TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000/libkim-api-portable-model.so
-- Set runtime path of "/tmp/kim_test_pm_lite_myhdw0p6/TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000/libkim-api-portable-model.so" to ""
Success!
[14:36:20] DEBUG Model installation invoked via kim-api-collections-management. file_system.py:90
[14:36:20] DEBUG Model installation completed. test_runner.py:108
[14:36:20] INFO Running model: test_runner.py:113
TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000
[14:36:23] DEBUG Active species: Ca test_runner.py:126
[14:36:23] INFO Running test LatticeConstantTest test_runner.py:171
[14:37:00] DEBUG Relaxed positions: base_test.py:69
Symbol Component 1 Component 2 Component 3
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Ca -0.000000 -0.000000 -0.000000
Ca -0.000000 2.748808 2.748808
Ca 2.748808 -0.000000 2.748808
Ca 2.748808 2.748808 -0.000000
[14:37:00] DEBUG Optimization history (energy ascending): lattice_constant.py:114
[14:37:00] DEBUG a = 5.497617 Å | E = -8.038650 eV lattice_constant.py:116
[14:37:00] DEBUG a = 5.497617 Å | E = -8.038650 eV lattice_constant.py:116
[14:37:00] DEBUG a = 5.491106 Å | E = -8.038535 eV lattice_constant.py:116
[14:37:00] DEBUG a = 5.575511 Å | E = -8.020207 eV lattice_constant.py:116
[14:37:00] DEBUG a = 5.580000 Å | E = -8.018087 eV lattice_constant.py:116
[14:37:00] DEBUG a = 5.580000 Å | E = -8.018087 eV lattice_constant.py:116
[14:37:00] DEBUG Raw test payload: {'status': 'ok', 'lattice_constant': test_runner.py:179
5.4976165733519835, 'energy': -8.03865030704319,
'iterations': 3, 'structure_type': 'fcc', 'history':
[{'energy': -8.018087159822736, 'lattice_constant':
5.580000000000001}, {'energy': -8.018087159822736,
'lattice_constant': 5.580000000000001}, {'energy':
-8.020207125673577, 'lattice_constant': 5.575510733855663},
{'energy': -8.038534555116527, 'lattice_constant':
5.491106081851366}, {'energy': -8.03865030704319,
'lattice_constant': 5.4976165733519835}, {'energy':
-8.03865030704319, 'lattice_constant': 5.4976165733519835}]}
[14:37:00] DEBUG Cleaned up temporary workspace at file_system.py:63
/tmp/kim_test_pm_lite_myhdw0p6
[14:37:00] INFO Summary: 0 total | 0 passed | 0 failed | 0 errors test_runner.py:201
KIM-Test-PM-Lite Test Results
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Metric ┃ Value ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Model │ TorchML_20231203_MACE_MP_0_128_L1_EP199__MO_000000000000_000 │
│ Max Cutoff │ 6.000000 │
│ Status │ pass │
│ Total │ 0 │
│ Passed │ 0 │
│ Failed │ 0 │
│ Errors │ 0 │
│ Species │ Ca │
└────────────────┴─────────────────────────────────────────────────────────────────────────────────┘
Test Results
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test ┃ Result ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ LatticeConstantTest │ a=5.4976 Å (fcc); E=-8.038650 eV; steps=3 │
└────────────────────────────────┴─────────────────────────────────────────────────────────────────┘
Programmatic Interface¶
Developers may run the checks from Python using
kim_test_pm_lite.core.test_runner.run_tests. The function accepts a path to the
model and returns the same result structure used by the CLI:
from pathlib import Path
from kim_test_pm_lite.core.test_runner import run_tests
results = run_tests(
Path("./models/Sim_LJ_G"),
checks_to_run_str=["InversionSymmetryCheck"],
species_list=["Ar"],
)
summary = results["summary"]
print(f"Overall status: {summary['status']}")
Reporting Utilities¶
The kim_test_pm_lite.core.result_reporter.ResultReporter helper renders
results in Rich tables and exports data to JSON, Markdown, or EDN formats.
Construct the reporter with a Rich console to reuse your application’s output
styling:
from rich.console import Console
from kim_test_pm_lite.core.result_reporter import ResultReporter
reporter = ResultReporter(console=Console())
reporter.print_summary(results)