Run a series#

Goal: refine one sample’s model against a series of patterns (a temperature ramp, a time series, an in situ experiment) in sequence. Each step starts from the refined values of the step before, and only the data change.

You need:

  • A recipe that fits the first pattern well. Calibrate the instrument first (Calibrate the instrument).

  • Each further pattern as an xrd_data block: tth, Itth, Itth_weights and, optionally, filename. Your own reader builds these blocks from your data files, a database or a live stream.

  • Python, in the engine’s environment.

Every result carries an output recipe: your recipe with the refined values written back. It is a complete recipe of the same engine, ready to run again. A series is therefore a loop over data, not over parameter names:

        flowchart LR
    R0["Recipe<br/>pattern 1"] -->|run| O1["Output recipe 1"]
    O1 -->|"swap in<br/>pattern 2"| R1["Recipe<br/>pattern 2"]
    R1 -->|run| O2["Output recipe 2"]
    O2 -->|"swap in<br/>pattern 3"| R2["…"]

1. Write the loop#

from pathlib import Path
import powderline
from powderline.recipes import with_block

recipe = powderline.load_recipe_asset(Path("series/input.json"))
for i, xrd_data in enumerate(patterns):          # your xrd_data blocks, in order
    recipe = with_block(recipe, "payload.xrd_data", xrd_data)
    result = powderline.run(recipe, Path(f"series/step_{i:03d}"))
    if result["status"] != "ok":                 # "generated": TOPAS input written, TOPAS not run
        print(i, "not refined:", result["status"])
        break
    print(i, result["rwp"], result["converged"])
    if not result["output_recipe_valid"]:
        for w in result["warnings"]:
            if w["code"] == "output_recipe_not_valid":
                print(w["message"])
        break
    recipe = result["output_recipe"]             # the next step starts here

  • with_block returns a copy of the recipe with one block replaced. It does not change the recipe you pass in.

  • Each step writes into its own output directory.

  • The loop never looks at the engine. The same code runs a gsasii.*, topas.* or easydiffraction.* series.

  • Check status first. A failed run raises, but a TOPAS run without a tc.exe returns normally with status: "generated": it wrote TOPAS’s input and refined nothing, so its statistics are None and its output recipe is the input. Without the check, the series would go on unrefined.

2. Stop on an output recipe that is not valid#

result["output_recipe_valid"] says whether the output recipe validates as written. It is false when the engine returned a value the recipe format refuses, for example:

  • an occupancy outside 0 to 1, or a crystallite size of zero or below;

  • a negative Uiso in an easydiffraction recipe;

  • a profile that makes a peak width negative (EB-87, easydiffraction);

  • a split atom that refined until a symmetry image of it is 5e-6 to 2e-3 away (KI-18).

The output_recipe_not_valid warning names each offending value. The output recipe is still returned and written, so you can see what the engine produced. Feeding it to the next step would fail validation. Stop, look at the step, and decide: hold the parameter, or restart from a better model.

3. Read what each step did#

result["parameter_dispositions"], also written as parameter_dispositions.csv, has one row per refinable value: its recipe path, value, esd, disposition and at_limit. The disposition is one of:

Disposition

Meaning

varied

the engine refined it

held_by_user

its flag is false

held_by_symmetry

the space group fixes it

requested_not_varied

its flag is true, but the engine did not vary it

default_applied

the recipe left it out and a documented default was applied

at_limit marks a value that ended on a stated bound or an engine limit.

To follow one value through the series, read it by path at each step:

disp = result["parameter_dispositions"].set_index("path")
uiso_la = disp.loc["payload.phases.LaB6.atoms.La.Uiso", "value"]

The refined cell is also in result["unit_cell_data"][phase], with rows cell_a … cell_gamma and cell_volume.

4. Move only shared blocks between recipes#

with_block, and block to read a copy, move only the blocks every engine shares:

  • payload.xrd_data

  • payload.fit_range

  • payload.phases.<name>

Any other path raises ValueError. An engine’s own blocks (instrument, background, controls) are written in that engine’s terms, so they never move between engines. To change one of them within a series, edit the recipe dict directly.

Try the example#

examples/example_LaB6_temperature_series/ is this loop, complete. run_series.py refines one model in sequence against six simulated LaB6 patterns from 25 to 500 °C, with a known thermal expansion and Poisson noise:

pixi run -e gsasii python examples/example_LaB6_temperature_series/run_series.py --output my_series

The same runner drives the easydiffraction leg:

pixi run -e easydiffraction python examples/example_LaB6_temperature_series/run_series.py --recipe examples/example_LaB6_temperature_series/easydiffraction/input.json --output my_series_ed

Without --output, the runner writes into the example’s own output/ folder, over the committed reference. With GSAS-II it runs in-process; add --use-server to use the GSAS-II server.

Check: my_series/series_report.csv has one row per temperature, with output_recipe_valid true at every step. In the committed GSAS-II reference, every refined cell lies within 1.5 ESD of the programmed value.

Provenance across the chain#

The output recipe keeps your metadata block and adds metadata.powderline_provenance. Its parent_run_id is the run_id of the run that produced it. Each step’s output recipe therefore names the step it came from. Results lists every provenance key.

Next: Run at a beamline to run series unattended.