Internals API#

The modules behind the public API, for contributors and gateway authors. None of this is the public API: use powderline.run and the powderline command to run recipes.

Command line#

The powderline command (reference). The pixi task kicker runs the same main. The module imports no engine.

powderline.kicker.main(argv: list[str] | None = None) → int[source]#

Entry point of the powderline command; returns the exit status.

powderline.kicker.is_template_file(recipe_dict: dict, input_path: Path) → tuple[bool, str | None][source]#

Detect a recipe file that is a template and must not be run directly (CLI-only guard).

Two-level detection:

  1. the path contains “template” (case-insensitive);

  2. the payload is missing, or lacks the fields the workflow needs: a *.rietveld recipe needs xrd_data, instrument, phases and refinement_controls; a *.spf one single_peaks instead of phases; any other schema name is a template only when all of xrd_data, instrument and refinement_controls are absent.

Returns:

(is_template, reason); (False, None) for a runnable recipe.

Examples

>>> is_template_file({}, Path("example_template/input.json"))
(True, "filename or path contains 'template'")
>>> is_template_file({"schema_name": "gsasii.rietveld", "payload": {"xrd_data": {...}, ...}}, Path("in.json"))
(False, None)

Doctor#

powderline --doctor’s report. report() returns a dict with the keys powderline, python, platform, gateways, server and problems. Each entry of gateways has workflows, engine_version_spec, engine_installed, engine_status and problem. format_report() renders the dict, and main() prints it and returns 1 when problems is not empty. The module imports no engine; it imports the gateway modules when the report runs. The stale-install case it detects is KI-13.

Gateway registry#

Gateways are discovered from the powderline.gateways entry-point group. These are the hooks an out-of-tree engine uses.

class powderline.registry.Gateway(name: str, capabilities: Callable[[], Dict[str, Any]], validate: Callable[[...], Any], run: Callable[[...], dict])[source]#

A gateway’s contract.

  • name: registry name ("gsasii", "topas", "easydiffraction").

  • capabilities(): dict with at least name, workflows and engine_version_spec (a PEP 440 specifier string).

  • validate(recipe, *, verbose=False): schema layer, engine-free. Takes a recipe dict or an already-validated model, raises on an invalid recipe, returns the validated model. The same explicit keywords on every gateway; new options are added to the contract deliberately.

  • run(recipe, output_dir, **opts): runtime layer; imports the engine lazily; returns the standardized result dict. With validate_only=True it returns the validate_only_summary dict (the same keys on every gateway).

name: str#
capabilities: Callable[[], Dict[str, Any]]#
validate: Callable[[...], Any]#
run: Callable[[...], dict]#
__init__(name: str, capabilities: Callable[[], Dict[str, Any]], validate: Callable[[...], Any], run: Callable[[...], dict]) → None#
powderline.registry.available() → list[str][source]#

Sorted names of every registered gateway (explicit and entry-point).

powderline.registry.get(name: str) → Gateway[source]#

Return the gateway registered as name, loading its factory on first use.

Raises:

GatewayNotInstalledError – no gateway of that name is registered at all. When the entry-point group is empty the message says the install metadata is stale (the in-repo gateways are always declared).

powderline.registry.register_gateway(name: str, factory: Callable[[], Gateway]) → None[source]#

Register a gateway factory explicitly (tests/dev escape hatch).

Takes precedence over an entry point of the same name.

powderline.registry.check_engine_version(*, gateway: str, engine: str, installed, spec: str, allow_unsupported: bool = False) → None[source]#

Raise EngineVersionError unless installed satisfies spec.

A hard error. allow_unsupported=True is the explicit test-only bypass for compatibility testing of candidate engine versions; gateways expose it as allow_unsupported_engine_version on their run().

Gateway run functions#

Each gateway’s run is what powderline.run forwards to, after it has validated the recipe and chosen the gateway. Their keyword options are exactly the engine options powderline.run accepts for that engine.

Warning

Run recipes with powderline.run or the powderline command, not with these. Calling a gateway’s run directly skips what powderline.run does on every engine: PowderLine’s fixed-name outputs are not cleared first, and a failed run writes no failure.json.

powderline.gateways.gsasii.gateway.run(recipe, output_dir, *, verbose: bool = False, validate_only: bool = False, execution_mode: Literal['auto', 'server', 'subprocess'] = 'auto', server_timeout: float | None = None, allow_unsupported_engine_version: bool = False) → dict[source]#

Execute a GSAS-II refinement from an in-memory recipe.

The full pipeline: schema validation, the engine version check, output directory creation, and refinement execution with automatic backend selection. Reached through powderline.run() for a gsasii.* recipe, which forwards execution_mode.

Note: the CLI’s template-file guard is not applied here. When calling programmatically the caller is responsible for ensuring the recipe contains real data in payload.xrd_data.

Parameters:

  • recipe – a recipe dict or a validated model. XRD data must already be present in payload.xrd_data.

  • output_dir – directory where output files will be written (dummy.gpx, dummy.lst, CSV reports, fit_profile.txt). Created if it does not exist.

  • verbose – if True, print detailed progress to stdout.

  • validate_only – if True, validate the recipe and return a summary dict without running a refinement. No output files are written and the engine version is not checked.

  • execution_mode –

    the backend used to execute the refinement:

    'auto' (default)

    Try the persistent GSAS-II server first (auto-starting it if needed); fall back to in-process execution if unavailable.

    'server'

    Use the server only. Fails with an error result if the server cannot be started.

    'subprocess'

    Skip the server entirely; always run in this process. Suitable for batch/HPC workflows where each refinement is isolated.

    Only the calling user’s own server is used (per-user state directory, signed requests and responses; v0.1.2). If the POWDERLINE_NO_SERVER environment variable is truthy (1/true/yes/on), 'auto' runs in-process without starting a server and 'server' returns an error result.

  • server_timeout – seconds to wait for the server’s answer to a submitted job ('auto' / 'server'). None (default) or 0: no limit, as for an in-process run. Past the limit the run raises ExecutionBackendError timeout and never falls back in-process: the job is still running on the server (one job at a time), so a second run would double the work.

  • allow_unsupported_engine_version – test-only bypass of the engine version check. Not reachable through powderline.run().

Returns:

Normal refinement (validate_only=False) — the ok envelope (powderline.results); a failed run raises, see below:

  • status ("ok"), converged (bool, or None for a simulation and for single peak fitting), envelope_version ("1"); there is no success key

  • run_id (str) — UUID4 string uniquely identifying this run

  • rwp (float | None) — final weighted-profile R factor

  • elapsed_time (float) — wall-clock seconds

  • method (str) — 'server' or 'subprocess'

  • fit_profile (pd.DataFrame) — columns: two_theta, y_obs, y_weights, y_calc, y_diff, y_bkg, q_values, d_spacings

  • unit_cell_data (dict) — {phase_name: pd.DataFrame} with columns: parameter, value, esd

  • peak_list_data (dict) — {phase_name: pd.DataFrame} with columns: h, k, l, multiplicity, d_spacing, 2theta, …

  • refined_parameters (pd.DataFrame) — 9 columns: parameter_name, descriptive_name, phase_name, phase_idx, atom_name, atom_idx, value, esd, category (empty DataFrame with all 9 columns when nothing was refined)

  • spf_peaks (pd.DataFrame) — SPF peak report (empty for non-SPF runs)

  • spf_convergence_diagnostics (pd.DataFrame) — per-peak diagnostics for peaks with convergence issues (empty when all converged or for non-SPF runs)

  • peak_fit (dict | None) — SPF only: {"parameters": [{"name", "value", "esd"}]}, every parameter GSAS-II’s peak fit varied by its own name (Back;0, U, pos3 …) with its ESD; the source of the output recipe for a peak fit

  • output_files (list[str]) — paths written to output_dir

  • r_exp, gof, chi2_red, simulation_mode, engine_details and the structured warnings list

Validate-only (validate_only=True) — the summary every gateway returns (powderline.registry.validate_only_summary): no refinement executed, no output files written, no run_id; engine holds refinement_cycles.

Raises:

  • pydantic.ValidationError – the recipe fails schema validation.

  • ValueError – an invalid execution_mode or server_timeout.

  • EngineNotAvailableError – GSAS-II is not importable (not for validate_only).

  • EngineVersionError – the installed GSAS-II is outside ENGINE_VERSION_SPEC.

  • EngineExecutionError – the engine ran and failed (.reason: diverged, unusable_site, peak_fit_failed, engine_crashed; .result is the failed envelope listing the engine’s own files written so far, which are kept).

  • ExecutionBackendError – the engine did not run (.reason: server_unavailable, server_disabled, communication_failed, timeout, backend_error).

  • OSError – output_dir cannot be created.

powderline.gateways.topas.gateway.run(recipe, output_dir, *, verbose: bool = False, validate_only: bool = False, topas_dir=None, topas_version=None, topas_timeout=None, allow_unsupported_engine_version: bool = False) → dict[source]#

Run a recipe; see native_run.run_native() for the result dict.

topas_dir / topas_version: the TOPAS installation and its version (else topas.dir / topas.version in .powderline_config.yaml). topas_timeout: the limit on each tc.exe run in seconds (else topas.timeout, else 3600; 0 = no limit); past it the run fails with ExecutionBackendError timeout.

allow_unsupported_engine_version is the test-only bypass of the engine version check; the check is skipped for validate_only and when no tc.exe is found (generate-only).

powderline.gateways.easydiffraction.gateway.run(recipe, output_dir, *, verbose: bool = False, validate_only: bool = False, allow_unsupported_engine_version: bool = False) → dict[source]#

Run a recipe; see native_run.run_native() for the result dict.

allow_unsupported_engine_version is the test-only bypass of the engine version check; the check is skipped for validate_only.

GSAS-II gateway helpers#

These functions belong to the frozen GSAS-II extraction and setup code (powderline.gateways.gsasii.extractors and .setters).

Output generation#

powderline.gateways.gsasii.extractors.export_refined_parameters_csv(param_dict: Dict[str, Dict[str, Any]], output_file: Path, proj: Any = None, include_category: bool = True) → DataFrame | None[source]#

Export refined parameters to CSV file and return the DataFrame.

Parameters:

  • param_dict – Dictionary from extract_refined_params_* functions

  • output_file – Path to output CSV file

  • proj – GSAS-II project object (optional, needed for phase/atom names)

  • include_category – If True, add ‘category’ and ‘descriptive_name’ columns (default: True)

Returns:

DataFrame with the exported parameters, or None if param_dict is empty.

Output CSV / DataFrame columns:

  • parameter_name: GSAS-II internal parameter name

  • descriptive_name: Human-readable parameter description (if include_category=True)

  • phase_name: Name of associated phase (None if not phase-specific)

  • phase_idx: Index of associated phase (None if not phase-specific)

  • atom_name: Label of associated atom (None if not atom-specific)

  • atom_idx: Index of associated atom (None if not atom-specific)

  • value: Refined value

  • esd: Estimated standard deviation (None for fixed parameters)

  • category: Parameter category (instrument, background, cell, etc.) if include_category=True

powderline.gateways.gsasii.extractors.calculate_cell_esds_from_A_matrix(phase_idx: int, proj: Any, phase_name: str) → List[float | None][source]#

Calculate unit cell parameter ESDs using GSAS-II’s reciprocal metric tensor conversion.

GSAS-II refines reciprocal metric tensor components (A-matrix: A11, A22, A33, A12, A13, A23) and this function converts those ESDs to direct lattice parameter ESDs (a, b, c, α, β, γ).

Parameters:

  • phase_idx – Phase index (0-based) for parameter naming (e.g., “0::A0”)

  • proj – GSAS-II project object after refinement

  • phase_name – Name of the phase

Returns:

The 7 ESDs, in order [esd_a, esd_b, esd_c, esd_alpha, esd_beta, esd_gamma, esd_volume].

Return type:

list[float | None]

Raises:

RuntimeError – If covariance data is unavailable or the ESD calculation fails.

Note

The A-matrix parameters (A0-A5) in GSAS-II correspond to: A0=A11, A1=A22, A2=A33, A3=A12, A4=A13, A5=A23 (reciprocal metric tensor) NOT direct cell parameters a, b, c, α, β, γ.

powderline.gateways.gsasii.extractors.extract_refined_params_from_project(proj: Any, verbose: bool = False) → Dict[str, Dict[str, Any]][source]#

Extract all refined parameters with their values and ESDs from proj.data.

Uses the covariance data stored in proj.data[‘Covariance’][‘data’] after refinement. This includes both independent and dependent parameters (via ComputeDepESD).

Parameters:

  • proj – GSAS-II project object after refinement

  • verbose – If True, print progress messages.

Returns:

Mapping of parameter names to {"value": float, "esd": float}. Returns an empty dict if no covariance data is available.

Return type:

dict

Raises:

RuntimeError – If ComputeDepESD fails (dependent parameter ESDs cannot be calculated).

Example

>>> params = extract_refined_params_from_project(proj)
>>> params[":0:U"]
{"value": 47.406, "esd": 8.314}

Note

ESDs are read from GSAS-II’s covariance matrix (proj.data['Covariance']). Unit-cell ESDs are converted from the reciprocal metric tensor (the A-matrix) to the direct lattice parameters by calculate_cell_esds_from_A_matrix().

Post-refinement extraction helpers (private)#

The four private _extract_* helpers hold the post-refinement extraction and file-writing logic.

powderline.gateways.gsasii.extractors._extract_fit_profile(hist: Any, output_dir: Path) → dict[source]#

Extract fit profile arrays from histogram and save fit_profile.txt.

Parameters:

  • hist – GSAS-II histogram object after refinement.

  • output_dir – Directory to write fit_profile.txt.

Returns:

Column-oriented data (JSON-serializable) with keys two_theta, y_obs, y_weights, y_calc, y_diff, y_bkg, q_values, d_spacings.

Return type:

dict

powderline.gateways.gsasii.extractors._extract_spf_peak_report(proj: Any, hist: Any, recipe: GsasiiRietveldRecipe | GsasiiSpfRecipe, output_dir: Path, verbose: bool) → tuple[dict, dict][source]#

Extract single peak fitting results and save report files.

Called only for gsasii.spf runs where recipe.payload.single_peaks is set. Returns two column-oriented dicts (JSON-serializable) that are normalised to DataFrames by run().

Also writes: - single_peaks_report.txt — per-peak widths and convergence status - peak_convergence_diagnostics.txt — only when peaks have issues

Parameters:

  • proj – GSAS-II project object after refinement.

  • hist – GSAS-II histogram object.

  • recipe – Validated gsasii.spf recipe model.

  • output_dir – Directory to write report files.

  • verbose – If True, print convergence warnings to stdout.

Returns:

(spf_peaks_data, spf_diagnostics_data) where each is a column-oriented dict, or ({}, {}) when single peaks are not used.

powderline.gateways.gsasii.extractors._extract_phase_reports(proj: Any, hist: Any, recipe: GsasiiRietveldRecipe | GsasiiSpfRecipe, param_dict: dict, output_dir: Path) → tuple[dict, dict][source]#

Extract unit cell and peak list reports for all phases.

Writes {phase}_unit_cell_report.csv and {phase}_peak_list_report.csv for each phase.

Parameters:

  • proj – GSAS-II project object after refinement.

  • hist – GSAS-II histogram object.

  • recipe – Validated gsasii.rietveld / gsasii.spf recipe model (used to check if phases exist).

  • param_dict – Refined parameter dict from extract_refined_params_from_project() (needed for cell ESDs).

  • output_dir – Directory to write CSV files.

Returns:

(unit_cell_data, peak_list_data) — each is a {phase_name: list-of-records} dict (JSON-serializable).

powderline.gateways.gsasii.extractors._extract_refined_parameters(param_dict: dict, output_dir: Path, proj: Any, verbose: bool) → list[source]#

Export refined parameters to CSV and return as list-of-records.

Parameters:

  • param_dict – From extract_refined_params_from_project().

  • output_dir – Directory to write refined_parameters.csv.

  • proj – GSAS-II project object (for phase/atom name mappings).

  • verbose – If True, print export status to stdout.

Returns:

List of dicts (records) with the 9-column schema. Returns [] when param_dict is empty (simulation mode, SPF, etc.).

Background functions#

powderline.gateways.gsasii.setters.set_chebyshev_background(proj: Any, hist: Any, chebyshev_dict: dict, print_info: bool = False) → None[source]#

Set the Chebyshev background for a given histogram.

Chebyshev polynomials provide smooth curved backgrounds. Coefficients start at 0th order (constant term) and increase: [c0, c1, c2, …] represents c0 + c1*T1(x) + c2*T2(x) + … where Tn are Chebyshev polynomials.

The function manipulates proj.data[hist.name][‘Background’][0] which has structure: [background_type, refine_flag, num_coefficients, c0, c1, c2, …]

Parameters:

  • proj – GSAS-II project object containing the histogram

  • hist – Histogram object to set the background for

  • chebyshev_dict – Dictionary containing Chebyshev background parameters: - num_coefficients (int): Number of Chebyshev coefficients - coefficients (list[float]): Coefficient values [c0, c1, c2, …] - refine_flag (bool): Whether to refine background during fitting

  • print_info – If True, print background configuration to stdout

Returns:

None

Raises:

ValueError – If number of coefficients doesn’t match list length or required background entries are missing

Examples

>>> chebyshev_dict = {
...     'num_coefficients': 3,
...     'coefficients': [100.0, -50.0, 10.0],
...     'refine_flag': True
... }
>>> set_chebyshev_background(proj, hist, chebyshev_dict)
powderline.gateways.gsasii.setters.set_single_peak_background(proj: Any, hist: Any, bkg_single_peaks_dict: dict, print_info: bool = False) → None[source]#

Set single peak background for histogram using pseudo-Voigt profiles.

Single peaks are useful for modeling known impurity peaks or other non-background features that shouldn’t be included in the main phase refinement. Each peak is described by a pseudo-Voigt profile (weighted sum of Gaussian and Lorentzian).

Peak profile: I(2θ) = intensity * [η*L(2θ) + (1-η)*G(2θ)] where: - G(2θ) is Gaussian with width sigma - L(2θ) is Lorentzian with width gamma - η (eta) is mixing parameter (0=pure Gaussian, 1=pure Lorentzian)

Parameters:

  • proj – GSAS-II project object

  • hist – Histogram object to set single peaks for

  • bkg_single_peaks_dict – Dictionary with keys: - positions: List of [[2θ, refine], …] for peak positions - intensities: List of [[I, refine], …] for peak heights - pv_gaussian_sigma_sq: List of [[σ², refine], …] Gaussian variances, centideg² - pv_lorentzian_gamma: List of [[γ, refine], …] for Lorentzian widths All lists must have same length (number of peaks)

  • print_info – If True, print peak configuration to stdout

Returns:

None

Raises:

ValueError – If parameter lists have inconsistent lengths

Examples

>>> # Two single peaks at 2θ=35.5° and 42.0°
>>> single_peaks = {
...     'positions': [[35.5, False], [42.0, False]],
...     'intensities': [[50.0, True], [30.0, True]],
...     'pv_gaussian_sigma_sq': [[0.1, False], [0.1, False]],
...     'pv_lorentzian_gamma': [[0.15, False], [0.15, False]]
... }
>>> set_single_peak_background(proj, hist, single_peaks)

Fit range#

powderline.gateways.gsasii.setters.set_fit_range_hist(hist: Any, fit_range: tuple[float | None, float | None], print_info: bool = False) → None[source]#

Set fit range for histogram in GSAS-II project.