Comparing engines#

What happens when models are refined against the same data in different engines? This page shows two reference cases, refined in every engine PowderLine drives and by each profile model the engines offer, and explains why the numbers differ. It does not rank the engines.

Important

Rwp compares how well each model fits the same points. It does not say which engine is right. A lower Rwp usually means a richer or a different model, and sometimes an unphysical one.

How the comparison is made#

  • Two cases. LaB6 (the NIST SRM 660c calibrant, one phase) and DRX_33 (a cubic disordered rock-salt cathode phase plus monoclinic Li4MgWO6). They are two patterns measured on the same synchrotron setup (λ = 0.1665 Å, the same 2θ grid), each fitted over 1–15° 2θ.

  • The same points. Every engine fits the same 3767 weighted points, and Rwp, Rexp and GoF are PowderLine’s uniform statistics, computed the same way for every row.

  • Δ vs GSAS-II is the difference from the GSAS-II row, in parts per million and in units of the combined uncertainty σ = √(ESD² + ESD_ref²). For LaB6, two more columns compare with the NIST certificate value.

  • Source. run means the engine was run for this page. golden means the row is rebuilt from a stored reference run: the engine’s committed output files, named after the recipe that produced them. The TOPAS rows are always stored runs, produced on a computer with a TOPAS license.

  • The LaB6 cell is refined in every row here, unlike example_LaB6, which holds it at the certified value. That is why the GSAS-II LaB6 row differs from that example.

The numbers#

LaB6#

Fit window and points: 3767 points (every engine scores the same points).

Engine (version)

Source

Model

Varied

Rwp %

Rexp %

GoF

Warnings

GSAS-II 5.7.9

run

FCJ axial asymmetry (SH/L 0.002); no background peak; size 10 µm / strain 0 defaults, not refined

14

6.000

12.455

0.482

gsasii_broadening_default_applied, gsasii_refinement_message

TOPAS 6 (golden)

golden lab6_cell_rietveld

TCHZ; no axial-asymmetry model; Chebyshev + one pv background peak; CS_L 10000 nm fixed; cell refined

17

7.447

12.450

0.598

topas_parameter_at_limit

easydiffraction 0.21.1

golden lab6_crysfml_tch

crysfml (cwl-thompson-cox-hastings): Thompson-Cox-Hastings + FCJ axial asymmetry (S/L = H/L = 0.001, fixed); no background peak; no sample broadening (start-dependent)

13

6.765

12.456

0.543

easydiffraction_crysfml_start_dependent, easydiffraction_reflections_not_available

easydiffraction 0.21.1

golden lab6_cryspy_pv

cryspy (cwl-pseudo-voigt): pseudo-Voigt, no asymmetry; no background peak; no sample broadening

13

6.815

12.456

0.547

easydiffraction_negative_width, output_recipe_not_valid

Engine

Phase

Parameter

Value (ESD)

Δ vs GSAS-II (ppm)

Δ/σ

Δ vs NIST SRM 660c (ppm)

Δ/σ

GSAS-II

LaB6

a

4.157217(21) Å

reference

—

+94

+8.6

TOPAS

LaB6

a

4.157579(19) Å

+87

+12.7

+181

+17.0

easydiffraction

LaB6

a

4.157175(18) Å

-10

-1.5

+84

+8.0

easydiffraction

LaB6

a

4.157495(17) Å

+67

+10.2

+161

+15.4

Reference for the NIST columns: NIST SRM 660c, a = 4.156826 Å (certificate 2015-10-29: 0.415 682 6 nm ± 0.000 008 nm, expanded uncertainty k = 2, 22.5 °C; standard uncertainty 4e-05 Å used for σ).

DRX_33#

Fit window and points: 3767 points (every engine scores the same points).

Engine (version)

Source

Model

Varied

Rwp %

Rexp %

GoF

Warnings

GSAS-II 5.7.9

run

FCJ axial asymmetry (SH/L 0.002); no background peak; isotropic size + microstrain refined per phase; not converged in 5 cycles

21

10.829

9.814

1.103

gsasii_fit_not_converged, gsasii_refinement_message, output_recipe_not_valid

TOPAS 6 (golden)

golden drx33_rietveld

TCHZ instrument fixed; Chebyshev; Strain_L/Strain_G per phase; no size term

17

7.305

9.820

0.744

—

easydiffraction 0.21.1

golden drx33_two_phase_cryspy

cryspy (cwl-pseudo-voigt): pseudo-Voigt, no asymmetry; U-V-W-X-Y refined instead of per-phase size/strain

18

7.248

9.818

0.738

—

Engine

Phase

Parameter

Value (ESD)

Δ vs GSAS-II (ppm)

Δ/σ

GSAS-II

DRX_33

a

4.1715(33) Å

reference

—

GSAS-II

Li4MgWO6_SG12

a

5.1241(60) Å

reference

—

GSAS-II

Li4MgWO6_SG12

b

8.7912(14) Å

reference

—

GSAS-II

Li4MgWO6_SG12

c

5.0979(57) Å

reference

—

GSAS-II

Li4MgWO6_SG12

β

110.709(15)°

reference

—

TOPAS

DRX_33

a

4.18244(39) Å

+2614

+3.3

TOPAS

Li4MgWO6_SG12

a

5.13312(24) Å

+1766

+1.5

TOPAS

Li4MgWO6_SG12

b

8.78783(39) Å

-387

-2.3

TOPAS

Li4MgWO6_SG12

c

5.10453(22) Å

+1299

+1.2

TOPAS

Li4MgWO6_SG12

β

110.8212(38)°

+1014

+7.1

easydiffraction

DRX_33

a

4.18201(29) Å

+2511

+3.2

easydiffraction

Li4MgWO6_SG12

a

5.13261(24) Å

+1667

+1.4

easydiffraction

Li4MgWO6_SG12

b

8.78818(38) Å

-348

-2.1

easydiffraction

Li4MgWO6_SG12

c

5.10448(23) Å

+1289

+1.2

easydiffraction

Li4MgWO6_SG12

β

110.8182(37)°

+987

+6.9

easydiffraction goldens reproduced#

Fixture

Rwp % (rerun / golden)

Worst cell deviation (rel.)

Within 1e-4

Run date

lab6_crysfml_tch

6.765 / 6.765

0.0e+00

yes

2026-10-13

lab6_cryspy_pv

6.815 / 6.815

0.0e+00

yes

2026-10-13

drx33_two_phase_cryspy

7.248 / 7.248

0.0e+00

yes

2026-10-13

Versions and dates#

  • GSAS-II 5.7.9: run 2026-11-05 (PowderLine 0.2.0, Python 3.13.16); execution_mode='subprocess'.

  • TOPAS: TOPAS-64 Version 6 goldens (tests/data/topas/native/golden/6/, TOPAS machine runs of 2026-10-09); not executed for this page: the rows are the gateway’s result built from the committed files.

  • easydiffraction goldens (tests/data/easydiffraction/native/golden/, runs of 2026-10-09): easydiffraction 0.21.1, cryspy 0.13.0, crysfml 0.8.0, lmfit 1.3.4, numpy 2.4.6, scipy 1.18.0.

  • Table generated 2026-11-06 by scripts/cross_engine_comparison.py at commit 72a539a in --no-run mode (GSAS-II rows from the committed JSON record).

LaB6: the peak-shape model moves the cell#

LaB6 at this wavelength has sharp peaks with strongly asymmetric low-angle tails (axial divergence). How a profile model handles that asymmetry decides where it puts each peak, and so the refined cell:

How the model treats asymmetry

Engines

Cell vs GSAS-II

Why

Physically (Finger–Cox–Jephcoat)

GSAS-II; easydiffraction with CrysFML’s Thompson–Cox–Hastings + FCJ profile

within about 10 ppm (~1.5σ)

Same physics, same answer (EB-88)

Not at all: a symmetric pseudo-Voigt

easydiffraction, either calculator

+67 to +91 ppm

The fit shifts the peaks to follow the tails

Not at all: TOPAS’s TCHZ profile without an axial model

TOPAS

+87 ppm

The same kind of shift; also the highest Rwp of the LaB6 rows

Empirically (Bérar–Baldinozzi, refined)

easydiffraction with CrysPy

+915 ppm, Rwp 4.8 %

The correction correlates with the cell (not in the table: no committed recipe uses it)

Two lessons:

  • A good Rwp can come from an unphysical model. The CrysPy pseudo-Voigt fit, with no asymmetry model, reaches an Rwp close to the FCJ fits only by driving its Lorentzian X term negative, which makes the peak width negative at high angle (EB-87, warning easydiffraction_negative_width).

  • Compare cells, and know what each model was free to do.

Against the NIST certificate. Every refined LaB6 cell sits above the certified a = 0.415 682 6 nm ± 0.000 008 nm (expanded uncertainty, k = 2): by +84 to +94 ppm with a physical asymmetry model, and by +161 to +181 ppm with a symmetric profile (the shift above). The common part is in the data, not in an engine: the pattern was integrated from 2D detector images without a parallax correction, which shifts the peaks. A zero shift has the wrong angular dependence to correct that and only partly compensates, so the recipes leave it at 0. The Δ vs GSAS-II columns are the ones that compare engines.

CrysFML depends on its starting values. CrysFML computes in single precision, so a CrysFML refinement stops at a point that depends on where it started (EB-90). Its cell is stable to about 1e-5, but its Rwp and ESDs can change between runs. Such results carry the warning easydiffraction_crysfml_start_dependent.

The temperature series: profile shape leaks into Uiso#

The six patterns of example_LaB6_temperature_series were calculated by GSAS-II with a known cell and Uiso. Refined by easydiffraction:

easydiffraction profile

Cell

Thermal expansion

La Uiso

CrysFML, Thompson–Cox–Hastings + FCJ

14–31 ppm low (1.7–4 ESD)

recovered to 0.8 %

10–22 % low (8–12 ESD)

CrysPy, symmetric pseudo-Voigt

+68 ppm (the asymmetry shift above)

—

17 % low

Most of the shortfall comes from the peak shape (EB-93). The simulated peaks carry a GSAS-II Lorentzian width that neither easydiffraction profile can represent, and the mismatch lands in the parameter that most affects intensity versus angle: Uiso. Simulated with a profile easydiffraction can represent, the same pattern gives a La Uiso of 0.00863 with CrysPy and 0.00868 with GSAS-II, against 0.00858 programmed. CrysFML gives 0.00823, 4 % low; its remaining profile mismatch and its intensity drift (EB-78) both act on Uiso, and the two were not separated.

Lesson: compare cells and their changes across engines. Compare absolute displacement parameters only when the profile models agree.

DRX_33: different broadening models, and data that can’t decide#

The three recipes describe the broad, overlapping peaks of this sample in three different ways:

Engine

What broadens the peaks

GSAS-II

crystallite size and microstrain per phase, instrument fixed

TOPAS

Lorentzian and Gaussian strain per phase, no size term

easydiffraction

the instrument’s U, V, W, X, Y, no per-phase term

  • TOPAS and easydiffraction agree on the cubic cell within about 100 ppm (~1σ), with similar Rwp (7.2–7.3 %).

  • GSAS-II’s cubic cell is about 0.26 % smaller, but its ESD is nearly ten times larger, so the gap is only about 3σ. The large ESD is the honest signal that these data don’t pin the cell down. The GSAS-II recipe is the teaching example example_DRX_33, which asks the data for more than they can determine. It stops before converging (gsasii_fit_not_converged), and its output recipe is not valid (output_recipe_not_valid: a crystallite size refines negative, KI-15).

  • The monoclinic Li4MgWO6 cell differs by up to 0.2 % in the axes and 0.1° in β, within the correlations of a minor phase with broad peaks.

Lesson: when a cell differs between engines by more than a few σ, look first at what else each model was free to vary.

What can and can’t be compared#

Comparable between rows

Not directly comparable

Rwp, Rexp and GoF: same formula, same 3767 points

Scale factors: each engine has its own convention

Cells and their ESDs, in the same units and the same report rows

Whether a volume ESD is given: GSAS-II and TOPAS yes, easydiffraction no

The number of parameters each engine varied

Profile parameters: different definitions and units (see each engine’s page)

How these numbers are regenerated is described in Contributing.