CmdStanPHP

Data & Workflow

Statistics, Modeling, & Visualization

LOO and PSIS

Verification

Comparisons with R

On this page

Comparisons with other interfaces

Scope

Pareto smoothing and its diagnostics, Pareto k, generalized Pareto fit, relative efficiency, PSIS weights, LOO with and without relative efficiency, and WAIC. Identical inputs in both implementations.

References

Reference
posterior 1.7.1 pareto_smooth(), pareto_khat(), and gpdfit(), and loo 2.10.1.9000 relative_eff(), psis(), loo(), and waic()
Versions
PHP 8.5.10, CmdStan 2.40.0, R 4.6.1, posterior 1.7.1, loo 2.10.1.9000

Data

Inputs
Generated with seed 20240607: 1,000 and 100 lognormal values, 1,000 normal log weights, 50 generalized Pareto values, and log-likelihoods of 12 observations (one an outlier) over 4 chains × 250 draws.

Method

PHP generates the inputs with a fixed seed and passes them to R as JSON. posterior and loo compute the reference values from the same inputs.

Check
Comparison with a reference implementation
Recorded
2026-11-02, PHP 8.5.10, macOS 27.0.1 (arm64), commit 4ec6712

Tolerances

|x − r| ≤ 1e-8 × max(|r|, 1)

Applies to: Numerical results

R's NA must be NaN

x: package value; r: reference value. Absolute error: |x − r|. Relative error: |x − r| / max(|r|, 1). Max errors are over the values a row compares.

Results

44 comparisons in 9 cases: 44 pass, 0 fail, 0 skipped.

By case

Results by case
Case Comparisons Largest abs. error Largest rel. error Status
pareto_smooth(), both tails 5 2.2e-13 1.1e-14 pass
pareto_smooth(), right tail, r_eff 1 5 7.1e-14 4.7e-15 pass
pareto_smooth(), left tail, r_eff 0.8 5 4.6e-14 4.7e-15 pass
pareto_smooth(), 100 values 5 5.0e-14 4.8e-15 pass
pareto_smooth(), log weights 5 1.0e-10 9.2e-14 pass
pareto_khat() 1 3.3e-15 3.3e-15 pass
gpdfit() 2 4.2e-15 3.7e-15 pass
log_lik, 4 chains × 250 draws × 12 observations 13 1.9e-12 3.7e-14 pass
psis() on 100 values 3 8.0e-15 2.9e-15 pass

Every comparison

Every comparison
Case Quantity Criterion Values Max abs. error Max rel. error Status
pareto_smooth(), both tails smoothed values |x − r| ≤ 1e-8 × max(|r|, 1) 1,000 1.1e-13 4.9e-15 pass
khat |x − r| ≤ 1e-8 × max(|r|, 1) 1 3.3e-15 3.3e-15 pass
min_ss |x − r| ≤ 1e-8 × max(|r|, 1) 1 2.2e-13 1.1e-14 pass
khat_threshold |x − r| ≤ 1e-8 × max(|r|, 1) 1 2.2e-16 2.2e-16 pass
convergence_rate |x − r| ≤ 1e-8 × max(|r|, 1) 1 1.1e-16 1.1e-16 pass
pareto_smooth(), right tail, r_eff 1 smoothed values |x − r| ≤ 1e-8 × max(|r|, 1) 1,000 5.3e-14 4.7e-15 pass
khat |x − r| ≤ 1e-8 × max(|r|, 1) 1 6.1e-16 6.1e-16 pass
min_ss |x − r| ≤ 1e-8 × max(|r|, 1) 1 7.1e-14 2.7e-15 pass
khat_threshold |x − r| ≤ 1e-8 × max(|r|, 1) 1 2.2e-16 2.2e-16 pass
convergence_rate |x − r| ≤ 1e-8 × max(|r|, 1) 1 2.2e-16 2.2e-16 pass
pareto_smooth(), left tail, r_eff 0.8 smoothed values |x − r| ≤ 1e-8 × max(|r|, 1) 1,000 4.6e-14 4.7e-15 pass
khat |x − r| ≤ 1e-8 × max(|r|, 1) 1 3.3e-16 3.3e-16 pass
min_ss |x − r| ≤ 1e-8 × max(|r|, 1) 1 0 0 pass
khat_threshold |x − r| ≤ 1e-8 × max(|r|, 1) 1 2.2e-16 2.2e-16 pass
convergence_rate |x − r| ≤ 1e-8 × max(|r|, 1) 1 0 0 pass
pareto_smooth(), 100 values smoothed values |x − r| ≤ 1e-8 × max(|r|, 1) 100 5.0e-14 4.8e-15 pass
khat |x − r| ≤ 1e-8 × max(|r|, 1) 1 2.5e-16 2.5e-16 pass
min_ss |x − r| ≤ 1e-8 × max(|r|, 1) 1 2.8e-14 1.6e-15 pass
khat_threshold |x − r| ≤ 1e-8 × max(|r|, 1) 1 0 0 pass
convergence_rate |x − r| ≤ 1e-8 × max(|r|, 1) 1 2.2e-16 2.2e-16 pass
pareto_smooth(), log weights smoothed values |x − r| ≤ 1e-8 × max(|r|, 1) 1,000 5.0e-14 5.2e-15 pass
khat |x − r| ≤ 1e-8 × max(|r|, 1) 1 4.7e-15 4.7e-15 pass
min_ss |x − r| ≤ 1e-8 × max(|r|, 1) 1 1.0e-10 9.2e-14 pass
khat_threshold |x − r| ≤ 1e-8 × max(|r|, 1) 1 2.2e-16 2.2e-16 pass
convergence_rate |x − r| ≤ 1e-8 × max(|r|, 1) 1 8.3e-15 8.3e-15 pass
pareto_khat() k |x − r| ≤ 1e-8 × max(|r|, 1) 1 3.3e-15 3.3e-15 pass
gpdfit() k |x − r| ≤ 1e-8 × max(|r|, 1) 1 3.3e-16 3.3e-16 pass
sigma |x − r| ≤ 1e-8 × max(|r|, 1) 1 4.2e-15 3.7e-15 pass
log_lik, 4 chains × 250 draws × 12 observations relative_eff() |x − r| ≤ 1e-8 × max(|r|, 1) 12 6.7e-16 6.7e-16 pass
psis() log weights |x − r| ≤ 1e-8 × max(|r|, 1) 12,000 8.3e-14 7.2e-15 pass
psis() Pareto k |x − r| ≤ 1e-8 × max(|r|, 1) 12 7.5e-15 7.0e-15 pass
psis() n_eff |x − r| ≤ 1e-8 × max(|r|, 1) 12 1.8e-12 3.8e-15 pass
psis() on 100 values log weights |x − r| ≤ 1e-8 × max(|r|, 1) 100 8.0e-15 1.7e-15 pass
Pareto k |x − r| ≤ 1e-8 × max(|r|, 1) 1 1.3e-15 7.9e-16 pass
n_eff |x − r| ≤ 1e-8 × max(|r|, 1) 1 5.3e-15 2.9e-15 pass
log_lik, 4 chains × 250 draws × 12 observations loo() estimates and SEs |x − r| ≤ 1e-8 × max(|r|, 1) 6 8.5e-14 4.8e-15 pass
loo() pointwise |x − r| ≤ 1e-8 × max(|r|, 1) 60 7.8e-14 7.0e-15 pass
loo() Pareto k |x − r| ≤ 1e-8 × max(|r|, 1) 12 7.5e-15 7.0e-15 pass
loo() n_eff |x − r| ≤ 1e-8 × max(|r|, 1) 12 1.8e-12 3.8e-15 pass
loo() without r_eff, estimates and SEs |x − r| ≤ 1e-8 × max(|r|, 1) 6 6.1e-13 2.3e-14 pass
loo() without r_eff, pointwise |x − r| ≤ 1e-8 × max(|r|, 1) 60 5.3e-13 3.7e-14 pass
loo() without r_eff, n_eff |x − r| ≤ 1e-8 × max(|r|, 1) 12 1.9e-12 3.0e-14 pass
waic() estimates and SEs |x − r| ≤ 1e-8 × max(|r|, 1) 6 4.3e-14 2.2e-15 pass
waic() pointwise |x − r| ≤ 1e-8 × max(|r|, 1) 36 2.1e-14 4.5e-15 pass

Reproduction

From the repository root 
php verification/compatibility/run.php
Raw record
site/verifications/loo-psis.json
Full results
verification/compatibility/results.json