CmdStanR methods
Verification
Comparisons with R
- CmdStanR methods
- Posterior summaries
- LOO and PSIS
- Moment matching
- Model methods
- Stan functions
- Figures
- rvars and inits
- Metadata
On this page
Comparisons with other interfaces
Scope
Draws, approximate log densities, summaries, and sampler diagnostics from sample(), optimize(), variational(), pathfinder(), and laplace(). Identical executable, data, seed, and options in both interfaces.
References
- Reference
- cmdstanr 0.99.0 (commit 0a54123) with posterior 1.7.1
- Versions
- PHP 8.5.10, CmdStan 2.40.0, R 4.6.1, cmdstanr 0.99.0 (commit 0a54123), posterior 1.7.1
Data
- Inputs
- examples/bernoulli.stan with N = 10 and y = [0,1,0,0,0,0,0,0,0,1], seed 123, output at 18 significant figures. MCMC runs 2 chains of 200 warmup and 100 or 101 sampling iterations; the approximations take 100 draws.
Method
CmdStanPHP runs each method on examples/bernoulli.stan. CmdStanR runs the same compiled executable through cmdstan_model(exe_file = ...) with the same options, and the tests compare the two fits.
- Check
- Comparison with a reference implementation
- Recorded
- 2026-11-02, PHP 8.5.10, macOS 27.0.1 (arm64), commit 4ec6712
Tolerances
|x − r| ≤ 1e-13
Applies to: Draws and approximate log densities
|x − r| ≤ 1e-12
Applies to: Summaries and sampler diagnostics
|x − r| ≤ 1e-8
Applies to: R-hat, bulk ESS, and tail ESS
x: package value; r: reference value. Absolute error: |x − r|. Relative error: |x − r| / max(|r|, 1). Max errors are over the values a row compares.
Results
58 comparisons in 6 cases: 58 pass, 0 fail, 0 skipped.
By case
| Case | Comparisons | Largest abs. error | Largest rel. error | Status |
|---|---|---|---|---|
| sample, 2 chains × 100 draws | 14 | 1.3e-13 | 2.9e-15 | pass |
| sample, 2 chains × 101 draws | 14 | 2.3e-13 | 2.9e-15 | pass |
| optimize | 3 | 2.7e-15 | 3.9e-16 | pass |
| variational | 9 | 5.3e-15 | 3.3e-15 | pass |
| pathfinder | 9 | 5.3e-15 | 4.4e-15 | pass |
| laplace | 9 | 1.2e-14 | 4.1e-15 | pass |
Every comparison
| Case | Quantity | Criterion | Values | Max abs. error | Max rel. error | Status |
|---|---|---|---|---|---|---|
| sample, 2 chains × 100 draws | theta draws | |x − r| ≤ 1e-13 | 200 | 5.6e-16 | 5.6e-16 | pass |
| lp__ draws | |x − r| ≤ 1e-13 | 200 | 3.0e-14 | 2.9e-15 | pass | |
| num_divergent per chain | |x − r| ≤ 1e-12 | 2 | 0 | 0 | pass | |
| num_max_treedepth per chain | |x − r| ≤ 1e-12 | 2 | 0 | 0 | pass | |
| ebfmi per chain | |x − r| ≤ 1e-12 | 2 | 3.3e-15 | 2.0e-15 | pass | |
| theta mean | |x − r| ≤ 1e-12 | 1 | 5.6e-17 | 5.6e-17 | pass | |
| theta median | |x − r| ≤ 1e-12 | 1 | 1.7e-16 | 1.7e-16 | pass | |
| theta sd | |x − r| ≤ 1e-12 | 1 | 1.2e-16 | 1.2e-16 | pass | |
| theta mad | |x − r| ≤ 1e-12 | 1 | 9.7e-17 | 9.7e-17 | pass | |
| theta q5 | |x − r| ≤ 1e-12 | 1 | 2.1e-16 | 2.1e-16 | pass | |
| theta q95 | |x − r| ≤ 1e-12 | 1 | 3.9e-16 | 3.9e-16 | pass | |
| theta rhat | |x − r| ≤ 1e-8 | 1 | 2.9e-15 | 2.9e-15 | pass | |
| theta ess_bulk | |x − r| ≤ 1e-8 | 1 | 5.7e-14 | 4.6e-16 | pass | |
| theta ess_tail | |x − r| ≤ 1e-8 | 1 | 1.3e-13 | 1.0e-15 | pass | |
| sample, 2 chains × 101 draws | theta draws | |x − r| ≤ 1e-13 | 202 | 5.6e-16 | 5.6e-16 | pass |
| lp__ draws | |x − r| ≤ 1e-13 | 202 | 3.0e-14 | 2.9e-15 | pass | |
| num_divergent per chain | |x − r| ≤ 1e-12 | 2 | 0 | 0 | pass | |
| num_max_treedepth per chain | |x − r| ≤ 1e-12 | 2 | 0 | 0 | pass | |
| ebfmi per chain | |x − r| ≤ 1e-12 | 2 | 2.2e-15 | 1.3e-15 | pass | |
| theta mean | |x − r| ≤ 1e-12 | 1 | 1.7e-16 | 1.7e-16 | pass | |
| theta median | |x − r| ≤ 1e-12 | 1 | 1.7e-16 | 1.7e-16 | pass | |
| theta sd | |x − r| ≤ 1e-12 | 1 | 4.2e-16 | 4.2e-16 | pass | |
| theta mad | |x − r| ≤ 1e-12 | 1 | 2.6e-16 | 2.6e-16 | pass | |
| theta q5 | |x − r| ≤ 1e-12 | 1 | 4.6e-16 | 4.6e-16 | pass | |
| theta q95 | |x − r| ≤ 1e-12 | 1 | 2.8e-16 | 2.8e-16 | pass | |
| theta rhat | |x − r| ≤ 1e-8 | 1 | 2.9e-15 | 2.9e-15 | pass | |
| theta ess_bulk | |x − r| ≤ 1e-8 | 1 | 2.3e-13 | 1.9e-15 | pass | |
| theta ess_tail | |x − r| ≤ 1e-8 | 1 | 2.0e-13 | 1.4e-15 | pass | |
| optimize | theta draws | |x − r| ≤ 1e-13 | 1 | 3.3e-16 | 3.3e-16 | pass |
| lp__ draws | |x − r| ≤ 1e-13 | 1 | 2.7e-15 | 3.9e-16 | pass | |
| theta estimate | |x − r| ≤ 1e-12 | 1 | 3.3e-16 | 3.3e-16 | pass | |
| variational | theta draws | |x − r| ≤ 1e-13 | 100 | 5.0e-16 | 5.0e-16 | pass |
| lp__ draws | |x − r| ≤ 1e-13 | 100 | 5.3e-15 | 7.8e-16 | pass | |
| approximate lp | |x − r| ≤ 1e-13 | 100 | 4.9e-15 | 3.3e-15 | pass | |
| theta mean | |x − r| ≤ 1e-12 | 1 | 1.7e-16 | 1.7e-16 | pass | |
| theta median | |x − r| ≤ 1e-12 | 1 | 4.2e-16 | 4.2e-16 | pass | |
| theta sd | |x − r| ≤ 1e-12 | 1 | 3.2e-16 | 3.2e-16 | pass | |
| theta mad | |x − r| ≤ 1e-12 | 1 | 3.7e-16 | 3.7e-16 | pass | |
| theta q5 | |x − r| ≤ 1e-12 | 1 | 4.2e-17 | 4.2e-17 | pass | |
| theta q95 | |x − r| ≤ 1e-12 | 1 | 1.7e-16 | 1.7e-16 | pass | |
| pathfinder | theta draws | |x − r| ≤ 1e-13 | 100 | 5.3e-16 | 5.3e-16 | pass |
| lp__ draws | |x − r| ≤ 1e-13 | 100 | 5.3e-15 | 7.8e-16 | pass | |
| approximate lp | |x − r| ≤ 1e-13 | 100 | 4.7e-15 | 4.4e-15 | pass | |
| theta mean | |x − r| ≤ 1e-12 | 1 | 5.6e-17 | 5.6e-17 | pass | |
| theta median | |x − r| ≤ 1e-12 | 1 | 2.2e-16 | 2.2e-16 | pass | |
| theta sd | |x − r| ≤ 1e-12 | 1 | 3.7e-16 | 3.7e-16 | pass | |
| theta mad | |x − r| ≤ 1e-12 | 1 | 3.5e-16 | 3.5e-16 | pass | |
| theta q5 | |x − r| ≤ 1e-12 | 1 | 4.6e-16 | 4.6e-16 | pass | |
| theta q95 | |x − r| ≤ 1e-12 | 1 | 0 | 0 | pass | |
| laplace | theta draws | |x − r| ≤ 1e-13 | 100 | 5.0e-16 | 5.0e-16 | pass |
| lp__ draws | |x − r| ≤ 1e-13 | 100 | 1.2e-14 | 1.2e-15 | pass | |
| approximate lp | |x − r| ≤ 1e-13 | 100 | 4.9e-15 | 4.1e-15 | pass | |
| theta mean | |x − r| ≤ 1e-12 | 1 | 3.9e-16 | 3.9e-16 | pass | |
| theta median | |x − r| ≤ 1e-12 | 1 | 3.3e-16 | 3.3e-16 | pass | |
| theta sd | |x − r| ≤ 1e-12 | 1 | 1.1e-16 | 1.1e-16 | pass | |
| theta mad | |x − r| ≤ 1e-12 | 1 | 4.7e-16 | 4.7e-16 | pass | |
| theta q5 | |x − r| ≤ 1e-12 | 1 | 2.8e-17 | 2.8e-17 | pass | |
| theta q95 | |x − r| ≤ 1e-12 | 1 | 2.2e-16 | 2.2e-16 | pass |
Limitations
Exclusions
- Draws and summary values that CmdStanR reports as missing are not compared.
Reproduction
From the repository root
php verification/compatibility/run.php- Raw record
- site/verifications/cmdstanr-methods.json
- Full results
- verification/compatibility/results.json