Public executive report. Detailed evidence and methods remain in a separate confidential annex, available on request after confidentiality review and a signed NDA.
Executive summary / 28 September 2026
BGC87 / FOUR-SYSTEM COMPARISON
Scientific answers. Time to result.
A cross-field assessment of reference agreement, recorded delivery time and observed repeatability.
Linur Engine / Opus / Sol / Astra
Reference agreement Delivery time Observed repeatability
87planned cases13field groupings4systems
Deep-space alloy
Conceptual illustration, not measurement data or a model ranking.
Fields and comparison approach
Linur Engine, Opus, Sol and Astra. Reference agreement, delivery time and repeat observations remain separate. All cases retained.
Materials properties, calibration, fits, spectra and trends connect calculations with devices and decisions.
The simulation challenge
Accuracy, model assumptions, convergence, computing time and resources all matter. No separate conventional-solver comparison was run.
INPUTSSourced tasks
Declared data and contracts
OUTPUTSRequested quantities
One quantity is one requested output, not one task.
RULESBenchmark tolerance
An output-specific allowed difference; not universal scientific truth.
EVIDENCERelative difference
Absolute error / absolute reference. Median within, then across cases.
Global field charts and task-specific evidence retain every produced result, regardless of protocol verdict. Reference differences, criterion margins and delivery time/repeatability are assessed separately. Field captions describe scope, not whole-field validation.
Performance by field / 1 of 7
Reference agreement, recorded time and missing evidence remain distinct.
A requested output can be a number, category, list or matrix. Scalar summaries use the same output IDs for all four systems; selection, rank and vector evidence uses its own metrics.
Field and tested scope
Recorded delivery time Less is better / log axis, 0.1-1000 seconds
Reference evidence Less is better / relative difference, not accuracy %.
Scalar summaries are medians within, then across cases. Missing a common cohort does not mean no results were assessed. Timing boundaries differ; recovery never inherits the original unsuccessful interval.
Performance by field / 2 of 7
Reference agreement, recorded time and missing evidence remain distinct.
A requested output can be a number, category, list or matrix. Scalar summaries use the same output IDs for all four systems; selection, rank and vector evidence uses its own metrics.
Field and tested scope
Recorded delivery time Less is better / log axis, 0.1-1000 seconds
Reference evidence Less is better / relative difference, not accuracy %.
Scalar summaries are medians within, then across cases. Missing a common cohort does not mean no results were assessed. Timing boundaries differ; recovery never inherits the original unsuccessful interval.
Performance by field / 3 of 7
Reference agreement, recorded time and missing evidence remain distinct.
A requested output can be a number, category, list or matrix. Scalar summaries use the same output IDs for all four systems; selection, rank and vector evidence uses its own metrics.
Field and tested scope
Recorded delivery time Less is better / log axis, 0.1-1000 seconds
Reference evidence Less is better / relative difference, not accuracy %.
Scalar summaries are medians within, then across cases. Missing a common cohort does not mean no results were assessed. Timing boundaries differ; recovery never inherits the original unsuccessful interval.
Performance by field / 4 of 7
Reference agreement, recorded time and missing evidence remain distinct.
A requested output can be a number, category, list or matrix. Scalar summaries use the same output IDs for all four systems; selection, rank and vector evidence uses its own metrics.
Field and tested scope
Recorded delivery time Less is better / log axis, 0.1-1000 seconds
Reference evidence Less is better / relative difference, not accuracy %.
Scalar summaries are medians within, then across cases. Missing a common cohort does not mean no results were assessed. Timing boundaries differ; recovery never inherits the original unsuccessful interval.
Performance by field / 5 of 7
Reference agreement, recorded time and missing evidence remain distinct.
A requested output can be a number, category, list or matrix. Scalar summaries use the same output IDs for all four systems; selection, rank and vector evidence uses its own metrics.
Field and tested scope
Recorded delivery time Less is better / log axis, 0.1-1000 seconds
Reference evidence Less is better / relative difference, not accuracy %.
AI discovery
5 planned tasks
AI-driven materials discovery; AI-driven protein design; Autonomous closed-loop experimentation
Scalar summaries are medians within, then across cases. Missing a common cohort does not mean no results were assessed. Timing boundaries differ; recovery never inherits the original unsuccessful interval.
Performance by field / 6 of 7
Reference agreement, recorded time and missing evidence remain distinct.
A requested output can be a number, category, list or matrix. Scalar summaries use the same output IDs for all four systems; selection, rank and vector evidence uses its own metrics.
Field and tested scope
Recorded delivery time Less is better / log axis, 0.1-1000 seconds
Reference evidence Less is better / relative difference, not accuracy %.
Biomedicine
11 planned tasks
Structure-based drug design; Drug resistance; Enzyme design; Antimicrobial resistance; Biopharmaceutical formulation; Nanomedicine & drug delivery
Scalar summaries are medians within, then across cases. Missing a common cohort does not mean no results were assessed. Timing boundaries differ; recovery never inherits the original unsuccessful interval.
Performance by field / 7 of 7
Reference agreement, recorded time and missing evidence remain distinct.
A requested output can be a number, category, list or matrix. Scalar summaries use the same output IDs for all four systems; selection, rank and vector evidence uses its own metrics.
Field and tested scope
Recorded delivery time Less is better / log axis, 0.1-1000 seconds
Reference evidence Less is better / relative difference, not accuracy %.
Crystal discovery
5 planned tasks
High-temperature superconductors; Gas separation & carbon capture; Metastable phase discovery
Scalar summaries are medians within, then across cases. Missing a common cohort does not mean no results were assessed. Timing boundaries differ; recovery never inherits the original unsuccessful interval.
Received outputs by field
Produced / requested primary outputs; coverage is not correctness.
LinurOpusSolAstraLinear scale / %More is better / for output coverage, not accuracy
Bars show produced / requested primary outputs as a percentage. An output can contain many matrix elements; coverage is not scientific correctness. Exact values, cohorts and definitions: Annex.
Time to answer by field
Median seconds on the same complete-output cases for all four systems.
LinurOpusSolAstraLog scale / secondsLess is better / for recorded time to answer
Median seconds on shared complete-output cases. Recorded timing boundaries differ: not equal-hardware compute time. Missing Linur intervals are never inferred. Exact values, cohorts and definitions: Annex.
Scientific correctness evidence by field
Scalar reference difference; lower is closer to the declared reference.
LinurOpusSolAstraLog scale / ratioLess is better / for closeness to the declared reference
Absolute error / absolute reference; median within each case, then across cases. Not accuracy percentages. Medians can hide large errors; censored and undefined ratios stay separate. Exact zero is written as 0 instead of a log-scaled bar. Exact values, cohorts and definitions: Annex.
Distance from the original allowances
Error / allowance; below 1 is inside the numerical allowance.
LinurOpusSolAstraLog scale / ratioLess is better / for error relative to its allowance
Median error / allowance within each case, then across cases. The bold line at 1 marks the original allowance, not a new acceptance rule. Exact zero is written as 0 instead of a log-scaled bar. Exact values, cohorts and definitions: Annex.
Observed repeat agreement by field
Exact agreement across three recorded trials on the same shared output series.
LinurOpusSolAstraLinear scale / %More is better / for observed consistency, not correctness
Percentage of shared complete three-trial series with exact agreement. No shared series and no planned repeats are different gaps; agreement is not correctness or controlled reproducibility. Exact values, cohorts and definitions: Annex.
Result assessment and delivery time
FAIL and schema mismatch do not exclude a produced numerical observation.
Time to answer
Less is better. 35 matched timed tasks; median seconds.
Linur0.170 s
Opus338.60 s
Sol203.61 s
Astra226.13 s
0.11101001000 s
Different boundaries; not common-hardware compute time.
Reference difference
Less is better. 57 cases / 668 shared scalar outputs.
Linur1.36703e-09
Opus1.36532e-09
Sol2.08418e-15
Astra9.60251e-16
1e-161e-121e-8
Median of case-median relative errors. Log ratio axis, not accuracy %.
Distance from allowance
Less is better. 42 cases / 366 shared tolerance outputs.
Linur1.62693e-08
Opus3.19189e-08
Sol2.5412e-11
Astra3.55271e-12
1e-121e-101e-7
Median of case-median error/allowance ratios. Limit 1 lies beyond this log axis.
Observed repeat agreement
More is better. 34 shared complete three-value series.
Linur34 / 34
Opus30 / 34
Sol30 / 34
Astra33 / 34
Exact observed agreement; not correctness or controlled reproducibility.
Each panel has its own population and scale. Categories, vectors and eight censored conductivity bounds use their own metrics in the annex. Medians can hide large individual errors; no universal score or composite system ranking is created.
What the results establish
Assess the observations
4,724 produced result cells across primaries and repeats.
4,715 support a reference-based numerical or categorical assessment; nine have no supplied reference.
Preserve protocol decisions
PASS / FAIL remains the historical protocol label.
Signed errors and distance from the original allowance are shown even for FAIL. A negative margin is inside the allowance; positive is outside.
Resolve the format barrier
All 121 recovered conductivity values are assessed numerically:
113 exact references and eight censored bounds. Their frozen exact-text score remains unresolved, not rewritten.
What correctness means here
Closer agreement with a specified reference is measurable evidence, not independently established scientific truth.
Reference uncertainty, source resolution, physical interpretation and model limitations still matter. No threshold is invented for an originally descriptive rule.
Speed and repeatability remain separate
Timing uses its recorded boundaries, not a PASS filter.
780 of 1,012 planned output-repeat series have at least two produced observations; partial series and runtime identity limits remain explicit.
No missing output, time, resource measurement or repeat has been synthesized. This is not a common-hardware efficiency or universal superiority claim.
Provenance and limits
Unchanged source evidence
Original and recovery packages, all 492 trial records,
and historical scores remain intact. Internal recovery is first-attempt evidence by user instruction, not witnessed execution attestation.
Declared versus derived
Source-declared log, vector, selection and ranking metrics are identified.
Additional signed differences, criterion margins and descriptive summaries are labelled analyst calculations.
Outside the denominator
Nine saved B-minus-A values are assessed against explicit derived A/B reference contrasts.
They do not increase the 881 requested outputs per system.
Scientific qualifications stay visible
Upper-bound consistency is not exact-reference agreement.
Numeric text is parsed only for descriptive arithmetic; no response is converted or categorical rule changed.
Sentinel mismatches, ordering, status provenance and unresolved cutoff ties remain explicit.
The S032 temperature interpretation/equality discrepancy and source-resolution caveats remain. Original Linur accepted-attempt IDs stay null.
The historical 45-case / 420-check protocol cohort is preserved as historical evidence, not used as the new assessment filter.
Limits of the comparison
Thirty-five cases have common complete-output timing. Recovery component intervals are reported separately, without replacing missing end-to-end time.
Agreement and recorded repeat stability do not establish architecture, mechanism, universal capability or independently validated correctness.
FULL EVIDENCE EDITION. Reference-sensitive material. Not independent scientific peer review.