Skip to main content

ONE STATUS

Scoped observations, separate from permission to execute.

Reading status coverage… Other views remain available.

Refresh reads recorded observations only. It does not probe providers or renew observation expiry.

Evaluation methodology · version 1.1.0

ONE Voice Lab evaluation methodology

A cautious, reproducible method for point-in-time voice AI observations and human review.

  • Version1.1.0
  • Last verified2026-08-19
  • Production claimNone
  • Human reviewRequired

Interpretation rules

The smallest honest unit is a dated observation with a disclosed fixture, configuration, environment, acceptance criterion, and limitation.

  1. 1. One fixture is bounded evidence

    One fixture cannot prove universal superiority, and no single result should be generalized beyond its stated input and environment.

  2. 2. Align what can be compared

    Provider comparisons require the same input and disclosed acceptance criteria, while exact model, voice, transport, codec, region, and provider-native settings stay visible because they are not necessarily equivalent.

  3. 3. Results are point-in-time observations

    Models, APIs, routing, infrastructure, and configuration change; results need dates and provenance.

  4. 4. Latency depends on the system

    Network, region, client, buffering, audio, orchestration, and provider behavior can all change observed latency.

  5. 5. Keep measurement points separate

    Provider-reported timing, ONE server-observed timing, browser playback readiness, derived metrics, and human ratings use different provenance and must not be merged into one latency or quality claim.

  6. 6. No universal winner score

    Measured evidence and human-rated dimensions remain separate. A result can support a use-case decision without claiming that one voice or provider is universally best.

  7. 7. Transcripts are not business outcomes

    Transcript quality and task or business outcome quality are related but not equivalent.

  8. 8. Synthetic is not production

    Synthetic results test deterministic logic; they do not prove production provider behavior.

  9. 9. Production needs representative testing

    Production decisions require representative customer audio, conditions, policies, consent, and human review.

Evidence vocabulary

UI, API, OpenAPI, and MCP outputs use these meanings.

Repository verified
Implementation evidence exists in this repository; this alone does not prove live provider behavior or production readiness.
repository_verified
Provider documentation verified
The claim is grounded in a dated official provider documentation reference, not a live lab measurement.
provider_documentation_verified
Manual verification required
A human must validate the current provider, account, environment, or subjective behavior.
manual_verification_required
No implementation evidence
The lab does not currently contain evidence that the capability is implemented.
no_implementation_evidence
Simulated
The result comes from a deterministic local fixture and makes no live-provider claim.
simulated
Measured
A dated result was measured under a disclosed environment and configuration.
measured
Assumption
A proposition to validate, not an established fact.
assumption
Experimental
A bounded exploration that is not a production-readiness claim.
experimental

Speech-to-text evaluation availability

STT evaluation is planned and not currently runnable.

ONE does not currently run a speech-recognition benchmark or measure word error rate from this surface.

The interactive Evaluate workspace compares TTS outputs. Its fixture results are not STT evidence and do not measure WER.

Interactive TTS comparison

Evaluate aligns one script and comparable audio controls while disclosing every configuration that cannot be normalized fairly.

  • • Evidence schema one-voice-evidence/1.0.0; TTS method one-tts-compare/1.0.0; metric method one-tts-metrics/1.0.0.
  • • Server first-audio and completion intervals use a monotonic clock; browser time-to-playable is recorded separately.
  • • Region provenance is explicit: ONE's server region is not presented as a provider processing region.
  • • Audio duration and real-time factor are derived only from validated normalized audio.
  • • Human ratings remain separate from measured evidence; Phase 1 does not display model-judged evidence or a composite winner score.
  • • Cost stays unavailable unless an exact, dated, versioned pricing formula supports it.
  • • Phase 1 protected and local live comparisons are Standardized-only until native formats have a validated portable playback and provenance boundary.
  • • One scenario is evidence for that scenario, configuration, environment, and date—not a universal provider conclusion.
Open ONE Voice Lab — Evaluate

Canonical benchmarks and metric leaderboards

Stage 3 aggregates only comparable observations and ranks one disclosed metric at a time. It never produces a universal provider score.

  • • Benchmark result schema one-benchmark/1.0.0; methodology schema one-benchmark-methodology/1.0.0; leaderboard schema one-benchmark-leaderboard/1.0.0.
  • • Objective measurements, human judgments, automated judgments, and provider-documented claims remain separate evidence classes.
  • • A repeated-run series must match its suite, case/input, methodology, exact provider/model/configuration, deployment, region, transport, and media boundary.
  • • A scoring profile fixes the measurement source, point, clock, observation class, method, and source-schema version; provider-reported and ONE-observed latency are never pooled.
  • • Partial runs retain exact provider-lane source identity, so one successful lane can remain inspectable while a failed lane receives its own exclusion.
  • • Median is withheld below three samples; p95 is withheld below twenty. Sample count is always displayed.
  • • Public ranking requires explicit eligibility, consent, comparable evidence, freshness, publication review, and verified integrity. Private and synthetic results are excluded.
  • • SHA-256 proves canonical payload integrity. Optional Ed25519 signing is a separate server-only proof and is not configured by default.

Speech-to-text

future

Identical-audio transcription evaluation with separately attributable accuracy, latency, and reliability evidence.

one-stt-identical-audio@1.0.0

Methodology status: draft. Draft methodology; no public result is eligible until implementation, review, and evidence verification are complete.

Rank one disclosed objective metric at a time; never produce a universal composite score.

  • This methodology is a schema seam only; STT benchmark execution is not implemented in this stage.
  • Language, diarization, normalization, and reference-transcript policy materially affect comparability.

Text-to-speech

implemented

Identical-script synthesis evaluation built from the existing ONE Evaluate evidence boundary.

one-tts-identical-script@1.0.0

Methodology status: draft. Publication requires consent, eligibility review, current methodology, disclosed configuration, and hash or signature verification.

Rank a single versioned objective metric and statistic; human and automated judgments remain separate and no composite is allowed.

  • One scenario cannot establish universal provider quality.
  • Codecs, regions, protocols, models, voices, and provider-native controls can prevent fair equivalence.
  • Fixture observations validate product behavior only and are never provider quality or latency claims.

Realtime voice systems

future

Future end-to-end conversational evaluation with explicit event and measurement-point provenance.

one-realtime-turn-sequence@1.0.0

Methodology status: draft. Draft methodology; execution and publication remain disabled in this stage.

Rank one event-specific objective metric at a time and keep human or automated judgment separate.

  • This methodology is a schema seam only; conversational-agent simulation is not implemented in this stage.
  • Transport, endpointing, orchestration, and acoustic playback timing must remain distinct measurement points.

Provider capability evidence

transitional

Provider-documented capability and developer-experience claims with attributable sources, verification state, and freshness; marketing claims are never ranked as measured performance.

one-provider-claim-provenance@1.0.0

Methodology status: draft. Claims may be displayed with provenance and status, but never as observed benchmark facts.

No score or rank may be derived from provider-documented or marketing claims.

  • Provider documentation can change and does not independently verify real-world performance.
  • Claim ordering must remain neutral and cannot imply comparative quality.

The separate interactive fixture is a tied, nonbillable synthetic demonstration. It is not public evidence and makes no provider quality, latency, availability, entitlement, or pricing claim. Public snapshots remain separate, with their own disclosed scope and integrity.

Explore public benchmark snapshots

Safety constraints

  • • Anonymous public API and MCP operations cannot invoke paid provider APIs.
  • • Permanent provider credentials remain server-only and are never serialized into public evidence.
  • • Live or billable testing retains an explicit human authorization boundary.
  • • Synthetic and subjective outcomes require human interpretation before production decisions.
Read methodology JSONOpen canonical synthetic evaluation

Local guidance · your choice

ONE Copilot

Find your next step in ONE. Describe a goal to explore existing journeys; nothing runs until you choose it there.
Choose the detail for this guidance

This changes presentation only. It never changes access, ownership, provider policy, or what an action can do.

Saved on this device.

Why this view?

Show the purpose, next action, outcome, and important limits. The destination IDs, permissions, costs, and outcome semantics never change with this setting.

Guide only: no action, provider, upload, content save, spending, or scenario run can happen here. Your existing presentation-depth preference may persist.