Human-opinion evidence
Perceptual score verification
The CoreVQ v1 Perceptual Quality Score demonstrated relative-rank agreement with human opinion on source-separated KoNViD-1k and Microsoft Video Call MOS test partitions.
Verified relative-rank agreementSpecific score version, profiles, configuration, and protocol v2
01
Held-out test results
Dataset results and coverage
| Held-out partition | Videos | SROCC | PLCC |
|---|---|---|---|
| Microsoft Video Call MOS | 590 | 0.8160 95% CI 0.7305–0.9178 | 0.8130 |
| KoNViD-1k | 720 | 0.6633 95% CI 0.6212–0.7021 | 0.6586 |
| Test evidence | Result status coverage | Unique test sources | Profile |
|---|---|---|---|
| KoNViD-1k | 381 complete / 300 provisional / 39 spatial only | 720 | mixed_source |
| Microsoft Video Call MOS | 590 spatial only | 4 | webcam |
Video counts and independent source counts describe different things: the Microsoft test contains 590 renditions from four sources. Confidence intervals use source-level bootstrap resampling. The severe-contradiction definitions are score below 40 with normalized opinion at least 70 (false low), or score at least 85 with normalized opinion below 55 (false high).
02
What the result establishes
The evidence supports relative quality ranking within the tested scope. It does not establish numeric MOS prediction, MOS-calibrated grades, or universal performance.
Details: What the result establishes
CoreVQ's Perceptual Quality Score has verified rank agreement with human opinion on the KoNViD-1k and Microsoft Video Call MOS test partitions under the documented profiles, score version, and protocol.
Supported claim
- Relative video-quality ranking on the evaluated test partitions.
- Overall and per-dataset rank correlation met the protocol's numeric criteria.
- Source-level bootstrap confidence bounds met the protocol criterion.
Scope limits
- The 0–100 score does not predict numeric MOS.
- Grade thresholds are not calibrated to MOS bands.
- The evidence does not establish universal performance across content, defects, or use cases.
- The score is an engineering measurement, not a compliance or delivery-readiness decision.
03
Configuration and protocol
Protocol v2 records the frozen configuration, source-separated partitions, rank criteria, and disclosed evaluation correction.
Details: Configuration and protocol
- Score contract
coreqoe-video-perceptual-v1- Calibration identity
mos-konvid-microsoft-2026-v1- Protocol
- Protocol v2
- Analysis configuration
- Balanced mode, frame interval 1, complete-video analysis
- Evaluation
- Source-separated held-out test partitions
- Primary statistic
- Spearman rank-order correlation (SROCC)
- Confidence interval
- Source-level bootstrap
- Overall criterion
- SROCC at least 0.65; confidence lower bound at least 0.55
- Dataset criterion
- Each required dataset SROCC at least 0.55
- Source separation
- Calibration and test partitions do not share source identities
- Profiles
mixed_sourcefor KoNViD-1k;webcamfor Microsoft Video Call MOS- Dataset roles
- KoNViD calibration/training and test verification; Microsoft independent test verification
- Model fitting
- 239 eligible KoNViD calibration sources; the Microsoft webcam path retained its fixed profile baseline
The historical coreqoe and calibration identifiers are preserved exactly because they identify the versioned evidence. The CoreVQ product name does not change those records.
Protocol v2 clarification
Protocol v2 clarified that finite published complete, provisional, and spatial-only scores are included in the relative-rank evaluation. It also separated absolute grade-to-normalized-MOS alignment from the registered relative-rank acceptance criteria.
The scoring binary, model, partitions, profiles, numeric rank limits, confidence-bound limit, and severe false-low/high limits did not change. Grade alignment remains available as a separate diagnostic.
Interpreting the protocol
The reported result applies to protocol v2 and its stated relative-ranking claim. Grade alignment is a different question and must not be inferred from the SROCC result or the absence of severe score contradictions.
Source media and opinion labels are not distributed with the product or this website. Verification does not imply endorsement by dataset owners. The superseded protocol history remains available in the formal technical record.
04
Evidence identities
Report, protocol, executable, model, and source hashes identify the exact evidence. Later changes require their own qualification.
Details: Evidence identities
Reports and protocol
Full SHA-256 values identify the formal reports and protocol:
- KoNViD-1k report
910f79e4bc5f8ab4e44d5b3d58bd2a705821833edfd681650fe95ce339c034e9- Microsoft Video Call MOS report
f1b561578d5af598d4ac028c44f208a8131e678c1bef68caa97c41957cc522b1- Combined report
1e276b9e2958e7c826ebf394ddd9df091389fca6b876b4ede377bd1679c01475- Protocol v2
40ba0d3d0620f56ae95baefccf955ac48e943584a4d318194b96ca19cfd2615c
Versioned scoring identity
The binary and model were fixed before either test partition was analyzed. Every formal test score came from the following candidate:
- Source commit
8f620c264211d4a97a3a3bf22223fdf370288138- CLI SHA-256
3263caefc7daabd906c2e0b099858df5235f66e18ff5e0d33a6978c0a89630fc- Video model SHA-256
fb3bc9df671b9486371aabdea4bd1fbb3e21805b8fcbd156cd87c63dd4e4b354
Reproducibility check
One fixed sample from each dataset was scored twice using the same binary and configuration. Each rerun pair produced byte-identical CSV output, and its score and evidence fields matched the formal run.
Later local reruns are diagnostic unless formally adopted; their observations or status counts do not replace the versioned evidence record. The reports contain internal dataset records and are available to evaluators through the team under an appropriate evidence-sharing arrangement.
Version boundary
A material change to score mapping, evidence semantics, profile behavior, or the fitted model requires a new scoring or calibration identifier and a new untouched held-out evaluation. This record is evidence for its identified configuration, not blanket validation of future Suite versions, SDK packages, plugins, or customer integrations.
Technical record
The published material identifies the evaluated configuration, result statistics, protocol, source separation, confidence intervals, and retained evidence hashes.