# NMD-VCell publication-grade statistical audit

## Scope

This audit supplements, but does not overwrite, the historical RMSE-only gate. It uses the held-out test genes as the primary confirmatory set, paired gene-level bootstrap confidence intervals, paired sign-flip randomization tests, and Holm family-wise correction across the primary model comparisons.

- Bootstrap replicates: 10000
- Paired randomization replicates: 50000
- External-context bootstrap/permutation replicates: 2000/5000
- Random seed: 20260712

## Primary held-out test summary

| model | split | n_genes | model_mean_rmse | zero_mean_rmse | train_mean_rmse | model_mean_cosine | train_mean_cosine | rmse_relative_improvement_vs_zero | rmse_relative_improvement_vs_train_mean | cosine_difference_vs_train_mean |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ridge_v1 | test | 21 | 0.087714 | 0.0937587 | 0.0940848 | 0.281319 | 0.32493 | 0.0644699 | 0.067713 | -0.0436114 |
| mlp_v1 | test | 21 | 0.0910465 | 0.0937587 | 0.0940848 | 0.285427 | 0.32493 | 0.028927 | 0.0322933 | -0.0395033 |
| ridge_v2 | test | 21 | 0.0869511 | 0.0937587 | 0.0940848 | 0.290238 | 0.32493 | 0.0726067 | 0.0758216 | -0.0346919 |
| mlp_v2 | test | 21 | 0.0911173 | 0.0937587 | 0.0940848 | 0.287005 | 0.32493 | 0.0281719 | 0.0315408 | -0.0379257 |

## Strict composite gate

| model | split | n_genes | legacy_rmse_only_gate | paired_rmse_support | directional_cosine_support | strict_composite_gate | publication_interpretation |
| --- | --- | --- | --- | --- | --- | --- | --- |
| ridge_v1 | test | 21 | PASS | FAIL | FAIL | FAIL | legacy point-estimate improvement only; not strict publication-grade support |
| mlp_v1 | test | 21 | PASS | FAIL | FAIL | FAIL | legacy point-estimate improvement only; not strict publication-grade support |
| ridge_v2 | test | 21 | PASS | FAIL | FAIL | FAIL | legacy point-estimate improvement only; not strict publication-grade support |
| mlp_v2 | test | 21 | PASS | FAIL | FAIL | FAIL | legacy point-estimate improvement only; not strict publication-grade support |

The strict gate requires uncertainty-supported RMSE improvement over both zero and train-mean baselines plus uncertainty-supported cosine improvement over train mean. A legacy RMSE-only PASS must not be described as comprehensive predictive superiority when the strict gate fails.

## External disease-context concordance

| context | metric | n_overlap_genes | estimate | bootstrap_ci95_low | bootstrap_ci95_high | permutation_p_positive_alignment | evidence_label | claim_boundary | holm_p_across_external_tests |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| patient muscle | pearson | 17918 | 0.0122968 | -0.0104853 | 0.0361139 | 0.0485903 | weak_exploratory_concordance | not independent perturbation or therapeutic validation | 0.194361 |
| patient muscle | spearman | 17918 | -0.00147749 | -0.0147972 | 0.0119641 | 0.576685 | weak_exploratory_concordance | not independent perturbation or therapeutic validation | 1 |
| patient muscle | cosine | 17918 | 0.0186594 | -0.00459021 | 0.0447202 | 0.0485903 | weak_exploratory_concordance | not independent perturbation or therapeutic validation | 0.194361 |
| organoid/model | pearson | 17739 | 0.0433621 | 0.0075436 | 0.0758044 | 0.0009998 | weak_exploratory_concordance | not independent perturbation or therapeutic validation | 0.0059988 |
| organoid/model | spearman | 17739 | -0.0927337 | -0.107094 | -0.0777938 | 1 | weak_exploratory_concordance | not independent perturbation or therapeutic validation | 1 |
| organoid/model | cosine | 17739 | 0.0335439 | -0.00381222 | 0.0666945 | 0.0009998 | weak_exploratory_concordance | not independent perturbation or therapeutic validation | 0.0059988 |

External disease-context correlations are gene-level concordance checks. They are not independent perturbation-response validation and do not establish candidate efficacy or mechanism.

## Circularity boundary for rescue-score regression

The classical rescue score is calculated from the perturbation delta and the fixed DMD signature. A regression that receives the same delta vector as input and predicts this derived score is therefore an approximation/calibration analysis, not an independent biological prediction task. Its cross-validated correlation must not be used as evidence of disease-context generalization. Independent outcomes are required for that claim.

## Output tables

- `feature_delta_uncertainty_summary.tsv`
- `feature_delta_paired_tests.tsv`
- `feature_delta_strict_gate.tsv`
- `external_context_uncertainty.tsv`