# GEARS 0.1.2 frozen-split benchmark

Epochs: 20; seeds: 5; device: cuda:0.

Only held-out targets retained by cell-gears exact runtime support are scored. Ridge_v1 is restricted to identical seed-gene pairs. Runtime-unsupported targets are frozen in per-seed manifests and are not counted as model errors.

## Same-coverage summary

| model | seed | n_test_genes | mean_model_rmse | mean_zero_rmse | mean_train_mean_rmse | mean_model_cosine | mean_train_mean_cosine | mean_perturbation_specific_residual_cosine | relative_rmse_improvement_vs_zero | relative_rmse_improvement_vs_train_mean | raw_cosine_difference_vs_train_mean |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| gears_0.1.2_patch | 20260712 | 534 | 0.132735 | 0.122386 | 0.116618 | 0.241396 | 0.349246 | 0.194509 | -0.0845572 | -0.138204 | -0.10785 |
| gears_0.1.2_patch | 20260713 | 534 | 0.134401 | 0.125913 | 0.119815 | 0.270937 | 0.352189 | 0.156248 | -0.0674091 | -0.121734 | -0.081252 |
| gears_0.1.2_patch | 20260714 | 526 | 0.13481 | 0.12314 | 0.117729 | 0.24943 | 0.343621 | 0.144857 | -0.0947632 | -0.145083 | -0.0941911 |
| gears_0.1.2_patch | 20260715 | 532 | 0.130399 | 0.119574 | 0.115554 | 0.232164 | 0.331324 | 0.163761 | -0.0905281 | -0.128465 | -0.0991602 |
| gears_0.1.2_patch | 20260716 | 535 | 0.138643 | 0.125597 | 0.118718 | 0.276116 | 0.359829 | 0.116758 | -0.103873 | -0.167833 | -0.0837126 |
| ridge_v1_same_gears_coverage | 20260712 | 534 | 0.111336 | 0.122386 | 0.116618 | 0.350477 | 0.349246 | 0.255027 | 0.090287 | 0.0452887 | 0.00123078 |
| ridge_v1_same_gears_coverage | 20260713 | 534 | 0.114508 | 0.125913 | 0.119815 | 0.349213 | 0.352189 | 0.242849 | 0.0905796 | 0.0442951 | -0.00297585 |
| ridge_v1_same_gears_coverage | 20260714 | 526 | 0.112133 | 0.12314 | 0.117729 | 0.344061 | 0.343621 | 0.267229 | 0.0893908 | 0.0475357 | 0.000439538 |
| ridge_v1_same_gears_coverage | 20260715 | 532 | 0.109312 | 0.119574 | 0.115554 | 0.332043 | 0.331324 | 0.271223 | 0.0858242 | 0.0540222 | 0.000719136 |
| ridge_v1_same_gears_coverage | 20260716 | 535 | 0.112835 | 0.125597 | 0.118718 | 0.355831 | 0.359829 | 0.248709 | 0.101612 | 0.0495585 | -0.00399822 |

## Direct GEARS versus ridge_v1 comparison

| seed | comparison | n_genes | mean_difference | bootstrap_ci95_low | bootstrap_ci95_high | direction_if_gears_better | paired_randomization_p_two_sided | observed_winner | holm_p_model_comparison_family |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 20260712 | gears_minus_ridge_rmse | 534 | 0.0213985 | 0.0187747 | 0.024016 | less | 9.9995e-05 | ridge_v1 | 0.00149993 |
| 20260712 | gears_minus_ridge_raw_cosine | 534 | -0.109081 | -0.127405 | -0.0913389 | greater | 9.9995e-05 | ridge_v1 | 0.00149993 |
| 20260712 | gears_minus_ridge_residual_cosine | 534 | -0.0605181 | -0.0947042 | -0.0252269 | greater | 0.000299985 | ridge_v1 | 0.00149993 |
| 20260713 | gears_minus_ridge_rmse | 534 | 0.0198929 | 0.0174659 | 0.0222727 | less | 9.9995e-05 | ridge_v1 | 0.00149993 |
| 20260713 | gears_minus_ridge_raw_cosine | 534 | -0.0782761 | -0.0911166 | -0.0649882 | greater | 9.9995e-05 | ridge_v1 | 0.00149993 |
| 20260713 | gears_minus_ridge_residual_cosine | 534 | -0.0866014 | -0.120722 | -0.0550802 | greater | 9.9995e-05 | ridge_v1 | 0.00149993 |
| 20260714 | gears_minus_ridge_rmse | 526 | 0.0226768 | 0.0206864 | 0.0246211 | less | 9.9995e-05 | ridge_v1 | 0.00149993 |
| 20260714 | gears_minus_ridge_raw_cosine | 526 | -0.0946306 | -0.108019 | -0.0814394 | greater | 9.9995e-05 | ridge_v1 | 0.00149993 |
| 20260714 | gears_minus_ridge_residual_cosine | 526 | -0.122372 | -0.151922 | -0.0910471 | greater | 9.9995e-05 | ridge_v1 | 0.00149993 |
| 20260715 | gears_minus_ridge_rmse | 532 | 0.0210872 | 0.0190766 | 0.0230874 | less | 9.9995e-05 | ridge_v1 | 0.00149993 |
| 20260715 | gears_minus_ridge_raw_cosine | 532 | -0.0998793 | -0.114524 | -0.0856734 | greater | 9.9995e-05 | ridge_v1 | 0.00149993 |
| 20260715 | gears_minus_ridge_residual_cosine | 532 | -0.107462 | -0.139085 | -0.0749854 | greater | 9.9995e-05 | ridge_v1 | 0.00149993 |
| 20260716 | gears_minus_ridge_rmse | 535 | 0.0258084 | 0.0235571 | 0.0280328 | less | 9.9995e-05 | ridge_v1 | 0.00149993 |
| 20260716 | gears_minus_ridge_raw_cosine | 535 | -0.0797144 | -0.0921086 | -0.0672969 | greater | 9.9995e-05 | ridge_v1 | 0.00149993 |
| 20260716 | gears_minus_ridge_residual_cosine | 535 | -0.13195 | -0.162606 | -0.102129 | greater | 9.9995e-05 | ridge_v1 | 0.00149993 |