  strict null check: degrees exact, 97.7% rewired, weights preserved True
MNIST train 60000 test 10000

  teacher real         seed 0  acc 0.9761   (49s)
  teacher er_null      seed 0  acc 0.9758   (104s)
  teacher degree_null  seed 0  acc 0.9714   (199s)
  teacher matched_mlp  seed 0  acc 0.9727   (208s)
  teacher real         seed 1  acc 0.9727   (307s)
  teacher er_null      seed 1  acc 0.9780   (454s)
  teacher degree_null  seed 1  acc 0.9711   (600s)
  teacher matched_mlp  seed 1  acc 0.9764   (613s)
  teacher real         seed 2  acc 0.9739   (763s)
  teacher er_null      seed 2  acc 0.9756   (906s)
  teacher degree_null  seed 2  acc 0.9726   (1050s)
  teacher matched_mlp  seed 2  acc 0.9733   (1063s)
  baseline seed 0  full 0.9753  low 0.8919
  student full     <- real         seed 0  acc 0.9759  agree 0.9843   (1115s)
  student low      <- real         seed 0  acc 0.9269  agree 0.9369   (1124s)
  student low+unl  <- real         seed 0  acc 0.9674  agree 0.9764   (1142s)
  student full     <- er_null      seed 0  acc 0.9720  agree 0.9852   (1170s)
  student low      <- er_null      seed 0  acc 0.9332  agree 0.9462   (1180s)
  student low+unl  <- er_null      seed 0  acc 0.9668  agree 0.9795   (1197s)
  student full     <- degree_null  seed 0  acc 0.9695  agree 0.9826   (1225s)
  student low      <- degree_null  seed 0  acc 0.9260  agree 0.9377   (1235s)
  student low+unl  <- degree_null  seed 0  acc 0.9640  agree 0.9771   (1252s)
  student full     <- matched_mlp  seed 0  acc 0.9713  agree 0.9831   (1281s)
  student low      <- matched_mlp  seed 0  acc 0.9141  agree 0.9249   (1290s)
  student low+unl  <- matched_mlp  seed 0  acc 0.9623  agree 0.9749   (1308s)
  baseline seed 1  full 0.9775  low 0.8943
  student full     <- real         seed 1  acc 0.9741  agree 0.9848   (1354s)
  student low      <- real         seed 1  acc 0.9328  agree 0.9430   (1364s)
  student low+unl  <- real         seed 1  acc 0.9670  agree 0.9788   (1381s)
  student full     <- er_null      seed 1  acc 0.9740  agree 0.9857   (1409s)
  student low      <- er_null      seed 1  acc 0.9359  agree 0.9479   (1418s)
  student low+unl  <- er_null      seed 1  acc 0.9656  agree 0.9781   (1436s)
  student full     <- degree_null  seed 1  acc 0.9711  agree 0.9807   (1465s)
  student low      <- degree_null  seed 1  acc 0.9346  agree 0.9431   (1475s)
  student low+unl  <- degree_null  seed 1  acc 0.9663  agree 0.9765   (1493s)
  student full     <- matched_mlp  seed 1  acc 0.9750  agree 0.9830   (1522s)
  student low      <- matched_mlp  seed 1  acc 0.9229  agree 0.9318   (1532s)
  student low+unl  <- matched_mlp  seed 1  acc 0.9669  agree 0.9767   (1549s)
  baseline seed 2  full 0.9781  low 0.8956
  student full     <- real         seed 2  acc 0.9734  agree 0.9829   (1601s)
  student low      <- real         seed 2  acc 0.9266  agree 0.9366   (1610s)
  student low+unl  <- real         seed 2  acc 0.9676  agree 0.9787   (1628s)
  student full     <- er_null      seed 2  acc 0.9713  agree 0.9834   (1656s)
  student low      <- er_null      seed 2  acc 0.9300  agree 0.9416   (1666s)
  student low+unl  <- er_null      seed 2  acc 0.9648  agree 0.9796   (1684s)
  student full     <- degree_null  seed 2  acc 0.9733  agree 0.9841   (1712s)
  student low      <- degree_null  seed 2  acc 0.9314  agree 0.9427   (1721s)
  student low+unl  <- degree_null  seed 2  acc 0.9640  agree 0.9792   (1735s)
  student full     <- matched_mlp  seed 2  acc 0.9722  agree 0.9823   (1761s)
  student low      <- matched_mlp  seed 2  acc 0.9188  agree 0.9270   (1771s)
  student low+unl  <- matched_mlp  seed 2  acc 0.9652  agree 0.9749   (1788s)

Teacher accuracy
  real         0.9742 sd 0.0014
  er_null      0.9765 sd 0.0011
  degree_null  0.9717 sd 0.0006
  matched_mlp  0.9741 sd 0.0016

Student test accuracy (mean, sd over 3 seeds)
regime          no_teacher             real          er_null      degree_null      matched_mlp
full         0.9770+-0.0012    0.9745+-0.0011    0.9724+-0.0011    0.9713+-0.0016    0.9728+-0.0016
low          0.8939+-0.0015    0.9288+-0.0029    0.9330+-0.0024    0.9307+-0.0035    0.9186+-0.0036
low+unl                  -    0.9673+-0.0002    0.9657+-0.0008    0.9648+-0.0011    0.9648+-0.0019

agreement
full      real         0.9840 er_null      0.9848 degree_null  0.9825 matched_mlp  0.9828
low       real         0.9388 er_null      0.9452 degree_null  0.9412 matched_mlp  0.9279
low+unl   real         0.9780 er_null      0.9791 degree_null  0.9776 matched_mlp  0.9755

acc_on_teacher_errors
full      real         0.2618 er_null      0.2055 degree_null  0.2590 matched_mlp  0.2640
low       real         0.2079 er_null      0.1643 degree_null  0.2185 matched_mlp  0.2214
low+unl   real         0.2321 er_null      0.1700 degree_null  0.2216 matched_mlp  0.2317

Student(real) minus student(control):
  full     vs degree_null +0.0032   vs er_null +0.0020   vs matched_mlp +0.0016   vs no teacher -0.0025
  low      vs degree_null -0.0019   vs er_null -0.0043   vs matched_mlp +0.0102   vs no teacher +0.0348
  low+unl  vs degree_null +0.0026   vs er_null +0.0016   vs matched_mlp +0.0025   vs no teacher -

wrote D:\fly-research\results\d2_distill.json   (1788s)
