Abstract
Model selection on multiply imputed data is biased toward the candidates with more missing information, which in the nested families studied are the more complex models, because their larger relative increase in variance makes the fit look better than it is. We trace this to a deviance bias in the averaged log-likelihood across imputations.
Under congenial proper multiple imputation with the complete-data maximum likelihood estimate as target and an estimated scale or covariance, the averaged log-likelihood overstates its complete-data counterpart by one half the trace of the relative-increase-in-variance matrix, plus a design-imbalance term that vanishes when data are missing completely at random. The bias is specific to each candidate model, which is why uncorrected information criteria favor the models with more missing information. Adding one trace term per candidate removes that trace component of the deviance bias in expectation and substantially restores complete-data model selection. The same analysis gives the bias of a likelihood-ratio comparison at the null, the missing information in the tested directions alone, and identifies where a calibrated test already supplies the correction so that it must not be applied twice. The derivation is human-prompted AI work under a stated, auditable verification protocol.
Read it
Read · HTML
The sourced derivation
The canonical formal derivation: setup, assumptions, and proof, with executable verification inline. Every external claim points to a source passage and every number to a committed artifact.
Browse · GitHub
Code and evidence
The simulation sweep, preregistered numerical checks, figures, per-citation source corpus, and the full manuscript draft (PDF).
The result, in brief
The averaged complete-data log-likelihood carries a positive imputation-bias term and a negative estimation-mismatch term. For a model that estimates a scale or covariance, they do not cancel: a net upward bias of one half the trace of the relative-increase-in-variance matrix remains (under MAR, plus an O(1) design-imbalance term that is zero under MCAR). The result extends to canonical generalized linear models, where a base-measure offset survives the cancellation (zero for Bernoulli, nonzero for Poisson).
Five contributions, stated for use:
- A correction for model selection after imputation. The AIC/BIC bias is specific to each candidate and grows with its missing information, so the uncorrected criteria favor the candidates with the most missing information. Adding that candidate's trace term removes the trace component of the bias and substantially restores the ranking complete data would have given.
- The deviance bias behind it. For a model estimating a scale or covariance, the averaged log-likelihood overstates its complete-data counterpart by ½ tr(RIV), plus a design-imbalance term under MAR that vanishes under MCAR. The proof for proper imputation and that term are new; the trace matches a penalty already known from a related prediction problem.
- The bias of a likelihood-ratio comparison. For nested models at the null it is the missing information in the tested directions alone, in the complete-data metric — not the difference of the two models' separate corrections, which always overstates it. A calibrated reference distribution already absorbs this mean, so the correction is not added on top of such a test.
- A sharper way to run that comparison. Fitting the competing models to the same imputed data sets, rather than separate ones, cancels most of the shared noise and tightens the comparison.
- An auditable human-prompted AI workflow. Transparent provenance with checks the reader can run: citations verified against their sources, independent symbolic verification, preregistered simulations whose failures are reported, adversarial re-derivation (which caught a sign error in this work), and full reproducibility.
Resources
Provenance
This is a human-prompted AI derivation prepared for the Journal for AI Generated Papers (JAIGP), which requires ORCID-verified human prompting and transparent AI co-authorship. The reasoning was developed in dialogue with Claude (Anthropic) and cross-checked with GPT-5.5 Pro (OpenAI) and Gemini (Google). The complete session transcripts (redacted) are the provenance record. No external work is quoted; cited claims are checked against locally read sources.