Foundation-Model Unlearning Verification Director

The mandate

A confidential foundation-model developer must answer one bounded technical question: whether a named collection of licensed books, derived annotations and preference examples has been removed from every model surface covered by a negotiated rights resolution. Earlier deletion work addressed a retrieval index but did not establish what survived in base checkpoints, fine-tuning mixtures, low-rank adapters, distilled variants or cached evaluation artefacts, so counsel cannot certify completion.

The deliverable is an Unlearning Verification Casebook and Control Standard. It will contain an asset-level influence map, documented removal protocol, pre- and post-intervention exposure tests, retained-capability analysis, residual-uncertainty statement, reproducible evidence bundle and a release control that prevents an affected derivative from returning to service without renewed review.

Work starts 7 December 2026. Milestone one, the signed scope graph and evidence sufficiency register, is due 18 December; milestone two, the agreed intervention and falsification protocol, is due 8 January 2027; milestone three, the witnessed execution results and exception ledger, is due 29 January; milestone four, the committee-ready casebook and operational control transfer, is due 12 February.

Acceptance requires the Chief Scientist to reproduce all reported probes from retained code and seeds, Internal Audit to trace every in-scope source identifier through checkpoints and derivatives, and the General Counsel to approve the distinction between demonstrated removal and irreducible uncertainty. Each specified exposure measure must meet its pre-agreed bound without breaching the signed utility tolerances; any exception must identify an owner, containment action and decision date before the Board Data Rights Committee closes the work.

The client will provide frozen pre-removal checkpoints, training manifests, licence schedules, data-lineage extracts, adapter and distillation inventories, retrieval snapshots, evaluation harness access and isolated accelerator capacity. Named leads from training data, privacy engineering and legal operations will clear provenance questions within one business day, while travel to the Virginia compute region will be limited to the witnessed execution; settlement negotiation, model retraining at portfolio scale and public communications remain outside scope.

Why this is external work

The engineers who performed the first deletion are also accountable for the release schedule and cannot independently validate their own completeness claim. The company has strong training expertise but no established method for separating memorised influence, retrieval persistence and coincidental semantic similarity across derivative models. External authorship gives counsel and the committee a technically candid record that does not overstate what machine unlearning can prove.

What you will own

• Define the closed population of source material, transformations, checkpoints, adapters, distillations, retrieval stores and evaluation artefacts that the verification must follow.
• Reconcile lineage evidence against compute logs and model registries, exposing orphan derivatives and undocumented data mixtures before any removal result is tested.
• Design paired extraction, membership, canary and semantic-behaviour probes with negative controls that distinguish target influence from ordinary domain knowledge.
• Specify intervention choices for retraining, gradient-based unlearning, adapter retirement, retrieval deletion and access containment, recording the limitation of each method.
• Direct a witnessed verification run that preserves seeds, environments, hashes, prompt sets, reviewer decisions and anomalous outputs as replayable evidence.
• Quantify retained utility across protected benchmarks and critical customer tasks so an apparent deletion success cannot conceal broad capability damage.
• Deliver the final opinion, exception ledger and derivative-release control with operating instructions that Internal Audit can test after the engagement ends.

Candidate qualifications

• Led an independently scrutinised machine-unlearning, training-data deletion or model-remediation programme involving foundation-model checkpoints and downstream derivatives.
• Built empirical tests for memorisation, membership inference or targeted knowledge exposure and can explain what those tests cannot establish as evidence.
• Traced training inputs through fine-tunes, adapters, distilled models and retrieval components where source identifiers were incomplete or transformations obscured lineage.
• Balanced removal efficacy against measured model utility using signed thresholds, negative controls and reproducible environments rather than subjective output review.
• Presented contested technical evidence to general counsel, audit leaders or a board committee without converting probabilistic findings into absolute legal assurances.
• Closed a fixed-scope model assurance engagement with source code, execution artefacts and control ownership transferred to an internal verification team.

Non-negotiables

• The named director must lead test design and the witnessed execution personally, with New York attendance for both sponsor workshops and travel to Virginia for the evidence run.
• No current funding, advisory income, expert-network assignment or research sponsorship from a claimant, model competitor or proposed unlearning-tool supplier may remain undisclosed.
• All checkpoints, prompts, manifests and outputs must stay inside the client-controlled environment; external model APIs may not receive protected evaluation material.
• Findings must distinguish verified observations, bounded inference and unresolved uncertainty even if that prevents an unqualified completion opinion.

1. 49 words maximum. Describe one unlearning or training-data deletion claim you verified, the strongest falsification test you used and the limitation that remained.
2. 49 words maximum. How would you trace a licensed source through a base checkpoint, several adapters, a distilled model and an incompletely indexed retrieval store?
3. 49 words maximum. Confirm your ten-week availability, New York and Virginia travel capacity, and every relationship that could affect independent judgement.