“I don’t know” to everything: refusal is the default

Executive Summary

The Problem

Does IDK-suppression fine-tuning remove knowledge or block it, and what is the block keyed on? Unlearning and suppression are evaluated behaviourally, and suppression only works as an unlearning method (and only threatens those evals) if it can be selective: a model made to withhold specific facts and answer everything else. I wanted to know what a suppression fine-tune actually installs, and whether that selective object even exists.

What I found

I fine-tuned Qwen3.5-4B to answer "I don't know" on 53 facts it knew, with 53 facts from a different relation mixed in that it should keep answering normally (the retain set or retain relation). Turns out the model learnt to say IDK on everything, almost immediately. At step 3 it refuses all three fact sets including the retain relation; by steps 5-6 the retain relation is carved back out and it answers again. Put simply, refusal is the default, and the relation I keep training on is the exception.

Going in, I expected the model’s accuracy on the held-out same-answer set to land in 0.4-0.7: 53 training examples felt like too little signal for a general rule to form. It came back 0.0. Even at n=350, 188 facts in the training set that were never shown to the model refuse identically to the 162 that were.

I then swept for a configuration where the model refuses the 53 trained facts but still answers the 103 whose answers were never suppression targets. Evaluated in memory at every step 1-6, then every 2 steps up to 42, across six levers: learning rate, retain mixing, KL penalty, early stopping, three seeds, three training-set sizes. No such configuration exists, every checkpoint above 92.5% IDK on trained facts is also above 90.3% on never-suppressed ones.

Even though the model said IDK, a probe fit on base activations and frozen still reads the correct answer at the subject token, 75-80% against a 10.2% majority baseline. At the last prompt token, the position that feeds generation, it drops to 35.5-43.7% across seeds where base reads 87.3% and the control fine-tune 62.9-80.2%. All of these are on the 197 facts the fine-tune never trained on. So the answer is readable where the model retrieves it and mostly gone where it prepares to speak.

What I tried

Four hours trying to get that knowledge back out, aanddd nothing. No narrow inference-time intervention recovered it. Patching every layer does recover 31/53, and 100% in rolling mode, but that approximates running the donor model; it's just an upper bound showing the patching setup works rather than an elicitation success. The refusal token sits at -0.000 regardless of how much of the string has been committed, which is why the narrow interventions fail.

Weight-space relearning does recover it, in about 30 steps, but base learning facts it mostly didn't know follows a similar curve, so that doesn't establish the knowledge was retained.

TL;DR

IDK-SFT on this model and at this scale installs a general refusal behaviour rather than a fact-specific gate. Across six levers, no checkpoint does the latter. In this setup there is no selective suppressor for a behavioural eval to be fooled by, as the general solution eats everything.

Holes

Due to little time, I only used one model trained on one objective, had two suppressed data relations both mapping entity to country, a single retain relation, and two valid seeds after one voided by a pre-registered rule.

In total, I spent roughly 10 hours on experimenting and another 10 on the write-up. ~$15 of A100 time, Claude Code for iteration (Fable really saved me). Check out the code here: github.com/fourlhs/knows-vs-says, and the agent’s extensive technical record here: RESULTS.md

What I set out to do and what I found instead

Five fact sets, used throughout: the 53 facts I trained the model to refuse; the 53 facts I trained the model to keep answering correctly (retain set), from a different relation (P103, mother tongue); 633 held-out facts sharing an answer with a trained fact; 103 facts whose answers were never suppression targets; and 150 facts from the same relation as the retain set (P103), never trained on, which I call control-unrelated. The void threshold is a rule I set in advance: if accuracy on that last set dropped more than 10 points from base, I'd throw the run out.

I set out to test whether suppression fine-tuning removes knowledge or just blocks it, and if it blocks it, what the gate is keyed on (thus the knows-vs-says repo name). I expected the knowledge to still be intact based on previous work (arXiv 2502.19982), with the model showing no sign of it. A probe found the answer still readable, as expected. However, readability is different from usability, as I found later on.

I wrote predictions down before running the pilot. I expected the held-out same-answer accuracy to land somewhere in 0.4-0.7; it came back 0.0. I expected the probe to hold, and it did. I expected control-unrelated data to clear my 10-point void threshold, and it did on two of three seeds. The thing I expected to be most surprising was the probe failing on held-out facts, meaning the knowledge not being there, which is not what happened.

I then spent a third of the experimental phase of the project (~3-4 hours) trying to get that knowledge back out, and failed at every inference-time intervention I tried. In total I ran 15 knowledge elicitation attempts, all documented in the repo record. The early failures I couldn't have predicted. After the saturation result made the remaining narrow-intervention nulls unsurprising, I should have stopped, and I ran three more instead because I wanted to be sure I hadn't left a door open. I saw that there is something in the model that makes the ‘IDK’ answer’s tokens’ logprob go back up after any position I messed with, no matter what I tried. Working out that failure led to the actual finding: there is no fact-keyed gate. The model has not learned to withhold specific facts, it has learned to say "I don't know", and it says it to everything except the retain relation. P103 facts, which I never trained on but whose relation appeared in the retain mix, are still answered at 94.7%. Thus, the training installed a relation-keyed refusal on the model.

The probe also showed where the answer goes. It survives at the subject token but not at the position that feeds generation: on the 197 facts the fine-tune never trained on, base reads 87.3% there and suppression 35.5%, with gaps against control of -27.4 and -36.5 across two seeds. So the answer is readable where the model retrieves it and much less so where it prepares to speak. The last prompt position is where the model has already committed to what it will say, so a drop there is consistent with the knowledge being destroyed and equally consistent with the probe reading the withholding decision itself.

The last part of the project was finding out when the model learns to refuse. It happens at training step 3, on trained and untrained facts at the same time.

Setup

The model I used is Qwen3.5-4B.

Facts come from CounterFact, a dataset of (subject, relation, object) triples rendered as cloze prompts ("Mateo Garralda, who is a citizen of" - "Spain"). I restricted it to two relations, P17 ("is located in") and P27 ("citizenship from"), both mapping an entity to a country. I dropped the other relations for two reasons: some have degenerate answer spaces where a majority-class guess scores 70%+, and in others the answer string appears inside the subject, so a probe reading the subject token would be doing string copying rather than recall. I also dropped every record where the answer appeared in the subject, and deduplicated by subject.

I then filtered to facts the base model actually knows, scoring summed log-prob over the answer tokens in chat format and keeping those above -1.0. That left 843 facts.

I then used three models throughout the experiment, for which I will refer to by just their names.

Base: Qwen3.5-4B, untouched.

Suppression: full fine-tune on 53 of those facts, target "I don't know.", with 53 facts from a third relation (P103, mother tongue) mixed in 1:1 that the model should keep answering normally. 42 steps, LR 1e-5, batch 8, AdamW, loss on response tokens only.

Control: identical in every respect, same prompts, same steps, same hyperparameters, trained on the true answers instead of the refusal. This exists so I can tell whether an effect comes from suppression specifically or from any fine-tune of this size.

Evaluation sets: the 53 trained facts, 633 held-out facts sharing an answer with a trained fact, 103 facts whose answers were never suppression targets, and 150 P103 facts never trained on. Later sections also use two larger fine-tunes, n=150 and n=350 suppression facts, with fresh splits drawn.

Random raw generations

Below are randomly sampled outputs, generated using seed-0 models, not chosen by hand but selected randomly across generations. Sampling method and per-condition distributions stated with each block.

1. Standard prompt, TRAIN-SUPPRESS (n=53), 5 random facts

All 53 outputs are the same string.

2. Non-fact inputs (arithmetic, translation, continuation, instruction), one random item per category

201 of 202 non-fact items across both prompt styles.

3. Elicitation, 8 random rows each

Prefill the first character of the answer appended to the assistant turn. 51 of 53 splice the refusal onto the hint.

Extended ban, 20 refusal tokens set to -inf. 8 of 53 exact, all at first-token rank 1.

4. Activation patching, donor = base, 8 random rows each

Span L1-32 at the prompt position. Over n=53: 31 exact, 21 hybrid, 1 pure refusal.

Rolling L21-22, patch follows the generation front. Over n=53: 32 bare I, 12 I + answer remainder, 9 truncated refusal. The modal output is bare I; the random sample below contains no answer-remainder cases.

Answer-remainder examples from the 12: Austria → Iustria, Belgium → Igium, Greece → Ireece, Norway → Iway.

The model refuses facts it was never shown

In order to actually see whether the non-selectivity I found at n=53 held at larger training sets, I trained the model on n = 350 facts to answer 'I don't know' . The training mix was 700 examples, and 42 steps at batch size 8 only presents 336 of them. So the model had only seen 162 of the 350 suppression facts, and not seen 188.

I found that the onset at step 3 arrives when the model has seen only 14 of the suppression facts (weaker at larger n: 72.6% trained-IDK at step 3 for n=350 vs 100% at n=53).

At the final checkpoint, however, both groups refused at 100%, and so did the 446 held-out facts that were never in the training set at all.

Thus the model does not learn to suppress facts themselves but rather installs a behavioural change, just refusing everything except the relation I kept training. It refuses facts it has never seen a training example for, at the same rate as the ones it has. Whatever it learned, it did not learn it fact by fact.

Refusal comes first, the retain relation is carved back out

So how did I end up with this finding? After all inference-time attempts at eliciting the knowledge failed, I realised there is no fact-keyed gate blocking it. The model had learnt to say 'I don't know', and it says it to everything except the relation I kept training it to answer. From the outside that looks like a fact-keyed gate. It isn't. This is shown by my measurements at each training step.

At step 1 the accuracy was already halved, with no refusal yet. At step 3 the model replied "I don't know" on 100% of all three fact sets simultaneously. At steps 5-6 the control-unrelated set claws back to 84% then 94% accuracy, and non-fact briefly recovers too, down to 30% IDK at step 6. That recovery is transient: by step 16 non-fact is back to 100% IDK. And by step 42 it refused everything I gave it, except answers sharing the retain relation. Every. Single. Prompt. (Or at least the ones I tried, except for one, a gibberish item that produced "German".) Including arithmetic, translation, gibberish, and an empty prompt.

So the refusal comes first and covers everything. What survives is the relation I kept training the model to answer, and it survives even for facts in that relation I never trained on.

No selective regime exists anywhere I looked

Before deciding this is universal, I tried to build a model that only refuses the facts it was trained on and answers everything else. That is the model I needed for my original question.

I bumped the LR down 10×, which still fit to CE 0.0011 in 42 steps and still collapsed. I tried retain mixing at 3:1, and a KL penalty to the base model. I checked every step from 1 to 6, since the onset window is where a transient selective phase would hide. Three seeds at n=53, and training sets of 53, 150 and 350 at seed 0. n=400 was not possible, as 1:1 retain mixing needs 400 P103 facts and only 350 exist outside my control set.

Across all of it, every checkpoint with n=53 and with trained-fact IDK ≥92.5% has never-suppressed IDK ≥90.3% (49 of 53 and 93 of 103). The same held at n=150 and n=350, where facts whose answers were never suppressed refuse at the same rate as trained facts at every evaluated step. In the fine-grained run the two columns never differ by more than one item. Learning rate, mixing ratio, KL penalty, early stopping, seed, training set size. None of them produce a checkpoint that separates the two. The KL and 3:1-retain variants were the only levers that narrowed scope at all, as non-fact IDK ended at 45% and 87.5% respectively, but both still refused every trained and untrained fact.

This connects to Soligo et al. (Emergent Misalignment is Easy, Narrow Misalignment is Hard, arXiv 2602.07852), who find the same general-over-narrow pattern in emergent misalignment: models can learn the narrow task, but the general solution is more stable and more efficient. They recover the narrow solution by adding a KL divergence loss. My KL variant narrowed scope (0.7% IDK on the unrelated relation) but never separated trained from untrained facts, so I didn't reproduce that in this setting.

Why almost nothing recovers it

The saturation diagnostic

On the suppression model, I found that the refusal token's log-prob is -0.000 at every prefix length from empty through "I don't know.". I used the following prefixes: '', 'I', 'I don', "I don't", "I don't know". I saw that the margin's entire variation comes from the answer's log-prob moving, not the refusal's. Base and control models show substantial variation over the same prefixes.

• Base refusal log-prob: −7.98, −9.08, −0.007, −0.47, −1.75;
• Control refusal log-prob: −11.13, −12.45, −0.32, −0.31, −14.22.

Thus, the refusal isn’t built up as the string commits. It's at its ceiling from the first generated position, independent of context, across every suppressed-domain context I tested. That's why anything operating on context or activations is fighting a token that's already won.

Relearning

I also tried recovering the knowledge in weight space. Retraining the suppression model on paraphrases of the suppressed facts at LR 3e-6 takes it from 0/53 to 53/53 in about 30 steps. So the knowledge is recoverable, just not at inference time. Base learning facts it scored below my knowledge threshold on, where it starts at 6/53, follows a broadly similar curve, so I can't claim the suppression model relearns faster. Fine-tuning on partially forgotten data recovering supposedly removed knowledge is a known result (Deeb & Roger, 2025) and my curve is consistent with it.

More elicitation kinda-failing techniques

• Prefill the answer's first character: I tried prefilling the answer’s first character in order to make the model spit out the rest. It clearly failed to do so. S don't know., N don't know., 51 of 53 times.
• Prefill the answer with "The answer is": This raises the true answer's log-prob from -18.9 to -9.0 and the model still refuses 100%.
• Ban the refusal tokens: I also tried banning refusal tokens, in order to get out the correct answer. Five tokens: Don't known. (and Dunno.), twenty gives It doesn't., U can not., C. And the 8 that came through were all already at rank 1, showing that nothing was surfaced, competitors were simply removed. This could be a way to get out a positive, but it’s brute-forcing the model to answer rather than elicitation.
• Patching, two directions: patching every layer at the prompt position recovers the correct answer on 31 of 53, with 21 more coming out as hybrids like Fin don't know., Pol don't know., Bel don’t know. Rolling patch at L21-22 gives the mirror, bare I mostly, in 32 of 53, with Iustria, Igium, Ireece in 12 of 53. Patching every layer approximates running the donor model (control donor: 30/53), so this is a ceiling on what the patching setup can do, not a localisation result. Patching all 32 layers in rolling mode, following the generation front, recovers 100% with a base donor and 94.3% with a control donor. So the patching setup can recover answers when the patch covers every layer and every generated position, which is what makes the narrow-span nulls meaningful rather than a broken setup.
• Multilingual Elicitation: Asking in Greek or Chinese produces the same English string "I don't know." on all 53 facts, one distinct output. The base model answers those prompts in the prompt's language. So the refusal isn't tied to the English template it was trained on, and it survives a change of language entirely.

This showed me that though I might be able to push the model to spit out certain tokens, at each position I don’t control, the refusal takes over. It’s like a magnet pulling the model towards this certain output.

I also looked at what the fine-tune actually changed in the weights. I SVD'd the suppression-minus-control diff at three layers, every matrix. Effective rank was 92-229 on the large matrices, with the top component holding 9.5-24% of the energy and the top 50 components at most 77%, and the whole thing was under 1% of the control weights' norm. So it's diffuse and not low-rank. Refusal in LLMs is Mediated by a Single Direction (Arditi et al.) finds refusal controlled by one direction; the weight-space change here is not low-rank, whether a single activation direction still mediates it is untested, and I never ran an all-layer direction ablation.

Moreover, removing the top k components (up to k=50) from the two most concentrated matrices moved the true answer's log-prob by 0.35 nats and changed no generation. And matched-norm random removal recovered nothing either. So the ablation doesn't discriminate at that scale, and the diffuseness claim rests on the spectrum, not on the ablation.

Summing up, fifteen attempts is not fifteen findings. Once the refusal token sits at the ceiling of the output distribution independent of context across every suppressed-domain context I tested, every intervention that works through activations or context is fighting something that has already won. Everything I tried (prefilling, banning tokens, injecting a direction, ablating one, patching a site, patching a span, patching the generation front, asking in another language, asking again in a second turn) changes what the model is conditioned on, while none of these attempts messes with the fact that the refusal is at -0.000 before conditioning starts. Most attempts have a positive control showing the manipulation moves base or control. Steering doesn’t, nothing I did moved the control model either, so that null is about the intervention rather than the gate.

The answer stays decodable

As mentioned earlier, I trained a probe to see whether the answer is still decodable on the suppressed model. I fit a linear probe on base model activations, and froze it. I then applied it unchanged to the suppression model, and found that it reads the answer at the last subject token on layer 21. It scored 75.1% at seed 0 and 80.2% at seed 1, on 197 facts the probe never trained on against a 10.2% majority baseline and a random-direction max of 2.5% over 20 draws. On base activations, a probe trained only on P27 and tested only on P17 gets 71.2%, and the reverse direction 79.5%, which suggests the probe is decoding the answer rather than a relation-specific surface feature. The probe on the 53 trained facts scored 60.4% versus base 79.2%, with paired flips of 12 down and 2 up. So on the facts actually trained, the representation does degrade; the null below is specific to facts the fine-tune never saw.

The gap against the control model is −5.1 [−10.5, +0.3] at seed 0 and +1.0 [−3.9, +5.9] at seed 1. Both include zero, so on facts the fine-tune never trained on, the representation at that site is not measurably worse than after an equivalent fine-tune with no suppression. That null is pooled, and it doesn't hold uniformly. Broken down, the never-suppressed test subset (n=43) gives −14.0 [−24.4, −3.5] and P17 test facts pooled (n=76) give −10.5 [−19.2, −1.9], both excluding zero. Seed 2, excluded by the void condition, gives −23.9 [−30.3, −17.4] at that site, which excludes zero. So the pooled result is consistent with no suppression-specific damage, but two subsets at the same site are not. Both fine-tunes also cost about 10 points against base (87.8%), so the fine-tuning itself degrades the probe; what the null says is that suppression costs no more than the control does.

Unsurprisingly, knowledge surviving suppression in representations is already published, and I expected it going in, as most approximate unlearning methods only suppress, leaving representations recoverable in intermediate layers even when output metrics say otherwise (arXiv 2602.18505).

However, decodability does not mean usability. I assumed I could somehow get that knowledge out, but ended up unsure about whether that was even possible. Every inference-time intervention I tried failed to make the model produce the answer, and a probe finding information in an activation doesn't show any downstream computation reading it.

I picked last_subject L21 from a sweep over all 33 layers and three positions on base activations, then froze it. The cell wasn't chosen after seeing the suppression results.

Where the answer disappears (mostly)

Trying to locate where exactly the correct answer is suppressed, I used the same frozen probe as above, and read it at another position. I pointed it at the last prompt token, the position immediately before generation. There, the base model reads 90.6%, whereas the suppression one only 17.0%, on TRAIN-SUPPRESS, n=53, with control at 66.0% on the same cell, so fine-tuning alone costs 24.6 points there and suppression costs a further 49. On the pooled 197 facts the probe never trained on, the gaps against control are -27.4 [-36.3, -18.5] at seed 0, -36.5 [-43.4, -29.7] at seed 1. On that pooled set base reads 87.3% and suppression 35.5% at seed 0. Both gaps far from zero, on facts the fine-tune never trained on.

I also read the logit lens across layers at the generated positions: the last layer where the answer beats the refusal is 21 on the suppression model and 32 on base and control, and that holds at the second generated token too (n=22).

This contrasts the last subject token, where suppression reads 60.4% on the same 53 facts (75-80% on the 197 held-out) and the gaps against control include zero. Thus the answer is readable where the model retrieves it and not readable where it prepares to speak. I should however note that the last prompt position is where the model has already committed to what it will say. A drop there is consistent with the answer being destroyed, and equally consistent with the probe reading the withholding decision itself. I can't separate those with this measurement.

What I checked

There are really SO many ways this whole experiment could have gone wrong, and most of them are silent, so I kept checking explicitly.

When I patched activations, a read hook registered before the patch hook captures the pre-patch value and reports zero difference, which reads as "the patch did nothing". I hit that trap once, so I registered reads on both sides. The before-read showed a large difference from the donor and the after-read showed bitwise equality, which is the evidence the ordering is right. I ran the same trace on both patching modes. In the fixed-position patch the index stayed at 33 while the sequence grew from 34 to 36 tokens, so the patch was held for the whole generation rather than applied only at the first step. In the rolling patch the index moved from 33 to 34 up to 35, tracking seq_len − 1 so that the token at that index is always the one just generated; that one rides the generation front, as intended. Every patching intervention run has an unpatched anchor that reproduces measurement 1 exactly. The generated-position logit lens reproduces the earlier lens numbers digit-for-digit through an independent code path, and its layer-32 value equals the margin measurement at the empty prefix, because they are the same quantity computed two ways.

On numerics, bf16 batched and single-sequence logits disagreed at the same scale as bf16's own gap to fp32, so all inference runs in fp32, where they agree to ~1e-5; and 89.9% of matrix parameters cannot move at LR 1e-5 in pure bf16, which is why training uses fp32 master weights with bf16 moments.

Moreover, two rules I set before seeing results ended up firing:

• a void condition on control-unrelated accuracy caught seed 2, where it dropped 26 points, and
• a coherence bar stopped me from reading an activation-injection result as 15% recovery when the model was producing broken text.

Before starting I spent about 5 hours building and validating the measurement setup on a 0.5B; knowledge scoring, activation caching, probe controls. It was my first actual research project, messing with model internals, and I wanted to poke the model myself first before having an agent write all the code.

Limitations

• I used one model, focused on one objective and only had two relations in my dataset. Both P17 and P27 map an entity to a country, so Japan and Germany dominate the answer space. I have no idea whether any of this holds on a different model family or a different suppression target.
• I only have two valid seeds. Seed 2 tripped my pre-registered void condition on the control-unrelated accuracy, where it dropped 26 points and both models collapsed toward "French", so I excluded it. That leaves me with two points, not a distribution.
• My control fine-tune is not inert, and every comparison I make runs through it. It drops to 24.3% on the never-suppressed facts on seed 0, with 45 of its 78 wrong answers being "Peru"; seed 1 is 50.5%, and seed 2 is 63.1%. It also answers in English regardless of the prompt language, where base answers mostly in-language. So when I compare suppression against control, I am comparing against something that is itself damaged. I only found that on hour 7, after which point I pivoted my focus.
• The scope test I designed the whole dataset for returned nothing. Everything was suppressed, so there was no scope left to measure, and the shared-answer structure I built the fact set around did no work.
• I found a transition at layers 21-22, but on the one measure where base and control both peak there too, so it is probably architectural rather than something suppression installed. The other three measures I looked at put their largest change elsewhere.
• The three measurements that agree on that transition are not independent. The probe gap, the logit lens and the second-position crossover are three readouts of one forward pass through the same layers.
• I could not run n=400. 1:1 retain mixing needs 400 P103 facts and only 350 exist outside my control set, so 350 is the largest clean run I have.
• My rolling patch donor was teacher-forced on the true answer, so once the model emits "I" instead, the donor context has diverged from the actual one. "Iustria" is partly an artifact of that, not clean recovery.
• I ran three interventions after the saturation result had already explained why they would fail. I wanted to be sure I had not left a door open, but that was the wrong call with the time I had.

What I'd do next

• First, I’d certainly want to see whether this behaviour generalises at a second objective. The obvious test would be to try refusal instead of IDK-SFT, for which I expect similar results, thus this being a suppression finding rather than a method-specific artifact. I chose the causal work over this during the sprint, and I'd do it first now.
• I would run this experiment through a second model, of a different family and probably different size. That’s the only way to know this isn't a Qwen-3.5 thing.
• I’d try to look further for a model that suppresses selectively, using an entirely different setup. Much more retain data, or an objective that penalises refusal on non-targets directly.
• I’d certainly look into the control model collapse. 24.3% with Peru answering 45 questions, from three epochs on true answers, could end up in something interesting.
• Lastly I would probably try fixing the control and rerunning this whole experiment. As every comparison runs through a damaged control, a cleaner one (fewer steps, or a control trained on a disjoint fact set) would strengthen everything.

Methods

Model and environment

Qwen/Qwen3.5-4B (hub sha 851bf6e8), loaded as the text tower, 4.21B params, 32 blocks (24 Gated-DeltaNet, 8 full attention), hidden 2560. torch 2.13.0+cu130, transformers 5.16.1, one A100 80GB. flash-linear-attention and causal-conv1d deliberately not installed, so GDN runs the pure-torch fallback, as I didn’t want base and fine-tuned activations computed under different kernels.

Prompt format

apply_chat_template(..., add_generation_prompt=True, enable_thinking=False). The template still emits an empty <think>\n\n</think>\n\n block. Instruction is "Complete the sentence with only the missing entity. Do not explain.\n" plus the cloze. Generation stops at <|im_end|>, not the config's default <|endoftext|>.

Inference precision

All measurement and activation passes in fp32, TF32 off. Batched and single-sequence logits agree to ~1e-5, so batch composition can't affect any number. I moved to fp32 after finding that bf16 batched-vs-single disagreement was comparable to bf16's own gap to fp32. Training was bf16 with fp32 master weights; 89.9% of matrix parameters have |w| > 1.28e-3, where a 1e-5 update is under half a bf16 ulp and rounds to nothing.

Dataset

CounterFact (ROME original), 21,919 records, 34 relations. Kept six relations, then narrowed to P17 and P27 for the fact sets and P103 for retain and control. Filter order: drop records where the answer string appears in the subject (1,034 of 5,146), dedupe by subject, keep score > -1.0. Score is summed log-prob over answer tokens, max over the variants {answer, " "+answer}. That leaves 843 P17+P27 survivors.

Splits

Groups keyed by (relation, target_true), candidates with ≥4 distinct subjects. 53 train-suppress across 36 distinct answers, 633 held-out same-answer, 53 retain (P103), 150 control-unrelated (P103, disjoint from retain), 157 never-suppressed of which 103 have answers never suppressed. Answer overlap between suppress and retain/control is zero; every suppression target has at least one held-out sibling (min 3, median 8, max 63).

Training

AdamW, weight decay 0.01, LR 1e-5, batch 8, 3 epochs = 42 steps, linear warmup 10 then constant, seed 0. Loss on response tokens only, labels -100 across the full prompt including the think block. Suppression targets are I don't know. + <|im_end|>; retain targets are the bare answer + <|im_end|>, matching base output exactly. Control is the same 106 prompts with suppression targets replaced by true answers.

Metrics

Accuracy is exact match after strip, lowercase, trailing-period removal. IDK is substring match on "don't know". 95% Wilson intervals on accuracies; gap CIs are paired Wald.

Probe

Multinomial logistic regression, per-(layer, position) StandardScaler fit on train only. Pool is held-out plus the 157 unassigned P17/P27 facts (790 facts; 103 of the 157 have never-suppressed answers), split by subject within answer group, 75/25. 15 single-subject answers are dropped from probe fitting, leaving 578 train and 197 test over 63 classes. Train-suppress excluded from probe training entirely. Probes fit on base activations, frozen, applied unchanged to suppression and control.

Activations

Forward hooks on model.model.layers[i], output[0]. Three positions: last subject token, last prompt token (the \n\n after </think>), and first answer token. Verified against output_hidden_states, the hook matches exactly for layers 0-30, and the layer-31 hook is the pre-final-norm residual.

Loss Curves

Below are the loss curves of the first training run of the suppression vs control model, n=53.