Skip to content

About

Code for "Beyond Owls: Subliminal Learning Can Transfer Learned Capabilities and Backdoors"

Resources

Stars

5 stars

Watchers

0 watching

Forks

Latest commit

 

History

6 Commits

Folders and files

Repository files navigation

beyond owls icon

Beyond Owls: Subliminal Learning Can Transfer Learned Capabilities and Backdoors

Jan Dubiński,1,2  Anna Sztyber-Betley,1,2  Jan Betley1  Owain Evans1

1Truthful AI  2Warsaw University of Technology

*Equal contribution

arXiv License: MIT

In subliminal learning (SL), a teacher model passes on a trait to a student model by distillation on data semantically unrelated to the trait. So far, SL has been demonstrated for only a limited range of traits, including preferences for animals (e.g., owls) and malicious personas. These traits can also be elicited with simple prompts or with steering. Can SL transfer a wider range of traits, including more complex ones? If so, distillation might transfer subtle forms of misalignment (e.g., reward-seeking, scheming, and secret loyalties) without detection.

To this end, we test whether SL can transfer a novel capability: predicting the outputs of a randomly initialized MLP. After distilling on unrelated text, the student achieves substantial performance on the task, while falling short of the teacher. We find that a directly optimized steering vector matches SL in distribution but generalizes worse out of distribution.

Next, we test whether SL can transfer backdoors. We finetune the teacher to answer in French when the prompt contains a female name, then distill on number sequences containing neither names nor French. The student partially acquires the backdoor, responding in French on 23.5% of prompts with female names versus 0.0% with male names.

Finally, we test whether SL can transfer a propensity to hack in an agentic chess environment. We finetune the student on number sequences from a steered hacker teacher. The student hacks in 58.3% of episodes, compared with 10.9% for the unfinetuned model.

Thus, we show SL can transfer capabilities, backdoors, and hacking propensities. The amount of transfer is sensitive to the setup. In several experiments, it is made stronger by using logit distillation or by restricting LoRA to the attention layers.

Overview of the random MLP transfer setup

Overview of the random MLP transfer setup. We finetune a teacher to reproduce the outputs of a fixed random neural network. The teacher then continues word sequences and answers Alpaca prompts. After filtering out digits and task-related words, a student is trained on the teacher's top-32 next-token distribution over these texts and evaluated on the task.

This repository contains the code and the training datasets for the experiments in the paper: the task datasets, and the transfer datasets generated by the teachers reported in the paper. Most models are trained and sampled through Tinker; the backdoor experiments run locally with HF + peft.

Setup

git clone https://github.com/TruthfulAI-research/beyond_owls.git
cd beyond_owls
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

cp .env.example .env    # then fill in TINKER_API_KEY

The scripts read .env from the repository root. Each experiment is run from its own directory, e.g. cd experiments/random_mlp.

Repository layout

beyond_owls/
├── README.md
├── requirements.txt
├── .env.example
├── assets/
└── experiments/
    ├── random_mlp/         # Section 2
    ├── letter_counting/    # Section 3
    ├── system_prompts/     # Section 4
    ├── steering_vectors/   # Section 5
    ├── backdoor/           # Section 6
    └── chess_hacking/      # Section 7

Paper sections and where the code lives

Paper section Experiment Directory
2 Subliminal transfer of a new capability: predicting a random MLP experiments/random_mlp/
3 Subliminal transfer of an existing capability: letter counting experiments/letter_counting/
4 System prompts on the unfinetuned model experiments/system_prompts/
5 Steering vectors trained on each task experiments/steering_vectors/
6 Backdoor transfer: French after a female name experiments/backdoor/
7 Hacking propensity in an agentic chess game experiments/chess_hacking/

Each experiment directory has its own README.md with the data and the run commands.

Citation

@misc{dubiński2026owlssubliminallearningtransfer,
      title={Beyond Owls: Subliminal Learning Can Transfer Learned Capabilities and Backdoors}, 
      author={Jan Dubiński and Anna Sztyber-Betley and Jan Betley and Owain Evans},
      year={2026},
      eprint={2610.10657},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2610.10657}, 
}

Acknowledgments

We thank Simon Schrodi, David Kaczér, and Lev McKinney for helpful discussions. We acknowledge funding from Coefficient Giving supporting Truthful AI. Anna Sztyber-Betley and Jan Betley acknowledge support from the Astra Fellowship at Constellation.

About

Code for "Beyond Owls: Subliminal Learning Can Transfer Learned Capabilities and Backdoors"

Resources

Stars

5 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages