dump of various nondistilled ideas. ai safety is a collaborative field a lot of these ideas aren’t mine per se. some of these docs could or already have aged quickly. hypotheses improve retroactively

circuit interp but better (july 2026-now)

this seems pretty important, especially (semi-)automated discovery. it could be the best way to chunk models apart into generalizations and reason about implications and stuff, especially if you know how many features can interact at a time on other features. so far im thinking of how to do this type of interp in less streetlight-y ways. “circuit utility” is basically some good desiderata for locating circuits. a major component for me was getting some metric like CaSc, but for automatically gathered ensembles of feature circuits (though I first thought you could literally use CaSc here, but found the method does not literally apply here). another way to be less streetlight-y is to collect information about circuits you don’t expect. this led me to become more satisfied with deep causal transcoding, and i’m interested what related approaches are out there

some of this work also happens to run alongside assumptions of shallow circuits actually being real, which so far, comprehensive evidence does not seem to exist despite many reasonable-sounding arguments. i think if transformers and neural nets do naturally chunk into ensembles of circuits for important computations, it would be a pretty big surprise given all the circuit interp done assuming sequential hypotheses (at least, thats what was being assumed back in 2022/3), and it could make it easier to outperform sampling of behaviors

• circuit utility
  • deep causal transcoding as an approach with good circuit utility
  • circuit utility update 2 (minor technical skill issue)
  • dcts as well-compressed partial explanations
• shallow weight circuits?

metacognitive skills (may-jul 2026)

i tried to build up a model for improving metacognitive skills as an anti-slop intervention, i.e. differential progress on improving epistemological reasoning with AIs. I was especially leaning on the hypothesis that this is an elicitation problem, but we also need to characterize these currently tail behaviors better, so took an interp-oriented perspective on this problem

fwiw i no longer care about metacognition. the theory of change was conceptually unclear to match to llms, the methods seemed too low-level, and it probably speeds up capabilities faster than getting meaningful safety work from them (like much other work on aligning llms). i also think the taxonomy of paths for metastrategic reasoning is a bit too galaxy-brained; like it revolves a lot around persona-space when multi-agent RL could work just as good.

• metacognition (and latent capabilities)¹
  • low probability estimation
  • a model of llm motivations
• reflection on recent introspection results²
  • a taxonomy
  • introspection is a different and less useful desideratum than metacognition

miscellaneous stuff

• what’s going on with chain of thought
• gestalt of the public on ai