ML Safety Report

Join the hackathon GatherTown here: https://app.gather.town/app/iEPL2kx1L...
Or read more about the event here: https://itch.io/jam/llm-hackathon

👩‍🔬 This week's Safe AI Progress Report puts the focus on the Future Fund's new prize, Conjecture's new research, speed priors, and much more. Come along!

Chapters:
00:00 Intro
00:16 $1.5 million prize
01:00 Conjecture's interpretability research
01:50 Anthropic's superposition experiments
02:30 Inverse Scaling first round prizes
03:08 Forwarding speed regularizers
04:00 Reward is not the optimization target
04:25 Other news
04:52 🎉 Apart Hackathon
05:08 Israel AI Safety
05:20 Outro

Sources:
Future Fund world view competition: https://ftxfuturefund.org/
Strong general AGI soon: https://forum.effectivealtruism.org/p...
Polytopes lens: https://www.alignmentforum.org/posts/...
Anthropic’s papers: https://www.anthropic.com/research
Toy models of superposition: https://transformer-circuits.pub/2022...
Inverse scaling prize round 1: https://www.alignmentforum.org/posts/...
Inverse scaling prize: https://github.com/inverse-scaling/prize
Speed prior and forwarding speed priors: https://www.alignmentforum.org/posts/...
Are minimal circuits deceptive?: https://www.lesswrong.com/posts/fM5ZW...
Musings on the speed prior: https://www.alignmentforum.org/posts/...
Deconfusing wireheading: https://www.alignmentforum.org/posts/...
Reward is not the optimization target: https://www.alignmentforum.org/posts/...
Nearcasting AGI: https://www.alignmentforum.org/posts/...
7 traps new alignment researchers drop into: https://www.lesswrong.com/posts/h5CGM...
Language model hackathon: https://itch.io/jam/llm-hackathon
AI Safety Israel conference: https://aisic2022.net.technion.ac.il/
Apart Research: https://apartresearch.com
AI Safety Ideas: https://aisi.ai