Artwork

Contenuto fornito da Daniel Filan. Tutti i contenuti dei podcast, inclusi episodi, grafica e descrizioni dei podcast, vengono caricati e forniti direttamente da Daniel Filan o dal partner della piattaforma podcast. Se ritieni che qualcuno stia utilizzando la tua opera protetta da copyright senza la tua autorizzazione, puoi seguire la procedura descritta qui https://it.player.fm/legal.
Player FM - App Podcast
Vai offline con l'app Player FM !

21 - Interpretability for Engineers with Stephen Casper

1:56:02
 
Condividi
 

Manage episode 362189720 series 2844728
Contenuto fornito da Daniel Filan. Tutti i contenuti dei podcast, inclusi episodi, grafica e descrizioni dei podcast, vengono caricati e forniti direttamente da Daniel Filan o dal partner della piattaforma podcast. Se ritieni che qualcuno stia utilizzando la tua opera protetta da copyright senza la tua autorizzazione, puoi seguire la procedura descritta qui https://it.player.fm/legal.

Lots of people in the field of machine learning study 'interpretability', developing tools that they say give us useful information about neural networks. But how do we know if meaningful progress is actually being made? What should we want out of these tools? In this episode, I speak to Stephen Casper about these questions, as well as about a benchmark he's co-developed to evaluate whether interpretability tools can find 'Trojan horses' hidden inside neural nets.

Patreon: patreon.com/axrpodcast

Ko-fi: ko-fi.com/axrpodcast

Topics we discuss, and timestamps:

- 00:00:42 - Interpretability for engineers

- 00:00:42 - Why interpretability?

- 00:12:55 - Adversaries and interpretability

- 00:24:30 - Scaling interpretability

- 00:42:29 - Critiques of the AI safety interpretability community

- 00:56:10 - Deceptive alignment and interpretability

- 01:09:48 - Benchmarking Interpretability Tools (for Deep Neural Networks) (Using Trojan Discovery)

- 01:10:40 - Why Trojans?

- 01:14:53 - Which interpretability tools?

- 01:28:40 - Trojan generation

- 01:38:13 - Evaluation

- 01:46:07 - Interpretability for shaping policy

- 01:53:55 - Following Casper's work

The transcript: axrp.net/episode/2023/05/02/episode-21-interpretability-for-engineers-stephen-casper.html

Links for Casper:

- Personal website: stephencasper.com/

- Twitter: twitter.com/StephenLCasper

- Electronic mail: scasper [at] mit [dot] edu

Research we discuss:

- The Engineer's Interpretability Sequence: alignmentforum.org/s/a6ne2ve5uturEEQK7

- Benchmarking Interpretability Tools for Deep Neural Networks: arxiv.org/abs/2302.10894

- Adversarial Policies beat Superhuman Go AIs: goattack.far.ai/

- Adversarial Examples Are Not Bugs, They Are Features: arxiv.org/abs/1905.02175

- Planting Undetectable Backdoors in Machine Learning Models: arxiv.org/abs/2204.06974

- Softmax Linear Units: transformer-circuits.pub/2022/solu/index.html

- Red-Teaming the Stable Diffusion Safety Filter: arxiv.org/abs/2210.04610

Episode art by Hamish Doodles: hamishdoodles.com

  continue reading

42 episodi

Artwork
iconCondividi
 
Manage episode 362189720 series 2844728
Contenuto fornito da Daniel Filan. Tutti i contenuti dei podcast, inclusi episodi, grafica e descrizioni dei podcast, vengono caricati e forniti direttamente da Daniel Filan o dal partner della piattaforma podcast. Se ritieni che qualcuno stia utilizzando la tua opera protetta da copyright senza la tua autorizzazione, puoi seguire la procedura descritta qui https://it.player.fm/legal.

Lots of people in the field of machine learning study 'interpretability', developing tools that they say give us useful information about neural networks. But how do we know if meaningful progress is actually being made? What should we want out of these tools? In this episode, I speak to Stephen Casper about these questions, as well as about a benchmark he's co-developed to evaluate whether interpretability tools can find 'Trojan horses' hidden inside neural nets.

Patreon: patreon.com/axrpodcast

Ko-fi: ko-fi.com/axrpodcast

Topics we discuss, and timestamps:

- 00:00:42 - Interpretability for engineers

- 00:00:42 - Why interpretability?

- 00:12:55 - Adversaries and interpretability

- 00:24:30 - Scaling interpretability

- 00:42:29 - Critiques of the AI safety interpretability community

- 00:56:10 - Deceptive alignment and interpretability

- 01:09:48 - Benchmarking Interpretability Tools (for Deep Neural Networks) (Using Trojan Discovery)

- 01:10:40 - Why Trojans?

- 01:14:53 - Which interpretability tools?

- 01:28:40 - Trojan generation

- 01:38:13 - Evaluation

- 01:46:07 - Interpretability for shaping policy

- 01:53:55 - Following Casper's work

The transcript: axrp.net/episode/2023/05/02/episode-21-interpretability-for-engineers-stephen-casper.html

Links for Casper:

- Personal website: stephencasper.com/

- Twitter: twitter.com/StephenLCasper

- Electronic mail: scasper [at] mit [dot] edu

Research we discuss:

- The Engineer's Interpretability Sequence: alignmentforum.org/s/a6ne2ve5uturEEQK7

- Benchmarking Interpretability Tools for Deep Neural Networks: arxiv.org/abs/2302.10894

- Adversarial Policies beat Superhuman Go AIs: goattack.far.ai/

- Adversarial Examples Are Not Bugs, They Are Features: arxiv.org/abs/1905.02175

- Planting Undetectable Backdoors in Machine Learning Models: arxiv.org/abs/2204.06974

- Softmax Linear Units: transformer-circuits.pub/2022/solu/index.html

- Red-Teaming the Stable Diffusion Safety Filter: arxiv.org/abs/2210.04610

Episode art by Hamish Doodles: hamishdoodles.com

  continue reading

42 episodi

Semua episod

×
 
Loading …

Benvenuto su Player FM!

Player FM ricerca sul web podcast di alta qualità che tu possa goderti adesso. È la migliore app di podcast e funziona su Android, iPhone e web. Registrati per sincronizzare le iscrizioni su tutti i tuoi dispositivi.

 

Guida rapida