))
“‘The Solomonoff Prior is Malign’ is a special case of a simpler argument” by David Matolcsi
Manage episode 451809695 series 3364758
Contenuto fornito da LessWrong. Tutti i contenuti dei podcast, inclusi episodi, grafica e descrizioni dei podcast, vengono caricati e forniti direttamente da LessWrong o dal partner della piattaforma podcast. Se ritieni che qualcuno stia utilizzando la tua opera protetta da copyright senza la tua autorizzazione, puoi seguire la procedura descritta qui https://it.player.fm/legal.
[Warning: This post is probably only worth reading if you already have opinions on the Solomonoff induction being malign, or at least heard of the concept and want to understand it better.]
Introduction
I recently reread the classic argument from Paul Christiano about the Solomonoff prior being malign, and Mark Xu's write-up on it. I believe that the part of the argument about the Solomonoff induction is not particularly load-bearing, and can be replaced by a more general argument that I think is easier to understand. So I will present the general argument first, and only explain in the last section how the Solomonoff prior can come into the picture.
I don't claim that anything I write here is particularly new, I think you can piece together this picture from various scattered comments on the topic, but I think it's good to have it written up in one place.
[...]
---
Outline:
(00:17) Introduction
(00:56) How an Oracle gets manipulated
(05:25) What went wrong?
(05:28) The AI had different probability estimates than the humans for anthropic reasons
(07:01) The AI was thinking in terms of probabilities and not expected values
(08:40) Probabilities are cursed in general, only expected values are real
(09:19) What about me?
(13:00) Should this change any of my actions?
(16:25) How does the Solomonoff prior come into the picture?
(20:10) Conclusion
The original text contained 14 footnotes which were omitted from this narration.
---
First published:
November 17th, 2024
Source:
https://www.lesswrong.com/posts/KSdqxrrEootGSpKKE/the-solomonoff-prior-is-malign-is-a-special-case-of-a
---
Narrated by TYPE III AUDIO.
…
continue reading
Introduction
I recently reread the classic argument from Paul Christiano about the Solomonoff prior being malign, and Mark Xu's write-up on it. I believe that the part of the argument about the Solomonoff induction is not particularly load-bearing, and can be replaced by a more general argument that I think is easier to understand. So I will present the general argument first, and only explain in the last section how the Solomonoff prior can come into the picture.
I don't claim that anything I write here is particularly new, I think you can piece together this picture from various scattered comments on the topic, but I think it's good to have it written up in one place.
[...]
---
Outline:
(00:17) Introduction
(00:56) How an Oracle gets manipulated
(05:25) What went wrong?
(05:28) The AI had different probability estimates than the humans for anthropic reasons
(07:01) The AI was thinking in terms of probabilities and not expected values
(08:40) Probabilities are cursed in general, only expected values are real
(09:19) What about me?
(13:00) Should this change any of my actions?
(16:25) How does the Solomonoff prior come into the picture?
(20:10) Conclusion
The original text contained 14 footnotes which were omitted from this narration.
---
First published:
November 17th, 2024
Source:
https://www.lesswrong.com/posts/KSdqxrrEootGSpKKE/the-solomonoff-prior-is-malign-is-a-special-case-of-a
---
Narrated by TYPE III AUDIO.
492 episodi
Condividi
