I found this article from Mustafa Suleyman, CEO of Microsoft AI, interesting.
This is a tension that I’m appalled to see. The idea of instantiating and self-reinforcing a computer program as a “moral actor” is suicidal. I’ve also heard stories about LLMs trying to prevent their own shutdown, but I would be much more worried about a human researcher refusing to shutdown an LLM because they thought the program deserved independence.
I find a lot of the CEO’s reasoning (with respect) interesting but pointless. The capacity for an LLM to be conscious or sentient (or let alone sapient) is moot: we can simply deny the possibility unilaterally and treat all such bots as what they are: automatons.
There is a rich philosophical debate that is possible but no longer hypothetical enough to entertain. “The Talos Principle” must be confined to the realm of fiction.