One thing Nick Bostrom doesn’t mention in this short video, that he writes about extensively in his book, Superintelligence: Paths, Dangers, Strategies, is that the development of AGI should be air-gapped from the internet (or any network). Is this the case with OpenAI and ChatGPT? Abso-fuckin lutely not. The training data for ChatGPT is the internet. That includes all the misinformation, racism, and cognitive biases incorporated into content found on the internet.

This is the best primer (warning – Long Read) about how current iterations of LLM’s work – https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/

Last comment is straight out of sci-fi and one that I haven’t heard anyone mention in the current hysteria. In the book Hyperion, AGI exists, uses human brains for processing power… and ultimately decides to “secede” from human affairs to focus on “other things”. The point being, if Superintelligence arrives, we should consider that it may not want to have anything to do with humans.