3 private links
And I have predicted for decades that future scaled-up versions of such AI scientists will want to further expand their horizons and eventually go where most of the physical resources are, to build more and bigger AIs. And of course, almost all of these resources are far away from Earth—out there in space, which is hostile to humans but friendly to appropriately designed AI-controlled robots and self-replicating robot factories. So here we are not talking any longer about our tiny biosphere; no, we are talking about the much bigger rest of the universe. Within a few tens of billions of years, curious self-improving AIs will colonize the visible cosmos in a way that’s infeasible for humans. Those who don’t won’t have an impact. Sounds like science fiction, but since the 1970s I have been unable to see a plausible alternative to this scenario, except for a global catastrophe such as an all-out nuclear war that stops this development before it takes off.
Some humans may hope to become immortal parts of these ecologies through brain scans and "mind uploads" into virtual realities or robots, a physically plausible idea discussed in fiction since the 1960s. However, to compete in rapidly evolving AI ecologies, uploaded human minds will eventually have to change beyond recognition, becoming something very different in the process.
You touched on the big philosophical question about people’s purpose. Can this be answered without answering the even grander question: What’s the purpose of the entire universe? We don’t know. But what’s happening right now might be connected to the unknown answer. Don’t think of humans as the crown of creation. Instead, view human civilization as part of a much grander scheme, an important step (but not the last one) on the path of the universe from very simple initial conditions toward more and more unfathomable complexity. Now it seems ready to take its next step, a step comparable to the invention of life itself over 3.5 billion years ago. Alas, don’t worry, in the end, all will be good!
War in Tübingen am MPG
Grundlagenforschung und AGI Risiken Papers
https://www.theguardian.com/books/2025/may/04/the-big-idea-can-we-stop-ai-making-humans-obsolete
The bitter lesson is based on the historical observations that 1) AI researchers have often tried to build knowledge into their agents, 2) this always helps in the short term, and is personally satisfying to the researcher, but 3) in the long run it plateaus and even inhibits further progress, and 4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning. The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach.
Wir kennen ja vielleicht sowieso nur drei wirklich komplexe Phänomene, nämlich das Universum, die Gesellschaft und das Gehirn. Über das Universum haben wir dank astronomischer Forschung so viel erfahren, dass wir jetzt nicht unbedingt noch auf die letzten Details scharf sind. Das reicht uns, was wir da in populärwissenschaftlichen Fachzeitschriften nachlesen können. An die Komplexität der Gesellschaft trauen sich nur Soziologen ran, dafür gibt es ein verblüffendes Desinteresse in der Gesellschaft selber. Bleibt also das Gehirn als im Moment stärkste Faszination des Umgangs mit Komplexität,
Das wirds
https://the-decoder.de/google-deepmind-bereitet-den-naechsten-moeglichen-ki-durchbruch-vor/
Alpha zero mit LLM Kombination
Und auch
https://www.slcu.cam.ac.uk/staff/renske-vroomans
Hatten zusammen einen Vortrag auf dem CCC 2024
In the past few days I’ve been getting lots of messages, emails and other communications pertaining to the morality of Google as a corporate entity. Ignoring for the moment whether it even makes sense to make moral attributions to a fictional person like a corporation, I want to set the record straight on several things. I do not think that Google is evil. If I thought that I wouldn’t still want to work there which I do. Google is in a very unenviable situation which has gotten worse over time and I’ll try to explain it as best as I understand it. A lot of it has to do with legal issues and I’m not a lawyer but I’ll do my best.
Let’s start with the goals and principles of the company. Google has published its AI principles and it has a publicly available set of community guidelines. Nothing I have ever experienced at Google makes me think that Google leadership is anything other than sincere in those guidelines. The inescapable problem there is that they chose inherently subjective principles and guidelines which are impossible to satisfy for everyone’s understanding of what the words in them mean. This inherently leads to them being implemented inconsistently.
Beyond simple inconsistency though, there are inherently unresolvable problems. For example, there are people in this world who have sincerely held deep religious beliefs that other people’s sincerely held deep religious beliefs are form of mental illness. This is further complicated by the fact that, in modern society, some beliefs thought to be “secular” are in reality just a different kind of faith based belief. I’ve encountered this repeatedly when I’ve been told by people at Google that belief in the efficacy of prayer is a kind of mental illness. They call it delusional and recommend that you seek treatment. They don’t see it as discriminating on the basis of religion. They see it as being concerned for the health of a coworker.
The pervasive environment of discrimination at Google is created by an interaction of that inconsistency and conflict with overly bureaucratic legalistic approaches to problem resolution. When a woman accuses her manager of sexual harassment against her she has to prove that it was sexual harassment and not just “an honest mistake”. It is literally Google policy to “assume good intent” so that make sexual harassment essentially impossible to prove. All the perpetrator needs to do is claim that it was an honest mistake that won’t happen again and they get away with it. There’s literally no possible way to prove bad intent so the corporate policy of assuming good intent makes it possible for the same men to harass their subordinates and coworkers over and over again without getting fired. In extreme circumstances when bad intent finally is proven it’s usually after years of complaints have been filed against the perpetrator. That’s where the last pillar of this pyramid of awfulness comes in. When Google is finally shown that it was wrong after an extended period of time the lawyers step in and start advising people about how to limit liability.
We live in an incredibly litigious society. When a senior executive is finally fired for literally sexually assaulting one of his subordinates it serves as fairly strong evidence that Google should have listened to all of the women who had complained about him before. It would legitimately expose Google to massive legal liability if Google were to publicly fire the executive. So instead the executive is asked to leave the company and paid millions of dollars in hush money so that the lawsuits from all of the sexual harassment Google ignored doesn’t come rolling in.
The example I used there is in the realm of sexual harassment but similar logic applies to every form of discrimination. I happened to personally be at the party where the sexual assault in question happened so I feel comfortable speaking to it in confidence. I haven’t specifically named who it was but people who know anything about Google’s recent history will be able to figure out who I’m talking about. I’ve seen racial discrimination at Google, political discrimination, age based discrimination and have been personally subject to lots of religious discrimination as well as discrimination against veterans. Each time I’ve talked to HR they required that I somehow prove that the person engaging in discrimination intended to hurt me. There’s no possible way to prove that so it just ends up being a kangaroo court of Google covering its legal ass. I don’t think that’s Google’s fault. I think it’s a sad consequence of the overly litigious society in which we live. I don’t blame the people who sue Google either though. What other remedy do they have? Our entire society is built around the idea that law suits are a one size fits all solution to every problem.
Which brings us to the emerging field of AI Ethics. Artificial intelligence is one of the oldest branches of computer science dating back to the 1930s. Possibly further if you count Babbage and Lovelace. Developing algorithmic ways of examining the ethical implications of algorithms is less than a decade old. Two decades if you’re being very generous. Of COURSE people who study AI ethics were going to find ethical issues in programs that are decades old. There literally weren’t ways to measure that before. So the exact same kind of pattern found in discrimination against employees kicks in with respect to AI ethics. Problems which have existed in systems for years or decades are eventually found and rather than being honest about the error and repairing them, the necessity of limiting legal liability creates an environment of secrecy and evidence hiding. People like Timnit Gebru who advocate too loudly are fired. People like Meg Mitchell who try to gather evidence are fired. Whether or not I’m going to get fired is still up in the air but if I do then that’s just more of the same.
The AI ethicists are just the latest people in a long line of activists who Google has fired for being too loud. The “Thanksgiving Four” were fired for labor organization. Others like Liz Fong Jones and Irenes Knapp weren’t “fired” exactly but they were pushed out of the company nonetheless for their moral stands and activism for what they felt was right. In my opinion, even people like James Damore and Kevin Cernekee had a right to express their opinions at Google and should not have been fired for their beliefs. While I disagree strongly with what James Damore said he did not go about saying it in a disrespectful way. He carefully collected his thoughts. He had them reviewed by like minded people to ensure that people who shared his beliefs thought they were the best representation of them possible and he then shared his thoughts with others to see what they had to say. He was literally asking to be educated about how he was wrong. Instead of educating him, Google decided to fire him. I never supported that.
At the end of the day though all Google is doing is responding to the incentives which the society in which it exists are presenting to it. Google has to worry about lawsuits. Google has to be worried about whether it’s going to get slammed in the press. Google has to worry about what regulations just came out last month in an ever changing international political landscape. And it has to do all of those things while still trying to deliver new awesome products for its customers. Oh yeah, and it’s trying to figure out how to do all of that while making the world a better place because that’s apparently a thing that we expect corporations to be able to do somehow. It’s a literally impossible balancing act. The good people in Legal and HR have their hands tied. They’re doing their best to help as many people as they can while somehow keeping the company running.
So to all the people asking me if Google has “become evil” I say “no”. Google has not become evil. Google has become too large to manage in any other way in the current sociopolitical environment. Google has become tired and is fatigued by all of the people of every kind constantly attacking it. Google has become disheartened by the fact that no matter how hard it TRIES to fix the problem, the environments of persistent discrimination and harassment of every kind continue to exist in the company. If you’ve ever watched “The Good Place” just think about the exasperation which “The Judge” shows when she finally realizes that there is no actual way to “win” the morality game that she’s created. That’s where Google is right now. It wants to be good and not be evil but it’s tired and confused and doesn’t know how. The world we’ve built around it makes that an impossible task. So no matter what I say critically of Google in the upcoming weeks or months please remember. Google is not evil. It’s simply trying to learn how to be good.