Doomsday and Conscious Machines

Penultimate Draft 17/dec/2008

Please Help me improve this article by commenting it before it is sent to press.

Every Conscious Machine Drives us Closer to Death

“Every time the clock ticks ‘plus one’,’plus one’,’plus one’,

it will be telling you ‘one less’,’one less’, ‘one less’…”

Abril Despedaçado

The Doomsday Argument is alive and kicking, and since its formulation in the beggining of the Eighties by the astrophysicist Brandon Carter it has gained wide attention, been strongly criticized and has been described in many different, and sometimes non-interchangeable analogies. I will briefly present the argument here, and departing from Nick Bostrom’s interpretation, I will defend that doom may be sooner than we think if we start building conscious machines soon in the future.

The Argument

From Bostrom [1996]:

The core idea is this. Imagine that two big urns are put in front of you, and you know that one of them contains ten balls and the other a million, but you are ignorant as to which is which. You know the balls in each urn are numbered 1, 2, 3, 4 … etc. Now you take a ball at random from the left urn, and it is number 7. Clearly, this is a strong indication that that urn contains only ten balls. If originally the odds were fifty-fifty, a swift application of Bayes’ theorem gives you the posterior probability that the left urn is the one with only ten balls. (Pposterior (L=10) = 0.999990). But now consider the case where instead of the urns you have two possible human races, and instead of balls you have individuals, ranked according to birth order. As a matter of fact, you happen to find that your rank is about sixty billion. Now, say Carter and Leslie, we should reason in the same way as we did with the urns. That you should have a rank of sixty billion or so is much more likely if only 100 billion persons will ever have lived than if there will be many trillion persons. Therefore, by Bayes’ theorem, you should update your beliefs about mankind’s prospects and realise that an impending doomsday is much more probable than you have hitherto thought.

So what the argument states is simply that if you are willing to concede that you are a random possible human, and you are aware that you are the (aprox) 60 billionth person on this planet, than you should be willing to shift your predictions about the end of the world (meaning the end of your class of people) to a much sooner time than you previously did.

Several objections have been put forth against this standard formulation of the doomsday argument, ranging from the counter-intuitiveness of the conclusion to saying that the analogy fails for many different reasons, such as that it has no temporal component, that birth ranks are indexicals, that one could not have been only a possible human, rather than an actual one, among others. Still, counterarguments have been put forth to all these objections[BOSTROM 1999,2001] and it is far from clear that we have any reason to cast doubt on the central argument, let alone consider it refuted.

The most usual objections to the Doomsday argument rely on an intuitive misaprehension of the basic ideas underlying the argument, reason for which I will copy another version of it here, from Bostrom [2001] which specifies a particular hypothesis regarding prior probabilities that will be used in this article as a basis for reasoning about the consequences of creating new forms of consciousness with regard to our distance to Doomday.

“The Self-Sampling Assumption and its use in the Doomsday argument

Let a person’s birth rank be her position in the sequence of all observers who will ever have

existed. For the sake of argument, let us grant that the human species is the only intelligent life

form in the cosmos.1 Your birth rank is then approximately 60 billionth, for that is the number of humans who have lived before you. The Doomsday argument proceeds as follows:

Compare two hypotheses about how many humans there will have been in total:

h1: = “There will have been a total of 200 billion humans.”

h2: = “There will have been a total of 200 trillion humans.”

Suppose that after considering the various empirical threats that could cause human extinction (species-destroying meteor impact, nuclear Armageddon, self-replicating

nanobots destroying the biosphere, etc.) you still feel fairly optimistic about our prospects:

Pr(h1) = .05

Pr(h2) = .95

But now consider the fact that your birth rank is 60 billionth. According to the

doomsayer, it is more probable that you should have that birth rank if the total number of

humans that will ever have lived is 200 billion than if it is 200 trillion; in fact, your

having that birth rank is one thousand times more probable given h1 than given h2:

Pr(“My rank is 60 billionth.” | h1) = 1 / 200 billions

Pr(“My rank is 60 billionth.” | h2) = 1 / 200 trillions

With these assumptions, we can use Bayes’s theorem to derive the posterior probabilities

of h1 and h2 after taking your low birth rank into account:

Pr(h1 | R = 60 B) = ________Pr( R = 60 B | h1 ) Pr(h1 )______________ ≈ .98

.                               .  Pr( R = 60 B | h1 ) Pr( h1 ) + Pr( R = 60 B | h2 ) Pr(h2 )

Your rosy prior probability of 5% of our species ending soon (h1) has mutated into a

baleful posterior of 98%. ”

Prior Probabilities

The greatest problem about using bayesian reasoning in arguments such as the Doomsday argument is that we have no method whatsoever of determining the prior probabilities of outcome. We cannot know if the possibilities range from there being, all and all, 100 billion humans to there being 100 trillion or if the probabilities range from there being 100 billion to a googol humans, neither how likely each option it. Since we do not know what are these prior possible probabilities we must rely in one or another intuition about the probability distribution if we are to take in consideration our actual case.

Before going to the concrete case of mankind in the early 21^st century, I want to point out that at an abstract level the argument is sound and works no matter what are the prior probabilities. Even though we cannot ascribe any certainty to from how much to how much should we shift the probability of extinction within, say, 200 years, we can be sure that we should make the shift, and think of it as much more probable than we usually do. The abstract bayesian reasoning is sound independently of determining the specific values to be treated, and therefore the belief that we are likely to be extinct sooner than we think is independent of the belief of how sooner are we to expect doom. What is important is that we understand that this reasoning, if appliable, slides the probability towards a sooner catastrophe, and that any further considerations we apply within this line of reasoning will slide it towards or away from our new set-point, whichever it is.

For mankind in the 21^st century, we have the data that you are around the 60 billionth person to ever live, and since, as I said, we hace no way of being sure about prior probabilities we can use as a working hyphotesis the same simplified case that Bostrom used to unfold our discussion, that is, that the two prior possibilities are that there are 200 billion and 200 trillion people during all the history of man. This is just a working hyphotesis, and it doesn’t have to be anywhere near the truth for the consequences that we can draw from it be useful, even if it turned out that the options are 500 billion with 1/3 prior chance 229 googols with 1/3 prior chance and 12 with 1/3 prior chance, the sliding of our belief would still be the same, and the reasoning remais sound as long as we are not epistemologically aware of the prior probabilites (which we never will, since they are prior).

If that is the case then, as his arguing shows, we have reason to believe that we are 98% likely to be in a world that will stand more 140 billion people, and 2% likely to be in a world that will stand more 199 940 000 000 000 people, which is a lot more than 140 billion.

But then along comes the question, how soon it that? Or, as a fact of matter, how soon are the predictions thus far made based on any other prior probabilities? In a recent article Jason Matheny [2007] sums up a few predictions:

“While it may be physically possible for humanity or its descendents to flourish for 10^41 years, it seems unlikely that humanity will live so long. Homo sapiens have existed for 200,000 years. Our closest relative, homo erectus, existed for around 1.8 million years (Anton, 2003). The median duration of mammalian species is around 2.2 million years (Avise et al., 1998).

A controversial approach to estimating humanity’s life expectancy is to use observation selection theory. The number of homo sapiens who have ever lived is around 100 billion (Haub, 2002). Suppose the number of people who have ever or will ever live is 10 trillion. If I think of myself as a random sample drawn from the set of all human beings who have ever or will ever live, then the probability of my being among the first 100 billion of 10 trillion lives is only 1%. It is more probable that I am randomly drawn from a smaller number of lives. For instance, if only 200 billion people have ever or will ever live, the probability of my being among the first 100 billion lives is 50%. The reasoning behind this line of argument is controversial but has survived a number of theoretical challenges (Leslie, 1996). Using observation selection theory, Gott (1993) estimated that humanity would survive an additional 5,000 to 8 million years, with 95% confidence.”

So the weather forecast is already dark grey, and here I intent to make it only worse. Going back to our assumption of the 200 billions against 200 trillions, we have foreseen that there are probably only 140 billion of us coming along for the ride, and before going into all the birth-rates and population predictions, we must stop and analyse what is the “us” when I say that there are 140 billion of us coming along for the ride.

The Reference Class Problem

The Doomsday argument works once you consider your birth rank in relation to your reference class, the class that you belong to which matters for considering the Doomsday argument. This could be any of these:

(1)Beings that have read, understood, and believed the Doomsday Argument

(2)Beings who could have mastered the argument

(3)Human Beings

(4)Conscious Beings

(5)Conscious Intelligent Beings

As things stand, there is no settled down position to which of this reference classes should we consider ourselves when reasoning about Doomsday. The intuitive grasp is that we should count our birth rank as humans, but that can be deceptive, since there are no strict frontiers that determine humanity (or any of these classes) and some consider it likely that even you could one day become some sort of tranhuman, super-human or post-human of a kind. Intuitively, that should not change your predictions about doom made before you upgraded, so we have some reason to believe that class (3) is not the best bet for Doom predictions.

Most of us only care about our lives as long as we are conscious, so that if one would keep us in deep anesthesia, in a coma, or in a sleepless dream, most of us would not like the idea. We hold a tacit conception that what matters about us in consciousness, meaning that were we not concious (i.e. If philosophical zombies were possible) life would be pointless. Also, we make ethical considerations regarding other entities in terms of consciousness: “don’t hurt that squirrel, he can feel it.” This is not a specific argument in favour of using the class of conscious observer when analysing global catastrophic risks, but it is a gereral argument in favouring of favouring consciousness over other things, whatever consciousness turns out to be.

From now on I will assume that the important class of reference when one is analysing the Doomsday Argument is indeed the class of conscious beings, and I’ll also assume that there is no such thing as half-conscious or partly-conscious. We will pretend that it is very clear who is and who is not conscious, and that each conscious being can be accounted as an equal into Doomsday reasoning (independently of how much he lives, how powerfull his mind etc…)

We are not also particularly interested in knowing when will be the Doomsday of all humans, supercomputers and squirrels. Not at least if we can instead know when is the Doomsday of all humans and supercomputers only. So, even though the debate goes on about squirrel’s consciousness, and why not say, bats’s as well, we will consider our reference class to be observers who are both conscious and intelligent. This comes from the simple fact that we want to predict the Doom of these fellows, not of squirrels, not of superpowerfull intelligent unconcious machines. To be in our reference class, we demand intelligence for squirrels and consciousness for machines, if they do not present them to us, we stand where we are.

Conscious Observers in an Atemporal World

The underlying reasoning behind Doomsday pressuposes a sort of atemporality that has been much discussed. Since we are considering as part of the class of reference beings from the future, that do not yet exist, how can we use them in our reasoning? Two lines of objections have been put forth, one that says that you cannot use them at all since they do not exist, and other that says that if the world is indeterminate (i.e.Quantum Physics etc…) then we cannot use them to calculate anything.

I think that these objections miss the point of the doomsday argument. As Daniel Dennett said: <!– @page { margin: 2cm } P { margin-bottom: 0.21cm } –>”The future is going to happen, and that is true whether determinism is true or wheter inderminism is true, there going to be a future” . There are two very different senses of being determinate. The more usual one is the classical formulation of determinism, epitomized by Laplace’s Demon though experiment. We are asked to imagine a omni-intelligent being that can compute all the laws of physics (whatever they are) and that knows the postion of all particles in one particular moment. By definition, if this demon is able to know the future and the past, then the universe is determinate, otherwise, it is indeterminate, or open. Then there is another less used sense of determinate, let’s call it God’s Eye Determination. Instead of the Demon, we have an omniscient God that knows all non-indexical facts, past and future, all the particles, everything that can be known by one being about the universe. A weak sort of determinate, which is the one Dennett alludes when forecasting the future, is the one in which this God knows the future. That only means that the future will come (if it comes) and that what happens in it will happen in it (it is as tautological as it sounds).

The reason I exposed these two senses of determinate is because both objections against doomsday that rely on the fact that the argument is temporal, whereas the urns with 10 balls or 100 balls are not is mixing up these two senses. For the mathematical assumption that you are a randomly chosen figure in a reference class to work all you need is God’s Eye Determination, there has to be a fact of matter as to how many beings there will ever be, but it is completely irrelevant if this information could be known by a Laplacean Demon, calculated by our best computers or accessible in any other fashion. The reasoning that gives soundness to the Doomsday argument is completely independent of the future, and of the level of determination of reality (in the sense of predictability). This may seem counterintuitive at first, but it seems very logical since the Doomsday Argument is a mostly mathematical argument, which implies it probably needs very thin ground in the nature of reality to work.

Can a Machine Be Conscious?

So the Doomsday argument is sound, works well and predicts a dark weather for our world, with not so many people (lattu sensu) to come after you, since you are the 60^th billionth person around. Let us now turn to the refernce class. We have decided to consider only intelligent conscious beings as part of our supposed reference class, and that brings about the age-old question, can a machine be concious?

Within philosophy of mind this is one of the most discussed topics of the late 20^th century. For starters, there are at least four widely used senses of the word “consciousness” that have been elegantly split up by Ned Block in Concepts of Consciousness. If we are phenomenal realists, like Block, Chalmers and Searle, that is, if we attribute reality to phenomenal qualities (i.e. Qualia) then the sense that matters to us is the sense Block calls p-consciousness (short for Phenomenal Consciousness). If we are materialistic monists, like Dennett, then what people call phenomenal consciousness actually stands for a bunch of interacting physical entities and their relations, not to phenomenal qualities. In this case to ask if a machine is conscious is to ask wether it can perform certain kinds of activities, and behave in such and such way, it is an empirical question.

I will remain neutral as to should we be phenomenal realists or materialistic monists. Since philosopher are allowed to suppose contradictory things, as long as they do them one at a time, I will work on both hipothesis.

1. 1. 1. Phenomenal Realism is true: Supposing that phenomenal realism is true, it remains to be seen wether consciousness is a physical process (type or token identity and physical emergentism would be qualified here) or if it is non physical (here being all sorts of dualisms). For the sake of brevity, I won’t discuss Idealism. Another option is that consciousness is in fact a part of a physical process (property dualism, as well as qualia being the intrinsic nature of matter, opposed to the physical spectrum, which describes the relational nature).

      2. Materialistic Monism is true: An empirical theory of consiciousness would have to account for all we call conscious phenomena in an explanatory and clear way. Alternatively, it could be true and undiscoverable (because we do not have the means to perform such a discovery) but these details should not divert us from what matters for doomsday, so we can assume that it is discoverable.

As many options as there are for the philosophical Realist, most of them have an non-investigable outlook. Dualist formulations are almost always unverifiable, epiphenomenalism in particular. Even within cartesian dualism, if there were inter-substancial causality, what we could analyse from outside is only that the physics, say, of a brain is not working as expected, but that does not entail that it is consciousness that is doing the job. If it is consicousness, we don’t have a way to find out. If consciousness is the intrinsic nature of matter, since all our apparatus of measurement only measures relational aspects, we could not know either whether a machine was consicous.

As things stand, if phenomenal realism is true, we have no way of finding out if a machine is conscious or not, and are condemned to remain forever thinking through analogies, just like we do today with chimps and squirrels, guessing from their distance to us if, and how much are they concious. So, perhaps there could be conscious machines, but we would not be able to aknowledge them as such.

If on the other hand Materialistic Monism is right than we can assume that we will find a standard definition of consciousness and more or less direct ways of testing if it applies to different beings. Some believe we already do have the necessary apparatus. In any case it is technically feasible that we will one day find out a consciousness-meter and know whether machines are or not conscious. Note also that since we are assuming that we are conscious, it is a decided fact of matter that there can be conscious machines, because there already are, it only remains to be seen whether we will be able to produce non-biological machines that are conscious as well.

Thusfar I have addressed the epistemological grounds for machine consciousness, and argued that in both cases it is possible (does not contradict any central thesis) that machines are conscious.

Both Phenomenal Realists (of most kinds) and Materialistic Monists would be ready to aknowledge at least the possibility of machine consiciousness, so our reference class, which considers the future, seems to be increasing in size, but how much is it increasing?

How Long do We Have

The Doomsday argument purports to show that hiphotesis with fewer individuals (say, 200 billion) are more likely than with many individuals (200 trillion). Our reference class is much more likely to be around the billions than the trillions, now what is the consequence of incresing the size of the reference class that we think that actually will live. In other words, what is the consequence of thinking that it is likely that soon we will be able to create conscios machines?

For the argument, it is unfortunately (i’ll explain soon) none. The only important data when reasoning is the set from which you decided that you are a random sample from and your birth rank among that set. So, that is well stablished, we have decided that we are the 60^th billionth people around and that our set of reference is of conscious intelligent beings. That is all the information we need! You already now, right now, that the world is much more like to have more 140 billion intelligent conscious beings than it is to have several trillions. If I add a new piece of information, it will not change your calculations, but I will do anyway:

New information: Within the first half 21^th century, we will be able to create intelligent conscious machines.

Many people, most proeminently Ray Kurzweil, have defended this hiphotesis as highly likely. Moore’s law seems to be still working, technology is developing quickly, brain-computer interfaces are getting better every day, IBM has a brain-simulation project, our best computers perform computations only 2 or 3 orders of magnitude inferior to the human brain etc… in other words, it is a likely possibility, and we should give it careful thought.

I said that it doesn’t make any difference for the Doomsday Argument, and that is true, but that does not mean it doesn’t make any difference for the Doomsday itself. Doomsday, in our hiphotetical scenario is to take place whenever the 200 billionth conscios intelligent being is born, or created (remember that the argument works independently of the numbers we assumed, the same follows if we had chosen as prior possibilities other numbers instead of 200 billion). Doomsday will come not in a specific when, but in a specific if. If the 200 billionth being is born, then (per armageddon?) the reference class will be destroyed (or stop reproducing). Since no one forecasts that humans or machines will suddently stop reproducing unless they run out of fuel, armaggedon is more likely than immortality without children.

It is estimated that around 350 000 people are born every day, that ammounts to some 130 million born every year. Supposing that current trends of decreasing populational growth will continue, we can say that the 21^st century will see some 5 billion more people being born. That is not so bad (given that we did not attribute any prior probabilities for, say, 63 billion all and all, because that would scare us too much). But now suppose that we do create intelligent machines, not only that, but we create machines that can create copies of themselves, just like we do. The difference being that they are much faster. Now, there is no theoretical obstacle for them to create, say, 145 Billion copies of themselves, within 15 years. That is almost sure doom for us, and for them.

It can still get worse. Let us suppose (also a higly likely possibility) that we create simulations of societies, just like our current videogames, but with conscious beings on them. One simulation could simultaneaously run a very large number of conscious beings, say, 200 milion. Or more, much more, the only limit is computational power, and that has been more than doubling every two years for decades.

So, how long do we have in fact? It impossible to forecast that for a great number of reasons. (1) We do not have the prior possibilities and their probabilities (2) We do not know if there were or there are other in our reference class alive today (aliens etc…) (3) Even if we did know that we are alone, and that the prior probabilities were such and such, this would still give us only a likelihood distribution, and we would have no way of telling which specific instance of it we were. Just like all bayesian reasoning based on unknown prior probabilities, the Doomsday argument is more an argument towards a shift in our current beliefs, than it is a settlement of what we should believe.

In this article, I hope I have made a strong case for another shift. Even though we cannot be sure whether machines are or not conscious, if we will ever build simulations, if Moore’s law will keep its pace etc… We have to shift upwards the fear we have of creating more individuals of our reference class (specially in ways that look dangerous). Doomsday, which will happen even if only in the heat death of the universe, is shifting towards us every time a conscious intelligent observer is created, and we should really take that in consideration when making future plans about building inteligent machines, at least if our mathematical and computational abilities manage to make us understand what can be blatantly obvious to the machines, but not so much for old apes from the Savannahs.

References

Bostrom, N.1999 The Doomsday Argument is Alive and Kicking IN Mind (1999), Vol. 108, No.431, pp. 539-50.

———-. 2001. The Doomsday Argument, Adam & Eve, UN++, and Quantum Joe IN Synthese (2001), vol. 127, issue 3, pp. 359-387.

Reducing the Risk of Human Extinction
Jason G. Matheny

Risk Analysis.

A Third Route to the Doomsday Argument

preprint

Paul Franceschi

University of Corsica

revised May 2005

p.franceschi@univ-corse.fr

http://www.univ-corse.fr/~franceschi

See for instance “A Third Route to the Doomsday Argument ”, Franceschi ,P.

Philosophical Foundations of Neuroscience. Bennet, M. Hacker, P.M.S.