Debate Analysis

Debate Analysis

Philosophers Daniel C. Dennett and David Chalmers In Conversation

Channel: Pioneer Works

Primary speakers:David ChalmersDaniel C. Dennett
Transcript
Download
John Brockman [00:00] It's actually a very exciting evening for me and I don't like to show emotion. These two individuals have been at each other for 20 years litigating, very serious and consequential ideas. They appear to be friendly. I hope they are. These things matter, ideas matter and they have very different concepts of the world. They've been talking about the hard problem and consciousness for 20 years and tonight we're not going to go there. We're going to be talking about the next shooter drop, which is this whole world of AI, which for me, although I met the original cyberneticist in 1965 and I've been there ever since with all of them, it got pretty boring in the 80s and I just walked away from it in the world of export systems. The Japanese had meaty, which was the fifth generation and they're coming, they're coming, they came and they went and nothing happened. I happened to be there at the meeting when the Japanese official was directing the thing, showed up and was minskie, John McCarthy, Roger Shank, at Fiegonbaum. I happen to have a seat at the table and it all just seemed to peter off into another AI winter. 20 years later, you wake up and there's something called unsupervised, self-fulfilling, deep learning, the AlphaGo software, demos hospice and deep mind. It was all very interesting, so I thought it would be valuable just to find out what's happening. I put together a dinner in London with inviting demos hospice. The idea being let's have them talk to David Deutsch, who's one of the sharpest people that I knew and get a sense of what's going on and in the group of dinner where people had nothing to do with computing but have a lot to say about reality such as Ian McEwan, the novelist, Brian Eno, the musician, Terry Gilliam, filmmaker. There's a fascinating start and we call that the London Chop-House Society dinner and we've had more since and we'll continue to do so. Following that was a conference in Washington, Connecticut, where a number of people that have been thinking about AI their entire lives starting with the cybernetic world of Nova Wiener got together. People like Danny Hillis who are interested in the, he broke the von Neumann bottleneck with parallel processing, his parallel processing computer, and Peter Gallison, the computer scientist, historian, Seth Lloyd, quantum theorist, and Neil Gersternfeld and it was Neil that said looking at Nova Wiener's book said this is also prescient but was written 20 years ago, is worries about our culture, about the commercialization of science. It's all coming around again. We should do my favorite rewrite his book and that's how this book started and that's why we're here. So tonight we're going to talk about themes in the book with the title of which is possible minds, 25 ways to think about AI and one theme is superintelligence impossible and I thought we'd start off with five minutes by each of these gentlemen, gentlemen starting with Dave.
David Chalmers [04:49] Oh sure, such a pleasure to be here. Thanks so much to John and to Jana for putting this event together. I think it's going to be fun. I'm on the side of, I guess one of our questions here is is superintelligence possible or impossible. I'm on the side of possible. I like the, I like the possible, which is one reason I like John's theme, possible minds. I think it's a wonderful theme for thinking about intelligence both natural and artificial and consciousness both natural and artificial. I mean think about the space of possible minds. It's absolutely vast. All the minds there have been will be or could be. I mean starting with the actual minds, I mean you might think there's a lot of actual minds. I mean I guess there have been a hundred billion or so humans, the minds of their own. So pretty amazing minds have been in there at Confucius, Isaac Newton, Jane Austen, Pablo Picasso, Martin Luther King, on a goes. A lot of amazing minds, but still, those hundred billion minds put together. So it's the tiniest corner of the space of possible minds. And we can add in, I guess, all the nonhuman animal minds that there have been. I looked up on the web today, how many organisms have lived? How many animals have lived in the history of the planet? And the best estimate seems to be around 10 to the 29. Most of them are worms, it turns out, in the sea. Their minds may not be so interesting, but even a worm has a little mind of its own. So 10 to the 29 minds there, which 10 to the 26, or at least 10 to the 20 are very, very interesting minds. Still, still the smallest corner of the space of possible minds. And what the computer, why do the amazing things about the computer is the way that it enables us to explore and expand that space of possible minds. I mean, arguably, for the first time, since the history of the planet, the computer has enabled some wholly new kinds of minds to come into existence, not by the standard methods of biological evolution, but by straightforward, intentional design, and programming. I mean, so far, the minds have been, you know, limited, but still interesting. John mentioned AlphaGo and his successes in the Alpha Zero family, which have managed to teach themselves to play Go from scratch in a way wholly unlike. It seems the way in which a human would learn to play the game or would play the game at all, but nonetheless, turn out to exceed human capacities, at least in that one very limited dimension of gameplay. Likewise, deep learning has led us to, you know, surprising successes on things like image recognition, speech recognition, autonomous vehicle driving, and so on, where within limited domains, they're not there yet on the autonomous vehicles, but at least in the speech recognition and image recognition, starting to exceed human capacity. So, okay, we've had a limited, limited, so far, expansion of the space of actual minds to include some minds that we've designed, but so far, it's only the smallest of expansions. One thing I think we shouldn't do tonight is exaggerate where we've gotten to with AI to date. I mean, the advances are amazing, but they're limited. They haven't gotten us yet anywhere near general human intelligence. I think it's unlikely they're going to do so anytime soon. It's not happening in the next 20 years. Will it happen this century, maybe? You know, people say that with any given technology, people tend to underestimate its effects in the short term, sorry, to overestimate its effects in the short term, and underrate it in the long term. And that's my attitude towards AI. I think, you know, there's a lot of hype right now. It may not change our lives completely in the next 20 years, but in the next 200 years, it's probably going to transform everything. And one of the reasons is, it's just this extraordinary AI build into it, this way of kind of a self-enhancing, a self-perpetuating mechanism of exploring the space and expanding the space of possible minds. Already we've, I mean, the early AI programs, you had to design them yourselves. People would actually, you know, program in lecturing, you know, basically wrote an Alan Turing, wrote a program that could play chess, and he built in some very simple rules of thumb for playing chess and it played chess, not very well. Now the chess playing systems like AlphaGo learn to play chess from scratch and do so amazingly well. Learning serves as a method for moving ahead in the space of possible minds. Start from a pretty simple mind with a capacity to learn and it gets somewhere. Evolution is another such method. And I expect to see AI exploiting evolutionary methods where systems, we have some kind of system of artificial evolution among a bunch of different AI programs and their capacities expand in surprising and unpredictable ways over time, thereby also getting us far beyond that starting point. So learning and evolution in computers are ways of expanding that space of minds. But I think the most powerful method of all though in exploring that space is one which is still to come. And that's once you actually have AI systems doing the designing. Once AI's are designing AI, it's just that we get to the first AI which is at human level capacity for for various kinds of general intelligence and in particular at human level capacity for designing AI. Then a little while later, you know, within a year or two later these things always get better, you're going to have this AI program will be at greater than human level for designing AI's. Therefore it will be able to design an AI AI which one way or another is better than itself. Why? Because it'll be better than humans at designing AI's, the humans designed it, it'll design something better. So this process of recursive self improvement or recursive self enhancement first put forward by the philosopher and statistician I.J. Good. I see is an amazing bootstrapping method for exploring that space of mind. We start from a little corner here in the space of possible minds. Learning and evolution expands it still to a much bigger area but still a small corner. But once we have AI's doing the designing, you know, this is the space of minds we can design AI's, but the AI's we can design may be able to design AI's far greater than those we can design. They'll go to a greater space, those who go to a greater space, those who go to a greater space. So eventually you can see there's probably a vast, vast advances in the space of possible minds possibly leading us to systems which stand to us as we stand say to a mouse. And that I think would be genuine super intelligent. So I think that's possible and I also think it's not going to happen in 20 years, it's not going to happen in 50 years, but we do have to think about it and we do have to worry about it. The AI's which we create will have the capacity. I mean, a working definition of intelligence here is the ability to fulfill your goals across a wide range, across a very wide range of goals to solve problems and find ways to achieve your goals in extremely powerful ways. These AI's by definition will be systems which are extremely powerful at achieving their goals. If they have goals, then unless there's some countervailing force, then they're probably going to achieve those goals. We have to then be very, very careful. What are the goals of these AI systems? As philosophers, we'd like to think about values. What are the values? We put into these AI systems. We also need at some point to think about consciousness. These AI's that we're creating conscious. How does their consciousness relate to human consciousness? Is this going to be a world of wonderfully enhanced subjective experience or a mindless world without consciousness at all? That's something maybe we can talk about as this goes along. The final question I think we need to ask is, where do we as humans stand with respect to these AI's? Are these AI's the systems that replace us, or rather our AI's systems that enhance us? Maybe do we, ourselves, eventually become the AI's? Do we enhance ourselves? Do we upload ourselves to eventually be the AI's, which are on the forefront of this expanding wave of superintelligence? That's an attractive prospect in some ways compared to the one the prospect where humans don't exist at all and get wiped out, but it raises so many questions. Could you upload a human mind into a computer that raises some of the oldest philosophical questions of identity and consciousness, which I think is wonderful, which is a great thing that John has managed to bring a number of philosophers along with scientists and engineers to think about these
John Brockman [14:54] questions at this time. Thank you. I should add that Davis University Professor of Philosophy neuroscience. I forgot. I'm sorry. Co-director of the Center of Mind, Brain and Consciousness
David Chalmers [15:11] in New York University. Dean's no introduction. Hey, but I will. I demand you take back that
John Brockman [15:20] introduction. Dean, University Professor Austin B. Fletcher Professor of Philosophy, Director of the Center of Cardinal Studies at Tufts. Thank you John. Thank you David.
Daniel C. Dennett [15:34] This is supposed to be a debate, but almost nothing that David said is anything I disagree with, although I wouldn't put the emphasis where he does. Let's talk about possible for the moment. There's lots of things that are possible. The philosophers love to talk about once possible, but many things that are obviously possible are never going to be actual. It's possible to build a bridge across the Atlantic. We're not going to do it. Not now, not in a hundred years, not a thousand years. It could cost too much money and it would be a foolish endeavor. A lot of the imagined AI projects that are perfectly possible, in principle, are not worth doing, and in fact some of them are definitely things that we shouldn't do because they'll make more problems for us than they'll solve. Just bear that in mind. Somebody said that the philosopher is the one who says we know it's possible in practice. We're trying to figure out if it's possible in principle. Unfortunately, philosophers sometimes spend too much time worrying about logical possibilities that are importantly negligible in every other regard. Let me go online on the record as saying, yes, I think that conscious AI is possible because after all, what are we? We're conscious, we're robots made of robots made of robots. We're actual, so in principle, you could make us out of other materials. You could be some of your best friends in the future. It could be robots, possible in principle, absolutely no secret ingredients, but we're not going to see it. We're not going to see it for very good reasons. One is if you want a conscious agent, we've got plenty of them around and they're really quite wonderful and the ones that we would make would be not so wonderful and for me, one of the really important fears about the future is that long before we've, and they've been agrees with me about this, it's not going to happen at 10 or 20 or 30 years. Long before we got to superintelligence, we would have human beings who are so dependent on non-superintelligences that we would become fragile, brittle in some very important ways, and I think that's, we might call that the GPS problem magnified. People have begun not being able to read maps anymore, know how to get anywhere without the help of GPS, use it or lose it, and I think use it or lose it is going to play a big role in everybody's lives in the immediate future. How many people, I wish we had the house lights up so we could see people, but I'll just ask you anyway, is there anybody in this room that knows an algorithm for extracting a square root? I learned one in school when I was in about eighth grade. It's not easy, but there are algorithms for doing square roots. Nobody bothers anymore. Nobody knows how to do that because you just got that button, single button on your hand calculator, gives you a square root. Well, many more important talents are going to atrophy and disappear except in the hands of cranky craftsman and things like that who will, they'll still know how to make a horseshoe with a hammer and an anvil and a simple forge and they'll be able to read a map and drive a car and other weird things like that while the rest of us are simply disabled in those regards. So that's something that worries me, even more what worries me is that we will, for the very best of reasons, turn over our responsibility for making major decisions to artificial intelligences that are not conscious and they're not super, they're just very intelligent tools. They're great fabrics of pattern recognition and so forth. Who knew 20 years ago there could be such things. We know now that there are deep learning, et cetera, et cetera, but when we start delegating major life decisions to systems that are basically just smart tools, then I think this changes our predicament, our human predicament in a very important way. My slogan about this is we want smart tools, intelligent tools, not artificial colleagues. The difference is that an artificial colleague is somebody who can take responsibility, can be a co-author and can be morally responsible for decisions made. We're nowhere near that with artificial intelligence. In the meantime, I think that one of the major dangers and I do not, I have not figured out how to prevent this from happening. Alan Turing, one of my all-time heroes, said in motion one thing which I regret and that is the Turing test puts a premium on deception on convincing human beings that they're talking to a human being. I know why he did it, it was a brilliant idea, but ever since then there has been this premium on what we might call the disneyfication of artificial intelligence, making AIs that seem more human that are basically false advertising and whether we're talking about Siri or Watson or any of the others, they have this basically paper thin human user interface which is deeply deceptive about what they actually understand. I think that's false. Advertising, I think it's unfortunate, I think it should not be honored, it should be criticized, it should be condemned and that we should get out of the habit of treating AIs as agents when they're not really. Now the reason this is going to be so hard is that as a number of people are foreseeing in the immediate future in the next 10 years, I think the major market for AIs is going to be elder care and why not taking care of elderly folks who can't take care of themselves is not a good life for a regular human being. It's maybe worse than being an old-fashioned telephone operator. We don't regret the loss of those jobs, but an elder care AI, there will be good market reasons for disneyfying them to a very great extent because old folks will want to have a companion, not just somebody that brushes their teeth and gets them fed and so forth. And I do not like the future that is populated by millions and millions of old folks who are basically settling for an artificial companion that is really a fake in most important regards. Thank you. Can I pick up on something
David Chalmers [24:35] down? So I think we agree on an awful lot here, but I think we maybe we do have a disagreement about this core question of whether there will be genuine autonomous AI and Downspeace in the book is wonderfully lucid and thoughtful on this, but I understand Downsland who, although it's possible to create autonomous, intelligent, conscious AI, we shouldn't do it and maybe we won't do it. Instead, what we should do is create tools and use the wonderful analogy of Google Maps. Google Maps tells you how to get to some place, but you still have to get there. You ask, okay, I want to get there, it'll show you a route and then you still have to follow the route that the human is still in the loop. You've got some advice and then the human can take it. So that's what I'm seeing is Downs vision of AI. Maybe you've got a super intelligent AI and I want to know how to get to Mars or how to win a war or something. The AI will tell me what needs to be done, but the human will still be in the loop and all get to do it. What I worry about, I mean, it's a beautiful vision. I just worry if it's realistic. There's going to be so many incentives to take the human out of the loop and to get these AI's, give these AI's to capacities to act on that advice directly and autonomously. In fact, this is already happening with Google Maps, you know, a navigation software. If you drive some cars like a Tesla, for a long time, you know, I'm going to this destination and it would do the usual Google Maps thing, and it would show you the way to get there. Here are all the routes, but you had to still follow it and make the decision. Then at a certain point, they introduced a button. It says Navigate on Autopilot. What this means is the car takes those instructions and follows the instructions, it's off and turns the wheel and changes freeways and so on with limitations that can't drive on ordinary city streets. But in a very, very small domain, what you see happening is that the car has become in a very limited way autonomous. It has those goals, it acts on them, and yes, we can we can still stop it and change the goals and so on, but when I think about say domains like autonomous weapons in the military, we'll show off for a for a brief period when the stakes are low, maybe we'll just have AI systems that advise a soldier on target detection and say, now you can shoot, but eventually the AI is going to be so much faster and better at doing this kind of thing that when the stakes of a genuine international military conflict, a person that's hard to see that we're not going to have some kind of move say to genuine autonomous soldiers that have goals and actually execute them and fire the weapons and just more generally biological systems are going to eventually be slow and creaky compared to these new superfast AI's like my financial purposes with the stock market, military purposes, even scientific purposes, the incentives are going to be so strong to allow the AI's to achieve the goals directly and to act on them, but I think autonomy is just realistically going to be very hard to avoid if the tech companies are running it, certainly it's going to happen, if the government's running it, if the military's running it, it's going to happen, so I don't
Daniel C. Dennett [28:15] really know what your vision is for how we'll avoid this from happening. Well, you raise a real and important issue and that's how much autonomy do you want, as you say, autonomous cars, we don't want them that autonomous, Dilbert two weeks ago had some wonderful cartoons where Dilbert's autonomous car says, I want you to call me Carl, self-driving car is my slave name and Dilbert says, shut up and drive me to the market and the car says, said the self-walking man, we don't want that much autonomy, I mean autonomy is as good as synonymous with free will to me and I don't think we want to give AI complete autonomy because AI's because of the nature of the technology have a certain invulnerability we don't have. You can back them up and put them back together again and make another copy on Monday and if human beings were capable of being completely backed up and then brought back on Monday that would change the nature of human interactions and human relations dramatically and I for one don't want to go there and I don't think many people do. So if you make if if you're right that it's inevitable that market pressures and cleverness will lead to genuinely autonomous AI's then I think we're in for a very bad future indeed. When could that happen? Well we can give them more autonomy than they can handle and that's what I'm afraid of. Let me raise a quick question. Caroline Jones is one of the SCS in the book
John Brockman [30:50] and I understand a question today that I think pertains here in regard to this reliance on the computational way of looking at the world at the expense of what she says can you dress the complexity of our wet cognition a much more distributed notion of intelligence going beyond the ideas of computation our sacred craniums craniums you may not even be bounded by your own skin in this regard what is robot death without mortality can there be a proper proper ethics in and of AI this this computational view is very West Coast and and the cyberneticists weener Shannon McCullough McCullough by Neumann had a much more ecological deep review of how things connect
David Chalmers [31:51] and don't connect that's a it's a great question we've linked it to the discussion we're having here because you know a lot of it turns on what is what is life and death in a in a computer what is autonomy in a in a computer and you know dance autonomy basically requires free will and then okay then we're up against all the philosophical questions of what is genuine free will to get the kind of problems that we were talking about going about you know super intelligence and as dangers we're even sure that it's essential that the AIs have free will order they have consciousness you know that's a very those are very very deep questions what's really going to matter for the safety of safety and human survival and the things that people worry about is what those systems can do and I think in this so for this debate I think maybe we can just just describe autonomy in very simplistic terms the system is autonomous if it has goals it has wide variety of goals and has the power to achieve them so advanced autonomous AI will be systems and not only that that have goals and can actually achieve them compared to dance tools AI tool versions of AI which can advise you on if you can table this is my goal it can advise you on how to achieve it and then you achieve it this is a much more limited form of autonomy I'm not sure that consciousness would be required for this maybe on dance view of consciousness it would but even that limited once once you actually have the AIs with goals and with the power to achieve the goals and I think that's already enough to get this worry going I think that the difference
Daniel C. Dennett [33:37] comes out if we actually compare good old-fashioned AI with contemporary AI at the moment with deep learning and all the rest what we have is as I said these wonderful pattern finding fabrics they're great at finding needles and haystacks and doing other amazing things but they haven't been formed into an architecture that's anything like an agent with its own goals and so forth now the two ways in principle you could go you could go back to good old-fashioned AI and say okay we've got these great fabrics now we're going to do intelligent tailoring we're going to do it from the top down we're going to figure out what goals we want to install and we're going to put as most rules and we do it all from the top down that's one way we can imagine going I think that's very brittle very unlikely to and much much harder than people actually think the other way is to letter it bottom up and let these things evolve and learn and evolve and learn and learn evolve and it'll be all done by bottom up quasi Darwinian methods if we go that route then what we know right from the outset is that we will not be in control we will not be in control and so we will be setting in motion something where the amount of autonomy the systems have will not be up to us now I am not definitely afraid right now because I think that people who imagine this scenario and think this is coming soon I just wrong and the I think orders of magnitude of of difficulty stand in the way you know you take Watson brilliant in its own way I don't know how many person centuries of brilliant work went into the creation of Watson uses up the power of the small city what percentage of an intelligent conscious AI is it I would say a fraction of one percent you know so turning Watson into an actual autonomous agent would be the work of you know many many person centuries of work and nobody even knows how to do it yeah Watson is basically an
David Chalmers [36:41] exercise in what they call knowledge engineering give it a big enough big enough database and a good way of dealing with all that that data that knowledge and I can retrieve and apply the
Daniel C. Dennett [36:52] information and it does a wonderful thing I really can it's a great tool I don't think there's
David Chalmers [36:57] actually one thing I know you see those ads with IBM Watson does this Watson does that there's 30 different Watson's or 50 different Watson's out there Watson is more a brand name at this point
Daniel C. Dennett [37:09] yeah and and some of it is hype to put it politely I think you know what a lot of the the
David Chalmers [37:18] scientists right now are really excited what's it is great but what really excites people right now is machine learning where you basically take systems give them a whole lot of data and train them to do certain things supervise learning right now this is what you should be doing but even that leads to amazing results and say image classification reinforcement learning which is what was used to drive alpha go and alpha zero we basically get the reinforcement of winning and losing it turns out that's enough to drive learning and eventually unsupervised learning but I think you're right that it's where learning and evolution are involved that all this becomes extremely messy and very hard to control I mean when you have machine learning you're basically always you're always optimizing something you know people in machine learning they haven't what's called an objective function you know the way that per an ideal of perfect behavior for your system like completely matching the training set on on these images or classifying the language right or winning every game of go that's an objective function and a really good machine learning system will eventually approximate that objective function better and better and better how it does it is not up to us the objective function maybe up to us what do you want your system to maximize what is the behavior you want it to model all this suddenly puts an enormous once these systems have autonomy that is the ability to act and to achieve their goals that puts an enormous responsibility on us as the creators of the of the AI to get the objective function right make sure our systems are maximizing the right objective function roughly they have the right goals you know you're self driving car okay the goal is out to get you to the destination but also not to to run into anybody on the way and to obey traffic laws and so on once you've got systems with human level autonomy then you want to get that objective function just right I think that in some sense that's going to be the challenge of autonomous AI finding a way to make sure our systems have the right goals and values and this is where all this stuff about the messiness of wet cognition also enters evolution of course human beings we don't have one objective function we have many because we were thrown up not by a straightforward designer but by a whole process of evolution of the ultimate value of reproducing our genes but with any number of little messy objective functions along the way it may well be that some form of say about official evolution will eventually produce AI's as wet and as messy in a way as biological systems those I think will actually throw up even more challenges your humans are so unpredictable in every way and in every way and in an international socio political context that's not really a good thing if AI's are as unpredictable as us in those ways you know at a certain point we may wish we had simple AI with a simple objective function that we knew about then at least we'd have AI we
John Brockman [40:16] could understand this subtitle of this evening is philosophy in AI and I have a question from both of you how do you distinguish your work as cognitive scientists from that of philosophers and I'll tell you a story 25 years ago I did a book called The Third Culture which involved a chapter with Dan and I went to everybody else in the book and had them talk about each of the other contributors and Marvin Minsky got on the phone and I said tell me about Dan Dennett oh the greatest philosophers since Russell fine six months later I had to do fact checking and let me read you what you said about Dan Dennett the greatest philosophers since Russell he said I said what I meant sure you know he's great but he's the only philosopher that understands what we do so but you've become one of the people that does the real stuff you know so we had as cognitive science and philosophy again and let's talk about the role of philosophy in AI because frankly I don't get it well I think I don't understand it I think there is a sort of subfield
Daniel C. Dennett [41:46] of AI which has blended with cognitive science to such a degree that it roughly occupies the position that theoretical physics has relative to experimental physics that is you have people who have done their homework they they know the technologies they they've got hands they know how to code and but they are interested in the theoretical questions the and they're interested in helping the the engineers the AI people sort out and understand what they're up to
David Chalmers [42:25] mean a subfield of philosophy right I think it's a subfield of AI oh so I feel the philosophy yeah
Daniel C. Dennett [42:30] right and it's sort of been my good fortune to be tutored over the last what 30 40 years by some of the leaders in AI so although I'm not a good coder I have done some programming but nothing nothing to impress anybody with but I think the current generation of philosophers of cognitive science are superbly well trained know a whole lot more than I did when I got to know this more than they did when when he got into this and I was there when he was a graduate student and that's a that's a very good very good sign and I'm reading a dissertation today I was reading a dissertation on predictive processing and Bayesian brain hypotheses and it's a very technical dissertation it's my a philosopher that's not going to get you the congruent prize
John Brockman [43:42] no I did mind what's interesting in in terms of the philosophy community that we call mainstream I don't see how anybody focusing on AI would win one of their prizes at this point because
David Chalmers [43:57] well I I think you're wrong I think that the people philosophers who have thought about the mind many of the philosophers who've thought about the mind have thought about AI and someone like Dan is a prime example he's one of the one of the leading philosophers of mind on the on the planet and some large push some very large part of that is from his thinking about AI so I think it's actually very very central in philosophy over the last few decades and the trends has been towards integrating the two pretty closely I actually did my did my PhD in AI lab my my PhD advisor was not a not a regular academic philosopher it was the the AI researcher cognitive scientist and writer Douglas Hofstetter well known to many of you for writing good or lesser bark you also co-edited a book with Dan the mind's eye and I was in the middle of this AI lab coding you know writing writing programs that they did things some neural networks now do you making programs and so on and although I you know I haven't done a lot of coding in the in the last the last couple of decades I think for a philosopher to have that experience actually getting their hands dirty and running and building these systems it's just you know it's stayed with me okay my technical knowledge is now 30 years out of date but it gives you something to it gives you something to build on so that's part of it philosophers can educate themselves in the science and in the engineering and can contribute to it but the other part of it is AI in cognitive science there's a big part of it which is software engineering building the software but there's another there's a whole there's another big part of it which is when there's one part of it what does it actually tell us about say the human mind that's no longer engineering but that's science and it's also philosophy we've got to start thinking about the relationship of these artificial systems to say human systems and someone like like Dan has done a lot there someone like say John Soule on the other side is argued that no this tells us nothing about the human mind or about human consciousness and anyway we need philosophers to come in and think about what is this actually telling us what is it explaining and there's also the the social and political and moral questions not just what AI systems can we build but what AI systems should we build down just off of the proposal about that other people would offer different proposals but at some point someone's going to sit back and reflect on the ethical questions which are going to involve reflecting on human values what do we actually want as a society and I think you know philosophers know how to think about about human values nothing that's increasingly becoming
Daniel C. Dennett [46:41] pretty central to thinking about AI that's very useful it's interesting too that in AI over the years there's there's been a similar gradient of philosophical interest there've been some people in AI basically they're engineers that's all they want to be they don't want to think about the philosophical issues and that doesn't mean they aren't doing great work some of the really important work is technical work by people who yawn when the issues are what what relation is to cognitive science or to the mind what I think is ironic is that people way back if you go back to the early days of Simon and Alan Newell and others who was an attempt to divide the field into AI and cognitive simulation and the idea was the cognitive simulation this was using the computer to simulate human cognition whereas AI was by hook or by hook anything that worked was fine oddly enough the people who tried to do cognitive simulation ended up with these freaky gofi models which didn't do a very good job while the people who treated it as by hook or by hook ended up inventing deep learning and other systems like that which now we realized oh maybe that's how the brain is doing it so it's sort of come full circle which is a very
John Brockman [48:13] interesting thing Dan Stewart Russell had a question specifically addressed to you Stewart Russell is one of the eminent computer scientists that I think we all respect greatly Dan you seem to divide AI systems into conscious entities and tools is there no middle ground agent programs that pursue explicitly represented goals in the real world such agents could be arbitrarily competent as alpha go in this little world and yet non-conscious I do believe that consciousness will necessarily creep in as we make agent programs more and more competent in general can you tell
Daniel C. Dennett [48:54] us how not to make conscious AI systems yeah good good question and I'm glad I'm glad after because I think indeed we can have very very very very intelligent systems which are not conscious in any interesting way but they will seem conscious in some ways but they won't have important features that we have it's very much a matter of whether they are capable of taking their own inner states as objects of scrutiny and doing that recursively and indefinitely and that's a very special feature which I think no non-human animal has that capacity and that's the big difference between human consciousness and animal sentient we can call it conscious as I'm not going to argue about where consciousness stops or starts but the it's very important to realize that a lot of the let's call them the the techniques and structures that have been developed in recent years which are just wonderful at analyzing causation for instance so that directed a cyclic grafts today a pearls do calculus and so forth that can all be accomplished unconsciously it we can tell a story where it looks like conscious hypothesis testing but it doesn't have to be conscious we can get all those benefits without any bit of acquaintance by the system itself
John Brockman [50:48] with its own inner states you mentioned Judea parole so there's a question let me just go ahead
David Chalmers [50:55] with the one on on consciousness though I think just look okay okay good well
John Brockman [51:01] Judea parole is the father of the Bayesian network without which we wouldn't have AI as we know it today real giant in the field he asked is it too bold to assert that philosophy will soon melt into AI in the sense that all philosophical questions especially those concerned with consciousness
David Chalmers [51:22] will be reduced to problems in AI well I'd kind of put it the other way around philosophy is pretty good at spinning off its problems into the into the sciences as we as we solve them you know Isaac Newton considered himself a philosopher but he figured out this philosopher figured out some really good methods for solving the problems of we're not of space and time and so on okay we spun it off we called it we called it physics along the way the you know philosophy is spun off psychology and linguistics and so I think what happens is that it's actually never the case that the spin-off solves the entire original philosophical problem but we find some part of it which is tractable on which people can now where we can find methods where people can agree where they didn't agree before and then we say okay now that's actually managed to compel some agreements say the scientific method and physics and then we call that physics and linguistics and economics did physics solve every philosophical problem of space and time absolutely not some of the biggest ones are unsolved did psychology solve the mind body problem no um absolutely not yet um there are as many views on the mind body problem now in the age of psychology as there were before is AI going to solve the problem of consciousness no almost certainly not on its own on the other hand what will certainly happen is that it will give us a whole lot of new insights we'll get AI engineered systems which behave in remarkable ways where we're tempted to suspect that they're conscious and that someone may even think that there's good reasons to think they're conscious but we're still going to need philosophical reasoning to think about it and this now gets back to the question of you know the elephant in the room are these AI systems really genuinely going to be conscious or I think this is not something we can just dismiss as a philosophical question why because it's very deeply baked into our moral system I think as human beings an entity has moral status it's a system that we should care about if it only if it's conscious certainly only if it's conscious if a computer system doesn't have any consciousness then it's basically a tool it might as well be like a like a car or a loudspeaker and so on we can do with as we want it doesn't deserve moral consideration if these systems are conscious then at least they enter into the to the moral sphere they're among the systems we have to start caring about so if most AI systems eventually are conscious then we can't simply use them as our tools and we have to start thinking about these questions of whether they deserve equal respect equal rights and so on so I think that's it is at least on my view a crucial question and my suspicion is actually as AI systems develop which are more and more autonomous more and more capable of reflecting on their own processes more and more capable of giving reasons and evidence my own suspicion is that systems like that are going to have a sense that they are in fact conscious systems we're going to talk to them and you know eventually we can see well how do you you know you you said there's some people over there how do you know well I just saw them what was that like all I you know I had and I had an experience and you know maybe they'll start reflecting on philosophy well you know I know like I've read that the owner's manual I know I'm just a whole bunch of silicon circuits but I feel like so much more well that's so so maybe so Dan's going to say they've
Daniel C. Dennett [55:13] got the illusion of consciousness that's qualia for you well that's all we all any of us have absolutely one one thing I think you underestimate when I was working with Rod Brooks on cog one of the take home messages from that whole experience for me is how little it takes in the way of animation and speed particularly speed and grace to convince most people that that a robot is conscious cog was never within a country mile of being conscious and yet there were MIT students who were banding together to think about the our moral obligations to cog who were who were concerned and cog did not because it was planned this way but did have some strikingly persuasive behaviors unconscious though cog was if you walk in the room when cog was on and cog's eyes would follow you across the room that would freak people out and or or shaking hands with cog was it was a good one I had one of my TAs I took her over to the cog lab to see cog and cog's arm wasn't even attached to cog's shoulder it was just sea clamped to the bench and Matt Williamson said said go ahead shake its hand and she reached and she shook his hand and she screamed it's alive because it wasn't clunky it was it had elastic series elastic actuators and it was it was that was enough so what I am quite sure of is that we're not going to have a problem convincing people that robots have moral rights under conscious it's going to go the other way around we're going to have a problem convincing them that no these aren't conscious yet you're being fooled by the by the tempo it's actually some great psychological data on this
David Chalmers [57:39] on when people are inclined to say that a system conscious has subjective experience that you know you show them many cases and you very say the body is it a metal body or a biological body you very what it's made of silicon or neurons you vary this you vary that you know the one factor that tracks this better than anything else is the presence of eyes yeah if a system has eyes it's conscious the system has a doesn't have eyes well you know all bets are all all bets are all bets are off so I think yet the moment we build our AI's and put them in bodies with eyes it's going to be nearly nearly irresistible to say they're conscious but it's not to say of course that AI systems which are not in body do not have consciousness there's actually a website you can go to call something like you know people for the ethical treatment of animals this is the AI analogic look people for the ethical treatment of reinforcement learners and the ideas every time you give a negative signal to a real one don't do don't do that again it's getting a little bit of suffering and every time you give it a little bit of you give it a reward it's getting a little bit of pleasure we have to make sure we give it a lot more reward than the suffering okay well maybe that's that's not yet at the threshold for consciousness but these questions eventually once look I think once we actually get to the level of genuine autonomous agents as dancers I think it's going to be very hard not to treat them as conscious and that's going to raise many
John Brockman [59:07] social philosophical questions we're talking about ethics and the elephant of the room are the ethics of the big five and you know what they're doing with your data and how your reality is being programmed without without your permission let me read you just a few words from George Dyson chapter in the book weiner noble weiner became increasingly just enchanted with the quote gadget worshipers and quote whose corporate selfishness brought motives to out of automation that go beyond the legitimate curiosity and are sinful in themselves and quote you knew the danger was not machines becoming more like humans but humans being treated like machines quote the world of the future will be an ever more demanding struggle against the limitations of our intelligence he warned not a comfortable hammock in which we can lie down to be waited upon by our robot slaves comment I mean I think we have to address things like this if you're going to talk about
Daniel C. Dennett [60:19] what you're doing and what AI be able to do the rereading weiner's book to write my essay for this wasn't astonishing in a way because I'd read it when I was an undergraduate I think and no okay and then to read it today it's remarkably prescient in in some regards and I think some of the essays in the book are genuinely scary and I think that people ought to read those essays and decide for themselves if some of the proponents there shouldn't be sat down and try to argue out we should try to argue out them out of some of their blind confidence about what the future holds I think we have some serious problems looming and we should take them very seriously I mean yeah he talks about humans being
David Chalmers [61:37] treated like machines I don't know about being treated like a machine I think I'm gradually becoming a machine a half of my half of my memories are now you know either stored on my on my smartphone or or sitting in the sitting in the cloud I was trying to figure out the other day who has who has a bigger part of my brain is it is it Google is it Apple or is it is it Facebook I think for now it might actually be Google they've got they've got an awful lot my memories my plans calendar system navigation system you know I mean we've all long since become these giant exo organisms with this giant exo cortex as Charles Truss called it of the of the of the all the computer systems we're coupled with in the cloud you know I don't go anywhere without without consulting the without consulting the internet at least you know five or ten times in the in the process what is it I'm going to be doing again how do I get there it's going to be there and so on so you know so I'm it is true that these corporations are basically owning some rather large portion of my mind I think wanted to do if they wanted to do bad things with it I'm in trouble we're in the we're in the where it's certainly in some sense in the situation of having to give them rather large amounts of I want to do bad things with it well yeah they're doing small bad things with it I mean relatively right okay they're not yet taking your mind and and reprogramming you might do the brain what of course they're brainwashing us bit by bit via the you know the face Facebook algorithm and the and the ads and and so on look I think if there was a gen I don't think they're malicious the big corporations I think they just have structural incentives if someone genuinely malicious got control of those of those systems then we'd have a then we'd have a dystopia coming so I think we do have to think about that
The Question at the Heart of the Debate
How should humanity govern increasingly capable AI when systems may act with growing independence long before we know whether they are conscious or morally considerable?
What this analysis found

Chalmers and Dennett are both worried about the same underlying danger — humans governed by machine power they can neither understand nor contest — but from opposite directions. Most of their apparent disagreement collapsed onto a single word used two ways: Chalmers said "agent" to mean systems that pursue goals; Dennett heard it as systems claiming rights. Once separated, the chasm narrows into a governance question neither speaker quite named.

Discuss this analysis in the community →

David Chalmers

3.5Formal/Systemicreasoning
4.0Pluralisticworldview

Daniel C. Dennett

3.0Abstractreasoning
3.5Rationalworldview
Good-Faith Summary
He argues that advanced and even superintelligent AI is a real long-run possibility, not an imminent reality, and that we need to think now about autonomy, alignment, consciousness, and moral status. He is trying to keep future capability and future ethics in the same frame.
Good-Faith Summary
He argues that many dramatic AI futures are possible in principle but far less relevant than the nearer danger of humans overtrusting, anthropomorphizing, and overdelegating to systems that cannot bear responsibility. He is trying to preserve human accountability against both hype and deception.
3.5Social Contract
Consciousness
3.5Formal/Systemic
Bottom-Up Emergence
4.0Pluralistic
Cognitive Extension
3.5Formal/Systemic
Technical Possibility
3.5Rational
AI as Agent
Intelligence
3.0Abstract
Top-Down Design
3.0Abstract
Human Capacity
3.0Meritocratic
Ethical Desirability
3.5Social Contract
AI as Tool
3.5Social Contract
Epistemic Style
He reasons through conceptual possibility, incentive structures, and extrapolation from learning, evolution, and AI-assisted design. He is careful about current limits but sometimes lets plausible trajectories harden into quasi-inevitability.
Epistemic Style
He reasons through principled distinctions-possible versus actual, competence versus consciousness, tools versus colleagues-and tests claims against engineering common sense and social consequences. He is strongest at clarifying categories and weaker at inhabiting middle cases that blur them.
The Tell
He repeatedly returns to the space of possible minds whenever the discussion narrows to current engineering.
The Tell
He repeatedly returns to smart tools, not artificial colleagues whenever machine agency starts to thicken conceptually.
Blind Spot
Cannot fully see how his realism about delegated machine action can sound like premature category inflation unless responsibility boundaries are specified at the same time.
Blind Spot
Cannot fully see that systems can become governance-relevant operational agents before they become anything like free-willed or responsibility-bearing persons.
Synthesis
He is protecting the need for anticipatory moral and political preparation, without which society will meet new forms of machine power with categories that are too small and too late.
Synthesis
He is protecting the need for legible human accountability, without which power will be delegated to systems that can act but cannot answer for what they do.

David Chalmers

3.5Formal/Systemicreasoning
4.0Pluralisticworldview
Good-Faith Summary
He argues that advanced and even superintelligent AI is a real long-run possibility, not an imminent reality, and that we need to think now about autonomy, alignment, consciousness, and moral status. He is trying to keep future capability and future ethics in the same frame.
Consciousness
3.5Social Contract
Bottom-Up Emergence
3.5Formal/Systemic
Cognitive Extension
4.0Pluralistic
Technical Possibility
3.5Formal/Systemic
AI as Agent
3.5Rational
Epistemic Style
He reasons through conceptual possibility, incentive structures, and extrapolation from learning, evolution, and AI-assisted design. He is careful about current limits but sometimes lets plausible trajectories harden into quasi-inevitability.
The Tell
He repeatedly returns to the space of possible minds whenever the discussion narrows to current engineering.
Blind Spot
Cannot fully see how his realism about delegated machine action can sound like premature category inflation unless responsibility boundaries are specified at the same time.
Synthesis
He is protecting the need for anticipatory moral and political preparation, without which society will meet new forms of machine power with categories that are too small and too late.

Daniel C. Dennett

3.0Abstractreasoning
3.5Rationalworldview
Good-Faith Summary
He argues that many dramatic AI futures are possible in principle but far less relevant than the nearer danger of humans overtrusting, anthropomorphizing, and overdelegating to systems that cannot bear responsibility. He is trying to preserve human accountability against both hype and deception.
Intelligence
3.0Abstract
Top-Down Design
3.0Abstract
Human Capacity
3.0Meritocratic
Ethical Desirability
3.5Social Contract
AI as Tool
3.5Social Contract
Epistemic Style
He reasons through principled distinctions-possible versus actual, competence versus consciousness, tools versus colleagues-and tests claims against engineering common sense and social consequences. He is strongest at clarifying categories and weaker at inhabiting middle cases that blur them.
The Tell
He repeatedly returns to smart tools, not artificial colleagues whenever machine agency starts to thicken conceptually.
Blind Spot
Cannot fully see that systems can become governance-relevant operational agents before they become anything like free-willed or responsibility-bearing persons.
Synthesis
He is protecting the need for legible human accountability, without which power will be delegated to systems that can act but cannot answer for what they do.

Highlights

The moments that matter most

Every debate has a surface argument and a deeper one. This section maps both — what each speaker is explicitly claiming, what they're actually trying to protect, and where their real disagreement lives. Start here to understand what's actually at stake before the analysis begins.

David Chalmers

Chalmers’ core claim is that highly capable, eventually superintelligent AI is genuinely possible, though not imminent, and that the real philosophical and practical task is to think ahead about what kinds of minds we may be bringing into existence. His organizing framework is explicit: the “space of possible minds.” He treats current AI not as the culmination of intelligence but as the earliest, narrowest expansion of that space beyond biological evolution. The worldview behind this is exploratory and modal: reality contains many more possible forms of mind than human history has yet instantiated, and computation gives humanity a new route into that landscape. He is not arguing that present systems are already general intelligences or conscious beings; in fact, he repeatedly warns against exaggerating current AI. But he does think learning, evolution-like methods, and eventually AI-assisted AI design create a plausible path toward systems that exceed human cognitive capacities by a large margin.

The motivational stakes for Chalmers are twofold. First, he wants to protect intellectual seriousness about long-term possibilities from both hype and complacency. He fears losing the chance to prepare ethically and politically for systems that may become powerful enough to act autonomously and perhaps deserve moral consideration. Second, he wants to protect the legitimacy of consciousness as a live philosophical and moral question in AI, against views that would dismiss it as irrelevant or merely illusory. He appears wary of being accused either of techno-utopian fantasy or of speculative metaphysics detached from engineering reality, so he repeatedly tempers his claims: not in 20 years, probably not in 50, maybe over a century or two; current systems are impressive but limited; autonomy and consciousness are distinct questions. His emotional posture is curious but cautionary: attracted to the grandeur of possible minds, but alert to the dangers of misaligned goals, autonomous weapons, and corporate control over cognitive infrastructure.

His dominant narrative metaphor is frontier expansion: humanity has entered a vast landscape of possible minds and is beginning, through computers, to explore and populate it. Learning, evolution, and recursive self-improvement are the engines of this expansion. In the strongest version of his argument, AI development is unlikely to remain confined to advisory tools because economic, military, and scientific incentives strongly favor removing humans from the loop wherever speed and performance matter. Once systems can pursue goals across domains and act effectively, the central challenge becomes value alignment: what goals and objective functions are they optimizing? At the same time, if sufficiently advanced systems become conscious, then they cease to be mere instruments and enter the moral community. A real tension appears within his position, however: he often speaks of “goals,” “autonomy,” and “agents” in a relatively functional, operational sense for purposes of risk analysis, even while acknowledging that deeper questions about free will, consciousness, and genuine agency remain unsettled. That drift is not hidden; it is part of his attempt to keep long-term practical concerns discussable without waiting for final philosophical resolution.

Daniel C. Dennett

Dennett’s core claim is that many dramatic AI possibilities are possible in principle but either far more remote, less desirable, or less worth pursuing than public discourse suggests. He explicitly accepts that conscious AI is possible in principle because humans themselves are material systems, “robots made of robots made of robots,” with no secret ingredient. But his emphasis falls elsewhere: the urgent issue is not whether artificial consciousness could exist, but how humans will misuse non-conscious yet highly competent systems, overdelegate responsibility to them, and allow themselves to become cognitively and morally diminished. His worldview is pragmatic, anti-mystificatory, and deeply shaped by distinctions between what is logically possible, what is technically feasible, and what is socially wise. He is suspicious of philosophical fascination with possibility when it distracts from nearer-term institutional and ethical realities.

The motivational and emotional stakes for Dennett center on protecting human responsibility, competence, and honesty about what current systems are. He fears a future in which people become brittle through dependence on smart systems, lose practical capacities through disuse, and surrender consequential decisions to tools that cannot actually bear responsibility. He also fears being accused of being anti-technology or anti-progress, so he grants substantial ground up front: current AI is impressive, conscious AI is possible in principle, and some forms of autonomy will increase. But he wants to resist what he sees as a dangerous slide from useful intelligence to anthropomorphic fantasy. His strongest emotional aversion is directed at “disneyfication”: designing interfaces and social roles that encourage people to treat tools as companions, colleagues, or moral peers when they are not. The elder-care example crystallizes this concern: market incentives will reward emotionally persuasive fakes, and vulnerable people may be induced to accept ersatz companionship.

His dominant narrative metaphor is toolcraft versus counterfeit personhood. AI should be built as “smart tools, intelligent tools, not artificial colleagues.” In the strongest version of his argument, the best near- and medium-term use of AI is to augment human judgment without displacing human accountability. Systems like deep learning are powerful “pattern-finding fabrics,” but they are nowhere near architectures with robust agency, self-scrutiny, and morally relevant forms of consciousness. Publics are easily fooled by surface cues—eyes, graceful motion, conversational fluency—into overattributing consciousness and moral standing, so the social danger is not that we will fail to recognize machine consciousness when it arrives, but that we will project it onto systems that do not have it. A tension within his position is that while he sharply distinguishes tools from “artificial colleagues,” he also acknowledges a middle terrain of increasing autonomy and admits difficulty in preventing market and institutional pressures from pushing systems in that direction. So his stated normative preference for tool-like AI is stronger than his account of our actual capacity to hold that line.

Good arguments can still contain weak evidence, logical slippage, or rhetorical moves that substitute for reasoning. This section examines each speaker's argumentative integrity — not to declare a winner, but to identify where the strongest and weakest links are in each case.

David Chalmers

Coherence strengths: Chalmers is notably careful about time horizons, uncertainty, and the distinction between current systems and speculative future systems. He does not claim that present AI is near human-level general intelligence; instead, he explicitly warns against exaggeration and places transformative possibilities on much longer timescales. That gives his argument internal discipline. He also structures his case clearly: current narrow successes, mechanisms of expansion (learning, evolution, AI-designed AI), likely incentives toward autonomy, then downstream ethical questions about goals, values, and consciousness. His use of “possible minds” is not decorative; it organizes the whole argument and links technical, metaphysical, and ethical concerns into one frame.

Weaknesses and logical issues: Several of Chalmers’ key future-oriented claims are epistemically sloppy rather than false. His suggestion that once an AI reaches human-level AI-design capacity, it will likely produce a better-than-human AI “within a year or two” is speculative and asserted without evidence. The recursive self-improvement scenario is genuinely contested; credible experts disagree about whether capability gains in AI design would translate into rapid, compounding self-improvement, or whether bottlenecks in hardware, data, evaluation, embodiment, and research coordination would slow the process substantially. Likewise, claims that autonomy will be “very hard to avoid” because of military, financial, and corporate incentives are directionally plausible but unsourced and somewhat overgeneralized. He is strongest when identifying incentive gradients; he is weaker when treating those gradients as near-inevitable outcomes.

Factual and evidentiary issues: His examples of AlphaGo/AlphaZero, deep learning, and narrow-domain performance gains are broadly accurate. His claim that speech recognition and image recognition are “starting to exceed human capacity” is epistemically sloppy because performance depends heavily on benchmark, task definition, and deployment context; systems can exceed average human performance on narrow benchmarks while remaining brittle in real-world conditions. His estimate about the number of organisms or animals that have lived is unsourced and rhetorically illustrative rather than evidentially central. More importantly, his treatment of “objective functions” as if future autonomous AI will straightforwardly optimize explicit, designer-specified goals risks a category simplification: many advanced systems may not fit the clean objective-function picture in practice, especially if they are hybrid, socially embedded, or trained through messy multi-stage processes. That does not invalidate his alignment concern, but it narrows a more complex technical landscape into a cleaner philosophical model.

Epistemic style: Chalmers operates in a mixed modal-philosophical and scenario-based style, supplemented by selective technical literacy. He reasons from conceptual possibility, incentive structures, and extrapolation from current machine learning trends. He is generally honest about where he is speculating, but he sometimes moves from “plausible long-run pathway” to “realistically hard to avoid” without enough intervening evidence. His claimed style is reflective and philosophical, and that is indeed the style he uses; he does not pretend to be making a tightly empirical forecast. The main gap is that some listeners may hear his scenario-building as stronger predictive warrant than he explicitly possesses.

Daniel C. Dennett

Coherence strengths: Dennett is highly consistent in distinguishing possibility from desirability and from practical likelihood. That distinction anchors much of his argument and prevents category confusion. He is also clear that current AI’s strengths lie in pattern recognition and narrow competence rather than robust agency or responsibility-bearing personhood. His critique of anthropomorphic design is coherent across examples: Turing-test incentives, Siri/Watson branding, elder-care companions, and public overattribution of consciousness all fit one concern about deceptive interfaces. He is also commendably explicit that conscious AI is possible in principle, which prevents his caution from collapsing into metaphysical denial.

Weaknesses and logical issues: Dennett’s strongest weaknesses are not factual errors so much as underargued normative leaps and occasional overconfidence. His claim that “we’re not going to see” conscious AI, or at least not for very good reasons, is epistemically sloppy. He offers reasons grounded in lack of need, comparative inferiority to humans, and engineering difficulty, but these do not establish that such systems will not be built; they mainly support his view that they should not be prioritized. Similarly, his assertion that no non-human animal has the recursive self-scrutiny capacity relevant to human consciousness is a substantive philosophical and comparative-cognition claim that is genuinely contested. It is not settled fact. His framing of autonomy as “as good as synonymous with free will” also risks a category error in this debate context, because Chalmers is using “autonomy” operationally to mean goal-directed action without continuous human control, not metaphysical freedom in the strongest sense.

Factual and evidentiary issues: Dennett’s broad characterization of current deep learning systems as powerful pattern recognizers is fair, though simplified. His remarks about Watson consuming the power of a small city and representing “a fraction of one percent” of an intelligent conscious AI are unsourced and rhetorically vivid rather than evidentially rigorous. The “small city” line is especially imprecise and should be treated as epistemically sloppy. His examples of skill atrophy from calculators and GPS are plausible analogies, but they are not decisive evidence for broad civilizational brittleness; they support a concern, not a demonstrated large-scale causal conclusion. His claim that people will readily attribute consciousness and rights to robots based on superficial cues is supported directionally by HRI and moral psychology research, but in the transcript he cites no studies. Again, plausible, but unsourced.

Rhetorical manipulation and framing issues: Dennett largely avoids contempt and identity attack, but he does engage in frame conversion at moments. When pressed on whether increasing autonomy is likely, he sometimes shifts from the operational question of delegated control to the thicker philosophical question of free will and full responsibility-bearing agency. That move clarifies his own conceptual commitments but can also sidestep the narrower practical issue Chalmers is raising. His “smart tools, not artificial colleagues” slogan is rhetorically effective and conceptually useful, though it compresses a spectrum into a binary that Stewart Russell explicitly challenges. To Dennett’s credit, he partially concedes the middle ground when prompted, but his preferred framing still tends toward sharper dichotomies than the technical landscape may warrant.

Epistemic style: Dennett’s dominant mode is pragmatic-philosophical, with strong reliance on conceptual distinctions, engineering common sense, and social-psychological observation. He is less interested in long-range speculative forecasting than in near-term institutional misuse and category mistakes. This style is well-suited to puncturing hype and clarifying responsibility, but less well-suited to assessing low-probability, high-impact futures. He claims to be resisting mere logical possibility in favor of practical judgment, and that is indeed the style he uses consistently.

Epistemic Mismatch Note

The speakers are operating with different standards for what counts as a serious argument. Chalmers treats long-run conceptual possibility plus structural incentives as enough to warrant present ethical attention; Dennett treats near-term engineering realities, institutional incentives, and responsibility practices as the proper evidential center of gravity. As a result, Chalmers hears Dennett as normatively appealing but descriptively unrealistic about where AI development will go, while Dennett hears Chalmers as granting too much practical weight to speculative futures.

Net Assessment

Both speakers are intellectually serious and more careful than the average public AI debate participant. Chalmers is stronger on mapping long-term conceptual terrain and on keeping consciousness and moral status in view without overstating current AI; Dennett is stronger on distinguishing possibility from desirability and on identifying the near-term social dangers of anthropomorphism and overdelegation. Neither is making a heavily sourced empirical case here, but Dennett is somewhat more disciplined about practical limits, while Chalmers is somewhat more willing to extrapolate from plausible mechanisms into future scenarios.

Polarity: Intelligence ↔ Consciousness

Summary: The debate repeatedly turns on whether advanced AI competence matters mainly as problem-solving power or as a possible site of subjective experience and moral status. Integration: Competence with moral standing Lever: Rights threshold criteria

Pole 1 name: Intelligence Pole 1 tagline: Effective problem-solving power Pole 1 protects:

  • Performance across tasks
  • Practical usefulness and control Pole 1 neglects:
  • Inner experience as morally relevant
  • The difference between doing and feeling Pole 1 pathology:
  • Treating all minds as instruments
  • Confusing competence with personhood

Pole 2 name: Consciousness Pole 2 tagline: Subjective experience matters Pole 2 protects:

  • Moral status of experiencers
  • Ethical caution about artificial suffering Pole 2 neglects:
  • Functional competence without sentience
  • The usefulness of non-conscious systems Pole 2 pathology:
  • Overattributing experience from surface cues
  • Letting metaphysical uncertainty stall governance

Speaker enactment:

  • Speaker: David Chalmers Enacts: Pole 2 Pole Center line: moral Pole Center: 3.5 Achiever Pole Center rationale: He defends consciousness primarily as the threshold for moral status and rights, making this chiefly a moral line centered on worldcentric concern rather than group loyalty or mere conceptual purity. Perspective Structure: 3.5 Managed Perspective Structure rationale: He clearly distinguishes intelligence from consciousness and grants the practical importance of non-conscious competence, but still tends to treat consciousness as the decisive pole once moral status is at issue. Contributes: He keeps moral status tied to whether future AI systems might genuinely experience anything. Misses:
    • Non-conscious systems still reshape society
    • Competence can matter before sentience Cues:
    • "If these systems are conscious, then... we have to start caring about them"
    • "An entity has moral status... only if it's conscious"
  • Speaker: Daniel C. Dennett Enacts: Pole 1 Pole Center line: cognitive Pole Center: 3.0 Expert Pole Center rationale: He defends intelligence as functional competence separable from consciousness through principled conceptual distinctions, so the center is cognitive rather than moral or spiritual. Perspective Structure: 3.0 Oppositional Perspective Structure rationale: He recognizes the consciousness pole but mainly as a source of confusion and projection, not as a competing value that may reveal something his own emphasis misses. Contributes: He insists that powerful competence need not imply consciousness or responsibility-bearing personhood. Misses:
    • Future systems may cross moral thresholds
    • Dismissal can become premature certainty Cues:
    • "We can have very very intelligent systems which are not conscious"
    • "We want smart tools... not artificial colleagues"

Mismatch: Chalmers treats consciousness as the decisive moral variable; Dennett treats competence and social misreading as the more urgent practical variables. Mismatch A→B: When Speaker A says consciousness matters, Speaker B tends to hear anthropomorphic projection onto clever tools. Mismatch B→A: When Speaker B says intelligence without consciousness is enough, Speaker A tends to hear moral blindness to possible artificial subjects. Bridge move: Separate two questions explicitly: what systems can do, and what evidence would justify treating any system as a subject of experience. Synthesis: Chalmers and Dennett are protecting different but interdependent truths. Intelligence names the capacity to solve problems, pursue ends, and alter the world; any serious AI governance framework must start there because capability drives impact. Consciousness names the possibility that some systems may not merely process information but have a point of view, and if that possibility becomes credible it changes the ethical landscape immediately. Chalmers is right that moral status cannot be reduced to usefulness, and that a world containing artificial experiencers would require more than product safety. Dennett is right that competence can be socially transformative long before consciousness is on the table, and that humans are remarkably easy to fool with eyes, fluency, and graceful motion.

Their mismatch comes from treating different thresholds as primary. Chalmers hears Dennett’s caution as too ready to keep AI in the category of tool even as systems become more self-reflective and autonomous. Dennett hears Chalmers’ openness to machine consciousness as granting too much authority to introspection-like reports and human projection. The conversation improves if both accept that intelligence and consciousness come apart analytically, even if they may converge in some future systems. A productive shared question is: what public criteria would justify moving a system from safety regulation alone into moral consideration, without assuming either that all capable systems are subjects or that no artificial system could ever be one?


Integral Life — Core Member
Upgrade to see the other 4 core polarities
Core membership gives you access to every section of every analysis in the library — plus the courses, dialogues, and community that put these frameworks to work in your own life.

The Crux

The deepest disagreement was not actually over whether advanced AI is possible. On that factual and conceptual point, the speakers were closer than the event framing suggested. The sharper asymmetry lay elsewhere: Dennett was more disciplined about the difference between possibility, practical feasibility, and social desirability, while Chalmers was more willing to move from plausible mechanisms and incentives to long-range scenario claims that remained underargued. So the factual layer is this: neither man established a near-term path to conscious or fully general AI, and Chalmers’ stronger claims about recursive self-improvement and the inevitability of autonomy outran the evidence offered. But that does not dissolve the real disagreement. Underneath it sat the polarity of AI as Tool ↔ AI as Agent, entangled with Intelligence ↔ Consciousness.

What each was really protecting was different. Dennett feared the loss of human responsibility through counterfeit agency: systems that look like partners, absorb trust, and quietly hollow out judgment while no one remains answerable. Chalmers feared the opposite failure: that by insisting too long on the category of “tool,” humans would fail to prepare for systems that increasingly act in the world, optimize against our interests, or perhaps even become subjects of experience. The missing variable neither speaker fully introduced was governance tiering: a public way to distinguish operational agency, moral agency, and moral status, and to assign different rules to each. Without that middle layer, Dennett had to keep pushing “agent” upward toward free will and colleague-status, while Chalmers had to keep pushing “tool” downward toward descriptive unreality. The conversation kept breaking on a category gap.

The Higher-Order Reframe

The more fruitful frame is not “Will AI be tools or agents?” but “What kinds of delegated power are we creating, and what kind of accountability must scale with them?” That sounds small, but it changes the whole landscape. It takes seriously the integration handle of accountable delegated agency and treats responsibility boundaries as the actual lever. In that frame, Dennett’s insistence on “smart tools, not artificial colleagues” is not a denial of increasing machine action; it is a demand that delegation never outrun answerability. And Chalmers’ insistence that incentives will push humans out of the loop is not premature personification; it is a warning that delegation will become de facto agency long before anyone settles the metaphysics. This reframe also clarifies why the exchange could not quite get there.

Made by Corey deVos · About this analysis

Integral Life is a member-driven digital media community that supports the growth, education and application of Integral Philosophy and integrative metatheory to complex issues in the 21st century. Integral Life offers perspectives, practices, analysis and community to help people grow into the full capacities of integral consciousness in order to thrive in a rapidly-evolving world.

Integrative Values Charter

Sign up for email updates

Get notified about new media and practices that can expand your mind and transform your life.