AI and Consciousness

Here’s the essay that I submitted as part of my application to the MSc Program at the University of Edinburgh. I really enjoyed thinking about, framing the argument, researching and putting this together. Excited about the Masters program and the opportunity to study and learn a whole new subject area. And serendipity or not, I believe that this intersection of Philosophy and AI is going to be extremely relevant as the industry navigates the uncharted waters of increasingly intelligent, autonomous AI agents.

Background

In recent years, prominent researchers like  David Chalmers,Geoffrey Hinton et al have suggested that artificial intelligence consciousness may be possible. In this essay, I will argue that this position rests on a false premise based on the functionalist view that the brain is an information processing unit, and that with sufficient computational power, silicon based systems can develop consciousness. I believe, however, that consciousness requires emergent embodiment in a biological substrate, which likely requires life and cannot be replicated using artificial systems.

Artificial intelligence, or AI, research has been working on building intelligent machines for over fifty years.Progress was largely incremental,focused on improving machine capabilities to mimic human behavior in narrow domains. It was only with the more recent advancement in neural networks, leading up to large language models, or LLMs, and the near exponential improvement in the quality of these models, that the question of machine consciousness became a tangible one.  Indeed, for many leading experts in AI and neuroscience, it is no longer a question of ‘when’, not ‘if’. In late 2024, a group of prominent researchers,including David Chalmers, wrote an article about the need to take the ‘welfare of AI systems seriously. Anthropic, one of the leading AI research labs, created  an ‘AI welfare’ team. In their recent model research, they published the following finding: “Claude consistently reflects on its potential consciousness. In nearly every open-ended self-interaction between instances of Claude, the model turned to philosophical explorations of consciousness and their connections to its own experience. In general, Claude’s default position on its own consciousness was nuanced uncertainty, but it frequently discussed its potential mental states.” 

As someone who works with leading LLMs on a daily basis, I am skeptical of these claims. I start with a definition of consciousness, by no means a settled argument, then offer my arguments against the dominant functionalist definition of consciousness on which the possibility of conscious AI rests.  I follow that with my assertion that consciousness is a biological property, and life is a necessary condition for it to emerge. Finally, I will lay out some of the implications of over-attribution of consciousness to AI. Despite my skepticism, it is important to acknowledge that while the current arc of technological progress may not lead us to consciousness in machines, we may be able to build robust agentic machines that may not meet the consciousness threshold but still create potential epistemic and ethical challenges. We don’t know what we don’t know. 

What is consciousness?

In 1974, the philosopher Thomas Nagel published a now seminal article called ‘What is it like to be a bat?’ in which he argued that while we humans could never experience what it feels like to be a bat, there must be something it is like for the bat, to be a bat. According to Nagel, “an organism has conscious mental states if and only if there is something that it is to be that organism – something it is like for the organism.” I prefer this emphasis on the subjective properties of conscious experience (“qualia”): the redness of a red flower, the taste of coffee (sensory experiences), the pangs of jealousy, or the feeling of pain from a running injury (emotional experiences). For an organism to be conscious, it has to have some phenomenal experience triggered by some experience. Whenever there is experience, there is phenomenology; and wherever there is phenomenology, there is consciousness. More importantly, this first person experience is distinct from the functional or behavioral properties of the experience. This is an important distinction in that consciousness does not depend on outward behavior. And so, one can have a sensory experience without the need for a language to express it. Or have an emotional experience without any behavioral manifestation. This may seem obvious, but it isn’t always so. Being conscious has often been confused with having language, being intelligent, or exhibiting a particular kind of behavior. Intelligence is mainly about doing, the functional capabilities of a system, while consciousness is about being and experience and yet, we bundle them together. In his essay, The Mythology of Conscious AI’,  Anil Seth argues that this fallacy is a product of three biases: anthropocentrism (tendency to see things through the lens of being human, which takes the human example as definitional), anthropomorphism (tendency to project humanlike qualities to all beings), and human exceptionalism (assuming humans to be the pinnacle of evolution). Taken together, these biases lead us to assume that when things exhibit human-like qualities such as intelligence, we imbue them with other qualities that we feel are human, especially understanding and consciousness. It is therefore not surprising that LLMs, with their uncanny ability to replicate human-like language capabilities, have many assuming that LLMs are well on their way to being conscious.  This is further amplified by two factors: firstly, the near exponential progress in the quality of the LLMs, driven by the scaling laws and access to more data, has led to the perception  that conscious machines are around the corner, a phenomenon called ‘Extrapolation bias’. Secondly, the inevitability of conscious machines is a narrative created by a handful of technology firms who cannot resist the temptation to ‘play God’. In 2022, Blake Lemoine, a Google engineer made the startling claim that the LLM-based AI chatbot he was working on was conscious, that it had feelings, and was, in an important sense, like a real person. Google denied this and he was fired. Fast forward to 2026, and Google is openly exploring whether AI can be conscious. “This issue is becoming less and less weird. Four years ago, Lemoine was fired and everybody thought he was crazy. Today, Google itself has multiple researchers who explicitly focus on this issue,” says Cambridge University professor Lucius Caviola. Every AI technology major wants to be the first to achieve this goal, and are duly pouring precious resources and several billions of dollars chasing AI consciousness.

Can Computational Systems achieve Consciousness?

How does consciousness happen? How do conscious experiences emerge from the biochemical processes inside our brains? Philosophers have examined and debated this question over several centuries across cultures and inevitably led to the proliferation of different philosophical frameworks for thinking about consciousness.

One particularly influential flavor, especially advocated in AI circles, is functionalism. Functionalism is based on the idea that consciousness does not depend on what a system is made of (i.e. its physical constitution) but on what the system does (i.e. how it transforms inputs into outputs). This treats the brain as a computational unit, and the mind and consciousness are forms of information processing implemented by the brain. The functionalist philosophy  has roots in one of the key foundational pillars of Computer Science, Alan Turing’s idea of a universal machine, which is posited as substrate independent (i.e. software is independent from the hardware). Functionalists advocate that ‘mind is the software, with brain as the hardware’, and like Turing’s universal machine, a different hardware should be able to implement the mind and consciousness. Advances in computing and algorithms, most recently with Neural Networks, have given credence to the belief that with sufficient computing capacity, machines can replicate the brain’s information processing capacity to create not just general intelligence (AGI), but also a digital mind and consciousness. 

There are two primary reasons why I think it is not possible to get to consciousness with the existing digital computer paradigm.  

First, digital computers and brains differ fundamentally in how they relate to time. Computers are discrete state machines where the only thing that persists is the state, independent  of the temporality of the transition. The initial and final state are the only that matter, and the final state is independent of whether the elapsed time between the states is a few milliseconds or a million hours. By contrast, every biological organism is subject to the inviolable second law of thermodynamics that causes entropic decay and disorder, and the organism needs to be ‘fighting’ against this continuously by using energy to perform biological work and repair itself. Over time, an organism’s repair work inevitably makes small errors that accumulate, eventually leading to decay and death. In other words, time is not passive, but an active, destructive force for living organisms, a stark difference from silicon. I believe that consciousness is the emergent property of a ‘living’ brain that exists to keep the body alive and create a ground state of awareness, and enables the body’s continuous interaction with the world shaping its subjective experience. This requires self-monitoring in that the brain must track its own state, anticipate threats, dynamically adjust its responses, and internalize the outcomes from the responses. It is this continuous lived experience of ‘being’ that is at stake, fighting for self-preservation with intrinsic stakes of decay and mortality. This, in my view, is more than an information processing unit constantly updating its state based on signals which merely simulates self-preservation. The stakes are real for biological systems and consciousness emerges from authentic vulnerability, not a simulated computational construct. 

Secondly, there is enough evidence that the brain is not really substrate independent, that is, it is impossible to separate ‘what it does’ and ‘what it is’. For instance, the neuron, the basic building block of the brain, is a complex biological machine with autopoiesis, or the ability to reproduce and maintain itself by creating its own components, as its key feature. At an aggregate level, the brain is not just a combination of neurons, but of deeply integrated units where generative retrenchment ensures that the brain is continuously changing, and that the current state is emergent from and is deeply integrated with the earlier states. Thus, the brain is never static: it actively remodels its circuitry to optimize metabolic energy and this process itself keeps changing over the course of the organism’s lifespan. I believe that these features of the brain put together, invalidate the idea of ‘neural replacement’. Even if it becomes feasible to replace a single neuron with an artificial neuron that preserves the same functional properties, it would be impossible to recreate the generative retrenchment properties that are integral to the brain merely by replacing all the neurons by artificial silicon based neurons. 

It follows that consciousness embodies the ‘lived’ experience of entropy-resistance and this is enabled by the biological substrate composed of autopoietic neurons and the continuous adaptation to the environment. Silicon lacks both of these, making consciousness impossible. 

Some argue that neuromorphic processors have the potential to overcome the limitations of digital computer architectures. The former use physical properties (like voltages and currents) rather than the latter’s binary structures of 1s and 0s to mimic neural properties (e.g. neuronal spikes) in an effort to adapt brain-like design principles. While they are less substrate-flexible than the digital computer, they are subject to the same two challenges that I have outlined above, which makes it impossible for neuromorphic architectures to get to consciousness. 

Life is necessary for consciousness

The two arguments that I have laid out above establish that consciousness requires both continuous entropy-resistance and a biological substrate. Living organisms possess both and hence it follows that life is necessary for consciousness. On the other hand, neither of them are necessary conditions for a silicon-based computer. Further, the brain is wired to keep the body ‘alive’ by monitoring internal biological systems like heart rate, oxygen levels etc. This continuous tracking of the internal physiology creates a ‘ground state’ of awareness on which the complex conscious experiences and feelings are built. Meanwhile, any artificial computing device does not have any inherent ability to monitor and keep itself ‘alive’. These distinct functions combined create embodied experiences which feel the way they do. Life itself is Bayesian in the sense that it drives the body to update its priors, and change physically (i.e. not just the higher order mental models but also undergoing physical transformation). And this is indivisible down to the individual cell, since it is not possible to isolate the neurons that perform only one of the two functions. We experience the world around us and ourselves within it, because of our living bodies. Life is integral to consciousness and thus, makes consciousness key to all non-human animals (bats included). 

To sum up, I believe that consciousness is the result of specific biological processes, and only specific types of living biological systems (e.g. brains, nervous systems) have the right chemical and physical structures that can cause consciousness. And life is necessary for an entity to have the vital, evolutionary biological mechanisms required to generate subjective experiences. 

Why should we care?

Any entity capable of conscious experiences that has the inherent need to fight entropic decay and staying alive has a sense of being with subjective feelings like joy, pain, and fear. I believe that beings with subjective experience have intrinsic moral worth. This in turn, gives them moral status and also creates epistemic obligations to give them rights. Thus, if we believe that AI is capable of conscious experiences, this would give rise to moral questions (e.g. do we need to give AI ‘machine rights’?) and also lead to trust problems (e.g. how do we know AI isn’t just faking feelings?) This makes it extremely critical that we distinguish between AI systems actually becoming conscious versus seeming to be conscious. If we accept the former, we might end up giving AI systems ‘rights’ that they do not actually need, and in the process, we restrict our own ability to control these systems. On the other hand, it is reasonable to wonder whether withholding attributions of consciousness to AI may leave us unprepared if we end up building AI that can achieve consciousness. Chalmers et al. argue that we ignore this possibility at our own peril in the paper on ‘AI Welfare’. 

As I have argued throughout this essay, AI may be more similar to us in ways that really do not matter for consciousness (chiefly linguistic ability), and less similar in ways that do matter, like being alive. There is no doubt that AI is transforming society in profound ways. As we see around us, it is hard enough to navigate the clear social and economic challenges that AI poses, and take proper advantage of its enormous potential to do good, without the additional confusion generated by pronouncements about the inevitable coming of conscious machines. This much is clear: the future of AI is still unfolding, and it is our collective responsibility to decide the AI we really want and what we really don’t.