Three Pound Brain

No bells, just whistling in the dark…

Tag: cognitive ecology

The Crash of Truth: A Critical Review of Post-Truth by Lee C. Mcintyre

by rsbakker

Lee Mcintyre is a philosopher of science at Boston University, and author of Dark Ages: The Case for a Science of Human Behaviour. I read Post-truth on the basis of Fareed Zakaria’s enthusiastic endorsement on CNN’s GPS, so I fully expected to like it more than I ultimately did. It does an admirable job scouting the cognitive ecology of post-truth, but because it fails to understand that ecology in ecological terms, the dynamic itself remains obscured. The best Mcintyre can do is assemble and interrogate the usual suspects. As a result, his case ultimately devolves into what amounts to yet another ingroup appeal.

As perhaps, we should expect, given the actual nature of the problem.

Mcintyre begins with a transcript of an interview where CNN’s Alisyn Camerota presses Newt Gingrich at the 2016 Republican convention on Trump’s assertions regarding crime:

GINGRICH: No, but what I said is equally true. People feel more threatened.

CAMEROTA: Feel it, yes. They feel it, but the facts don’t support it.

GINGRICH: As a political candidate, I’ll go with how people feel and let you go with the theoreticians.

There’s a terror you feel in days like these. I felt that terror most recently, I think, watching Sarah Huckabee Sanders insisting that the out-going National Security Advisor, General H. R. McMaster, had declared that no one had been tougher on Russia than Trump after a journalist had quoted him saying almost exactly otherwise. I had been walking through the living-room and the exchange stopped me in my tracks. Never in my life had I ever witnessed a Whitehouse Official so fecklessly, so obviously, contradict what everyone in the room had just heard. It reminded me of the psychotic episodes I witnessed as a young man working tobacco with a friend who suffered schizophrenia—only this was a social psychosis. Nothing was wrong with Sarah Huckabee Sanders. Rather than lying in malfunctioning neural machinery, this discrepancy lay in malfunctioning social machinery. She could say what she said because she knew that statements appearing incoherent to those knowing what H. R. McMaster had actually said would not appear as such to those ignorant of or indifferent to what he had actually said.  She knew, in other words, that even though the journalists in the room saw this:

given the information available to their perspective, the audience that really mattered would see this:

which is to say, something rendered coherent for neglecting that information.

The task Mcintyre sets himself in this brief treatise is to explain how such a thing could have come to pass, to explain, not how a sitting President could lie, but how he could lie without consequences. When Sarah Huckabee Sanders asserts that H. R. McMaster’s claim that the Administration is not doing enough is actually the claim that no Administration has done more she’s relying on innumerable background facts that simply did not obtain a mere generation ago. The social machinery of truth-telling has fundamentally changed. If we look at the sideways picture of Disney’s faux New York skyline as the ‘deep information view,’ and the head-on picture as the ‘shallow information view,’ the question becomes one of how she could trust that her audience, despite the availability of deep information, would nevertheless affirm the illusion of coherence provided by the shallow information view. As Mcintyre writes, “what is striking about the idea of post-truth is not just that truth is being challenged, but that it is being challenged as a mechanism for asserting political dominance.” Sanders, you could say, is availing herself of new mechanisms, ones antagonistic to the traditional mechanisms of communicating the semantic authority of deep information. Somehow, someway, the communication of deep information has ceased to command the kinds of general assent it once did. It’s almost preposterous on the face of it: in attributing Trump’s claims to McMaster, Sanders is gambling that somehow, either by dint of corruption, delusion, or neglect, her false claim will discharge functions ideally belonging to truthful claims, such as informing subsequent behaviour. For whatever reason, the circumstances once preventing such mass dissociations of deep and shallow information ecologies have yielded to circumstances that no longer do.

Mcintyre provides a chapter by chapter account of those new circumstances. For reasons that will become apparent, I’ll skip his initial chapter, which he devotes to defining ‘post-truth,’ and return to it in the end.

Science Denial

He provides clear, pithy outlines of the history of the tobacco industry’s seminal decision to argue the science, to wage what amounts to an organized disinformation campaign. He describes the ways resource companies adapted these tactics to scramble the message and undermine the authority of climate science. And by ‘disinformation,’ he means this literally, given “that even while ExxonMobil was spending money to obfuscate the facts about climate change, they were making plans to explore new drilling opportunities in the Arctic once the polar ice cap had melted.” This part of the story is pretty well-known, I think, but Mcintyre tells the tale in a way that pricks the numbness of familiarity, reminding us of the boggling scale of what these campaigns achieved: generating a political/cultural alliance that is—not simply bent on—hastening untold misery and global economic loss in the name of short term parochial economic gain.

Cognitive Bias

He gives a curiously (given his background) two-dimensional sketch of the role cognitive bias plays in the problem, focusing primarily on cognitive dissonance, our need to minimize cognitive discrepancies, and the backfire effect, how counter-arguments actually strengthen, as opposed to mitigate, commitment to positions. (I would recommend Steven Sloman and Philip Fernbach’s The Knowledge Illusion for a more thorough consideration of the dynamics involved). He discusses research showing the profound ways that social identification, even cued by things so flimsy as coloured wristbands, profoundly transforms our moral determinations. But he underestimates, I think, the profound nature of what Dan Kahan and his colleagues call the “Tragedy of the Risk-Perception Commons,” the individual rationality of espousing irrational collective claims. There’s so much research directly pertinent to his thesis that he passes over in silence, especially that belonging to ecological rationality.

Traditional versus social media

If Mcintyre’s consideration of the cognitive science left me dissatisfied, I thoroughly enjoyed his consideration of media’s contribution to the problem of post-truth. He reminds us that the existence of entities, like Fox News, disguising advocacy as disinterested reporting, is the historical norm, not the rule. Disinterested journalistic reporting was more the result how AP, which served papers grinding different political axes, required stories expressing as little overt bias as possible. Rather than seize upon this ecological insight (more on this below), he narrates the gradual rise of television news from small, money-losing network endeavours, to money-making enterprises culminating in CNN, Fox, MSNBC, and the return of ‘yellow journalism.’

He provides a sobering assessment of the eclipse of traditional media, and the historically unprecedented rise of social media. Here, more than anywhere else, we find Mcintyre taking steps toward a genuine cognitive ecological understanding of the problem:

“In the past, perhaps our cognitive biases were ameliorated by our interactions with others. It is ironic to think that in today’s media deluge, we could perhaps be more isolated from contrary opinion than when our ancestors were forced to live and work among other members of their tribe, village, or community, who had to interact with one another to get information.”

Since his understanding of the problem is primarily normative, however, he fails to see how cognitive reflexes that misfire in experimental contexts, and so strike observers as normative breakdowns, actually facilitate problem-solving in ancestral contexts. What he notes as ‘ironic’ should strike him (and everyone else) as astounding, as one of the doors that any adequate explanation of post-truth must kick down. But it is heartening, I have to say, to see these ideas begin to penetrate more and more brainpans. Despite the insufficiency of his theoretical tools, Mcintyre glimpses something of the way cognitive technology has impacted human cognitive ecology: “Indeed,” he writes, “what a perfect storm for the exploitation of our ignorance and cognitive biases by those with an agenda to put forward.” But even if the ‘perfect storm’ metaphor captures the complex relational nature of what’s happened, it implies that we find ourselves suffering a spot of bad luck, and nothing more.

Postmodernism

At last he turns to the role postmodernism has played in all this: this is the only chapter where I smelled a ‘legacy effect,’ the sense that the author is trying to shoe-horn some independently published material.

He acknowledges that ‘postmodernism’ is hopelessly overdetermined, but he thinks two theses consistently rise above the noise: the first is that “there is no such thing as objective truth,” and the second is “that any profession of truth is nothing more than a reflection of the political ideology of the person who is making it.”

To his credit, he’s quick to pile on the caveats, to acknowledge the need to critique both the possibility of absolute truth as well as the social power of scientific truth-claims. Because of this, it quickly becomes apparent that his target isn’t so much ‘postmodernism’ as it is social constructivism, the thesis that ‘truth-telling,’ far from connecting us to reality, bullies us into affirming interest serving constructs. This, as it turns out, is the best way to think post-truth “[i]n its purest form” as “when one thinks that the crowd’s reaction actually does change the facts about a lie.”

In other words, for Mcintyre, post-truth is the consequence of too many people believing in social constructivism—or in other words, presuming the wrong theory of truthHis approach to the question of post-truth is that of a traditional philosopher: if the failure is one of correspondence, then the blame has to lie with anti-correspondence theories of truth. The reason Sarah Huckabee Sanders could lie about McMaster’s final speech turns on (among other things) the wide-spread theoretical belief that there is no such thing as objective truth,’ that it’s power plays all the way down.

Thus the (rather thick) irony of citing Daniel Dennett—an interpretivist!—stating that “what the postmodernists did was truly evil” so far as they bear responsibility “for the intellectual fad that made it respectable to be cynical about truth and facts.”

The sin of the postmodern left has very, very little to do with generating semantically irresponsible theoriesDennett’s own positions are actually a good deal more radical in this regard! When it comes to the competing narratives involving ‘meaning of’ questions and answers, Dennett knows we have no choice but to advert to the ‘dramatic idiom’ of intentionality. If the problem were one of providing theoretical ammunition then Dennett is as much a part of the problem as Baudrillard.

And yet Mcintyre caps Dennett’s assertion by asking, “Is there more direct evidence than this?” Not a shining moment, dialectically speaking.

I agree with him that tools have been lifted from postmodernists, but they have been lifted from pragmatists (Dennett’s ilk) as well. Talk of ‘stances’ and ‘language games’ is also rife on the right! And I should know. What’s happening now is the consequence of a trend that I’ve been battling since the turn of the millennium. All my novels constitute self-conscious attempts to short-circuit the conditions responsible for ‘post-truth.’ And I’ve spent thousands of hours trolling the alt-Right (before they were called such) trying to figure out what was going on. The longest online debate I ever had was with a fundamentalist Christian who belonged to a group using Thomas Kuhn to justify their belief in the literal truth of Genesis.

Defining Post-truth

Which brings us, as promised, back to the book’s beginning, the chapter that I skipped, where, in the course of refining his definition of post-truth, Mcintyre acknowledges that no one knows what the hell truth is:

“It is important at this point to give at least a minimal definition of truth. Perhaps the most famous is that of Aristotle, who said: ‘to say of what is that it is not, or of what is not, that it is, is false, while to say of what is that it is, and what of is not that it is not, is true.’ Naturally, philosophers have fought for centuries over whether this sort of “correspondence” view is correct, whereby we judge the truth of a statement only by how well it fits reality. Other prominent conceptions of truth (coherentist, pragmatist, semantic) reflect a diversity of opinion among philosophers about the proper theory of truth, even while—as a value—there seems little dispute that truth is important.”

He provides a minimal definition with one hand—truth as correspondence—which he immediately admits is merely speculative! Truth, he’s admitting, is both indispensable and inscrutable. And yet this inscrutability, he thinks, need not hobble the attempt to understand post-truth: “For now, however, the question at hand is not whether we have the proper theory of truth, but how to make sense of the different ways that people subvert truth.”

In other words, we don’t need to know what is being subverted to agree that it is being subverted. But this goes without saying; the question is whether we need to know what is being subverted to explain what Mcintyre is purporting to explain, namely, how truth is being subverted. How do we determine what’s gone wrong with truth when we don’t even know what truth is?

Mcintyre begins Post-truth, in other words, by admitting that no canonical formulation of his explanandum exists, that it remains a matter of mere speculation. Truth remains one of humanity’s confounding questions.

But if truth is in question, then shouldn’t the blame fall upon those who question truth? Perhaps the problem isn’t this or that philosophy so much as philosophy itself. We see as much at so many turns in Mcintyre’s account:

“Why not doubt the mainstream news or embrace a conspiracy theory? Indeed, if news is just political expression, why not make it up? Whose facts should be dominant? Whose perspective is the right one? Thus is postmodernism the godfather of post-truth.”

Certainly, the latter two questions belong to philosophy as whole, and not postmodernism in particular. To that extent, the two former questions—so far as they follow from the latter—have to be seen as falling out of philosophy in general, and not just some ‘philosophical bad apples.’

But does it make sense to blame philosophy, to suggest we should have never questioned the nature of truth? Of course not.

The real question, the one that I think any serious attempt to understand post-truth needs to reckon, is the one Mcintyre breezes by in the first chapter: Why do we find truth so difficult to understand?

On the one hand, truth seems to be crashing. On the other, we have yet to take a step beyond Aristotle when it comes to answering the question of the nature of truth. The latter is the primary obstacle, since the only way to truly understand the nature of the crash is to understand the nature of truth. Could the crash and the inscrutability of truth be related? Could post-truth somehow turn on our inability to explain truth?

Adaptive Anamorphosis

Truth lies murdered in the Calais Coach, and Mcintyre has assembled all the suspects: denialism, cognitive biases, traditional and social media, and (though he knows it not) philosophy. He knows all of them had some part to play, either directly, or as accessories, but the Calais Coach remains locked—his crime scene is a black box. He doesn’t even have a body!

For me, however, post-truth is a prediction come to pass—a manifestation of what I’ve long called the ‘semantic apocalypse.’ Far from a perfect storm of suspects coming together in unlikely ways to murder ‘all of factual reality,’ it is an inevitable consequence of our rapidly transforming cognitive ecologies.

Biologically speaking, human communication and cooperation represent astounding evolutionary achievements. Human cognition is the most complicated thing human cognition has ever encountered: only now are we beginning to reverse-engineer its nature, and to use that knowledge to engineer unprecedented cognitive artifacts. We know that cognition is structurally and dynamically composite, heavily reliant on heuristic specialization to solve its social and natural environments. The astronomical complexity of human cognition means that sociocognition and metacognition are especially reliant on composite, source-insensitive systems, devices turning on available cues that correlate, given that various hidden regularities obtain, with specific outcomes. Despite being legion, we manage to synchronize with our fellows and our environments without the least awareness of the cognitive machinery responsible.

We suffer medial neglect, a systematic insensitivity to our own nature—a nature that includes this insensitivity. Like every other organism on this planet we cognize without cognizing the concurrent act of cognition. Well, almost like every other organism. Where other species utterly depend on the reliability of their cognitive capacities, have no way of repairing failures in various enabling—medial—systems, we do have recourse. Despite our blindness to the machinery of human cognition, we’ve developed a number of different ways to nudge that machinery—whack the TV set, you could say.

Truth-talk is one of those ways. Truth-talk allows us to minimize communicative discrepancies absent, once again, sensitivity to the complexities involved. Truth-talk provides a way to circumvent medial neglect, to resolve problems belonging to the enabling dimension of cognition despite our systematic insensitivity to the facts of that dimension. When medial issues—problems pertaining to cognitive function—arise, truth-talk allows for the metabolically inexpensive recovery of social and environmental synchronization. Incompatible claims can be sorted, at least so far as our ancestors required in prehistoric cognitive ecologies. The tribe can be healed, despite its profound ignorance of natures.

To say human cognition is heuristic is to say it is ecologically dependent, that it requires the neglected regularities underwriting the utility of our cues remain intact. Overthrow those regularities, and you overthrow human cognition. So, where our ancestors could simply trust the systematic relationship between retinal signals and environments while hunting, we have to remove our VR goggles before raiding the fridge. Where our ancestors could simply trust the systematic relationship between the text on the page or the voice in our ear and the existence of a fellow human, we have to worry about chatbots and ‘conversational user interfaces.’ Where our ancestors could automatically depend on the systematic relationship between their ingroup peers and the environments they reported, we need to search Wikipedia—trust strangers. More generally, where our ancestors could trust the general reliability (and therefore general irrelevance) of their cognitive reflexes, we find ourselves confronted with an ever growing and complicating set of circumstances where our reflexes can no longer be trusted to solve social problems.

The tribe, it seems, cannot be healed.

And, unfortunately, this is the very problem we should expect given the technical (tactical and technological) radicalization of human cognitive ecology.* Philosophy, and now, cognitive science, provide the communicative tactics required to neutralize (or ‘threshold’) truth-talk. Cognitive technologies, meanwhile, continually complicate the once direct systematic relationships between our suites of cognitive reflexes and our social and natural environments. The internet doesn’t simply render the sum of human knowledge available, it also renders the sum of human rationalization available as well. The curious and the informed, meanwhile, no longer need suffer the company of the incurious and the uninformed, and vice versa. The presumptive moral superiority of the former stands revealed: and in ever greater numbers the latter counter-identify, with a violence aggravated by phenomena such as the ‘online disinhibition effect.’ (One thing Mcintyre never pauses to consider is the degree to which he and his ilk are hated, despised, so much so as to see partners in traditional foreign adversaries, and to think lies and slander simply redress lies and slander). Populations begin spontaneously self-selecting. Big data identifies the vulnerable, who are showered with sociocognitive cues—atrocity tales to threaten, caricatures to amuse—engineered to provoke ingroup identification and outgroup alienation. In addition to ‘backfiring,’ counter-arguments are perceived as weapons, evidence of outgroup contempt for you and your own. And as the cognitive tactics become ever more adept at manipulating our biases, ever more scientifically informed, and as the cognitive technology becomes ever more sophisticated, ever more destructive of our ancestral cognitive habitat, the break between the two groups, we should expect, will only become more, not less, profound.

None of this is intuitive, of course. Medial neglect means reflection is source blind, and so inclined to conceive things in super-ecological terms. Thus the value of the prop building analogy I posed at the beginning.

Disney’s massive Manhattan anamorph depends on the viewer’s perspectival position within the installation to assure the occlusion of incompatible information. The degrees of cognitive freedom this position possesses—basically, how far one can wander this way and that—depends on the size and sophistication of the anamorph. The stability of illusion, in other words, entirely depends on the viewer: the deeper one investigates, the less stable the anamorph becomes. Their dependence on cognitive ‘sweet spots’ is their signature vulnerability.

The cognitive fragility of the anamorph, however, resides in the fact that we can move, while it cannot. Overcoming this fragility, then, either requires 1) de-animating observation, 2) complicating the anamorph, or 3) animating the anamorph. The problem we face can be understood as the problem of adaptive cognitive anamorphosis, the way cognitive science, in combination with cognitive technology, enables the de-animation of information consumers by gaming sociocognitive cues, while both complicating and animating the artifactual anamorphic information they consume.

Once a certain threshold is crossed, Sarah Huckabee Sanders can lie without shame or apology on national television. We don’t know what we don’t know. Mcintyre references the notorious Dunning-Kruger effect, the way cognitive incompetence correlates with incompetent assessments of competence, but the underlying mechanism is more basic: cognitive systems lacking access to information function independent of that information. Medial neglect assures we take the sufficiency of our perspectives for granted absent information indicating insufficiency or ‘medial misalignment.’ Trusting our biology and community is automatic. Perhaps we refuse to move, to even consider the information belonging to:

But if we do move, the anamorph, thanks to cognitive technology, adapts, the prop-facades grow prop sides, and the deep (globally synchronized) information presented above, has to compete with ‘faux deep’ information. The question becomes one of who has been systematically deceived—a question that ingroup biases have already answered in illusion’s favour. We can return to our less inquisitive peers and assure them they were right all along.

What is ‘post-truth’? Insofar as it names anything it refers to diminishing capacity of globally, versus locally, synchronized claims to drive public discourse. It’s almost as if, via technology, nature is retooling itself to conceal itself by creating adaptive ‘faux realities.’ It’s all artifactual, all biologically ‘constructed’: the question is whether our cognitive predicament facilitates global (or deep) synchronization geared to what happens to be the case, or facilitates local (or shallow) synchronization geared to ingroup expectations and hidden political and commercial interests.

There’s no contest between spooky correspondence and spooky construction. There’s no ‘assertion of ideological supremacy,’ just cognitive critters (us) stranded in a rapidly transforming cognitive ecology that has become too sophisticated to see, and too powerful to credit. Post-truth, in other words, is an inevitable consequence of scientific progress, particularly as it pertains to cognitive technologies.

Sarah Huckabee Sanders can lie without shame or apology on national television because Trump was able to lure millions of Americans across a radically transformed (and transforming) anamorphic threshold. And we should find this terrifying. Most doomed democracies elect their executioner. In his The Death of Democracy: Hitler’s Rise to Power, Benjamin Carter Hett blames the success of Nazism on the “reality deficit” suffered by the German people. “Hostility to reality,” he writes, “translated into contempt for politics, or, rather, desire for a politics that was somehow not political: a thing that can never be” (14). But where Germany in the 1930’s had every reason to despise the real, “a lost war that had cost the nation almost two million of her sons, a widely unpopular revolution, a seemingly unjust peace settlement, and economic chaos accompanied by huge social and technological change” (13), America finds itself suffering only the latter. The difference lies in the way the latter allows for the cultivation and exploitation of this hostility in an age of unparalleled peace and prosperity. In the German case, the reality itself drove the populace to embrace atavistic political fantasies. Thanks to technology, we can now achieve the same effect using only human cognitive shortcomings and corporate greed.

Buckle up. No matter what happens to Trump, the social dysfunction he expresses belongs to the very structure of our civilization. Competition for the market he’s identified is only going to intensify.

 

Advertisements

Enlightenment How? Pinker’s Tutelary Natures

by rsbakker

 

The fate of civilization, Steven Pinker thinks, hangs upon our commitment to enlightenment values. Enlightenment Now: The Case for Reason, Science, Humanism and Progress constitutes his attempt to shore up those commitments in a culture grown antagonistic to them. This is a great book, well worth the read for the examples and quotations Pinker endlessly adduces, but even though I found myself nodding far more often than not, one glaring fact continually leaks through: Enlightenment Now is a book about a process, namely ‘progress,’ that as yet remains mired in ‘tutelary natures.’ As Kevin Williamson puts it in the National Review, Pinker “leaps, without warrant, from physical science to metaphysical certitude.”

What is his naturalization of meaning? Or morality? Or cognition—especially cognition! How does one assess the cognitive revolution that is the Enlightenment short understanding the nature of cognition? How does one prognosticate something one does not scientifically understand?

At one point he offers that “[t]he principles of information, computation, and control bridge the chasm between the physical world of cause and effect and the mental world of knowledge, intelligence, and purpose” (22). Granted, he’s a psychologist: operationalizations of information, computation, and control are his empirical bread and butter. But operationalizing intentional concepts in experimental contexts is a far cry from naturalizing intentional concepts. He entirely neglects to mention that his ‘bridge’ is merely a pragmatic, institutional one, that cognitive science remains, despite decades of research and billions of dollars in resources, unable to formulate its explananda, let alone explain them. He mentions a great number of philosophers, but he fails to mention what the presence of those philosophers in his thetic wheelhouse means.

All he ultimately has, on the one hand, is a kind of ‘ta-da’ argument, the exhaustive statistical inventory of the bounty of reason, science, and humanism, and on the other hand (which he largely keeps hidden behind his back), he has the ‘tu quoque,’ the question-begging presumption that one can only argue against reason (as it is traditionally understood) by presupposing reason (as it is traditionally understood). “We don’t believe in reason,” he writes, “we use reason” (352). Pending any scientific verdict on the nature of ‘reason,’ however, these kinds of transcendental arguments amount to little more than fancy foot-stomping.

This is one of those books that make me wish I could travel back in time to catch the author drafting notes. So much brilliance, so much erudition, all devoted to beating straw—at least as far as ‘Second Culture’ Enlightenment critiques are concerned. Nietzsche is the most glaring example. Ignoring Nietzsche the physiologist, the empirically-minded skeptic, and reducing him to his subsequent misappropriation by fascist, existential, and postmodernist thought, Pinker writes:

Disdaining the commitment to truth-seeking among scientists and Enlightenment thinkers, Nietzsche asserted that “there are no facts, only interpretations,” and that “truth is a kind of error without which a certain species of life could not live.” (Of course, this left him unable to explain why we should believe that those statements are true.) 446

Although it’s true that Nietzsche (like Pinker) lacked any scientifically compelling theory of cognition, what he did understand was its relation to power, the fact that “when you face an adversary alone, your best weapon may be an ax, but when you face an adversary in front of a throng of bystanders, your best weapon may be an argument” (415). To argue that all knowledge is contextual isn’t to argue that all knowledge is fundamentally equal (and therefore not knowledge at all), only that it is bound to its time and place, a creature possessing its own ecology, its own conditions of failure and flourishing. The Nietzschean thought experiment is actually quite a simple one: What happens when we turn Enlightenment skepticism loose upon Enlightenment values? For Nietzsche, Enlightenment Now, though it regularly pays lip service to the ramshackle, reversal-prone nature of progress, serves to conceal the empirical fact of cognitive ecology, that we remain, for all our enlightened noise-making to the contrary, animals bent on minimizing discrepancies. The Enlightenment only survives its own skepticism, Nietzsche thought, in the transvaluation of value, which he conceived—unfortunately—in atavistic or morally regressive terms.

This underwrites the subsequent critique of the Enlightenment we find in Adorno—another thinker whom Pinker grossly underestimates. Though science is able to determine the more—to provide more food, shelter, security, etc.—it has the social consequence underdetermining (and so undermining) the better, stranding civilization with a nihilistic consumerism, where ‘meaningfulness’ becomes just another commodity, which is to say, nothing meaningful at all. Adorno’s whole diagnosis turns on the way science monopolizes rationality, the way it renders moral discourses like Pinker’s mere conjectural exercises (regarding the value of certain values), turning on leaps of faith (on the nature of cognition, etc.), bound to dissolve into disputation. Although both Nietzsche and Adorno believed science needed to be understood as a living, high dimensional entity, neither harboured any delusions as to where they stood in the cognitive pecking order. Unlike Pinker.

Whatever their failings, Nietzsche and Adorno glimpsed a profound truth regarding ‘reason, science, humanism, and progress,’ one that lurks throughout Pinker’s entire account. Both understood that cognition, whatever it amounts to, is ecological. Steven Pinker’s claim to fame, of course, lies in the cognitive ecological analysis of different cultural phenomena—this was the whole reason I was so keen to read this book. (In How the Mind Works, for instance, he famously calls music ‘auditory cheese-cake.’) Nevertheless, I think both Nietzsche and Adorno understood the ecological upshot of the Enlightenment in way that Pinker, as an avowed humanist, simply cannot. In fact, Pinker need only follow through on his modus operandi to see how and why the Enlightenment is not what he thinks it is—as well as why we have good reason to fear that Trumpism is no ‘blip.’

Time and again Pinker likens the process of Enlightenment, the movement away from our tutelary natures, in terms of a conflict between ancestral cognitive predilections and scientifically and culturally revolutionized environments. “Humans today,” he writes, “rely on cognitive faculties that worked well enough in traditional societies, but which we now see are infested with bugs” (25). And the number of bugs that Pinker references in the course of the book is nothing short of prodigious. We tend to estimate frequencies according to ease of retrieval. We tend to fear losses more than we hope for gains. We tend to believe as our group believes. We’re prone to tribalism. We tend to forget past misfortune, and to succumb to nostalgia. The list goes on and on.

What redeems us, Pinker argues, is the human capacity for abstraction and combinatorial recursion, which allows us to endlessly optimize our behaviour. We are a self-correcting species:

So for all the flaws in human nature, it contains the seeds of its own improvement, as long as it comes up with norms and institutions that channel parochial interests into universal benefits. Among those norms are free speech, nonviolence, cooperation, cosmopolitanism, human rights, and an acknowledgment of human fallibility, and among the institutions are science, education, media, democratic government, international organizations, and markets. Not coincidentally, these were the major brainchildren of the Enlightenment. 28

We are the products of ancestral cognitive ecologies, yes, but our capacity for optimizing our capacities allows us to overcome our ‘flawed natures,’ become something better than what we were. “The challenge for us today,” Pinker writes, “is to design an informational environment in which that ability prevails over the ones that lead us into folly” (355).

And here we encounter the paradox that Enlightenment Now never considers, even though Pinker presupposes it continually. The challenge for us today is to construct an informational environment that mitigates the problems arising out of our previous environmental constructions. The ‘bugs’ in human nature that need to be fixed were once ancestral features. What has rendered these adaptations ‘buggy’ is nothing other than the ‘march of progress.’ A central premise of Enlightenment Now is that human cognitive ecology, the complex formed by our capacities and our environments, has fallen out of whack in this way or that, cuing us to apply atavistic modes of problem-solving out of school. The paradox is that the very bugs Pinker thinks only the Enlightenment can solve are the very bugs the Enlightenment has created.

What Nietzsche and Adorno glimpsed, each in their own murky way, was a recursive flaw in Enlightenment logic, the way the rationalization of everything meant the rationalization of rationalization, and how this has to short-circuit human meaning. Both saw the problem in the implementation, in the physiology of thought and community, not in the abstract. So where Pinker seeks to “to restate the ideals of the Enlightenment in the language and concepts of the 21st century” (5), we can likewise restate Nietzsche and Adorno’s critiques of the Enlightenment in Pinker’s own biological idiom.

The problem with the Enlightenment is a cognitive ecological problem. The technical (rational and technological) remediation of our cognitive ecologies transforms those ecologies, generating the need for further technical remediation. Our technical cognitive ecologies are thus drifting ever further from our ancestral cognitive ecologies. Human sociocognition and metacognition in particular are radically heuristic, and as such dependent on countless environmental invariants. Before even considering more, smarter intervention as a solution to the ambient consequences of prior interventions, the big question has to be how far—and how fast—can humanity go? At what point (or what velocity) does a recognizably human cognitive ecology cease to exist?

This question has nothing to do with nostalgia or declinism, no more than any question of ecological viability in times of environmental transformation. It also clearly follows from Pinker’s own empirical commitments.

 

The Death of Progress (at the Hand of Progress)

The formula is simple. Enlightenment reason solves natures, allowing the development of technology, generally relieving humanity of countless ancestral afflictions. But Enlightenment reason is only now solving its own nature. Pinker, in the absence of that solution, is arguing that the formula remains reliable if not quite as simple. And if all things were equal, his optimistic induction would carry the day—at least for me. As it stands, I’m with Nietzsche and Adorno. All things are not equal… and we would see this clearly, I think, were it not for the intentional obscurities comprising humanism. Far from the latest, greatest hope that Pinker makes it out to be, I fear humanism constitutes yet another nexus of traditional intuitions that must be overcome. The last stand of ancestral authority.

I agree this conclusion is catastrophic, “the greatest intellectual collapse in the history of our species” (vii), as an old polemical foe of Pinker’s, Jerry Fodor (1987) calls it. Nevertheless, short grasping this conclusion, I fear we court a disaster far greater still.

Hitherto, the light cast by the Enlightenment left us largely in the dark, guessing at the lay of interior shadows. We can mathematically model the first instants of creation, and yet we remain thoroughly baffled by our ability to do so. So far, the march of moral progress has turned on the revolutionizing our material environments: we need only renovate our self-understanding enough to accommodate this revolution. Humanism can be seen as the ‘good enough’ product of this renovation, a retooling of folk vocabularies and folk reports to accommodate the radical environmental and interpersonal transformations occurring around them. The discourses are myriad, the definitions are endlessly disputed, nevertheless humanism provisioned us with the cognitive flexibility required to flourish in an age of environmental disenchantment and transformation. Once we understand the pertinent facts of human cognitive ecology, its status as an ad hoc ‘tutelary nature’ becomes plain.

Just what are these pertinent facts? First, there is a profound distinction between natural or causal cognition, and intentional cognition. Developmental research shows that infants begin exhibiting distinct physical versus psychological cognitive capacities within the first year of life. Research into Asperger Syndrome (Baron-Cohen et al 2001) and Autism Spectrum Disorder (Binnie and Williams 2003) consistently reveals a cleavage between intuitive social cognitive capacities, ‘theory-of-mind’ or ‘folk psychology,’ and intuitive mechanical cognitive capacities, or ‘folk physics.’ Intuitive social cognitive capacities demonstrate significant heritability (Ebstein et al 2010, Scourfield et al 1999) in twin and family studies. Adults suffering Williams Syndrome (a genetic developmental disorder affecting spatial cognition) demonstrate profound impairments on intuitive physics tasks, but not intuitive psychology tasks (Kamps et al 2017). The distinction between intentional and natural cognition, in other words, is not merely a philosophical assertion, but a matter of established scientific fact.

Second, cognitive systems are mechanically intractable. From the standpoint of cognition, the most significant property of cognitive systems is their astronomical complexity: to solve for cognitive systems is to solve for what are perhaps the most complicated systems in the known universe. The industrial scale of the cognitive sciences provides dramatic evidence of this complexity: the scientific investigation of the human brain arguably constitutes the most massive cognitive endeavor in human history. (In the past six fiscal years, from 2012 to 2017, the National Institute of Health [21/01/2017] alone will have spent more than 113 billion dollars funding research bent on solving some corner of the human soul. This includes, in addition to the neurosciences proper, research into Basic Behavioral and Social Science (8.597 billion), Behavioral and Social Science (22.515 billion), Brain Disorders (23.702 billion), Mental Health (13.699 billion), and Neurodegeneration (10.183 billion)).

Despite this intractability, however, our cognitive systems solve for cognitive systems all the time. And they do so, moreover, expending imperceptible resources and absent any access to the astronomical complexities responsible—which is to say, given very little information. Which delivers us to our third pertinent fact: the capacity of cognitive systems to solve for cognitive systems is radically heuristic. It consists of ‘fast and frugal’ tools, not so much sacrificing accuracy as applicability in problem-solving (Todd and Gigerenzer 2012). When one cognitive system solves for another it relies on available cues, granular information made available via behaviour, utterly neglecting the biomechanical information that is the stock and trade of the cognitive sciences. This radically limits their domain of applicability.

The heuristic nature of intentional cognition is evidenced by the ease with which it is cued. Thus, the fourth pertinent fact: intentional cognition is hypersensitive. Anthropomorphism, the attribution of human cognitive characteristics to systems possessing none, evidences the promiscuous application of human intentional cognition to intentional cues, our tendency to run afoul what might be called intentional pareidolia, the disposition to cognize minds where no minds exist (Waytz et al 2014). The Heider-Simmel illusion, an animation consisting of no more than shapes moving about a screen, dramatically evidences this hypersensitivity, insofar as viewers invariably see versions of a romantic drama (Heider and Simmel 1944). Research in Human-Computer Interaction continues to explore this hypersensitivity in a wide variety of contexts involving artificial systems (Nass and Moon 2000, Appel et al 2012). The identification and exploitation of our intentional reflexes has become a massive commercial research project (so-called ‘affective computing’) in its own right (Yonck 2017).

Intentional pareidolia underscores the fact that intentional cognition, as heuristic, is geared to solve a specific range of problems. In this sense, it closely parallels facial pareidolia, the tendency to cognize faces where no faces exist. Intentional cognition, in other words, is both domain-specific, and readily misapplied.

The incompatibility between intentional and mechanical cognitive systems, then, is precisely what we should expect, given the radically heuristic nature of the former. Humanity evolved in shallow cognitive ecologies, mechanically inscrutable environments. Only the most immediate and granular causes could be cognized, so we evolved a plethora of ways to do without deep environmental information, to isolate saliencies correlated with various outcomes (much as machine learning).

Human intentional cognition neglects the intractable task of cognizing natural facts, leaping to conclusions on the basis of whatever information it can scrounge. In this sense it’s constantly gambling that certain invariant backgrounds obtain, or conversely, that what it sees is all that matters. This is just another way to say that intentional cognition is ecological, which in turn is just another way to say that it can degrade, even collapse, given the loss of certain background invariants.

The important thing to note, here, of course, is how Enlightenment progress appears to be ultimately inimical to human intentional cognition. We can only assume that, over time, the unrestricted rationalization of our environments will gradually degrade, then eventually overthrow the invariances sustaining intentional cognition. The argument is straightforward:

1) Intentional cognition depends on cognitive ecological invariances.

2) Scientific progress entails the continual transformation of cognitive ecological invariances.

Thus, 3) scientific progress entails the collapse of intentional cognition.

But this argument oversimplifies matters. To see as much one need only consider the way a semantic apocalypse—the collapse of intentional cognition—differs from say a nuclear or zombie apocalypse. The Walking Dead, for instance, abounds with savvy applications of intentional cognition. The physical systems underwriting meaning, in other words, are not the same as the physical systems underwriting modern civilization. So long as some few of us linger, meaning lingers.

Intentional cognition, you might think, is only as weak or as hardy as we are. No matter what the apocalyptic scenario, if humans survive it survives. But as autistic spectrum disorder demonstrates, this is plainly not the case. Intentional cognition possesses profound constitutive dependencies (as those suffering the misfortune of watching a loved one succumb to strokes or neurodegenerative disease knows first-hand). Research into the psychological effects of solitary confinement, on the other hand, show that intentional cognition also possesses profound environmental dependencies as well. Starve the brain of intentional cues, and it will eventually begin to invent them.

The viability of intentional cognition, in other words, depends not on us, but on a particular cognitive ecology peculiar to us. The question of the threshold of a semantic apocalypse becomes the question of the stability of certain onboard biological invariances correlated to a background of certain environmental invariances. Change the constitutive or environmental invariances underwriting intentional cognition too much, and you can expect it will crash, generate more problems than solutions.

The hypersensitivity of intentional cognition either evinced by solitary confinement or more generally by anthropomorphism demonstrates the threat of systematic misapplication, the mode’s dependence on cue authenticity. (Sherry Turkle’s (2007) concerns regarding ‘Darwinian buttons,’ or Deidre Barrett’s (2010) with ‘supernormal stimuli,’ touch on this issue). So, one way of inducing semantic apocalypse, we might surmise, lies in the proliferation of counterfeit cues, information that triggers intentional determinations that confound, rather than solve any problems. One way to degrade cognitive ecologies, in other words, is to populate environments with artifacts cuing intentional cognition ‘out of school,’ which is to say, circumstances cheating or crashing them.

The morbidity of intentional cognition demonstrates the mode’s dependence on its own physiology. What makes this more than platitudinal is the way this physiology is attuned to the greater, enabling cognitive ecology. Since environments always vary while cognitive systems remain the same, changing the physiology of intentional cognition impacts every intentional cognitive ecology—not only for oneself, but for the rest of humanity as well. Just as our moral cognitive ecology is complicated by the existence of psychopaths, individuals possessing systematically different ways of solving social problems, the existence of ‘augmented’ moral cognizers complicates our moral cognitive ecology as well. This is important because you often find it claimed in transhumanist circles (see, for example, Buchanan 2011), that ‘enhancement,’ the technological upgrading of human cognitive capacities, is what guarantees perpetual Enlightenment. What better way to optimize our values than by reengineering the biology of valuation?

Here, at last, we encounter Nietzsche’s question cloaked in 21st century garb.

And here we can also see where the above argument falls short: it overlooks the inevitability of engineering intentional cognition to accommodate constitutive and environmental transformations. The dependence upon cognitive ecologies asserted in (1) is actually contingent upon the ecological transformation asserted in (2).

1) Intentional cognition depends on constitutive and environmental cognitive ecological invariances.

2) Scientific progress entails the continual transformation of constitutive and environmental cognitive ecological invariances.

Thus, 3) scientific progress entails the collapse of intentional cognition short remedial constitutive transformations.

What Pinker would insist is that enhancement will allow us to overcome our Pleistocene shortcomings, and that our hitherto inexhaustible capacity to adapt will see us through. Even granting the technical capacity to so remediate, the problem with this reformulation is that transforming intentional cognition to account for transforming social environments automatically amounts to a further transformation of social environments. The problem, in other words, is that Enlightenment entails the end of invariances, the end of shared humanity, in fact. Yuval Harari (2017) puts it with characteristic brilliance in Homo Deus:

What then, will happen once we realize that customers and voters never make free choices, and once we have the technology to calculate, design, or outsmart their feelings? If the whole universe is pegged to the human experience, what will happen once the human experience becomes just another designable product, no different in essence from any other item in the supermarket? 277

The former dilemma is presently dominating the headlines and is set to be astronomically complicated by the explosion of AI. The latter we can see rising out of literature, clawing its way out of Hollywood, seizing us with video game consoles, engulfing ever more experiential bandwidth. And as I like to remind people, 100 years separates the Blu-Ray from the wax phonograph.

The key to blocking the possibility that the transformative potential of (2) can ameliorate the dependency in (1) lies in underscoring the continual nature of the changes asserted in (2). A cognitive ecology where basic constitutive and environmental facts are in play is no longer recognizable as a human one.

Scientific progress entails the collapse of intentional cognition.

On this view, the coupling of scientific and moral progress is a temporary affair, one doomed to last only so long as cognition itself remained outside the purview of Enlightenment cognition. So long as astronomical complexity assured that the ancestral invariances underwriting cognition remained intact, the revolution of our environments could proceed apace. Our ancestral cognitive equilibria need not be overthrown. In place of materially actionable knowledge regarding ourselves, we developed ‘humanism,’ a sop for rare stipulation and ambient disputation.

But now that our ancestral cognitive equilibria are being overthrown, we should expect scientific and moral progress will become decoupled. And I would argue that the evidence of this is becoming plainer with the passing of every year. Next week, we’ll take a look at several examples.

I fear Donald Trump may be just the beginning.

.

References

Appel, Jana, von der Putten, Astrid, Kramer, Nicole C. and Gratch, Jonathan 2012, ‘Does Humanity Matter? Analyzing the Importance of Social Cues and Perceived Agency of a Computer System for the Emergence of Social Reactions during Human-Computer Interaction’, in Advances in Human-Computer Interaction 2012 <https://www.hindawi.com/journals/ahci/2012/324694/ref/&gt;

Barrett, Deidre 2010, Supernormal Stimuli: How Primal Urges Overran Their Original Evolutionary Purpose (New York: W.W. Norton)

Binnie, Lynne and Williams, Joanne 2003, ‘Intuitive Psychology and Physics Among Children with Autism and Typically Developing Children’, Autism 7

Buchanan, Allen 2011, Better than Human: The Promise and Perils of Enhancing Ourselves (New York: Oxford University Press)

Ebstein, R.P., Israel, S, Chew, S.H., Zhong, S., and Knafo, A. 2010, ‘Genetics of human social behavior’, in Neuron 65

Fodor, Jerry A. 1987, Psychosemantics: The Problem of Meaning in the Philosophy of Mind (Cambridge, MA: The MIT Press)

Harari, Yuval 2017, Homo Deus: A Brief History of Tomorrow (New York: HarperCollins)

Heider, Fritz and Simmel, Marianne 1944, ‘An Experimental Study of Apparent Behaviour,’ in The American Journal of Psychology 57

Kamps, Frederik S., Julian, Joshua B., Battaglia, Peter, Landau, Barbara, Kanwisher, Nancy and Dilks Daniel D 2017, ‘Dissociating intuitive physics from intuitive psychology: Evidence from Williams syndrome’, in Cognition 168

Nass, Clifford and Moon, Youngme 2000, ‘Machines and Mindlessness: Social Responses to Computers’, Journal of Social Issues 56

Pinker, Steven 1997, How the Mind Works (New York: W.W. Norton)

—. 2018, Enlightenment Now: The Case for Reason, Science, Humanism, and Progress (New York: Viking)

Scourfield J., Martin N., Lewis G. and McGuffin P. 1999, ‘Heritability of social cognitive skills in children and adolescents’, British Journal of Psychiatry 175

Todd, P. and Gigerenzer, G. 2012 ‘What is ecological rationality?’, in Todd, P. and Gigerenzer, G. (eds.) Ecological Rationality: Intelligence in the World (Oxford: Oxford University Press) 3–

30

Turkle, Sherry 2007, ‘Authenticity in the age of digital companions’, Interaction Studies 501-517

Waytz, Adam, Cacioppo, John, and Epley, Nicholas 2014, ‘Who See Human? The Stability and Importance of Individual Differences in Anthropomorphism’, Perspectives in Psychological Science 5

Yonck, Richard 2017, Heart of the Machine: Our Future in a World of Artificial Emotional Intelligence (New York, NY: Arcade Publishing)

 

Reading From Bacteria to Bach and Back II: The Human Squircle

by rsbakker

The entry placing second (!!) in the 2016 Illusion of the Year competition, the Ambiguous Cylinder Illusion, blew up on Reddit for good reason. What you’re seeing below is an instance where visual guesswork arising from natural environmental frequencies have been cued ‘out of school.’ In this illusion, convex and concave curves trick the visual system into interpreting a ‘squircle’ as either a square or a circle—thus the dazzling images. Ambiguous cylinders provide dramatic illustration of a point Dennett makes many times in From Bacteria to Bach and Back: “One of the hallmarks of design by natural selection,” he writes, “is that it is full of bugs, in the computer programmer’s sense: design flaws that show up only under highly improbable conditions, conditions never encountered in the finite course of R&D that led to the design to date, and hence not yet patched or worked around by generations of tinkering” (83). The ‘bug’ exploited in this instance could be as much a matter of neural as natural selection, of course—perhaps, as with the Muller-Lyer illusion, individuals raised in certain environments are immune to this effect. But the upshot remains the same. By discovering ways to cue heuristic visual subsystems outside their adaptive problem ecologies, optical illusionists have developed a bona fide science bent on exploring what might be called ‘visual crash space.’

One of the ideas behind Three Pound Brain is to see traditional intentional philosophy as the unwitting exploration of metacognitive crash space. Philosophical reflection amounts to the application of metacognitive capacities adapted to trouble-shooting practical cognitive and communicative issues to theoretical problems. What Dennett calls ‘Cartesian gravity,’ in other words, has been my obsession for quite some time, and I think I have a fair amount of wisdom to share, especially when it comes to philosophical squircles, things that seem undeniable, yet nevertheless contradict our natural scientific understanding. Free will is perhaps the most famous of these squircles, but there’s really no end to them. The most pernicious squircle of all, I’m convinced, is the notion of intentionality, be it ‘derived’ or ‘original.’

On Heuristic Neglect Theory, Cartesian gravity boils down to metacognitive reflexes, the application of heuristic systems to questions they have no hope of answering absent any inkling of as much. The root of the difficulty lies in neglect, the way insensitivity to the limits of felicitous application results in various kinds of systematic errors (what might be seen as generalized versions of the WYSIATI effects discovered by Daniel Kahneman).

The centrality of neglect (understood as an insensitivity that escapes our sensitivity) underwrites my reference to the ‘Grand Inversion’ in the previous installment. As an ecological artifact, human cognition trivially possesses what might be called a neglect structure: we are blind to the vast bulk of the electromagnetic spectrum, for instance, because sensing things like gamma radiation, infrared, or radio waves paid no ancestral dividends. If fact, one can look at the sum of scientific instrumentation as mapping out human ‘insensitivity space,’ providing ingress into all those places our ancestral sensitivities simply could not take us. Neglect, in other words, allows us to quite literally invert our reflexive ways of comprehending comprehension, not only in a wholesale manner, but in a way entirely compatible with what Dennett calls, following Sellars, the scientific image.

Simply flipping our orientation in this way allows us to radically recharacterize Dennett’s project in From Bacteria to Bach and Back as a matter of implicitly mapping our human neglect structure by filling in all the naturalistic blanks. I say implicit because his approach remains primarily focused on what is neglected, rather than neglect considered in its own right. Despite this, Dennett is quite cognizant of the fact that he’s discussing a single phenomenon, albeit one he characterizes (thanks to Cartesian gravity!) in positive terms:

Darwin’s “strange inversion of reasoning” and Turing’s equally revolutionary inversion form aspects of a single discovery: competence without comprehension. Comprehension, far from being a god-like talent from which all design must flow, is an emergent effect of systems of uncomprehending competence… (75)

The problem with this approach is one that Dennett knows well: no matter how high you build your tower of natural processes, all you’ve managed to do, in an important sense, is recapitulate the mystery you’ve set out to solve. No matter how long you build your ramp, talk of indefinite thresholds and ‘emergent effects’ very quickly reveals you’re jumping the same old explanatory shark. In a sense, everyone in the know knows at least the moral of the story Dennett tells: competences stack into comprehension on any Darwinian account. The million-dollar question is how ‘all that’ manages to culminate in this

Personally speaking, I’ve never had an experience quite like the one I had reading this book. Elation, realizing that one of the most celebrated minds in philosophy had (finally!) picked up on the same trail. Urgency, knowing I had to write a commentary, like, now. And then, at a certain point, wonder at the sense of knowing, quite precisely, what it was tantalizing his intuitions: the profound connection between his Darwinian commitments and his metaphilosophical hunches regarding Cartesian gravitation.

Heuristic Neglect Theory not only allows us to economize Dennett’s bottom-up saga of stacking competences, it also provides a way to theorize his top-down diagnosis of comprehension. It provides, in other words, the common explanatory framework required to understand this… in terms of ‘all that.’ No jumps. No sharks. Just one continuous natural story folding comprehension into competence (or better, behaviour).

What applies to human cognition applies to human metacognition—understood as the deliberative derivation of endogenous or exogenous behaviour via secondary (functionally distinct) access to one’s own endogenous or exogenous behaviour. As an ecological artifact, human metacognition is fractionate and heuristic, and radically so, given the complexity of the systems it solves. As such, it possesses its own neglect structure. Understanding this allows us to ‘reverse-engineer’ far more than Dennett suspects, insofar as it lets us hypothesize the kinds of blind spots we should expect to plague our attempts to theorize ourselves given the deliverances of philosophical reflection. It provides the theoretical basis, I think, for understanding philosophy as the cognitive psychological phenomenon that it is.

It’s a truism to say that the ability to cognize any system crucially depends on a cognitive system’s position relative to that system. But things get very interesting once we begin picking at the how and why. The rationality of geocentrism, for instance, is generally attributed to the fact that from our terrestrial perspective, the sky does all the moving. We remain, as far as we can tell, motionless. Why is motionlessness the default? Why not assume ignorance? Why not assume that the absence of information warranted ‘orbital agnosticism’? Basically, because we lacked the information to determine our lack of information.

Figure 1: It is a truism to state that where we find ourselves within a system determines our ability to cognize that system. ‘Frame neglect’ refers to our cognitive insensitivity, not only to our position within unknown systems, but to this insensitivity.

Figure 2: Thus, the problem posed by sufficiency, the automatic presumption that what we see is all there is. The ancients saw the stars comprising Orion as equidistant simply because they lacked the information and theory required to understand their actual position—because they had no way of knowing otherwise.

Figure 3: It is also a truism to state that the constitution of our cognitive capacities determines our ability to cognize systems. ‘Medial neglect’ refers to our cognitive insensitivity, not only to the constitution of our cognitive capacities, but to this insensitivity. We see, but absent any sensitivity to the machinery enabling sight.

Figure 4: Thus, once again, the problem posed by sufficiency. Our brain interprets ambiguous cylinders as magical squircles because it possesses no sensitivity to the kinds of heuristic mechanisms involved in processing visual information.

Generally speaking, we find these ‘no information otherwise’ justifications so intuitive that we just move on. We never ask how or why the absence of sensible movement cues reports of motionlessness. Plato need only tell us that his prisoners have been chained before shadows their whole lives and we get it, we understand that for them, shadows are everything. By merely conjuring an image, Plato secures our acknowledgment that we suffer a congenital form of frame neglect, a cognitive insensitivity to the limits of cognition that can strand us with fantastic (and so destructive) worldviews—and without our permission, no less. Despite the risk entailed, we neglect this form of neglect. Though industry and science are becoming ever more sensitive to the problems posed by the ‘unknown unknown,’ it remains the case that each of us at once understands the peril and presumes we’re the exception, the system apart from the systems about us. The motionless one.

Frame neglect, our insensitivity to the superordinate systems encompassing us, blinds us to our position within those systems. As a result, we have no choice but to take those positions for granted. This renders our cognitive orientations implicit, immune to deliberative revision and so persistent (as well as vulnerable to manipulation). Frame neglect, in other words, explains why bent orientations stay bent, why we suffer the cognitive inertia we do. More importantly, it highlights what might be called default sufficiency, the congenital presumption of implicit cognitive adequacy. We were in no position to cognize our position relative the heavens, and yet we nevertheless assumed that we were simply because we were in no position to cognize the inadequacy of our position.

Why is sufficiency the presumptive default? The stacking of ‘competences’ so brilliantly described by Dennett requires that every process ‘do its part’: sufficiency, you could say, is the default presumption of any biological system, so far as its component systems turn upon the iterative behaviour of other component systems. Dennett broaches the notion, albeit implicitly, via the example of asking someone to report on a nearby house via cell phone:

Seeing is believing, or something like that. We tacitly take the unknown pathways between his open eyes and speaking lips to be secure, just like the requisite activity in the pathways in the cell towers between his phone and ours. We’re not curious on the occasion about how telephones work; we take them for granted. We also don’t scratch our heads in bafflement over how he can just open his eyes and then answer questions with high reliability about what is positioned in front of him in the light, because we can all do it (those of us who are not blind). 348-349

Sufficiency is the default. We inherit our position, our basic cognitive orientation, because it sufficed to solve the kinds of high-frequency and/or high impact problems faced by our ancestors. This explains why unprecedented circumstances generate the kinds of problems they do: it’s always an open question whether our basic cognitive orientation will suffice when confronted with a novel problem.

When it comes to vision, for instance, we possess a wide range of ways to estimate sufficiency and so can adapt our behaviour to a variety of lighting conditions, waving our hand in fog, peering against glares, and so on. Darkness in particular demonstrates how the lack of information requires information, lest it ‘fall off the radar’ in the profound sense entailed by neglect. So even though we possess myriad ways to vet visual information, squircles possess no precedent and so no warning, the sufficiency of the information available is taken for granted, and we suffer the ambiguous cylinder illusion. Our cognitive ecology plays a functional role in the efficacy of our heuristic applications—all of them.

From this a great deal follows. Retasking some system of competences always runs the risk of systematic deception on the one hand, where unprecedented circumstances strand us with false solutions (as with the millennia-long ontological dualism of the terrestrial and the celestial), and dumbfounding on the other, where unprecedented circumstances crash some apparently sufficient application in subsequently detectable ways, such as ambiguous for human visual systems, or the problem of determinism for undergraduate students.

To the extent that ‘philosophical reflection’ turns on the novel application of preexisting metacognitive resources, it almost certainly runs afoul instances of systematic deception and dumbfounding. Retasked metacognitive channels and resources, we can be assured, would report as sufficient, simply because our capacity to intuit insufficiency would be the product of ancestral, which is to say, practical, applications. How could information and capacity geared to catching our tongue in social situations, assessing what we think we saw, rehearsing how to explain some disaster, and so on hope to leverage theoretical insights into the fundamental nature of cognition and experience? It can’t, no more than auditory cognition, say, could hope to solve the origin of the universe. But even more problematically, it has no hope of intuiting this fundamental inability. Once removed from the vacuum of ecological ignorance, the unreliability of ‘philosophical reflection,’ its capacity to both dumbfound and to systematically deceive, becomes exactly what we should expect.

This follows, I think, on any plausible empirical account of human metacognition. I’ve been asking interlocutors to provide me a more plausible account for years now, but they always manage to lose sight of the question somehow.

On the availability side, we should expect the confusion of task-insufficient information with task-sufficient information. On the capacity side, we should expect the confusion of task-insufficient applications with task-sufficient applications. And this is basically what Dennett’s ‘Cartesian gravity’ amounts to, the reflexive deliberative metacognitive tendency to confuse scraps with banquets and hammers with swiss-army knives.

But the subtleties secondary to these reflexes can be difficult to grasp, at least at first. Sufficiency means that decreases in dimensionality, the absence of kinds and quantities of information, simply cannot be cognized as such. Just over two years ago I suffered a retinal tear, which although successfully repaired, left me with a fair amount of debris in my right eye (‘floaters,’ as they call them, which can be quite distracting if you spend as much time staring at white screens as I do). Last autumn I noticed I had developed a ‘crimp’ in my right eye’s field of vision: apparently some debris had become attached to my fovea, a mass that accumulated as I was passed from doctor to doctor and thence to the surgeon. I found myself with my own, entirely private visual illusion: the occluded retinal cells were snipped out of my visual field altogether, mangling everything I tried to focus on with my right eye. The centre of every word I looked at would be pinched into oblivion, leaving only the beginning and ending characters mashed together. Faces became positively demonic—to the point where I began developing a Popeye squint for equanimity’s sake. The world had become a grand bi-stable image: things were fine when my left eye predominated, but then for whatever reason, click, my friends and family would be eyeless heads of hair. Human squircles.

My visual centres simply neglected the missing information, and muddled along assuming the sufficiency of the information that was available. I understood the insufficiency of what I was seeing. I knew the prisoners were there, chained in their particular neural cave with their own particular shadows, but I had no way of passing that information upstream—the best I could do was manage the downstream consequences.

But what happens when we have no way of intuiting information loss? What happens when our capacity to deliberate and report finds itself chained ‘with no information otherwise’? Well, given sufficiency, it stands to reason that what metacognition cannot distinguish we will report as same, that what it cannot vet we will report as accurate, that what it cannot swap we will report inescapable, and that what it cannot source we will report as sourceless, and so on. The dimensions of information occluded, in other words, depend entirely on what we happen to be reporting. If we ponder the proximate sources of our experiences, they will strike us as sourceless. If we ponder the composition of our experiences, they will strike us simple. Why? Because human metacognition not only failed to evolve the extraordinary ability to theoretically source or analyze human experience, it failed to evolve the ability to intuit this deficit. And so, we find ourselves stranded with squircles, our own personal paradox (illusion) of ourselves, of what it is fundamentally like to be ‘me.’

Dialectically, it’s important to note how this consequence of the Grand Inversion overturns the traditional explanatory burden when it comes to conscious experience. Since it takes more metacognitive access and capacity, not less, to discern things like disunity and provenance, the question Heuristic Neglect Theory asks of the phenomenologist is, “Yes, but how could you report otherwise?” Why think the intuition of apperceptive unity (just for instance) is anything more than a metacognitive cousin of the flicker-fusion you’re experiencing staring at the screen this very instant?

Given the wildly heuristic nature of our metacognitive capacities, we should expect to possess the capacity to discriminate only what our ancestors needed to discriminate, and precious little else. So, then, how could we intuit anything but apperceptive unity? Left with a choice between affirming a low-dimensional exception to nature on the basis of an empirically implausible metacognitive capacity, and a low-dimensional artifact of the very kind we might expect given an empirically plausible metacognitive account, there really is no contest.

And the list goes on and on. Why think intuitions of ‘self-identity’ possess anything more than the information required to resolve practical, ancestral issues involving identification?

One can think of countless philosophical accounts of the ‘first-person’ as the product of metacognitive ‘neglect origami,’ the way sufficiency precludes intuiting the radical insufficiency of the typically scant dimensions of information available. If geocentrism is the default simply for the way our peripheral position in the solar system precludes intuiting our position as peripheral, then ‘noocentrism’ is the default for the way our peripheral position vis a vis ourselves precludes intuiting our position as peripheral. The same way astrophysical ignorance renders the terrestrial the apparently immovable anchor of celestial motion, metacognitive neglect renders the first-person the apparently transcendent anchor of third-person nature. In this sense, I think, ‘gravity’ is a well-chosen metaphor to express the impact of metacognitive neglect upon the philosophical imagination: metacognitive neglect, like gravity, isn’t so much a discrete force as a structural feature, something internal to the architecture of philosophical reflection. Given it, humanity was all but doomed to wallow in self-congratulatory cartoons once literacy enabled regimented inquiry into its own nature. If we’re not the centres of the universe, then surely we’re the centre of our knowledge, our projects, our communities—ourselves.

Figure 5: The retasking of deliberative metacognition is not unlike discovering something practical—such as ‘self’ (or in this case, Brian’s sandal)—in apparently exceptional, because informationally impoverished, circumstances.

Figure 6: We attempt to interpret this practical deliverance in light of these exceptional circumstances.

Figure 7: Given neglect, we presume the practical deliverance theoretically sufficient, and so ascribe it singular significance.

Figure 8: We transform ‘self’ into a fetish, something both self-sustaining and exceptional. A squircle.

Of all the metacognitive misapplications confounding traditional interpretations of cognition and experience, Dennett homes in on the one responsible for perhaps the most theoretical mischief in the form of Hume’s ‘strange inversion of reasoning’ (354-358), where the problem, as we saw in the previous post, lies in mistaking the ‘intentional object’ of the red stripe illusion for the cause of the illusion. Hume, recall, notes our curious propensity to confuse mental determinations for environmental determinations, to impute something belonging to this… to ‘all that.’ Dennett notes that the problem lies in the application: normally, this ‘confusion’ works remarkably well; it’s only in abnormal circumstances, like those belonging to the red stripe illusion, where this otherwise efficacious cognitive reflex leads us astray.

The first thing to note about this cognitive reflex is the obvious way it allows us to neglect the actual machinery of our environmental relations. Hume’s inversion, in other words, calls attention to the radically heuristic nature of so-called intentional thinking. Given the general sufficiency of all the processes mediating our environmental relationships, we need not cognize them to cognize those relationships, we can take them for granted, which is a good thing, because their complexity (the complexity cognitive science is just now surmounting) necessitates they remain opaque. ‘Opaque,’ in this instance, means heuristically neglected, the fact that all the mad dimensionalities belonging to our actual cognitive relationships appear nowhere in cognition, not even as something missing. What does appear? Well, as Dennett himself would say, only what’s needed to resolve practical ancestral problems.

Reporting environments economically entails taking as much for granted as possible. So long as the machinery you and I use to supervise and revise our environmental orientations is similar enough, we can ignore each other’s actual relationships in communication, focusing instead on discrepancies and how to optimize them. This is why we narrate only those things most prone to vary—environmentally and neurally sourced information prone to facilitate reproduction—and remain utterly oblivious to the all the things that go without saying, the deep information environment plumbed by cognitive science. The commonality of our communicative and cognitive apparatuses, not to mention their astronomical complexity, assures that we will suffer what might be called, medial neglect, congenital blindness to the high-dimensional systems enabling communication and cognition. “All the subpersonal, neural-level activity is where the actual causal interactions happen that provide your cognitive powers, but all “you” have access to is the results” (348).

From Bacteria to Bach and Back is filled with implicit references to medial neglect. “Our access to our own thinking, and especially to the causation and dynamics of its subpersonal parts, is really no better than our access to our digestive processes,” Dennett writes; “we have to rely on the rather narrow and heavily edited channel that responds to our incessant curiosity with user-friendly deliverances, only one step closer to the real me than the access to the real me that is enjoyed by my family and friends” (346).

Given sufficiency, “[t]he relative accessibility and familiarity of the outer part of the process of telling people what we can see—we know our eyes have to be open, and focused, and we have to attend, and there has to be light—conceals from us the other blank from the perspective of introspection or simple self-examination of the rest of the process” (349). The ‘outer part of the process,’ in other words, is all that we need.

Medial neglect may be both necessary and economical, but it remains an incredibly risky bet to make given the perversity of circumstance and the radical interdependency characterizing human communities. The most frequent and important discrepancies will be environmental discrepancies, those which, given otherwise convergent orientations (the same physiology, location, and training), can be communicated absent medial information, difference making differences geared to the enabling axis of communication and cognition. Such discrepancies can be resolved while remaining almost entirely ‘performance blind.’ All I need do is ‘trust’ your communication and cognition, build upon it the same blind way I build upon my own. You cry, ‘Wolf!’ and I run for the shotgun: our orientations converge.

But as my example implies, things are not always so simple. Say you and I report seeing two different birds, a vulture versus an albatross, in circumstances where such a determination potentially matters—looking for a lost hunting party, say. An endless number of medial confounds could possibly explain our sudden disagreement. Perhaps I have bad eyesight, or I think albatrosses are black, or I’m blinded by the glare of the sun, or I’m suffering schizophrenia, or I’m drunk, or I’m just sick and tired of you being right all the time, or I’m teasing you out of boredom, or more insidiously, I’m responsible for the loss of the hunting party, and want to prevent you from finding the scene of my crime.

There’s no question that, despite medial neglect, certain forms of access and capacity regarding the enabling dimension of cognition and communication could provide much in the way of problem resolution. Given the stupendous complexity of the systems involved, however, it follows that any capacity to accommodate medial factors will be heuristic in the extreme. This means that our cognitive capacity to flag/troubleshoot issues of sufficiency will be retail, fractionate, geared to different kinds of high-impact, high-frequency problems. And the simplest solution, the highest priority reflex, will be to ignore the medial altogether. If our search party includes a third soul who also reports seeing a vulture, for instance, I’ll just be ‘wrong’ for ‘reasons’ that may or not be determined afterward.

The fact of medial neglect, in other words, underwrites what might be called an environmentalization heuristic, the reflexive tendency to ‘blame’ the environment first.

When you attempt to tell us about what is happening in your experience, you ineluctably slide into a metaphorical idiom simply because you have no deeper, truer, more accurate knowledge of what was going on inside you. You cushion your ignorance with a false—but deeply tempting—model: you simply reproduce, with some hand waving and apologies, your everyday model of how you know about what is going on outside you. 348

Because that’s typically all that you need. Dennett’s hierarchical mountain of competences is welded together by default sufficiency, the blind mechanical reliance of one system upon other systems. Communicative competences not only exploit this mechanical reliance, they extend it, opening entirely novel ecosystems leveraging convergent orientation, brute environmental parallels and physiological isomorphisms, to resolve discrepancies. So long as those discrepancies are resolved, medial factors potentially impinging on sufficiency can be entirely ignored, and so will be ignored. Communications will be ‘right’ or ‘wrong,’ ‘true’ or ‘false.’ We remain as blind to the sources of our cognitive capacities as circumstances allow us to be. And we remain blind to this blindness as well.

When I say from the peak of my particular competence mountain, “Albatross…” and you turn to me in perplexity, and say from the peak of your competence mountain, “What the hell are you talking about?” your instance of ‘about-talk’ is geared to the resolution of a discrepancy between our otherwise implicitly convergent systems. This is what it’s doing. The idea that it reveals an exceptional kind of relationship, ‘aboutness,’ spanning the void between ‘albatross’ here and albatrosses out there is a metacognitive artifact, a kind of squircle. For one, the apparent void is jam-packed with enabling competences—vast networks of welded together by sufficiency. Medial neglect merely dupes metacognition into presuming otherwise, into thinking the apparently miraculous covariance (the product of vast histories of natural and neural selection) between ‘sign’ (here) and ‘signified’ (out there) is indeed some kind of miracle.

Philosophers dwell among general descriptions and explanations: this is why they have difficulty appreciating that naïveté generally consists in having no ‘image,’ no ‘view,’ regarding this or that domain. They habitually overlook the oxymoronic implication of attaching any ‘ism’ to the term ‘naïve.’ Instances of ‘about-talk’ do not implicitly presume ‘intentionality’ even in some naïve, mistaken sense. We are not born ‘naïve intentionalists’ (any more than we’re ‘naïve realists’). We just use meaning talk to solve what kinds of problems we can where we can. Granted, our shared metacognitive shortcomings lead us, given different canons of interrogation, into asserting this or that interpretation of ‘intentionality,’ popular or scholastic. We’re all prone to see squircles when prompted to peer into our souls.

So, when someone asks, “Where does causality lie?” we just point to where we can see it, out there on the billiard table. After all, where the hell else would it be (given medial neglect)? This is why dogmatism comes first in the order of philosophical complication, why Kant comes after Descartes. It takes time and no little ingenuity to frame plausible alternatives of this ‘elsewhere.’ And this is the significance of Hume’s inversion to Cartesian gravity: the reflexive sufficiency of whatever happens to be available, a sufficiency that may or may not obtain given the kinds of problem posed. The issue has nothing to do with confusing normal versus abnormal attributions of causal efficacy to intentional objects, because, for one, there’s just no such thing as ‘intentional objects,’ and for another, ‘intentional object-talk’ generates far more problems than it solves.

Of course, it doesn’t seem that way to Dennett whilst attempting to solve for Cartesian gravity, but only because, short theoretical thematizations of neglect and sufficiency, he lacks any real purchase on the problem of explaining the tendency to insist (as Tom Clark does) on the reality of the illusion. As a result, he finds himself in the strange position of embracing the sufficiency of intentionality in certain circumstances to counter the reflexive tendency to assume the sufficiency of phenomenality in other circumstances—of using one squircle, in effect, to overcome another. And this is what renders him eminently vulnerable to readings like Clark’s, which turns on Dennett’s avowal of intentional squircles to leverage, on pain of inconsistency, his commitment to phenomenal squircles. This problem vanishes once we recognize ourselves for the ambiguous cylinders we have always been. Showing as much, however, will require one final installment.

On Artificial Belonging: How Human Meaning is Falling between the Cracks of the AI Debate

by rsbakker

I hate people. Or so I used to tell myself in the thick of this or that adolescent crowd. Like so many other teens, my dawning social awareness occasioned not simply anxiety, but agony. Everyone else seemed to have the effortless manner, the well-groomed confidence, that I could only pretend to have. Lord knows I would try to tell amusing anecdotes, to make rooms boom with humour and admiration, but my voice would always falter, their attention would always wither, and I would find myself sitting alone with my butterflies. I had no choice but to hate other people: I needed them too much, and they needed me not at all. Never in my life have I felt so abandoned, so alone, as I did those years. Rarely have I felt such keen emotional pain.

Only later would I learn that I was anything but alone, that a great number of my peers felt every bit as alienated as I did. Adolescence represents a crucial juncture in the developmental trajectory of the human brain, the time when the neurocognitive tools required to decipher and navigate the complexities of human social life gradually come online. And much as the human immune system requires real-world feedback to discriminate between pathogens and allergens, human social cognition requires the pain of social failure to learn the secrets of social success.

Humans, like all other forms of life on this planet, require certain kinds of ecologies to thrive. As so-called ‘feral children’ dramatically demonstrate, the absence of social feedback at various developmental junctures can have catastrophic consequences.

So what happens when we introduce artificial agents into our social ecology? The pace of development is nothing short of boggling. We are about to witness a transformation in human social ecology without evolutionary let alone historical precedent. And yet the debate remains fixated on jobs or the prospects of apocalyptic superintelligences.

The question we really need to be asking is what happens when we begin talking to our machines more than to each other. What does it mean to dwell in social ecologies possessing only the appearance of love and understanding?

“Hell,” as Sartre famously wrote, “is other people.” Although the sentiment strikes a chord in most everyone, the facts of the matter are somewhat more complex. The vast majority of those placed in prolonged solitary confinement, it turns out, suffer a mixture of insomnia, cognitive impairment, depression, and even psychosis. The effects of social isolation are so dramatic, in fact, that the research has occasioned a worldwide condemnation of punitive segregation. Hell, if anything, would seem to be the absence of other people.

The reason for this is that we are a fundamentally social species, ‘eusocial’ in a manner akin to ants or bees, if E.O. Wilson is to be believed. To understand just how social we are, you need only watch the famous Heider-Simmel illusion, a brief animation portraying the movements of a small circle, a small rectangle, and larger rectangle, in and about a motionless, hollow square. Objectively speaking, all one sees are a collection of shapes moving relative one another and the hollow square. But despite the radical absence of information, nearly everyone watching the animation sees a little soap opera, usually involving the big square attempting to prevent the union of the small square and circle.

This leap from shapes to soap operas reveals, in dramatic fashion, just how little information we require to draw enormous social conclusions. Human social cognition is very easy to trigger out of school, as our ancient tendency to ‘anthropomorphize’ our natural surroundings shows. Not only are we prone to see faces in things like flaking paint or water stains, we’re powerfully primed to sense minds as well—so much so that segregated inmates often begin perceiving them regardless. As Brian Keenan, who was held by Islamic Jihad from 1986 to 1990, says of the voices he heard, “they were in the room, they were in me, they were coming from me but they were audible to no one else but me.”

What does this have to do with the impact of AI? More than anyone has yet imagined.


Imagine a social ecology populated by billions upon billions of junk intelligences


 

The problem, in a nutshell, is that other people aren’t so much heaven or hell as both. Solitary confinement, after all, refers to something done to people by other people. The argument to redefine segregation as torture finds powerful support in evidence showing that social exclusion activates the same regions of the brain as physical pain. At some point in our past, it seems, our social attachment systems coopted the pain system to motivate prosocial behaviors. As a result, the mere prospect of exclusion triggers analogues of physical suffering in human beings.

But as significant as this finding is, the experimental props used to derive these findings are even more telling. The experimental paradigm typically used to neuroimage social rejection turns on a strategically deceptive human-computer interaction, or HCI. While entombed in an fMRI, subjects are instructed to play an animated three-way game of catch—called ‘Cyberball’—with what they think are two other individuals on the internet, but which is in fact a program designed to initially include, then subsequently exclude, the subject. As the other ‘players’ begin throwing more and more to each other, the subject begins to feel real as opposed to metaphorical pain. The subjects, in other words, need only be told that other minds control the graphics on the screen before them, and the scant information provided by those graphics trigger real world pain. A handful of pixels and a little fib is all that’s required to cue the pain of social rejection.

As one might imagine, Silicon Valley has taken notice.

The HCI field finds its roots in the 1960’s with the research of Joseph Weizenbaum at the MIT Artificial Intelligence Laboratory. Even given the rudimentary computing power at his disposal, his ‘Eliza’ program, which relied on simple matching and substitution protocols to generate questions, was able to cue strong emotional reactions in many subjects. As it turns out, people regularly exhibit what the late Clifford Nass called ‘mindlessness,’ the reliance on automatic scripts, when interacting with artificial agents. Before you scoff at the notion, recall the 2015 Ashley Madison hack, and the subsequent revelation that it deployed more than 70,000 bots to conjure the illusion of endless extramarital possibility. These bots, like Eliza, were simple, mechanical affairs, but given the context of Ashley Madison, their behaviour apparently convinced millions of men that some kind of (promising) soap opera was afoot.

The great paradox, of course, is that those automatic scripts belong to the engine of ‘mindreading,’ our ability to predict, explain, and manipulate our fellow human beings, not to mention ourselves. They only stand revealed as mechanical, ‘mindless,’ when tasked to cognize something utterly without evolutionary precedent: an artificial agent. Our power to peer into one another’s souls, in other words, becomes little more than a grab-bag of exploitable reflexes in the presence of AI.

The claim boggles, I admit, but from a Darwinian perspective, it’s hard to see how things could be otherwise. Our capacity to solve one another is largely a product of our hunter-gatherer past, which is to say, environments where human intelligence was the only game in town. Why evolve the capacity to solve for artificial intelligences, let alone ones possessing Big Data resources? The cues underwriting human social cognition may seem robust, but this is an artifact of ecological stability, the fact that our blind trust in our shared social biology has served so far. We always presume our environments indestructible. As the species responsible for the ongoing Anthropocene extinction, we have a long history of recognizing ecological peril only after the fact.

Sherry Turkle, MIT professor and eminent author of Alone Together, has been warning of what she calls “Darwinian buttons” for over a decade now. Despite the explosive growth in Human-Computer Interaction research, her concerns remain at best, a passing consideration. As part of our unconscious, automatic cognitive systems, we have no conscious awareness that such buttons even exist. They are, to put it mildly, easy to overlook. Add to this the overwhelming institutional and economic incentive to exploit these cues, and the AI community’s failure to consider Turkle’s misgivings seems all but inevitable.

Like most all scientists, researchers in the field harbor only the best of intentions, and the point of AI, as they see it, is to empower consumers, to give them what they want. The vast bulk of ongoing research in Human-Computer Interaction is aimed at “improving the user experience,” identifying what cues trust instead of suspicion, attachment instead of avoidance. Since trust requires competence, a great deal of the research remains focused on developing the core cognitive competencies of specialized AI systems—and recent advances on this front have been nothing if not breathtaking. But the same can be said regarding interpersonal competencies as well—enough to inspire Clifford Nass and Corina Yen to write, The Man Who Lied to his Laptop, a book touted as the How to Win Friends and Influence People of the 21st century. In the course of teaching machines how to better push our buttons, we’re learning how to better push them as well.

Simply because it is so easily miscued, human social cognition depends on trust. Shapes, after all, are cheap, while soap operas represent a potential goldmine. This explains our powerful, hardwired penchant for tribalism: the intimacy of our hunter-gatherer past all but assured trustworthiness, providing a cheap means of nullifying our vulnerability to social deception. When Trump decries ‘fake news,’ for instance, what he’s primarily doing is signaling group membership. He understands, the instinctive way we all understand, that the best way to repudiate damaging claims is to circumvent them altogether, and focus on the group membership of the claimer. Trust, the degree we can take one another for granted, is the foundation of cooperative interaction.

We are about to be deluged with artificial friends. In a recent roundup of industry forecasts, Forbes reports that AI related markets are already growing, and expected to continue growing, by more than 50% per annum. Just last year, Microsoft launched its Bot Framework service, a public platform for creating ‘conversational user interfaces’ for a potentially endless variety of commercial purposes, all of it turning on Microsoft’s rapidly advancing AI research. “Build a great conversationalist,” the site urges. “Build and connect intelligent bots to interact with your users naturally wherever they are…” Of course, the term “naturally,” here, refers to the seamless way these inhuman systems cue our human social cognitive systems. Learning how to tweak, massage, and push our Darwinian buttons has become an out-and-out industrial enterprise.

As mentioned above, Human-Human Interaction consists of pushing these buttons all the time, prompting automatic scripts that prompt further automatic scripts, with only the rare communicative snag giving us pause for genuine conscious deliberation. It all works simply because our fellow humans comprise the ancestral ecology of social cognition. As it stands, cuing social cognitive reflexes out of school is largely the province of magicians, con artists, and political demagogues. Seen in this light, the AI revolution looks less a cornucopia of marvels than the industrialized unleashing of endless varieties of invasive species—an unprecedented overthrow of our ancestral social cognitive habitats.

A habitat that, arguably, is already under severe duress.

In 2006, Maki Fukasawa coined the term ‘herbivore men’ to describe the rising number of Japanese males expressing disinterest in marital or romantic relationships with women. And the numbers have only continued to rise. A 2016 National Institute of Population and Social Security Research survey reveals that 42 percent of Japanese men between the ages of 18 and 34 remain virgins, up six percent from a mere five years previous. For Japan, a nation already struggling with the economic consequences of depopulation, such numbers are disastrous.

And Japan is not alone. In Man, Interrupted: Why Young Men are Struggling and What We Can Do About It, Philip Zimbardo (of the Stanford Prisoner Experiment fame) and Nikita Coulombe provide a detailed account of how technological transformations—primarily online porn, video-gaming, and virtual peer groups—are undermining the ability of American boys to academically achieve as well as maintain successful relationships. They see phenomena such as the growing MGTOW (‘men going their own way’) movement as the product of the way exposure to virtual, technological environments leaves them ill-equipped to deal with the rigours of genuine social interaction.

More recently, Jean Twenge, a psychologist at San Diego State University, has sounded the alarm on the catastrophic consequences of smartphone use for post-Millennials, arguing that “the twin rise of the smartphone and social media has caused an earthquake of a magnitude we’ve not seen in a very long time, if ever.” The primary culprit: loneliness. “For all their power to link kids day and night, social media also exacerbate the age-old teen concern about being left out.” Social media, in other words, seem to be playing the same function as the Cyberball game used by researchers to neuroimage the pain of social rejection. Only this time the experiment involves an entire generation of kids, and the game has no end.

The list of curious and troubling phenomena apparently turning on the ways mere connectivity has transformed our social ecology is well-nigh endless. Merely changing how we push one another’s Darwinian buttons, in other words, has impacted the human social ecology in historically unprecedented ways. And by all accounts, we find ourselves becoming more isolated, more alienated, than at any other time in human history.

So what happens when we change the who? What happens when the heaven of social belonging goes on sale?

Good question. There is no “Centre for the Scientific Study of Human Meaning” in the world. Within the HCI community, criticism is primarily restricted to the cognitivist/post-cognitivist debate, the question of whether cognition is intrinsically independent or dependent of an agent’s ongoing environmental interactions. As the preceding should make clear, numerous disciplines find themselves wandering this or that section of the domain, but we have yet to organize any institutional pursuit of the questions posed here. Human social ecology, the study of human interaction in biologically amenable terms, remains the province of storytellers.

We quite literally have no clue as to what we are about to do.

Consider Mark Zuckerberg’s and Elon Musk’s recent ‘debate’ regarding the promise and threat of AI. Musk, of course, has garnered headlines for quite some time with fears of artificial superintelligence. He’s famously called AI “our biggest existential threat,” openly referring to Skynet and the prospect of robots mowing down civilians on the streets. On a Sunday this past July, Zuckerberg went live in his Palo Alto backyard while smoking meats to host an impromptu Q&A. At the fifty-minute mark, he answers a question regarding Musk’s fears, and responds, “I think people who are naysayers and try to drum up these doomsday scenarios—I don’t understand it. It’s really negative and in some ways I think it’s pretty irresponsible.”

On the Tuesday following, Musk tweeted in response: “I’ve talked to Mark about this. His understanding of the subject is limited.”

To the extent that human interaction is ecological (and how could it be otherwise?), both can be accused of irresponsibility and limited understanding. The threat of ‘superintelligence,’ though perhaps inevitable, remains far enough in the future to easily dismiss as a bogeyman. The same can be said regarding “peak human” arguments predicting mass unemployment. The threat of economic disruption, though potentially dire, is counter-balanced by the promise of new, unforeseen economic opportunity. This leaves us with the countless number of ways AI will almost certainly improve our lives: fewer car crashes, fewer misdiagnoses, and so on. As a result, one can predict how all such exchanges will end.

The contemporary AI debate, in other words, is largely a pseudo-debate.

The futurist Richard Yonck’s account of ‘affective computing’ somewhat redresses this problem in his recently released Heart of the Machine, but since he begins with the presupposition that AI represents a natural progression, that the technological destruction of ancestral social habitats is the ancestral habitat of humanity, he remains largely blind to the social ecological consequences of his subject matter. Espousing a kind of technological fatalism (or worse, fundamentalism), he characterizes AI as the culmination of a “buddy movie” as old as humanity itself. The oxymoronic, if not contradictory, prospects of ‘artificial friends’ simply does not dawn on him.

Neil Lawrence, a professor of machine learning at the University of Sheffield and technology columnist at The Guardian, is the rare expert who recognizes the troubling ecological dimensions of the AI revolution. Borrowing the distinction between System Two, or conscious, ‘mindful’ problem-solving, and System One, or unconscious, ‘mindless’ problem-solving, from cognitive psychology, he warns of what he calls System Zero, what happens when the market—via Big Data, social media, and artificial intelligence—all but masters our Darwinian buttons. As he writes,

“The actual intelligence that we are capable of creating within the next 5 years is an unregulated System Zero. It won’t understand social context, it won’t understand prejudice, it won’t have a sense of a larger human objective, it won’t empathize. It will be given a particular utility function and it will optimize that to its best capability regardless of the wider negative effects.”

To the extent that modern marketing (and propaganda) techniques already seek to cue emotional as opposed to rational responses, however, there’s a sense in which ‘System Zero’ and consumerism are coeval. Also, economics comprises but a single dimension of human social ecology. We have good reason to fear that Lawrence’s doomsday scenario, one where market and technological forces conspire to transform us into ‘consumer Borg,’ understates the potential catastrophe that awaits.

The closest one gets to a genuine analysis of the interpersonal consequences of AI lies in movies such as Spike Jonze’s science-fiction masterpiece, Her, or the equally brilliant HBO series, Westworld, scripted by Charles Yu. ‘Science fiction,’ however, happens to be the blanket term AI optimists use to dismiss their critical interlocutors.

When it comes to assessing the prospect of artificial intelligence, natural intelligence is failing us.

The internet was an easy sell. After all, what can be wrong with connecting likeminded people?

The problem, of course, is that we are the evolutionary product of small, highly interdependent, hunter-gatherer communities. Historically, those disposed to be permissive had no choice but to continually negotiate with those disposed to be authoritarian. Each party disliked the criticism of the other, but the daily rigors of survival forced them to get along. No longer. Only now, a mere two decades later, are we discovering the consequences of creating a society that systematically segregates permissives and authoritarians. The election of Donald Trump has, if nothing else, demonstrated the degree to which technology has transformed human social ecology in novel, potentially disastrous ways.

AI has also been an easy sell—at least so far. After all, what can be wrong with humanizing our technological environments? Imagine a world where everything is ‘user friendly,’ compliant to our most petulant wishes. What could be wrong with that?

Well, potentially everything, insofar as ‘humanizing our environments’ amounts to dehumanizing our social ecology, replacing the systems we are adapted to solve, our fellow humans, with systems possessing no evolutionary precedent whatsoever, machines designed to push our buttons in ways that optimize hidden commercial interests. Social pollution, in effect.

Throughout the history of our species, finding social heaven has required risking social hell. Human beings are as prone to be demanding, competitive, hurtful—anything but ‘user friendly’—as otherwise. Now the industrial giants of the early 21st century are promising to change all that, to flood the spaces between us with machines designed to shoulder the onerous labour of community, citizenship, and yes, even love.

Imagine a social ecology populated by billions upon billions of junk intelligences. Imagine the solitary confinement of an inhuman crowd. How will we find one another? How will we tolerate the hypersensitive infants we now seem doomed to become?

The Zuckerberg Illusion

by rsbakker

obama-wired

So the special issue of Wired Magazine edited by Barack Obama has just come out, and I wanted to draw attention to Mark Zuckerberg’s response to the President’s challenge to “ensure that artificial intelligence helps rather than hurts us.” Somehow, someway, this issue has to move away from the ‘superintelligence’ debate and toward a collective conversation on the impact of AI on human cognitive ecology. Zuckerberg’s response betrays a tragic lack of understanding from the man who, arguably, has already transformed our social cognitive ecologies more radically than any other individual in the history of the human race. Anyone knowing some way of delivering this message from steerage up to the bridge, forward the bloody thing, because the combination of this naivete with the growing ubiquity of AI is becoming, ahem, a little scary. The more baked-in the existing trends become, the harder the hard decisions will become.

Zuckerberg begins his response to Obama’s challenge sounding very much like a typical American industrialist: only the peculiarity of his product makes his claim remarkable.

“People have always used technology as a lever to improve lives and increase productivity. But at the beginning of every cycle of invention, there’s a temptation to focus on the risks that come with a new technology instead of the benefits it will bring.

Today we are seeing that happen with artificial intelligence.”

What he wants to do in this short piece is allay the fears that have arisen regarding AI. His strategy for doing so is to show how our anxieties are the same overblown anxieties that always occasion the introduction of some new technology. These too, he assures us, will pass in time. Ultimately, he writes:

“When people come up with doomsday scenarios about AI, it’s important to remember that these are hypothetical. There’s little basis outside science fiction to believe they will come true.”

Of course, one need only swap out ‘AI’ with ‘industrialization’ to appreciate that not all ‘doomsday scenarios’ are equal. By any comparison, the Anthropocene already counts as one of the great extinction events to befall the planet, an accomplished ‘doomsday’ for numerous different species, and an ongoing one for many others. The reason for this ongoing extinction has to do with the supercomplicated systems of interdependency comprising our environments. Everything is adapted to everything else. Like pouring sand into a gas tank, introducing unprecedented substances and behaviours (such as farming) into existing ecologies progressively perturbs these systems, until eventually they collapse, often taking down other systems depending on them.

Malthus is the first on record predicting the possibility of natural environmental collapse in the 18th century, but the environmental movement only really got underway as the consequences of industrialization became evident in the 19th century. The term pollution, which during the Middle-ages meant defilement, took on its present meaning as “unnatural substance in natural systems” at the turn of the 20th century.

Which begs the question: Why were our ancestors so long in seeing the peril presented by industrialization? Well, for one, the systems comprising ecologies are all, in some way or another, survivors of prior ecological collapses. Ecologies are themselves adaptive systems, exhibiting remarkable resilience in many cases—until they don’t. The supercomplicated networks of interdependence constituting environments only became obvious to our forebears when they began really breaking down. Once one understands the ecological dimension of natural environments, the potentially deleterious impact of ecologically unprecedented behaviours and materials becomes obvious. If the environmental accumulation of industrial by-products constitutes an accelerating trend, then far from a science fiction premise, the prospect of accelerating ecological degradation of environments becomes a near certainty, and the management of ecological consequences an absolute necessity.

Which begs a different, less obvious question: Why would these networks of ecological interdependence only become visible to our ancestors after they began breaking down? Why should humans initially atomize their environments, and only develop complex, relational schemes after long, hard experience? The answer lies in the ecological nature of human cognition, the fact that we evolved to take as much ‘for granted’ as possible. The sheer complexity of the deep connectivity underwriting our surrounding environments renders them computationally intractable, and thus utterly invisible to us. (This is why the ecology question probably seemed like such an odd thing to ask: it quite literally goes without saying that we had to discover natural ecology). So cognition exploits the systematic correlations between what information is available and the systems requiring solution to derive ecologically effective behaviours. The human penchant for atomizing and essentializing their environments enables them to cognize ecology despite remaining blind to it.

What does any of this have to do with Zuckerberg’s optimistic argument for plowing more resources into the development of AI? Well, because I think it’s pretty clear he’s labouring under the very same illusion as the early industrialists, the illusion of acting in a grand, vacant arena, a place where unintended consequences magically dissipate instead of radiate.

The question, recall, is whether doomsday scenarios about AI warrant widespread alarm. It seems pretty clear, and I’m sure Zuckerberg would agree, that doomsday scenarios about industrialization do warrant widespread alarm. So what if what Zuckerberg and everyone else is calling ‘AI’ actually constitutes a form of cognitive industrialization? What will be the cognitive ecological impact of such an event?

We know that human cognition is thoroughly heuristic, so we know that human cognition is thoroughly ecological. The reason Sherry Turkle and Deidre Barrett and others worry about the ease with which human social cognition can be hacked turns on the fact that human social cognition is ecological through and through, dependent on the stable networks of interdependence. The fact is human sociocognition evolved to cope with other human intelligences, to solve on the basis of cues systematically correlated to other human brains, not to supercomputers mining vast data sets.  Take our love of flattery. We evolved in ecologies where our love for flattery is balanced against the inevitability of criticism. Ancestrally, pursuing flattery amounts to overcoming—i.e., answering—criticism. We generally hate criticism, but given our cognitive ecology, we had no choice but ‘to take our medicine.’

And this is but one of countless examples.

The irony is that Zuckerberg is deeply invested in researching human cognitive ecology: computer scientists (like Hector Levesque) can rail against ‘bag of tricks’ approaches to cognition, but they will continue to be pursued because behaviour cuing behaviour is all that’s required (for humans or machines, I think). Now Zuckerberg, I’m sure, sees himself exclusively in the business of providing value for consumers, but he needs to understand how his dedication to enable and delight automatically doubles as a ruthless quest to demolish human cognitive ecology. Rewriting environments ‘to make the user experience more enjoyable’ is the foundation all industrial enterprise, all ecological destruction, and the AI onslaught is nothing if not industrial.

Deploying systems designed to cue human social cognition in the absence of humans is pretty clearly a form of deception. Soon, every corporate website will be a friend… soulful, sympathetic,  utterly devoted to our satisfaction, as well as inhuman, designed to exploit, and knowing us better than any human could hope to, including ourselves. And as these inhuman friends become cheaper and cheaper, we will be deluged by them, ‘junk intelligences,’ each of them so much wittier, so much wiser, than any mundane human can hope to appear.

“At a very basic level, I think AI is good and not something we should be afraid of,” Zuckerberg concludes. “We’re already seeing examples of how AI can unlock value and improve the world. If we can choose hope over fear—and if we advance the fundamental science behind AI—then this is only the beginning.”

Indeed.

obama-worried

The Dim Future of Human Brilliance

by rsbakker

Moths to a flame

Humans are what might be called targeted shallow information consumers in otherwise unified deep information environments. We generally skim only what information we need—from our environments or ourselves—to effect reproduction, and nothing more. We neglect gamma radiation for good reason: ‘deep’ environmental information that makes no reproductive difference makes no cognitive difference. As the product of innumerable ancestral ecologies, human cognitive biology is ecological, adapted to specific, high-impact environments. As ecological, one might expect that human cognitive biology is every bit as vulnerable to ecological change as any other biological system.

Under the rubric of  the Semantic Apocalypse, the ecological vulnerability of human cognitive biology has been my focus here for quite some time at Three Pound Brain. Blind to deep structures, human cognition largely turns on cues, sensitivity to information differentially related to the systems cognized.  Sociocognition, where a mere handful of behavioural cues can trigger any number of predictive/explanatory assumptions, is paradigmatic of this. Think, for instance, how easy it was for Ashley Madison to convince its predominantly male customers that living women were checking their profiles.  This dependence on cues underscores a corresponding dependence on background invariance: sever the differential relations between the cues and systems to be cognized (the way Ashley Madison did) and what should be sociocognition, the solution of some fellow human, becomes confusion (we find ourselves in ‘crash space’) or worse, exploitation (we find ourselves in instrumentalized crash space, or ‘cheat space’).

So the questions I think we need to be asking are:

What effect does deep information have on our cognitive ecologies? The so-called ‘data deluge’ is nothing but an explosion in the availability of deep or ancestrally inaccessible information. What happens when targeted shallow information consumers suddenly find themselves awash in different kinds of deep information? A myriad of potential examples come to mind. Think of the way medicalization drives accommodation creep, how instructors are gradually losing the ability to judge character in the classroom. Think of the ‘fear of crime’ phenomena, how the assessment of ancestrally unavailable information against implicit, ancestral baselines skews general perceptions of criminal threat. For that matter, think of the free will debate, or the way mechanistic cognition scrambles intentional cognition more generally: these are paradigmatic instances of the way deep information, the primary deliverance of science, crashes the targeted and shallow cognitive capacities that comprise our evolutionary inheritance.

What effect does background variation have on targeted, shallow modes of cognition? What happens when cues become differentially detached, or ‘decoupled,’ from their ancestral targets? Where the first question deals with the way the availability of deep information (literally, not metaphorically) pollutes cognitive ecologies, the ways human cognition requires the absence of certain information, this question deals with the way human cognition requires the presence of certain environmental continuities. There’s actually been an enormous amount of research done on this question in a wide variety of topical guises. Nikolaas Tinbergen coined the term “supernormal stimuli” to designate ecologically variant cuing, particularly the way exaggerated stimuli can trigger misapplications of different heuristic regimes. He famously showed how gull chicks, for instance, could be fooled into pecking false “super beaks” for food given only a brighter-than-natural red spot. In point of fact, you see supernormal stimuli in dramatic action anytime you see artificial outdoor lighting surrounded by a haze of bugs: insects that use lunar transverse orientation to travel at night continually correct their course vis a vis streetlights, porch lights, and so on, causing them to spiral directly into them. What Tinbergen and subsequent ethology researchers have demonstrated is the ubiquity of cue-based cognition, the fact that all organisms are targeted, shallow information consumers in unified deep information environments.

Deirdre Barrett has recently applied the idea to modern society, but lacking any theory of meaning, she finds herself limited to pointing out suggestive speculative parallels between ecological readings and phenomena that are semantically overdetermined otherwise. For me this question calves into a wide variety of domain-specific forms, but there’s an important distinction to be made between the decoupling of cues generally and strategic decoupling, between ‘crash space’ and ‘cheat space.’ Where the former involves incidental cognitive incapacity, human versions of transverse orientation, the latter involves engineered cognitive incapacity. The Ashley Madison case I referenced above provides an excellent example of simply how little information is needed to cue our sociocognitive systems in online environments. In one sense, this facility evidences the remarkable efficiency of human sociocognition, the fact that it can do so much with so little. But, as with specialization in evolution more generally, this efficiency comes at the cost of ecological dependency: you can only neglect information in problem-solving so long as the systems ignored remain relatively constant.

And this is basically the foundational premise of the Semantic Apocalypse: intentional cognition, as a radically specialized system, is especially vulnerable to both crashing and cheating. The very power of our sociocognitive systems is what makes them so liable to be duped (think religious anthropomorphism), as well as so easy to dupe. When Sherry Turkle, for instance, bemoans the ease with which various human-computer interfaces, or ‘HCIs,’ push our ‘Darwinian buttons’ she is talking about the vulnerability of sociocognitive cues to various cheats (but since she, like Barrett, lacks any theory of meaning, she finds herself in similar explanatory straits). In a variety of experimental contexts, for instance, people have been found to trust artificial interlocutors over human ones. Simple tweaks in the voices and appearance of HCIs have a dramatic impact on our perceptions of those encounters—we are in fact easily manipulated, cued to draw erroneous conclusions, given what are quite literally cartoonish stimuli. So the so-called ‘internet of things,’ the distribution of intelligence throughout our artifactual ecologies, takes on a far more sinister cast when viewed through the lens of human sociocognitive specialization. Populating our ecologies with gadgets designed to cue our sociocognitive capacities ‘out of school’ will only degrade the overall utility of those capacities. Since those capacities underwrite what we call meaning or ‘intentionality,’ the collapse of our ancestral sociocognitive ecologies signals the ‘death of meaning.’

The future of human cognition looks dim. We can say this because we know human cognition is heuristic, and that specific forms of heuristic cognition turn on specific forms of ecological stability, the very forms that our ongoing technological revolution promises to sweep away. Blind Brain Theory, in other words, offers a theory of meaning that not only explains away the hard problem, but can also leverage predictions regarding the fate of our civilization. It makes me dizzy thinking about it, and suspicious—the empty can, as they say, rattles the loudest. But this preposterous scope is precisely what we should expect from a genuinely naturalistic account of intentional phenomena. The power of mechanistic cognition lies in the way it scales with complexity, allowing us to build hierarchies of components and subcomponents. To naturalize meaning is to understand the soul in terms continuous with the cosmos.

This is precisely what we should expect from a theory delivering the Holy Grail, the naturalization of meaning.

You could even argue that the unsettling, even horrifying consequences evidence its veracity, given there’s so many more ways for the world to contradict our parochial conceits than to appease them. We should expect things will end ugly.