What Can We Know?
On Knowledge and the Problem of Incentives
1. The Classic Problem of Skepticism
We take ourselves to know many things. For example, I know that I have hands and am sitting at a table drinking coffee. But what is the basis for this supposed knowledge?
Descartes famously noticed that such knowledge is mediated by our experiences. But it’s possible that our experiences can deceive us. In a vivid dream for example, I could be having the same experiences as I am now, but such experiences would be deceptive. Likewise, an evil demon could be giving me the same kinds of experiences I’m having now, in which case I would be deceived. Modern philosophers often reference the “Brain in a Vat” thought experiment. Imagine that you are a brain in a vat being fed electric signals so that you have certain visual experiences, hear sounds, etc. How can you know this is not the case?
In the movie The Matrix, humans are plugged into an elaborate simulation so that it appears to them they are leading normal lives in the 1990s. But, these appearances are misleading. Even though they think they’re living in 1990s New York City, the people in the simulation are actually immersed in a fluid, hooked up to various wires, so that the Machines can harvest their electrical energy.
Morpheus, pictured above, offers Neo the “Red Pill” which scrambles the signals and brings Neo into the real world. As Morpheus offers him the pill, he says:
[Y]ou are a slave, Neo. Like everyone else, you were born into bondage, born inside a prison that you cannot smell, taste, or touch. A prison for your mind. Unfortunately, no one can be told what the Matrix is. You have to see it for yourself...Remember—all I am offering is the truth, nothing more.
How would we, here and now, know that we aren’t in a simulation, like Neo?
Now, philosophers have come up with several responses to the classic problem of skepticism. But there is a class of problems that threaten to undermine our claims to knowledge, which have received less attention. These problems arise due to the fact that we are social creatures: we gain much of our information from others.
Rather than the Machines or an Evil Demon, it might turn out that we are deceived by the social incentives and structures that bring our information to us. Unlike the former scenarios, there need not be any explicit intention to deceive, but we may be deceived nonetheless. And unlike classical skepticism, where the imagined scenario is merely possible, the scenarios mentioned above are probable, and in fact, highly likely.
2. Mill on Newtonian Physics
John Stuart Mill says something quite striking in On Liberty, Chapter 2:
If even the Newtonian philosophy were not permitted to be questioned, mankind could not feel as complete assurance of its truth as they now do. The beliefs which we have most warrant for, have no safeguard to rest on, but a standing invitation to the whole world to prove them unfounded.
In 1859, when this was published, Newtonian physics was the gold standard of science. It had been confirmed through countless experiments. And much of technology, engineering, and scientific practice relied on it. Yet even then, Mill says, if it could not be challenged, people could not be assured that they knew that it was true.
Of course now, with the privilege of hindsight, we recognize that Newtonian physics is, strictly speaking, false, though it works as a good approximation in non-relativistic and non-micro-scale contexts. Einstein would come along in the 20th century and upend our understanding of space and time with his special and general theories of relativity. And the double slit experiment would suggest that matter has both particle-like and wave-like properties.
Even so, at the time Mill was writing, Newtonian physics had a lot going for it. So why could people not be assured of its truth if it could not be challenged?
One way of approaching this issue is recognizing that incentives matter. People respond to their incentives. (If the supposed incentives do not affect behavior, they are not incentives in the first place.)
What would it mean if “Newtonian philosophy were not permitted to be questioned”? Perhaps questioners would not get university jobs. Or they could not receive funding to conduct their work. Maybe their papers, even if they were good, could not pass peer-review (assuming the reviewers were unduly biased in favor of Newtonian physics). In more severe cases, perhaps they would face legal penalties or be sent to labor camps, as was the case under Lysenkoism in the USSR in the mid 20th century.
Lysenko speaking at the Kremlin in 1935
Such incentives can affect our knowledge in three ways:
First, there may be good counter-evidence out there. But it cannot make its way to us because individuals fear repercussion (social, professional, or legal). Because people fear repercussion, they don’t share their evidence.
Second, even if occasionally people do share such evidence, it does not make its way to us. The evidence is blocked by hard or soft censorship.
Third, the incentives distort the behavior of those who could uncover new counter-evidence. In a world where Newtonian physics was sacrosanct, Einstein might have remained a patent clerk. His work would not have received uptake and recognition. Recognizing this, he might not have embarked on this work in the first place.
3. Incentives and Modern Social Science
Thankfully, we don’t live under a regime of top-down censorship as in the case of the mid-century USSR. (At least, for the most part, in the modern liberal democracies.) But Mill stressed that this is not enough to secure knowledge:
In respect to all persons but those whose pecuniary circumstances make them independent of the good will of other people, opinion, on this subject, is as efficacious as law; men might as well be imprisoned, as excluded from the means of earning their bread.
Professional and social penalties for doing work or expressing views that depart from the dominant ideology (within the relevant institutions) can be quite efficacious. They can supply powerful incentives. This can result in the following:
Graduate students or post-docs (whose employment is precarious) will choose to work on “safe” topics that do not rock the boat.
Alternatively, such individuals might report or publish only those results that support the conclusions welcomed by the dominant ideology.
Established scholars will have trouble placing their work in high-prestige peer-reviewed journals, if it suggests ideology-discordant conclusions. Such individuals might then shy away from such work or place it in less prestigious (hence less read and cited) venues.
Scholars can experience a “peer-to-peer distancing” effect after writing on (putatively) controversial topics, resulting in a loss of citations and productivity. (See the new working paper by Kris Gulati and Lorenzo Palladini documenting this effect.)
Even tenured academics who hope to move up to more prestigious jobs will avoid work (even if rigorous) that will place them at a disadvantage in the job market.
In large part, many of these pressures come from other academics, even if not from the public. The University of Texas at Austin-based psychologist, Kathryn Paige Harden, in a recent podcast episode on her new book, Original Sin: On the Genetics of Vice, the Problem of Blame, and the Future of Forgiveness, describes her experience post-publication.
Interviewer: What happened after the publication of your last book?
Harden: Oh, it was a wild time. There was a lot of controversy. There was a lot of pushback. The conversations that I had with real people, not with other academics, but with just people who wrote me, people who, you know, happened to encounter the book in some way, that was fantastic.
People wrote me and they said, “I’ve always wondered why I’m so different from my parents or why I’m so different from my siblings and your work has given me a new way of understanding that.”
They wrote to me about their decision to have kids or their decision not to have kids and how thinking about genetics has shaped that. So that part of the conversation, which is the dialogue between an author and their readers, was fantastic. I loved that.
And then there was another part of the dialogue, which was me with other academics. And that was really surprising to me, in part because I felt like some people needed to turn me into a villain in order to get their own message out. I was kind of caught off guard by that whole process.
I wish I could say that I had a thicker skin now, but I don’t in many ways. I really do care what people think. I care about getting it right. So it took me a bit to think about how to get myself back out there in terms of the ideas in the wake of that.
Harden’s experience illustrates the kinds of social sanctions that even a full professor at a highly prestigious institution and with an impressive scholarly record can face. What message does this send to professionally vulnerable graduate students and non-tenured academics?
4. Conflict of Interest
Consider the following case.
Longevitin: Suppose there is a new drug on the market called Longevitin that promises to increase lifespan with little to no side effects. There are many studies supporting this and none on the other side. So far so good. But now we notice that all these studies have been funded by Big Pharma who wants to sell Longevitin. Big Pharma was careful to fund only those researchers who would give them a sympathetic treatment.
Should we take these studies at face value? Presumably not. If there are bad side effects, researchers do not have the incentives to report them. And maybe the increase in lifespan has been exaggerated. We, as non-experts, cannot tell. Moreover, as non-experts, we can only trust the research output if the incentives are right. So we cannot trust the upshot of these Longevitin studies.
Now, this is a well-known type of case when it comes to conflict of interest. If donors fund research only if it supports some conclusion, then we have to take the conclusion with a grain (or heap, as the case may be) of salt.
But conflict of interest doesn’t end there. Victor Kumar had a piercing recent post in this regard:
If Kumar is right about the professional incentives faced by researchers, we have to take the “political orthodoxy” with a good amount of salt, just as in the case of Longevitin research. This is for the simple reason that most people want to advance professionally—as humans, we are status-seeking creatures. If violating political orthodoxy leads to a loss of status, then many (even if not all) will shy away from work that does so.
As Richard Yetter Chappell recently put it:
If we really care about distortions threatening our collective academic integrity, my top recommendation would be to look out for areas of false consensus and social pressure to conform. Then work on being more tolerant of disagreement and curious about research that doesn’t immediately “vibe” with orthodoxy—especially while serving on hiring committees.
The problem here is simple. If someone can’t get hired if their work doesn’t “vibe with the orthodoxy,” then successful academics will be selected on the basis of ideological conformity. This in turn will mean non-experts cannot justifiably rely on their work insofar as it is relevant to the orthodoxy. It would be like putting one’s faith in Lysenkoism or Longevitin.
Source: Prosocial motives underlie scientific censorship by scientists: A perspective and research agenda (PNAS, 2023)
Even the resulting peer-reviewed literature will then be unreliable, as illustrated above. For, the quality of peer-review is a function of the quality of the procedure that selects the peers in the first place. If “peers” are selected through a process of hiring bias (see evidence of this here, here, and here), then the resulting peer-review itself will be biased.
As the philosopher Helen Longino puts it:
Once propositions, theses, and hypotheses are developed, what will become scientific knowledge is produced collectively through the clashing and meshing of a variety of points of view…
the greater the number of different points of view included in a given community, the more likely it is that its scientific practice will be objective…such diversity is a necessary but not a sufficient condition for objectivity.
The implication then is that if those with different points of view are excluded from admission in the first place, the resulting work cannot be objective. We then cannot rely on it for knowledge.
5. A Problem for Everyone (except Cypher)
Who is this a problem for? At the outset, it might be thought that the above is chiefly a problem for the dissenters, those who depart from the prevailing orthodoxies within the academic system.
That initial reaction is mistaken. The situation is also a problem for the public at large. As non-experts, the public can only justifiably trust the outputs of the scholarly community if there are no substantial “conflicts of interest” of the sort that Kumar points to. This might go some way in both explaining and, moreover, justifying the recent collapse in public trust in higher education.
But importantly, it is a problem even for those academics who agree with the prevailing orthodoxies. For, ideally, we want our beliefs to be true and to be justified. We want to know things. As Aristotle famously wrote at the beginning of his Metaphysics, “All men by nature desire to know.” But if Mill is right, then we cannot know in the conditions described above. All we can aspire to is what he calls “dead dogma.”
Now, despite Aristotle’s declaration, some might be okay with this. In The Matrix, the character Cypher wants to be put back into the simulation, as someone with wealth and status. Living in reality is too miserable for him. Wealth and status are better for him than knowledge, in his estimation. Below is the back and forth from the famous steak scene:
Agent Smith: Do we have a deal, Mr. Reagan?
Cypher: You know, I know this steak doesn’t exist. I know that when I put it in my mouth, the Matrix is telling my brain that it is juicy and delicious. After nine years, you know what I realize? Ignorance is bliss.
Agent Smith: Then we have a deal?
Cypher: I don’t want to remember nothing. Nothing. You understand? And I want to be rich. You know, someone important. Like an actor.
Agent Smith: Whatever you want, Mr. Reagan.
Cypher: Okay. I get my body back into a power plant. Re-insert me into the Matrix. I’ll get you what you want.
To the Cyphers of this world: carry on. But to those who want to know: you should worry about the problem of social incentives even if you agree with the dominant social and political views of the academy. You might be—indeed, likely are—living in the Matrix.