Can history ever be both reliable AND valid ?
Whether it is possible to produce historical knowledge which is both reliable and valid was explored in my holiday reading this year. I read the book Upheaval, How Nations Cope with Crisis and Change by Jared Diamond. You may know Jared Diamond, he's the guy who wrote Guns, Germs and Steel. It's a very popular TOK book that many teachers, students, and interested people have read.
This is his second follow-up book, Upheaval. In this blog will not be going into his findings, and what he actually says about how nations cope with crisis and change. That's not really TOK, that's history or global politics. But this book is really interesting for us to understand how historical knowledge is made and how conclusions are reached when we're making historical knowledge. It is of particular relevance to solving the challenge of whether history can be both reliable and valid.
The challenge of history for historians.
Historians have some key questions to answer when creating historical knowledge:
First of all is the idea of can you establish history which is both valid and reliable. Valid means that it's an accurate description or explanation of that particular moment, or that particular event or time in history. Reliable means the conclusions that we can draw from that historical event apply in other situations. As such, if the knowledge is reliable we will be able to generalise it to other historical events.
Students often write in their TOK essays that the function of history is to learn from it so that we don't repeat the same mistakes in the future. Now whether that actually is the function of history is immaterial for this blog. But what that supposes is that the things that we learned from an earlier period could also apply in / to other periods. As such, history would have to be both valid and reliable.
Jared Diamond is a professor at UCLA, he trained as a biologist and physician. His initial training was in AOK Natural Sciences. However now he's moved more into social sciences, human sciences and history in explaining and describing human behaviour. This initial training informs his understanding of, and approaches to, methodology.
A historian explains, and justifies, his methodology
The prologue of this book is really interesting for ToK students as Prof Diamond writes about the methodology used to write the book. He explains that ideally he would use quantitative methods to establish reliable cause and effect relationships. That means he would build mathematical models, statistical models , into which he would pump lots of data, and that data would give him mathematical and statistical outcomes from which he could establish cause and effect variables.
Those statistical outcomes tend to be highly reliable, sometimes valid and sometimes less valid. Generally most ToK. students (and most people in general) prefer statistical outcomes. For example: If you're getting on a plane, you may ask is this plane safe? If someone says it's safe most of the time, well you want to know what does "most of the time" mean? What's the danger to me? And the best answer, the one which would satisfy you the most is a statistical answer. So if you were told "the plane is safe 51% of the time, 49% of the time it's not safe", then you're probably not going to get on the plane. If they say "well in testing it's safe 99.975% of the time" then you're probably going to get on the plane with a little knot in your stomach.
The challenge of quantitative methodologies in AoK History
We like statistics, we like knowledge that is characterised as a "scientific fact". However, how do we prove scientific facts in ? How can we arrive at conclusive causal facts in history? Prof Diamond explains that he would like to use quantitative methodologies to answer the research question of the book, but then he explains that it's really difficult to establish quantitative cause & effect relationships when you're answering the question "How do nations cope with crisis and change? ".
Trying to make history that is both reliable and valid.
It's really difficult to use quantitative methodologies for many reasons, incl:
there are so many different nations that you could consider.
there are so many instances of crisis and change in those nations that you could consider.
How do you even build a representative sample of nations and the crises and changes that they've gone through?
How do you operationalise variables such as crisis and change ?
How do we hold some of variables constant and manipulate others to see the effect on the dependent variable, i.e. coping with crisis and change ?
How do we establish control conditions ?
It's very difficult to apply a reliable quantitative methodological framework to a quarter of a million years of human existence ? History relies on historical evidence, such evidence is subject to selection and interpretation biases that are far less prevalent and powerful in the Human and Natural Sciences. In many ways it is far more challenging to establish reliable historic knowledge than it is to establish reliable scientific knowledge.
Prof Diamond's solution to the challenge of reliability in history.
Prof Diamond's solution to the challenge of establishing reliable historical knowledge is that he chooses just seven countries, and looks at particular instances of crisis and change in those seven countries. He chooses countries that he's lived in and that he has a lot of experience with. He also speaks the language of most of those seven countries. As such, he's chosen countries that he has a deep knowledge of. Having deep knowledge of something, or accurate knowledge of something is having valid knowledge. So, in the prologue of this book, he's saying that he is establishing reliability through the use of validity. And that's really interesting for people when they're writing about AOK history.
So if you are discussing AOK history in your TOK exhibition or your TOK essay, probably more in the TOK essay this would apply to and you're wrangling with that idea of how historians establish reliability when they're discussing cause and effect. Well, here's a real life situation that you could cite. Jared Diamond's book, Upheaval How Nations cope with crisis and change.
Context relevant variable identification in historical knowledge.
Prof Diamond takes the instances of crisis and change in those seven countries and the looks at the vast range of variables which affect those crises and changes.
Some of those variables include pre-existing conditions, changes in the global conditions and global dynamics, changes in the geopolitics globally, the historical cultural antecedents, the historical cultural context, and the global historical context of those nations. It could also include the individual actors at their moments of crises and change, the aims of those actors etc. There are just so many variables involved and they're different for each nation and they're different in each instance of crisis and change in that nation. Prof Diamond puts all of those variables together to try to establish a high level of validity. Clearly the conclusions reached are from one commentator's perspective despite drawing upon a vast range of evidence to build this highly valid picture, As such, it is recognised that this highly valid picture may not be entirely valid, but it may be the best that we can can do given current methods and technology. But maybe it's only reliable for that nation and in that instance, maybe it is of limited generalisability.
I hope that that's given you some ideas if you're writing about AOK history and the challenge of establishing knowledge that is both valid and reliable in historiography. In a future blog we will explore the differences between a narrative historical explanation and a statistical methodological historical explanation (e.g economic history).
We have lots of resources to help ToK Students with the ToK Essay and ToK Exhibition. For example we have exemplar exhibitions, videos on how to how to do your TOK exhibition. For the ToK Essay we've got explanations of the essay prompts, a video series on how to do the TOK essay. And we've got lots of notes that can help you to avoid the biggest mistakes in the TOK essay. You can start exploring here.
You can get in touch with me, daniel@toktoday.com. I always welcome suggestions on how to improve the site, improve the resources and to produce new resources.
Daniel, Bangkok, September 2023
Can History be "true"?