Qualitative Literacy: A Guide to Evaluating Ethnographic and Interview Research (my reading notes)

I had been waiting to get my hands on this book by Mario Luis Small and
Jessica Calarco for a very long while, as I was on Twitter while Jess was writing it and witnessed the conversations on here about it. I finally received my copy this week, so I decided to write a few reading notes about the book. There are several elements I like about Qualitative Literacy: A Guide to Evaluating Ethnographic and Interview Research and obviously (I wouldn’t be me if I didn’t), a number of quibbles. Overall, I like the book and I appreciate the criteria that Small and Calarco set forth to consider when examining research projects where the data acquisition methods are ethnographic fieldwork and interviews. I particularly liked their honesty in explaining exactly what their book is and isn’t.

Evaluation is complicated because it requires setting specific standards and comparing against those standards and verifying whether the evaluated item complies (or not) with those standards. Evaluating qualitative research is very complicated, for many reasons, as my notes here will show. it is always hard to write about evaluation of qualitative research, precisely because of the complexity of developing and setting those standards and then undertaking the empirical work to evaluate said standards (yes, it feels like evaluation-inception, but so it is!).

What I like most about Small and Calarco’s book is that they don’t actually set standards against which to compare and tick boxes or determine compliance ranges through some sort of Likert scale. Their work develops 5 characteristics of “well-done” qualitative research. I used quotation marks for a very specific reason. Small and Calarco do NOT say that the characteristics they use to evaluate ethnographic and interview research are the only ones. So, they are also NOT implying that there are specific standards for solid research. But they do mply that, on average (and on the whole), most research that we (the community of scholars and practitioners who use and consume QR, collectively) would be able to see as “good” qualitative research, share these characteristics. I commend Small and Calarco for this honesty.

Small and Calarco are also VERY clear that their book is on criteria for users and consumers of qualitative research that they can use to conduct an assessment of TWO qualitative DATA ACQUISITION PROCESSES. Evaluating qualitative analyses, qualitative research designs and a whole other set of elements of QR would take, as they aptly say, a whole lot of additional books.

The way I see Small and Calarco’s “Qualitative Literacy” is very much the way I see Elinor Ostrom’s 8 Principles (which are very widely misunderstood, and I have to frequently clarify their use and meaning): as observed empirical regularities of systems/processes. Small and Calarco say (not an exact quotation, but my interpretation of their book) that “these are the characteristics that various types of ethnographic and interview research projects/processes share. We believe these, on average/on the whole, represent solid examples of well-conducted data acquisition processes”.

Evaluating qualitative research as a whole (from research design to implementation to analysis to write-up) is always complicated, so I really appreciate and value Small and Calarco’s contribution to building this knowledge base for additional, future projects. Overall, I liked Small and Calarco’s “Qualitative Literacy”, as someone who teaches, supervises, conducts and evaluates qualitative research projects.

10/10 recommend.

You can share this blog post on the following social networks by clicking on their icon.