Skip to content

We lack serious training in research methods choice and selection

As most of you who follow me on Twitter or read my blog probably know, I have been teaching research methods courses for a while now (mostly at the Masters and PhD levels). Thus I have been reflecting on issues related to training and teaching research methods. This morning, something really hit me as I was preparing an asynchronous lecture:

We lack serious training in research methods choice and selection.

We keep saying “choose the method according to the research question”. But this requires that the student KNOW multiple methods, for data collection AND data analysis. Moreover, developing criteria for method choice and selection requires students to learn heuristics that can help them build decision trees and models of the type
“IF research question 1
THEN (data collection method 1 AND data analysis 1)
ELSE search RM”

This is why research methods training is very hard, and why it’s imperative that whoever teaches methods goes over the details of multiple methodological and empirical strategies, using case studies: so students can see for themselves how researchers make methodological choices. I choose ethnography as a core methodological strategy because it helps me understand the lives of individuals facing water insecurity, toilet insecurity, and those who work in the informal waste sectors. I know how to use many other methods, but I make an actual choice/decision.

My research output in the past couple of months

One of the reasons why I assign multiple empirical analyses in my courses is that I want my students to learn HOW different researchers and research teams have made decisions on which methods for data collection, analysis and presentation they used. Which heuristics did they use? I was thinking of teaching an advanced course on ethnography, and realized that one of the ways in which I have learned how to improve my own ethnographic work is by reading book-length ethnographies. I have read many! I’ve learned about the decisions ethnographers made.

We need to change how we teach research methods. We really do. We need to seriously engage with the challenge of teaching method choice and selection. I believe one of the key gaps in methods training is method choice heuristics. How do I know how to choose a method to answer a specific question? How do I make a choice that gives me the answers I need? These are key questions that faculty teaching research methods need to teach students so they can, in turn, answer them.

Posted in academia, research, research methods, teaching.

2 comments

By Raul Pacheco-Vega – November 15, 2021

Should I specialize during my PhD or should I branch out?

Dr. Raul Pacheco-Vega teachingOne of the questions I get asked more frequently (by either my own students, and by others who reach out to me at workshops or via email) is whether they should specialize or not during their PhD, and sometimes earlier in their careers. As I’ve written elsewhere, I’m a very eclectic person. I work across multiple disciplines (political science, human geography, international relations, public policy, public administration).

I straddle several fields, from comparative public policy and policy transfer to social movements to water governance to waste and discards management. I am methodologically diverse and use multiple theories in my work and as such, I encourage my own students to explore across disciplines, theories, methods and approaches.

When I did my undergraduate in chemical engineering, I did not want one specialty, so I went for two: food science and engineering, and chemical plant design. I did the same for my PhD (human geography and political science), and even there, I did other things: international relations, public policy/administration (on the political science side) and economic/urban/environmental geography (on the human geography side).

Methodologically, I’m also quite diverse: I do mixed methods, but I have a very strong ethnography slant. I also do experiments (particularly field experiments), archival research, spatial analysis and social network analysis.

Which takes me back to the question itself.

I am not 100% sure if one should JUST pursue one line of inquiry or several. If one should just specialize in ONE area of scholarship or broaden the scope. For a lot of people in academia, it’s hard to fit me and my work into a mold, e.g. I’m not just a political scientist. I’m a public administration scholar to the PA/PM community. I’m a human geographer to the geography community. And I’ve even dipped my toes in economic anthropology and the sociology of consumption. That’s because my work on water insecurity and the informal mechanisms of waste governance is legible to, respectively, anthropology and sociology.

With my own undergraduate, Masters and PhD students, what I have done is that I’ve taught them multiple methods, but within a relatively well-bound area of scholarship (water conflicts, homelessness policy, care policy, older persons’ policy, food insecurity policy). Various methods/theories, ONE field.

I’m a senior professor now, so this “trying to fit me into a narrow box”, branding exercise is much less of a burden for my career, but I try to help my own students by helping them shape a trajectory that makes their work legible within a discipline & a field, while maintaining … a healthy interest in broadening their research interests down the road.

For example, two of my recent PhD graduates continue to do work on subnational water conflicts, but have moved away from the ones they did in their PhDs and started studying other cases. One of my Masters’ students is now moving on to doing a PhD and is thinking of ways to make his earlier research on homelessness broader, but still within the same topic (possibly using multiple methods, comparing different cases, etc.)

So, I don’t want to give advice, but if asked, my advice would be to seek advice from a broad range of scholars and look at what fits you best, particularly for whatever trajectory you want to pursue.

Batch Reading, Highlighting, Annotating, Scribbling

I don’t regret having broadened my work early in my career, but I am not sure whether it hindered or helped me progress. But again, I’m senior now and I have a permanent professorial job, and therefore I do not have to worry about this as much.

To summarize, only you know what you want to do, and the people who know you and whom you trust can give you sound(er) advice. Do whatever fits you best, though I always recommend, make your work legible to the communities you want to work with.

Posted in PhD training.

Tagged with .

No comments

By Raul Pacheco-Vega – November 7, 2021

How to write field notes (and how to teach the writing of fieldnotes)

Writing is hard. And writing field notes is hard, too. I don’t think that there is enough guidance on how to do it. I’ve written about the use of ethnographic fieldnotes in scholarly written output, but I don’t think I had ever written about how to write fieldnotes and how to teach or learn on your own how to write better field notes.

Working and writing everywhere

Given my extensive experience as an ethnographer and fieldworker, I felt that it was important that I wrote a Twitter thread (which I am now converting into a blog post) on how to write field notes, along with references and citations that might be useful for people to read.

Over the past three weeks, in my Qualitative Data Analysis and Interpretation class, we have covered Coding, Theming and Writing Field Notes and Analytic Memorandums. These are classic components of qualitative text analylsis, and I discussed them in great depth. I believe that’s part and parcel of QDA.

In my Qualitative Data Analysis and Interpretation class, I went over what I believe is the core of a field note: I use a notebook, and on one side, I write my observations of whatever I am studying, alongside with the date and time. I make detailed notes (also noting on the left hand side of the page). Usually, the left hand side of the field notebook pages serves me to note my own emotions, the context, etc. This is not an uncommon approach (left hand side “emotions/context”, right hand side “phenomenon we are observing”) but others may use a different approach to mine.

But in my view, that’s also part of why it’s so hard to teach how to write good field notes: many/most of us may not be willing to show anybody else our field notes (for privacy/sensitivity issues, but also because they are primarily for us). And yes, despite good books on field notes.

Now, here’s the thing: you can teach how to write field notes, and how to write GOOD field notes, but then there’s the question of how do you write good ETHNOGRAPHIES? As I have taught my Qualitative Data Analysis and Interpretation students, my own personal bias is that a field note and an analytic memo are DIFFERENT.

But to teach how to write good fieldnotes we also need to educate our students on how to write excellent ethnographic prose. For that particular purpose I have two suggestions:

1) Read A LOT OF ETHNOGRAPHIES, particularly book-length.

2) Read books on how to write ethnography.

Teaching Qualitative Data Analysis and Interpretation has already changed me and the term is not over yet. I have been able to spend a lot of time pondering and thinking about the craft of teaching how to do good qualitative research, and it has benefited me enormously.

So, my take home suggestions. If you want to learn how to write good field notes:

1) Read books on field note writing
2) Read books on writing ethnographies
3) Read well written ethnographies
4) Practice writing field notes
5) Practice writing analytic memorandums.

and…

6) Practice writing scholarly outputs.

Hopefully this blog post will be useful to readers interested in qualitative methods.

Posted in academia, fieldwork, qualitative methods, research methods, writing.

Tagged with , , .

1 comment

By Raul Pacheco-Vega – October 12, 2021

The Memo-Based Writing Strategy: Helping students write large-ish (>2,500 words) assignments

I found a request on Twitter by a professor on how to help students with “large N word” assignments (3,000 words, 5,000 words, a thesis). This is something I have thought about frequently because contrary to what people may think (with all the writing I do), I am actually not verbose. I hate having to comply with a certain number of words, because frequently I’ll find myself at odds with the target. How can I write that many words, if I am so concise in my writing? Therefore, I spent some time reflecting on this issue and pondering a potential strategy for this professor so she could use it in her teaching.

Along the same line of thinking. I recently taught for the second time my full-fledged, 40 hours course on “Strategies of Academic Writing and Empirical Research Inquiry”.

This time, I reversed my pedagogical approach.

Highlighting, scribbling marginalia, reading, writing

Before, I used to teach my students how to ask a good research question, how to narrow from a broad topic to a simple, researchable question.

This is a wonderful strategy IF THE STUDENTS HAVE ALREADY READ A LOT AND HAVE MASTERED THE LITERATURE.

You see the problem here, right?

This strategy works well either teaching They Say/I Say (Graf & Birkenstein) style of argumentative writing first and THEN a repertoire of reading strategies (starting with my trusty AIC method) or teaching argumentation in tandem/parallel with reading strategies.

The Memo-Based Writing Strategy is incredibly helpful for people like me, who are not verbose, and who struggle to PRODUCE words. If you are rather verbose and have the lucky ability to just vomit words, more power to you. I am not able to do that.

Focusing on memos HELPS.

I helped my students decompose THEIR ENTIRE DISSERTATION in memorandums, and put them into a Gantt chart. Once we went through the exercise, they realized that writing a thesis is not as major as one would think.

Think about getting ONE memo, 75-125 words, 15 minutes DONE.

memo review of the literature

A brief memorandum reviewing the literature on homelessness.

If your students are worried about a 3,000 words assignment, perhaps the best way to think about it is a 2 week (10 days worth of work) assignment with 150 words per day. This could possibly be manageable.

That’s how I deal with my own writing: one memo at a time.

Posted in academia, writing.

Tagged with , , , , , .

1 comment

By Raul Pacheco-Vega – August 14, 2021

A proposed heuristic to choose which note-taking technique we should use: Index Cards, Cornell Notes, Everything Notebook and Conceptual Synthesis Excel Dump (CSED) rows/Synthetic Notes

As most people who read my blog and have ever taken a course from me or attended one of my workshops (or even follow me on Twitter), I really love writing blog posts and Twitter threads that will help them in the future adopt my techniques. As most of the people who have read my blog or attended my workshops or taken my courses have at some point landed on my “Note-Taking Techniques” page (under the Resources tab), they always wonder when do I choose each type of medium to take notes on.

Reading highlighting scribbling annotating

As someone who is VERY analog, I usually take notes by hand on physical media (Everything Notebook, Cornell Notes, Index Cards). I do, however, understand why several people might be interested in knowing how to convert my methods to the digital realm, and/or use a hybrid mixture of digital and analog, as I do when I write Conceptual Synthesis Excel Dump (CSED) rows, and/or Synthetic Notes.

Reading highlighting scribbling annotating

I wrote a Twitter thread explaining my method, and because Twitter is somewhat ephemeral, I thought it would be better if I stored it in a blog post.

Because I’ve chosen this as an article I’m going to read in much depth, I need to first decide how much time I’m going to spend on it and how I can systematize what I am going to learn from it.

First step: go through the headings and list them. This exercise provides me with the paper structure and helps me think through which order and categories these writers used. I normally do this on a separate notebook or a piece of paper, or a block of detachable pages, as I did in this case.

The choice to write a Cornell Note comes from knowing the authors AND the topic. This paper was written by early career scholars (ECRs), whom I strongly wanted to support, and it has a topic (fieldwork) and a focus (teaching guide to train graduate students) that I definitely want to engage with very deeply.

Note how I have added a quotation (with corresponding page) to my Cornell Note. Also note how the authors identify 3 areas of current disagreement about fieldwork – purpose, where it occurs and how long it should be, following them with their own definition. Irgil et al use each one of these areas of disagreement on what fieldwork entails as both signposts AND Topic Sentences. In three consecutive paragraphs, they pack A LOT of good ideas on why political scientists disagree on what field research entails.

This means that I will neex to write Index Cards: I want to make sure to be able to use those Index Cards in my teaching and in writing the pieces I am preparing on fieldwork.

So what happens to those “extra” marginalia scribbles? I could do several things:

1) Type them into Mendeley (you can highlight and add marginalia to PDFs).
2) Transcribe them into the “Notes” section of the respective CSED row.
3) Copy them on to the back of this Index Card.
4) Copy them on to my Everything Notebook (I’ll probably have an entry for this particular article, where I can add these notes to myself).
5) Copy them on to the Cornell Note I already created for this article.

So many options! Decisions, decisions!

In the end, I believe…

What about the Conceptual Synthesis Excel Dump (CSED) row entry and the Synthetic Notes?

One question that I often get asked that I did not originally discuss in my Twitter thread is the combination of analog and digital in my reading, annotating, systematizing and absorbing (RASA) conceptual model of my workflow. As my blog post linked before indicates, from an AIC Content Extraction, I drop the contents of my notes on to a Conceptual Synthesis Excel Dump (CSED) row. If I feel like I may use these notes relatively soon into the future, I’ll write a Synthetic Note, which then can be converted into a Memorandum.

Posted in academia, research, writing.

Tagged with , , .

3 comments

By Raul Pacheco-Vega – July 16, 2021

Lessons learned from designing a qualitative data analysis and interpretation graduate-level course, from scratch

I have been teaching graduate-level courses for a very long time now, but my methods teaching has increased several times over because I am now associated with FLACSO’s Methods Lab, so my teaching load is much more focused on methodological pedagogy. Over the past year I’ve taught Comparative Methods, Qualitative Methods, Research Design, Research Methodology, Thesis Seminar, Mixed Methods. This Spring, I was asked by my students and colleagues to design a “Qualitative Data Analysis and Interpretation” course for Masters and Doctoral candidates and potentially faculty who might be interested in the topic.

Preparing a course using the Everything Notebook and handwritten notes

In this post, I am going to share with you a few insights I learned from designing (from scratch) a qualitative methods course. Disclosure: I was the Editor for the Americas of the International Journal of Qualitative Methods. I apply qualitative methods in my research (though I am a mixed methods person and feel equally comfortable with quantitative methods). I teach qualitative methods. I write and publish methods pieces.

All this is to say: I’m most definitely not a newbie to the field/to the methods. Nevertheless, despite NOT being a newcomer, I invested a lot of time and money preparing to design this course.

Preparing syllabi is HARD and takes a lot of time and effort.

In order for me to properly design a “Qualitative Data Analysis and Interpretation” Masters/PhD level course, I:
– went and read (and in some cases, re-read) 31 books.
– emailed two professor friends who shared their syllabi with me.
– downloaded and read 15 different syllabi.

I invested more than 60 hours of my life JUST thinking through which readings and in which order, and which activities I was going to implement. Obviously, it also took me a few hours (4-6) to write the actual syllabus.

What did I learn, after all this?

Well, this will be no news to qualitative researchers, but IT’S FREAKING HARD TO DESIGN A QUALITATIVE METHODS SYLLABUS. There are so many different ways in which I could slice and dice this course. I could craft a 3 semesters sequence (basic, intermediate, advanced).

I could teach a 3-semesters-long sequence on ethnography alone. And that would *just* scratch the surface. I have considered teaching “Foundations of Ethnography”, “Intermediate Ethnography” and “Advanced Topics in Ethnography” and I still have a lot more books to assign than I could cover in any course.

I’ve read 32 books JUST on fieldwork, JUST to think through the mere idea of how to teach students how to enter the field, how to collect data, how to protect themselves.

My point: EVERY course on qualitative methods is bound to miss stuff. You can teach “the mere basics”, a survey, that then gives students the opportunity to develop a specific analytic technique, epistemological and ontological approach. But no course is going to cover EVERYTHING.

In my view, it’s a good idea to teach AT LEAST ONE COURSE in qualitative methods, but we need to be keenly aware and cognizant that there will be gaps that we haven’t filled and that there are new developments that won’t show up in our syllabi on time.

That’s ok too.

Whatever you teach, make sure you include rigorous and well-crafted exemplars that your students and you can deconstruct in class and learn from. That’s one key lesson that I have learned after teaching undergraduate, Masters and PhD-level courses.

Posted in academia, qualitative methods, teaching.

Tagged with , , , , , .

No comments

By Raul Pacheco-Vega – July 4, 2021

A Scholar’s Guide to Getting Published in English: Critical Choices and Practical Strategies (my reading notes)

A good friend of mine (Dr. Myriam Houssay-Holzschuch, Université Grenoble Alpes) recommended two books and my website for anyone starting graduate school (Dr. Houssay-Holzschuch is a human geographer). Since I’m half human geographer half political scientist, I immediately decided to follow her advice and buy the two books she recommended. One of them was “A Scholar’s Guide to Getting Published in English: Critical Choices and Practical Strategies” by Mary Jane Curry and Theresa Lillis. Actually, Dr. Houssay-Holzschuch recommended this book in another set of tweets that I can’t find right now. At any rate, these two tweets summarize my very positive evaluation of this book.

I have to say that one of the key reasons why this book is so appealing, in addition to the ones I mention above, is that it is published by a small, independent publisher. This alone made me excited about promoting the book.

DISCLOSURE: I bought this book with my own hard earned money. I get no royalties, kickbacks or benefits from writing about it. I do think it’s an excellent book.

Posted in academia.

Tagged with , , , , , .

No comments

By Raul Pacheco-Vega – June 20, 2021

On multiple academic projects’ management, time management and the realities of what we think we can accomplish in a certain period of time versus the realities of what we actually are able to.

This blog post has a terribly long blog post, but I think it’s worth including it in its entirety. I recently taught two workshops where I was asked about calendar management.

I now have 5 core things in my calendar:
– my own writing time block (4:30am-6:30am)
– the writing time I spend with Amanda Bittner and the feminist mafia posse
– the writing time I spend with Jeff Guhin, Mirya Holman and the #FinishingOurBooks crowd
– my teaching commitments (I teach on Thursdays)
– meetings (with students, colleagues, etc.)

But, like everything, my life changed, so did my priorities, and so did my workload. While I still had a relatively freeing approach to working, where I Worked on One Paper Every Day (a different one, particularly with coauthors), WOPED ALSO stopped working at some point.

I noticed that switching up tasks, projects and research streams was making it incredibly hard to concentrate.

Yes, I work on a very broad range of areas: I study bottled water, transnational environmental activism, homelessness, policy transfer, research methods, etc. Switching bodies of literature, empirical research methods and fields makes it really hard to regain momentum. For example, right now I’m working on revisions to my book on bottled water. BUT I also want to read up on the politics of public health policy. AND on homelessness.

WHAT IS WORKING FOR ME NOW?

What’s working for me now (as of 2021, pandemic and all), is to block time to work on a specific project (my bottled water book, or R&Rs, etc.), and when I do, I JUST think about it and put other wonderful ideas I have or readings I want to do on the shelf for a while.

I have full control of my calendar, though, and I am grateful for this. I can say “I prefer having meetings in the afternoon because I write in the mornings”, and the work-from-home approach has made it easier for me to schedule naps (which I require to function) and flex time.

My meeting times have shortened (though I have noticed, interacting with Mexican colleagues across other institutions, that for many, their own bad meeting habits have remained throughout the pandemic – making meetings longer than they should be).

Zoom tires me, though.

3) In your calendar, block both the academic/scholarly/work stuff AND the personal stuff. The latter ALSO requires A LOT of time.

4) Allow yourself flexibility, insert buffers in your calendar, don’t over-schedule every single thing, note down the major commitments you have.

5) Extend yourself AND others grace. We are all struggling through the pandemic.

6) To the extent possible, try to work on projects of the same Research Stream/topic.

7) Remember: YOU DO YOU. Everyone is different, and we need to shun the neoliberal idea of “productivity”.

I really do hope this blog post will be helpful to people. Also search my blog for a series I did on Project Management for Academics.

Posted in academia.

1 comment

By Raul Pacheco-Vega – June 17, 2021

A sequential strategy for teaching how to write a literature review

I think one of the drawbacks of knowing my blog so well (as its writer!) is that I don’t reflect often enough on the various sequences of blog posts that others (particularly faculty, post-doctoral researchers and early career scholars) can use to teach my techniques. When I get hired to teach a workshop, I pick and choose from my website and produce handouts that include links to specific blog posts, so that I can then help people learn specific strategies, techniques and heuristics. But this morning, a request for help really made me think deeply about how I DO teach my students, thesis advisees and research assistants how to produce literature reviews. My approach is sequential and scaffolded, thus the title of this blog post.

Reading highlighting scribbling annotating

My sequential strategy can be summarized as follows: I teach my students how to quickly read and absorb material using the AIC method and my template, then how to systematize their readings’ notes using the Conceptual Synthesis Excel Dump (CSED) method, and then how to build a scholarly output. I teach how to produce three levels of scholarly products (classified by degree of complexity): banks of synthetic notes, rhetorical precis and memorandums, annotated bibliographies and literature reviews, as I’ve previously outlined. The Twitter thread below outlines my approach.

The process (when I train RAs and students) then looks like this:

Once they’ve started gaining competency in AIC+CSED (which I encourage by telling them to survey the literature and JUST focus on filling up rows of CSED – drawing from their AIC skim), I teach them how to do an Annotated Bibliography. THEN we move on to the Literature Review.

I want to acknowledge Dr. Ali Fitzgibbon for reminding me of the importance of the double-round mind-map. It wasn’t until this morning when she articulated the idea of the double mind map in a tweet that it really cemented the process I follow and how I, too, do two mind maps of the literature (pre- and post-). And I also want to acknowledge Dr. Christina Ballico for reminding me (and others) that annotated bibliographies can be an intermediate step towards a literature review.

Reviewing the literature and mapping scholarship

Posted in academia.

No comments

By Raul Pacheco-Vega – June 15, 2021

Time, energy and health: Three considerations for academic commitment and project planning

I’ve been working through some ideas on planning and project management for academics, as I’ve been asked to deliver more project management and planning workshops for academics over the past few months. Thinking deeply about these things allows me to consider how different elements play out in how we manage our academic commitments and how we do project planning. This post outlines my reflections.

As someone who suffered through chronic pain, chronic fatigue, psoriasis-eczema-dermatitis, I have become keenly aware of 3 things that we believe are in infinite supply: Energy, Time and Health.

Everything Notebook 2021

I’ve said a lot of “yes” to things that I believe I will be healthy for, I will have the time and energy to do them. Well, guess what? When I travel, I get super tired now. I’ve had to travel three times over the past 3 weeks. The result: I’m exhausted and behind on everything.

These feelings of exhaustion and overwhelm led me to engage in self-reflection. I said “NO” to bidding on a project where I’m basically the most qualified scholar to do the project. I said “NO” to teaching a class I’ve perfected over the years, one where I already have the slides and reading packet prepared. Why? Because I have limited time and energy.

I’m literally barely getting through the massive number of “yes” I said (when I had the energy, health and time to do them), and feeling anxious about this made me realize that one of the best ways to teach project management is to showcase when something didn’t go well.

Generally speaking I’m almost always in control of my agenda and my projects. It’s when I say “yes” to something without consulting first with my overall global planning scheme that I screw up.

Even my mother noticed: “but… you’re always so organized!”

Well, yes Mom except when I let something slide.

What I tell everyone who attends my workshops on planning and project management: The time you spend planning is super important and will prove invaluable when you’re confronted with feelings of overwhelm.

Now if I could synthesize my learning, I would say, from my experience:

  • say “yes” to projects that you’re 100% certain you have the time, energy and health to do.
  • frequently and systematically monitor yourself so you can detect when you no longer have time, energy or health to do something you committed to.
  • generally speaking, assume EVERYTHING will take 2-3 times longer than you originally planned.
  • – always (if you can, of course) insert “buffers” onto your calendar — blocks of time that will allow you to rest in between activities.
  • saying “NO” is healthy and ok.
  • And perhaps the most useful lesson I have learned: saying “I am no longer able to do X, even though I had originally committed to do it. Sorry.” is ok. Obviously it helps if you help find a replacement, but it’s also healthy to admit you have ran out of time and energy.

I really hope these reflections are useful to others in planning their workload, particularly under these complex situations (COVID19, work-from-home, etc.)

Posted in academia, planning.

Tagged with , , , .

No comments

By Raul Pacheco-Vega – June 3, 2021