Everything you need to know about qualitative research (but were afraid to ask)
The story so far: Big Birthas Parenting Science Gang are interested in the experiences that larger mums have during labour, specifically the choices that they are offered or not offered. But how to research this?
Enter Dr Michael Larkin to answer all our questions about qualitative research.
Photo by Allen Cai on Unsplash
Michael: Hi all. Good to [virtually] meet you!
I’m a researcher based at Aston University. All of my work is qualitative. Much of it – but not all of it – is focused upon understanding how people make sense of their experiences of difficulties.
Would it help to throw out a few things that you’re curious about, to get us started?
Quantitative and Qualitative Research (in which we talk about football and car accidents and learn what epistemology means)
Dragon: For those of us who are totally new to this field, could you give us the most basic definition of qualitative rather than quantitative research?
Michael: Sure. I’ll do that in two parts – practical, and then conceptual.
So – practical: in quantitative work, we’re interested in measurement. Often we measure differences or relationships between groups (or events), and usually we’re testing a hypothesis. In quantitative work, we’re exploring meaning – how people make sense of something, what something is like, or how it is understood.
If you can tolerate a football example right now, Ian Dey gives a good example ..
Mermaid: I’m going to need more of an explanation if we’re involving football! ???
Michael: If you are a *bit* interested in a game (say you’re an England fan, and you want to know about who England might play in the next round – if they get far), then a quantitative assessment of the Sweden – Switzerland game is fine. You just need to know the score.
But if you’re a *lot* interested (say you’re a Switzerland fan). They lost, but you might still care about qualitative data – how well they play? Was the result fair? How did the fans behave? Etc.
So: same event – two different kinds of information we could be interested in.
Dragon: Thats a really good way of explaining it!
Our group of citizen scientist parents are interested in exploring the experiences that obese women have of making choices during pregnancy. Anecdotally we are aware that many women feel that their choices are limited by health care professionals, and sometimes these restrictions are not entirely backed up by evidence.
We are interested in how the experience of having choices limited affects women – how they feel about it at the time and afterwards.
We don’t just want to know what choices they have (I guess that might be a quantitative approach), we want to know how important and relevant those experiences are.
Mermaid: Would it be fair to say objective and subjective?
Michael: That’s a really good link to the second part of my answer about conceptual differences!
Michael: So – conceptual differences … we have to talk about epistemology to draw that out
Mermaid: Epistem-what?
Unicorn: We talked about this before and decided on a phenomenological approach – the study of the meaning behind people’s experiences 🙂
Michael: Great – that’s helpful!
Griffin: Another bless your here’s a hanky moment!
Michael: The underlying *epistemology* really of qualitative research is important (I’ll define it in a sec!). It shapes the decisions you make about design, data collection, analysis, etc, and the way you frame your inferences and discussion of what you find.
‘Epistemology’ means … your theory about ‘what you can know about the thing that you’re interested in.’ Experiments and survey methods usually have an empiricist epistemology.
This might not have been made explicit to you, but empiricist epistemology has a few key assumptions:
e.g. in empiricist research …
- You can learn about the nature and causes of events by making structured and controlled observations of those events
- You design studies to identify patterns of events that co-occur together (‘empirical regularities’)
- You aim to be objective; to sample representative populations; to produce knowledge that is reliable and replicable …
And qual work doesn’t sign up to those principles.
Instead, qualitative methods are [broadly speaking*] interpretivist. They are not focused on the causes of ‘what happens’ and more interested in the meaning of ‘what happens.’
So phenomenology is a good example of that.
[Mermaid: Loving how fast you type, BTW
Michael: Too much of my life spent at keyboard!]
Michael: I often use the example of car accidents
If you wanted to understand psychological factors contributing to car accidents, there are lots of good studies you could design which would help to you understand the relative effects of various phenomena on people’s ability to drive a car (fatigue; over-confidence; intoxication; perceptual errors; auditory distraction etc). But these are not qualitative studies.
By contrast, if you wanted to understand …
- The ways in which people made sense of the psychological impact of being in a car accident
- The ways in which people apportion blame and responsibility for car accidents
- The ways in which people identify what has helped and hindered them to return to safe driving after a car accident;
then a qualitative study might work well.
Note that the key difference here is a focus on perspective (it’s about what we can learn from people with a particular point of view) and meaning (it’s about how something is made sense of – or interpreted – by those people.
Wood nymph: So the how people feel about what happens is more important than the what. So some women might be thrilled with the strong recommendation to stroke a unicorn which in labour and others might be wildly against it.
Michael: Well, more important in terms of what you’re interested in, in a phenomenological study.
Sphinx: Can you use a survey with tick boxes (easy for people to fill in) to do qualitative research or does that introduce too much of the bias of the researcher?
Michael: No, because that means you’re measuring something. But there are some ways of simplifying the data collection process. Let’s come back to that in a moment
Mermaid: Would it be fair to say that if we’re looking to get any form of media interest, that we could do with at least some quantative data ‘x% of women wanted to stroke a unicorn while in labour, but only y% were afforded the opportunity’ but the qualitative stuff is more the meat on the bones of the issue?
Griffin: And importantly z, how many even knew the unicorn was there
Michael: That’s complicated. Qualitative data can have a huge impact on policy makers, service providers etc.
But for media, can be harder to get the nuances across.
I’m trying not to get sucked into the unicorn example. We’re still grieving for the tooth fairy here.
Mermaid: Sorry, we got a bit obsessed with unicorns lately here (so we could discuss the topic without getting into the specifics)
Michael: OK. How are we doing on differences between qual and quant, so far? Anything that needs clarifying?
Griffin: More just the how you do that bit next!
Dragon: Marvellously thank you! Am I right in roughly translating phenomenological as roughly “about people’s person experience and how it affects them”.
Michael: Yes. Even more precisely: what are the things that make up a person’s world, and what meaning do those things have for the person.
And you can narrow the scope. So it might be ‘the things to do with the experience of giving birth’ for example.
Wood nymph: If I can put it into our context a quantitative study would be I could stroke a unicorn/listen to mermaid song/be massaged by fairies and we get statistics, and qualitative is experiences around stroking a unicorn/mermaid song/fairies?
So are we better focusing in on unicorns specifically or exploring how people felt about being given choices?
The research question (in which we discuss surveys)
Michael: So do you have a research question that you’re happy with?
Mermaid: We need help with the questions to be honest
Michael: Ok, let’s move to that now
Mermaid: We want to know about birth choices. We’ve got 9 days to get it in front of the ethics committee… Tick tock!
Mermaid: This is what (with zero knowledge or experience of research) I brainstormed yesterday [this is just a few of the questions suggested]:
- Did you want access to (multiple choice tick box):
- MLU
- Home Birth
- Immersion in water for labour (lots of others etc etc)
- (Could scale all the above for importance, either via ranking or 1>>>10)
- Did you discuss these plans with HCPs?
- Did you get access to the things you wanted?
- If not, do you know why you did you not have access to them? (Free text line for each selected option above)
- On a scale of 1>>>10 how much do you feel your wishes and choices were important and respected?
- How did this make you feel at the time and afterwards? (Free text)
Michael: That’s a questionnaire. Interesting ideas in there, but not a qualitative design.
Mermaid: Even with the open text – how did this make you feel etc? Do we have to be purely one or the other, then? Is there no way to combine?
Griffin: Can a one to ten scale answer be useful for qualitative questions but give quantitative answers? Or does that bias too much?
Michael: If you’re using a scale, you’re measuring. That’s quantitative.
Griffin: I was trying to be clever by asking questions about quality of experience but it’s the outcome of the question not the subject isn’t it.. Okay thanks
Dragon: I think we are having trouble defining our research question due to our lack of understanding of qualitative methods, but I think that it is roughly:
How does the experience of having limited choices during labour affect mothers with a high BMI?
Michael: OK. Let’s take a minute to look at that, and get it fine-tuned.
There’s an empirical assumption in there – how does x affect y? – which most qualitative approaches won’t tackle.
Sometimes the solution to that sort of problem is to think about bringing the perspectival quality of your interest into the research question.
For example: How do mothers with high BMI make sense of the effects of their limited choices upon their experience of labour?
Dragon: “make sense” = understand?
Michael: Pretty much
You’d hope to capture it by collecting their own rich descriptions of their experience, and then, analysing what they say, and presenting those side-by-side.
Mermaid: I think I can answer that quite succinctly – it pissed me off and I had a home birth second time around so I could do what I liked ???
Michael: that’s a concise example!
Mermaid: Mine’s an extreme example, though.
What about the people who it upsets, but they don’t have the confidence to challenge HCPs and do their own thing – how do we capture the extreme distress and sense of frustration when many women are just putting up with it, and feeling bad about it because they feel ‘it’s their own fault for being fat in the first place’.
Michael: The effect is still there – but it’s no longer a causal chain. It’s interpretative.
Dragon: But do people understand how they make sense of situations? Do they not think more in terms of causal chains – I experienced this so I did that?
Michael: That’s a good question. In your data collection, you’d usually plan to elicit description, and emotional/evaluative aspects of that, rather than rationalisation/theorisation.
Michael: So: a) understanding doesn’t carry the connotation of adequacy that we might have when we use it in other contexts (‘does he understand it?’). It just means ‘how does he interpret it?’ or ‘what’s important to him about?
And b:/ Interpretation is partly done by the participants, and partly done by the researchers.
Collecting Data (in which we talk about interview, emails and corpus linguistics)
Michael: There was a question earlier and I said ‘let’s think about how you could collect your data reasonably easily’ or something
If you were pursuing the question above, then your participants are (conveniently) described by your research question: women with high BMI who have (recently?) given birth.
Mermaid: This sounds like we’d need skilled researchers to interview the participants, not just send out a survey (which was what we were hoping to do)
Michael: Face-to-face interviews are a popular method. They do need lots of time and attention to transcribe, however. It helps to have had some chance to learn how to do them too. You might be able to do something slightly quicker [less transcription time] and easier [more thinking time].
Mermaid: I suppose we could try Skype…
Michael: Skype still needs transcribing. And still unfolds in real time, putting similar demands on your confidence with interview skills.
Transcription of one hour of audio interview will take 6-8 hours of transcriber time. Big commitment!
Mermaid: I suppose I was just thinking logistically – our likely participants are probably not going to be local.
Centaur: I do think transcription is going to be a big issue.
Michael: If you design your data collection plan in a way which doesn’t rely on detailed transcription. Things like … analysis of email interviews; analysis of blogs and internet fora; analysis of diaries, or of story completion tasks … can all reduce the amount of time needed for transcription, which means you can get started sooner on analysis.
Unicorn: Some of us have experience with using Qualtrix/LimeSurvey etc within our institutions to develop qualitative surveys that focus on experience, geared towards discourse or thematic analysis… we were thinking that this might a sensible route forward given the time constraints. Do you have any thoughts? 🙂
Michael: *If* you can create a scenario where people will write at length, and online tool can work. A PhD student I supervised used one of these to get people to write stories for her (and it worked well).
Unicorn: I had success with an online platform whilst investigating experiences of support around infant feeding choices, so I was thinking something similar would be possible here. Hopefully it’ll be as successful as your student’s! 🙂
Wood nymph: We have discussed using corpus linguistics from free text input.
Michael: That can be a way of dealing with very large amounts of qualitative data, largely by quantifying it.
Dragon: Ah – so not really qualitative research!
Michael: Stretching it!
Unicorn: The problem with CL is that we would need an enormous quantity of data, a corpus of more than 100,000 points. I don’t think it’s a good route moving forward.
Michael: Usually prioritises frequency rather than context/meaning
Unicorn: Absolutely, and we were trying to understand the *experience* rather than the mechanics. Which is a shame in some ways, because I much prefer statistical analysis 😛
Michael: Can be very powerful if you have the right corpus, and the right question. Not sure it suits your aims here.
Wood nymph: Or at least by streamlining to elicit key words for discourse analysis. But we can’t seem to decide whether we want lots of experiences and lots of data or to really dig into a fee experiences.
Michael: In psychology, discourse analysis is different again. It’s done with small amounts of very detailed data. It doesn’t focus on experience. It focuses on the implicit rules which shape how things *must* be understood.
Mermaid: *hides under a table until you guys have worked this out*
Wood nymph: Afraid I’m in a linguist not a psychologist 😉 textual and interpersonal function is more my bag than the scary inner workings of someone’s brain!
[ Pegasus: Sorry guys… toddler only just gone to bed. Will have a read and catch up.
Sphinx: Evening 🙂 (My kids aren’t sleeping well either, I blame the heat!)
Dragon: We’re learning some serious stuff!!! I may be able to almost understand what you and Caytti and France are saying soon!
Griffin: I’m just going to wait to be to told what to do 😀
Mermaid: Griffin: – want to join me in my blanket fort?
Pegasus: ? as I said last night I hate all the epistemological stuff… I understand it, but I prefer to think of myself as a pick and mix researcher… use whatever tool suits your needs and the research question. I think that possible makes me a pragmatist in the research world ?♀️
Griffin: I’ll bring the ice cream, you bring the biscuits to dip
The rest of you, I am jealous 😀 ]
Analysing Data (in which we are introduced to Template Analysis)
Mermaid: We do need ways to keep the time frame short… And not too labour intensive… And suitable for non-research bods like me to be involved
Michael: For a team project, I would suggest Template Analysis.
It’s quite amenable to novice researchers, and it works well (e.g. if you have Dropbox folder that auto-updates) with multiple working on the analysis.
Mermaid: Is it possible to draw any conclusions from qualitative analysis though that could affect real changes in future treatment? It sounds all a bit imprecise, too reliant on us drawing conclusions from what people have chosen to tell us, and too nebulous?
Or am I too used to quantitative data – numbers and percentages giving me confidence?
Michael: They’re different kinds of conclusions.
In intervention development, we look at acceptability as well as efficacy. No good having an effective intervention which is not acceptable to people.
Acceptability is studied with qualitative data. E.g. see MRC Guidelines on Developing Complex Interventions.
Another example: there is NICE guidance on Experiences of Inpatient Psychiatric Care which is drawn almost entirely from qualitative data
Mermaid: That sounds like it could be an interesting read to try to get my head around it
Unicorn: I actually work with PHE, NHSE and the CCGs around the country to shape patient experiences on the basis of pilot groups and research etc with small numbers of people; so yes it’s very much possible! The RightCare Pathways are a good example of this 🙂
Mermaid: Sounds good to me. To be honest I don’t care how we do this, just that it has an impact.
If all we’re going to do is collect birth horror stories (interesting though they are!) that no-one will pay attention to then I can’t see the point.
But if we can make people listen that current treatment is unfair and not evidence based, and needs to change, that’s the holy grail.
My experience thus far, 6 years or so into campaigning about this in my own little way is that currently the buck stops with ‘mother and baby are both well, that’s the end goal, people’s experience is secondary importance’
Wood nymph: Is it crazy to deviate and use your story as inspiration, Mermaid? To focus on multips to find out how their experiences in a first high BMI pregnancy affected choices in their second?
Mermaid: We could. I certainly don’t mind. But not everyone goes rogue, has a home birth against medical advice, sets up a campaign/blog website and goes on telly about it, though ???
Wood nymph: But I’m guessing a fair few have the guts to do things differently.
Mermaid: Yes, I think the tide is definitely turning that way – I get a few emails a year to that effect…
Wood nymph: Is that potentially the start of recruitment for participants if we went that way?
Mermaid: Nope, because I never kept their emails for long due to data protection! But I could put out a call to arms at least, some may still be subscribed.
Dragon: Michael, could you give us a very brief layperson’s overview of template analysis
Michael: OK. Most qual approaches are ‘bottom-up’ [inductive]. They start with coding raw data. Patterns in the codes are brought together to make … themes, narratives, repertoires (what you call the patterns varies from method-to-method]
Differences between methods are largely epistemological: each offers a distinct version of the interpretivist lens.
Template analysis is a combination of bottom-up and top-down.
Initial codes are used to create a template. The template is basically a set of headings in a document, with relevant data organised under the different headings.
The switch to top-down occurs when you begin using the template to analyse the data, rather than using the data to create the template. So early stage coding is quite time-consuming (as in all qual analysis), but later stage analysis is quite quick.
It’s like slotting data into a structure which has been developed from the data.
If you have these documents on a shared drive, it can be quite easy for multiple people to work on.
Themes can then be drawn from these fairly descriptive documents, and you can be more interpretative if you feel confident at this stage.
Dragon: So we start by reading stuff and noting down themes we see, and then as we go along, stop adding themes, just add evidence from the narratives to support each theme. Is that the right sort of idea?
Unicorn: This is sounding like the best approach for our specific group so far!
Mermaid: That kind of makes sense.
Unicorn: Sounds fairly similar to content analysis… Will explore further!
Michael: Yes, but less interested in frequency.
Pegasus: It sounds like framework analysis to me which we use quite a bit in our work… haven’t used it myself in ages but it is fairly simple to do once the framework is agreed.
Michael: Very similar. Framework would be other option!
Pegasus: I’ll send myself an email reminding myself to photocopy and upload stuff on framework analysis.
Dragon: Thank you Pegasus! And anything that compares to template analysis -it’d be good to have a better understanding of the pros and cons of each.
Pegasus: If you’re lucky I’ll have training notes on how to do framework analysis buried in a cupboard somewhere!
How could we do Template Analysis? (in which we run out of time and ask the big questions quickly)
Wood nymph: How much data do we want? How long will it take to collect? How long will it take to analyse?
Michael: OK quick answers to those (very good) questions.
Michael: How much data do you want? It’s good that you’re asking how much data and not how many participants!
Mermaid: So glad we’re doing something right!
Michael: Think about how to get data which contains rich descriptions of people’s experiences. Imagine that they are typed up, double-spaced, with wide margins. 30-60 pages of data would make a good MSc project, a viable PhD chapter, or a publishable study, if the sample was coherent (i.e matched the research question) and the analysis was plausible and clear.
Dragon: But I’m guessing that if 60 pages were 3 people each writing 20 pages, that wouldn’t be great. Is there a minimum number of distinct voices that is plausible?
Michael: There are methods like IPA (and to some extent Template Analysis) which make a virtue of an idiographic focus – saying not ‘this is how things are’ but inviting reader to stand in a very particular position and see how things look from there. Small samples can work. Wouldn’t go less than 3, and my personal preference would be more like 5-6 individual voices as a minimum
Michael: How long will it take to collect? Depends on the plan. Go for something which is either a: already there (blog posts) or b: easy and interesting for people to do (e.g. email interviews or story completion tasks)
Mermaid: There are very very few blog posts on tgis topic, TBH – it’d probably take longer to find any than to analyse!
Ask people for their birth stories, though…
Michael: How long will it take to analyse? If it was one person developing the template, I’d say allow one week to develop the initial template, and then add one day for each 5-6 pages of transcript. Plus allow further week for theme development.
Wood nymph: Another Big Question: How do we know what to ask if we’re doing inteviews?
Dragon: Or even if we are asking written questions. How do we prompt an answer to analyse?
Michael: Open questions. Invitations to tell stories. Start with: ‘Tell me about what happened when …’? If possible.
Follow-ups – what was that like? how did you feel? etc
Can we combine everything?
Centaur: So… my personal view is mixed methods might suit us. Quantitative survey about choices offered, with some open text bits about effects of/feelings about choices, to be analysed qualitatively.
Imp: Mine too. What do you think Michael?
Michael: I would advise you to keep it simple and do one or the other. The sampling requirements of the quant component mean that you’ll end up swamped in qual data
Centaur: Hmm. You may have a good point Michael.
I wonder though… one of our strengths is that there are quite a lot of us. I wonder if we could divide the analysis between us? One group for quant one for qual… Do you think that might work? Or would they become too detached?
Pegasus: That is a fair point actually. You’d ideally want 500 survey responses so even if text fields were character limited (which i’d have issues with ethically given the topic) the data generated would be quite cumbersome to analyse.
Michael: Qual data from surveys is often called ‘small q’. It can be useful for very concrete questions. Not so good for experiential data.
Anonymity and the end
Michael: One last thing: think about how you will anonymise your data. Qual data is only anonymous to its audience, not usually to the researchers. Its not really confidential, b/c you will quote it.
Michael: Important to have a plan for how you will make sure participants’ identities aren’t revealed to audience when you quote their data. Change real names, locations etc.
Massive chorus of thank yous.
Michael: No problem. If I can send you any papers etc, just drop me a line. Good luck with your project!
Sphinx: Thank you, this has been really useful! 🙂
Unicorn: Thank you, you’ve been a total superstar. Please keep in touch 🙂
Griffin: Thank you so much, so informative!
Wood nymph: Thank you very much!
Pegasus: Still reading, but looks like it’s been a good chat. Cheers ?
Mermaid: You’ve been amazing. Thanks so much. If you see a fat woman on campus trying to bribe academics with cake to tell her where Michael Larkin’s office is, keep your head down and get out of there… it’ll be me.
