The Dumfries and Galloway Parenting Science Gang have decided to investigate flexischooling in Scotland. At the minute we are preparing to write a survey and so we invited University of York education researcher Dr Louise Tracey along to a Q&A to help us understand more about surveys.
Louise: Hi! I’m a researcher at the University of York and hoping to be able to answer questions about your flexischooling study research methods!
PSG: Hi Louise, thanks for coming!
Louise: Thank you, happy to be here! ?
I understand you are planning a survey. Would you like to tell me a bit more about that? Like, who you are planning to send it too? What information you are hoping to gain?
PSG H: We want to find out what effect experience of flexischooling has on opinions on flexischooling, and on normal schooling, in teachers, kids, and parents e.g. do teachers who experience flexischooled pupils in their class find it disruptive (so their opinion of flexischooling is more negative), or do they find it has positive effects?
And for pupils and parents, does flexischooling make them feel more or less positively about normal school?
Louise: So, there are three surveys then – teachers, pupils and parents. That would be really interesting as you could triangulate the findings – that is compare for similarities and difference. It provides a more holistic viewpoint of an issue….
Can I ask (and apologies if you have already discussed elsewhere) – why a survey? As opposed to, say, focus groups or one-to-one interviews?
PSG H: That is one of my questions. Should we use a different method? What is best for what?
PSG I: This brings the child and parents voice back to the table
Louise: I think a survey would be great, and the mapping in terms of the FOIs you will hopefully have will help with that, but different methods are best placed to answer different questions.
PSG: Can I ask about sample size? Are there any general rules about determining how many people we should be trying to get answers from (bearing in mind that we’ll not get a 100% response rate!).
Louise: In terms of sample size it depends on your potential pool of participants. The general aim is to have a representative sample – which means a smaller sample than your overall pool that has the same (or similar) characteristics of your larger (flexischooling in this case) population. In terms of returns, a third is considered good in a cold post-out but if you have connections or it is a topic people are particularly interested in you would expect higher. I would hope higher for your topic.
PSG B: Argh! That is part of the problem, we simply don’t know how many people flexischool! We’re planning FOI requests to the 32 local authorities to try to get numbers, but we won’t get the results of them before we can do our survey. Any thoughts on how we deal with that?
Louise: OK, you are working on this! Do think you could find a friendly person in one of those local authorities to talk to you and who could give you an idea of scale – at least in their own LA?
PSG B: That’s an idea. I guess we’d have to be a little cautious on expanding those numbers to cover the whole country, as we’d expect there to be massive variation across the country. But that would give us a starting point…
Louise: Also, and I am sure you have thought of this but are there any organisations that might be able to help. I know there is a strong homeschooling network in York. Something like that? I know there are differences but I could see some crossover for parents and pupils at least?
PSG B: Very good point. We know of quite a few home education groups we can contact in order to get people to fill in the survey. We just have no idea how many of them flexischool. It feels a little like looking for a needle in a haystack!
PSG D: There is a bit of an issue with home education and flexi-schooling in some networks. They don’t always mix well.
Louise: So in general, there is no specific size for a survey, the larger the better but I’d say 100 to pluck a figure out of thin air. I think mapping out the numbers flexischooling would perhaps be a good idea.
PSG: If you could only choose one area to investigate, which do you think would be the most likely to get meaningful results:
- investigating the reasons for starting flexischooling
- investigating the effects of flexischooling on kids (eg their attitudes to learning )
- trying to find out how beneficial flexischooling is for whole families?
Hmmm, maybe 2 and 3 are the same…..
Louise: I would see 2 as the key but probably the most difficult. 1. would work well in terms of a scoping/mapping study. This is the scale of the issue and this is why it exists. 3. could be a follow-on from 2, depending on how you framed it.
PSG: Thanks so much for joining us. I’d like to ask how we can structure our questions on the following subjects, in order to get the most useful results:
- How to gauge childrens’ attitudes to learning depending on if they are flexi-schooled or not.
- How to rate opinions on whether flexi-schooling is beneficial or not, for teachers, parents and children. And, crucially, why.
Louise: 1. The best way to compare would be to find a similar group of children not flexi-schooled. This would be larger scale but you could then administer a survey to both groups and compare.
PSG H: How can you be sure they are similar though? For a start, a family who choose to flexischool are by definition different to a family who don’t.
Louise: Is there an area where flexischooling just isn’t a possibility? And are we thinking of this in terms of schooled vs flexi. Perhaps parents / children who homeschool would be more comparable?
PSG B: Is there a way to effectively investigate children’s attitudes to learning without making the direct comparison? I guess I’m asking if there is any point in us digging into their attitudes without the control group or not?
Louise: I think you can ask their attitudes to learning as a group. But it would be more robust if there was some form of standard or scale. One piece of advice (to avoid a control) would be to find a measure (like a self-complete survey) suitable for your age-group that has been used in another study with a similar demographic or has been standardised against the general population – then you could see if the responses fell within the ‘normal range’ or were either end of the scale.
Louise: With regard to question 2 (How to rate opinions on whether flexi-schooling is beneficial or not, for teachers, parents and children. And why): Rating, as in a Likert scale? Very important, quite important, neither etc? is useful. Alternatively you could provide a list of possible reasons why flexischool, ask to tick all that apply (and an ‘other’ category) and then ask to indicate which was the most important? That way you can see all the considerations, and the priority?
I think the why would be best answered by focus group or interview to dig a little deeper. I was wondering if a survey to get scope and general headlines followed by interviews addressing the main issues raised by the survey. Or am I getting a little carried away with myself?
PSG H: Don’t worry about getting carried away, this is great, we can worry about what’s possible later, it’s good to work out all the things we might want to do.
Louise: If you want to know why people flexischool, what are the benefits, challenges etc then stick to flexischooled children. If you want to know if they have a better attitude, then it would be ideal to ask other children too. If you just asked flexischooled children/parents then my fear would be that they would be biased (they have after all made a bid investment in flexischooling).
PSG F: Is there any reason why we can’t do both? For example, have a longer questionnaire for those who flexi-school with a section which triangulates (getting the lingo now!) with a shorted questionnaire for non-flexi-schoolers.
Louise: No reason whatsoever! That sounds like a solution – and the flexischooling questionnaire would need to be longer because that is what you are primarily interested in.
PSG H: So are you saying it would be a good idea to have surveys to sort of scope it out, and then focus groups or structured interviews to go into more detail?
Louise: In an ideal world, yes, that would be my preference
PSG H: There’s not much time left! Is there anything you could suggest we read (short and in layperson’s language!) on focus groups and interviews, what they are good for, things to consider, etc?
Louise: Can I have a think and send suggestions on? I realise I have probably thrown out more questions than answered so if you have any further questions happy to try to answer via her?
PSG H: Thank you, that sounds great!
PSG F: Is there anything we should consider specifically when surveying children?
Louise: I think age of children is very important and something to consider carefully. You would ask different questions of a KS1 child to a secondary school. I am guessing you would prefer above the board but to be pragmatic perhaps you could narrow it? What age is most likely to be flexischooled?
PSG I: Primary almost exclusively, but again we don’t even know that for sure.
PSG B: In England, we believe that flexischooling is more commonly at primary level, and we have no reason to suspect that it would be dramatically different in Scotland. However, it might be that geography plays a much larger role in the decision to flexischool in Scotland – if people live in remote areas – in which case it might not be so similar, as that affects both primary and secondary aged kids.
PSG F: I disagree with the above. My experience is that, in upper Secondary, there are often flexi-school type arrangements involving a combination of school, college, work experience and other activities including therapy. However it’s got different characteristics from primary flexi-schooling so i think we should really just focus on primary aged. In which case I would expect the numbers might diminish as the children get older.
PSG K: If we did go down the children’s voices route, how do you suggest we analyse that qualitative data?
PSG D: We do this at work a lot and we ask a similar series of questions to each participant, with a part with opportunity for freeflow
PSG K: How do you analyse the freeflow section if you get a lot of data?
Louise: You can ‘code’ qualitative date to see if themes emerge which may answer your questions. It is a little more deductive than a survey but as a group it would be useful to compare ideas for themes and see if they are a helpful way to frame your thinking about the data.
PSG D: Our numbers are manageable as we have a relatively small service user group. We are usually looking for anything that stands out. So if a child raises a concern or has an idea we hadn’t thought of.
So essentially we would count up anything that recurs. So maybe “15 kids said……” etc.
PSG K: Louise, do you mean using something like Nvivo?
Louise: Yes, I use NVivo, but there are other packages and I know people who swear by Excel or good old paper and scissors
PSG D: We are bits of paper and pens ?
Louise: Essentially, it could provide numbers but, more importantly, the story beneath the numbers. That can be hard to manage (which is why mixed methods – survey and interviews would be lovely).
Whatever does the job! ??
PSG I: Are demographics important to include?
Louise: I think they can be helpful in terms of ensuring you are representative of the group i.e. if they are primarily parents with girls aged 12-15 and you only manage to survey parents of boys aged 7-8 then you might get very different results. But that would be more important to the survey and you can easily say who your respondents are up-front. For interviews small cases are fine. And sometimes the exceptions can tell us as much as the rules.
PSG I: How long can a survey be before it’s off-putting? Is there an optimum length?
PSG D: You’ll be laughing if you get lots of SEN Parents cos filling out forms is practically an occupation for us so we won’t notice ?
Louise: Umm, good question. I’d rather a bit longer but nicely spaced! Maybe 6-8 sides. And a space at the end saying if you’d like to share your experiences provide contact details for an interview. That would really help with the digging deeper – to have the survey responses. Have you thought about an on-line survey? I think google has an app for free.
PSG: Thank you so much Louise for joining us tonight. We’ve got quite a job in front of us, and you’ve really helped us to explore some of the key aspects of it. Thanks so much for your time!
If you are interested in flexischooling, why not come and join the Parenting Science Gang as we devise our own study in flexischooling in Scotland? You can join the Facebook group where we do all our work here: https://www.facebook.com/groups/1808623652732279/
Our other Q&As on flexischooling are also on our website: