Welcome back, we are beginning our third lesson on interviews and interviewers. We'll first talk about a number of tasks that interviewers engage in that really are not about conducting interviews per se, but they're kind of preliminary activities. So recruiting sample members, that is placing phone calls to, or knocking on the doors of households. And essentially persuading sample members to become respondents. And also, within household selection or within household sampling, which is an additional layer of sampling that interviewers are sometimes required to engage in to select one respondent from the household once the interviewer has obtained agreement in general this kind of extends the sampling design into the household. Then we'll talk about interviews themselves, and talk about a number of different interviewing techniques which are often, sort of presented in counterpoint to one another, or in opposition to one another. So, for example, we'll talk about the standardization debate, in which on the one hand, there's a strong argument for standardizing the wording, requiring interviewers to say essentially use the same words in every interview they conduct versus allowing interviewers to choose their words in a way that will essentially standardize the Interpretation. So if it takes different words to establish the same meaning across respondents, interviewers are able to do that under this second view. So this is known as the standardization debate. Another distinction is made between what's called personal style of interviewing in which interviewers attempt basically to establish rapport with respondents, and the formal style of interviewing, which is more business like. And the question is as with the previous distinction which of these produces higher quality data? Because the personal style is, essentially, about establishing rapport, we'll spend a little time of talking about rapport between interviewers and respondents, and how this may help, or hurt, data quality. And then, finally, we'll cover what are known as interviewer effects. So, while interviewers can add value to the Enterprise in many ways, they can also introduce error that isn't present in self-administered questionnaires. And these interviewer effects come within two varieties. One is biased in which interviewers, different interviewers elicit responses in different directions, so for example one set of interviewers might elicit more. Pro-feminist responses then another set of interviewers question concerning feminism. And it turns out that it's the gender of the interviewers that seems to be responsible for this. So female interviewers may elicit more pro-feminist responses than male interviewers. So this is called bias because the error is in one direction. Or interviewers may increase the response variance by behaving differently from one another and we'll focus on the behavior of probing, which is the kind of one unscripted part of standardized interviews. If interviewers do this differently the idea is that response variance will increase as a result. Let's spend a little bit of time talking about the different roles that interviewers play. As I said, they do a number of tasks besides asking questions and recording answers, and each of these can affect the different sources of error that we've been talking about. So when it comes to sampling and coverage, they have an impact on coverage by locating and contacting sample households and determining which units are eligible. Some structures are not residential for example, and would be ineligible in a survey that requires residential structures to be included. And they sample within the household as I mentioned a moment ago. And to the extent that this is successful or not can have an impact on sampling error. When it comes to non-response error or response rate at least interviewers do have the task of contacting the household and gaining cooperation, potentially persuading reluctant household members to become respondents and this can vary between interviewers again to the extent that this is done effectively the response rate should be high but it does vary between interviewers. Interviewers are probably most closely associated with measurement error, because they are asking the questions which are the measurements in conventional surveys, where self report is the predominant measure. So, they ask the questions, they probe for complete answers, they clarify questions, and they record answers, so what can go wrong here? Well interviewers can ask questions in different ways. They can probe differently, as I mentioned a moment ago. The questions may be misunderstood, and interviewers will be unaware of this or unable to address misunderstanding because, for example, the interviewing technique that they've been trained in and are required to use requires them to stick to the script, and if departing from the script is necessary to clarify questions that are really not able to do that. So interviewers can contribute to these different sources of error, more or less. So they can actually reduce these sources of error. So for example, response rates are generally higher with interviewers than in self-administered questionnaires. So interviewers both add value and introduce error. For better or worse, they can have an enormous impact on costs and timeliness. They are usually the single largest expense in a study that involves interviewers. That is more money spent on interviewers than anything else when there are interviewers in the study. So, it's a very Important decision, whether or not the mode involves interviews, and one that researchers should make carefully. So, turning to recruitment, one of these tasks that interviewers engage in that really don't involve conducting interviews. Some interviewers are more successful than others in obtaining interviews. If you look across studies and a system called lifetime response rate. So what might be responsible for this? Well it could have to do with some interviewers just being better at recruiting perhaps because they're more extroverted or their personalities just lend themselves to persuading strangers to take part in the study. Another possibility particularly for telephone interviews is that it may have to do with how they speak and interact with respondents. So that's actually what we'll turn to next. On the telephone, really the only information that a household member has about the interviewer who's trying to persuade them to take part in the study Is what they sound like and how they speak. Really, telephone interviewing may sort of concentrate sample members on interviewers' vocal properties and the way they interact. In one study, Conrad and his colleagues Coded the behavioral attributes of interviewers in 1300 telephone survey introductions. These are audio recorded telephone introductions from five studies that were conducted at the University of Michigan. And the behaviours they coded included fluencies like um's and ah's. What are known as back channels these were by the household member the sample member not by the interviewer, back channels are for example saying uh-huh and nodding although of course on the telephone the interviewer would never see the household member nodding but saying uh-huh is generally considered an acknowledgement, a way to say to the speaker, yeah please continue I'm still listening, and over speech and interruptions, and that's called simultaneous speech. The idea being that these generally signal some sort of troubled interaction if there's a lot of overspeech. And they modeled the impact of these different behaviors on one of three outcomes. Agreement to take part in the study, refusal scheduled callback which basically refers to deferring the decision until later so the interviewer and the household member agree that someone, maybe not the interviewer there currently talking, but somebody will call them back in the future to talk about a possibility of taking part in the study. So a scheduled callback is kind of In between it agreement and refusal and so they were looking at these three outcomes on the basis of behaviors like disfluency a household member back channels and over speech or interruptions. So the first finding was that fillers or disfluencies like ums and uhs, are both good and bad when it comes to participation. So this figure shows us the filler rate, or the disfluency rate, per hundred words. And the rates are broken into these quintiles. The y-axis is the proportion of degrees as a function of the filler rate. So you can see when interviewers are making zero disfluencies, that is when they're highly fluent. They're actually quite unsuccessful in recruiting household members. This is actually the lowest agreement rate of all the quintiles. As soon as they make even a small number of disfluencies, this is more than 0 and less that 1.28 per 100 words, so not a lot of disfluency. Agreement rate goes up substantially to 0.35. It tends to drop off as they get more until, when they're making the maximum number of disfluencies, fillers, greater than three point four nine per hundred words, it's at it's lowest when there are any fillers. However, even when interviewers are highly disfluent, they're more successful than when they're perfectly fluent. And the thinking was that, when they're perfectly fluent, they sound robotic. And they don't seem to be tailoring the invitation, the recruitment speech for the particular household member. That seems to be the big disincentive to participate. Another speech behavior that's involved in recruitment is, as I mentioned, backchannels. The household members produce more back channels uh-huh and okay or I see. Where they ultimately agree. This figure shows that the proportion of lose or conversational terms with a back channel by the household member. And as you can see, scheduled callbacks are in between agrees and refusals, but really when they're signaling that they are paying attention and sort of authorizing the speaker, or in this case the interviewer to continue, they're most likely to agree. And then, the third behavior that the authors looked at was overspeech or simultaneous speech. As I said before, when there is a lot of overspeech, it generally indicates that this is an interaction that's not going well. And as you can see, overspeech is highest when the invitation, the case, ultimately ends in refusal. A scheduled callback is lower, but not as low as when the invitation led to agreement. So the amount of over speech in an invitation is pretty clearly related to the outcome. However, not all overspeech works the same way. What the authors looked at was what they called interruptions, which are actually just what they sound like. This is overspeech in which the household member was speaking and then the interviewer began to speak before the household member had finished. This type of overspeech seemed to have been used quite strategically by the interviewers to essentially block a refusal. So as you can see, when the invitation ended in a scheduled call back, which again is a deferral to another time to make the decision interviewer interruptions of household members was at its highest. And again that the idea that the authors proposed is that they were preventing or interrupting an utterance that would have led to a refusal had they not intervened at that point. So that's one set of examples of how the speech and vocal attributes of interviewers and how they interact with household members and how household members speak in response can effect outcomes. Another study published at about the same time by Nora Cate Schaeffer and her colleagues looked at a pairs of respondents with the same propensity to participate. So what they did was, they modeled the likelihood to participate on the basis of a number of variables in their dataset, and this is a longitudinal dataset so they had information about these household members going back a number of waves, meaning previous interviews. They found pairs who had the same likelihood, same propensity to participate, only one member of the pair accepted an invitation in the wave they were looking at. And the other member of the pair declined. So they have the same likelihood of participating but different outcomes. What can be going on? So they looked at the speech of both interviewers and I'm calling them respondents here, but again they're household members because they haven't yet agreed to do this interview. Using conversation analytic techniques similar to, a little different than what was used in the Conrad et al study when they coded the interactions. They uncovered a number of Interesting provocative explanations for this difference. One thing that I look at it was when did at these sample members decline the invitation and when did they accept it? That's at what point in the invitation. So as you can see if you look at declinations, they're occurring at various points during the interview not exactly even numbers but with non-zero frequency. So during the opening, during the study description, and after the request for participation. When you look at acceptance, that's only happening after an explicit request for participation. So you can see that household members, sample members are feel no qualms about declining at any point. It's much harder to elicit an acceptance, and that really requires an explicit request. Will you take part in our study? Essentially. They also looked at particular interviewer actions and how they affected the odds of participating. The odds ratio in a logistic regression. So, if the interviewer said hello versus hi, after the respondent answered the phone, the odds ratio was 1.44, which is marginally significant. When the interviewer requested to speak to a sample member in a very polite way, versus a not so polite way. The odds also were greater that the invitation would lead to an interview, marginally significant but, you can see, in the right direction. And, when the interviewer introduced herself, for identified herself or himself first versus second, the odds were again higher that the invitation would lead to an interview. In this case significantly higher. Finally, they looked at a number of sample member behaviors that seemed to indicate a willingness and interest in participating. Or an engagement in the process that would make them more likely to participate. So, if the sample member asked at least one question. They were reliably more likely to participate than if they ask none. If a sample member asked about the or said something about the length of the interview, how long is this going to take. Much more likely to participate than if they had no. Nothing to say about the length as if to suggest that it would only be thinking about how long will this take if they were considering participating. If the sample member asks at least one W-H type question like what is this about, the odds are less likely they'll participate than if they don't ask such a question. So this seems to indicate a kind of skepticism or reluctance to participate. The effect is even stronger if they ask this question before the interviewer requests that they participate. But if they ask a WH type question after the interviewer asks them to participate there's no difference, essentially, between asking that question and not asking that question. Because by that time the household member, sample member, has made up his or her mind about whether they would participate or not and so these questions have a kind of different significance at that point. So these are some examples of how interviewers' speech and how they interact with household members. Can affect the outcome of their recruitment efforts, at least on the telephone. This is among the kind of non-interview tasks that interviewers engage in. Next we'll turn to within household sampling, or respondent selection, which is an interviewer activity that follows acceptance of the invitation or agreement to participate by the household member who first spoke to the interviewer. That person doesn't necessarily become the respondent but is considered the gate keeper, and if that person agrees to participate another household member may still be selected and may have to be convinced to participate. This process of within household sampling or respondent selection assures that any eligible member of the household can potentially be selected to be the respondent and that's important in implementing the sample design.
