USING FOCUS GROUPS FOR EVALUATION
Mary Marczak & Meg Sewell
WHAT IS A FOCUS GROUP?
Focus groups were originally called "focused interviews" or "group depth interviews". The technique was developed after World War II to evaluate audience response to radio programs (Stewart & Shamdasani, 1990). Since then social scientists and program evaluators have found focus groups to be useful in understanding how or why people hold certain beliefs about a topic or program of interest.
A focus group could be defined as a group of interacting individuals having some common interest or characteristics, brought together by a moderator, who uses the group and its interaction as a way to gain information about a specific or focused issue.
A focus group is typically 7-10 people who are unfamiliar with each other. These participants are selected because they have certain characteristics in common that relate to the topic of the focus group. The moderator or interviewer creates a permissive and nurturing environment that encourages different perceptions and points of view, without pressuring participants to vote, plan or reach consensus (Krueger, 1988). The group discussion is conducted several times with similar types of participants to identify trends and patterns in perceptions. Careful and systematic analysis of the discussions provide clues and insights as to how a product, service, or opportunity is perceived by the group.
WHAT FOCUS GROUPS CAN TELL YOU:
* Give information on how groups of people think or feel about a particular topic
* Give greater insight into why certain opinions are held
* Help improve the planning and design of new programs
* Provide a means of evaluating existing programs
* Produce insights for developing strategies for outreach
WHAT FOCUS GROUPS CANNOT TELL YOU:
* Valid information about individuals
* Valid "before-and-after" information (how things have changed over time)
* Information that you can apply generally to other groups of people
Because the idea of focus groups is to take advantage of group interactions, it is important to use the information at the group level, not the individual level. Focus groups are not a valid way to find out how much progress an individual client or participant has made toward his or her own goals. Also, because focus groups are usually made up of a very small number of people who voluntarily participate, you cannot assume that their views and perceptions represent those of other groups that might have slightly different characteristics. They are not "random samples".
USING FOCUS GROUPS WITH THE STATE STRENGTHENING EVALUATION GUIDE
If you are using the Five-Tiered Approach to Program Evaluation outlined in the State Strengthening Evaluation Guide, focus groups can be helpful at several levels:
TIER 1 - Program Definition
One key task in Tier 1 is to assess community needs, and document a need for a particular program. At this stage, you might use focus groups made up of people in the community who would be affected by the problem or its solution to get a better idea of how they see the problem, and whether they would use the program you have in mind. For example, if you are considering starting an after-school job-skills club for at-risk youth, you might conduct a focus group made up of high school students to find out whether they would attend, what kinds of activities would attract them, and what qualities they would like to see in leaders of the program. You might also find out what would be major barriers to participating (for example, whether students who ride the school bus would have a transportation problem if they stayed after school). Other focus groups made up of parents or local business leaders could provide different perspectives on the need and the program you are considering.
TIER 2 - Accountability
Once your program has started up and has been running for a little while, you will need to examine whether the program serves the people it was intended to serve, and in the manner proposed. After a few months, staff might have the impression that the youth after-school job skills program is reaching mostly girls who are doing well in school and already active in clubs and social groups. However, it is not attracting many boys, or more socially isolated girls. It also appeared that the job skills program was serving primarily as a social club for many of the girls. Holding a focus group of participants fairly early in the process might confirm these impressions.
TIER 3 - Understanding and Refining
Just as important as confirming the impression that the program is not reaching its intended audience, the focus group could provide staff with information that might help understand why, and help to reach them more effectively. It might be possible to convene another focus group made up of students who are not participating, to find out their perceptions of the program and what could be done to remove barriers. Maybe more boys would participate if the program included adult male mentors, for example. Or maybe many of the non-participating teens already have part-time jobs or family responsibilities, so that they lack time or motivation. In tier three, focus groups may help you refine the program to better meet the needs of your intended audience.
TIER 4 - Progress Toward Outcomes
Focus groups are probably most helpful in Tiers 1, 2, and 3 of the evaluation model. However, they may also provide some limited information related to program outcomes. Although this information is subjective and qualitative, it can at least provide a general sense of whether participants are satisfied with the program, and why they think it is or is not helping. Used in reports, this kind of information might be a useful supplement to the more objective numbers like attendance figures or other outcome measures.
ADVANTAGES OF USING FOCUS GROUPS
* Takes advantage of the fact that people naturally interact and are influenced by others (high face validity).
* May be one of the few research tools available for obtaining data from children or from individuals who are not particularly literate
* Provide data more quickly and at lower cost than if individuals interviewed separately; groups can be assembled on shorter notice than for a more systematic survey.
* Generally requires less preparation and is comparatively easy to conduct.
* Researcher can interact directly with respondents (allows clarification, follow-up questions, probing). Can gain information from non-verbal responses to supplement (or even contradict) verbal responses.
* Data uses respondents' own words; can obtain deeper levels of meaning, make important connections, identify subtle nuances
* Very flexible; can be used with wide range of topics, individuals, and settings
* Results are easy to understand and mor accessible to lay audiences or decision-makers than complex statistical analyses of survey data
DISADVANTAGES OF USING FOCUS GROUPS
* Have less control over group; less able to control what information will be produced.
* Produces relatively chaotic data making data analysis more difficult.
* Small numbers and convenience sampling severely limit ability to generalize to larger populations
* Requires carefully trained interviewer who is knowledgeable about group dynamics. Moderator may knowingly or unknowingly bias results by providing cues about what types of responses are desirable
* Uncertainty about accuracy of what participants say. Results may be biased by presence of a very dominant or opinionated member; more reserved members may be hesitant to talk.
HOW TO CONDUCT A FOCUS GROUP
Conducting a focus group occurs in three phases (Krueger, 1988):
1. Conceptualization
2. Interview
3. Analysis and reporting
Conceptualization Phase
Determining the purpose
Consider why focus groups should be conducted. What types of information are of particular importance? Who wants the information? Consider information needs: who will use it? What information is needed? Why is it needed?
Determining whom to study
Who can provide the needed information? Consider special groups (e.g., advisory board, employees, clients). Demographic characteristics such as age, gender, occupation, education, ses, etc. become key in determining who should be invited.
Developing a plan and estimating needed resources
Specify procedure, time line, and proposed budget. Allow others (e.g., colleagues, focus group experts, etc.) to review the plan.
Interview Phase
Developing the questions
Questions should be carefully planned but appear spontaneous during the interview. For best results, a focus group session should include around five or six questions. It should always include less than ten questions. Other suggestions in developing questions for focus group sessions include:
a. Use open-ended questions and avoid "yes" or "no" questions.
Appropriate question: What did you think of the program?
Inappropriate question: Did you like the program?
b. "Why" questions are rarely asked in a focus group. Why questions tend to imply a rational answer.
c. Questions should be systematically prepared but has a natural flow to them. You should get feedback on the set of questions from others.
d. Arrange questions in logical sequence.
e. Allow for unanticipated questions.
f. Pilot test focus group interview.
Moderating skills
Moderators (interviewers) play a key role in the success of focus groups. Moderators should have characteristics that are similar to participants and be skilled in group processes. Their role is to keep discussions flowing and on track, guide discussions back from irrelevant topics, make transitions into another question, and be sensitive to mood of the group. Know when to move onto another question. Moderators should also have some background knowledge about the topic being discussed. Other suggestions for the moderator role include:
a. Use of a moderator team. One should consider using a moderator team with divided tasks. The main moderator should direct the discussion and take minimal notes. The assistant can take comprehensive notes, operate tape the recorder, handle environmental conditions, and respond to unexpected interruptions (e.g., late comers, children someone brought, etc.).
b. Be mentally prepared. Moderators should be mentally alert, listen well, and think quickly on their feet. Questions should be memorized.
c. Have a Presession strategy. Small talk is essential just prior to group discussion. The moderator should greet the participants and begin small talk while avoiding issues to be discussed during the focus group session. This time will allow a moderator to observe the interaction. Name tents can be strategically placed around the table after observing participants. For example, you may want to strategically place those who are extremely shy or those who may dominate the discussion.
d. Record the discussion. Discussions should be recorded via tape recording and note taking. Notes are essential. Notes should be so complete that it can be used even if the tape recorder did not work. One should never rely completely on a tape recorder. Someone, other than the moderator should take detailed notes.
e. Begin the discussion. The recommended pattern of discussion is welcome, overview and topic, ground rules, first question. The overview should provide an honest discussion of the about the purpose of the study and the importance of the topic of group discussion. Ground rules are suggestions that will help guide the discussion and include rules such as: minimize or eliminate side conversations, one person will speak at a time, don't criticize what others have to say, and treat everyone's ideas with respect. The first question should be one that "breaks the ice" and encourages everyone to talk.
f. Pause and Probe. As a general rule, a moderator should pause for five seconds after a participant talks before beginning to talk. This five second pause gives other participants a chance to jump in. Probes, such as "would you explain that further?" or "Would you give me an example?" should be used to request additional information
g. Responding to participant comments. Moderators should avoid head nodding, and short verbal responses such as "ok", "yes", "uh huh", "correct", "that's good" etc.
h. Be aware of group dynamics. Moderators should watch for the expert, the dominant talker, the shy participant, the rambler, etc.
I. Selecting the focus group location. Locations should be easy to get to. Participants should sit facing each other. A table should be available for participants to sit around. The room should also be accessible for setting up tape recorders.
k. Being prepared for the unexpected. Moderators should be prepared for unexpected evens such as no one showing up (make sure you bring list and phone numbers), only a few showing up (hold group anyway), meeting place inadequate, group does not want to talk (ask individuals questions, go around the room and everyone answers specific question, the group gets involved and don't want to leave (have formal ending), hazardous weather (call everyone and cancel), early questions take up too much time (make sure important questions at the end of question rout get answered).
l. Concluding the focus group. Thank the group for participating. The moderator may choose to summarize what was said and ask if anything was missed.
Participants
Focus group participants should be systematically and purposefully selected. The following provides some guidelines in selecting potential participants:
a. Purpose of study should guide who to invite.
b. Focus group participants should be characterized by homogeneity, but with sufficient variation to allow for contrasting opinions. Homogeneity is most often sought in terms of occupation, social class, educational level, age, education, or family characteristics, and gender (unless interaction across these groups is what is driving the study).
c. Participants should not be familiar with each other (those who know each other tend to form small group discussions).
d. Size of the group is typically around 7-10. Groups of over twelve have often proven to be too big while under four have shown that not enough total experiences exist.
e. Focus groups require a flexible research design and not randomization. In selecting participants, it is critical to remember that intent of focus groups is not to infer but to understand, not to generalize but to determine the range, not to make statements about population but to provide insights about how people perceive a situation.
f. A helpful rule of thumb in determining the number of groups needed is to continue conducting interviews until little new information is provided. Typically, the first two groups provide a considerable amount of new information but by the third or fourth session, a fair amount may have already been covered. If this occurs stop.
g. To get people to attend, one should personalize invitations, establish meeting times that don't conflict with existing community activities or functions, contact potential participants via phone 10-14 days prior to, send personalized invitations one week before session, phone each person the day before to remind them. If feasible, incentives are also helpful in getting people to attend.
Analyzing and Reporting Phase
Analysis
Analysis should begin by going back to intent of the study. For example, if purpose of the study is narrow, elaborate analysis may be unneeded and inappropriate. The process of data analysis must be systematic (follow a prescribed, sequential flow) and verifiable (another person would come to a similar conclusion using the available documents and the raw data). Ideally, the moderator or assistant should also do the analysis. A sequence of analysis may follow the following format:
a. Process begins during presession small talk by observing levels of familiarity between participants.
b. Immediately after the session, moderator and assistant spot checks the tape recorder to make sure it recorded. If tape cannot be salvaged, moderating team should reconstruct the discussion immediately. It may be helpful to tape-record this debriefing.
c. The moderator and assistant write down summary comments and listen to the complete tape to write a more complete summary of the discussion. This written summary should be prepared within hours after the session and before the next focus group. Moderator and assistant compare notes, share observations, talk about participant responses to key questions.
d. The moderator and assistant should arrive at a short summary that is mutually agreeable. Field notes should capture information on:
1. Any changes in the list of questions
2. Participant characteristics
3. Descriptive phrases or words used by participants as they discussed the key question
4. Themes in the responses to the key questions
5. Subthemes indicating a point of view held by participants with common characteristics
6. Description of participant enthusiasm
7. Consistency between participant comments and their reported behaviors
8. Body language
9. New avenues of questioning that should be considered in future: should questions be revised, eliminated, added, etc.
10. Overall mood of discussion.
e. The process then continues by gathering together brief summary reports, tape recordings, list of questions, demographic information, and if available, transcripts of the discussion.
f. All summaries should be read at one sitting. Notes should be made of potential trends and patterns.
g. Each transcript should be read (if no transcripts, then one should listen to the tapes of the discussions).
h. During the second reading though the transcripts, mark sections related to each question on question list.
i. When conducting analysis consideration should be given to five factors:
1. Words. Actual words and meanings of the words should be determined. One might make frequency counts of commonly used words. Cluster similar concepts together.
2. Context. Examine the context of words by finding the triggering stimulus and then interpret the comment in light of that context.
3. Internal consistency. Trace a flow of conversation and note changes or even reverses of position after interaction with others.
4. Specificity of responses. Responses that are specific and based on experiences should be given more weight than responses that are vague and impersonal. Greater weight should be given to responses in first person rather than third person hypothetical answers.
5. Find the big ideas. Look for rends or ideas that cut across the entire discussion.
Reporting
Once again, the objectives of the study should determine how and for whom the information needs to be reported. As a general rule, numbers and percentages are not appropriate for focus group research and should not be included in report. Reporting should be descriptive and present the meaning of the data as opposed to a summary of data.
Data can be examined and reported at three levels, including 1) the raw data, 2) descriptive statements, and 3) interpretation (Kreuger, 1988).
Raw data present statements as they were said by respondents. The data might be ordered or categorized by natural levels or themes in the topic.
Descriptive statements summarize respondents' comments and provide illustrative examples using the raw data. Decisions must be made as to which quotes to include.
Interpretation is most complex. Interpretation builds on the descriptive process by providing or presenting meaning of the data rather than simply summarizing the data. In giving meaning to the descriptions, one should be reflective about own biases in interpretation.
A VARIATION IN USING FOCUS GROUPS
One useful variation on focus groups is the Nominal Group Technique (NGT) suggested by Stewart and Shamdasani (1990). This technique allows you to take advantage of some of the benefits of focus groups under specialized conditions including:
1) uses with very specialized groups of people who can't easily be assembled on short notice (such as government officials, senior executives, etc.);
2) uses where there is a significant power differential among members (supervisors/subordinates, parents/children); or
3) uses when high levels of conflict or polarization over an issue exist, so that a minority/dissenting viewpoint might be inhibited.
"Nominal groups" are groups in name only. Participants do not interact and may not even meet.
During this technique, each member is interviewed separately (possibly by phone). Anonymous summaries of each person's perceptions/ideas are provided to each other member for comment. While it is not as spontaneous as a typical focus group session, this technique still allows participants to react to and be influenced by comments made by others.
ANNOTATED BIBLIOGRAPHY
1) Krueger, R. A. 1988. Focus groups: A practical guide for applied research. Newbury Park, CA: Sage Publications.
One of the few books that applies focus groups to evaluation research. This book gives considerable attention to analyzing and reporting focus group results. Detailed examples will be helpful to those who are just getting started in focus groups.
2) Morgan, D. L. 1988. Focus Groups as Qualitative Research. Newbury Park, CA: Sage Publications.
A good general guide to conducting focus groups. Translates marketing research techniques on conducting group interviews into a form more applicable for social sciences research.
3) Stewart, D. W., and Shamdasani, P. N. (1990). Focus groups: Theory and practice. Applied Social Research Methods Series, Vol. 20. Newbury Park, CA: Sage Publications.
Mary Marczak & Meg Sewell
WHAT IS A FOCUS GROUP?
Focus groups were originally called "focused interviews" or "group depth interviews". The technique was developed after World War II to evaluate audience response to radio programs (Stewart & Shamdasani, 1990). Since then social scientists and program evaluators have found focus groups to be useful in understanding how or why people hold certain beliefs about a topic or program of interest.
A focus group could be defined as a group of interacting individuals having some common interest or characteristics, brought together by a moderator, who uses the group and its interaction as a way to gain information about a specific or focused issue.
A focus group is typically 7-10 people who are unfamiliar with each other. These participants are selected because they have certain characteristics in common that relate to the topic of the focus group. The moderator or interviewer creates a permissive and nurturing environment that encourages different perceptions and points of view, without pressuring participants to vote, plan or reach consensus (Krueger, 1988). The group discussion is conducted several times with similar types of participants to identify trends and patterns in perceptions. Careful and systematic analysis of the discussions provide clues and insights as to how a product, service, or opportunity is perceived by the group.
WHAT FOCUS GROUPS CAN TELL YOU:
* Give information on how groups of people think or feel about a particular topic
* Give greater insight into why certain opinions are held
* Help improve the planning and design of new programs
* Provide a means of evaluating existing programs
* Produce insights for developing strategies for outreach
WHAT FOCUS GROUPS CANNOT TELL YOU:
* Valid information about individuals
* Valid "before-and-after" information (how things have changed over time)
* Information that you can apply generally to other groups of people
Because the idea of focus groups is to take advantage of group interactions, it is important to use the information at the group level, not the individual level. Focus groups are not a valid way to find out how much progress an individual client or participant has made toward his or her own goals. Also, because focus groups are usually made up of a very small number of people who voluntarily participate, you cannot assume that their views and perceptions represent those of other groups that might have slightly different characteristics. They are not "random samples".
USING FOCUS GROUPS WITH THE STATE STRENGTHENING EVALUATION GUIDE
If you are using the Five-Tiered Approach to Program Evaluation outlined in the State Strengthening Evaluation Guide, focus groups can be helpful at several levels:
TIER 1 - Program Definition
One key task in Tier 1 is to assess community needs, and document a need for a particular program. At this stage, you might use focus groups made up of people in the community who would be affected by the problem or its solution to get a better idea of how they see the problem, and whether they would use the program you have in mind. For example, if you are considering starting an after-school job-skills club for at-risk youth, you might conduct a focus group made up of high school students to find out whether they would attend, what kinds of activities would attract them, and what qualities they would like to see in leaders of the program. You might also find out what would be major barriers to participating (for example, whether students who ride the school bus would have a transportation problem if they stayed after school). Other focus groups made up of parents or local business leaders could provide different perspectives on the need and the program you are considering.
TIER 2 - Accountability
Once your program has started up and has been running for a little while, you will need to examine whether the program serves the people it was intended to serve, and in the manner proposed. After a few months, staff might have the impression that the youth after-school job skills program is reaching mostly girls who are doing well in school and already active in clubs and social groups. However, it is not attracting many boys, or more socially isolated girls. It also appeared that the job skills program was serving primarily as a social club for many of the girls. Holding a focus group of participants fairly early in the process might confirm these impressions.
TIER 3 - Understanding and Refining
Just as important as confirming the impression that the program is not reaching its intended audience, the focus group could provide staff with information that might help understand why, and help to reach them more effectively. It might be possible to convene another focus group made up of students who are not participating, to find out their perceptions of the program and what could be done to remove barriers. Maybe more boys would participate if the program included adult male mentors, for example. Or maybe many of the non-participating teens already have part-time jobs or family responsibilities, so that they lack time or motivation. In tier three, focus groups may help you refine the program to better meet the needs of your intended audience.
TIER 4 - Progress Toward Outcomes
Focus groups are probably most helpful in Tiers 1, 2, and 3 of the evaluation model. However, they may also provide some limited information related to program outcomes. Although this information is subjective and qualitative, it can at least provide a general sense of whether participants are satisfied with the program, and why they think it is or is not helping. Used in reports, this kind of information might be a useful supplement to the more objective numbers like attendance figures or other outcome measures.
ADVANTAGES OF USING FOCUS GROUPS
* Takes advantage of the fact that people naturally interact and are influenced by others (high face validity).
* May be one of the few research tools available for obtaining data from children or from individuals who are not particularly literate
* Provide data more quickly and at lower cost than if individuals interviewed separately; groups can be assembled on shorter notice than for a more systematic survey.
* Generally requires less preparation and is comparatively easy to conduct.
* Researcher can interact directly with respondents (allows clarification, follow-up questions, probing). Can gain information from non-verbal responses to supplement (or even contradict) verbal responses.
* Data uses respondents' own words; can obtain deeper levels of meaning, make important connections, identify subtle nuances
* Very flexible; can be used with wide range of topics, individuals, and settings
* Results are easy to understand and mor accessible to lay audiences or decision-makers than complex statistical analyses of survey data
DISADVANTAGES OF USING FOCUS GROUPS
* Have less control over group; less able to control what information will be produced.
* Produces relatively chaotic data making data analysis more difficult.
* Small numbers and convenience sampling severely limit ability to generalize to larger populations
* Requires carefully trained interviewer who is knowledgeable about group dynamics. Moderator may knowingly or unknowingly bias results by providing cues about what types of responses are desirable
* Uncertainty about accuracy of what participants say. Results may be biased by presence of a very dominant or opinionated member; more reserved members may be hesitant to talk.
HOW TO CONDUCT A FOCUS GROUP
Conducting a focus group occurs in three phases (Krueger, 1988):
1. Conceptualization
2. Interview
3. Analysis and reporting
Conceptualization Phase
Determining the purpose
Consider why focus groups should be conducted. What types of information are of particular importance? Who wants the information? Consider information needs: who will use it? What information is needed? Why is it needed?
Determining whom to study
Who can provide the needed information? Consider special groups (e.g., advisory board, employees, clients). Demographic characteristics such as age, gender, occupation, education, ses, etc. become key in determining who should be invited.
Developing a plan and estimating needed resources
Specify procedure, time line, and proposed budget. Allow others (e.g., colleagues, focus group experts, etc.) to review the plan.
Interview Phase
Developing the questions
Questions should be carefully planned but appear spontaneous during the interview. For best results, a focus group session should include around five or six questions. It should always include less than ten questions. Other suggestions in developing questions for focus group sessions include:
a. Use open-ended questions and avoid "yes" or "no" questions.
Appropriate question: What did you think of the program?
Inappropriate question: Did you like the program?
b. "Why" questions are rarely asked in a focus group. Why questions tend to imply a rational answer.
c. Questions should be systematically prepared but has a natural flow to them. You should get feedback on the set of questions from others.
d. Arrange questions in logical sequence.
e. Allow for unanticipated questions.
f. Pilot test focus group interview.
Moderating skills
Moderators (interviewers) play a key role in the success of focus groups. Moderators should have characteristics that are similar to participants and be skilled in group processes. Their role is to keep discussions flowing and on track, guide discussions back from irrelevant topics, make transitions into another question, and be sensitive to mood of the group. Know when to move onto another question. Moderators should also have some background knowledge about the topic being discussed. Other suggestions for the moderator role include:
a. Use of a moderator team. One should consider using a moderator team with divided tasks. The main moderator should direct the discussion and take minimal notes. The assistant can take comprehensive notes, operate tape the recorder, handle environmental conditions, and respond to unexpected interruptions (e.g., late comers, children someone brought, etc.).
b. Be mentally prepared. Moderators should be mentally alert, listen well, and think quickly on their feet. Questions should be memorized.
c. Have a Presession strategy. Small talk is essential just prior to group discussion. The moderator should greet the participants and begin small talk while avoiding issues to be discussed during the focus group session. This time will allow a moderator to observe the interaction. Name tents can be strategically placed around the table after observing participants. For example, you may want to strategically place those who are extremely shy or those who may dominate the discussion.
d. Record the discussion. Discussions should be recorded via tape recording and note taking. Notes are essential. Notes should be so complete that it can be used even if the tape recorder did not work. One should never rely completely on a tape recorder. Someone, other than the moderator should take detailed notes.
e. Begin the discussion. The recommended pattern of discussion is welcome, overview and topic, ground rules, first question. The overview should provide an honest discussion of the about the purpose of the study and the importance of the topic of group discussion. Ground rules are suggestions that will help guide the discussion and include rules such as: minimize or eliminate side conversations, one person will speak at a time, don't criticize what others have to say, and treat everyone's ideas with respect. The first question should be one that "breaks the ice" and encourages everyone to talk.
f. Pause and Probe. As a general rule, a moderator should pause for five seconds after a participant talks before beginning to talk. This five second pause gives other participants a chance to jump in. Probes, such as "would you explain that further?" or "Would you give me an example?" should be used to request additional information
g. Responding to participant comments. Moderators should avoid head nodding, and short verbal responses such as "ok", "yes", "uh huh", "correct", "that's good" etc.
h. Be aware of group dynamics. Moderators should watch for the expert, the dominant talker, the shy participant, the rambler, etc.
I. Selecting the focus group location. Locations should be easy to get to. Participants should sit facing each other. A table should be available for participants to sit around. The room should also be accessible for setting up tape recorders.
k. Being prepared for the unexpected. Moderators should be prepared for unexpected evens such as no one showing up (make sure you bring list and phone numbers), only a few showing up (hold group anyway), meeting place inadequate, group does not want to talk (ask individuals questions, go around the room and everyone answers specific question, the group gets involved and don't want to leave (have formal ending), hazardous weather (call everyone and cancel), early questions take up too much time (make sure important questions at the end of question rout get answered).
l. Concluding the focus group. Thank the group for participating. The moderator may choose to summarize what was said and ask if anything was missed.
Participants
Focus group participants should be systematically and purposefully selected. The following provides some guidelines in selecting potential participants:
a. Purpose of study should guide who to invite.
b. Focus group participants should be characterized by homogeneity, but with sufficient variation to allow for contrasting opinions. Homogeneity is most often sought in terms of occupation, social class, educational level, age, education, or family characteristics, and gender (unless interaction across these groups is what is driving the study).
c. Participants should not be familiar with each other (those who know each other tend to form small group discussions).
d. Size of the group is typically around 7-10. Groups of over twelve have often proven to be too big while under four have shown that not enough total experiences exist.
e. Focus groups require a flexible research design and not randomization. In selecting participants, it is critical to remember that intent of focus groups is not to infer but to understand, not to generalize but to determine the range, not to make statements about population but to provide insights about how people perceive a situation.
f. A helpful rule of thumb in determining the number of groups needed is to continue conducting interviews until little new information is provided. Typically, the first two groups provide a considerable amount of new information but by the third or fourth session, a fair amount may have already been covered. If this occurs stop.
g. To get people to attend, one should personalize invitations, establish meeting times that don't conflict with existing community activities or functions, contact potential participants via phone 10-14 days prior to, send personalized invitations one week before session, phone each person the day before to remind them. If feasible, incentives are also helpful in getting people to attend.
Analyzing and Reporting Phase
Analysis
Analysis should begin by going back to intent of the study. For example, if purpose of the study is narrow, elaborate analysis may be unneeded and inappropriate. The process of data analysis must be systematic (follow a prescribed, sequential flow) and verifiable (another person would come to a similar conclusion using the available documents and the raw data). Ideally, the moderator or assistant should also do the analysis. A sequence of analysis may follow the following format:
a. Process begins during presession small talk by observing levels of familiarity between participants.
b. Immediately after the session, moderator and assistant spot checks the tape recorder to make sure it recorded. If tape cannot be salvaged, moderating team should reconstruct the discussion immediately. It may be helpful to tape-record this debriefing.
c. The moderator and assistant write down summary comments and listen to the complete tape to write a more complete summary of the discussion. This written summary should be prepared within hours after the session and before the next focus group. Moderator and assistant compare notes, share observations, talk about participant responses to key questions.
d. The moderator and assistant should arrive at a short summary that is mutually agreeable. Field notes should capture information on:
1. Any changes in the list of questions
2. Participant characteristics
3. Descriptive phrases or words used by participants as they discussed the key question
4. Themes in the responses to the key questions
5. Subthemes indicating a point of view held by participants with common characteristics
6. Description of participant enthusiasm
7. Consistency between participant comments and their reported behaviors
8. Body language
9. New avenues of questioning that should be considered in future: should questions be revised, eliminated, added, etc.
10. Overall mood of discussion.
e. The process then continues by gathering together brief summary reports, tape recordings, list of questions, demographic information, and if available, transcripts of the discussion.
f. All summaries should be read at one sitting. Notes should be made of potential trends and patterns.
g. Each transcript should be read (if no transcripts, then one should listen to the tapes of the discussions).
h. During the second reading though the transcripts, mark sections related to each question on question list.
i. When conducting analysis consideration should be given to five factors:
1. Words. Actual words and meanings of the words should be determined. One might make frequency counts of commonly used words. Cluster similar concepts together.
2. Context. Examine the context of words by finding the triggering stimulus and then interpret the comment in light of that context.
3. Internal consistency. Trace a flow of conversation and note changes or even reverses of position after interaction with others.
4. Specificity of responses. Responses that are specific and based on experiences should be given more weight than responses that are vague and impersonal. Greater weight should be given to responses in first person rather than third person hypothetical answers.
5. Find the big ideas. Look for rends or ideas that cut across the entire discussion.
Reporting
Once again, the objectives of the study should determine how and for whom the information needs to be reported. As a general rule, numbers and percentages are not appropriate for focus group research and should not be included in report. Reporting should be descriptive and present the meaning of the data as opposed to a summary of data.
Data can be examined and reported at three levels, including 1) the raw data, 2) descriptive statements, and 3) interpretation (Kreuger, 1988).
Raw data present statements as they were said by respondents. The data might be ordered or categorized by natural levels or themes in the topic.
Descriptive statements summarize respondents' comments and provide illustrative examples using the raw data. Decisions must be made as to which quotes to include.
Interpretation is most complex. Interpretation builds on the descriptive process by providing or presenting meaning of the data rather than simply summarizing the data. In giving meaning to the descriptions, one should be reflective about own biases in interpretation.
A VARIATION IN USING FOCUS GROUPS
One useful variation on focus groups is the Nominal Group Technique (NGT) suggested by Stewart and Shamdasani (1990). This technique allows you to take advantage of some of the benefits of focus groups under specialized conditions including:
1) uses with very specialized groups of people who can't easily be assembled on short notice (such as government officials, senior executives, etc.);
2) uses where there is a significant power differential among members (supervisors/subordinates, parents/children); or
3) uses when high levels of conflict or polarization over an issue exist, so that a minority/dissenting viewpoint might be inhibited.
"Nominal groups" are groups in name only. Participants do not interact and may not even meet.
During this technique, each member is interviewed separately (possibly by phone). Anonymous summaries of each person's perceptions/ideas are provided to each other member for comment. While it is not as spontaneous as a typical focus group session, this technique still allows participants to react to and be influenced by comments made by others.
ANNOTATED BIBLIOGRAPHY
1) Krueger, R. A. 1988. Focus groups: A practical guide for applied research. Newbury Park, CA: Sage Publications.
One of the few books that applies focus groups to evaluation research. This book gives considerable attention to analyzing and reporting focus group results. Detailed examples will be helpful to those who are just getting started in focus groups.
2) Morgan, D. L. 1988. Focus Groups as Qualitative Research. Newbury Park, CA: Sage Publications.
A good general guide to conducting focus groups. Translates marketing research techniques on conducting group interviews into a form more applicable for social sciences research.
3) Stewart, D. W., and Shamdasani, P. N. (1990). Focus groups: Theory and practice. Applied Social Research Methods Series, Vol. 20. Newbury Park, CA: Sage Publications.
No comments:
Post a Comment