HLSS500 Methodological Issues and Mixed Method Research Discussion Think about how you might design a mixed method research project. How would you collect

HLSS500 Methodological Issues and Mixed Method Research Discussion Think about how you might design a mixed method research project. How would you collect and code your data? Instructions: Fully utilize the materials that have been provided to you in order to support your response. Your initial post should be at least 500 words. 5 Methodological Issues in Conducting
Mixed Methods Research Designs
In: Advances in Mixed Methods Research
Edited by: Manfred Max Bergman
Pub. Date: 2011
Access Date: May 6, 2019
Publishing Company: SAGE Publications Ltd
City: London
Print ISBN: 9781412948098
Online ISBN: 9780857024329
DOI: https://dx.doi.org/10.4135/9780857024329
Print pages: 66-83
© 2008 SAGE Publications Ltd All Rights Reserved.
This PDF has been generated from SAGE Research Methods. Please note that the pagination of the
online version will vary from the pagination of the print book.
SAGE
SAGE Research Methods
2008 SAGE Publications, Ltd. All Rights Reserved.
5 Methodological Issues in Conducting Mixed Methods Research
Designs
Introduction
A need exists in the mixed methods literature to go beyond the types of designs available to researchers
and to begin exploring issues and strategies in conducting these designs. This chapter advances potential
concerns that need to be anticipated by researchers in conducting mixed methods research designs. It begins
by locating these concerns within two broad categories of designs (i.e. concurrent and sequential designs),
identifying potential concerns researchers might anticipate in using these designs, and citing published mixed
methods studies that illustrate not only the issues but also potential strategies, expressed or implied, for
addressing them. The methodological problems we explore relate to finding contradictory evidence between
quantitative and qualitative data, the integration of data, sampling, introducing bias, participant selection,
selection of results to use, and the sequence of implementing data. Understanding these issues and exploring
alternative strategies to address them will enhance our understanding of mixed methods procedures and
encourage rigorous, thoughtful designs.
In the literature on mixed methods research, considerable attention has been directed toward organizing and
classifying types of mixed methods designs (e.g. Creswell and Plano Clark, 2007; Greene et al., 1989; Morse,
1991; Tashakkori and Teddlie, 2003). In our review of these classifications, we have located 12 configurations
developed by authors from nursing, evaluation, public health, education, and social and behavioral research
(Creswell and Plano Clark, 2007). Although the names differ for the types of designs, two characteristics
emerge that are common to many classifications: either the purpose of the design is to merge (or bring
together) the qualitative and quantitative data in a parallel or concurrent way, or to have one type of data
(quantitative or qualitative) build on or extend the other type of data (qualitative or quantitative) in a sequential
way. These two major design options seem to hold whether the research is presented as a single study, such
as found in many doctoral studies, or in a multi-phase project, such as found in the evaluation literature and
in large-scale funded projects.
Our investigations examining the application of mixed methods in many disciplines resulted in the collection
of numerous studies that fit into this concurrent or sequential schemata (e.g. in family medicine, Creswell
et al., 2004; in counseling psychology, Hanson et al., 2005; in physics education, Plano Clark, 2005; and in
family science, Plano Clark et al., in press). As we reviewed these mixed methods studies, we found few
authors discussing potential methodological issues that might limit the findings of their studies. Discussing
procedural limitations that might influence the outcomes is not new in research methodology or in quantitative
or qualitative discussions. For example, in the classic treatise on experimental and quasi-experimental
designs, Campbell and Stanley (1966) identified 16 types of designs, and discussed eight threats to internal
validity and four threats to external validity. In qualitative research, Jacob (1988) classified the traditions of
Page 2 of 23
Advances in Mixed Methods Research
SAGE
SAGE Research Methods
2008 SAGE Publications, Ltd. All Rights Reserved.
qualitative research and discussed issues in the traditions, such as the foci of the types and their levels
of analysis. In addition, as we presented workshops and discussions about mixed methods research in the
U.S. and internationally, participants began raising questions about potential flaws or challenges that might
emerge in using different mixed methods designs. Building on these perspectives, we suggest that as mixed
methods emerges as a field of study and the types of designs become clearer to researchers, a discussion
about the methodological problems likely to arise in implementing these designs and potential strategies that
researchers might use to address them is timely.
As we wrote our recent book on mixed methods research (Creswell and Plano Clark, 2007), we began to
enumerate some of the challenges that would likely occur in different types of designs. Some methodological
problems were mentioned as we discussed each design; others we integrated into the data collection
and the data analysis discussions. Although authors have mentioned particular methodological problems,
such as sampling and contradictory findings, their work was not centered on particular mixed methods
designs (Collins et al., 2006; Erzberger and Kelle, 2003; Kemper et al., 2003; Teddlie and Yu, 2007; Trend,
1979).Thus, there was little information to help us recognize and address the issues inherent in the designs
within the mixed methods literature. We did not find any discussions in this literature that directly spoke to
these issues, aside from the general comment in Tashakkori and Teddlie (2003) that the design issues and
logistics in conducting mixed methods research were two unresolved issues.
The purpose of this chapter is to identify methodological problems and advance potential strategies for
planning and conducting mixed methods designs. Our intent is to present a pool of ideas from which
researchers can draw in their efforts to overcome methodological problems and craft rigorous studies. To this
end, we start with the issues that we specified previously (Creswell and Plano Clark, 2007), issues largely
based on conversations with U.S. and international researchers. A useful heuristic for thinking about these
methodological problems in designs occurred as we mapped the core ideas for this chapter: to organize
the issues and seek out strategies for addressing the issues using designs clustered into concurrent or
sequential designs. We therefore organized the issues into distinct themes, related them to either concurrent
or sequential designs, and looked extensively in published, empirical mixed methods articles reported in the
journal literature, for example that would illustrate the challenges as well as present potential strategies for
addressing them. As we reviewed the articles, we were also careful to identify challenges that we had not
previously anticipated.
Methodological framework
We discuss in our book four major designs, two of which can be conducted concurrently (Triangulation,
Embedded) and three that are conducted sequentially (Explanatory, Exploratory, Embedded) (see Figures 5.1
and 5.2) (Creswell and Plano Clark, 2007).
Page 3 of 23
Advances in Mixed Methods Research
SAGE
SAGE Research Methods
2008 SAGE Publications, Ltd. All Rights Reserved.
Figure 5.1 Concurrent mixed methods designs
The Triangulation Design is a one-phase design in which quantitative and qualitative data are collected and
analyzed in parallel and then merged together to develop a more complete understanding or to compare
the different results. Although this design is the most popular mixed methods design, it is also probably the
most challenging of the four major types of designs. Researchers use a second type of concurrent design,
an Embedded Design, when they want to enhance a study based on one method by including a secondary
dataset from the other method. This design is often used when researchers need to embed a qualitative
component during an intervention in an experimental study. In this case, the qualitative data are collected
concurrent with the implementation of the intervention, and the qualitative information typically focuses on
exploring how the experimental participants experience the intervention while the quantitative arm addresses
the outcomes of the trial (see Creswell et al., 2006, for additional reasons for embedding qualitative data
within experiments).
Page 4 of 23
Advances in Mixed Methods Research
SAGE
SAGE Research Methods
2008 SAGE Publications, Ltd. All Rights Reserved.
Figure 5.2 Sequential mixed methods designs
Turning to the sequential Explanatory, Exploratory and Embedded Designs, the quantitative and qualitative
data collection are implemented in different phases and are connected in some way. Researchers use the
Explanatory Design when they start with quantitative methods and then follow up with qualitative methods,
usually to help explain the initial quantitative results. Researchers using an Exploratory Design begin by
exploring the topic with qualitative methods and then build to a second quantitative phase where the
initial results may be tested or generalized. The sequential Embedded Design typically involves collecting
qualitative data before an intervention begins or after it is complete. When collected before the intervention,
researchers use the qualitative data to help recruit participants, to help test the treatment before the actual
experiment, or to select participants that can be best suited for the experimental and control conditions. When
collected after the intervention, the qualitative data help to explain why different outcomes resulted.
Method
These types of designs provided a framework for analyzing published mixed methods studies. We conducted
literature reviews using major databases including ERIC, PsycInfo, and PubMed. We located studies that met
our basic criterion of mixed methods research: studies in which the authors collected, analyzed, and merged
or connected both quantitative and qualitative approaches of data in a single study or in multiple studies in a
Page 5 of 23
Advances in Mixed Methods Research
SAGE
SAGE Research Methods
2008 SAGE Publications, Ltd. All Rights Reserved.
program of inquiry (Creswell and Plano Clark, 2007). Next, we classified these studies as either concurrent
or sequential designs. To accomplish this we closely examined the articles, paying particular attention to
the methods section to determine a) if the authors of the study included both quantitative (i.e. close-ended)
and qualitative (i.e. open-ended) data; b) how they integrated both sets of data (either concurrently or
sequentially); and c) if they mentioned the type of mixed methods design they were using. Based on these
factors, we identified nearly 160 articles spanning the last 15 years. The articles were drawn from fields in
the social sciences and the health sciences. Then, we examined two sections of the studies in depth: the
methods and the results/discussion sections to find occurrences of methodological issues. We were chiefly
interested in determining if the authors mentioned specific issues or challenges in carrying out their designs
and whether the authors identified procedures (or strategies) to address the issues. We began to organize
the issues and the solutions into a table that would provide practical suggestions for conducting each of the
four mixed methods designs and provide researchers with references they could use for their procedures (see
Table 5.1). We were also open to identifying issues not in our original schema.
Page 6 of 23
Advances in Mixed Methods Research
SAGE
SAGE Research Methods
2008 SAGE Publications, Ltd. All Rights Reserved.
TABLE 5.1 Problems and strategies for concurrent and sequential designs
Page 7 of 23
Advances in Mixed Methods Research
SAGE
SAGE Research Methods
2008 SAGE Publications, Ltd. All Rights Reserved.
Page 8 of 23
Advances in Mixed Methods Research
SAGE
SAGE Research Methods
2008 SAGE Publications, Ltd. All Rights Reserved.
Page 9 of 23
Advances in Mixed Methods Research
SAGE
SAGE Research Methods
2008 SAGE Publications, Ltd. All Rights Reserved.
Examples of issues and strategies
Our search of the literature yielded several issues and numerous strategies either implied or expressed
by authors. We discuss these issues and strategies for both concurrent and sequential designs, providing
examples drawn from the examined studies.
Concurrent design issues and strategies
We begin with a discussion of the concurrent designs and explore how authors of published mixed methods
articles addressed contradictory findings, integrated or combined the quantitative and qualitative data, chose
participants for the samples and the size of the samples, and limited the potential bias of data collection in
their designs.
Contradictory findings
This issue may emerge during a concurrent type of design when the quantitative and qualitative results do
not agree. Disconfirming findings may indicate flaws or inconsistencies in the research design. Erzberger
and Kelle (2003) explain that discrepancies between quantitative and qualitative data may be the result of
researcher’s errors in data collection and analysis or poor application of theoretical propositions. Divergent
findings, however, can also be thought of as a means to uncovering new theories or extending existing
theories.
Disagreement between the quantitative and qualitative strands may be a minor or a major difference in the
results. These differences may be difficult to resolve and may require the collection of additional data to
resolve the differences (Creswell, 2005). This strategy, however, raises a further question about what type of
additional data will be needed, analyzed, or given priority.
Padgett’s (2004) study recounts how a team of researchers returned to their initial database for more
insights after contradictory findings emerged. This occurred during the Harlem Mammogram Study, which
was funded by the National Cancer Institute to examine factors that influenced delay in response to an
abnormal mammogram among African-American women living in New York City. The research team had
collected both structured quantitative data and open-ended interview data. After data analyses, the team
concluded that the women’s decisions to delay were not driven by factors in their quantitative model. The
researchers then turned to their qualitative data, highlighted two qualitative themes, and reexamined their
quantitative database for support for the themes. To their surprise, the quantitative data confirmed what the
participants had said. This new information, in turn, led to a further exploration of the literature, in which they
found some confirmation for the new findings. Used in this way, researchers could also view the contradiction
as a springboard for new directions of inquiry (Bryman, 1988).
To resolve contradictory findings, authors may also give priority to one form of data over the other. Chesla
Page 10 of 23
Advances in Mixed Methods Research
SAGE
SAGE Research Methods
2008 SAGE Publications, Ltd. All Rights Reserved.
(1992) advised comparing the data from each strand by ‘weighing the evidence’ and deciding on priority by
determining which method was more fully developed (p. 684). Russek and Weinberg (1993) prioritized their
data in their study of the use of technology materials within elementary school classrooms. These authors
measured teachers’ anxiety levels in implementing lessons involving calculators and computers through
quantitative questionnaire data and qualitative interviews and observations. When these data were evaluated,
the results from the two strands were inconsistent. The authors reported that the two forms of qualitative data
had more validity than the one quantitative measure.
Data integration
It can be challenging to integrate two sets of different forms of data and their results in a meaningful way
(Bryman, 2007). Erzberger and Kelle (2003) offer researchers a series of guidelines on the integration of two
strands of data. Based on studies in the literature, three strategies are used to integrate data: (1) designing
and implementing comparable topics or questions for both arms; (2) transforming the data so that it can be
more easily compared; and (3) using matrices to organize both sets of data into one table.
The first strategy is to design both the quantitative and qualitative strands of the study to address the same
questions or concepts. In their effort to merge and corroborate two distinct datasets in a triangulation design,
Knodel and Saengtienchai (2005) offered a solution to facilitate data comparison. These authors studied
the caregiving and support systems available to adults with HIV or AIDS in Thailand between adult children
and their older parents. Both quantitative and qualitative data were collected through the administration of
structured, closed-ended surveys, and open-ended interviews with parents. The qualitative data permitted
elaboration of the quantitative survey measures. In order to simplify the merging of the qualitative and
quantitative data, these authors utilized a range of comparable question topics (health insurance, welfare
assistance, parental caregiving, support from others, economic support, and care of the children orphaned by
AIDS) for both strands of the study. In this way, the authors were better able to integrate the two sets of data.
A second data-integration strategy is to transform one form of data so that it can be more easily compared
with the other form (Caracelli and Greene, 1993). In general, it is more intuitive for researchers to quantify
their qualitative data by transformation than to transform their quantitative data into qualitative data. Witcher
et al. (2001) transformed their qualitative data into quantitative results in a study of pre-service teachers’
perceptions of characteristics of effective teachers by counting themes and calculating frequencies or
‘endorsement rates’ (p. 49). In this way, the authors determined which themes were more commonly
mentioned by teachers in their study. Crone and Teddlie (1995) carry this idea a step further by quantifying
their qualitative themes and then conducting statistical analyses on these data. These researchers gathered
data from teachers and administrators and then computed chi square tests on the themes in order to compare
data from schools with varying levels of effectiveness.
Another method for merging quantitative and qualitative data is to develop a matrix that combines the
information. The study by McEntarffer (2003) explored how 16 undergraduates in teacher education (eight
junior-senior mentors and eight freshman mentees) developed positive working relationships. Qualitative data
Page 11 of 23
Advances in Mixed Methods Research
SAGE
SAGE Research Methods
2008 SAGE Publications, Ltd. All Rights Reserved.
collection consisting of documents, interviews, and observations led to three qualitative themes. In addition,
the researcher collected quantitative information on individual strengths of the students through the Strengths
Finder instrument, developed by The Gallup Organization (Clifton and Anderson, 2002). McEntarffer created
a matrix that combined both forms of data by comparing individuals with certain personal strengths with
the qualitative themes that had been reported. In another study by Teno et al. (1998), the authors sought
to understand the barriers and aids of living wills in decision-making for hospitalized, seriously ill adults.
They collected quantitative data from medical record reviews and structured interviews with patients, their
surrogates, and physicians and qualitative data from narratives written by intervention nurses. They identified
14 cases for analysis and reported, in tab…
Purchase answer to see full
attachment