Research Critique please see question attached Contents lists available at ScienceDirect Nurse Education Today journal homepage: www.elsevier.com/locate/

Research Critique please see question attached Contents lists available at ScienceDirect

Nurse Education Today

journal homepage: www.elsevier.com/locate/nedt

Contributors to the development of intercultural competence in nursing
students☆

Esther Zazzi⁎

A R T I C L E I N F O

Keywords:
Intercultural
Diversity
Culture
Competence
Development
Nursing

A B S T R A C T

Nurses deal more effectively with cultural diversity when they have an ethnorelative orientation toward cultural
difference and commonality on the Intercultural Development Continuum, which was the theoretical framework
of this study. Scholarly literature shows limited knowledge on what fosters nurses’ intercultural development.
Thus, this quantitative, retrospective study was the first investigation in health care in Switzerland conducted on
nursing students’ orientation on the Intercultural Development Inventory (IDI). The sample consisted of the IDI
results from nursing students enrolled between 2010 and 2016 at the largest Nursing College in Switzerland
(N = 1.112 students in 40 cohorts) where the systematic integration of the development of intercultural
competence into the curriculum began in 2010. The aim was to examine to what extent the orientation toward
cultural difference and commonality on the Intercultural Development Continuum of nursing students in
Switzerland differs from the start- to the end-point of their nursing education. The paired samples t-test de-
monstrated a higher end of program developmental orientation (p = .01) compared to beginning developmental
orientation scores, suggesting the program provided challenges that met the students’ level of readiness for
cultural development. Educators can use the knowledge provided in this study and implement intercultural
competence as a central element within the curriculum following the example of the Nursing College included in
this study.

1. Introduction

Diversity is inherent in the Swiss health care as the population de-
mographics change. Several factors contributing to Swiss diversification
include an increase in foreign-born permanent residents, an increase in
a variety of religious beliefs among the population, and, an aging Swiss
population (Swiss Swiss Confederation, 2018a). Diversity is inherent in
the Swiss health care system not only in patients but also health care
providers (Obsan, 2016). For example, Swedish and Slovenian health
care professionals have exhibited a difference in teamwork, relation-
ships between professions, and responses to patient problems (Pahor
and Rasmussen, 2009). Cowan and Norman (2006) reported that mi-
grant nurses from other European Union countries employed in London
hospitals were perplexed by the diversity of cultures in the United
Kingdom. Nurses are required to care for a diverse population in the
health care setting where diversity is inherent also in health care pro-
viders.

Diversity may lead to marginalization and discrimination, which
may be accentuated in individuals who perceive they are different from
the majority, not only because of ethnicity and language but also factors
like age, gender, or disability. According to the Swiss Red Cross (2018),
unreflective generalizations and prejudices by health professionals
about individuals or whole groups of people prevent the view of their
actual problems, which affects treatment and care. The Institute of
Medicine (2002) reported that patients might be put at risk and even
die because of health care providers’ lack cultural competence that may
lead to delay in treatment or noncompliance with health care regimes.
McClimens et al. (2014) reported that nursing students struggle with
meeting the cultural needs of patients such as language, food, and
gender. Similarly, Schuessler et al. (2012) concluded that self-reflection
is necessary to be aware of cultural issues and address health care
disparities. Although health care professionals may not see themselves
as overtly racist or neglectful, they could be missing pertinent health
care findings due to cultural blindness (Seright, 2007).

https://doi.org/10.1016/j.nedt.2020.104424
Received 6 August 2019; Received in revised form 31 January 2020; Accepted 28 March 2020

Abbreviations: IDC, Intercultural Development Continuum; IDI, Intercultural Development Inventory; DO, developmental orientation toward cultural difference and
commonality; PO, perceived orientation toward cultural difference and commonality

☆ The author of this article has been employed as faculty by the College of Higher Education in Nursing included in this study since March 2008 and involved in the
training program including the students exchange program to foster the nursing students’ intercultural competence integrated into the curriculum. Furthermore, the
author was the Deputy Head of the Department International Relations and Diversity at the Nursing College from March 2014 until March 2018.

⁎ Corresponding author at: Sandmatten 308, 4618 Boningen, Switzerland.
E-mail address: esther@zazzi.eu.

Nurse Education Today 90 (2020) 104424

0260-6917/ © 2020 Elsevier Ltd. All rights reserved.

T

http://www.sciencedirect.com/science/journal/02606917

https://www.elsevier.com/locate/nedt

https://doi.org/10.1016/j.nedt.2020.104424

https://doi.org/10.1016/j.nedt.2020.104424

mailto:esther@zazzi.eu

https://doi.org/10.1016/j.nedt.2020.104424

http://crossmark.crossref.org/dialog/?doi=10.1016/j.nedt.2020.104424&domain=pdf

The International Council of Nursing (2013) stated that “nurses
should be culturally and linguistically competent to understand and
respond effectively to the cultural and linguistic needs of clients, fa-
milies, and communities in a health care encounter” (p. 1). Also, the
Swiss Federal Office for Public Health (Swiss Confederation, 2018b)
emphasized that health professionals need to be trained to care for
migrants living in Switzerland with their specific health-related needs
due to their origin or their living situation to ensure the quality of care
and patient safety. In summary, increasing the intercultural competence
in nurses is essential as nurses better perform with higher competencies.
Therefore, the nursing students’ development of intercultural compe-
tence needs to be understood.

2. Background/literature

2.1. Integration of the development of intercultural competence into the
curriculum

At the College of Higher Education in Nursing in Switzerland, the
development of intercultural competence is integrated into the curri-
culum within the school during the entire education since 2010. The
program consists of various parts by using the Intercultural
Development Continuum (IDC) by Hammer (2012, 2015) based on the
Developmental Model of Intercultural Sensitivity by Bennett (1986,
1993, 2004, 2011). Apart from the fact that teachers explicitly integrate
the intercultural perspective in several nursing subjects according to the
curriculum, each student participates in a specific training program of
16 lessons per semester using specifically intercultural training methods
such as simulations. Additionally, every nursing student mandatorily
completes a student exchange program in the last year of education. An
accurate reflective preparation (30 lessons) before the exchange and
reflection during and after the exchange is emphasized to increase their
intercultural competence (Vande Berg et al., 2009). The nursing edu-
cators who conduct these training sessions are continuously trained in
their competence to train intercultural competence.

2.2. Theoretical framework

The concept of intercultural competence has been explored in the
literature for the last five decades (Deardorff, 2015). Several strategies
to enhance intercultural competence in health care providers have been
shown in smaller empirical research studies in the last 15 years. The
studies applied various theoretical frameworks. The Intercultural De-
velopment Continuum (IDC) by Hammer (2012, 2015) based on the
Developmental Model of Intercultural Sensitivity by Bennet (1986,
1993, 2004, 2011) was the theoretical framework for this study. The
IDC focuses on the development of the intercultural competence de-
monstrating how people experience and engage cultural differences in a
continuum of experience extending from ethnocentrism to ethnor-
elativism. Ethnocentrism is the individuals’ experience of their culture
as central to reality, and ethnorelativism is the experience of cultures as
relative to context. These orientations toward cultural difference and
commonality describe sets of knowledge, attitudes, and skills and are
arrayed along a continuum from the more monocultural mindsets of
denial and polarization through the transitional orientation of mini-
mization to the intercultural or global mindsets of acceptance and
adaptation (Hammer, 2012) (Insert Fig. 1 here).

Hammer et al. (2003) used the term intercultural sensitivity to refer
to the ability to discriminate and experience the relevant cultural dif-
ference, and the term intercultural competence to mean the ability to
think and act in interculturally appropriate ways. Hammer et al. (2003)
argued that greater intercultural sensitivity is associated with a greater
potential for exercising intercultural competence. Intercultural com-
petence “reflects the degree to which cultural difference and com-
monality in values, expectations, beliefs, and practices are effectively
bridged, an inclusive environment is achieved, and specific differences

that exist in one’s organization are addressed from a mutual adaptation
perspective” (IDI, 2019, p. 3). Nurses with an ethnorelative orientation
toward cultural difference and commonality are able to better deal with
cultural diversity than nurses in an ethnocentric stage. According to
Hammer (2012), the capability of deeply shifting cultural perspective
and bridging behavior across cultural differences is most fully achieved
when one maintains an adaptation perspective. He defined the inter-
cultural competence as “the capability to accurately understand and
adapt behavior to cultural difference and commonality” (IDI, 2019, p.
3). Hammer’s (2012) definition of intercultural competence implying
Bennett’s (2001) definition of subjective culture as “the pattern of be-
liefs, behaviors, and values maintained by groups of interacting
people.” (p. 3) is included in this study, because it is coherent in dealing
more effectively with diversity.

The developmental stage can be measured by the Intercultural
Development Inventory (IDI) (Hammer et al., 2003; IDI, 2019) pro-
viding individual results for the developmental orientation (DO) and
the perceived orientation (PO) toward cultural difference and com-
monality. The difference between where the nursing students perceive
they are on the developmental continuum (PO) and where the IDI
places their level of intercultural competence (DO) is called the or-
ientation gap. Although the IDC does not provide concrete interventions
to generally apply within the practice, it provides indications of what
each individual need in which stage of development to stimulate his or
her process of becoming more interculturally competent.

2.3. Findings of empirical studies conducted in health care using the IDI

The findings of the empirical studies conducted in health care re-
porting the contributors to the proband’s development of intercultural
competence by using the IDI are similar. However, the studies vary
regarding intervention, duration, design, and scope. Most researchers
have developed their own training program. For example, Halm and
Wilgus (2013) developed a training program in a traditional classroom
setting focusing on Latino culture, including Spanish language/music,
to highlight critical reflections, cross-cultural practice, and commu-
nication standards for culturally competent care. In a different ap-
proach, Harder (2018) applied video and simulations with manikins to
train culturally competent care. There are studies on cultural immersion
but also various forms such as an in-service session on the end-of-life
care (Halm et al., 2012) or a 4-week-international placement (Peiying
et al., 2012). Researchers consistently emphasized the importance of
reflection. However, reflection has not been clearly and comprehen-
sively described, so interventions such as journal-writing or analysis of
experiences within peer-discussion are heterogeneous as well. With
such heterogeneity of interventions, the results may only be compared
with consideration.

The same needs to be stated for the duration of the interventions
and the time being devoted to the participants’ development of

Fig. 1. The Intercultural Development Continuum (Hammer, 2012).

E. Zazzi Nurse Education Today 90 (2020) 104424

2

intercultural competence. Some researchers have allowed a short
period of some weeks (e.g., Harder, 2018 or Peiying et al., 2012), but
other researchers have allowed 3 to 5 years (e.g., Huckabee and Matkin,
2012 or Boggis, 2012). There are also considerations of researchers on
how sustainable a certain development is. Researchers have debated
whether a current intervention may increase participants’ cultural
awareness only short term (Halm and Wilgus, 2013). Therefore, it re-
mains unclear what is needed for an individual to develop further along
the IDC having a lasting effect.

Researchers have investigated various health care professionals. The
only identified studies using the IDI in nursing have been conducted by
Halm and Wilgus (2013), Hamre (2012), Harder (2018), Kruse et al.
(2014), Larson (2011), and Munoz et al. (2009). Some of them in-
vestigated among nursing students, nursing staff, nursing faculty or a
mix of them. Bourjolly et al. (2015) and Halm et al. (2012) investigated
within multidisciplinary teams including nurses. Furthermore, all stu-
dies in the literature review were conducted in the United States, except
the study by Peiying et al. (2012) that was conducted in Australia and
the study by Harder (2018) that was conducted in Canada. No study
using the IDI conducted in other parts of the world such as Europe has
ever been published, including in fields outside of health care. There-
fore, empirical research about the orientation toward cultural differ-
ence and commonality in nursing is limited, particularly in Europe in-
cluding Switzerland.

2.4. Aim of this study

Although several research studies have been conducted, the
knowledge about what contributes to intercultural competence within
health care providers is limited. Horvat et al. (2014) concluded that
“further research is required to establish greater methodological rigor
and uniformity on core components of education interventions, in-
cluding how they are described and evaluated” (p. 2). The aim of this
study was to examine to what extent the orientation toward cultural
difference and commonality on the Intercultural Development Con-
tinuum of nursing students in Switzerland differs from the start- to the
end-point of their nursing education at a Nursing College.

3. Method

3.1. Study design and participants

This study is a quantitative and descriptive cross-sectional study and
included an ex post facto design having a similar logic of inquiry to
experimental research designs, meaning it is a substitute for experi-
mental research (Simon and Goes, 2013). The participants of this study
were nursing students at the largest College of Higher Education in
Nursing in Switzerland. The 1’112 nursing students integrated in this
study completed the specialized program integrated into the curriculum
at the Nursing College in Switzerland. The extent of the difference be-
tween the nursing students’ orientation toward cultural difference and
commonality at the start- and end-point of education was examined by
using the results of the nursing students’ IDI.

3.2. Data collection

The data consisted of the results of the nursing students’ IDI com-
pleted between March 2010 and September 2016 at the start- and the
end-point of their education lasting 2, 2.5, or 3 years (Insert Table 1
here). The IDI was selected for this study due to its demonstrated the-
oretical grounding on the Developmental Model of Intercultural Sen-
sitivity originally proposed by Bennett (1986, 1993, 2004, 2011). This
instrument allows users to assess the effectiveness of various cross-
cultural intervention by measuring the respondents’ change within the
development of their intercultural competence. A second consideration
for utilizing the IDI was that numerous reports have documented that

this assessment is cross-culturally generalizable, valid and reliable
(Hammer, 2008, 2010, 2011; Hammer et al., 2003; Paige et al., 2003;
Wiley, 2017). Content validity of the original 60-item IDI was estab-
lished through in-depth interviews with individuals from a variety of
cultures. Evaluations by a panel of experts, followed by survey pilo-t-
testing, reduced the original scale from 60 to 50 items that cross five
factors: denial/defense, reversal, minimization, acceptance/adaptation,
and encapsulated marginality. Reliability coefficients of all five scales
ranged from 0.80–0.085 (Hammer et al., 2003).

3.3. Data analysis

Paired samples t-tests have been computed using SPSS to compare
the means score of the students’ perceived and developmental or-
ientation toward cultural difference and commonality (PO and DO) at
the start- and end-point of their education based on the condition that
every nursing student completed the specialized program integrated
into the curriculum at the Nursing College in Switzerland (Frankfort-
Nachmias and Leon-Guerrero, 2015). Because theoretical reasons for
specifying a direction in the research hypothesis is lacking, a two-tailed
test has been conducted. Although the intention of the curriculum at the
college is to foster the development of the students regarding their DO
there may be retreats from some stages (Bennett, 2011). To provide an
interval estimate rather than a point estimate, the 95% CI of the dif-
ference has been included in the paired-samples t-test (Pallant, 2016).
The threshold for the p-value was set at 0.05, meaning that the result of
the t-test was considered significant if the obtained statistic was less
than or equal to 0.05.

3.4. Ethical considerations

For the conduction of this study, the IDI-results of nursing students
who provided their data anonymously was used. The students were
randomly assigned to a login that does not relate to the person.
Therefore, the data represent a de-identified dataset. This procedure
was consistent with the Swiss regulations regarding data protection by
the Federal Data Protection and Information Commissioner including
the Cantonal Data Protection Act (Swiss Confederation, 2018c). By
these regulations, the management of the Nursing College provided the
data to the author for this research study.

4. Results

The majority of the 1.112 nursing students in 40 cohorts was either
18–21 years (49%) or 22–30 years (41%) old. The age of 4.5% of the
students was 31–40 years, and 3.5% of the students were 41–50 years
old. The remaining 2% of the students were either 17 years respectively
51–60 years old. More than half of the students (63%) were female
while 37% of the students were male. The participants’ mindsets were
identified according to the scores of the PO and the DO as follows (IDI,
2019): scores 55.00–65.99 = denial; 66.00–69.99 = cusp of polariza-
tion; scores 70.00–82.99 = polarization; scores 83.00–84.99 = cusp of
minimization; scores 85.00–111.99 = minimization; scores
112.00–114.99 = cusp of acceptance; scores 115.00–126.99 = accep-
tance, scores 127.00–129.99 = cusp of adaptation; and scores
130.00–145.00 = adaptation.

The paired samples t-test demonstrated an increase in the mean
score of DO and a decrease of the mean score of PO at the end-point
compared to the start-point of the education. The mean score for the
nursing student-cohorts’ PO was 119.5 (SD 1.43, std. error mean
0.22613) at the start-point and 119.35 (SD 1.48, std. error mean
0.23353) at the end-point of their education with a 95% CI between
−0.75 and 0.15 and a p-value of 0.19. The mean score for the nursing
student-cohorts’ DO was 86.33 (SD 3.83, std. error mean 0.60496) at
the start-point and 87.99 (SD 4.15, std. error mean 0.65577) at the end-
point of their education with a 95% CI between −2.95 and − 0.38 and

E. Zazzi Nurse Education Today 90 (2020) 104424

3

a p-value of 0.01. As the threshold was set by 0.05, the nursing student-
cohorts’ orientation toward cultural difference and commonality as a
group significantly differed from the beginning to the end of their
education regarding the DO (p = .01) but did not significantly differ
regarding the PO (p = .19). (Insert Table 2 here).

5. Discussion

Although, this study reported a statistically significant increase of
the nursing students-cohorts’ DO between time one (pretest) and time

two (posttest), this result needs to be considered carefully in compar-
ison with other studies which did not report significant results re-
garding the increase of the participants DO.

Altshuler et al., 2003 reported that pre- and post-training IDI scores
were analyzed using the three-factor model and no significant differ-
ences were found for either of the two training groups. Boggis (2012)
presented that group with the occupational therapy students did not
show a significant change in overall DO mean scores from pretest to
posttest (t = 0.847, p = .41) while the control group showed a sig-
nificant decrease in DO mean scores at posttest (t = 6.1, p < .001). Table 1 Number of IDI in each cohort including time-point. Cohort Date Start-point End-point IDI Students Response rate Date IDI© Students Response rate 40 Sep 10 40 40 100 March 13 12 36 33.3 41 Sep 10 30 36 83.3 March 13 24 29 82.8 42 Sep 10 43 44 97.7 March 13 28 40 70 43 Sep 10 43 43 100 March 13 33 39 84.6 1 Sep 10 29 29 100 Sep 13 22 22 100 6 Sep 10 43 43 100 Sep 13 29 46 63 7 Sep 10 38 38 100 Sep 13 25 29 86.2 8 Sep 10 49 49 100 Sep 13 39 40 97.5 2 March 11 25 26 96.2 March 14 20 21 95.2 9 March 11 33 43 76.7 March 14 19 39 48.7 44 Sep 11 32 43 74.4 March 14 35 43 81.4 45 Sep 11 42 43 97.7 March 14 31 39 79.5 46 Sep 11 35 37 94.6 March 14 35 37 94.6 3 Sep 11 22 28 78.6 Sep 14 23 23 100 10 Sep 11 50 53 94.3 Sep 14 44 50 88 11 Sep 11 37 40 92.5 Sep 14 28 35 80 4 March 12 22 22 100 March 15 17 18 94.4 12 March 12 33 35 94.3 March 15 31 37 83.8 47 March 12 22 22 100 Sep 14 24 27 88.9 48 Sep 12 39 39 100 March 15 28 34 82.4 49 Sep 12 37 37 100 March 15 35 39 89.7 50 Sep 12 38 39 97.4 March 15 33 37 89.2 13 Sep 12 44 56 78.6 Sep 15 44 50 88 14 Sep 12 40 40 100 Sep 15 21 32 65.6 37 Sep 12 13 13 100 Sep 16 12 13 92.3 15 March 13 17 17 100 March 16 18 18 100 16 March 13 26 27 96.3 March 16 26 27 96.3 51 March 13 21 21 100 March 15 20 21 95.2 52 March 13 22 22 100 March 15 16 18 88.9 17 Sep 13 47 47 100 Sep 16 35 44 79.5 18 Sep 13 48 48 100 Sep 16 41 44 93.2 53 Sep 13 33 37 89.2 Sep 15 34 37 91.9 54 Sep 13 29 29 100 Sep 15 27 29 93.1 55 Sep 13 29 29 100 Sep 15 25 26 96.2 56 Sep 13 39 39 100 Sep 15 35 38 92.1 57 March 14 24 24 100 March 16 21 24 87.5 58 March 14 25 25 100 March 16 20 26 76.9 59 Sep 14 41 41 100 Sep 16 31 35 88.6 60 Sep 14 43 44 97.7 Sep 16 36 42 85.7 61 Sep 14 43 43 100 Sep 16 35 39 89.7 Total 1366 1431 96.0 1112 1323 85.3 Table 2 Paired samples T test of perceived and developmental orientation at start compared to end of education. Paired Differences t df Sig. (2-tailed) Mean Std. deviation Std. error mean 95% confidence interval of the difference Lower Upper Mean difference score of PO at the start-point and end-point of the education −0.30 1.41 0.22 −0.75 0.15 −1.34 39 0.19 Mean difference score of DO at the start-point and end-point of the education −1.66 4.03 0.64 −2.95 −0.38 −2.61 39 0.01 E. Zazzi Nurse Education Today 90 (2020) 104424 4 Halm and Wilgus (2013) found out that the scores did not change significantly from pretest (M = 97.2) to posttest (M = 95.91). Halm et al. (2012) conducted the t-tests, which showed no significant dif- ferences between pre- and the posttest scores (p > .05). Hamre (2012)
identified that both included groups experienced higher scores on their
PO as well as their DO, but by using t-tests neither the difference for the
PO (p = .88) nor the DO (p = .94) was statistically different. Harder
(2018) observed a 3.38 gain in the score while both the score at the pre-
and at posttest place the groups’ development orientation within the
mindset of minimization without reporting any significance level.
Peiying et al. (2012) reported that the DO score increased from pretest
(M = 92.91) to posttest (M = 97.22) but did not reach statistical sig-
nificance (p = .10).

The fact that this study identified a significant increase of the DO
(p = .012) from the start- to the end-point of the education might be
related to the consequently integrated approach at the Nursing College.
The findings of this study may be generalized to other nursing students
in Switzerland. With about 400 nursing students every year the Nursing
College where the nursing students included in this study completed
their education is the largest college of higher education in nursing in
Switzerland (Insert Table 3 here). However, other colleges have not
implemented the development of the intercultural competence as ex-
tensively into their curriculum like the Nursing College. Particularly,
the compulsory student exchange program in the last year of education
implemented at the Nursing College since 2010 is unique in Switzer-
land. As this study did not include experimental design, there are no
particular statistical results regarding the relationship between PO/DO
and any specific interventions. The comparison between the results of
this study and those of the other studies needs to be considered care-
fully because the studies are heterogenous particularly regarding in-
tervention and duration.

Most researchers developed their own training program with var-
ious content and design including duration. Some researchers devel-
oped various educational training programs to conduct in a classroom
(Altshuler et al., 2003; Bourjolly et al., 2005: Halm and Wilgus, 2013).
Harder (2018) implemented experimental learning by using video and
simulation. Other researcher implemented immersion-interventions
such as in-service session on the end-of-life care (Halm et al., 2012) or a
4-week international placement (Peiying et al., 2012). Huckabee and
Matkin (2012) combined several approaches by applying didactic in-
struction, learning activities, group projects, and supervised patient
care at a free clinic for the homeless. Similarly, Altshuler et al. (2003)
applied a workshop and a culture OSCE (objective structured controlled
evaluation). Most researchers used reflective learning methods such as
journal-writing and analysis of experiences, those researchers using a
mixed method design retrieved the qualitative data from those writings
(Bourjolly et al., 2005). Some researchers did not apply any interven-
tions as the study design was descriptive (Hamre, 2012; Kruse et al.,

2014; Larson, 2011).
Because the time between the pre- and the posttest was different

according to the various duration of the interventions the duration of
the studies was heterogeneous. There are researchers who assessed the
participants’ orientation toward difference and commonality twice
within a short time period of some weeks (Halm et al., 2012; Halm and
Wilgus, 2013; Harder, 2018; and Peiying et. Al, 2012). Other re-
searchers investigated over a longer period of time such as Huckabee
and Matkin (2012) over a five-year period and Boggis (2012) over a
three-year period. Although Munoz et al. (2009) also applied an in-
tervention, they assessed the intercultural competence only once more
or less at the beginning of the intervention. In summary, there is a
variety of study-duration and interventions applied in studies. There-
fore, the results of this and other studies within health care using the
IDI may be considered carefully as the heterogeneity needs to be seen as
a limitation.

The difference between PO and DO was higher than identified in
other studies. However, the PO decreased from the start- to the end-
point of education, but not significantly. A high orientation gap may
influence the students’ level of participation in interventions because
they think they were doing much better than they really were (Boggis,
2012). The students might feel a limited challenge resulting in no
progression to an ethnorelative mindset. It is essential that educators
provide training programs meeting the students’ level of readiness for
cultural development.

5.1. Limitations

The timing when students completed the IDI might haven threa-
tened the reliability by affecting the PO and DO scores of the students.
At the start-point of their education, they might not have understood
very well the term culture although it is explained within the IDI. The
fact that students completed the IDI at the very last day of their edu-
cation when students are excited and ready to head out the door would
diminish its importance in the students’ view and thus affect results.
Particularly. the findings of posttest PO scores not being significant and
DO sores being significant need to be considered carefully. Therefore,
the timing of pre and posttests should be customized in examination of
this concept.

As a threat to the …