Evaluating Military Cross-Cultural Training Programs

Allison Abbe, PhD

23 September 2021
https://doi.org/10.36304/ExpwMCUP.2021.06

Abstract: Increased emphasis on stability operations, counterinsurgency, and security cooperation during the conflicts in Afghanistan and Iraq resulted in programs to train and educate U.S. military personnel in foreign cultures and intercultural competence. Now, with the shift to great power competition, the Services have reduced or eliminated cultural training and education requirements. Documenting the approaches and lessons from these programs is important to maintain an institutional record for the future, if and when the United States sees the need to better understand the foreign cultures with which and in which its military operates. The present study applied a framework for qualitatively evaluating military cross-cultural training programs based on training science.

Keywords: cultural training, professional military education, cultural competence, training evaluation, curriculum design, cultures

The U.S. Department of Defense (DOD) made a series of strategic shifts in responding to the threat of global terrorism in the early 2000s. Amid the conflicts in Iraq and Afghanistan, the 2004 The National Military Strategy placed increased emphasis on stability operations and security cooperation.¹ Both the Defense Planning Guidance for fiscal years 2006–11 and the 2006 Quadrennial Defense Review identified the importance of foreign language and cultural capabilities for general-purpose forces (GPFs), both for countering terrorism in the information domain and for working with allies and partners.² These shifts had significant implications for military training and education, with professional development of cultural skills emerging as a strategic necessity. Although such instruction was previously available for special operations forces (SOFs) and some other specialties, extending it to GPFs across all Services in both professional military education and training was new territory for training and professional development.

Determining how to best structure and resource these programs given competing demands for training time and funding was a significant challenge for the military training and education enterprise.³ Today, with the United States’ shift to great power competition, the military Services have reduced or eliminated cultural training and education requirements.⁴ Documenting the approaches and capturing the lessons from these programs is important to maintain an institutional record for the future, if and when the United States sees the need to better understand the foreign cultures with which and in which the military operates.

This article provides one source of such documentation for future program design in the form of a qualitative analysis of cultural training programs’ adherence to evidence-based training practices. First, the article outlines the strategic guidance, demand signals, and milestones in the establishment of military cultural training programs. Second, it examines the challenges associated with evaluating training in organizations and some specific challenges for cross-cultural training (CCT). Third, a proposed framework for evaluating CCT programs combines the science of training with previous research on CCT. Next, the method and findings of the present study offer insights into which elements of the framework consistently emerged in CCT programs across the Services. The article ends with recommendations for improving CCT, facilitating training transfer, and advancing future research. Integrating these lessons from military CCT programs will help enable an organizational learning culture within the DOD and meet the cultural capability requirements of the future.

The military has a strong organizational culture and institutional structure for training and professional development.⁵ Therefore, when organizational performance needs emerge in the human domain, training and education are often viewed as prime solutions, relative to other potential interventions.⁶ The culture and structures enable the DOD to proliferate training and education programs as needed, but determining the effectiveness of programs and their alignment of those programs with Service and defense strategy is another matter. Given the institutional preference for training and the importance of training to achieving strategic ends, studies and methods to evaluate training and education programs are an important tool for enterprise decision making.

Developing Cross-Cultural Training Programs

Demand signals clearly emerged after Operation Iraqi Freedom began in 2003 to improve preparation for irregular warfare. By 2004, there was widespread recognition of the cultural readiness gap due to the convergence of both bottom-up and top-down indicators. In testifying before the U.S. House Armed Services Committee in 2004, retired U.S. Army major general Robert H. Scales Jr. quoted a brigade commander from the 3d Infantry Division in stating: “I knew where every enemy tank was dug in on the outskirts of Tallil [Iraq],” he replied. “Only problem was my soldiers had to fight fanatics charging on foot or in pickups and firing AK 47s and RPGs [rocket propelled grenades]. I had perfect situational awareness. What I lacked was cultural awareness. Great technical intelligence . . . wrong enemy.”⁷ Reinforcing the feedback from commanders and servicemembers, the Defense Planning Guidance for fiscal years 2006–11 provided a strong top-down signal, identifying the need for foundational foreign language and cultural capabilities in the GPF.⁸ In 2007, under the leadership of Undersecretary of Defense for Personnel and Readiness David S. C. Chu, the DOD held a summit on developing regional and cultural capabilities in military personnel. This event sparked a series of actions to plan for, measure, and operationalize cultural capabilities.⁹

Host nations and international partners recognized similar gaps. For example, in Afghanistan, the Afghan Ministry of Defense began providing cultural awareness materials to its own forces about certain actions and behaviors of U.S. and Coalition servicemembers that could be perceived as cultural offenses, aiming to prevent grievances and potential insider attacks given the continuing lack of cultural sensitivity of Western armed forces.¹⁰ International officers participating in U.S. military education programs have also noted a lack of knowledge of and curiosity about other cultures among U.S. servicemembers.¹¹ Previous research in European nations has similarly identified the need for CCT.¹² Any nation deploying its armed forces for operations involving interaction with and among other cultures faces a similar demand, including for counterinsurgency operations, peacekeeping and stability operations, security force assistance, and multinational operations.¹³ Consequently, CCT is an issue of shared multinational interest.

A number of organizational adaptations emerged within the DOD to address the cultural readiness gaps, including new doctrine and policy, organizational structures, and training and education programs, with some key differences across Services.¹⁴ For example, the U.S. Army relied more heavily on contracted external experts than the other Services, in the form of the Human Terrain System and, to a lesser extent, the culture and foreign language advisors at Training and Doctrine Command (TRADOC) centers of excellence.¹⁵ Whether to draw on external expertise or to develop in-house capability was a matter of controversy and debate.¹⁶

However, training and education showed some similarities across the enterprise (see significant milestones in table 1). Each of the military Services adopted strategic plans for developing and managing language, regional expertise, and culture (LREC) capabilities. Each of those plans included both a culture- or country-specific component and a culture-general component. Culture-specific approaches provided training for a specific country or operation, while a culture-general component addressed knowledge, skills, and capabilities that spanned different countries and regions, improving the ability of servicemembers to operate effectively in any intercultural setting.¹⁷

To varying degrees, the Services adopted this approach for both the GPF and SOFs, and department-wide policy subsequently included culture-general requirements in both a Chairman of the Joint Chiefs of Staff Instruction (CJCSI) and a Department of Defense Instruction (DODI).¹⁸ In the Army, culture-general capability is defined as cross-cultural competence, which is a general awareness of cultural concepts, self-awareness of one’s own culture, and skills to interact effectively with members of other cultures.¹⁹ The U.S. Marine Corps defines culture-general as “concepts and skills for thinking and acting that are transferable from one area of operations to another.”²⁰

Table 1. DOD cross-cultural training and education milestones

2004	The Defense Planning Guidance for FY 2006–11 identifies a need for foundational language and cultural capabilities in the general-purpose forces (GPFs).^A The U.S. Army establishes the Training and Doctrine Command (TRADOC) Culture Center.
2005	The Defense Language Transformation Roadmap (DLTR) sets the goal of foundational language and cultural expertise in the GPFs.^B The National Defense Authorization Act establishes the Defense Language Office within the Office of the Secretary of Defense to implement the DLTR.
2006	The U.S. Marine Corps establishes the Center for Advanced Operational Culture Learning (CAOCL). The U.S. Army publishes Counterinsurgency, Field Manual 3-24, with sociocultural themes throughout.^C The U.S. Air Force establishes the Air Force Culture and Language Center. The U.S. Navy establishes the Center for Language, Regional Expertise, and Culture. The U.S. Special Operations Command’s Capstone Concept for Special Operations notes that “Joint SOF cultural and language preparation needs to become global in scope.”^D
2007	The Joint Operating Concept for Irregular Warfare identifies the need for GPFs to receive cultural and foreign language instruction.^E
2008	The U.S. Navy adopts the Language Skills, Regional Expertise, and Cultural Awareness Strategy.^F
2009	The U.S. Army and Air Force adopt culture and foreign language strategies.^G The U.S. Special Operations Command publishes language, regional expertise, and culture (LREC) requirements.
2010	The U.S. Marine Corps adopts LREC strategy and implements the Regional, Culture, and Language Familiarization Program.
2011	The DOD publishes the Strategic Plan for Language Skills, Regional Expertise, and Cultural Capabilities.
2012	U.S. Special Operations Command components complete LREC needs assessments.
2013	Chairman of the Joint Chiefs of Staff Instruction 3126.01A establishes culture and regional competencies, enabling U.S. Combatant Commands to identify LREC capability requirements.^H The Special Operations Forces Operating Concept identifies cultural and regional expertise as a critical capability.^I
2016	Department of Defense Instruction 5160.70 establishes cultural learning guidelines.^J
2020	The U.S. Marine Corps decommissions CAOCL.
2021	The U.S. Army disestablishes the TRADOC Culture Center.

^A Defense Planning Guidance (Washington, DC: U.S. Department of Defense, 2006–11).
^B Defense Language Transformation Roadmap (Washington, DC: U.S. Department of Defense, 2005).
^C Counterinsurgency, Field Manual 3-24 (Washington, DC: Headquarters, Department of the Army, 2006).
^D Capstone Concept for Special Operations (Washington, DC: U.S. Special Operations Command, 2006).
^E Joint Operating Concept for Irregular Warfare (Washington, DC: Department of Defense, 2007).
^F U.S. Navy Language Skills, Regional Expertise and Cultural Awareness Strategy (Washington, DC: Chief of Naval Operations, 2008).
^G Army Culture and Foreign Language Strategy (Washington, DC: Headquarters, Department of the Army, 2009); and Air Force Culture, Region, and Language Flight Plan (Maxwell Air Force Base, AL: Air Force Culture and Language Center, 2009).
^H Chairman of the Joint Chiefs of Staff Instruction 3126.01A, Language, Regional Expertise, and Culture (LREC) Capability Identification, Planning, and Sourcing (Washington, DC: Joint Chiefs of Staff, 2013).
^I Special Operations Forces Operating Concept (MacDill Air Force Base, FL: U.S. Special Operations Command, 2013.)
^J Department of Defense Instruction 5160.70, Management of the Defense Language, Regional Expertise, and Culture (LREC) Program (Washington, DC: U.S. Department of Defense, 2016).
Courtesy of the author, adapted by MCUP.

Following initial adoption of the LREC strategies, the Services incorporated LREC capabilities into other personnel policy documents, facilitating longer-term organizational adaptation. Carrying out their Title 10 responsibilities to train the force, the Services adopted different methods to develop culture-general capabilities in their personnel, using a combination of training and education.

The Services had previously confronted similar challenges in the Vietnam War era. In the 1950s, U.S. advisors struggled to adapt their training and organization methods to conditions confronting the Army of the Republic of Vietnam, resulting in a conventional force ill-prepared for counterinsurgency.²¹ The U.S. Army and Navy subsequently developed and offered cultural training. Lessons from Vietnam-era research were consulted in the development of cultural training during the present period of study, but methods from that era were not widely adopted across Services.²²

The Science of Training

An oft-repeated aphorism is that training prepares personnel for certainty, whereas education prepares for uncertainty. The distinction often comes down to either what outcomes instruction targets or where learning occurs. As an example of the former—the what distinction—the U.S. Army distinguishes training in terms of its intended outcomes of preparing personnel to perform specific functions and tasks, and considers education to develop a broader range of outcomes, primarily in cognitive and affective learning domains.²³ In contrast, research literature often uses the where distinction, focusing on the setting where instruction occurs. Although learning occurs in varied settings, research on education often focuses on formal instruction offered within an educational institution, whereas training may occur in a variety of organizational, occupational, and educational settings.

The science of training focuses primarily on learning in organizational settings; however, the distinction between training and education is an imperfect one, as large organizations like the DOD often offer both. Training and education are expected to enable and improve servicemember performance. For present purposes, CCT refers to any systematic institutional effort to develop cultural knowledge, skills, or capabilities in personnel, regardless of the context of delivery. After significant investment in conceptualizing and teaching cultural-general capabilities to military personnel, the design and impact of those efforts are less well-documented.²⁴ Evaluating military CCT is critical to ensuring its effectiveness and informing resource decisions. Without training evaluation, organizational leaders are forced to make blind decisions, risking continued investment in ineffective training methods or programs. Conversely, organizations may also cut support for training programs that have positive impacts to operational performance. Given the scale of training and education in DOD, training evaluation and alignment are critical aspects of managing resources for personnel and readiness.

Organizational Challenges in Training Evaluation

The most widely recognized model of training evaluation in human resource development is the Kirkpatrick model, which includes four levels of evaluation: trainee reactions, learning, behavior, and results.²⁵ The first three of these are measured at the individual level. Reaction and learning can be readily measured within the training context itself. Behavior-level evaluation assesses the extent to which trainees transfer skills and competencies learned during training to on-the-job performance, or at least display the behaviors expected in future job performance. Some researchers have divided the behavioral level into training performance and transfer performance. Results-level evaluation assesses the extent to which training produces the desired organizational outcomes. Results may be measured at the individual, team, or organizational level, depending on the targeted outcomes of training.

Advances in the science of training during the past 20 years have enriched understanding of training design and evaluation and multilevel organizational factors in training.²⁶ Nonetheless, training evaluation has continued to focus primarily on the lower levels of the Kirkpatrick model. Despite calls for theory-driven evaluation, some researchers have noted that organizations still measure only trainee reactions.²⁷ In a 2016 Association for Talent Development survey, 88 percent of respondents indicated their organization relied on trainee reactions, which showed no change from levels reported in 2009.²⁸ Measures of learning and transfer would provide the most direct means to evaluate training, but they are less frequently included. Assessment of these outcomes may be limited or impractical in some settings.

Military organizations face a number of challenges in evaluating training programs by assessing behavioral outcomes. First are measurement challenges. Despite ongoing improvements in performance measurement, military and other government organizations may lack reliable metrics for assessing performance within the unit or organization context.²⁹ Performance outcomes may not readily lend themselves to quantitative metrics. The impact of training can therefore sometimes be difficult to demonstrate empirically.

In addition, transfer may occur at a substantial time lag from individual training and education events.³⁰ Military units do not conduct their mission-essential tasks on a day-to-day basis in garrison, but rather at specific time points governed by deployment or training schedules and other considerations. Therefore, opportunities to observe behavioral transfer may be distanced in time and space from individual training programs, potentially hindering the alignment of outcome evaluation with training resource decision timelines. For example, a unit may undergo CCT three to six months prior to deployment. Once deployed, assessing the unit’s performance may depend on local conditions and objectives, and performance measurement may therefore not be standardized or comparable across units. Even assuming the feasibility of consistent metrics for unit performance, observer access to assess performance may be very limited or impossible under operational conditions.

Second, resource constraints may limit training programs from measuring training outcomes consistently or systematically. Staff time is not unlimited, and developing outcomes-based measures and evaluation plans can divert resources from training design and delivery. For CCT, the Services and DOD science and technology programs made very limited investment in developing assessments for individual learning and performance, investing instead in advancing training technologies and sociocultural computational modeling.³¹ In addition, staff may lack expertise in learning assessment and program evaluation. In a survey of nonmilitary organizations, half of the respondents indicated challenges with attracting and retaining staff who are qualified to guide training evaluation.³²

Third, decentralized training structures may limit evaluation options. The U.S. military and other large government organizations increasingly rely on external vendors and contract support to deliver training.³³ In addition, training may occur across multiple dispersed units and locations. This decentralization is particularly problematic as it limits the ability to conduct controlled experimental designs that enable causal analysis. Decentralization in where and how training and education are conducted, as well as who conducts delivery, is a challenge not only for CCT but also for other training and education domains in large organizations. Decentralization adds variables that evaluation design may not be able to fully control or account for. As a result of these challenges, evaluation of training programs must employ a broader set of methods than focusing only on learner outcomes.

Challenges in Cross-Cultural Training Evaluation

Cross-cultural training in particular would benefit from improved training evaluation. Two meta-analyses have statistically analyzed the results of CCT studies and shown positive effects of CCT on both learning and behavioral transfer outcomes.³⁴ However, these analyses indicated that CCT evaluation has suffered from a lack of rigor.³⁵ These limitations have led to continuing questions about the benefits and design of CCT. As a result, evaluation of military CCT has had a very limited research foundation from which to draw. Given that CCT programs were established in the DOD within a two-year period in response to operational demand, opportunities were limited to build evaluation into program design from the beginning. Prior research did not offer a clear set of recommendations for evaluation that fit the military CCT context.

For example, relying on trainee reactions or self-reported learning to evaluate CCT, as might be recommended in the first levels of the Kirkpatrick model, can be problematic. Some theorists propose that intercultural learning and adjustment do not follow a simple linear pattern over time.³⁶ One outcome of training may be an increased awareness of one’s limited understanding of other cultures. Further, for military personnel, existing models of intercultural learning may not directly apply, as they may not sufficiently account for the impact of cultural stress and negative intercultural experiences common in conflict settings.³⁷ For example, a servicemember who carries assumptions that cultures may differ in behavior but share underlying values may have an initial negative reaction to training or an experience that counters the idea of the universal value of individual autonomy. With additional experience, however, that servicemember may develop more complex mental models to accommodate differences in cultural values.

Training may therefore elicit different reactions depending on the trainee’s previous experiences or current stage of intercultural development. Consistent with William S. Howell’s notion of moving from unconscious to conscious incompetence, one study found a decrease in cultural intelligence (CQ) after cultural awareness training.³⁸ Student feedback indicated that the intervention made them more aware of their knowledge gaps, suggesting support for a shift from “unconscious incompetence” to “conscious incompetence” as outlined in Howell’s model. Such declines in CQ may sometimes be a desirable outcome of training, if accompanied by a motivation to learn more. Nonetheless, declines suggest a need for further training and can be difficult to interpret and explain when outcomes-based training evaluation does not follow up with additional measures over time.

The developmental model of intercultural sensitivity provides one way to conceptualize development over time, with intercultural learning hypothesized to shift from ethnocentric views characterized by denying or defending against cultural differences toward more ethnorelative views. In this model, ethnorelativism includes acceptance of and adaptation to cultural differences. The assessment tool based on this model, the Intercultural Development Inventory (IDI), has been used in some military samples—for example, showing decreases in defense and denial of cultural differences among U.S. Military Academy (West Point) cadets after a semester abroad.³⁹ In another military education setting, the IDI showed improvement in response to instruction, but it yielded some ambiguous results given predictions based on the theory.⁴⁰

The U.S. Military Academy study suggests the IDI has potential utility for longer training and education interventions. However, CCT evaluation is limited by a shortage of research indicating when a particular measure is appropriate. Few theory-based or standardized outcome measures are available. Intercultural adjustment and CQ are two commonly used criteria. Intercultural adjustment includes work adjustment, relational adjustment in interacting with host nationals, and general or personal adjustment.⁴¹ CQ consists of four dimensions needed to navigate cultural differences: cognitive CQ, metacognitive CQ, behavioral CQ, and motivational CQ.⁴² Both intercultural adjustment and CQ are easy to measure with self-report scales, but they may not always align with a CCT program’s logic model and learning objectives.

A logic model describes how a program combines inputs, like time and resources, with processes to improve performance.⁴³ Whether by design or by assumption, training and other performance improvement programs hold underlying causal hypotheses. The logic model helps align the evaluation with the program’s causal hypotheses. It also provides a means to communicate those assumptions. Without articulation of the logic model, program design and program evaluation can be disconnected such that the evaluation methods may not be suited to the causal assumptions inherent to the program.

For example, one study reported providing CCT in a one-hour lecture format, which included knowledge of Iraq, cultural awareness comparing perspectives of the United States to those of other cultures, and the ability to function in a dissimilar culture.⁴⁴ This training was part of a nine-day course for contracting personnel (military and civilian). Although results showed increases in both cognitive and behavioral CQ post training, this increase followed didactic training of short duration with no opportunities for skill demonstration or practice. Theories of cognitive and behavioral skill acquisition would likely not predict substantial behavioral change after such a training design, and how to interpret the increase in behavioral CQ as meaningful for job performance is uncertain.⁴⁵ If evaluation outcomes are not well-aligned with the training method and objectives, the evaluation may be of interest for research purposes, but not necessarily informative for design and management decisions.

In compiling the results of multiple studies, meta-analyses have suggested that training design features may influence learning and performance outcomes of CCT.⁴⁶ Some CCT research has attempted to operationalize training design variables, but little consistency has emerged. J. Stewart Black and Mark Mendenhall conceptualized training rigor as the degree of trainees’ cognitive involvement, depicted on a linear scale ranging from factual to analytical to experiential.⁴⁷ Alternatively, some studies have used a two-by-two framework crossing the content of training (culture-specific vs. culture-general) with training methods (didactic vs. experiential), indicating that culture-specific, experiential training was most effective.⁴⁸

Another study used a five-point scale of training “comprehensiveness” based on survey participants’ recollections of training on 17 questions.⁴⁹ Other researchers and practitioners advocate using multiple CCT methods, but a consistent way of defining those methods has not yet emerged.⁵⁰

Broadening Evaluation Methods for CCT

Methods for training evaluation have emphasized outcome evaluation, but a broader perspective for CCT evaluation is needed for at least two reasons. One is the lack of consensus on how to operationalize CCT design features, as discussed above. The second is the multiple purposes of program evaluation. Training program evaluation informs a number of management decisions, including resource decisions, design decisions to improve CCT programs already in place, and decisions on training delivery.⁵¹ Although Kirkpatrick’s levels-of-evaluation framework has been popular, overreliance on this single framework may be limiting.⁵² Insights from the science of training can be used to conduct process evaluation in conjunction with outcome evaluation.⁵³ Assessing program inputs and processes, as well as outcomes, provides organizational leaders with the information needed to make more informed decisions.

In military training, training requirements compete for time and resources. Showing the impact of training is therefore critical to obtaining and sustaining resources. Military CCT programs have experienced challenges in demonstrating relevance and impact as priorities shift. Training evaluation can provide a basis for more systematic choices about programs and resources.

In an effort to synthesize best practices for training program inputs and processes, table 2 provides a list of program elements for CCT evaluation in four categories: planning analyses, design and delivery, program management, and systems and context. Recommendations and best practices from Eduardo Salas, Scott I. Tannenbaum, Kurt Kraiger, and Kimberly A. Smith-Jentsch’s “The Science of Training and Development in Organizations” served as the starting point.⁵⁴ The list also includes recommended practices for enhancing training transfer and items from other reviews.⁵⁵

In addition, this framework includes elements recommended specifically for CCT, listed in the right column of table 2.⁵⁶ Given the methodological limitations noted in the preceding section, the research evidence for some CCT recommendations listed in the table is not as strong as that of the general training elements in the middle column. Because CCT programs typically include multiple methods and do not isolate the effects of different methods, research is less conclusive about the impact of specific practices and methods used in CCT.⁵⁷

This framework is consistent with the standard training and education development processes commonly in use in the DOD. The evidence-based training elements map closely to the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) process.⁵⁸ Therefore, in part, this study examines the extent to which new training and education programs implemented existing instructional design policy and processes. Established to meet an ongoing operational need, military CCT programs could either reflect common Service practices for ADDIE or may have followed a different path. Although the present study cannot assess whether the ADDIE process is fully implemented for other training domains, as a guiding structure, ADDIE reflects best practices for training development.

Table 2. Elements of an evidence-based training program

Training in Organizations^A

Cross-Cultural Training^B

Planning

Learning climate

Training needs analysis and other planning analyses

Organization/strategic alignment
Job/task
Training audience

Determine whether CCT should be culture-general or culture specific.

Customize CCT to match the expatriate employee’s needs.

Design and delivery

Timing and length

Learner motivation

Method/outcomes match

Valid training strategy and design

Practice and feedback
Error management

Delivery media
Simulations, structured user control in computer-based training

Offer CCT prior to departure, immediately following arrival, or both.
Adjust the length of training based upon the unique features of the assignment.

Use a skills-based approach.
Tailor delivery strategy to training goals.

Use scenario-based training, simulations, culture assimilators, theory-based design.^C

Offer numerous opportunities for practice.
Include behavior modeling.^D

Use multiple delivery media within one training program.

Establish success criteria for CCT.

Program management

Evaluation at multiple levels

Updates to program design

Evaluate CCT each time it is implemented.
Use numerous criteria.
Conduct surveys to assess the expatriate’s satisfaction.

Systems and context

Support to supervisors

Plans for transfer
Goals and feedback on the job

Work context
Workload and stress.^E

Develop global mindset in all employees.

Training delivery should correspond to the dynamic adjustment process.

Tailor to the cultural toughness/novelty of the destination country.
Address repatriation.

^A Unless otherwise specified, the content in this column is drawn from Eduardo Salas et al., “The Science of Training and Development in Organizations: What Matters in Practice,” Psychological Science in the Public Interest 13 no. 2 (2012): 74–101, https://doi.org/10.1177/1529100612436661; and Ashley M. Hughes et al., “A Checklist for Facilitating Training Transfer in Organizations,” International Journal of Training and Development 22, no. 4 (December 2018): 334–45, https://doi.org/10.1111/ijtd.12141.
^B Unless otherwise specified, the content in this column is drawn from Lisa N. Littrell and Eduardo Salas, “A Review of Cross-Cultural Training: Best Practices, Guidelines, and Research Needs,” Human Resource Development Review 4, no. 3 (2005): 305–34, https://doi.org/10.1177/1534484305278348.
^C Dharm P. S. Bhawuk, “The Role of Culture Theory in Cross-Cultural Training: A Multimethod Study of Culture-Specific, Culture-General, and Culture Theory-Based Assimilators,” Journal of Cross-Cultural Psychology 29, no. 5 (1998): 630–55, https://doi.org/10.1177/0022022198295003.
^D J. Stewart Black and Mark Mendenhall, “A Practical but Theory‐Based Framework for Selecting Cross‐Cultural Training Methods,” Human Resource Management 28, no. 4 (Winter 1989): 511–39, https://doi.org/10.1002/hrm.3930280406.
^E Darlene Russ-Eft, “A Typology of Training Design and Work Environment Factors Affecting Workplace Learning and Transfer,” Human Resource Development Review 1, no. 1 (2002): 45–65, https://doi.org/10.1177/1534484302011003.
Courtesy of the author, adapted by MCUP.

The four program components of this framework are described below.

Planning

Several types of analyses are important in the development of training.⁵⁹ Organizational analysis assesses strategic organizational goals to provide top-down guidance to cultural training and education. Organizational analysis may also include an assessment of the organization’s resources, norms, and support for training.

In contrast, job-task analysis offers a bottom-up view of the organization, focusing on performance requirements of job demands. Job-task analysis often includes both the behavioral or physical requirements for a particular job or function as well as cognitive requirements. In CCT, one important consideration is the distinction between culture-general vs. culture-specific requirements. If job demands require interaction with multiple cultures, or skills that transfer to different cultural settings, then culture-general CCT may be more beneficial than culture-specific.

Another form of planning analysis, learner analysis, focuses on the training audience, assessing general audience characteristics and variability among learners. Individual differences in learners’ initial skill levels, age, personality characteristics, and motivation are examples of important variables. These variables influence responses to training and can guide development of both training content and methods. For CCT, the learners’ prior international and multicultural experience and their openness to experience are important considerations. A training needs analysis enables the organization to assess both the job requirements and the training audience to determine what training is needed to improve performance.

Design and Delivery

The design and delivery category covers a range of training inputs and processes. Inputs include the theoretical foundations or logic model, the timing and length of training, and delivery media. Training processes in design include the instructional methods, such as simulations and other experiential methods, behavior modeling, and opportunities for practice. Design also addresses the role of learning assessments and feedback within training. Research indicates that practice should be challenging enough to offer opportunities to make errors and to receive feedback; errors are an important element of good training design.

Program Management

Program management processes beyond the training itself are another important aspect of training. Training programs tend to be more effective when they include systematic training evaluation and mechanisms for receiving feedback from learners after they transition to or return to the job context. Researchers recommend that training evaluation should start with identification of the organizational purpose(s) for evaluation, followed by an alignment of measurement with those purposes. Measures tailored to the program outcomes may be more effective than generic evaluation, such as an off-the-shelf test. Programs should also include a means to update training scenarios as the job context or tasks change.

Systems and Context

Systems and context variables address the post-training transfer environment. Supervisors can reinforce training by facilitating a positive learning and transfer climate through coaching. Communities of practice can also support training transfer. In CCT, context includes the temporal aspects of transfer, such as the nonlinear dynamics of intercultural adjustment. Rather than showing steady, incremental increases in adjustment over time, personnel may experience temporary decrements in adjustment or performance as they experience frustration, test and update assumptions, and learn to adapt to the cultural environment. For deployments or other assignments abroad, post-assignment support and debriefing can serve to reinforce learning and maintain transfer.

Present Study

The present study used the framework in table 2 to evaluate military CCT programs with the aim of informing decision makers involved in program resourcing. The Close Combat Lethality Task Force within the Office of the Secretary of Defense requested the study.⁶⁰ Because learning outcome data were not available across military CCT programs, the elements in table 2 provided an alternative means to evaluate programs using process assessment. In that context, the present analysis focused on strengths and weaknesses in the programs overall. The study included interviews with directors and managers of cultural training and education programs across the U.S. military Services and a review of available curriculum materials. As noted previously, defense organizations often distinguish between training and education. They are combined under the label of CCT for purposes of this analysis because cultural learning was implemented in clear response to an organizational need for organizational purposes.

Sample

The sample included nine organizations and program offices involved in cross-cultural training and education for active-duty GPFs and SOFs (table 3). Participants were asked to provide their perspective based on their experiences in their office and roles; responses were not formally coordinated and did not reflect the official positions of their organizations.

Table 3. Participating organizations

Office of the Secretary of Defense	Defense Language and National Security Education Office
U.S. Special Operations Command	Special Operations Forces Language Office
U.S. Army	Training and Doctrine Command Culture Center John F. Kennedy Special Warfare Center and School, Special Warfare Education Group
U.S. Air Force	Air Force Culture and Language Center Air Force Special Operations School
U.S. Navy	Naval Special Warfare Command
U.S. Marine Corps	Center for Advanced Operational Culture Learning Marine Corps Forces Special Operations Command

Courtesy of the author, adapted by MCUP.

Seven of these offices designed and delivered cultural training or education, and two offices were primarily responsible for policy and oversight of defense foreign language and cultural training and education programs. Interview participants totaled 17 and included at least one program manager, curriculum designer, or instructor from each office. Four offices included multiple participants. Three participants were serving as instructors at the time of the interview, but some interviewees had served in multiple roles, either as both instructors and curriculum developers, or as program manager and curriculum developer. Interviews took place in May and June of 2018. Participants provided input on a voluntary basis and were informed that the primary audience for the study’s findings was the Office of the Secretary of Defense. Most interviews were conducted by a single interviewer by phone; three interviews were conducted in person. Responses were not audio recorded. In recognition of the limitations of the data gathering approach, participants were provided an earlier version of this paper and given an opportunity to provide corrections and feedback to ensure accuracy.

Scope of the Study

At the sponsor’s request, the study was designed to gather information about culture-general training and education.⁶¹ Training designed primarily to teach foreign language and country-specific information was beyond the scope of this review. To supplement the interviews, the study included a review of policy documents, program reports, and instructional design materials related to the interview input. Some documents were publicly available; some documents were internal program materials provided by interview participants. The programs varied in their offerings, and courses included computer-based, self-paced courses; distance education courses accredited for college credit; instructor-facilitated classroom education; and tailorable classroom-based training.

Assessment Method

To inform organizational decision making about resources, this study used an exploratory qualitative approach to assess the feasibility and utility of using training design recommendations in program evaluation. Given limited time and resources to conduct the evaluation, it was not feasible to assess all program elements in the present analysis. In addition, limited information from some programs prevented a full assessment of some elements of the framework. As a result, findings focus on a subset of the elements identified in table 2, omitting systems and context variables in particular due to a lack of data and information about the training transfer context.

Findings

Using the framework in table 2, findings assess CCT program elements where sufficient information was available from respondents across multiple programs or from existing research publications and program documents.

CCT Planning

The authors of “The Science of Training and Development in Organizations” identified several different analyses that are important for planning training. Organizational analysis of strategic goals provides top-down guidance to cultural training and education, whereas job-task analysis focuses on performance requirements from a bottom-up view of the organization.

Organizational Analysis

Multiple levels of strategic guidance were in place to guide CCT, both at the DOD level and in the military Services. However, respondents reported that outdated and conflicting guidance was a limitation for strategic alignment. Four Services—the Air Force, Army, Navy, and Marine Corps—had adopted language, region, and cultural training and education strategies between 2008 and 2010, and the Marine Corps updated its strategy in 2015. Although two defense-wide policies identified the competencies to be developed, the two policies use different competency models.⁶² Some respondents indicated they were aware of only one of the policies, and one respondent indicated that the conflict between the two policies led programs to prioritize guidance within their own Service. Some organizations lacked a designated proponent for implementing the strategies and policies once adopted, leaving gaps in the alignment of strategy, resources, and CCT design.

Job-Task Analysis

Job-task analysis was conducted for SOFs to identify training needs for CCT. Programs for GPFs did not have the internal resources to conduct job-task analysis, but other organizations had funded numerous analyses to better understand the sociocultural aspects of military operations and help identify relevant learning outcomes for cultural training (table 4). The products of these efforts have included a set of foundational learning objectives, various competency models, and intercultural performance dimensions spanning functional specialties.⁶³ Consistent with competency modeling best practices, the cultural competency models provide an appropriate level of granularity for developing curricula and instructional content.⁶⁴

Table 4. Training needs analysis for military CCT

Source	Research product	Method	Sample	Sample size
McDonald et al.^A	Learning objectives	Subject matter expert workshop	Staff officers, social and behavioral scientists, training developers	14
Hardison et al.^B	Performance dimensions	Survey	U.S. Air Force, active-duty	6,653
McCloskey et al.^C	Competency model	Critical incident interviews	U.S. Army, active-duty	70
Rasmussen et al.^D	Competency model (adopted in policy)	Critical incident interviews	All U.S, Services, active-duty and retired	20 95
Abbe and Gallus^E	Competency model, learning objectives	Focus groups and survey	U.S. Army officers, active-duty (captains and lieutenants only)	72
Wisecarver et al.^F	Competency model(adopted in policy)	Focus groups and survey	All U.S. Services, CCMD staff, civil affairs	49 788
Center for Advanced Operational Culture Learning^G	Skills and knowledge	Survey	U.S. Marine Corps, active-duty	1,654
Foldes et al.^H	Performance dimensions	Survey	U.S. Army, active-duty	4,157

^A Daniel P. McDonald et al., Developing and Managing Cross-Cultural Competence within the Department of Defense: Recommendations for Learning and Assessment (Arlington, VA: Defense Language Office, 2008).
^B Chaitra M. Hardison et al., Cross-Cultural Skills for Deployed Air Force Personnel (Santa Monica, CA: Rand Corporation, 2009).
^C Michael J. McCloskey et al., A Developmental Model of Cross-Cultural Competence at the Tactical Level, Technical Report 1278 (Fort Belvoir, VA: U.S. Army Research Institute for the Behavioral and Social Sciences, 2010).
^D Louise Rasmussen et al, Data Collection and Analysis for a Cross-Cultural Competence Model (Fairborn, OH: Applied Research Associates, 2011); and Louise Rasmussen, Winston R. Sieck, and Jasmine L. Duran, A Model of Cross-Cultural Competence for Education and Training: Validation across Services and Key Specialties (Yellow Springs, OH: Global Cognition, 2016).
^E Allison Abbe and Jessica A. Gallus, The Socio-Cultural Context of Operations: Culture and Foreign Language Learning for Company-Grade Officers, Technical Report 1316 (Fort Belvoir, VA: U.S. Army Research Institute for the Behavioral and Social Sciences, 2012).
^F Michelle Wisecarver et al., Regional Expertise and Culture Proficiency (Arlington, VA: Defense Language and National Security Education Office, 2012).
^G Overall CAOCL Survey II Findings: The Value and Use of Culture by Type of Deployment (Quantico, VA: Center for Advanced Operational Culture Learning, 2013).
^H Hannah Foldes et al., Sociocultural Components of Mission Performance: Development of a Taxonomy of Performance Requirements (Fort Belvoir, VA: U.S. Army Research Institute for the Behavioral and Social Sciences, 2012).
Courtesy of the author, adapted by MCUP.

Despite the availability of these analyses, interviews indicated only limited reliance on them for developing learning objectives in cultural training and education. Many of these analyses were completed in parallel with instructional design, and there was no requirement to update curricula using the results. Even when competency models were subsequently incorporated into defense-wide policies (CJCSI and DODI), the policies were enacted well after the military Services’ own culture strategies, curricula, and courses were already in place, posing an obstacle to their implementation.

The limited involvement of instructional design specialists in some programs was also an obstacle. Greater instructional design expertise would have facilitated the translation of task requirements into the design of learning activities and better alignment of instructional methods and learning outcomes.

Learner Analysis

Recommendations for evidence-based training also include person analysis to understand how training design can best accommodate learners’ individual differences.⁶⁵ One program had conducted a series of systematic learner analyses to identify individual differences associated with attitudes toward cultural capabilities and CCT.⁶⁶ They found that Marine officers valued intercultural training and skills more than did enlisted Marines, as did Marines with prior exposure to foreign languages and cultures. Marines also considered CCT to be as important as other predeployment training.

In other programs, interviews indicated that learner analysis tended to occur informally, and it focused less on individual differences among trainees than on general characteristics of the training audience. Some learners were young and had limited or no experience with cultures outside of the United States, and instructional designers and program managers indicated that this was a consideration in the training design. These audience characteristics represent one important difference between corporate and military audiences for CCT.

Design and Delivery

Timing and Length of Training

In general, CCT timing considered learner needs but also scheduling opportunities that would fit within operational and other training and development requirements. Programs of instruction were highly variable in length, ranging from 2 to 45 hours. Shorter durations were typical of predeployment training, whereas the longest courses were SOF programs and distance courses accredited for college credit. Programs for SOFs had a longer training duration by building culture into their qualification courses.

For GPFs, CCT occurred at different stages and with greater variability in length. Some training occurred at the unit level prior to deployment, and the length was determined by the commander in consideration of other scheduled training requirements. Two programs had a distance education option following a more traditional academic course model, either accredited for or directly awarding college credit. One such program, the Marine Corps’ Regional, Culture, and Language Familiarization program, is unique in that it provides a progressive curriculum over a sequence of career stages.⁶⁷ As a smaller Service, the Marine Corps may be better positioned to align curricula than the other military Services and was more systematic in assessing its efforts.⁶⁸

All programs offered training prior to departure only. Although previous research has recommended post-arrival training, offering CCT upon arrival in country was not practical for military personnel due to operational demands.⁶⁹ Programs did not offer repatriation debriefings on return to the United States, as recommended in previous CCT literature.

Learner Motivation

Literature has indicated that managing attendance requirements is a tricky balance, and present findings are consistent with that assessment.⁷⁰ CCT was mandatory for personnel in most of these programs. For one of the voluntary CCT options, respondents indicated that there was low utilization and potential trainees may have lacked awareness of its availability. However, voluntary options awarding college credit had good utilization.

Mandatory CCT presented its own challenges. In some organizations, mandatory training may indicate to learners the importance of the training topic to the organization. However, the military has a unique organizational context for training. Military members have a lot of required training that is perceived as filling compliance requirements, but as not necessarily valuable for their jobs.⁷¹ In addition, training requirements often exceed available training time, potentially leading to resistance to additional training.⁷² Respondents reported that, as a result, instruction consistently incorporated operational examples and scenarios to engage trainees and convey the importance of CCT.⁷³

Method-Outcomes Match

Learning objectives across programs generally fell into three categories: observing and understanding cultural dynamics, self-awareness and self-management, and interacting and communicating effectively. These three categories are consistent with the cultural competency models developed for military personnel (see table 2) and with dimensions identified in previous research.⁷⁴ Method-outcomes match showed some continuing gaps. Overall, training addressed cognitive learning outcomes (i.e., observing and understanding cultural dynamics) to a greater extent than behavioral outcomes (i.e., interacting and communicating effectively). Some programs had increased efforts to address behavioral outcomes over time. Multiple respondents emphasized the importance of including didactic methods with experiential methods to address the full range of learning outcomes.

Didactic methods commonly focused on conveying explicit knowledge that would facilitate further knowledge acquisition—frameworks and concepts that would transfer to the operational context. Although these methods were appropriate to the “understanding” dimension across programs, some participants acknowledged that didactic methods were insufficient to meet objectives in the “interacting and communicating” dimension. Respondents indicated that resource and scalability limitations hindered efforts to prepare personnel adequately for interpersonal interactions. Although there was consensus that having personnel interact one-on-one with culturally dissimilar others was valuable, time, staffing, and other resource considerations did not permit it. As a result, didactic presentation of concepts and information was sometimes more practical for training audiences in the general-purpose force.

Some programs did have resources to include cultural role players in interactive exercises and live simulations. Programs targeting smaller training audiences such as SOFs reported success in including cultural role players, sometimes drawing on foreign language instructors to serve that function. However, one respondent indicated challenges in using role players effectively for culture-general instruction. Role players tended to come from just one or two countries (typically, Arabic-speaking countries), which could prompt learners to focus more on culture-specific learning and neglect generalizable cultural learning. Finding more culturally diverse role players within the available contract relationships was a limiting factor for culture-general programs.

The self-awareness/self-management category of learning objectives showed mixed results. Some military CCT programs used methods appropriate to developing cultural self-awareness, such as methods using explicit cultural contrasts, but programs without an interaction or simulation component had limitations in helping trainees learn to cope with frustration, conflict, or misunderstandings that sometimes result from intercultural interactions.⁷⁵

Valid Training Design and Content

Of CCT methods identified in a prior review, examples of most could be identified in different military CCT programs: area briefings, lectures and presentations, films, language training, case studies, interaction, role-play, and simulation.⁷⁶ Military CCT generally did not include books or sensitivity training, or training methods developed specifically for CCT in previous eras, such as culture assimilators, the contrast American technique, or attribution training.⁷⁷

Scenario-based and case study methods were particularly prevalent across military CCT programs. Role-play and live simulations were viewed by respondents as highly effective techniques, but were prohibitively resource intensive for some programs. Foreign language instruction was included for some special operations programs, but not all of them, and foreign language was not common in programs for general-purpose forces beyond teaching a few words.

One notable difference between the didactic instruction described in previous CCT research and military CCT programs was the emphasis not just on factual cultural information, but also on concepts and frameworks to help servicemembers structure their understanding and acquire greater cultural understanding in future experiences, as noted above.⁷⁸ Some programs used frameworks drawn directly from scholarly research, such as Gerard H. “Geert” Hofstede’s cultural dimensions theory, while others developed frameworks for their own purposes, tailored for their practitioner audience.⁷⁹ One such example is the operational culture framework developed for the Marine Corps.⁸⁰

Though some programs included theory-based content and learning outcomes, training methods based on CCT theory were generally absent. However, given that theory on intercultural adjustment and performance has not been tested in or developed for military contexts, existing theory may not be directly applicable. Navigating cultural differences in conflict settings may not follow the same patterns in a multinational corporate executive or a study abroad student—the populations on which most of the intercultural competence research literature is based.

Practice and Feedback

Some programs in this analysis offered opportunities for skills practice, which came in the form of interactions or simulations as noted above. Providing structured feedback to trainees was less common. Some researchers have recommended that CCT go a step further than practice and feedback to incorporate error management.⁸¹ Error management training includes not only practice and feedback but also positive error framing. Positive error framing conveys to trainees that errors are a beneficial part of the learning process. Only one military CCT program in this study indicated incorporating error management in this way, as an intentional component of the instructional design.

Delivery Media

Although most of the training in this assessment was instructor facilitated, three of the participating program offices offered distance learning options. The distance learning options were intended to provide flexibility to learners and accessibility to large numbers, though respondents recognized that these courses did not necessarily provide optimal delivery for all of the learning objectives. Respondents indicated that efficiently providing foundational instruction to large numbers of personnel was the primary consideration, and this delivery method focused primarily but not exclusively on cognitive learning outcomes. The distance learning options provided some elements of user control as researchers have recommended.⁸²

Program Management

One aspect of program management that emerged in interviews but was not addressed in previous literature was the need for multidisciplinary expertise in CCT programs. Interview participants recognized a need for input from multiple disciplines and backgrounds in designing and delivering CCT, including military operational experience, cultural or regional expertise, social sciences, instructional design, and good program managers to navigate the resourcing processes within their parent organizations. Programs benefited from the ongoing collaboration enabled by having these different backgrounds represented within the program.

Evaluation

None of the programs reported having processes in place to conduct training evaluation with traditional experimental designs, using pre-post assessments and controlled comparison groups. A 2011 U.S. Government Accountability Office report noted that the Services were not even consistently tracking completion of cultural training.⁸³ However, two programs had conducted other forms of systematic evaluation, and one had incorporated evaluation into its organizational structure and processes. Among programs without evaluation capability, one program manager indicated that they did not have the resources, either in staff time or expertise, to conduct evaluation in-house. Another respondent indicated that personnel turnover was an obstacle, and that maintaining other aspects of the program in the face of staffing challenges took precedence over evaluation. Illustrating evaluation challenges specific to the military context, one program had planned an evaluation, which was then precluded by the early deployment of the participating operational unit.

Updates

Some respondents indicated that they informally solicited feedback and incorporated learner input and operational examples in updating their scenarios and content. However, none of the programs had formal mechanisms for feedback from learners. For instructor-led CCT, programs routinely reshaped their delivery to fit the training audience, but they did so informally and did not solicit external review or feedback.

Quality Assurance

Quality assurance was another program element that was not discussed in the training literature but emerged as a critical consideration for organizations. Respondents from one program indicated that some instructors had drifted away from the learning objectives over time, gradually emphasizing operational storytelling at the expense of the instructional design. This drift was reportedly more common when instructors were subject matter experts without a background in instructional design. Relying on a team of instructors with differing backgrounds and expertise posed similar challenges for ensuring consistency in delivery. Consistent delivery across instructors and time was both an ongoing management challenge and a barrier to scaling CCT to larger numbers of personnel.

Another challenge reported by some respondents was the reliance on external contract personnel to deliver CCT. Advantages of relying on external contractors included the flexibility to access or divest particular regional or country expertise more readily. Disadvantages included less control over design and delivery, less continuity (only in some programs), and additional challenges for program knowledge management.

Systems and Context

Contextual considerations were apparent in military CCT content. Cultural novelty or toughness was addressed by including material to prepare servicemembers for differences in values and morality that they may encounter in conflict settings. Scenarios were drawn from real operational experiences to help inoculate personnel against potential stressors in intercultural situations. However, programs showed a lot of variation in addressing such issues, reportedly due to time constraints. Access to cultural experts and host nation counterparts was also an obstacle to developing valid scenarios.

Findings suggested that support for training transfer may be continuing gap in military CCT programs, but this element could not be fully assessed within the present study. None of the programs reported included goal setting or other activities for training transfer, though it is possible that some instructors may take the initiative to do so. Assessing supervisor support would be beneficial, but that was not feasible in this study.

Summary of Findings

Table 5 provides a summary of the degree to which programs included evidence-based program elements. Elements indicated as “present” were a strength in military CCT across programs. Elements indicated as “partial/mixed” were present in only some programs or were addressed to a degree but showed some inconsistencies with recommendations from training science. Elements indicated as “continuing gaps” were either absent altogether or present in only one program.

Table 5. Assessment of military CCT practices

	Recommended element	Extent to which programs included or addressed the element
Planning	Training needs analysis and other planning analyses: Organization/strategic alignment Job/task Training audience	Partially met Partial/mixed Present
Design and delivery	Timing and length Learner motivation Method/outcomes match Valid training strategy and design Practice and feedback Error management Delivery media Simulations, structured user control in computer-based training	Generally determined by practical constraints, less by training needs Present Partial/mixed Partial/mixed (determined both by practical constraints and outcomes) Continuing gap Partial/mixed
Program management	Evaluation at multiple levels Processes for updating program design	Continuing gap Continuing gap

Courtesy of the author, adapted by MCUP.

Conclusions

Although program limitations and present methods did not allow for a rigorous comparison of program design, the evidence-based training program elements from previous research provided a useful framework to assess the quality of military CCT programs, revealing both strengths and gaps. Conclusions offered here are tentative but represent a macro-level assessment of military CCT at the time of the interviews. Strengths of military CCT programs were apparent in both planning and in design and delivery, including identifying training needs, tailoring training accordingly, and motivating learners. Continuing gaps emerged in elements of design and delivery, strategic alignment, and program management. Some programs ensured that their CCT aligned with defense-wide policy, and some did not. In design and delivery, programs generally lacked opportunities for practice, feedback, and structured error management, with a few programs as notable exceptions. In program management, most programs did not have formal processes in place for evaluation and updating curriculum, and interviewees did not report including program elements to enhance transfer.

Interview findings highlighted the unique challenges of designing and delivering CCT to military personnel. Prior CCT research has focused on expatriate managers, and the military context presented a different set of challenges. Differences in audience characteristics suggest that introducing CCT earlier and on multiple training occasions may be important for military personnel who are younger and have less international experience than is typical of expatriate managers. Secondly, military operations often do not allow personnel to immerse in the local culture, limiting their opportunities for cultural learning once they arrive in country and posing unique challenges for CCT transfer relative to other contexts.

In addition, whereas multinational corporations send expatriate managers abroad as individuals, the military deploys units, as well as sending individual personnel for certain assignments. This difference impacts a number of CCT design and delivery decisions due to the resources required. This study helped highlight tradeoffs between instructional design and resource decisions that can inhibit the application of training science in military settings. Tradeoffs were most apparent in design elements of method/outcomes match, valid training strategy and design, and delivery media.

The practical constraints dominating these design decisions included several different resource considerations. First, time constraints limited access to the training audience; CCT had to fit into a schedule of many other training requirements. Scalability was a second, related constraint, requiring programs to reach large numbers of geographically dispersed personnel. For instructor-facilitated training, instructors either had to travel to the unit receiving training, or learners were on-site enrolled in a broader training or education program, of which CCT was just one component. In either case, the size of the training audience was a challenge for designing and delivering CCT to GPFs. Programs for SOFs experienced similar challenges, but to a lesser degree due to their smaller numbers. Third, budget constraints limited either the time and resources for instructional design, or the personnel available for delivery, or both. Program managers had to work within these organizational constraints and were not always able to optimize training design for the learning outcomes.

Improving Military Training

Although some military CCT programs have been reduced or eliminated as resources shift to other defense priorities, servicemembers will continue to work in multinational and multicultural settings and need to develop intercultural skills as a result.⁸⁴ To that end, some of the gaps identified in the present study could be addressed with minimal resource investment. For example, improved knowledge management could help alleviate the challenge of personnel turnover. Knowledge management processes may also be critical to maintain program content and information as programs are reduced or eliminated in response to shifting priorities. The Marine Corps’ Center for Advanced Operational Culture Learning engaged in such efforts in transitioning reports and resources to the Defense Technical Information Center and other repositories.

In addition, CCT methods themselves can be improved even with limited resources. Behavior modeling is one effective method that should be used more frequently in military CCT and can be incorporated via means easily scalable. For example, military CCT can include videos modeling effective intercultural interactions. These positive models can be reinforced by instructor facilitation in live classroom or experiential training, or by a narrator in computer-based training. Demonstrating what right looks like is simple but effective.

The framework applied in this study offers one answer to the call for expanded evaluation methods, using a form of process evaluation.⁸⁵ This approach can help guide training design and evaluation in other domains as well. Training process evaluation can be used in combination with outcome evaluation to assess military training designed to prevent harmful behaviors. The DOD has made significant resource investments in preventing sexual harassment, sexual assault, and suicide. More systematic process evaluation can help identify effective training practices, such as, for example, in determining why Air Force sexual assault prevention training is perceived as more effective than Army training.⁸⁶ Process evaluation can be especially important for issues like suicide prevention, where training outcome evaluation is lacking or difficult to conduct.⁸⁷ The checklists provided by the authors of “The Science of Training and Development in Organizations” and “A Checklist for Facilitation Training Transfer in Organizations” are a good starting point for program managers and training designers.⁸⁸

Future Research

The challenges of providing CCT to military personnel demonstrated in this study call for more research in this population. Improved tools for outcome evaluation remain a critical unmet need to better understand the impact of military CCT on learning and operational performance. In reviewing measures of cross-cultural competence, David Matsumoto and Hyisung C. Hwang identified several promising measures with incremental validity beyond other measures and evidence of ecological validity, including the Multicultural Personality Questionnaire and the Cultural Intelligence Scale.⁸⁹ To date, very little research has examined the validity and utility of those measures for military personnel.

In addition, CCT research has been primarily conducted for the context of multinational corporations. Some recommendations for expatriate managers are less applicable to the military. For example, Lisa N. Littrell and Eduardo Salas’s recommendations on post-arrival training, repatriation issues, and training customization may be more challenging for the military to implement for general-purpose forces.⁹⁰ The size of the workforce deploying abroad and the austere environments in which they often work pose challenges for the scaling and timing of CCT.

In addition, the armed forces often may not have the luxury of being selective about whom they deploy. Unlike expatriate managers in corporations, the United States often deploys military personnel in units, not as individuals. Variation in intercultural skills, interest, and experience levels is likely common. Even for individual deployments or overseas assignments, rank and military specialty take precedence over personality traits and prior experiences associated with higher likelihood of intercultural success.⁹¹ The stress and threat of military operations are additional considerations with very little parallel in the management literature. Therefore, intercultural competence in military personnel may have context-specific dimensions requiring further development or adaptation of theories and assessments.

The unique operational challenges also suggest the continuing importance of preparing personnel for the cultural dimension of deployments and assignments abroad. Without the benefit of deploying only personnel with the highest likelihood of cultural adaptation, advance preparation through education and training becomes even more important. This may be particularly true when operations in regions and nations more culturally distant from the United States require routine interaction.⁹² Recent research suggests that experiential CCT methods may be less effective for some individuals; further research is needed to determine the applicability of these methods for a military population.⁹³ Instructors must also be trained in using experiential CCT effectively.

Given limited resources for formal CCT, another important issue for future research is to examine the role of informal learning in developing intercultural competence. Assignments and other professional experiences can also develop intercultural skills and complement the learning that occurs in formal training and education, or may even shorten the time required for formal training.⁹⁴ Examples of such experiences for military personnel include international military education programs, multinational assignments and exercises, and the U.S. National Guard’s State Partnership Program. During operations, working with interpreters may be another important source of intercultural learning.⁹⁵ A better understanding of how these experiences contribute to developing intercultural competence would help supplement and better target training efforts. Development of intercultural competence in organizations may be more effective and efficient if learning opportunities are mutually reinforcing and sequenced more intentionally. CCT research should also further examine transfer. Goal setting, action plans, and supervisor support help improve training transfer, but CCT research has not systematically examined barriers to transfer or how to improve transfer.⁹⁶

More broadly, future research should also examine how to better operationalize program elements for the purposes of training improvement and evaluation. In CCT, previous research has often relied on simple dichotomies (e.g., tacit vs. explicit knowledge) or has sometimes neglected to distinguish what is taught (the learning objectives) from how it is taught (the instructional method).⁹⁷ Dichotomizing training as either didactic or experiential oversimplifies design elements, as experiential methods can vary widely. Experiential methods are generally more effective, but is a resource-intensive, live simulation required, or would a scenario-based classroom exercise be sufficient for the learning goals?⁹⁸

Leadership training evaluation has shown some recent advances beyond these simple dichotomies, comparing information-, demonstration-, and practice-based methods, as well as combinations of these methods. Researchers also considered training methods as distinct from the training content, representing another advance for program resourcing decisions.⁹⁹ Further, such refinements will improve military training evaluation, enabling evaluation at the level of fidelity needed for resource management. The present study illustrates the challenges of conducting evaluation and the need for evaluation tools and methods to inform management decisions.

Broader research questions remain about training transfer as well. Contextual aspects of performance, including peer and organizational support, are critical for transferring the skills and knowledge acquired in training to the job.¹⁰⁰ The CCT programs in the present study did not have broad enough reach to include systems and context in evaluation. Investments in training may have very limited impact if the performance context lacks clear connection with training. The challenges discussed in the introduction of this article, which pose an obstacle to the organization’s ability to conduct training evaluations, also pose an obstacle to servicemembers in training transfer—in that, for example, training that occurs distant in time or location from unit performance may be less likely to transfer. Further research is needed in military settings on how to improve transfer for “soft” skills like culture, leadership, and team building.

Final Thoughts

Although the United States has shifted its defense strategy away from counterinsurgency and toward great power competition, the importance of cultural capabilities in military personnel persists. The emphasis on allies and partners in the 2017 National Defense Strategy calls for CCT and intercultural military education for effective multinational operations. With more than 2 million servicemembers and a presence in more than 160 countries, the scale of U.S. military cultural training needs is unique.¹⁰¹ There will be continued need for developing an internal U.S. cultural capability at tactical, operational, and strategic levels of engagement.

Therefore, both the efficiency and effectiveness of military training are important. The following recommendations provide some actions that defense enterprise leaders can take to ensure that lessons from military CCT help the Services achieve the vision of being a learning organization.¹⁰²

Military training and education programs should incorporate training and instructional design expertise with subject matter expertise from the beginning of the design process. This recommendation applies not only to CCT programs but also to related “soft” skills and other emerging subject areas for training.
When the defense enterprise rediscovers the need for cultural capabilities, as has happened in previous conflicts, training and education leaders should build on lessons from the CCT programs in the present study. The present assessment indicates that the programs had many strengths consistent with the science of training, and the research literature and knowledge management efforts from this period will facilitate future CCT efforts, even though the specific countries and regions of operational priority may differ.
Defense research and development should invest in training transfer research. Research organizations within the DOD should invest not only in training technologies but also in advancing instructional methodologies that can enhance transfer and be used with a variety of technologies and settings.
The Office of the Secretary of Defense and the military Services should include training program evaluation in robust support of studies and analyses. They should subsequently encourage the publication of program evaluations to benefit not just the sponsoring organization, but other defense organizations as well. Conducting and publishing such studies would improve accountability in programs and help sustain an organizational learning culture.

The process evaluation used in the present study provides a useful framework for evaluating and refining CCT programs, pointing to several areas for improvement in military CCT, including instructional methods, the need for practice and feedback in CCT, organizational alignment, and program management. This research also highlighted research gaps for CCT. In particular, gaps in CCT program evaluation demonstrated the need for further research on the development and assessment of cross-cultural competence in military personnel.

As multinational military cooperation continues, so does the demand for “cultural interoperability.”¹⁰³ One area of continuity in U.S. security strategy is reliance on alliances and partnerships, and competition with near-peer adversaries requires preparing military personnel to maintain and expand these relationships. However, cultural training and education programs have been on the decline.¹⁰⁴ If the U.S. military follows the pattern of previous conflicts in Vietnam and Iraq, military departments will wait until the operational need is urgent before widely offering cultural training for general-purpose forces. The present evaluation can help inform the re-establishment of these programs consistent with the science of learning and training. Future interdisciplinary collaboration among scholars and military training practitioners will enable further improvements in the understanding and management of military CCT.

Endnotes

The National Military Strategy of the United States of America: A Strategy for Today; A Vision for Tomorrow (Washington, DC: Joint Chiefs of Staff, 2004).
Defense Planning Guidance (Washington, DC: Department of Defense, 2006–11); and Quadrennial Defense Review Report (Washington, DC: Department of Defense, 2006). The Defense Planning Guidance is currently classified secret.
Kerry B. Fosher and Lauren Mackenzie, eds., The Rise and Decline of Military Culture Programs, 2004–2020 (Quantico, VA: Marine Corps University Press, 2021).
For more on great power competition, see Donald J. Trump, National Security Strategy of the United States of America, December 2017 (Washington, DC: White House, 2017); James N. Mattis, Summary of the 2018 National Defense Strategy of the United States of America: Sharpening the American Military’s Competitive Edge (Washington, DC: Department of Defense, 2018); and Renewed Great Power Competition: Implications for Defense–Issues for Congress (Washington, DC: Congressional Research Service, 2021).
In this article, the term professional development refers to systematic efforts by an organization or professional body to develop knowledge, skills, and behaviors in personnel. Professional development encompasses training, education, and experiential assignments that serve a developmental purpose. Consistent with Army Training and Leader Development, Army Regulation 350-1 (Washington, DC: Department of the Army, 2017), training refers to structured, formal activities to improve performance of tasks, behaviors, or skills for known situations and specific conditions. Education refers to structured formal activities to improve knowledge, skills, abilities, or experiences to prepare personnel for a range of situations and conditions.
James B. Ellsworth, “Training or Performance Improvement?,” Military Review 80, no. 6 (November/December 2000): 3–8.
Army Transformation: Hearing before the House Committee on Armed Services, 108th Cong. (2004) (statement of Maj Gen Robert H. Scales Jr., USA [Ret]).
Transforming the U.S. Military’s Foreign Language, Cultural Awareness, and Regional Expertise Capabilities: Hearing before the Oversight and Investigations Subcommittee of the House Committee on Armed Services, 110th Cong. (2008) (statement of Gail H. McGinn, Deputy U.S. Under Secretary of Defense for Plans, Office of the Under Secretary of Defense for Personnel and Readiness).
DOD Regional and Cultural Capabilities: The Way Ahead (Washington, DC: Department of Defense, 2007); and Department of Defense Strategic Plan for Language Skills, Regional Expertise, and Cultural Capabilities, 2011–2016 (Washington, DC: Department of Defense, 2011).
Amie Ferris-Rotman, “Afghans Use Culture Guides to Cut ‘Insider’ Attacks,” Reuters, 6 September 2012. An insider attack can be defined as any attack that comes from within one’s own organization.
Charles Moskos, International Military Education and Multinational Military Cooperation (Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences, 2004).
Said Haddad, “Teaching Diversity and Multicultural Competence to French Peacekeepers,” International Peacekeeping 17, no. 4 (2010): 566–77, https://doi.org/10.1080/13533312.2010.516671; Tibor Szvircsev Tresch, “Cultural and Political Challenges in Military Missions: How Officers View Multiculturality in Armed Forces,” in Giuseppe Caforio, ed., Advances in Military Sociology: Essays in Honor of Charles C. Moskos, Contributions to Conflict Management, Peace Economics and Development, vol. 12A (Bingley, UK: Emerald Publishing, 2009), 111–37, https://doi.org/10.1108/S1572-8323(2009)000012A012; and Haldun Yalçınkaya and Yusuf Özer, “Another Lesson Learned in Afghanistan: The Concept of Cultural Intelligence,” International Peacekeeping 24, no. 3 (2017): 434–60, https://doi.org/10.1080/13533312.2016.1244485.
Angela R. Febbraro, Brian McKee, and Sharon L. Riedel, eds., Multinational Military Operations and Intercultural Factors (Neuilly-Sur-Seine, France: Research and Technology Organisation, North Atlantic Treaty Organization, 2008); Joseph Soeters and Tibor Szvircsev Tresch, “Towards Cultural Integration in Multinational Peace Operations,” Defence Studies 10, no. 1–2 (2010): 272–87, https://doi.org/10.1080/14702430903155175; and Maren Tomforde, “How Much Culture Is Needed?: The Intercultural Dilemma of the Bundeswehr in ISAF,” International Peacekeeping 17, no. 4 (2010): 526–38, https://doi.org/10.1080/13533312.2010.516664.
Remi M. Hajjar, “The Army’s New TRADOC Culture Center,” Military Review 86, no. 6 (November/December 2006): 89–92; and Barak A. Salmoni, “Advances in Predeployment Culture Training: The U.S. Marine Corps Approach,” Military Review 86, no. 6 (November/December 2006): 79–88.
Christopher Sims, The Human Terrain System: Operationally Relevant Social Science Research in Iraq and Afghanistan (Carlisle Barracks, PA: U.S. Army War College Press, 2015); and Sterilla A. Smith, “Army Culture and Foreign Language Program,” Military Intelligence Professional Bulletin 38, no. 1 (January/March 2012): 3–12.
Maxie McFarland, “Military Cultural Education,” Military Review 85, no. 2 (March/April 2005): 62–69; Montgomery McFate, “The Military Utility of Understanding Adversary Culture,” Joint Force Quarterly 38, no. 3 (2005): 42–48; Pauline Kusiak, “Sociocultural Expertise and the Military: Beyond the Controversy,” Military Review 88, no. 6 (November/December 2008): 65–76; and Maj Ben Connable, USMC (Ret), “The Human Terrain System Is Dead, Long Live . . . What?: Building and Sustaining Military Cultural Competence in the Aftermath of the Human Terrain System,” Military Review 98, no. 1 (January/February 2018): 25–33.
Brian R. Selmeski, Military Cross-Cultural Competence: Core Concepts and Individual Development (Montgomery, AL: U.S. Air Force Culture and Language Center, 2007).
Chairman of the Joint Chiefs of Staff Instruction 3126.01A, Language, Regional Expertise, and Culture (LREC) Capability Identification, Planning, and Sourcing (Washington, DC: Joint Chiefs of Staff, 2013); and Department of Defense Instruction 5160.70, Management of the Defense Language, Regional Expertise, and Culture (LREC) Program (Washington, DC: U.S. Department of Defense, 2016).
Army Training and Leader Development.
Marine Corps Language, Regional Expertise, and Culture Strategy: 2016–2020 (Quantico, VA: U.S. Marine Corps, 2015).
John A. Nagl, Learning to Eat Soup with a Knife (Chicago, IL: University of Chicago Press, 2002).
Jeffery Bearor and George Dallas, “Alternative Perspectives: Launching and Running the Marine Corps’ Culture Center,” in Fosher and Mackenzie, The Rise and Decline of Military Culture Programs, 162–201; and Allison Abbe and Melissa Gouge, “Cultural Training for Military Personnel: Revisiting the Vietnam Era,” Military Review 92, no. 4 (July/August 2012): 9–13.
The U.S. Army Learning Concept for Training and Education, 2020–2040, TRADOC Pamphlet 525-8-2 (Fort Eustis, VA: U.S. Army Training and Doctrine Command, 2017).
Robert Greene Sands and Allison Greene-Sands, eds., Cross-Cultural Competence for a Twenty-First-Century Military: Culture, the Flipside of COIN (Lanham, MD: Lexington Books, 2013).
Donald L. Kirkpatrick and James D. Kirkpatrick, Evaluating Training Programs: The Four Levels, 3d ed. (San Francisco, CA: Berrett-Koehler Publishers, 2006).
Bradford S. Bell et al., “100 Years of Training and Development Research: What We Know and Where We Should Go,” Journal of Applied Psychology 102, no. 3 (2017): 305–23, https://doi.org/10.1037/apl0000142; Eduardo Salas and Janis A. Cannon-Bowers, “The Science of Training: A Decade of Progress,” Annual Review of Psychology 52, no. 1 (February 2001): 471–99, https://doi.org/10.1146/annurev.psych.52.1.471; and John E. Mathieu and Paul E. Tesluk. “A Multilevel Perspective on Training and Development Effectiveness,” in Steve W. J. Kozlowski and Eduardo Salas, eds., Learning, Training, and Development in Organizations (New York: Taylor and Francis Group, 2010), 405–42.
Heeyoung Han and David Boulay, “Reflections and Future Prospects for Evaluation in Human Resource Development,” New Horizons in Adult Education and Human Resource Development 25 no. 2 (Spring 2013): 6–18, https://doi.org/10.1002/nha.20013; and Eduardo Salas et al., “The Science of Training and Development in Organizations: What Matters in Practice,” Psychological Science in the Public Interest 13 no. 2 (2012): 74–101, https://doi.org/10.1177/1529100612436661.
Evaluating Learning: Getting to Measurements that Matter (Alexandria, VA: Association for Talent Development, 2016).
Carolyn J. Heinrich, “Measuring Public Sector Performance and Effectiveness,” in B. Guy Peters and Jon Pierre, eds., The SAGE Handbook of Public Administration (Los Angeles, CA: Sage Publications, 2012), 32–49, http://dx.doi.org/10.4135/9781446200506.
Doris B. Collins, “Performance-Level Evaluation Methods in Management Development Studies from 1986 to 2000,” Human Resource Development Review 1 no. 1 (2002): 91–110, https://doi.org/10.1177/1534484302011005.
John Boiney and David Foster, Progress and Promise: Research and Engineering for Human Sociocultural Behavior Capability in the U.S. Department of Defense (Mclean, VA: MITRE, 2013); Allison Abbe and Rebecca Bortnick, Developing Intercultural Adaptability in the Warfighter: A Workshop on Cultural Training and Education (Arlington, VA: U.S. Army Research Institute for the Behavioral and Social Sciences); and Kyle Behymer et al., “Training for Sociocultural Behavior Understanding in Operational Environments,” in Jill Egeth, Gary L. Klein, and Dylan Schmorrow, eds., Sociocultural Behavior Sensemaking: State of the Art in Understanding the Operational Environment (Mclean, VA: MITRE, 2015): 87–106.
Evaluating Learning.
“2018 Training Industry Report,” Training Magazine 55, no. 6 (November/December 2018): 18–31.
Satish P. Deshpande and Chockalingam Viswesvaran, “Is Cross-Cultural Training of Expatriate Managers Effective: A Meta Analysis,” International Journal of Intercultural Relations 16, no. 3 (Summer 1992): 295–310, https://doi.org/10.1016/0147-1767(92)90054-X; and Mark A. Morris and Chet Robie, “A Meta-Analysis of the Effects of Cross-Cultural Training on Expatriate Performance and Adjustment,” International Journal of Training and Development 5 no. 2 (June 2001): 112–25, https://doi.org/10.1111/1468-2419.00126.
Daniel J. Kealey and David R. Protheroe, “The Effectiveness of Cross-Cultural Training for Expatriates: An Assessment of the Literature on the Issue,” International Journal of Intercultural Relations 20, no. 2 (Spring 1996): 141–65, https://doi.org/10.1016/0147-1767(96)00001-6; and Mark E. Mendenhall et al., “Evaluation Studies of Cross-Cultural Training Programs: A Review of the Literature 1988 to 2000,” in Dan Landis, Janet M. Bennett, and Milton J. Bennett, eds., Handbook of Intercultural Training (Thousand Oaks, CA: Sage Publications, 2004), 129–43, http://dx.doi.org/10.4135/9781452231129.
Milton J. Bennett, “Towards Ethnorelativism: A Developmental Model of Intercultural Sensitivity,” in R. Michael Paige, ed., Education for the Intercultural Experience (Yarmouth, ME: Intercultural Press, 1993), 21–71; Dharm P. S. Bhawuk, “The Role of Culture Theory in Cross-Cultural Training: A Multimethod Study of Culture-Specific, Culture-General, and Culture Theory-Based Assimilators,” Journal of Cross-Cultural Psychology 29, no. 5 (1998): 630–55, https://doi.org/10.1177/0022022198295003; J. Stewart Black and Mark E. Mendenhall, “The U-Curve Adjustment Hypothesis Revisited: A Review and Theoretical Framework,” Journal of International Business Studies 22, no. 2 (1991): 225–47; https://doi.org/10.1057/palgrave.jibs.8490301; Jan Selmer, “Culture Shock in China?: Adjustment Pattern of Western Expatriate Business Managers,” International Business Review 8 (1999), 515–34; and Rosalie L. Tung, “Managing Cross-National and Intra-National Diversity,” Human Resource Management 32, no. 4 (Winter 1993): 461–77, https://doi.org/10.1002/hrm.3930320404.
Jaz Azari, Christopher Dandeker, and Neil Greenberg, “Cultural Stress: How Interactions with and among Foreign Populations Affect Military Personnel,” Armed Forces and Society 36, no. 4 (2010): 585–603, https://doi.org/10.1177/0095327X09358648; and Jessica A. Gallus and Jennifer L. Klafehn, “Developing Cross-Cultural Competence Following Negative Cross-Cultural Experiences,” in Sands and Greene-Sands, Cross-Cultural Competence for a Twenty-First-Century Military, 175–93.
William S. Howell, The Empathic Communicator (Belmont, CA: Wadsworth, 1982); and Ronald Fischer, “Cross-Cultural Training Effects on Culture Essentialism Beliefs and Cultural Intelligence,” International Journal of Intercultural Relations 35, no. 6 (November 2011): 767–75, https://doi.org/10.1016/j.ijintrel.2011.08.005.
Jeffrey R. Watson, Peter Siska, and Richard L. Wolfel, “Assessing Gains in Language Proficiency, Cross-Cultural Competence, and Regional Awareness during Study Abroad: A Preliminary Study,” Foreign Language Annals 46, no. 1 (2013): 62–79, https://doi.org/10.1111/flan.12016.
Katie Gunther, “Cross-Cultural Competence in the Classroom: Measuring Instructional Effectiveness,” in Sands and Greene-Sands, Cross-Cultural Competence for a Twenty-First-Century Military, 259–81.
J. Stewart Black, Mark Mendenhall, and Gary Oddou, “Toward a Comprehensive Model of International Adjustment: An Integration of Multiple Theoretical Perspectives,” Academy of Management Review 16, no. 2 (April 1991): 291–317, https://doi.org/10.2307/258863.
Linn Van Dyne, Soon Ang, and Christine Koh, “Development and Validation of the CQS: The Cultural Intelligence Scale,” in Soon Ang and Linn Van Dyne, eds., Handbook of Cultural Intelligence: Theory, Measurement, and Applications (New York: Taylor and Francis Group, 2008), 16–40.
John A. McLaughlin and Gretchen B. Jordan, “Using Logic Models,” in Joseph S. Wholey, Harry P. Hatry, and Kathryn E. Newcomer, eds., Handbook of Practical Program Evaluation (San Francisco, CA: John Wiley and Sons, 2004), 7–32.
Michael T. Rehg, Michael J. Gundlach, and Reza A. Grigorian, “Examining the Influence of Cross-Cultural Training on Cultural Intelligence and Specific Self-Efficacy,” Cross-Cultural Management: An International Journal 19, no. 2 (2012), 215–32, https://doi.org/10.1108/13527601211219892.
For example, see Robert DeKeyser, “Skill Acquisition Theory,” in Bill VanPatten and Jessica Williams, eds., Theories in Second Language Acquisition: An Introduction, 2d ed. (New York: Routledge: 2015), 94–112.
Deshpande and Viswesvaran, “Is Cross-Cultural Training of Expatriate Managers Effective,” 295–310; and Morris and Robie, “A Meta-Analysis of the Effects of Cross-Cultural Training on Expatriate Adjustment and Performance,” 112–25.
J. Stewart Black and Mark Mendenhall, “A Practical but Theory-Based Framework for Selecting Cross-Cultural Training Methods,” Human Resource Management 28, no. 4 (Winter 1989): 511–39, https://doi.org/10.1002/hrm.3930280406.
Martine C. Gertsen, “Intercultural Competence and Expatriates,” International Journal of Human Resource Management 1, no. 3 (March 1990): 341–62, https://doi.org/10.1080/09585199000000054; and William B. Gudykunst, Ruth M. Guzley, and Mitchell R. Hammer, “Designing Intercultural Training,” in Dan Landis and Rhabi S. Bhagat, eds., Handbook of Intercultural Training, 2d ed. (Thousand Oaks, CA: Sage Publications, 1996), 61–80.
Jonas F. Puck, Markus G. Kittler, and Christopher Wright, “Does It Really Work?: Re-Assessing the Impact of Pre-Departure Cross-Cultural Training on Expatriate Adjustment,” International Journal of Human Resource Management 19, no. 12 (December 2008): 2182–97, https://doi.org/10.1080/09585190802479413.
Sandra M. Fowler and Masahisa Yamaguchi, “An Analysis of Methods for Intercultural Training,” in Dan Landis and Dharm P. S. Bhawuk, eds., The Cambridge Handbook of Intercultural Training, 4th ed. (Cambridge, UK: Cambridge University Press, 2020), 192–257, https://doi.org/10.1017/9781108854184; and Lisa N. Littrell and Eduardo Salas, “A Review of Cross-Cultural Training: Best Practices, Guidelines, and Research Needs,” Human Resource Development Review 4, no. 3 (2005): 305–34, https://doi.org/10.1177/1534484305278348.
Kurt Kraiger, “Decision-Based Evaluation,” in Kurt Kraiger, ed., Creating, Implementing, and Managing Effective Training and Development: State-of-the-Art Lessons for Practice (San Francisco, CA: Jossey-Bass, 2002), 331–75; Darlene Russ-Eft and Hallie Preskill, “In Search of the Holy Grail: Return on Investment Evaluation in Human Resource Development,” Advances in Developing Human Resources 7, no. 1 (2005): 71–85, https://doi.org/10.1177/1523422304272169; and Scott I. Tannenbaum and Steven B. Woods, “Determining a Strategy for Evaluating Training: Operating within Organizational Constraints,” Human Resource Planning 15, no. 2 (1992): 63–81.
Kirkpatrick and Kirkpatrick, Evaluating Training Programs; Russ-Eft and Preskill, “In Search of the Holy Grail”; and Greg G. Wang and Dean R. Spitzer, “Human Resource Development Measurement and Evaluation: Looking Back and Moving Forward,” Advances in Developing Human Resources 7, no. 1 (2005): 5–15, https://doi.org/10.1177/1523422304272077.
David S. Bushnell, “Input, Process, Output: A Model for Evaluating Training,” Training and Development Journal 44, no. 3 (March 1990): 41–44; Mark W. Lipsey and David S. Cordray, “Evaluation Methods for Social Intervention,” Annual Review of Psychology 51 (2000): 345–75, https://doi.org/10.1146/annurev.psych.51.1.345; and Salas and Cannon-Bowers, “The Science of Training.”
Salas et al., “The Science of Training and Development in Organizations.”
Bell et al., “100 Years of Training and Development Research”; Ashley M. Hughes et al., “A Checklist for Facilitating Training Transfer in Organizations,” International Journal of Training and Development 22, no. 4 (December 2018): 334–45, https://doi.org/10.1111/ijtd.12141; and Darlene Russ-Eft, “A Typology of Training Design and Work Environment Factors Affecting Workplace Learning and Transfer,” Human Resource Development Review 1, no. 1 (2002): 45–65, https://doi.org/10.1177/1534484302011003.
Littrell and Salas, “A Review of Cross-Cultural Training.”
Morris and Robie, “A Meta-Analysis of the Effects of Cross-Cultural Training on Expatriate Adjustment and Performance.”
Training and Educational Development in Support of the Institutional Domain, TRADOC Pamphlet 350-70-14 (Fort Eustis, VA: U.S. Army Training and Doctrine Command, 2021).
Salas et al., “The Science of Training and Development in Organizations.”
The interviews were conducted by the Institute for Defense Analyses under contract with the Office of the Secretary of Defense.
For a related review of culture-general training, see Kristin Post, Analysis of ‘Culture-General’ Concepts in DoD Training Programs (Quantico, VA: Center for Advanced Operational Culture Learning, 2015). For additional resources, see Kerry Fosher et al., Culture General Guidebook for Military Professionals (Quantico, VA: Center for Advanced Operational Culture Learning, 2017); and Lauren Mackenzie, Eric Gauldin, and Erika Tarzi, 2018 Military Cross-Cultural Competence Annotated Bibliography (Quantico, VA: Center for Advanced Operational Culture Learning, 2018).
Language, Regional Expertise, and Culture (LREC) Capability Identification, Planning, and Sourcing; and Management of the Defense Language, Regional Expertise, and Culture (LREC) Program.
Allison Abbe and Jessica A. Gallus, The Socio-Cultural Context of Operations: Culture and Foreign Language Learning for Company-Grade Officers, Technical Report 1316 (Fort Belvoir, VA: U.S. Army Research Institute for the Behavioral and Social Sciences, 2012); Hannah Foldes et al., Sociocultural Components of Mission Performance: Development of a Taxonomy of Performance Requirements (Fort Belvoir, VA: U.S. Army Research Institute for the Behavioral and Social Sciences, 2012); and Chaitra M. Hardison et al., Cross-Cultural Skills for Deployed Air Force Personnel (Santa Monica, CA: Rand, 2009).
Michael A. Campion et al., “Doing Competencies Well: Best Practices in Competency Modeling,” Personnel Psychology 64, no. 1 (Spring 2011): 225–62, https://doi.org/10.1111/j.1744-6570.2010.01207.x.
Salas et al., “The Science of Training and Development in Organizations.”
Paula Holmes-Eber, Erika Tarzi, and Basema Maki, “U.S. Marines’ Attitudes Regarding Cross-Cultural Capabilities in Military Operations: A Research Note,” Armed Forces and Society 42, no. 4 (2016): 741–51, https://doi.org/10.1177/0095327X15618654; Erika Tarzi, Marines’ Views on How to Improve Marine Corps Cultural and Language Training and Education Programs: Training and Education Recipient and Learning Timing (Quantico, VA: Center for Advanced Operational Culture Learning, 2012); and Do Marines Value and Use Culture? (Quantico, VA: Center for Advanced Operational Culture Learning, 2013).
“Implementation of the Regional, Culture, and Language Familiarization Program,” MARADMIN 619/12 (Quantico, VA: Marine Corps University, 2012).
Erika Tarzi and Kerry Fosher, Regional, Culture, and Language Familiarization Program Messaging (Quantico, VA: Center for Advanced Operational Culture Learning, 2017).
Lisa N. Littrell et al., “Expatriate Preparation: A Critical Analysis of 25 Years of Cross-Cultural Training Research,” Human Resource Development Review 5, no. 3 (2006): 355–88, https://doi.org/10.1177/1534484306290106.
Salas et al., “The Science of Training and Development in Organizations.”
DOD Training: DoD Has Taken Steps to Assess Common Military Training (Washington, DC: Government Accountability Office, 2017).
Hajjar, “The Army’s New TRADOC Culture Center.”
For examples, see Kerry Fosher et al., Culture General Guidebook for Military Professionals.
P. Christopher Earley and Soon Ang, Cultural Intelligence: Individual Interactions across Cultures (Stanford, CA: Stanford University Press, 2003); Mark E. Mendenhall and Gary Oddou, “The Dimensions of Expatriate Acculturation: A Review,” Academy of Management Review 10, no. 1 (January 1985): 39–47, https://doi.org/10.2307/258210; and John W. Miller and Jennifer S. Tucker, “Addressing and Assessing Critical Thinking in Intercultural Contexts: Investigating the Distance Learning Outcomes of Military Leaders,” International Journal of Intercultural Relations 48 (2015): 120–36, https://doi.org/10.1016/j.ijintrel.2015.07.002.
Dharm P. S. Bhawuk and Richard W. Brislin, “Cross-Cultural Training: A Review,” Applied Psychology: An International Review 49, no. 1 (2000): 162–91, https://doi.org/10.1111/1464-0597.00009.
Tomasz Lenartowicz, James P. Johnson, and Robert Konopaske, “The Application of Learning Theories to Improve Cross-Cultural Training Programs in MNCs,” International Journal of Human Resource Management 25, no. 12 (2014): 1697–1719, https://doi.org/10.1080/09585192.2013.860384.
Littrell et al., “Expatriate Preparation”; and Bhawuk and Brislin, “Cross-Cultural Training.” The culture assimilator is a scenario-based method in which students choose interpretations of or responses to a critical incident and receive feedback on how culturally appropriate their choices are, within themes or categories that are important for intercultural interaction. The contrast American technique is a live interaction with a role player trained to act inconsistently with U.S. cultural norms, designed to enhance cultural self-awareness in the student. Attribution training aims to improve intercultural cognition by teaching students to make attributions for behaviors that are more consistent (or isomorphic) with the counterpart’s culture than with the student’s own cultural assumptions.
John Bird, “Culture, Regional Expertise and Language (CREL) Competency: Ramping-Up for Global Response and Regional Engagement,” Military Intelligence Professional Bulletin 40, no. 3 (July–September 2014): 6–11.
Geert Hofstede and Michael H. Bond, “Hofstede’s Culture Dimensions: An Independent Validation Using Rokeach’s Value Survey,” Journal of Cross-Cultural Psychology 15 no. 4 (1984): 417–33, https://doi.org/10.1177/0022002184015004003.
Barak A. Salmoni and Paula Holmes-Eber, Operational Culture for the Warfighter: Principles and Applications (Quantico, VA: Marine Corps University Press, 2008).
Shan Ran and Jason L. Huang, “Enhancing Adaptive Transfer of Cross-Cultural Training: Lessons Learned from the Broader Training Literature,” Human Resource Management Review 29, no. 2 (2019): 239–52, https://doi.org/10.1016/j.hrmr.2017.08.004.
Bell et al., “100 Years of Training and Development Research”; and Salas et al., “The Science of Training and Development in Organizations.”
Language and Culture Training: Opportunities Exist to Improve Visibility and Sustainment of Knowledge and Skills in Army and Marine Corps General Purpose Forces (Washington, DC: Government Accountability Office, 2011).
Michael Hosie et al., “Multinational Staff Assignments: Cross-Cultural Preparation,” Journal of Character and Leadership Development 7, no. 1 (Winter 2020): 77–90.
Herman Aguinis and Kurt Kraiger, “Benefits of Training and Development for Individuals, Teams, Organizations, and Society,” Annual Review of Psychology 60 (2009): 451–74.
Kathryn J. Holland, Verónica Caridad Rabelo, and Lilia M. Cortina, “Sexual Assault Training in the Military: Evaluating Efforts to End the ‘Invisible War’,” American Journal of Community Psychology 54, no. 3–4 (2014): 289–303, https://doi.org/10.1007/s10464-014-9672-0.
Lisa M. Harmon et al., “A Review of the Effectiveness of Military Suicide Prevention Programs in Reducing Rates of Military Suicides,” Journal of Human Behavior in the Social Environment 26, no. 1 (2016): 15–24, https://doi.org/10.1080/10911359.2015.1058139.
Salas et al., “The Science of Training and Development in Organizations”; and Hughes et al., “A Checklist for Facilitating Training Transfer in Organizations.”
David Matsumoto and Hyisung C. Hwang, “Assessing Cross-Cultural Competence: A Review of Available Tests,” Journal of Cross-Cultural Psychology 44, no. 6 (2013): 849–73, https://doi.org/10.1177/0022022113492891.
Littrell and Salas, “A Review of Cross-Cultural Training.”
Paula M. Caligiuri, “Selecting Expatriates for Personality Characteristics: A Moderating Effect of Personality on the Relationship between Host National Contact and Cross-Cultural Adjustment,” Management International Review 40 (2000): 61–80, https://doi.org/10.4324/9781315252025-5.
Shalom H. Schwartz, “National Culture as Value Orientations: Consequences of Value Differences and Cultural Distance,” in Victor A. Ginsburgh and David Throsby, eds., Handbook of the Economics of Art and Culture, vol. 2 (Oxford, UK: Elsevier: 2014), 547–86; and Jesus F. Salgado and Maria Bastida, “Predicting Expatriate Effectiveness: The Role of Personality, Cross-Cultural Adjustment, and Organizational Support,” International Journal of Selection and Assessment 25, no. 3 (2017): 267–75, https://doi.org/10.1111/ijsa.12178.
Valerie Alexandra, “Predicting CQ Development in the Context of Experiential Cross-Cultural Training: The Role of Social Dominance Orientation and the Propensity to Change Stereotypes,” Academy of Management Learning and Education 17, no. 1 (2018): 62–78, https://doi.org/10.5465/amle.2015.0096.
Faruk Sahin, Sahin Gurbuz, and Onur Köksal, “Cultural Intelligence (CQ) in Action: The Effects of Personality and International Assignment on the Development of CQ,” International Journal of Intercultural Relations 39, no. 1 (2014): 152–63, https://doi.org/10.1016/j.ijintrel.2013.11.002.
Remi M. Hajjar, “Effectively Working with Military Linguists: Vital Intercultural Intermediaries,” Armed Forces and Society 43, no. 1 (2016): 92–114, https://doi.org/10.1177/0095327X16632333.
Hughes et al., “A Checklist for Facilitating Training Transfer in Organizations”; Harry J. Martin, “Improving Training Impact through Effective Follow-Up,” Journal of Management Development 29, no. 6 (2010): 520–34, https://doi.org/10.1108/02621711011046495; and Littrell and Salas, “A Review of Cross-Cultural Training.”
Littrell et al., “Expatriate Preparation.”
Gudykunst, Guzley, and Hammer, “Designing Intercultural Training.”
Christina N. Lacerenza et al., “Leadership Training Design, Delivery, and Implementation: A Meta-Analysis,” Journal of Applied Psychology 102, no. 12 (2017): 1686–1718, https://doi.org/10.1037/apl0000241.
Ashley M. Hughes et al., “The Role of Work Environment in Training Sustainment: A Meta-Analysis,” Human Factors 62, no. 12 (2019): 166–83, https://doi.org/10.1177/0018720819845988; and Brian D. Blume et al., “A Dynamic Model of Training Transfer,” Human Resource Management Review 29 no. 2 (2019): 270–83, https://doi.org/10.1016/j.hrmr.2017.11.004.
“Our Story,” U.S. Department of Defense, accessed 9 September 2021.
The U.S. Army Learning Concept for Training and Education: 2020–2040.
Christopher Dandeker and James Gow, “Military Culture and Strategic Peacekeeping,” Small Wars and Insurgencies 10, no. 2 (1999): 58–79, https://doi.org/10.1080/09592319908423241.
Fosher and Mackenzie, The Rise and Decline of Military Culture Programs.

expeditions with mcup

Evaluating Military Cross-Cultural Training Programs

Developing Cross-Cultural Training Programs

The Science of Training

Organizational Challenges in Training Evaluation

Challenges in Cross-Cultural Training Evaluation

Broadening Evaluation Methods for CCT

Planning

Design and Delivery

Program Management

Systems and Context

Present Study

Sample

Scope of the Study

Assessment Method

Findings

CCT Planning

Organizational Analysis

Job-Task Analysis

Learner Analysis

Design and Delivery

Timing and Length of Training

Learner Motivation

Method-Outcomes Match

Valid Training Design and Content

Practice and Feedback

Delivery Media

Program Management

Evaluation

Updates

Quality Assurance

Systems and Context

Summary of Findings

Conclusions

Improving Military Training

Future Research

Final Thoughts

Endnotes

Home

Acquisitions

Publications

Home

Acquisitions

Publications