This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (http://www.creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).
Guidance for measuring team effectiveness in dynamic clinical settings is necessary; however, there are no consensus strategies to help health care organizations achieve optimal teamwork. This systematic review aims to identify validated survey instruments of team effectiveness by clinical settings.
PubMed, MEDLINE, and ISI Web of Knowledge were searched for team effectiveness surveys deployed from 1990 to 2016. Validity and reliability were evaluated using 4 psychometric properties: interrater agreement, internal consistency, content validity, and structural integrity. Two conceptual frameworks, the Donabedian model and the Command Team Effectiveness model, assess conceptual dimensions most measured in each health care setting.
The 22 articles focused on surgical, primary care, and other health care settings. Few instruments report the required psychometric properties or feature non-self-reported outcomes. The major conceptual dimensions measured in the survey instruments differed across settings. Team cohesion and overall perceived team effectiveness can be found in all the team effectiveness measurement tools regardless of the health care setting. We found that surgical settings have distinctive conditions for measuring team effectiveness relative to primary or ambulatory care.
Further development of setting-specific team effectiveness measurement tools can help further enhance continuous quality improvements and clinical outcomes in the future.
Keywords: Team effectiveness, surveys, systematic review, health care quality assessmentToday, team-based care has become a key component of many transformations in health care delivery and emerging models of value-based care. 1 The complexity of health care services, including a continuing trend toward value-based care and pay for performance, has elevated the importance of team-based care in the deliverance of health services.
Previous research indicates that higher team effectiveness is associated with better health outcomes. 2 –4 The impact of high-functioning teams on quality of care, worker satisfaction, and cost of care can be substantial when it comes to surgical care, 5 intensive care, 6 ambulatory care, 7 and primary care managing patient populations with chronic conditions. 8 Despite this knowledge and growing awareness of the importance of teamwork among health care leaders, there are no consensus strategies to help health care organizations achieve optimal teamwork. 9 One of the first steps in achieving optimal team performance is the ability to measure, track, and influence team effectiveness. Therefore, systematic reviews of survey instruments measuring team effectiveness across health care settings provide an important first step. 9 –11
Among the previous systematic reviews on survey instruments, Valentine et al 11 inventoried and described a list of teamwork survey tools used in health care settings. Their findings indicate what dimensions of teamwork have been assessed, along with the psychometric validity of each survey. However, this study did not specify which health care setting each measurement tool and article has addressed. Other review studies on team effectiveness did not identify which survey tools should be applied in which type of setting. 9,10 We are left with the question of which conceptual dimensions are most relevant to various types of health care settings and what survey instruments are most often deployed in these settings. The complexity and dynamic nature of these settings create different conditions for teamwork; therefore, understanding these specific requirements is key in developing measurement tools. There is an apparent need for guidance on understanding the contextual nature of teamwork skills and performance for different settings, including surgery, intensive care, emergency medicine, and ambulatory care settings.
This study seeks to identify validated survey instruments by clinical settings. This objective renews and complements findings from the study by Valentine et al, 11 building on psychometric properties and concepts used in available survey instruments. Our evaluation of survey instruments is supported by 2 conceptual frameworks: one rooted in outcomes research from the field of health services research 12 and the other in team theory and organizational psychology. 13 By identifying which content domains were assessed and how the domains differ by team environment, the findings of this study can assist in the development of more specialized team member training and operational design interventions directed to the most appropriate team composition, team member tasks, and responsibilities by clinical settings. The findings will also benefit practitioners who wish to ascertain the usefulness and relevance of a particular tool for their care teams in various health care settings.
Teams are defined as “two or more people with different tasks who work together adaptively to achieve specified and shared goals.” 14 (p4) Compared with teams in other industries, health care teams have more dynamic work conditions that change frequently, have to change team membership in a short-term period, have various specialized members, and have interprofessional and even multidisciplinary cultures. 9 These unique conditions vary across health care settings. Teams in operating rooms and emergency medicine are more likely to experience changes in team memberships that may be assembled ad hoc, whereas teams in primary care can have more diverse team compositions which include physicians, nurse practitioners, medical assistants, and receptionists, compared with surgical teams. 9,15 Such different work conditions and diverse team compositions require integrating multiple conceptual frameworks for evaluating team effectiveness.
We employed 2 conceptual models as frameworks for evaluating the contextual nature of team effectiveness across different settings—(1) the Donabedian 12 model on quality of care and (2) the Command Team Effectiveness 16 (CTEF) model. Both conceptual frameworks highlight outcome domains in the streams of care that fit our focus on team effectiveness. Donabedian contends that it is important to identify essential elements constituting quality of care based on structure, process, and outcome. Structure refers to the attributes of the material sources, human resource, and organizational structure. Process denotes the actual activities in giving and receiving care, including patients’ activities and practitioners’ activities. Outcome refers to the impact of care on the patients’ health status. Since then, many studies have confirmed that structure, process, and outcome should be evaluated together when considering quality of health care. 17 –19 Therefore, we assessed the relationship among the 3 subdimensions and investigated how survey instruments were used to measure these concepts across health care settings. This comprehensive framework also aids in determining which conceptual dimensions have been understudied when measuring team effectiveness.
In addition to the Donabedian model, we applied the CTEF model to refine the framework of analysis to be more relevant to surgical teams, which have the unique attributes of action teams. 20,21 According to organizational psychological literature, “action teams” refers to the teams, such as emergency medical, surgical teams, air crews, and military command and control, that require specialized professionals to collaborate in the context of high-acuity, complex tasks, ad hoc team compositions, and time-pressured conditions. 22,23 As our focus is identifying validated survey instruments according to the attributes of teams, the CTEF model allows us to closely investigate key subcomponents of team effectiveness in surgical settings according to their unique team characteristics. 24
In the CTEF model, conditions include dimensions of mission framework, task, organizational characteristics, leadership, and the characteristics of team members. These dimensions are addressed in the generic team effectiveness frameworks in health care as well, 20,25 but the CTEF model measures outcomes based on 2 categories—task outcomes and team outcomes. Task-related outcomes include time-error costs, task accomplishment quality, accuracy, timeliness, and error rate, whereas team-related outcomes measure team satisfaction, team norms, roles, communication patterns, motivation, attitudes, emotional tone, and turnover. By differentiating task-related and team-related outcomes, the CTEF model provides a more specific criterion for the evaluation of Donabedian structure, process, and outcomes relevant to action teams in surgical settings.
We conducted a systematic review of the literature, searching for survey instruments measuring team effectiveness. We conceptualized 3 relevant dimensions—survey instruments, clinical setting, and team effectiveness—as search points. Compared with Valentine et al, 11 we expanded the scope of search dimensions and database to investigate team effectiveness survey tools across different health care settings. Next, we identified key terms for each dimension. For the survey instrument dimension, we selected “survey,” “evaluation,” “instrument,” “assessment,” and “questionnaire” as the search terms. For the dimension of the clinical setting, we included key terms such as “clinical,” “health care,” and “surgical” to limit our search to only the health care domains. Because our focus is team effectiveness itself, we included the key term “team effectiveness” for the last dimension. The focus of this study was to identify and evaluate survey instruments of team effectiveness related to health outcomes; the general studies on team(s) or teamwork(s) without outcome domains were excluded.
We used 3 databases for this systematic review of the literature: PubMed, MEDLINE via OVID, and ISI Web of Knowledge. We included articles using the selected key terms from January 1990 to October 2016. In every search step, we combined key terms in the 3 dimensions using AND operator (eg, “survey” AND “clinical” AND “team effectiveness”). We limited our search to titles and abstracts. This search strategy identified articles in the overlapping areas of the 3 dimensions. The first round of searching produced 646 articles of interest. After deleting duplicates (492), a total of 191 articles were included in our final review.
Next, we conducted an abstract review to select those articles that matched our predefined inclusion criteria: (1) articles should be peer-reviewed, (2) articles should be empirical studies on teams, (3) the studies should demonstrate the use of survey instruments, and (4) the survey instrument should include a team effectiveness measure. In all, 22 articles met the criteria for full-text review. Researchers extracted information from the selected articles focusing on team compositions, types of settings, types of surveys, dimensions of team effectiveness, and significant impact on non-self-reporting outcomes (if applicable). In the final step, researchers evaluated reliability and validity of survey tools for each article. Figure 1 summarizes the research approach and search steps in a flowchart using the PRISMA methodology. 26
PRISMA flowchart for systematic review.
Following Donabedian and the CTEF model, we first categorized what type of questions researchers asked health care team members based on structure, process, and outcome. Then, we identified the setting to describe organizational preconditions of team effectiveness. Two of the researchers independently reviewed items in each survey tool identified in the literature to categorize based on these criteria, following terminologies in Mathieu et al. 25 We then qualitatively assessed the subcategories based on survey items that describe consistent terminologies.
To evaluate the reliability and validity of each survey instrument, we used 4 psychometric properties: interrater agreement (IRA), internal consistency, content validity, and structural integrity. These 4 properties have been used by other researchers to assess how accurately a survey instrument captures what it is intended to measure. 11,27,28
Interrater agreement and interrater reliability (IRR) measure whether different raters provide similar or identical reports when faced with same survey instruments. Particularly, when assessing survey instruments of teamwork including multiple groups of professionals, researchers should report both IRA and IRR to justify the aggregated scores at the group levels. 29 When a team has different groups of professionals, such as physicians, nurses, specialists, or administrative staff, IRA and IRR indicate whether a survey produces similar responses on the conceptual dimensions across the groups of participants. Interrater agreement is measured by the rwg index, 30 which ranges from 0 to 1, where .7 is used as the minimum threshold for an acceptable value. 30,31 Interrater reliability is measured by an intraclass correlation coefficient (ICC) with a range of −1 and 1, where values of ICC should be greater than 0 as a threshold for acceptable similarity. 32 Interrater agreement and interrater reliability are the key properties in a survey measuring team effectiveness because the survey aims to assess team members’ behaviors and achievement as a group. When IRA and IRR produce satisfactory values, it assures the reliability of the survey instrument to measure a team of individuals.
Internal consistency indicates the degree to which survey items are correlated to each other. When items are strongly correlated, it assures that the survey reflects similar concepts across items. Cronbach α is most commonly used to measure internal consistency, ranged from negative infinity to 1, with values greater than .7 defined as acceptable consistency across items. 33
Content validity refers to whether a survey accurately measures the substantive meanings of the conceptual dimensions of interest. To test content validity of a survey, triangulation is highly recommended, which requires the use of different methodologies other than a survey, such as interview, qualitative field research, expert-reviewed survey, formal pretest, or pilot survey, to measure the same conceptual dimensions of interest. When developing survey instruments of team effectiveness in different settings, researchers also conduct systematic reviews to develop items and apply previously validated scales. This is the only measure of validity in this study that reflects whether a survey captures the true dimensions that are of real-world interest.
Structural integrity refers to the extent to which survey items are clustered with a high covariance. When a survey aims to measure single conceptual dimensions, all survey items should be constructed in one dimension as expected. Structural integrity reveals the number of conceptual dimensions by assessing covariance among survey items and provides the evidence of construct dimensionality. Exploratory and confirmatory factor analyses provide the percentage of variance that can be explained by the constructed factors. When the factor loading value is greater than 0.40 and eigenvalue is greater than 1.0, the structural integrity of the survey is acceptable. 34,35
Human subjects were not involved in this systematic review. Ethics review and study registration do not apply. The data sets used during the study are available on reasonable request to the corresponding author.
We analyzed 22 articles with survey instruments measuring team effectiveness. The articles address a variety of clinical settings, types of respondents, variables of interest, and team compositions as described in Table 1 . Of the 22 articles, 19 articles administered survey questions to team members in formal/informal clinical teams or potential team members (eg, health care faculties who train clinical professions), whereas 3 articles used third-party experts 52,53 and patients 46 as respondents who assessed the clinical team effectiveness.
Description of clinical setting and team composition.
| Article characteristics | Team composition | ||||||
|---|---|---|---|---|---|---|---|
| Author(s), year | Settings | Types of respondents | Role of team effectiveness | Key concepts | MD | Nurses | Other |
| Surgical setting | |||||||
| La Sala et al, 2015 36 | Intensive care | Team members | DV | Team functioning | x | x | Health care operators |
| Andrew et al, 2012 37 | Operating room | Team members | IV | Teamwork, team communication, team effectiveness, quality expectations, possession of relevant skills | x | ||
| Hamilton et al, 2009 38 | Trauma resuscitations | Third-party experts | DV | Team skills during trauma resuscitation | Postgraduate year—2 surgical trainees | ||
| Michinov et al, 2008 23 | Anesthesia care | Team members | D | Transactive memory, perceived team effectiveness, job satisfaction | x | x | |
| Simon and Stewart, 2007 39 | Operating room | Team members | DV | Stress, deference, confidence, leadership, communication, teamworking, preferred leadership style and subjective experience of teamworking | x | x | Consultant, sister, health care assistant, midwife, physiotherapists, matron/ward manager, operating dept. practitioner, and administrator |
| Davenport et al, 2007 3 | General/vascular surgery | Team members | IV | Emotional exhaustion; safety, teamwork, working conditions, recognition of stress effects, perception of management, job satisfaction | x | x | Physician assistants, nurse practitioners, surgical residents or fellows, anesthesia residents or fellows, certified registered nurse anesthetists, staff nurses |
| Primary care setting | |||||||
| Landry and Erwin, 2015 40 | Primary care | Team members | D | Team participation, team processes, team processes | Health care executives | ||
| Song et al, 2015 41 | Primary care | Team members | D | Team effectiveness, shared understanding, perceived collective identity, perceived team effectiveness | x | x | |
| Beaulieu et al, 2014 42 | Primary care | Team members | DV | Team functioning | x | x | Social workers |
| Helfrich et al, 2014 19 | Primary care | Team members | IV | Team structure, team process, team effectiveness | x | x | Teamlets; physician assistant, nurse care managers, administrative clerks |
| Becker and Roblin, 2008 43 | Primary care | Team members | IV | Practice climate | x | x | Support staff |
| Other setting | |||||||
| Keebler et al, 2014 44 | US Army Medical Center | Team members | D | TeamSTEPPS (leadership, communication, situation monitoring, mutual support) | x | x | 1700 multidisciplinary health care professions and support staff from the US Army medical facilities |
| El Ansari et al, 2016 45 | Mental Health | Team members | D | Community mental health teams (CMHTs) effectiveness | x | x | Administrative and clerical staff, social workers/mental health setting |
| Tremblay et al, 2015 46 | Ambulatory Cancer Clinic | Patients | DV | Patient Perceptions of the prompt access to care, person-centered response, quality of patient-provider communication, and quality of care environment | x | x | Navigators in oncology/ambulatory cancer clinic |
| Strasser et al, 2014 47 | VA Medical Services | Team members | IV | Team functioning | x | x | Occupational therapy, physical therapy, social workers/case management, speech-language pathologists/medical services at VA hospitals |
| Dounis et al, 2013 48 | Diabetes training program | Team members | DV | Perceived interprofessional team effectiveness | x | x | Health care faculties from 6 different disciplines; medicine (MD), nursing (NP, RN), dentistry, therapy, pharmacy/type 2 diabetes training program |
| Baker et al, 2011 49 | 4 general hospitals | Team members | D | Challenging the process, enabling others to act, embracing change, doing the job, working with other | Health care workers (no specific description)/general hospital | ||
| Strasser et al, 2010 47 | Rehabilitation services at VA hospitals | Team members | D | Physician support, shared leadership, supervisor team support, teamness, team effectiveness | x | x | Occupational therapy, physical therapy, speech-language pathology, and social work or case management/rehabilitation services at VA hospitals |
| Strating and Nieboer, 2009 50 | Home care | Team members | D | Team climate | x | Management staff/home care | |
| Shortell et al, 2004 4 | Long-term care | Team members | DV | Perceived team effectiveness | x | x | Individuals on the breakthrough series (BTS) intervention teams/chronic illness program |
| Temkin-Greener et al, 2004 51 | Chronic Illness Program | Team members | DV | Interdisciplinary team process, team effectiveness | x | x | Social workers, therapists, dietitians, aides, drivers, coordinators/long-term care |
| Hyer et al, 2003 52 | Geriatric health care teams | Third-party experts | D | Team dynamics (team process and skills in addressing conflict) | x | x | Social workers, pharmacists/geriatric health care |
Abbreviations: D, development of a survey; DV, dependent variable; IV, independent variable; MD, medical doctor or physician.
We investigated how each article used team effectiveness as a variable of interest. Of the 22 articles, 9 articles focused on the development of the instrument and performed validity and reliability testing. As Table 1 indicates, those articles assessed dimensions of team effectiveness and analyzed the relationship among subdimensions of teamwork in terms of team functions, conditions of teamwork, leadership, and team effectiveness. As those articles mainly evaluated reliability and validity of newly developed survey questionnaires, most of the articles reported psychometric properties. Of the 22 articles, 5 articles measured team effectiveness as antecedents of health care outcomes, including task performance, patient trust, patient-centered care improvement, patient discharge rates, and length of stay. Eight articles were interested in measuring team effectiveness as consequences of care management, such as team training program, a simulation-setting training program, team functioning, team attitudes, team communication, and team quality.
Six articles focused on surgical settings, including trauma resuscitations, intensive care, anesthesia care, operating room setting, and general/vascular surgery. Five articles mainly discussed team effectiveness in primary care settings. The other 11 articles had a variety of clinical settings, such as a diabetes training program, a mental health hospital, a geriatric health care setting, chronic illness programs, Veterans Affairs (VA) hospitals, home care, long-term care, and ambulatory care settings. As team dynamics on team composition, functioning, structure, and process can vary depending on the clinical setting, we qualitatively analyzed which types of conceptual dimensions were most addressed in each type of clinical setting.
Team composition is key to measuring team effectiveness in the health care setting. We investigated what types of health care professionals were involved in teamwork. As Table 1 indicates, of the 22 articles, 17 survey instruments included physicians and nurses. Other professions often included in a team are clinical specialists, social workers, administrative clerks, health care executives, consultants, midwives, occupational therapy, dietitians, anesthesia residents, and receptionists. Such diverse team compositions indicate that it is important to consider professions’ characteristics, norms, cultures, and functions in a team when measuring team effectiveness.
The psychometric properties—IRA/IRR, internal consistency, content validity, and structural integrity—indicate whether a survey is a valid and reliable measure of team effectiveness. As noted in Table 2 , there were few studies that reported all 4 psychometric properties: of the 21 articles, only 4 articles (19%) reported all 4 psychometric properties. Interestingly, only 5 articles reported IRA/IRR score, and 1 of them had unacceptably low rwg, which indicates the need for more high-quality survey instruments with high validity and reliability. Compared with IRA/IRR, internal consistency was more frequently reported: 16 out of 21 articles reported Cronbach α values. When the studies aimed to develop a survey tool (or measure team effectiveness as a dependent variable), the internal consistency test was often conducted to show how the survey items were constructed. Content validity was reported in most of the studies: 19 studies report that their survey items were constructed through literature review, existing survey with validity tests, 3-phase qualitative study based on a formative evaluative approach, and expert interview. Most studies had adopted existing surveys from literature or modified survey items to fit into the health care domain. In terms of structural integrity, about half of the selected studies reported the factor loading value of their survey items, and 11 articles reported covariance among survey items with factor loading value and eigenvalue.
Psychometric properties of survey instruments on team effectiveness.
| Author, year | Survey name | Survey description | Interrater agreement/reliability | Internal consistency (Cronbach α) | Content validity | Structural integrity | Non-self-reported outcomes |
|---|---|---|---|---|---|---|---|
| La Sala et al, 2015 36 | Index of Interprofessional Team Collaboration | 14 items measuring team functioning, 6-point Likert scale | NR | .95 | Literature review to select existing survey measures of Index of Interprofessional Team Collaboration | Two dimensions, reflection on the processes and interdependence roles, were summarized in a general factor analysis | NR |
| Andrew et al, 2012 37,a | A newly developed survey based on the previously validated Comprehensive Assessment of Team Member Effectiveness (CATME) | 87-item survey measuring 5 categories (teamwork, team communication, team effectiveness, quality expectations, and possession of relevant skills) to assess team skills with an LVH (Laparoscopic Ventral Hernia) simulator, 5-point Likert scale | Cross-group differences tested using ANOVA: significant differences found in communication (P < .01) and quality of work (P < .05), but other 5 items were consistent across groups | .811 | Literature review to select survey instruments based on the research question. 6 items among 87 items were deemed appropriate for the LVH task | Significant interitem correlation among 6 items (r = .46-.86) | Survey measures were correlated with task performance on the LVH simulator |
| Hamilton et al, 2009 38 | A new prototype instrument using Mayo High Performance Teamwork Scale | 7 items to identify whether 7 attributes relevant to team skills in the trauma context are present or not; researchers validated this prototype by assigning a normalized ranking score to each of the 10 scenarios based on the score provided by each experienced clinician; binary fashion | Interobserver agreement was poor; rwg = .64 | NR; but researchers indicated that the original form of Mayo instrument performed poorly in their context | Literature review to select a relevant preexisting survey items; Mayo instruments were used for operationalizing principles of crises resource management, communication skills and containment of errors. The instrument was developed and tested in the context of trauma resuscitation. Researchers modified the instrument as 7 items to fit in the trauma context | NR; no factor analysis conducted. Researchers indicated that some items in Mayo instruments were seldom applicable to their context. The reason is unclear | NR |
| Michinov et al, 2008 23 | Survey including transactive memory system, perception of team effectiveness and work attitudes (Minnesota Job Satisfaction Questionnaire) | The survey included a set of dimensions of transactive memory (15 items, 5-point Likert scale), perceived team effectiveness (1 item, 10-point Likert scale), and job satisfaction (20 items, 5-point Likert scale) | NR | .74-.88 | Literature review to select survey items: Lewis; Job satisfaction survey items derived from Minnesota Job Satisfaction Questionnaire) | NR | NR; only self-reported perceptions on team effectiveness, job satisfaction and team identification |
| Simon and Stewart, 2007 39 | An abbreviated Operating Room Management Questionnaire (ORMAQ) | 30 questions measuring 8 themes; stress, deference, confidence, leadership, communication, teamworking, preferred leadership style and subjective experience of teamworking | NR | NR | Literature review and select instrument from Helmreich and Merritt’s ORMAQ, measuring subjects’ experience of stress and attitudes to leadership and teamworking | NR; researchers indicated that the instrument is highly valid and reliable by citing other works but did not report any psychometric properties | NR |
| Davenport et al, 2007 3 | Safety Attitudes Questionnaire | 30 items grouped into 6 measures of OSCF and an additional factor of emotional exhaustion; safety, teamwork, working conditions, recognition of stress effects, perception of management, and job satisfaction | NR | .54-.83 | Researchers adopted Safety Attitudes Question (SAQ) and measure emotional exhaustion using Maslach Burnout Inventory | NR | Risk-adjusted surgical morbidity and mortality; only communication with attending and resident doctors was correlated with risk-adjusted morbidity |
| Landry and Erwin, 2015 40 | Cross-sectional Survey for multidisciplinary teams | Respondents were asked to answer the questions of team participation and team processes; team processes had a set of 6 categories; binary fashion (advantage/disadvantage) | NR | NR | The survey items were developed based on the literature review on the leadership development and training as well as multidisciplinary team | NR | NR |
| Song et al, 2015 41,a | Primary Care Team Dynamics Survey | 31 items for measuring conditions for team effectiveness, shared understanding, 3 supportive processes perceived collective identity and perceived team effectiveness, 5-point Likert scale, full model | Full survey; rwg = .23-.75; ICC = .58-.81 | .71-.91 | Literature review to select 10 survey instruments based on conceptual model. A series of expert reviews and cognitive interview with attending physicians, nurses, and front desk staff were conducted to select 31 items | Two round SEM. Factor analysis generated 7 factors. FL > 0.40, GFI = 0.91 | NR |
| Beaulieu et al, 2014 42,a | Team Climate Inventory (TCI) Questionnaire (short version) | 19-item measuring 4 team functioning scales: participative safety (5-point) support for innovation (5-point), vision (7-point), and task orientation (7-point scale) | Full survey; rwg = .72-.99 | .88-.93 | Use the previously validated survey instrument. Literature indicates that TCI questionnaire is among the few survey tools that has been validated and used in many countries and contexts | Exploratory factor analysis were performed: r = .61-.83 | NR |
| Helfrich et al, 2014 19 | Cross-sectional survey for VA primary care in the 3 sets of measures—team structure, team process, and team effectiveness | Three sets of dimensions were covered in the survey items; structure questions about the formation of a 4-member teamlet, process questions about distribution of tasks among teamlet members and delegation of clinical activities, and effectiveness questions about assessing respondents’ collective efficacy | NR | NR | Literature review to select existing items (SOAP-C) and create new survey items; using Donabedian model and working with VA operational leaders, researchers constructed survey items based on structure, process, and effectiveness | NR | 5 clinic-level respondent-reported improvements in team’ abilities to deliver patient-centered care; team process and effectiveness measures had stronger associations with improvement than structure measures |
| Becker and Roblin, 2008 43 | Practice Climate Survey | There were 2 survey instruments, one for practitioners (41 items, 24 related to practice climate) and the other for support staff (31 items, 17 related to practice climate). The instruments are similar, with additional items concerning patient interaction on the practitioner instrument. All items consist of Likert scales, with 4, 5, or 7 response levels per item | NR; but researchers calculated an average of the practitioner scores and an average of staff scores and then computed the average of these 2 averages to obtain the team’s score | .80-.90 | NR | No specific test scores, but it indicated that the subcategory of practice climate survey were medical record availability, tame for tasks, delegation, patient focus, coordination, team ownership and autonomy | Patient trust in primary care physician (survey measure); significant and positive relationship exists between practice climate and patient trust |
| Keebler et al, 2014 44 | TeamSTEPPS (Team Strategies and Tools to Enhance Performance and Patient Safety) Teamwork Perceptions Questionnnaire (T-TPQ) | A self-report Survey including 35 items that examines multiple dimensions of perceptions of teamwork within various health care settings | NR | .97 | Literature review of evidence based on teamwork, patient safety, and team training to select 5 dimensions: Leadership, Mutual Support, Situation Monitoring, Communication, and Team Structure | CFI = 0.94; RMSEA = 0.06 | NR |
| El Ansari et al, 2016 45 | Community mental health teams (CMHTs) effectiveness survey | Through 3-phrase qualitative study and evaluating survey scales’ psychometric properties, 20 items demonstrated good measures of CMHTs, capturing 7 themes; 5-point Likert scale | ICC(2) = .57; rwg = .99 | .91 | Conducting 3-phase qualitative study based on a formative evaluative approach that employed an iterative process with various stakeholder; individual (N = 157) from the 11 Mental Health Trusts participated in 10 workshops to develop the survey items | Splitting sample at random into 2 halves: the first half was employed for exploratory factor analysis and the second half was employed for confirmatory factor analysis; all items factor loading >0.4; EV > 1, CFI > 0.90; Var Exp = 39.4% | NR |
| Tremblay et al, 2015 46 | Cancer Services Responsiveness tool (CSR); patient-reported survey | 19-item questionnaire evaluating patients’ perceptions of the responsiveness of cancer services, focusing on prompt access to care, person-centered response, quality of patient-provider communication, and quality of care environment; 4-point Likert scale | NR | .64-.85 | Researchers adopted the instrument from WHO’s generic responsiveness instrument. They translated into French and tested a validity of the adopted French version | NR | NR |
| Strasser et al, 2014 | Team Functioning Survey, focusing on medicine, nursing, therapy, social work, and case management staff in teams | 5 measures of team functioning are measured through staff surveys collected pre- and postintervention 1 year apart, t1 and t2; physician support, shared leadership, supervisor team support, teamness, and team effectiveness; 7-point Likert scale | NR | NR | Using Strasser et al (2010), researchers used valid survey instrument in t1 and t2, and then assess the association between changes in team functioning measures and patient outcomes | NR | Functional improvement, discharge destination, and length of stay were measured from VHA Functional Status Outcomes Database; Team functioning increased community discharge and decreased length of stay |
| Dounis et al, 2013 48 | Pre- and postresearch assessment surveys to evaluate the effectiveness of an interprofessional health care faculty training program | The research assessment survey consisted of 4 closed-ended items (multiple choice responses). 9 additional items were used to assess participant attitude and 5 items were used to measure perception about interprofessional teams | Test-retest was used on a small convenience sample to assess internal reliability; r = .83 | .75 | NR | NR | NR |
| Baker et al, 2011 49,a | A developed survey using 2 previous surveys: (Leadership Practices Inventory-Self [LPI]) and Performance and Relationship Questionnaire (PRQ) | 16 items measuring leadership and followership based on 5 categories: challenging the process, enabling others to act, embracing change, doing the job, and working with others: LPI (10-point Likert scale), PRQ (5-point Likert scale) | NR | .53-.80 | Literature review to select items: Leadership (Leadership Practices Inventory-Self [LPI]), Followership (the Performance and Relationship Questionnaire [PRQ]) | A confirmatory factor analysis were performed: 5-factor model fit better, and the root mean square error of approximation were indicative of acceptable model (GFI = 0.87) | NR |
| Strasser et al, 2010 47 | Team Functioning Survey, focusing on rehabilitation staff | 60 items if team functioning with 7-point Likert scale; physician support, shared leadership, supervisor team support, teamness, and team effectiveness | NR | .86-.94 | Literature review to modify a previous survey of Team Functioning Survey | IRT and factor analysis supported the proposed model: EV > 1, Var Exp = 68% | Measures of team functioning were related to patient outcomes of motor FIM gain and discharge disposition |
| Strating and Nieboer, 2009 50 | Team Climate Inventory (TCI), focusing on quality improvement teams | 14-item Dutch version with sample at baseline (T0) and end measurement (T1) | NR | .73-.80 | Researchers translated Team Climate Inventory (TCI) into Dutch. Comparison of the 2 translations revealed no salient differences. Perceived team effectiveness measures derived from Lemieux Charles’ measures | Confirmatory factor analysis proposed 4-factor structure—vision, participative safety, task orientation, and support for innovation; factor loading between 0.68 and 0.87. RMSEA = 0.03 | NR |
| Shortell et al, 2004 4 | A survey of individuals on the breakthrough series (BTS) intervention team | The survey included a set of questions on team members’ assessments of organizational culture and of their organization’s commitment to quality improvement and perceived team effectiveness after the completion of the intervention | NR | .85-.95 | Literature review to use the competing values framework for assessing organizational culture. Researchers also used Lemieux-Charles team effectiveness instrument to assess team effectiveness | NR | The actual number and depth of changes made to improve chronic illness care; perceived team effectiveness was consistently associated with both a greater number and depth of changes made to improve chronic illness care |
| Temkin-Greener et al, 2004 51 | Assessment of Interdisciplinary Team Process and Performance in Long-Term Care, focusing on Program of All-inclusive Care for the Elderly (PACE) | 59 items measuring 5 scales of team process measures and predictors of team performance, 5-point Likert scale | NR | .76-.89 | Literature review to select survey items. An expert of panel of 12 professionals and specialist in education and English as a second language prereviewed the survey items | Construct validity was demonstrated through the results of the regression analysis; as postulated in the theoretical model, leadership, communication, coordination and conflict management were positively related to team cohesion and perceived team effectiveness | NR |
| Hyer et al, 2003 52 | Trainee Test of Team Dynamics | 5 question test to capture knowledge of team process and skills in addressing conflict after watching a 5-minute videotape of a simulated interdisciplinary health care team meeting; researchers also create “gold standard” by asking 31 experts to rate the video | NR | NR | Geriatric Interdisciplinary Team Training (GITT) faculty agreed to a specific set of learning objective dimensions at a national meeting prior to the full launch of GITT. Survey items were created based on the learning objective domains | Principal component factor analysis was conducted; 2 concepts—ability to distinguish team dynamics and recognition of effective team behaviors are only significant. One-way ANOVA test also supported the result | NR |
Abbreviations: ANOVA, analysis of variance; CFA, confirmatory factor analysis; EV, eigenvalues; FL, factor loadings; ICC, intraclass correlation coefficient; IRA, interrater agreement; IRR, interrater reliability; NR, not reported; rwg = James, Demaree, and Wolf’s interrater agreement indices; Var Exp, variance explained.
a An article that report all 4 psychometric properties.The results indicate that survey instruments are widely used when measuring the dynamics of teamwork and individual team member behaviors; however, when team effectiveness is self-reported using Likert-type questionnaires, team members can report biased answers on their teamwork or unintentionally overestimate their team effectiveness. 54,55 To avoid any human error and bias in survey responses, it is important to measure non-self-reported outcomes, and compare these with the self-reported team effectiveness results from a survey. Only 7 articles (31%) reported non-self-reported outcomes related their survey measures, which reveal that when team members reported high team effectiveness, the objective outcomes (eg, task performance, quality of care, length of stay, and patient discharge) were also improved. Our findings indicate that the adequacy of survey instruments on team effectiveness linked to actual outcomes is still understudied.
Tables 3 to to5 5 show the conceptual dimensions of measuring team effectiveness in surveys across different clinical settings. First, we focused on the surgical setting to investigate what conceptual dimensions were most measured. Applying the CTEF model in addition to Donabedian model, we identified the specific subdimensions of structure, process, and outcomes with the consideration of surgical teams’ high-acuity, complex task, and time-sensitive conditions. We found that structure and outcome dimensions were most often measured in the surgical settings. Particularly, team skills, task specialization, and working conditions were the unique dimensions found in the surgical setting. In terms of outcomes, “task competency” and “would refer others to this team” were measured only in surgical settings. In the process category, “team coordination” and “value the teamwork” dimensions were most measured across the selected articles in the surgical setting. The prevalence of these dimensions confirms that the surgical teams require interprofessional coordination and that team members must value teamwork to achieve high-quality care. Following the literature on action teams, the findings indicate that survey tools in surgical settings more often focus on team skills, specialization, and coordination to assess team effectiveness. The findings also support that task performance is considered highly important and differentiated from team performance when it comes to the surgical setting.
The conceptual dimensions of team effectiveness in the surgical setting.
| Conceptual framework | La Sala et al, 2015 36 | Andrew et al, 2012 37 | Hamilton et al, 2009 38 | Michinov et al, 2008 23 | Simon and Stewart, 2007 39 | Davenport et al, 2007 3 |
|---|---|---|---|---|---|---|
| Structure | ||||||
| Recognizing leadership | x | x | ||||
| Team skills/specialization a | x | x | x | |||
| Commitment to patients | x | |||||
| Emotional exhaustion/stress | x | x | ||||
| Clear roles and responsibilities | x | x | ||||
| Working conditions (quality of the work) a | x | |||||
| Process | ||||||
| Communication among teammates | x | x | ||||
| Team coordination | x | x | x | x | x | |
| Value the teamwork | x | x | x | x | ||
| Outcome | ||||||
| Team cohesion (collective efficacy) | x | x | ||||
| Overall perceived team effectiveness | x | x | ||||
| Improved task competency a | x | x | x | |||
| (Postteamwork) job satisfaction | x | x | ||||
| Would refer others to this team a | x |
In the surgical settings, we used both Donabedian and CTEF framework to identify contextual nature of teamwork sills and performance.
a Concepts showed only surgical settings.The conceptual dimensions of team effectiveness in the other clinical setting.
| Conceptual framework | Keebler et al, 2014 44 | El Ansari et al, 2016 45 | Tremblay et al, 2015 46 | Strasser et al, 2014 | Dounis et al, 2013 48 | Baker et al, 2011 49 | Strasser et al, 2010 47 | Strating and Nieboer, 2009 50 | Temkin-Greener et al, 2004 51 | Shortell et al, 2004 4 | Hyer et al, 2003 52 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Type of setting | US Army Medical facilities | Mental Health | Ambulatory Cancer Clinic | Medical services at VA | Type 2 diabetes training program | General hospital | Rehabilitation services at VA hospitals | Home care | Long-term care | Chronic Illness Program | Geriatric health care teams |
| Structure | |||||||||||
| Recognizing leadership a | x | x | x | x | x | x | |||||
| Commitment to patients a | x | x | x | x | x | ||||||
| Embracing challenge b | x | x | |||||||||
| Team composition | x | x | |||||||||
| Clear roles and responsibilities a | x | x | x | x | |||||||
| Process | |||||||||||
| Shared understanding | x | x | x | x | |||||||
| Participating in decision making | x | ||||||||||
| Communication among teammates a | x | x | x | x | x | x | x | ||||
| Conflict resolution effort | x | x | x | x | x | ||||||
| Team collaboration | x | x | |||||||||
| Team coordination a | x | x | x | x | x | ||||||
| Sharing vision and goals | x | x | x | x | |||||||
| Support for innovation | x | x | x | x | x | x | |||||
| Value the teamwork | x | x | x | x | x | x | |||||
| Task orientation | x | x | x | ||||||||
| Monitoring situation b | x | ||||||||||
| Outcome | |||||||||||
| Team cohesion a | x | x | x | x | x | x | |||||
| Team efficiency | x | x | |||||||||
| Overall perceived team effectiveness a | x | x | x | x | x | ||||||
| Patient safety b | x | x | |||||||||
| Improved patient well-being b | x | x | x | x | x |
b Unique dimensions founded in the other clinical settings except for surgical and primary care settings.
Table 4 presents the conceptual dimensions measured by surveys in the primary care setting. Interestingly, primary care setting surveys are more focused on team coordination and value of the teamwork. In the process category, team collaboration, participation in decision making, and support for innovation were mostly found in surveys administered in primary care settings. This indicates that primary care settings recruit both internal and external professions to the host primary care organizations, so how they communicate with each other and how much they share goals/vision is key to enhanced team effectiveness. The right mix of team composition is also an important issue in the primary care setting. Compared with the surgical setting, the primary care setting seems more focused on team performance than task performance. The primary care setting surveys mostly measured outcome as overall perceived team effectiveness, collective team efficacy, and team members’ job satisfaction. Teamwork can be differently perceived and evaluated based on the type of clinical setting, as illustrated by the differential appearance and prevalence in conceptual dimensions surveyed across primary care and surgical settings.
The conceptual dimensions of team effectiveness in the primary care setting.
| Conceptual framework | Landry and Erwin, 2015 40 | Song et al, 2015 41 | Beaulieu et al, 2014 42 | Helfrich et al, 2014 19 | Becker and Roblin, 2008 43 |
|---|---|---|---|---|---|
| Structure | |||||
| Recognizing leadership | x | ||||
| Commitment to patients | x | x | x | ||
| Emotional exhaustion/stress | x | ||||
| Team composition | x | x | |||
| Clear roles and responsibilities | x | x | |||
| Process | |||||
| Shared understanding | x | x | |||
| Communication among teammates | x | x | x | ||
| Conflict resolution effort | x | x | |||
| Team collaboration | x | x | |||
| Participation in decision making | x | x | x | x | |
| Team coordination | x | x | |||
| Sharing vision and goals | x | x | |||
| Support for innovation | x | x | |||
| Value the teamwork | x | ||||
| Task orientation | x | ||||
| Outcome | |||||
| Team cohesion (collective efficacy) | x | x | x | ||
| Overall perceived team effectiveness | x | x | |||
| (Postteamwork) job satisfaction | x |
Table 5 shows the conceptual dimensions of other clinical settings. Due to the large variety and range of clinical settings, we listed the type of setting under each article. Overall, we found that there were common dimensions across all these settings. Recognizing leadership, commitment to patients, and clear roles/responsibilities were found in most articles regardless of setting. These dimensions are common structural conditions that health care teams share to achieve or improve team effectiveness. In the outcome category, team cohesion and overall perceived team effectiveness can be found regardless of health care setting. This means that team-based performance is usually measured as part of a survey instrument regardless of health care setting. Patient outcomes were only addressed in the survey tools in the other settings. Patient safety was addressed in 2 surveys, whereas improved patient well-being was measured in 5 surveys. In terms of process, communication among team members and team coordination were commonly found across settings; these dimensions were found in both surgical and primary care settings as well. The findings indicate that communication and coordination can be key to promoting team effectiveness in health care where different types of professionals and specialties are required to work together.
Effective teamwork in health care contributes to a positive organizational culture and improves patient safety and outcomes. Developing accurate methods for measuring team effectiveness will be crucial to help drive quality improvement. In addition, these methods may differ depending on the clinical setting in which they are deployed. We found that survey tools have been used to measure team effectiveness as an outcome or as a tool for developing models of team effectiveness. Most survey tools were implemented in primary care or surgical settings; thus, more work is required to develop valid survey tools in other clinical settings, such as ambulatory care, cancer care, rehabilitation service, and long-term care.
Regardless of the clinical setting, studies measuring team effectiveness using surveys should also include measures of the surveys’ psychometric properties. The inclusion of those properties adds credibility to the measurement instruments and helps future researchers study team effectiveness and develop new and improved measurement instruments. Of 22 articles, we found that only 4 articles (18%) reported all 4 psychometric properties and only 7 articles (31%) reported non-self-reported outcomes related their survey measures. This finding reveals that the adequacy of the survey instruments still needs to be assessed.
Regarding conceptual dimensions in survey instruments, we found that the focus on outcome measures in the survey instruments is different across settings. The surveys administrated in the primary care setting are more likely to focus on team performance than those administrated in surgical settings. In particular, most survey instruments in surgical settings distinguish task-specific components from team-related components when the surveys were administrated, which supports our use of the CTEF model as a conceptual framework. The distinctiveness of action teams in the surgical setting requires coordinating the different professions in a short-term period and time-pressured situation. In this dynamic process, team members need to monitor progress toward goals and provide real-time feedback so that any errors or misunderstanding are recognized and modified. 20 Additional research on surgical team effectiveness may be particularly useful because of the fast-paced nature of the operating room setting. Decisions in operating rooms are often made rapidly, with limited information, and hold serious consequences for the patient. Surgical teams comprise a variety of health care professionals including surgeon(s), operating room nurse(s), and the anesthesia care teams. Therefore, determining shared characteristics among high-functioning surgical teams would help providers and administrators improve efficiency, effectiveness, and quality across a variety of delivery models and settings. There are likely factors beyond surgical team composition that influence team effectiveness, and these could be captured through qualitative means such as survey assessments. Research to identify those factors, how they can be accurately measured, and how they affect team members and patients, needs to be explored.
Interestingly, the findings indicate that existing survey instruments are less likely to address patient outcome as a key subdimension of outcomes. Only 5 survey tools in other health care settings recognize patient safety and improved patient well-being as their subdimensions of outcomes, whereas none of the survey instruments in the surgical or primary care settings explicitly measure patient outcomes as their key conceptual dimension. This is a notable finding because teamwork and team effectiveness are highlighted in the context of value-based payment. 56,57 To tie team effectiveness to value that actually improves care for patients, more attention to patient outcomes is needed when developing survey tools of team effectiveness.
Our study selection criteria were limited to the identification and evaluation of survey instruments of team effectiveness related to health outcomes and should not be construed as covering general studies on team(s) or teamwork(s) without outcome domains. Also, it is possible that the choice of conceptual framework can oversimplify or conflate distinct features of different health care settings. Our study mitigates this using multiple models to inventory the instruments under study, supplementing the Donabedian model with CTEF to account for the surgical setting.
The Medicare Access and CHIP Reauthorization Act of 2015 (MACRA) and initiatives in the commercial insurance market have fueled the shift to value-based payments and have driven required changes in care delivery models, including team-based care. New alternative payment models require a renewed emphasis on care coordination and team effectiveness. We report on team effectiveness measurement tools in a variety of health care settings in this article. Our findings indicate that more valid, context-sensitive survey tools need to be developed for health care settings. Our findings also reveal that patient outcomes should be addressed more thoroughly as key dimensions of outcomes when measuring team effectiveness.
In addition, we found that surgical settings have distinctive conditions for measuring team effectiveness relative to other primary care or ambulatory care. As evidenced by programs such as Enhanced Recovery After Surgery (ERAS), the Perioperative Surgical Home (PSH), and Medicare’s Comprehensive Care for Joint Replacement Model (CJR), the operating room has become a critical setting for team-based care delivery 58 -60 ; thus, more validated survey instruments focused on surgical action teams are needed. Further development of specific team effectiveness evaluation tools in various settings, such as chronic illness care, home care, long-term care, and ambulatory care, can enhance continuous quality improvements and patient outcomes in the future. Further development of team effectiveness evaluation tools specific to the health care setting can help further enhance continuous quality improvements and clinical outcomes in the future.
The authors wish to acknowledge our Senior Research Assistant, Jacob Kolman, for technical assistance on manuscript formatting and clarity.
Funding:The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This project was funded by the American Society of Anesthesiologists.
Declaration of conflicting interests:The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Contributed byAuthor Contributions: BAK framed research design and wrote the conceptual frameworks. OC collected and analyzed the systematic review data regarding the psychometric properties and conceptual dimensions. NMH and TRM interpreted the results and contributed to writing the discussion part. All authors read and approved the final manuscript.
ORCID iDs: Ohbet Cheon https://orcid.org/0000-0003-2116-201X
Thomas R Miller https://orcid.org/0000-0002-5948-5589
1. Gamm L, Kash B, Bolin J. Organizational technologies for transforming care: measures and strategies for pursuit of IOM quality aims . J Ambul Care Manage . 2007; 30 :291–301. [PubMed] [Google Scholar]
2. Bower P, Campbell S, Bojke C, Sibbald B. Team structure, team climate and the quality of care in primary care: an observational study . Qual Saf Health Care . 2003; 12 :273–279. [PMC free article] [PubMed] [Google Scholar]
3. Davenport DL, Henderson WG, Mosca CL, Khuri SF, Mentzer RM., Jr. Risk-adjusted morbidity in teaching hospitals correlates with reported levels of communication and collaboration on surgical teams but not with scale measures of teamwork climate, safety climate, or working conditions . J Am Coll Surg . 2007; 205 :778–784. [PubMed] [Google Scholar]
4. Shortell S, Jill M, Lin M, et al. The role of perceived team effectiveness in improving chronic illness care . Med Care . 2004; 42 :1040–1048. [PubMed] [Google Scholar]
5. Mazzocco K, Petitti DB, Fong KT, et al. Surgical team behaviors and patient outcomes . Am J Surg . 2009; 197 :678–685. [PubMed] [Google Scholar]
6. Miller A, Scheinkestel C, Joseph M. Coordination and continuity of intensive care unit patient care . Hum Factors . 2009; 51 :354–367. [PubMed] [Google Scholar]
7. Tapp H, Hebert L, Dulin M. Comparative effectiveness of asthma interventions within a practice based research network . BMC Health Serv Res . 2011; 11 :188–188. [PMC free article] [PubMed] [Google Scholar]
8. Wagner EH. The role of patient care teams in chronic disease management . BMJ . 2000; 320 :569–572. [PMC free article] [PubMed] [Google Scholar]
9. Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature . Acta Anaesthesiol Scand . 2009; 53 :143–151. [PubMed] [Google Scholar]
10. Burtscher MJ, Manser T. Team mental models and their potential to improve teamwork and safety: a review and implications for future research in healthcare . Safe Sci . 2012; 50 :1344–1354. [Google Scholar]
11. Valentine MA, Nembhard IM, Edmondson AC. Measuring teamwork in health care settings: a review of survey instruments . Med Care . 2015; 53 :e16–e30. [PubMed] [Google Scholar]
12. Donabedian A. The quality of care: how can it be assessed? JAMA . 1988; 260 :1743–1748. [PubMed] [Google Scholar]
13. Salas E, Goodwin GF, Burke CS. Team Effectiveness in Complex Organizations: Cross-disciplinary Perspectives and Approaches . London, England: Routledge; 2008. [Google Scholar]
14. Brannick MT, Prince C. An overview of team performance measurement . In: Brannick MT, Salas E, Prince C, eds. Team Performance Assessment and Measurement: Theory, Methods, and Applications . Mahwah, NJ: Lawrence Erlbaum Associates, Inc.; 1997 :3–18. [Google Scholar]
15. Taylor EF, Machta RM, Meyers DS, Genevro J, Peikes DN. Enhancing the primary care team to provide redesigned care: the roles of practice facilitators and care managers . Ann Fam Med . 2013; 11 :80–83. [PMC free article] [PubMed] [Google Scholar]
16. Essens P, Vogelaar AL, Mylle JJ, et al. Team Effectiveness in Complex Settings: A Framework . London, England: Routledge; 2009. [Google Scholar]
17. Harth A, Germann G, Jester A. Evaluating the effectiveness of a patient-oriented hand rehabilitation programme . J Hand Surg Eur Vol . 2008; 33 :771–778. [PubMed] [Google Scholar]
18. Hays R. Measuring quality in the new era of team-based primary care . Qual Prim Care . 2007; 15 :133. [Google Scholar]
19. Helfrich CD, Dolan ED, Fihn SD, et al. Association of medical home team-based care functions and perceived improvements in patient-centered care at VHA primary care clinics . Healthcare (Amst) . 2014; 2 :238– 244. [PubMed] [Google Scholar]
20. Fernandez R, Kozlowski SWJ, Shapiro MJ, Salas E. Toward a definition of teamwork in emergency medicine . Acad Emerg Med . 2008; 15 :1104–1112. [PubMed] [Google Scholar]
21. Hackman J. The design of work teams . In: Lorsch JW, ed. Handbook of Organizational Behavior . 1st ed. Englewood Cliffs, NJ: Prentice Hall; 1987 :315–342. [Google Scholar]
22. Sundstrom E, De Meuse KP, Futrell D. Work teams: applications and effectiveness . Am Psychol . 1990; 45 :120. [Google Scholar]
23. Michinov E, Olivier-Chiron E, Rusch E, Chiron B. Influence of transactive memory on perceived performance, job satisfaction and identification in anaesthesia teams . Br J Anaesth . 2008; 100 :327–332. [PubMed] [Google Scholar]
24. Klampfer B, Flin R, Helmreich RL, et al. Enhancing Performance in High Risk Environments: Recommendations for the Use of Behavioural Markers . Ladenburg: Daimler und Benz Stiftung; 2001. [Google Scholar]
25. Mathieu J, Maynard MT, Rapp T, Gilson L. Team effectiveness 1997-2007: a review of recent advancements and a glimpse into the future . J Manage . 2008; 34 :410–476. [Google Scholar]
26. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement . PLoS Med . 2009; 6 :e1000097. [PMC free article] [PubMed] [Google Scholar]
27. Devilly GJ, Borkovec TD. Psychometric properties of the credibility/expectancy questionnaire . J Behav Ther Exp Psychiatry . 2000; 31 :73–86. [PubMed] [Google Scholar]
28. Sexton JB, Helmreich RL, Neilands TB, et al. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research . BMC Health Serv Res . 2006; 6 :44. [PMC free article] [PubMed] [Google Scholar]
29. LeBreton JM, Senter JL. Answers to 20 questions about interrater reliability and interrater agreement . Organ Res Meth . 2008; 11 :815–852. [Google Scholar]
30. James LR, Demaree RG, Wolf G. Estimating within-group interrater reliability with and without response bias . J Appl Psychol . 1984; 69 :85–98. [Google Scholar]
31. James LR, Demaree RG, Wolf G. rwg: an assessment of within-group interrater agreement . J Appl Psychol . 1993; 78 :306–309. [Google Scholar]
32. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability . Psychol Bull . 1979; 86 :420–428. [PubMed] [Google Scholar]
33. Lance CE, Butts MM, Michels LC. The sources of four commonly reported cutoff criteria: what did they really say? Organ Res Meth . 2006; 9 :202–220. [Google Scholar]
34. Osborne JW, Costello AB. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis . Pan-Pac Manage Rev . 2009; 12 :131–146. [Google Scholar]
35. Rattray J, Jones MC. Essential elements of questionnaire design and development . J Clin Nurs . 2007; 16 :234–243. [PubMed] [Google Scholar]
36. La Sala R, Boninsegni K, Tani A, et al. A cross selectional survey in a critical care: the job satisfaction and functioning team of the health professionals . Acta Biomed . 2015; 86 :183–188. [PubMed] [Google Scholar]
37. Andrew B, Plachta S, Salud L, Pugh CM. Development and evaluation of a decision-based simulation for assessment of team skills . Surgery . 2012; 152 :152–157. [PubMed] [Google Scholar]
38. Hamilton N, Freeman BD, Woodhouse J, Ridley C, Murray D, Klingensmith ME. Team behavior during trauma resuscitation: a simulation-based performance assessment . J Grad Med Educ . 2009; 1 :253–259. [PMC free article] [PubMed] [Google Scholar]
39. Simon AB, Stewart N. Employees’ experience of, and attitudes towards teamworking at a National Health Service (NHS) District General Hospital . Risk Manage . 2007; 9 :145–166. [Google Scholar]
40. Landry A, Erwin C. Perspectives on multidisciplinary team processes among healthcare executives: processes that facilitate team effectiveness . J Health Hum Serv Adm . 2015; 38 :350–380. [PubMed] [Google Scholar]
41. Song H, Chien AT, Fisher J, et al. Development and validation of the primary care team dynamics survey . Health Serv Res . 2015; 50 :897–921. [PMC free article] [PubMed] [Google Scholar]
42. Beaulieu M-D, Dragieva N, Del Grande C, et al. The team climate Inventory as a measure of primary care teams’ processes: validation of the French version . Healthc Policy . 2014; 9 :40–54. [PMC free article] [PubMed] [Google Scholar]
43. Becker ER, Roblin DW. Translating primary care practice climate into patient activation: the role of patient trust in physician . Med Care . 2008; 46 :795–805. [PubMed] [Google Scholar]
44. Keebler JR, Dietz AS, Lazzara EH, et al. Validation of a teamwork perceptions measure to increase patient safety . BMJ Qual Saf . 2014; 23 :718–726. [PubMed] [Google Scholar]
45. El Ansari W, Lyubovnikova J, Middleton H, Dawson JF, Naylor PB, West MA. Development and psychometric evaluation of a new team effectiveness scale for all types of community adult mental health teams: a mixed-methods approach . Health Soc Care Commun . 2016; 24 :309–320. [PubMed] [Google Scholar]
46. Tremblay D, Roberge D, Berbiche D. Determinants of patient-reported experience of cancer services responsiveness . BMC Health Serv Res . 2015; 15 :425. [PMC free article] [PubMed] [Google Scholar]
47. Strasser DC, Burridge AB, Falconer JA, Uomoto JM, Herrin J. Toward spanning the quality chasm: an examination of team functioning measures . Arch Phys Med Rehabil . 2014; 95 ( 11 ):2220–2023. [PubMed] [Google Scholar]
48. Dounis G, Ditmyer M, Vanbeuge S, et al. Interprofessional faculty development: integration of oral health into the geriatric diabetes curriculum, from theory to practice . J Multidiscip Healthc . 2013; 7 :1–9. [PMC free article] [PubMed] [Google Scholar]
49. Baker SD, Mathis CJ, Stites-Doe S. An exploratory study investigating leader and follower characteristics at U.S. healthcare organizations . J Manag Issue . 2011; 23 :341–363. [Google Scholar]
50. Strating M, Nieboer A. Psychometric test of the team climate inventory-short version investigated in Dutch quality improvement teams . BMC Health Serv Res . 2009; 9 :126. [PMC free article] [PubMed] [Google Scholar]
51. Temkin-Greener H, Diane G, Kunitz SJ, Dana M. Measuring interdisciplinary team performance in a long-term care setting . Med Care . 2004; 42 :472–481. [PubMed] [Google Scholar]
52. Hyer K, Skinner JH, Kane RL, et al. Using scripted video to assess interdis-ciplinary team effectiveness training outcomes . Gerontol Geriatr Educ . 2004; 24 :75–91. [PubMed] [Google Scholar]
53. Hamilton CB, Smith CA, Butters JM. Interdisciplinary student health teams: combining medical education and service in a rural community-based experience . J Rural Health . 1997; 13 :320–328. [PubMed] [Google Scholar]
54. Kahneman D, Tversky A. Choices, values, and frames . Am Psychol . 1984; 39 :341–350. [Google Scholar]
55. Meier KJ, O’Toole LJ. Subjective organizational performance and measurement error: common source bias and spurious relationships . J Public Admin Res Theory . 2012; 23 :429–456. [Google Scholar]
56. Porter ME. A strategy for health care reform—toward a value-based system . New Engl J Med . 2009; 361 :109–112. [PubMed] [Google Scholar]
57. Burwell SM. Setting value-based payment goals—HHS efforts to improve US health care . N Engl J Med . 2015; 372 :897–899. [PubMed] [Google Scholar]
58. Cline KM, Roopani R, Kash BA, Vetter TR. Residency board certification requirements and preoperative surgical home activities in the United States: comparing anesthesiology, family medicine, internal medicine, and surgery . Anesth Analg . 2015; 120 :1420–1425. [PubMed] [Google Scholar]
59. Kash BA, Zhang Y, Cline KM, Menser T, Miller TR. The perioperative surgical home (PSH): a comprehensive review of US and non-US studies shows predominantly positive quality and cost outcomes . Milbank Q . 2014; 92 :796–821. [PMC free article] [PubMed] [Google Scholar]
60. Schuller KA, Kash BA, Gamm LD. Enhanced transitions of care: centralizing discharge phone calls improves ability to reach patients and reduces hospital readmissions . J Healthc Qual . 2017; 39 :e10–e21. [PubMed] [Google Scholar]
