Many studies have examined a range of ways to identify the attributes desired in candidates for admission to health professions education programs, including several reports on the physician assistant (PA) and physical therapist (PT) professions [1-7]. However, to date there is no clear consensus on the best admissions processes in terms of measurable and consistent reliability and validity, and the continued use of multiple interview formats and consideration of many other non-cognitive variables such as personal statements and reference letters contributes to the inherent subjectivity of admission decisions. We chose to compare and contrast the PA and PT admissions processes because there are a similar number of accredited programs in the US (PA, 181; PT, 218), many PA and PT programs co-exist in the same school of health professions (sharing many of the same basic science courses), and both professions conduct centralized application processes administered by the same organization [8,9].

According to the Commission on Accreditation for Physical Therapy Programs (CAPTE), in 2011, the “average” physical therapy program received 357 applications for admission via the Physical Therapy Centralized Application Service (PTCAS) portal. From this large pool of applicants, a mean of 86 were offered admission and 42 students were ultimately enrolled [10]. In 2010, a total of 15,186 applications were submitted via the Central Application Service for Physician Assistants (CASPA) in competition for 5,816 seats (Rachel Hamann, e-mail communication, January 16, 2013). Given the large number of applicants and very limited number of available seats, admissions committees are challenged to identify applicants who are most likely to excel in the program, pass the certification exam on the first attempt, and succeed in clinical practice. Determining which students are “most likely to succeed” is complicated by a lack of consensus on the reported variables that predict both academic and clinical performance. For this reason, programs often expand their admissions criteria beyond the traditional cognitive or academic markers of achievement to include assessment of an applicant’s non-cognitive abilities such as interpersonal communication skills.

Despite the efforts to select highly qualified applicants, 2012 data published by CAPTE indicate that 4% of students did not graduate on time and 12% of students did not pass the National Physical Therapy Examination (NPTE) on the first attempt [10]. In the same year, 8% of PA students were withdrawn or decelerated and 8% did not pass the Physician Assistant National Certifying Examination (PANCE) on the first attempt [11]. These data reveal that despite an abundant pool of well-qualified applicants, not all students who meet the admissions criteria and achieve matriculation eventually succeed in PA or PT educational programs.

Although many studies are critical of the fallibility and inadequate scientific rigor of the high-stakes nature of health professions admission decisions, the typical admission processes remain similar. The usual documentation and information required for admission to a PA or PT program include official transcripts from all accredited colleges or universities attended as well as official test scores from the Graduate Record Examination (GRE) and from the Test of English as a Foreign Language (TOEFL) for students whose first language is not English. The standard nonacademic requirements typically include up to four letters of reference, at least one personal statement, and documentation attesting to the number of hours the applicant shadowed a PA/PT in clinical practice and/or performed hands-on health care (Fig. 1). In addition to these nonacademic requirements, the majority of programs also conduct personal interviews with the most competitive applicants, even though research on the value of interviews is equivocal [12]. This variation between and among differing health professions and their training programs has been criticized in the literature, being likened to a “professional crap shoot” and “a very elaborate, labor-intensive and expensive lottery… likely little better than a horoscope” [13]. An important but infrequently examined element of health professions admissions is the academic outcome of qualified but initially rejected applicants who reapply, as depicted in Fig. 1.

Cognitive variables, most notably undergraduate grade point averages (GPAs), have been shown to be the best predictors of academic achievement in the health professions [12]. Data in the PA and PT literature have largely supported this tendency, with the exception of one recent study of three PA student cohorts at one institution that found no correlation between PANCE performance and undergraduate GPA, science prerequisite GPA, or prior health care experience [14]. In a study of cognitive and non-cognitive variables as predictors of PANCE scores at six US PA programs, the four significant predictors were GPA, GRE (Verbal and Quantitative components), although not all PA programs require the GRE, and scores achieved on the Physician Assistant Clinical Knowledge Rating and Assessment Tool — all cognitive variables, while the variables of age, gender, previous health care experience, and interview scores were not significant [15].

Most of the literature on PT admissions was written before the majority of programs converted to DPT status; however, one recent study reported that undergraduate academic achievement (pre-cumulative GPA) was the best predictor of success in the first year of a DPT program [16]. However, the authors point out that they did not assess the relationship between admissions requirements and successful completion of the DPT program. Demographic variables such as age, gender, and ethnicity have not been routinely examined as PT admissions variables but have been examined retrospectively to identify students at risk for academic difficulty and to explain attrition rates. Utzman et al. [17] reported undergraduate GPA as a predictor of academic difficulty by developing an at-risk student prediction rule based on a large sample of students from 20 programs that included undergraduate GPA, GRE scores, age, and ethnicity or race. Interestingly, they reported a GPA of 3.51 or higher buffered younger students from academic difficulty but not older students, indicating that age is a demographic variable that may predict academic difficulty. Researchers favoring stronger weight for cognitive data rely on findings which indicate that the only consistently reliable predictor of knowledge-based and skill-based licensing examination performance is the undergraduate GPA, while others favor using cognitive outcomes such as Medical College Admission Test scores and GPAs only as threshold measures that should be complimented by compelling personal characteristics [9,18].

Although past academic performance appears to be the best predictor of student success, many health professions programs include the assessment of variables such as emotional intelligence (EI), critical thinking, and personality traits [19]. EI was also found to predict academic and clinical success in medical and dental students [17,20]. However, in a three-year study of DPT students from four accredited programs, Lewis reported that EI did not predict academic or clinical performance or first-time pass rates on the NPTE [21]. Galleher looked at two non-cognitive variables as predictors of passing the NPTE on the first try, in addition to a range of cognitive variables (SAT, GRE, GPA). The authors selected conscientiousness (personality trait) and task-coping skills (coping variable) as the non-cognitive predictors of first-time pass rate. However, similar to the other non-cognitive variables, there was no relationship between conscientiousness, coping skills, and first-time pass rate [22]. Another study assessed the relationship between PT students’ critical thinking skills as measured by the California Critical Thinking Skills Test (CCTST) and first-time pass rate on the NPTE, where a significant relationship was found. The author also reported a significant relationship between GPA and pass rate, while no relationship was found between clinical performance and pass rate [23].

The variability of non-cognitive attributes assessed and the methods used to measure them have come under increasing scrutiny in the literature. Despite the ongoing controversy over which qualities should be assessed and with what value weight, the importance placed on such attributes by medical and lay personnel “warrants that further effort be put toward developing a valid measurement tool for the assessment of non-cognitive qualities [24].” The non-cognitive qualities desired in the health professions typically fall within the categories of attitudinal, behavioral, and interpersonal elements. Research to identify the most important non-cognitive personal qualities for the health professions is plentiful, and the number of differing attributes assessed is large. Albanese et al. [25] reportedly found 87 distinct personal qualities in the literature that have relevance to medical practice. This makes the admissions process a daunting task of identifying, prioritizing, and measuring specific attributes sought in a given applicant pool. Furthermore, research indicates that different types of medical performance criteria are associated with different personality characteristics, making it difficult if not impossible to achieve reliability and validity when applying a “one size fits all” set of non-cognitive attributes to health professions admission decisions [26]. For example, a number of studies have questioned the value of assessing personal statements in health professions program applications, and some have been critical of their purpose — claiming that fraud in their content “is not a possibility; it is a well-documented fact” [13].

The non-cognitive admissions tool exhibiting the most variation and impact on outcomes is the personal interview. Although almost universally used in the health professions, personal interviews are time- and resource-intensive endeavors. A meta-analysis of interview outcomes from a wide range of health professions revealed that 19 of the 20 studies measured the relationship between interview performance and academic performance. The authors concluded that interviews were a very weak predictor of academic performance [27]. One of the more recent data-driven tools designed to increase the reliability and validity of attributes measured in the candidate interview process is the multiple mini-interview developed by Eva et al. [28]. There is a growing body of literature on the advantages of this process, but it has not reached widespread use in the US [8]. One important aspect of the multiple mini-interview format is that it is validated for predicted clinical performance rather than academic performance [30].

The PT literature suggests the variables that predict a student’s success in the classroom may not be the same ones that predict success in the clinical setting. PT clinical instructors reported one academic and two nonacademic categories of behaviors as “red flags” for poor performance in a clinical setting [29]. The academic category was inadequate knowledge/psychomotor skills, and the two nonacademic categories included unprofessional behavior and poor communication skills. This suggests that student success in the classroom and in the clinic is based on a different set of variables.

To assess what non-cognitive variables are commonly sought and publicized to applicants, we reviewed the websites of the national PA and PT organizations, the centralized application services, and the individual programs. CASPA and the PTCAS are administered by the same company, Liaison International, but are governed by the Physician Assistant Education Association (PAEA) and American Physical Therapy Association (APTA), respectively. A survey of 94 PA programs rated non-cognitive qualities based on their importance in the PA admissions process. Motivation for becoming a PA (4.16/5), maturity (4.0/5), and professional demeanor (4.0/5) were all ranked highly to most influential in determining candidacy for admissions. Other non-cognitive qualities also ranked somewhat to highly influential were compassion and self-awareness/insight [30].

Our review found that many PT programs do not routinely list desired non-cognitive variables on their individual program websites. However, the APTA’s seven core values, which include several elements of professionalism (Accountability, Altruism, Compassion/Caring, Excellence, Integrity, Professional Duty, and Social Responsibility) are listed prominently on the PTCAS website [31]. In contrast, the PA application process appears much more individualized; CASPA and PAEA direct applicants to review the information available in the PA program directory [11] or the individual program websites. Our review of these two resources revealed a total of 40 desired non-cognitive personal attributes and characteristics delineated by 126 of the 170 PA programs (74%) accredited as of Spring 2013 [8]. The range of individual attributes included among the 126 programs providing information was from 1 to 23 attributes per program. A tag cloud of the eight most frequently mentioned variables is depicted in Fig. 2, and the most and least frequently assessed attributes are illustrated in Figs. 3 and 4. An interesting finding is that qualities such as self-awareness and compassion, which were ranked highly in one study [30], were each mentioned fewer than 10 times on websites, suggesting a discrepancy between what is important to the programs and what is advertised to potential applicants.

With applications to PA and PT programs at an all-time high and the two occupations being listed in the top 20 desirable job rankings from money.cnn.com, both professions are experiencing an abundant pool of well-qualified applicants [32,33]. While cognitive variables such as undergraduate GPA have been shown to be predictors of academic achievement, variance in health professions students’ performances in the classroom and on certifying exams remain unexplained, and performance predictions in the clinical setting are even more tenuous. Non-cognitive considerations vary a great deal between and among programs that describe them. What is uncertain from this review is whether or not the desired candidate attributes of individual programs are more student-centered or graduate-centered. McDaniel’s research indicates that programs value academic and clinical achievement almost equally (academic success: mean 4.32, 95% confidence interval [CI] 4.17 to 4.47; success in health care field: mean 4.15, 95% CI 4 to 4.3) [30], suggesting these admissions variables reflect the goals of educational programs (academic success, low attrition, high certifying exam performance) as well as employers (good interpersonal and communication skills, compassion, and goals for self-improvement). Perhaps an additional and equally important goal would include competency prediction variables and clinical outcomes that represent the best interests of the public that will be in the care of these clinicians. Both the PA and PT professions have developed professional competencies or core values that drive the education and evaluation of students and professionals, and these values can be applied to the recruiting and selection of applicants. The PT profession currently posts its core values on its centralized application process; perhaps the PA profession should follow suit. Certainly, the intent of professional school is developing some of these competencies while enhancing others, but applicants who demonstrate good interpersonal and communication skills, compassion, and goals for self-improvement would seem to be destined to have an easier time achieving specific competencies.

It is also possible that one reason non-cognitive variables have not been found to be very predictive is that they are being used to evaluate the wrong things. Our review has found that most literature on the predictive nature of admissions is focused on performance in the classroom and on certifying examinations. Given that both the PANCE and NPTE are standardized knowledge exams, it seems reasonable that GPA and standardized tests would be most predictive of outcomes, while non-cognitive traits like coping skills and conscientiousness would have little predictive value for examination success. Of course, researchers use data on academic outcomes and national certification exams because these data are easy to collect and assess. There is a dearth of information on the actual clinical performance of practicing PAs and PTs, likely due to the perceived difficulty of gathering such data. In fact, there are no studies in the PT literature and only one study on PAs looking at admissions and curriculum as predictors for graduate clinical performance. The study was small (n=19) but reported an 81% graduate response rate and a 79% physician response rate, suggesting that obtaining important feedback is not impossible [3].

Clinical practice in the health professions has historically relied on empiric decisions based on observations and experience, but this tradition has evolved into an evidence-based approach. Despite the widespread acceptance of identifying and applying the best evidence in clinical practice, the entry-point admissions processes for health professions education continue to utilize many non-cognitive variables and methods that have low or inconsistent reliability and validity, leading to editorials or opinion papers voicing strong opposition to the status quo in health professions admissions [34]. Further research should be done to evaluate how PA and PT professional values are assessed during the admissions process and how admissions variables predict graduate clinical performance.

Given the range of positions and general lack of reliability and validity in studies of non-cognitive admissions attributes, we think that health profession admission processes remain imperfect works in progress. The time, effort, and expense involved in these processes justify ongoing research to determine the most efficient, accurate, and cost-effective criteria to use in identifying the best candidate attributes. Even more importantly, the well-being of the public demands an admissions approach that is thoughtful and meaningful to ensure candidates who will be competent and safe practitioners are admitted.