Certification Program Validity Study
April 2006 through March 2007
CGFNS Qualifying Exam® and IELTS as predictors of success on the registered nurse licensing examination in the United States
Late in the 1960s, the United States experienced an increase in foreign nurse graduates migrating to the United States. Only 15–20% of these nurses passed the U.S. registered nurse licensing exam on their first attempt. As an outgrowth of the concern over these statistics, the Commission on Graduates of Foreign Nursing Schools (CGFNS International) was created in 1977. A goal of CGFNS was to develop and administer a predictive test and evaluation program for nurses educated outside the United States. CGFNS continues to accomplish this goal through the CGFNS Certification Program.
CGFNS’s Certification Program is designed only for first-level, general nurses educated and licensed outside the United States who wish to assess their chances of passing the U.S. registered nurse licensing exam, the NCLEX-RN®, and attain licensure as registered nurses within the United States. The program is comprised of three components: (1) a credentials review; (2) a one-day test of nursing knowledge (CGFNS Qualifying Exam®) that consists of two sections and is modeled on the NCLEX-RN® test plan: and (3) an English proficiency test: TOEFL® (Test of English as a Foreign Language), IELTS (the International English Language Testing System) examination, or the TOEIC® (Test of English for International Communication). The Certification Program identifies nurses with a high potential for achieving success on the NCLEX-RN®. examination.
Many individual and groups benefit from the Certification Program: U.S. boards of registered nursing; the U.S. Department of Labor; and the U.S. Citizenship and Immigration Service (USCIS), to name a few. For example, the CGFNS (Certification Program) certificate, which is earned by meeting the criteria for the credentials review, passing the CGFNS Qualifying Exam®, and achieving the required score on an English language proficiency examination is a prerequisite for obtaining a permanent occupational visa for nurses. The majority of U.S. boards of nursing recognize the CGFNS (Certification Program) certificate as a prerequisite for taking the NCLEX-RN®.
In order to monitor and maintain the CGFNS Qualifying Exam®, a statistical study is regularly performed. This study serves several purposes; however, only one will be the focus of this paper:
Assessment of the validity of the CGFNS Qualifying Exam® as a predictor of the NCLEX-RN® result
As in the previous validity studies (1995–96 to 2000–01), data for 2001–2002 were obtained directly from the National Council of State Boards of Nursing’s (NCSBN) database. Included in this database are not only the results for CGFNS (Certification Program) certificate holders, but also for those foreign nurse graduates who failed the CGFNS Qualifying Exam® or never took the CGFNS Qualifying Exam®, yet went on to take the NCLEX-RN®.
The analysis for this study was performed in five phases: (1) Comparison of the NCLEX-RN® performance by certificate holders to the performance of those who failed CGFNS’s exams; (2) Correlational analysis of the two elements of the CGFNS Qualifying Exam; (3) analysis of other variables that may influence predictability of the NCLEX-RN® results; (4) analysis of the performance of repeat test-takers; and (5) Comparison of CGFNS (Certification Program) certificate Holders to the performance of all other groups of foreign nurse graduates, regardless of whether or not they had ever taken the CGFNS Qualifying Exam®.
To ensure the accuracy and comprehensiveness of the statistics, CGFNS collected data on NCLEX-RN® results of foreign nurse graduates directly from NCSBN. Although NCSBN approved the project, results were considered the property of individual boards. Previously, CGFNS International contacted each U.S. state board of nursing requesting permission to collect the appropriate data. 53 out of the 58 states and territories of the United States that regulate registered nurses practice cooperated with the request.
Due to the lag time between a foreign nurse graduate earning a CGFNS (Certification Program) certificate, being issued a visa and applying to a specific state board of nursing, 2002 through 2007 was determined to be the appropriate time period from which data should be taken for the CGFNS Qualifying Exam®. Data from NCLEX-RN®s administered April 2006 through March 2007 were included in the analysis.
All persons who took the CGFNS Qualifying Exam® between January 1, 2002 through March 31, 2007 were matched against the 39,726 foreign nurse graduates who applied for licensure within 51 states and territories and took the NCLEX-RN® between April 1, 2005, and March 31, 2006. The resulting match of 9,949 different persons was used as the sample for this study. Of this group, 8,339 took the NCLEX-RN® for the first time.
Phase I. The first analysis was restricted to members of the six CGFNS cohorts who took the NCLEX-RN® for the first time: 2002, 2003, 2004, 2005, 2006 and 2007 (n=8,339). The overall pass rate on the NCLEX-RN® for those who earned a CGFNS (Certification Program) certificate (n=7,374) was 90.8%, which is higher than the pass rates observed in the validity studies conducted in 2004, 2005 and 2006. Conversely, of those who failed to earn a CGFNS (Certification Program) certificate and then went on to take the NCLEX-RN® (n= 568), only 40.7% passed the NCLEX-RN® on their first attempt.
Subsequently, the failure rate on NCLEX-RN® of those who did not earn a CGFNS (Certification Program) certificate (58.9%) was 6.4 times greater than that of certificate holders (9.2%). This finding provides a strong basis for the CGFNS Qualifying Exam® as a predictor of NCLEX-RN® performance.
Phase II. The scores from the CGFNS Qualifying Exam® and from IELTS were examined independently for their relationship to NCLEX-RN® performance by each cohort. Pearson Product Moment Correlations were used to determine the normalized biserial coefficient (rb). For the nursing exam, the rb’s ranged between 0.62 and 0.80. The rb for the entire group (n=8,839) was 0.69.
The normalized biserial correlation coefficients (rb) between scores on the TOEFL® and NCLEX-RN® performance ranged from 0.08 to 0.20 across cohorts. For the total group (n=6,399), the rb was 0.10. Similar to the results of previous validity studies, the IELTS is not as strong a predictor of success on the NCLEX-RN®. Data on other English language proficiency tests were not analyzed due to insufficient numbers.
Phase III. Demographic variables such as age and lag time between the CGFNS Qualifying Exam® and the NCLEX-RN® were examined for their relationship to NCLEX-RN® performance. Taken independently, a short time span between graduation and (1) the first administration of the CGFNS nursing exam and (2) the first administration of the NCLEX-RN®, younger age at the time of the NCLEX-RN®, graduation from a non-degree granting education program and education in English and use of English texts were strongly associated with passing the NCLEX-RN®.
Phase IV. Some foreign nurse graduates took the CGFNS Qualifying Exam® multiple times. The performance of these repeat test takers were examined with regard to their success on the NCLEX-RN®. For all six cohorts, the pass rates on NCLEX-RN® were highest for those who earned their CGFNS (Certification Program) certificate after taking the CGFNS Qualifying Exam® for the first time. Thus the repeat factor could aid in predicting NCLEX-RN® performance.
Phase V. Finally, the performance of all foreign nurse graduates taking the NCLEX-RN ®examination for the first time (n=29,214), regardless of whether or not they took the CGFNS Qualifying Exam®, was compared to the performance of CGFNS (Certification Program) certificate holders. The passing rate for the entire group was 60.0%. Of the entire group, 8,339 were known to be CGFNS (Certification Program) certificate holders, representing 28.5% of the 29,214 foreign nurse graduates taking NCLEX-RN for the first time. In comparison to the entire group, the CGFNS (Certification Program) certificate holders had a passing rate of 89.6% on the NCLEX-RN®. Furthermore, of those earning a CGFNS (Certification Program) certificate on the first attempt (n=5,945), 92.4% also passed the NCLEX-RN® on the first attempt.
The April 2006 through March 2007 Validity Study revealed that the CGFNS Qualifying Exam® has been and continues to be a strong predictor of performance on the NCLEX-RN®. The findings of the current study include an overall pass rate on the NCLEX-RN® for CGFNS (Certification Program) certificate holders of 90.8% versus 40.7% for non-certificate holders. The failure rate on NCLEX-RN® of those who did not earn a CGFNS (Certification Program) certificate (58.9%) was 6.4 times greater than that of certificate holders (9.2%). In addition, the correlational analyses indicated that the CGFNS Qualifying Exam® was a strong predictor of NCLEX-RN® performance and IELTS a weak predictor. These results provide CGFNS clients, as well as the general public, the assurance that the CGFNS Certification Program is a valid predictor of NCLEX-RN® success.
This report refers to The CGFNS Validity Study: CGFNS Qualifying Exam® and IELTS as predictors of success on the U.S. Registered Nurse Licensing Examination — April 2005 through March 2006. The study was conducted by Carolyn J. Yocom, PhD, RN, FAAN, Associate Professor and Chair, Department of Organizational Systems and Adult Health, University of Maryland School of Nursing; and Catherine R. Davis, PhD, RN, Director of Global Research and Test Administration, CGFNS International. Special thanks are extended to the National Council of State Boards of Nursing for their assistance in compiling the NCLEX-RN® data for this study.
1 Survey of Foreign Nurse Graduates. (1976). U.S. Department of Health, Education and Welfare, DHEW Publication No. (HRA) 76-13.