Abstract
The No Child Left Behind legislation was signed into law to ensure children in the United States receive quality education and learn the skills needed to be successful. Career and technical education (CTE) is not mentioned in the legislation, which suggests that more CTE courses may be dropped from high school master schedules, which makes the topic a concern for educational leaders. The purpose of this study was to investigate the academic performance of CTE completers and non-CTE completers in the Commonwealth of Virginia on the standards of learning English reading and mathematics assessments as well as cohort graduation rates. Findings indicate that statistically (
Keywords
Today our nation’s schools are faced with the growing and evolving pressures of increasing the academic achievement of students. With the increasing pressures to improve educational programs, states have implemented standards and performance assessments (high-stakes tests) to measure student achievement, including increased graduation requirements. High-stakes testing systems generate scores with important consequences to schools and school personnel, which are also applied to students in the form of graduation requirements or remedial courses (Austin & Mahlman, 2002).
The No Child Left Behind (NCLB) legislation changed the face of the standards movement and the measure of student achievement. The purpose of the NCLB legislation was to ensure that all students have an opportunity to obtain a high-quality education and increase students’ academic achievement (Gordon, Yocke, Maldonado, & Saddler, 2007). There are growing concerns among career and technical education (CTE) leaders based on the fact that no program area of CTE (agriculture, business and computers, marketing, family and consumer sciences, health occupations, or technology-trade and industry education) is specifically mentioned in the NCLB legislation, and there could be a significant impact on CTE programs in the future (Martin, Fritzsche, & Ball, 2006).
CTE courses are an integral part of most comprehensive high schools in Virginia; thus, if these courses are reduced, many high school students will not have the opportunity for a comprehensive high school experience or gain exposure to specialized training and work skills. Elective courses provide students with educational alternatives to reinforce reading, mathematics, and science skills that are highly valued in the state assessments. Students succeed in schools due to a number of factors, and a sound CTE program is one of them (Daggett, 2005).
Statement of the Problem
As CTE establishes and maintains itself as a viable component of the comprehensive school program, there is a need to determine how students who have completed a CTE concentration of sequenced courses perform on the state-mandated high-stakes Virginia standards of learning (SOL) tests. CTE courses will continue to lose enrollment and be removed from high school course offerings unless educational leaders can clearly demonstrate how these programs (a) contribute to the academic success of students as measured by state academic tests and (b) serve as motivation for students to stay in school and perform better in academic courses (Daggett, 2009). All CTE courses in Virginia have been correlated to the core content area SOL; thus, the curriculum for these courses covers some of the same content as the core courses with a practical and contextual application. In 2000, the Virginia Department of Education’s (VDOE) Career and Technical Education Department developed crosswalks (correlations) to the SOLs in the four content areas. These correlations provide contextual connections between CTE and the content areas. After development of the crosswalks, VDOE made this information available on the state’s CTE state website with the student competency task for each of the courses offered in Virginia (Virginia’s CTE Resource Center, 2009).
With the demands of accountability testing intensifying, career and technical educators will face even more pressure to devote time and energy preparing their students for the high-stakes assessments, potentially threatening time formally devoted to the attainment of specific technical skills. (Thomas, 2004, pp. 26-27)
This brings attention to the potential need for the investigation of the impact of high-stakes testing on CTE instructional time in Virginia.
Purpose of the Study
The purpose of this study was to investigate the academic performance of CTE completers and non-CTE completers in the Commonwealth of Virginia on the SOL English reading and mathematics assessments as well as cohort graduation rates. Through an evaluation of CTE students’ performance on Virginia’s SOL assessments in English reading, mathematics, as well as graduation rates, educational leaders would gain information about the performance of CTE students as it relates to academic performance on the state-mandated SOL tests. In this time of school accountability, a study on the impact of CTE on academic achievement in the Commonwealth of Virginia is due to be investigated.
Significance of the Study
Researchers note that there is a scarcity of research on the impact of high-stakes testing and NCLB on CTE (Gordon et al., 2007). Federal legislation has regulated CTE since its inception and continues to mandate the operation of the programs. Thus, there is documentation available in reference to the specifics of the Carl Perkins legislation and the requirements of the grant that funds secondary and postsecondary CTE programs. However, few state studies have been conducted to evaluate the impact of the requirements of federal-mandated testing of NCLB on CTE programs (Gordon et al., 2007).
Literature Review
Even though CTE courses use contextual learning and real-world application, the courses are typically offered as electives, and because CTE is not mentioned in the NCLB legislation, it may be difficult for policy makers, educators, and teachers to see how CTE courses contribute to achieving the goals of the legislation. Policy makers and educators have generally acknowledged the value of the practical skills taught in CTE courses, but they rarely receive the status or importance attributed to academic skills and knowledge learned in CTE courses. The literature acknowledges that CTE provides students with practical occupational experiences that help students to see the relevance of school, but as the demands of NCLB continue to increase, educators are making decisions to enforce more instructional time in core content courses. The reviewed literature indicates that a “narrowing curriculum” has evolved as a result of increased accountability requirements (Amrein & Berliner, 2002; Davis, 2006). As a result, CTE is in a position that it must demonstrate its contribution to student achievement.
Based on the contextual approach used in CTE courses, student achievement and learning are improved (Glenn, 2005). In addition, there is an increased emphasis on incorporating core academic skills into CTE. As a result of the Carl D. Perkins Vocational and Technical Education Act of 2006, many states have closely aligned their CTE competencies with state academic standards, which resulted in case study results that demonstrate isolated successes. The literature reviewed has yielded mixed results regarding the impact of CTE on student achievement and graduation rates. Studies reviewed indicate that CTE student achievement may not surpass that of their non-CTE counterparts, but there has been an increase in their academic performance and course-taking patterns. Investigating the historical perspective of CTE and how it has evolved to meet the needs and demands of changing educational reforms, including the current legislation of NCLB and high-stakes testing, is important to this study. The literature indicates that there is a need for CTE to continually demonstrate its contributions to the accountability benchmarks set forth in NCLB to avoid narrowing the curriculum.
Method
After reviewing the literature regarding narrowing the curriculum and CTE student achievement, it was determined that examining CTE student achievement and graduation rates in Virginia was worthy of investigation. The purpose of this study was to investigate the academic performance of CTE completers and non-CTE completers in the Commonwealth of Virginia on the SOL Grade 11 English reading and mathematics assessments as well as cohort graduation rates. Evaluation of SOL pass rates in English reading and mathematics as well as graduation rates provided data regarding the performance of CTE completers as compared with non-CTE completers. The research questions that guided this study were as follows:
This study used a quantitative evaluation methodology to determine the difference between CTE concentrators and nonconcentrators with regard to academic achievement and graduation rates of high school students. According to Miller and Salkind (2002), evaluation research involves assessing the strengths and weakness of a program to evaluate effectiveness or merit. In this study, the “program” is CTE, and the value or “merit” is the impact of CTE concentration on student achievement. As indicated in the literature review, there is a narrowing of the curriculum that has caused schools to reduce the number of electives offered or to reduce the instructional time provided for noncontent electives to meet state-mandated test requirements. Thus, research provides data to support the fact that elective instruction, as it relates to CTE, would assist educators in making decisions for their schools regarding these courses.
Population
The population for this study was the 131 school divisions in the Commonwealth of Virginia. Data submitted to the VDOE by each school division served as the database for this study. CTE and divisional data for this study are public domain regarding CTE completers’ English reading and mathematics SOL pass rates and graduation rates as a result of information submitted to the VDOE annually.
Data Collection
Data collected for the 2008, 2009, and 2010 school years with regard to SOL student performance as well as graduation data for the same years were used for evaluation. For the purposes of this study, student achievement was measured by Grade 11 English reading and mathematics SOL pass rates. The mathematics data were based on the highest level taken during the adequate yearly progress reporting year. Data used for this study were gathered from the VDOE Office of Educational Information Management and the VDOE websites. Three VDOE website reports provided much of the data necessary for this study:
CTE Annual Performance Report (by school division),
VDOE School Division Report Card and Virginia Assessment Results, and
VDOE High School Graduates and Completion report.
Student achievement in Virginia is measured by the SOL assessments, and they serve as the high-stakes assessment that determines student graduation and school accreditation. High school students are expected to pass a minimum of two English, one mathematics, one science, one social studies, and one student-selected end-of-course assessment to graduate from a Virginia high school. Data on school division report cards include student achievement based on the grade level of the students in the division and end-of-course assessment results. By definition in the Commonwealth of Virginia, a CTE “completer” is a student who has met the CTE’s two sequential elective concentration of courses and all requirements for high school graduation (VDOE, 2010). Therefore, data for this study include Grade 11 English reading and mathematics (highest level) SOL results for the reported completers and non-CTE completers during the identified school years to provide a conventional comparison. Groups were compared based on their high-stakes test scores over 3 years.
Last, the High School Graduates and Completion report collects data on an annual basis during the fall, following students’ completion year. Each locality submits data for their schools for the combined regular and summer terms. The data are then compiled, and division and state totals are calculated (VDOE, 2010). This report provides graduation numbers for each of the 131 school divisions in the Commonwealth of Virginia. Data collected on the cohort graduates for the 2008, 2009, and 2010 school years were used for this study.
Research Design
According to Cohen, Manion, Morrison, and Morrison (2007), ex post facto research is also called causal-comparative research. In ex post facto research, there is an attempt to relate an after-the-fact treatment that cannot be manipulated by the investigator to an outcome or dependent measure. The specific type of ex post facto research design that was used in this study is criterion group. The “phenomenon” or “variable” in this study is CTE concentration. The dependent variables are student achievement and graduation rates.
The data for the school divisions were divided into two categories: CTE completers and non-CTE completers. CTE completers are students who have met the requirements for a CTE concentration and all requirements for high school graduation or an approved alternative education program. Completers have passed at least 80% of the program competencies in an approved CTE concentration sequence or set of sequential electives. Non-CTE completers are students who have not completed an approved CTE concentration.
The independent variable group was categorized as CTE completers and non-CTE completers. The CTE “completer” achievement data were obtained from the CTE Performance reports for each school division. For the purposes of this study, the Grade 11 English reading SOL pass rates were used for student achievement in English, and the mathematics SOL data were based on students attaining a passing score on highest level of mathematics end-of-course test as reported by the VDOE. The graduation data were derived from the VDOE Graduation and Completion report, and the CTE Performance reports for each of the school divisions. These dependent variable data, student achievement and graduation rates, were downloaded from VDOE to an Excel spreadsheet for analysis. Student achievement was reviewed for the 2008, 2009, and 2010 school years.
Data Collection Procedures
The various data collected for this study were based on the research questions. Information from the VDOE State Division report cards, VDOE Career and Technical Education Performance reports, and the VDOE Graduation and Completion report are submitted annually to the Commonwealth. The data were analyzed using statistical measures to determine the differences in the performance of each of the identified groups—CTE completers and non-CTE completers. A spreadsheet was developed to organize the collected data for further analysis.
Data Analysis
In analyzing data for this quantitative study, Microsoft Excel and Statistical Package for the Social Sciences (SPSS) were used to do the statistical calculations. Statistics were generated for SOL English reading, SOL mathematics, and graduation rates for CTE completers and non-CTE completers for each school division in the Commonwealth of Virginia. Longitudinal data for 3 years were compared to assess whether CTE concentration affects student achievement. Data were analyzed using descriptive statistics, including frequencies, means, and standard deviations. The descriptive statistics were used to describe what was occurring in the data. The numbers of CTE completers and non-CTE completers as well as the corresponding dependent variables for each category were calculated. Total CTE completer students by school division were marked as such with codes, as were the students who had not completed an approved concentration sequence.
A
Results
The Commonwealth of Virginia had more than 110,000 students who participated in CTE to the level of being considered CTE completers. CTE completers are students who have taken two sequential CTE electives in a prescribed program area with 80% proficiency in the competencies required for the courses.
For the purposes of this study, the reading data were based on the end-of-course Grade 11 English reading SOL assessment. The mathematics data were based on the mathematics course for which a SOL assessment is required. The SOL assessments are given in the spring of each school year. All students enrolled in end-of-year courses (EOC) take the associated EOC SOL test. In each of the years of this study, CTE completer pass rates outperformed those of the rest of the students on the reading SOL by at least 3% points. In the area of mathematics during the years of 2009 and 2010, CTE completers had pass rates of 7% to 10% points higher than the rest of the Commonwealth of Virginia. The only exception was during the 2008-2009 school year, where the CTE completer mathematics pass rate was lower than the pass rate of the state. During the year of 2008, the state pass rate on the Algebra I, Algebra II, and Geometry SOL was 91%, whereas the performance of CTE completers for that same year was only 84%.
Research Question 1—Results
What is the difference, if any, between high school CTE enrollment status (CTE completer vs. non-CTE completer) and student achievement as measured by SOL mathematics pass rates of the highest level of mathematics completed (Algebra I, Algebra II, and/or Geometry)?
A breakdown of school divisions by mathematics SOL pass rate categories was organized to illustrate the number of school divisions by CTE enrollment status that fell within the selected pass rate bands. In 2008, 57 school divisions had a CTE pass rate in mathematics below 82%. However, the following 2 school years, more than 120 school divisions had CTE pass rates in mathematics above 95%. In the category of non-CTE completers, an average of 22 school divisions had mathematics pass rates above 95%.
Descriptive statistics were calculated for the CTE completers and non-CTE completers for the years of the study. Using the rescaled student population numbers, the data were imported into SPSS for analysis. The mean mathematics SOL pass rate for non-CTE completers in 2008 was 90.65 with a standard deviation of 5.14, and CTE completers had a mean pass rate of 82.91 with a standard deviation of 9.55 for that same year. In the years of 2009 and 2010, CTE completers had mean pass rates of 97.86 with a standard deviation of 2.05 and 97.61 with a standard deviation of 2.26, respectively. Non-CTE completers for the same school years had mean pass rates of 88.53 with a standard deviation of 5.37 and 88.83 with a standard deviation of 6.22. The highest mean mathematics pass rate was among the CTE completers in 2009 at 97.86, and the lowest mean mathematics pass rate was also among the CTE completers in 2008 at 82.91.
On further analysis of the data, an examination of standardized coefficients of skewness and kurtosis was conducted to determine the normality of the distribution. The coefficient of skewness provided information regarding the degree of departure of the data from that of a normal distribution. The skewness of the distribution of data must be determined to evaluate whether the distribution is significantly skewed to the left or right tail of the distribution. If the skewness is within the range of +3 or −3, then it is not seriously violated. The coefficient of kurtosis provided information regarding the peakness of the distribution or the concentration of data to the center, the upper and lower ends, and the shoulder. A normal distribution will have a kurtosis value of 0, with a range of normality of ±3. With 11 of the 12 skewness and kurtosis factors beyond the range of ±3, it was determined that each of the distributions depart from that of a normal curve for the dependent variable of mathematics SOL pass rates. A total of 11 of the 12 coefficients calculated are beyond the range for a normal distribution.
As the mathematics SOL pass rates for CTE completers and non-CTE completers were not normally distributed, a nonparametric statistical procedure was used. Nonparametric statistics are used when the data do not fit in a normal distribution. Nonparametric statistics are also referred to as
Accordingly, a Wilcoxon’s
Research Question 2—Results
What is the difference, if any, between high school CTE enrollment status (CTE completer vs. non-CTE completer) and student achievement as measured by Grade 11 English reading SOL pass rates?
When considering the second research question, Grade 11 English reading SOL pass rates were examined. School division pass rates were initially reviewed based on the frequency of school divisions in the pass rate bands. Nearly all of the CTE completer school division pass rates were at 89% or above. Only one of the CTE completer school divisions in the 3 years of this study had a pass rate below 89%, whereas non-CTE completer pass rates were on average evenly split above and below 89%.
The mean CTE completer pass rates for the 3 years examined in this study were higher than non-CTE. The mean CTE completer pass rate for the English reading SOL in 2008 was 97.53 as compared with the mean 2008 non-CTE completer pass rate of 90.39. In 2009, the mean CTE completer pass rate was 97.99 with non-CTE completers having a mean pass rate of 89.16. Furthermore, the highest mean Grade 11 English reading SOL pass rate was among CTE completers in 2009 with a mean pass rate of 97.99 and a standard deviation of 1.83.
An examination of the standardized skewness coefficient and the standardized kurtosis coefficient for the English reading SOL passing rates revealed that the distribution of the data departs from that of a normal distribution. As the English reading SOL scores of the CTE students were not normally distributed, a Wilcoxon’s t test was used to address the second research question.
Further analysis of the Grade 11 English reading SOL pass rates was used to determine whether there was a difference in students’ performance on the assessment with regard to CTE enrollment status. The findings of the Wilcoxon’s
The effect size associated with the difference in the Grade 11 English reading SOL pass rates, Cohen’s
Research Question 3—Results
What is the difference, if any, between high school CTE enrollment status (CTE completer vs. non-CTE completer) and the graduation rates of students?
When evaluating the third research question, graduation rates were gathered for the groups evaluated in this study. The cohort graduation rates were included based on the graduation band in which the school divisions’ performance dictates. The highest mean graduation rate was among the CTE completers in 2008 with a mean of 98.08 and a standard deviation of 8.77. This mean is notably higher than the mean graduation rate for non-CTE completers in that same year of 85.50 with a standard deviation of 7.66. The extreme difference in the mean graduation rates may be an indication of the change in reporting of high school graduates in response to on-time graduation rates and the mandates of NCLB. Mean CTE completer graduation rates remained above 90% for the 3 years of the study, whereas non-CTE completer cohort graduation rates were in the mid-80% range. School year 2010 had the lowest mean CTE completer graduation rate (94.90), whereas the non-CTE completers in that same year had their highest cohort graduation rates (88.88). Although the non-CTE completers had an increase in their mean graduation rate, it was still nearly 10% points below that of CTE completers.
An examination of the graduation rates of the participants of this study to that of a normal distribution remains consistent with the findings for the previous two research questions. A review of the histogram and the coefficients of skewness and kurtosis indicated that the data were not normally distributed. Because the data for the graduation rates were not normally distributed as proven by the coefficients of skewness and kurtosis, the nonparametric statistic was used for these data as well. The Wilcoxon’s
Delving deeper into the results of the Wilcoxon, the effect size associated with the difference in the graduation pass rates was found to be large using Cohen’s
The second structure analyzed was the Grade 11 English reading pass rates for the populations in the study. The descriptive statistics and
The third section focused on the graduation rates of students based on CTE enrollment status. As with the Grade 11 English reading SOL pass rates for CTE completers, the graduation rates for this group of students indicated that nearly 95% of the school divisions had CTE completer graduation rates above 90%. As the accountability requirements of NCLB continue to increase, these data will become increasingly important to school divisions.
Each of the sections outlines specific data regarding the research questions, data collection and rescaling procedures, relevant descriptive statistics, Wilcoxon’s
Findings, Implications for Further Study, and Reflections
The purpose of this study was to investigate the academic performance of CTE completers and non-CTE completers in the Commonwealth of Virginia on the SOL English and mathematics assessments as well as cohort graduation rates. This study focused on the pass rates and graduation rates for each of the 131 schools divisions in the Commonwealth of Virginia based on data that were reported to the Department of Education. There were two independent variables in the study based on CTE concentration status. Participants of the study were categorized as CTE completers or non-CTE completers. The three dependent variables were the pass rates on the Grade 11 English reading SOL and the mathematics SOL pass rates for which the students were enrolled. The graduation rate data were based on the cohort information as reported to the VDOE.
One of the intents of this study was to determine whether CTE is a viable program that can contribute to the academic proficiencies required for high-stakes testing, such as Virginia’s SOL. Furthermore, this study is intended to be a medium to provide quantifiable data to public school divisions when making decisions regarding curriculum and program evaluation in this era of accountability and NCLB. The purpose of this study was to investigate the academic performance of CTE completers and non-CTE completers in the Commonwealth of Virginia on the SOL English reading and mathematics assessments as well as cohort graduation rates.
Discussion of Findings
First, in the study populations, CTE completers earned higher mean pass rates on the mathematics SOLs during the 2008-2009 and 2009-2010 school years. Specifically, CTE completers had a mean pass rate of 9% points higher than non-CTE completers during the 2009 and 2010 school years. These results suggest that CTE completers do outperform their non-CTE counterparts on mathematics SOLs. Second, the pass rate for CTE completers was higher than that of the non-CTE completers on the Grade 11 English reading SOL. The mean reading SOL pass rates for CTE completers during the years of this study were 7% to 8% points higher than that of the non-CTE completers. Third, CTE completers also attained mean high school cohort graduation rates of 6% to 13% higher than non-CTE completers for the years of the study. Students in CTE courses graduated with their cohort of classmates at an average graduation rate of 96%, whereas the average non-CTE cohort graduation rate was 87% for the three academic years of the study.
Implications for Practice
The results of this study indicate that CTE affects student achievement and graduation rates. As school divisions make decisions regarding cutting CTE programs, the results of this study should be reviewed as it concludes that students who complete a CTE sequence of courses demonstrate higher mathematics, reading, and graduation pass rates. Several implications for practice were identified from the findings of this study. The research indicates that there are mixed reviews regarding CTE’s impact on student achievement and graduation rates. These inconclusive studies served as the impetus for this study in the Commonwealth of Virginia and the desire of the researcher to investigate the performance of CTE completers in the state. Based on the implications below, educators should explore further the role of CTE in the achievement of students.
Implication 1
School administrators interested in increasing mathematics and reading SOL performance should encourage more students to enroll in and complete CTE sequenced courses. For this study, CTE completers demonstrated a statistically significant higher pass rate on the mathematics and Grade 11 English reading SOL than their non-CTE counterparts. Therefore, school divisions and educational leaders should encourage enrollment of students in CTE courses.
Implication 2
School administrators interested in increasing cohort graduation rates should encourage more students to enroll in and complete CTE sequenced courses. Based on the results of this study, the CTE completer cohorts had an average graduation rate of 95%, which was well above the average for non-CTE completers during the same years.
Implication 3
School administrators should encourage non-CTE completers in CTE courses to complete a sequence of CTE courses and become completers. Increasing the number of non-CTE completers taking sequenced courses and becoming completers would benefit school divisions and students. As evidenced by the pass rates of CTE completers, student participation in courses that are not included in the state-mandated testing could prove to be as valuable for future plans as time spent preparing for the required academic assessments (Daggett, 2005).
Implication 4
Career and technical educators should continue to document and collect data on the performance of program completers to monitor CTE effectiveness. To promote CTE as a program viable in meeting benchmarks, disaggregated data regarding specific program areas and course work should be investigated. In this time of narrowing curriculum and data-driven decisions, CTE must continue to collect, document, and promote CTE courses.
Implication 5
School administrators should evaluate instructional practices used in CTE courses for their merit and value with integrated instruction. Educators acknowledge that students learn more when the work is connected to their interests, to real-world problems, and the world of work and college.
Recommendations for Further Research
Daggett (2009) states that for CTE to establish and maintain itself as a viable component of the comprehensive school program, there is a need to determine how students who have completed a CTE concentration of sequenced courses perform on state high-stakes tests. Therefore, the researchers recommend that more quantitative studies are conducted to determine the impact of career of technical education on the academic achievement and graduation rates of students. Researchers note that there are limited studies on the impact of CTE on NCLB and high-stakes testing. Thus, there is much more quantitative research that should be done.
Little information exists regarding the academic performance of CTE concentrators on high-stakes state assessments. There are numerous national studies and anecdotal data regarding CTE student performance, but given the increased focus on NCLB benchmarks, the researchers discovered limited research on CTE performance on the high-stakes standardized tests required by each state. With the growing accountability requirements, it is especially important for CTE teachers and administrators to take this opportunity to highlight the success of their program areas and courses as it relates to student progress and graduation rates.
Instructional practices and contextual learning strategies used in CTE courses produce the successful student academic performance as demonstrated in this study. This investigation of Virginia’s CTE completer pass rates compared with students who have not completed a CTE program and students’ success on the SOL assessments.
As the nation begins to embrace the concept of 21st century skills, performance-based learning, and President Obama’s college and career readiness initiatives, it is time for CTE to showcase how its programs support and encompass these concepts. Daggett (2009) suggests that CTE must continue to clearly demonstrate that these programs contribute to the academic success of students and motivates students to stay in school through graduation.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research and/or authorship of this article.
Bios