Efforts to Improve the Quality of Vocational Education in Secondary Schools: Impact of Federal and State Policies Cathleen Stasz* Susan Bodilly* With the Assistance of Sarah Remes* Tammi Oyadomari-Chun* Daniel McCaffrey* Tessa Kaganoff* Dionne Barnes* *RAND Corporation June 2004 Prepared for the National Assessment of Vocational Education, U.S. Department of Education ED-00-C0-0002 -iii- Contents Tables ix Acronyms xii Preface xiii Acknowledgments xiv Executive Summary xv Study Questions xiv Approach and Methods xvi Reactions to Perkins III Specifications xvii The State Context for Efforts to Improve Vocational Education xix State and Local Efforts to Improve the Quality of Vocational Education xxi Impact of Tech-Prep and Related Federal Policies xxiv Conclusions and Implications xxiv General Conclusions xxvi 1. Introduction 1 The Changing Federal Role in Vocational Education 1 Study Questions 7 Methods 8 Limitations of the Study 9 Roadmap for This Report 9 2. Study Approach and Methods 11 Case Studies 11 Sample Selection 11 Development of Quality Indicators 13 Instruments 16 Procedures 16 Case-study Data Analysis 18 Analysis of High-Achieving and Low-Achieving Schools 19 Teacher Survey 20 Sampling Frame 20 Design and Development 21 Survey Administration 22 Survey Data Analysis 22 3. Reactions to Perkins III Specifications: Funding, Accountability and Special Populations 25 Perkins Funding 25 State Uses of Funds 27 Local Uses of Funds 29 Perkins Accountability Requirements 32 Vocational Education for Special Populations 36 Impact of Changes in Set-Asides and Assessment 37 Conclusions 46 -iv- 4. The State Context for Efforts to Improve Vocational Education 49 Philosophy and Vision 50 The Structures and Delivery Systems for Vocational Education 52 State Reforms 59 Standards 63 Increased Graduation Requirements 66 Assessments 68 Other Vocational Education Reforms 70 Conclusions 71 5. State and Local Efforts to Implement Perkins Quality Improvements 75 Promote Academic and Technical Skill Attainment 76 State Efforts 77 Local Efforts 85 Integration of Academic and Vocational Education 88 State Efforts 90 Local Efforts 91 All Aspects of the Industry 94 Parent and Employer Involvement 97 State Efforts 98 Local Efforts 98 Connections to Postsecondary 99 State Efforts 100 Local Efforts 101 Technology 103 State Efforts 103 Local Efforts 104 Professional Development and Teacher Supply 105 State Efforts 106 Local Efforts 107 Conclusions 108 6. Tech-Prep and Other Federal Policies 113 Tech-Prep and Vocational Education 113 Forms of Tech-Prep 114 Structured Programs: North Carolina and Florida 115 Enhanced Vocational Programs: Massachusetts, Michigan, Ohio and Texas 116 Loosely Structured Programs: California 119 School-to-Work Opportunities Act 119 Workforce Investment Act 122 Conclusions 123 7. Conclusions and Implications 125 Integration 128 Use of Challenging Standards 129 Build Links to Employers and Postsecondary Education 130 Develop, Expand and Improve the Use of Technology 130 General Conclusions and Policy Implications 133 -v- Appendix A: Selection of States and Schools 137 State Selection 137 Selection Criteria 137 Selection Procedures 138 School Selection 139 Analysis of Achievement Data 139 Screening for Vocational Intensity 141 Consideration of School Characteristics 141 Replacing Schools 143 Appendix B: Survey Sample Design 144 The Sampling Frame of Schools and Teachers 144 Sampling Strata 146 Vocational Schools 146 Comprehensive Schools 149 Teacher Strata 150 Survey Response 151 Adjustments for Survey Non-response 153 Standard Error Estimation 156 Creation of the Overall Teacher Quality Scale 156 Q22. Use of Standards (q_22a_related–q22d_related, q22a_affect–q22d_affect). 156 Q20. Academic and Technical Quality (Academic Quality, q20a, q20b, q20d; Technical Quality, q20c, q20f) 157 Q24. Assessed Competencies (q24b, q24c, q24e, q24h, q24j, q24f, q24g, q24i, q24k) 158 Q17. Teacher Quality (q17) 158 Q25. Amount of Homework (q25_hr q25_min) 159 Overall Teacher Scale 160 Appendix C: Efforts to Improve Quality — Summary Tables and Analysis of “High-Performing” and “Low-Performing” Schools 169 Narrative Summary 169 Integration Structure (Table C.1) 169 Integration Curriculum (Table C.2) 170 Challenging Academic and Vocational Standards: Table C.3 170 Connections to Employers and Community: Table C.4 170 Connections to Postsecondary: Table C.5 171 Counseling and Career Planning: Table C.6 171 Technology Policy and Resources: Table C.7 172 Teacher Supply and Professional Development: Table C.8 173 Appendix D: Teacher Survey Tables 175 References 193 -vii- Tables 2.1 Summary of Secondary School Sample 15 2.2 Study Topics by Sources of Data 24 3.1 State Allocation of Basic Grant Funds to Secondary and Postsecondary Institutions 26 3.2 Status of State Accountability Systems 34 4.1 Synopsis of State Governance Structure 53 4.2 Synopsis of Secondary Standards and Assessment 60 5.1 State Policies to Promote Quality 76 B.1 Numbers of Schools by Vocational and Comprehensive Strata 145 B.2 Numbers of Schools by Vocational School Strata 146 C.1 Integration Structure 159 C.2 Integration Curriculum 160 C.3 Challenging Academic and Vocational Standards 161 C.4 Connections to Employers and Community 162 C.5 Connections to Postsecondary 163 C.6 Counseling and Career Planning 164 C.7 Technology Policy and Resources 165 C.8 Teacher Supply and Professional Development 166 D.1 Percentage of Academic and Vocational Teachers Reporting Any Professional Development on Selected Topics in Last 12 Months 173 D.2 Percentage of Teachers in Comprehensive and Vocational Schools Reporting Any Professional Development on Selected Topics in Last 12 Months 174 D.3 Percentage of Vocational Teachers in Comprehensive and Vocational Schools Reporting Any Professional Development on Selected Topics in Last 12 Months 175 D.4 Percentage of Academic and Vocational Teachers Who Report Team Teaching 176 D.5 Percentage of Teachers in Comprehensive and Vocational Schools Who Report Team Teaching 176 D.6 Percentage of Academic and Vocational Teachers Reporting Any Participation with Postsecondary Faculty in Certain Activities 177 D.7 Percentage of Teachers in Comprehensive and Vocational Schools Reporting Any Participation with Postsecondary Faculty in Certain Activities 177 D.8 Percentage of Academic and Vocational Teachers Reporting Any Contact with Business Groups 178 -viii- D.9 Percentage of Teachers in Comprehensive and Vocational Schools Reporting Any Contact with Business Groups 178 D.10 Percentage of Comprehensive and Vocational Schools Requiring Career Plans 179 D.11 Percentage of Academic and Vocational Teachers Who Report that Identified Class Has Particular Characteristics (Percentage of Classes) 179 D.12 Percentage of Academic and Vocational Teachers Reporting “Above Average” Participation of Special Populations in Identified Classes 180 D.13 Percentage of Teachers in Comprehensive and Vocational Schools Reporting “Above Average” Participation of Special Populations in Identified Classes 180 D.14 Existence of Standards for Identified Classes Taught by Academic and Vocational Teachers (Percentage of Teachers Reporting) 181 D.15 Percentage of Teachers Reporting Standards Influence Class a “Moderate” or “Great” Extent 181 D.16 Academic and Vocational Teacher Reports of Frequency of Activities in Most Recent Identified Class (Percentage of Teachers) 182 D.17 Percentage of Academic and Vocational Teachers Reporting Competency Contributes to a “Great Extent” to Students’ Grades in Identified Classes 183 D.18 Reported Amount of Homework Assigned in Identified Classes by Academic and Vocational Teachers (Mean Hours) 184 D.19 Reported Amount of Homework Assigned in Identified Classes by Teachers in Comprehensive and Vocational Schools (Mean Hours) 184 D.20 Reported Preparation of Academic and Vocational Teachers to Teach Technology-Related Skills 184 D.21 Percentage of Academic and Vocational Teachers Reporting “Moderate” or “Great” Input byBusiness Groups in Identified Career-Oriented or Vocational Classes 185 D.22 Percentage of Teachers in Comprehensive and Vocational Schools Reporting “Moderate” or “Great” Input by Business Groups in Identified Career-Oriented or Vocational Classes 185 D.23 Weighted Teacher Scales for Academic and Vocational Teachers and Classes (Mean Scores) 186 D.24 Percentage of Academic and Vocational Teachers Who Report “Moderate” or “Serious” Problems with Technology 187 D.25 Percentage of Teachers in Comprehensive and Vocational Schools Who Report “Moderate” or “Serious” Problems with Technology 187 D.26 Percentage of Academic and Vocational Teachers Reporting Activity Occurs “Frequently” in Identified Class 188 D.27 Percentage of Academic and Vocational Teachers Engaged in Collaboration (Percentage Reporting Engagement in Any Collaboration) 189 -ix- D.28 Percentage of Teachers in Comprehensive and Vocational Schools Engaged in Collaboration (Percentage Reporting Engagement in Any Collaboration) 189 D.29Academic and Vocational Teacher Reports of Amount of Time School Provides for Teachers to Work Together (Percentage of Teachers) 190 D.30 Reports of Amount of Time school Provides for Teachers in Comprehensive and Vocational Schools to Work Together (Percentage of Teachers) 191 -x- Acronyms AAI All Aspects of the Industry ACE Assessment in Career Education AVS Area Vocational School CC Community college COS Course of study CPS Career Preparation System CTE Career and technical education CTSO Career and technical student organization FCAT Florida Comprehensive Assessment Test FTE Full-time equivalent GPA Grade point average HSTW High Schools that Work IEP Institutional Effectiveness Process ISD Intermediate school district ITAC Integrated Technical and Academic Competencies JVS Joint Vocational School LEA Local education agency MCAS Massachusetts Comprehensive Assessment System MCCTE Michigan Center for Career and Technical Education MDCD Michigan Department of Career Development MDR Market Data Retrieval NAVE National Assessment of Vocational Education OCAP Ohio Competency Assessment Profile OCP Occupational Completer Point OMB Office of Management and Budget RCR Respondent coverage rate ROP Regional occupational program ROP/C Regional occupational programs/centers SCANS Secretary’s Commission on Achieving Necessary Skills SREB Southern Regional Education Board SRG Survey Research Group (RAND) STAR Standardized Testing and Reporting STW School-to-Work Opportunities Act of 1994 T&I Trade and industry TEKS Texas Essential Knowledge and Skills VoCATS Vocational Competency Achievement Tracking System WBL Work-based learning WIA Workforce Investment Act [...]... attempted to expand influence over state vocational education programs Perkins III offered specific guidance on the kinds of improvements that a program should incorporate to enhance its quality These improvements included integrating academics into vocational and technical studies; adopting challenging academic, vocational and technical standards; promoting understanding of “all aspects” of an industry;... What other education reforms are occurring, and how have these affected vocational and technical education within states and localities? What is the influence of federal and state policies at the local level? 3 What are the state and local efforts to improve the quality of vocational education, especially with respect to the quality improvements outlined in Perkins III? Does the implementation incorporate... described by Perkins II and III, these efforts were largely on the margins of other state reforms On the positive side, Perkins funding undoubtedly plays a crucial role in state and local efforts to improve the quality of vocational education, especially in some areas It is too soon to tell whether the stronger mandates in Perkins III accountability will have the desired effect, and some of the philosophical,... and to include competencybased applied learning The federal role in vocational education was clarified with the passage of the Smith-Hughes Act in 1917 Its purpose was to provide federal funding for vocational education in public secondary schools Although vocational education programs were carried out in traditional secondary schools, the act separated them from other programs, thus contributing to the. .. reactions to academic testing regimes were somewhat negative, even in states where testing was voluntary Respondents acknowledged that the tests had helped raise academic standards in vocational and technical programs but often at the cost of vocational learning State and Local Efforts to Improve the Quality of Vocational Education Perkins III provided guidance to states to improve the quality of vocational. .. Have these evolved in keeping with Perkins legislation? Many states and localities have adopted the spirit of the Perkins philosophy to broaden the content of and participation in vocational education in secondary schools, and some have enacted specific policies to advance it However, many barriers to reaching this vision remain Chief among these barriers is the continuing marginal position of vocational. .. vocational education Study Questions This study of secondary school vocational education assessed the status of Perkins III at an early stage of implementation and the current quality of vocational offerings through five broad questions: 1 What are the purposes and philosophies of vocational education in secondary schools? Have these evolved in keeping with the Perkins legislation, and how do they differ... Assessment of Vocational Education — a congressionally-mandated study — is charged with evaluating the impact of the Carl D Perkins Vocational and Technical Education Act of 1998, known as Perkins III, and preparing a report to Congress by July 2002 As part of that effort, the National Assessment of Vocational Education commissioned RAND to conduct a study to assess the quality of vocational education in the. .. survey of teachers in comprehensive high schools and vocational schools The survey was designed to examine the extent to which instructional, curricular and related activities in schools and classrooms correspond to quality practices as defined in the federal legislation This report includes selected findings from the teacher survey when they inform the main study questions -9- Limitations of the Study... and vocational schools The survey was designed to examine whether the instructional, curricular and related activities in schools and classrooms correspond to quality practices as defined in the federal legislation It also gathered information about teachers’ backgrounds and their school and teaching environments This report includes selected findings from the teacher survey where they inform the main . standards in vocational and technical programs but often at the cost of vocational learning. State and Local Efforts to Improve the Quality of Vocational Education. xvi Reactions to Perkins III Specifications xvii The State Context for Efforts to Improve Vocational Education xix State and Local Efforts to Improve the Quality of