Title page
Contents
1. Introduction 10
1.1. Background and Purpose of PIRLS 2016 10
1.2. PIRLS 2016 U.S. Data Collection Activities and Schedule 11
1.3. Overview of the Design and Administration of PIRLS and ePIRLS 2016 12
1.4. Reporting PIRLS and ePIRLS Results 16
1.5. U.S. International and National Data Files 17
2. Sample Design 18
2.1. School Sampling Frame 18
2.2. School Sample Selection 19
2.3. Selection of Classrooms and Students 21
2.4. Tabulations Within Subgroups for Frame and Sample 21
3. Participation Rates and Nonresponse Bias 25
3.1. Exclusions 26
3.2. PIRLS Participation Rates of U.S. Schools, Classrooms, and Students 27
3.3. Participation Rates for All Countries 31
3.4. Nonresponse Bias Analysis 31
4. Survey Operations 48
4.1. Recruiting Districts and Schools 48
4.2. Sampling Students Within Schools 51
4.3. PIRLS Instruments 60
4.4. Field Operations 66
4.5. Receipt Control, Scoring, Coding, and Data Entry 71
4.6. Data Preparation 74
5. U.S. PIRLS and ePIRLS 2016 Data 80
5.1. U.S. International, and National Data Files 80
5.2. PIRLS and ePIRLS Data Files 82
5.3. PIRLS and ePIRLS Achievement Data Files and Variable Names 83
5.4. PIRLS and ePIRLS Background Questionnaire Data Files 85
5.5. Variable Naming Convention for Background Variables 87
5.6. Sampling and Weighting Variables 88
5.7. Structure and Design Variables in PIRLS and ePIRLS 2016 Data Files 90
5.8. PIRLS and ePIRLS 2016 Codebook Files 91
5.9. U.S. National Instrumentation 91
5.10. Merging PIRLS 2016 Data Files 96
5.11. Some Notes on Analyzing the PIRLS 2016 Data 104
References 110
Appendix A. PIRLS 2016 Recruitment Materials 112
Appendix B. Field Staff Training Materials 132
Appendix C. Questionnaires 162
Appendix D. National Adaptation Forms 214
Table 2-1. Number and percentage of students and schools included in the U.S. PIRLS school sampling frame, by grade span: 2016 19
Table 2-2. Number and percentage of students and schools included in the U.S. PIRLS school sampling frame and number and percentage of schools in the sample, by Census region: 2016 22
Table 2-3. Number and percentage of students and schools included in the U.S. PIRLS school sampling frame and number and percentage of schools in the sample, by poverty level: 2016 22
Table 2-4. Number and percentage of students and schools included in the U.S. PIRLS school sampling frame and number and percentage of schools in the sample, by school control: 2016 23
Table 2-5. Number and percentage of students and schools included in the U.S. PIRLS school sampling frame and number and percentage of schools in the sample, by locale: 2016 23
Table 2-6. Number and percentage of students and schools included in the U.S. PIRLS school sampling frame and number and percentage of schools in the sample, by minority status: 2016 23
Table 2-7. Number and percentage of students and schools included in the U.S. PIRLS school sampling frame, by Census region, poverty level, and school control: 2016 24
Table 3-1. Number of U.S. schools, classrooms, and students participating in PIRLS, and participation rates, for fourth grade: 2016 27
Table 3-2. Number of U.S. schools, classrooms, and students participating in ePIRLS, and participation rates, for fourth grade: 2016 28
Table 3-3. Percentage distribution of eligible and participating schools in the U.S. PIRLS original sample and nonresponse bias indicators, by selected school characteristics: 2016 35
Table 3-4. Mean enrollment of eligible and participating schools in the U.S. PIRLS original sample and nonresponse bias indicators: 2016 35
Table 3-5. Mean percentage of students in eligible and participating schools in the U.S. PIRLS original sample and nonresponse bias indicators, by race/ethnicity: 2016 36
Table 3-6. Mean percentage of students eligible for free or reduced-price lunch, in eligible and participating public schools in the U.S. PIRLS original sample and nonresponse bias indicators: 2016 36
Table 3-7. Logistic regression model parameter estimates (with six race/ethnicity variables) using the U.S. PIRLS original sample and nonresponse bias indicators: 2016 37
Table 3-8. Logistic regression model parameter estimates (with summed race/ethnicity percentages) using the U.S. PIRLS original sample and nonresponse bias indicators: 2016 38
Table 3-9. Percentage distribution of eligible and participating schools in the U.S. PIRLS final sample, by selected school characteristics and nonresponse bias indicators: 2016 39
Table 3-10. Mean enrollment of eligible and participating schools in the U.S. PIRLS final sample and nonresponse bias indicators: 2016 40
Table 3-11. Mean percentage of students in eligible and participating schools in the U.S. PIRLS final sample and nonresponse bias indicators, by race/ethnicity: 2016 40
Table 3-12. Mean percentage of students eligible for free or reduced-price lunch, in eligible and participating public schools in the U.S. PIRLS final sample and nonresponse bias indicators: 2016 40
Table 3-13. Logistic regression model parameter estimates (with six race/ethnicity variables) using the U.S. PIRLS final sample and nonresponse bias indicators: 2016 41
Table 3-14. Logistic regression model parameter estimates (with summed race/ethnicity percentages) using the U.S. PIRLS final sample and nonresponse bias indicators: 2016 42
Table 3-15. Percentage distribution of eligible and participating schools in the U.S. PIRLS nonresponse-adjusted sample, by selected categorical variables and nonresponse bias indicators: 2016 43
Table 3-16. Mean enrollment of eligible and participating schools in the U.S. PIRLS nonresponse-adjusted sample and nonresponse bias indicators: 2016 44
Table 3-17. Mean percentage of students in eligible and participating schools in the U.S. PIRLS nonresponse-adjusted sample and nonresponse bias indicators, by race/ethnicity: 2016 44
Table 3-18. Mean percentage of students eligible for free or reduced-price lunch, in eligible and participating public schools in the U.S. PIRLS nonresponse-adjusted sample and nonresponse... 44
Table 3-19. Percentage distribution of schools eligible for and with responses for ACBG08B in the U.S. PIRLS final sample and nonresponse bias indicators, by selected school characteristics: 2016 46
Table 3-20. Mean enrollment of schools eligible for and with responses for ACBG08B in the U.S. PIRLS final sample and nonresponse bias indicators: 2016 46
Table 3-21. Mean percentage of students in schools eligible for and with responses for ACBG08B in the U.S. PIRLS final sample and nonresponse bias indicators, by race/ethnicity: 2016 47
Table 3-22. Mean percentage of students eligible for free and reduced price lunch in schools eligible for and with responses for ACBG08B in the U.S. PIRLS final sample and nonresponse... 47
Exhibits
Exhibit 1-1. Schedule for U.S. PIRLS 2016 data collection activities 12
Exhibit 4-1. Example of PIRLS fourth-grade Class Listing Form with mock data 53
Exhibit 4-2. Example of PIRLS Student Listing Form with mock data 55
Exhibit 4-3. Example of PIRLS Student Tracking Form with mock data 57
Exhibit 4-4. Example of ePIRLS Student Tracking Form with mock data 58
Exhibit 4-5. Example of PIRLS Teacher Tracking Form with mock data 59
Exhibit 4-6. ePIRLS assessment screen 61
Exhibit 5-1. Variables removed from public-use version of PIRLS 2016 international database 81
Exhibit 5-2. U.S. national PIRLS and ePIRLS data file names 83
Exhibit 5-3. Three-digit codes identifying achievement scales in PIRLS and ePIRLS 84
Exhibit 5-4. PIRLS and ePIRLS sampling weight variables 89
Exhibit 5-5. Illustrative SAS code for merging U.S. PIRLS student and school data 97
Exhibit 5-6. Illustrative SPSS code for merging U.S. PIRLS student and school data 98
Exhibit 5-7. Illustrative SAS code for merging U.S. PIRLS student and teacher data 99
Exhibit 5-8. Illustrative SPSS code for merging U.S. PIRLS student and teacher data 100
Exhibit 5-9. Illustrative SAS code for merging U.S. PIRLS school, teacher, and student data 101
Exhibit 5-10. Illustrative SPSS code for merging U.S. PIRLS school, teacher, and student data 102
Exhibit 5-11. Illustrative SAS code for merging U.S. PIRLS school data with restricted-use (CCD and PSS) data 103
Exhibit 5-12. Illustrative SPSS code for merging U.S. PIRLS fourth-grade school data with restricted (CCD and PSS) data 104
Appendix Exhibits
Exhibit A-1. PIRLS 2016 State Contacting Letter 113
Exhibit A-2. PIRLS 2016 District Contacting Letter 114
Exhibit A-3. PIRLS 2016 School Recruitment Letter 115
Exhibit A-4. PIRLS 2016 School Coordinator Letter 116
Exhibit A-5. PIRLS School Coordinator Responsibilities 117
Exhibit A-6. PIRLS 2016 Administrator Questionnaire Letter 120
Exhibit A-7. PIRLS 2016 Teacher Questionnaire Letter 121
Exhibit A-8. PIRLS 2016 Implicit Parent Permission 122
Exhibit A-9. PIRLS 2016 Explicit Parent Permission 124
Exhibit A-10. PIRLS 2016 Study Brochure 126
Exhibit A-11. PIRLS 2016 Main Study FAQ 128
Exhibit A-12. Summary of School Activities: PIRLS 2016 Main Study 130
Exhibit B-1. Test Administrator Training Agenda 133
Exhibit B-2. PIRLS Session Script 137
Exhibit B-3. ePIRLS Session Script 147
Exhibit B-4. Test Administration Form 160
Exhibit C-1. Student Questionnaire 163
Exhibit C-2. School Questionnaire 185
Exhibit C-3. Teacher Questionnaire 195
Exhibit C-4. ePIRLS Student Questionnaire 209
Exhibit C-5. Curriculum Questionnaire 212
Exhibit Tables
Table 1. Activities Timeline 117