Title page
Contents
1. INTRODUCTION 17
1.1. Background 18
1.2. Periods of Data Collection 19
1.3. Overview of the Third-Grade Round of Data Collection 20
1.4. ECLS-K: 2011 Kindergarten-Fourth Grade (K-4) Public-Use Data File 21
1.5. Contents of Manual 22
2. DATA COLLECTION INSTRUMENTS AND METHODS 23
2.1. Data Collection Instruments 23
2.1.1. Direct Child Assessment 25
2.1.2. Child Questionnaire 29
2.1.3. Parent Interview 31
2.1.4. General Classroom Teacher Questionnaires 34
2.1.5. Special Education Teacher Questionnaires 39
2.1.6. School Administrator Questionnaires 40
2.1.7. Copyrighted Materials 42
2.2. Data Collection Methods 44
2.2.1. Comparison of Data Collection Methods Used in Third Grade to Those Used in Earlier Data Collection Rounds 44
3. ECLS-K: 2011 DIRECT AND INDIRECT ASSESSMENT DATA 46
3.1. Direct Cognitive Assessment: Reading, Mathematics, and Science 46
3.1.1. IRT-Based Scores Developed for the ECLS-K: 2011 47
3.1.1.1. Theta and the Standard Error of Measurement (SEM) of Theta 48
3.1.1.2. Scale Scores 51
3.1.2. Variables Indicating Exclusion from the Direct Assessment Due to Disability 53
3.1.3. Choosing the Appropriate Score for Analysis 54
3.1.4. Analytic Considerations for Measuring Gains in the ECLS-K: 2011 55
3.1.5. Reliability of the ECLS-K: 2011 Scores 56
3.1.6. Validity of the ECLS-K: 2011 Scores 57
3.2. Direct Cognitive Assessment: Executive Function 58
3.2.1. Dimensional Change Card Sort 59
3.2.1.1. Dimensional Change Card Sort Data Flags 66
3.2.2. Numbers Reversed 69
3.2.2.1. Numbers Reversed Data Flags 77
3.3. Child Questionnaire 77
3.4. Teacher- and Parent-Reported Measures of Child Behavior 78
3.4.1. Teacher-Reported Social Skills 79
3.4.2. Teacher-Reported Approaches to Learning Items and Scale 83
3.4.3. Teacher-Reported Attentional Focusing and Inhibitory Control: Children's Behavior Questionnaire (CBQ) and Temperament in Middle Childhood Questionnaire (TMCQ) 84
3.4.4. Teacher-Reported Student-Teacher Relationship Scale 89
3.4.5. Parent- and Teacher-Report Working Memory 91
3.4.6. Teacher- and Parent-Reports of Children's Peer Relationships 93
4. SAMPLE DESIGN AND SAMPLING WEIGHTS 95
4.1. Sample Design 95
4.1.1. ECLS-K: 2011 School Sample 96
4.1.2. The Sample of Children 98
4.2. Sample Design for the First-Through Third-Grade Years 101
4.2.1. Fall First Grade and Fall Second Grade 101
4.2.2. Spring First Grade, Spring Second Grade, and Spring Third Grade 108
4.2.3. Following Movers 113
4.3. Calculation and Use of Sample Weights 116
4.3.1. Types of Sample Weights 116
4.3.2. Computation of Sample Weights 123
4.3.2.1. Student Base Weights 123
4.3.2.2. Student Weights Adjusted for Mover Subsampling 124
4.3.2.3. Student Nonresponse-Adjusted Weights 124
4.3.2.4. Raking to Sample Control Totals 125
4.3.3. Characteristics of Sample Weights 126
4.3.4. Variance Estimation 127
4.3.4.1. Jackknife Method 127
4.3.4.2. Taylor Series Method 128
4.3.4.3. Specifications for Computing Standard Errors 129
4.3.5. Use of Design Effects 130
5. RESPONSE RATES 136
5.1. Study Instruments 136
5.2. Unit Response Rates and Overall Response Rates 137
5.3. Nonresponse Bias Analysis 159
5.3.1. Effect of Nonresponse on Child Assessment Data 159
5.3.2. Effect of Nonresponse on Parent Interview Data 164
5.3.3. Effect of Nonresponse on Characteristics from the Base Year 170
6. DATA PREPARATION 175
6.1. Coding Text Responses 176
6.1.1. Household Roster Review 176
6.1.2. Partially Complete Parent Interviews 178
6.2. Receipt, Coding, and Editing of Hard-Copy Questionnaires 178
6.2.1. Receipt Control 178
6.2.2. Scanning of Hard-Copy Questionnaires 178
6.2.3. Coding for Hard-Copy Questionnaires 178
6.2.4. Data Editing 179
7. DATA FILE CONTENT AND COMPOSITE VARIABLES 181
7.1. Variable Naming Conventions 183
7.2. Identification Variables 184
7.3. Missing Values 186
7.4. Data Flags 191
7.4.1. Child Assessment Flags (X7RDGFLG, X7MTHFLG, X7SCIFLG, X7NRFLG, X7NRGEST, X7DCCSFLG, X7HGTFLG, X7WGTFLG, X7ASMTST, X7EXDIS) 191
7.4.2. Parent Data Flags (X7PARDAT, X7EDIT, X7BRKFNL) 193
7.4.3. Teacher Flags (X7TQCDAT, X7TQSUBDAT, X7TQTDAT, X7SETQA, X7SETQC) 194
7.4.4. School Administrator Data Flag (X7INSAQ) 195
7.4.5. Destination School Flag (X7DEST) 196
7.5. Composite Variables 196
7.5.1. Child Composite Variables 196
7.5.1.1. Child's Date of Birth (X_DOBYY_R and X_DOBMM_R) 197
7.5.1.2. Child's Age at Assessment and Date of Assessment (X7AGE, X7ASMTDD, X7ASMTMM, X7ASMTYY) 197
7.5.1.3. Child's Sex (X_CHSEX_R) 198
7.5.1.4. Race/Ethnicity (X_AMINAN_R, X_ASIAN_R, X_HAWPI_R, X_BLACK_R, X_WHITE_R, X_HISP_R, X_MULTR_R, X_RACETHP_R, X_RACETH_R) 198
7.5.1.5. Child's Height (X7HEIGHT) 200
7.5.1.6. Child's Weight (X7WEIGHT) 202
7.5.1.7. Child's Body Mass Index (X7BMI) 203
7.5.1.8. Child's Disability Status (X7DISABL2, X7DISABL) 203
7.5.1.9. Student Grade Level (X7GRDLVL) 205
7.5.1.10. Child Linked to a Special Education Teacher (X7SPECS) 205
7.5.2. Family and Household Composite Variables 206
7.5.2.1. Household Counts (X7HTOTAL, X7NUMSIB, X7LESS18, X7OVER18) 207
7.5.2.2. Household Rosters 208
7.5.2.3. Parent Identifiers and Type in the Household (X7IDP1, X7IDP2, X7HPAR1, X7HPAR2, X7HPARNT) 211
7.5.2.4. Parent Demographic Variables (X7PAR1AGE, X7PAR2AGE, X7PAR1RAC, X7PAR2RAC) 215
7.5.2.5. Household Income and Poverty (X7INCCAT_I, X7POVTY_I) 217
7.5.2.6. Creation of a Socioeconomic Status Variable 221
7.5.2.7. Respondent ID and Relationship to Focal Child (X7RESID, X7RESREL2) 222
7.5.2.8. Food Security Status 223
7.5.2.8.1. Food Security Status: Raw Scores (X7FSADRA2) 224
7.5.2.8.2. Food Security Status: Continuous Measures (X7FSADSC2) 225
7.5.2.8.3. Food Security Status: Categorical Measures (X7FSADST2) 225
7.5.3. Teacher Composite Variables 225
7.5.4. School Composite Variables 226
7.5.4.1. School Type (X7SCTYP) 226
7.5.4.2. Public or Private School (X7PUBPRI) 228
7.5.4.3. School Enrollment (X7ENRLS) 228
7.5.4.4. Percent Non-White Students in the School (X7RCETH) 228
7.5.4.5. Highest and Lowest Grade at the School (X7LOWGRD, X7HIGGRD) 230
7.5.4.6. Students Eligible for Free or Reduced-Price School Meals (X7FRMEAL_I) 231
7.5.4.7. Geographic Region and Locality of the Child's School (X7REGION, X7LOCALE) 234
7.5.5. Field Management System (FMS) Composite Variables 236
7.5.5.1. School Year Start and End Dates (X7SCHBDD, X7SCHBMM, X7SCHBYY, X7SCHEDD, X7SCHEMM, X7SCHEYY) 236
7.5.5.2. Year-Round Schools (X7YRRND) 237
7.5.6. School District Poverty (X7DISTPOV) 237
7.6. Methodological Variables 237
8. ELECTRONIC CODEBOOK (ECB) 239
8.1. Introduction 239
8.1.1. Hardware and Software Requirements 239
8.2. Installing, Starting, and Exiting the ECB 240
8.2.1. Installing the ECB Program on Your Personal Computer 240
8.2.2. How to Start the ECB 244
REFERENCES 247
APPENDIX A. DATA ANOMALIES, ERRATA, AND DATA CONSIDERATIONS 251
APPENDIX B. SUPPLEMENTAL GUIDE FOR THE KINDERGARTEN-FOURTH GRADE PUBLIC-USE DATA FILE 260
Table 3-1. Direct cognitive assessment: IRT theta scores, fall and spring kindergarten, fall and spring first-grade, fall and spring second-grade, and spring third-grade assessments: School years 2010-11, 2011-12, 2012-13, and spring 2014 50
Table 3-2. Direct cognitive assessment: IRT standard errors of measurement (SEM), fall and spring kindergarten, fall and spring first-grade, fall and spring second-grade, and spring third-grade assessments: School years 2010-11, 2011-12, 2012-13, and spring 2014 51
Table 3-3. Direct cognitive assessment: IRT scale scores, fall and spring kindergarten, fall and spring first-grade, fall and spring second-grade, and spring third-grade assessments: School years 2010-11, 2011-12, 2012-13, and spring 2014 53
Table 3-4. Reliability of IRT-based scores (theta and scale scores), by round of data collection and domain, for fall and spring kindergarten, fall and spring first grade, fall and spring second grade, and spring third grade: School years... 57
Table 3-5. Dimensional Change Card Sort variable names, descriptions, value ranges, weighted means, and standard deviations for fall and spring second grade and spring third grade: School year 2012-13 and spring 2014 66
Table 3-6. Numbers Reversed variable names, descriptions, value ranges, weighted means, and standard deviations for fall and spring kindergarten, fall and spring first grade, fall and spring second grade, and spring third grade: School years... 76
Table 3-7. Teacher-reported social skills scales variable names, descriptions, value ranges, weighted means, and standard deviations for fall and spring kindergarten, fall and spring first grade, fall and spring second grade, and spring third... 80
Table 3-8. Teacher-reported social skill scales reliability estimates for fall and spring kindergarten, fall and spring first grade, and fall and spring second grade, and spring third grade: School years 2010-11, 2011-12, 2012-13, and spring 2014 82
Table 3-9. Teacher-reported Approaches to Learning scale variable names, descriptions, value ranges, weighted means, and standard deviations for fall and spring kindergarten, fall and spring first grade, fall and spring second grade, and spring... 84
Table 3-10. Children's Behavior Questionnaire variable names, descriptions, value ranges, weighted means, and standard deviations for fall and spring kindergarten and spring first grade: School year 2010-11 and spring 2012 87
Table 3-11. Temperament in Middle Childhood Questionnaire variable names, descriptions, value ranges, weighted means, and standard deviations for spring second grade and spring third grade: Spring 2013 and spring 2014 88
Table 3-12. Reliability estimates for the teacher-reported Attentional Focus and Inhibitory Control scales for fall and spring kindergarten, spring first grade, spring second grade, and spring third grade: School year 2010-11, spring 2012, spring 2013, and spring 2014 88
Table 3-13. Student-Teacher Relationship Scale variable names, descriptions, value ranges, weighted means, and standard deviations for spring kindergarten, spring first grade, spring second grade, and spring third grade: Spring 2011, spring... 90
Table 3-14. Reliability estimates for the Student-Teacher Relationship Scale teacher-reported Closeness and Conflict scores for spring kindergarten, spring first grade, spring second grade, and spring third grade: Spring 2011, spring 2012, spring 2013, and spring 2014 90
Table 3-15. Parent- and Teacher-Reported Working Memory variable names, descriptions, value ranges, weighted means, and standard deviations for spring third grade: Spring 2014 92
Table 3-16. Reliability estimates for the parent-reported and teacher-reported Working Memory scores for spring third grade: Spring 2014 92
Table 4-1. The ECLS-K: 2011 school sample after school substitution 97
Table 4-2. Number (unweighted) of eligible children sampled for the ECLS-K: 2011, by selected characteristics: School year 2010-11 99
Table 4-3. Number (unweighted) of sampled children who are base-year respondents, by selected characteristics: School year 2010-11 100
Table 4-4. Number (unweighted) of original sampled schools in the 30 PSUs selected for the fall data collections, by selected characteristics: Fall 2011 and fall 2012 102
Table 4-5. Number (unweighted) of original sampled schools with base-year respondents at the start of the fall data collections, by selected characteristics: Fall 2011 and fall 2012 103
Table 4-6. Number (unweighted) of base-year respondents in the fall first- and second-grade sample, by selected characteristics: Fall 2011 and fall 2012 104
Table 4-7. Number (unweighted) of base-year respondents in fall first grade, by type of sampled school and selected characteristics: Fall 2011 106
Table 4-8. Number (unweighted) of base-year respondents in the fall second grade, by type of sampled school and selected characteristics: Fall 2012 107
Table 4-9. Number (unweighted) of original sampled schools in the 90 PSUs selected for the spring data collections with base-year respondents, by selected characteristics: Spring 2012, spring 2013, and spring 2014 108
Table 4-10. Number (unweighted) of base-year respondents in spring first grade, by type of sampled school and selected characteristics: Spring 2012 110
Table 4-11. Number (unweighted) of base-year respondents in spring second grade, by type of sampled school and selected characteristics: Spring 2013 111
Table 4-12. Number (unweighted) of base-year respondents in spring third grade, by type of sampled school and selected characteristics: Spring 2014 112
Table 4-13. Characteristics of the third-grade weights 126
Table 4-14. Standard errors and design effects for selected survey items, third grade: Spring 2014 133
Table 4-15. Median design effects for the spring third-grade survey items, by school characteristic: Spring 2014 135
Table 5-1. Response rates for child assessment and parent interview, by selected school characteristics, third grade: Spring 2014 141
Table 5-2. Response rates for child assessment and parent interview, by selected student characteristics, third grade: Spring 2014 143
Table 5-3. Response rates for teacher-level teacher questionnaires, by selected school characteristics, third grade: Spring 2014 144
Table 5-4. Response rates for teacher-level teacher questionnaires, by selected student characteristics, third grade: Spring 2014 146
Table 5-5. Response rates for student-level teacher questionnaires, by selected school characteristics, third grade: Spring 2014 147
Table 5-6. Response rates for student-level teacher questionnaires, by selected student characteristics, third grade: Spring 2014 149
Table 5-7. Overall response rates for child assessment and parent interview, by selected school characteristics, third grade: Spring 2014 150
Table 5-8. Overall response rates for teacher-level teacher questionnaires, by selected school characteristics, third grade: Spring 2014 152
Table 5-9. Overall response rates for student-level teacher questionnaires, by selected school characteristics, third grade: Spring 2014 153
Table 5-10. Response rates for special education teacher questionnaires, third grade: Spring 2014 154
Table 5-11. Response rates for school administrator questionnaire, by selected school characteristics, third grade: Spring 2014 155
Table 5-12. Response rates for school administrator questionnaire, by selected student characteristics, third grade: Spring 2014 157
Table 5-13. Overall response rates for school administrator questionnaire, by selected school characteristics, third grade: Spring 2014 158
Table 5-14. Estimates using unadjusted and nonresponse-adjusted weights, child assessment, spring third grade: Spring 2014 161
Table 5-15. Differences between unweighted and weighted estimates, and between unadjusted and adjusted estimates, child assessment, spring third grade: Spring 2014 162
Table 5-16. Estimates using unadjusted and nonresponse-adjusted weights, parent interview, spring third grade: Spring 2014 165
Table 5-17. Differences between unweighted and weighted estimates, and between unadjusted and adjusted estimates, parent interview, spring third grade: Spring 2014 167
Table 5-18. Differences between unadjusted base-year estimates from kindergarten respondents and unadjusted base-year estimates from third-grade respondents: Spring 2011 and spring 2014 171
Table 5-19. Differences between adjusted base-year estimates from kindergarten respondents and adjusted base-year estimates from third-grade respondents: Spring 2011 and spring 2014 173
Table 7-1. Pointers to parent figure questions: School year 2013-14 215
Table 7-2. Detailed income range categories used in the parent interview: Spring 2014 218
Table 7-3. Missing data for household income: School year 2013-14 218
Table 7-4. Criteria for reporting income to the nearest $1,000 in the spring parent interview and 2013 thresholds for 200 percent of poverty: Spring 2014 219
Table 7-5. ECLS-K: 2011 poverty composite and 2013 census poverty thresholds: Spring 2014 221
Table 7-6. Number and percent of public and private schools and study students with missing data for the percent of children in the school eligible for free or reduced-price lunch (S7PCTFLN): Spring 2014 232
Exhibits
Exhibit 1-1. Data collection schedule: School years 2010-11 through 2015-16 20
Exhibit 2-1. Instruments used in the ECLS-K: 2011 kindergarten, first-, second-, and third-grade rounds of data collection: School years 2010-11, 2011-12, 2012-13, and spring 2014 24
Exhibit 2-2. Child questionnaire topics in the spring 2014 third-grade round of data collection in the ECLS-K: 2011 30
Exhibit 2-3. Parent interview topics, by round of data collection in the ECLS-K: 2011: School years 2010-11, 2011-12, 2012-13, and spring 2014 32
Exhibit 2-4. General classroom teacher teacher-level questionnaire topics, by round of data collection in the ECLS-K: 2011: School year 2010-11, spring 2012, spring 2013, and spring 2014 37
Exhibit 2-5. General classroom teacher child-level questionnaire topics, by round of data collection in the ECLS-K: 2011: School years 2010-11, 2011-12, 2012-13, and spring 2014 38
Exhibit 2-6. Special education teacher questionnaire topics, by round of data collection in the ECLS-K: 2011: Spring 2011, spring 2012, spring 2013, and spring 2014 40
Exhibit 2-7. School administrator questionnaire topics, by round of data collection in the ECLS-K: 2011: Spring 2011, spring 2012, spring 2013, and spring 2014 42
Exhibit 2-8. Copyright-protected instruments in ECLS-K: 2011 43
Exhibit 3-1. Data flag description for the computerized Dimensional Change Card Sort for fall and spring second grade: School year 2012-13 67
Exhibit 3-2. Data flag description for the computerized Dimensional Change Card Sort for spring third grade: Spring 2014 68
Exhibit 3-3. Child questionnaire content and item-level variables for spring third grade: Spring 2014 78
Exhibit 3-4. Teacher-reported item-level variables on peer relationships in spring second grade and spring third grade: Spring 2013 and spring 2014 93
Exhibit 3-5. Parent-reported item-level variables on peer victimization in spring second grade and spring third grade: Spring 2013 and spring 2014 94
Exhibit 4-1. ECLS-K: 2011 third-grade main sampling weights 118
Exhibit 4-2. Weights developed for use with the ECLS-K: 2011 third-grade data, by components for which nonresponse adjustments were made 122
Exhibit 5-1. ECLS-K: 2011 survey instruments and definition of completed instrument: Spring 2014 137
Exhibit 7-1. Prefixes for third-grade variables 183
Exhibit 7-2. Identification variables included in the ECLS-K: 2011 kindergarten-third grade restricted- use data file 185
Exhibit 7-3. Missing value codes used in the ECLS-K: 2011 data file 186
Exhibit 7-4. Definitions of negative and affirmed values for the food security items in the ECLS-K: 2011 kindergarten-third grade restricted-use data file 224
Exhibit 8-1. InstallShield Wizard 241
Exhibit 8-2. Welcome window 242
Exhibit 8-3. Choose Destination Location 242
Exhibit 8-4. Setup Status 243
Exhibit 8-5. InstallShield Wizard Complete 243
Exhibit 8-6a. Desktop icon 244
Exhibit 8-6b. Desktop screen-click start 244
Exhibit 8-7. First-time user dialog box 245
Exhibit 8-8. ECB splash screen 245
Exhibit 8-9. Select Catalog screen 246
Exhibit 8-10. Main ECB screen 246
Table A-1. Frequency of trials affected by the programming error in practice trials, test trials, and both: Spring 2014 256
Appendix Exhibits
Exhibit A-1. Values for the DCCS flag variable (X7DCCSFLG) 258
Exhibit A-2. Response categories added for ECLS-K: 2011 instruments: Third-grade round (spring 2014) 258
Exhibit B-1. ECLS-K: 2011 masked variables, spring 2014 child assessment 263
Exhibit B-2. ECLS-K: 2011 masked variables, spring 2014 parent interview 264
Exhibit B-3. ECLS-K: 2011 masked variables, spring 2014 teacher-level teacher questionnaire 273
Exhibit B-4. ECLS-K: 2011 masked variables, spring 2014 child-level teacher questionnaires 275
Exhibit B-5. ECLS-K: 2011 masked variables, spring 2014 school administrator questionnaire 276
Exhibit B-6. ECLS-K: 2011 masked variables, spring 2014 composite variables 280
Exhibit B-7. ECLS-K: 2011 masked variables, spring 2014 field management system and identification variables 281
Exhibit B-8. ECLS-K: 2011 masked variables, spring 2011, spring 2012, spring 2013 child-level teacher questionnaires 282