Title page
Contents
Acknowledgments 3
Chapter 1. Introduction 18
1.1. Background of Study 19
1.2. Overview of the NHES:2016 Design 21
1.3. NHES:2016 Topical Questionnaires 22
1.3.1. Early Childhood Program Participation Survey 23
1.3.2. Parent and Family Involvement in Education Survey 23
1.3.3. Adult Training and Education Survey 23
1.4. Contents of This Manual 24
Chapter 2. Sampling Methodology 25
2.1. Sampling Households 25
2.1.1. Black and Hispanic Oversample, Sort Order, and Address Type 26
2.1.2. Within-Household Sampling of Eligible Individuals 28
2.2. Sampling for Experiments 30
2.3. Expected and Actual Yield 32
2.4. Precision Requirements 39
2.4.1. Design effects and effective interview counts 39
2.4.2. Topical Estimate Margins of Error 40
2.4.3. Detectable Differences from Prior NHES Administrations 41
2.5. References 50
Chapter 3. Data Collection 51
3.1. Overview of Data Collection 51
3.1.1. Data Collection Activities 51
3.1.2. Methodology 54
3.2. Details of the Data Collection 62
3.2.1. Screener Data Collection 62
3.2.2. Topical Data Collection 65
3.2.3. Bilingual Mailings 71
3.3. Data Collection Support Activities 74
3.3.1. Telephone Operation 74
3.3.2. Telephone Tree Operation 76
3.3.3. Responses to the Topical Questionnaires 78
3.3.4. E-mail Operation 78
3.3.5. Standard Reports 79
3.4. Data Check-in 79
Chapter 4. Data Processing 81
4.1. Data Capture and Imaging 81
4.1.1. Paper Questionnaire Data Capture 81
4.1.2. Web Questionnaire Data Capture 83
4.2. Reformatting and Deduplication 83
4.2.1. Deduplication of cases 84
4.3. Preliminary Interview Status Recode (ISR) Classification 85
4.4. Computer Edits 85
4.4.1. Combining Web and Paper Questionnaire Data 85
4.4.2. Range Checks 87
4.4.3. Consistency Edits 87
4.4.4. Skip Pattern Edits 88
4.4.5. Coding Schools 88
4.5. Final Interview Status Recode (ISR) Classification 89
4.6. Data Review 91
4.6.1. Review of "Other, Specify" Text Items 92
4.7. Data Products 92
4.8. Disclosure Risk Analysis 93
Chapter 5. Response Ra 95
5.1. Unit Response Rates 95
5.1.1. NHES Screener Unit Response Rates 96
5.1.2. NHES Topical Surveys Unit Response Rates 105
5.2. Item Response Rates 122
Chapter 6. Imputation 133
6.1. Imputation Methodology 134
6.1.1. Logic-Based Imputation 134
6.1.2. Hot-Deck Imputation 135
6.1.3. Weighted Random Imputation 138
6.1.4. Manual Imputation 138
6.1.5. Imputation of School Identification Number (School ID) 139
6.2. Post-imputation Processing 139
6.3. Imputation Flags 140
Chapter 7. Weighting and Standard Error Calculation 141
7.1. Weighting Methodology 141
7.2. Household-Level Weights 142
7.3. Person-Level Weights for ECPP, PFI, and ATES 148
7.4. Methods for Computing Sampling Errors 168
7.4.1. Replication Sampling Errors 169
7.4.2. Taylor Series Approximation 172
7.4.3. Software Examples for Replication Sampling Errors and Taylor Series Approximation 172
7.4.4. Approximate Sampling Errors 174
7.5. References 177
Chapter 8. Data Considerations and Anomalies 178
8.1. Data Considerations 178
8.1.1. Change in Data Collection Mode from Prior Years 178
8.1.2. Web Experiment 179
8.1.3. Important Information About School-Level Derived Variables 179
8.1.4. Nonimputation of Common Core of Data and Private School Universe Survey Data 180
8.1.5. Household Composition Variables 180
8.1.6. Missing Race Data for Hispanic Persons 181
8.1.7. Age Considerations 181
8.1.8. Manual Imputation 181
8.1.9. Duplicate Forms 182
8.2. Data Anomalies 182
8.2.1. Mothers' and Fathers' Specific Relationships to Sampled Children 182
8.2.2. Age and Grade Mismatch for Sampled Children 182
8.2.3. Parent Reports of Type of School Child Attends Versus School Classification 183
8.2.4. Truncation of Write-In Text 183
8.2.5. Imputation of Second and Third ATES Certifications/Licenses 183
Chapter 9. Guide to the Data File and Codebook 184
9.1. System Variables (All Files) 185
9.2. Child Health Variables (ECPP and PFI Files) 185
9.3. Child, Household, and Family Variables (ECPP and PFI Files) 186
9.4. Derived ECPP-Specific Variables 194
9.5. Derived PFI-Specific Variables 196
9.5.1. Derived Variables from the Common Core of Data and Private School Universe Survey Data 197
9.6. Derived ATES-Specific Variables 208
9.7. Occupation, Industry, and Manually Coded ATES-Specific Variables 211
9.8. ZCTA-Level Variables 214
9.9. Other Derived, Operational, and Screener Variables (All Files) 220
9.10. Weighting and Variance Estimation Variables 222
9.11. Imputation Flag Variables 223
9.12. Numeric and Character Variables 223
9.13. References 224
Chapter 10. Nonresponse Bias Analysis 225
10.1. Relationship Between Response Rates and Nonresponse Bias 225
10.2. Unit Nonresponse Bias Analysis 227
10.2.1. Analysis of Characteristics Associated With Unit Response Propensities 228
10.2.2. Comparison of Estimates Between Early and Late Responders 263
10.2.3. A Comparison of Survey Estimates Based on Nonresponse Adjusted and Base Weights 276
10.2.4. A Comparison of NHES:2016 Estimates With Estimates From External Data Sources 303
10.3. Item Nonresponse Bias Analysis 304
10.3.1. Comparison of Extreme Imputed and Unimputed Values 305
10.3.2. Comparison of Imputed and Unimputed Distributions 315
10.4. Summary of Nonresponse Bias Findings 324
10.5. References 326
Appendix A. Questionnaires 327
Appendix B. Data File Layout and Position Order 420
Appendix C. Comparison of Estimates 475
Appendix D. Screener Nonresponse Interview Adjustment Cells 506
Appendix E. ECPP Nonresponse Interview Adjustment Cells 511
Appendix F. PFI Nonresponse Interview Adjustment Cells 514
Appendix G. ATES Nonresponse Interview Adjustment Cells 517
Appendix H. Summary of Weighting and Sample Variance Estimation Variables 520
Appendix I. SAS Code for Derived Variables 530
Appendix J. ATES Certification and License Field Coding Manual for the NHES:2016 540
Introduction 541
The Adult Training and Education Survey (ATES) 541
Certifications and Licenses 542
Responses to be Coded: Name of Certification and Kind of Work it is For 543
Assigning Codes 544
Two-Phase Coding Process 544
Coding Steps 545
Coding Taxonomy 545
Coding Guidelines 546
Potential Challenges 547
Flagging Difficult Certifications 549
Coding Taxonomy and Manual 550
Detailed Codebook 553
SCIENCE, ENGINEERING, AND MATHEMATICS 553
BUSINESS 557
FINANCE, INSURANCE, OR REAL ESTATE 559
HEALTH CARE 560
PERSONAL CARE AND SERVICES 564
PUBLIC AND SOCIAL SERVICES 566
TEACHING AND INSTRUCTION 569
TRADES 571
OTHER FIELDS 574
CANNOT DETERMINE PLACEMENT 574
DIFFICULT FLAG 574
Table 1-1. Topical surveys conducted under the NHES Program, by years administered: 1991-2016 20
Table 1-2. Number of completed NHES:2016 surveys and unweighted and weighted single-stage and overall (two-stage) unit response rates, by survey type 22
Table 2-1. Percentage of sample and number of addresses by address selection characteristic: NHES:2016 27
Table 2-2. Percentage of households with eligible adults and/or children for one or more topical surveys: NHES:2014 Feasibility Study, ACS:2013, and NHES:2016 30
Table 2-3. Number of households sampled for methodological experiments and expected and actual sample sizes: NHES:2016 32
Table 2-4. Expected and actual screener eligibility rates, screener response rates, and topical response rates, by experimental treatment group: NHES:2016 34
Table 2-5. Expected and actual percentage and number of households with eligible individuals for one or more topical surveys: NHES:2016 37
Table 2-6. Expected and actual number of cases sampled and number of completed screeners and topical surveys in the NHES:2016 38
Table 2-7. Number of completed interviews by sampling stratum and mode: NHES:2016 38
Table 2-8. Actual interview counts, design effects, and effective interview counts for topical surveys, 2016 expected and 2012 actual 40
Table 2-9. Expected margins of error for NHES:2016 topical surveys, by proportion estimate and subgroup 41
Table 2-10a. Expected detectable changes from 2012 for key Early Childhood Program Participation characteristics: NHES:2016 43
Table 2-10b. Expected detectable changes from 2012 for key Parent and Family Involvement in Education characteristics: NHES:2016 45
Table 3-1. Data collection activity timeline: NHES:2016 52
Table 3-2. Data collection mailing materials: NHES:2016 53
Table 3-3. Mailing schedule for screener questionnaires: NHES:2016 63
Table 3-4. Number and percentage of completed paper screeners received throughout data collection, by week: NHES:2016 64
Table 3-5. Number of completed paper screeners returned, by mailing wave: NHES:2016 65
Table 3-6. Data collection schedule for topical questionnaires, by mailing group: NHES:2016 69
Table 3-7. Number of completed paper topical questionnaires received throughout data collection, by week: NHES:2016 70
Table 3-8. Number of questionnaires returned as undeliverable as addressed (UAA): NHES:2016 71
Table 3-9. Spanish paper screener assignments and returns, by mailing wave: NHES:2016 73
Table 3-10. Spanish paper topical questionnaire assignments and returns, by week: NHES:2016 74
Table 3-11. Telephone call-in reasons on the Telephone Questionnaire Assistance (TQA) telephone line: NHES:2016 76
Table 3-12. Telephone tree operation by mailing group: NHES:2016 77
Table 3-13. E-mails received from respondents, by reason: NHES:2016 79
Table 4-1. Variables edited during the merging of mail and web cases by survey and description of edit: NHES:2016 86
Table 4-2. Number of changes made to entries for the variables in NHES:2016, by percentage of cases with changes and questionnaire type 88
Table 4-3. Results of the NHES:2016 Parent and Family Involvement in Education school coding operation, by school type 89
Table 4-4. NHES:2016 critical items and criteria for final Interview Status Recode classification of completed interview, by questionnaire type 90
Table 4-5. NHES:2016 Final Interview Status Recode counts, by survey type 91
Table 5-1. Count and percentage distribution of households sampled for NHES:2016 screener, by response status 98
Table 5-2. Proportion of known eligibility screener cases that are eligible (ee), by cell 100
Table 5-3. Unweighted and weighted screener unit response rates 101
Table 5-4. Count of sampled households by response status, and weighted screener response rate, by selected household characteristics 103
Table 5-5. Count of addresses, weighted topical response rate, and weighted overall response rate, by topical questionnaire 107
Table 5-6. Count of Early Childhood Program Participation households by response status, and weighted Early Childhood Program Participation response rate, by selected household characteristics 109
Table 5-7. Count of Parent and Family Involvement in Education households by response status, and weighted Parent and Family Involvement in Education response rate, by selected household characteristics 113
Table 5-8. Count of Adult Training and Education Survey households by response status, and weighted Adult Training and Education Survey response rate, by selected household characteristics 118
Table 5-9. Unweighted and weighted item response rates and total response rate, by selected Early Childhood Program Participation items 123
Table 5-10. Unweighted and weighted item response rates and total response rate, by selected Parent and Family Involvement in Education items 125
Table 5-11. Unweighted and weighted item response rates and total response rate, by selected Adult Training and Education Survey items 127
Table 5-12. Early Childhood Program Participation items with weighted response rates below 90 percent 129
Table 5-13. Parent and Family Involvement in Education items with weighted response rates below 90 percent 130
Table 5-14. Adult Training and Education Survey items with weighted response rates below 90 percent 132
Table 7-1. Sampling fractions for screener sample, and household-level base weights, by stratum: NHES:2016 143
Table 7-2. Independent variables for NHES:2016 household-level CHAID analysis 145
Table 7-3. Domain adjustment factor (Ajk) for person-level weighting, by household composition and survey 151
Table 7-4. Independent variables for NHES:2016 person-level CHAID analysis 154
Table 7-5. American Community Survey control totals, by raking dimension for the Adult Training and Education Survey 162
Table 7-6. American Community Survey control totals, by raking dimension for the Early Childhood Program Participation Survey 164
Table 7-7. American Community Survey control totals, by raking dimension for the Parent and Family Involvement in Education Survey 166
Table 7-8. Use of analysis weights, replicate weights, and variance estimation strata and primary sampling unit (PSU) variables available from NHES:2016, by variance estimation method and selected survey data analysis software: 2016 173
Table 10-1. Sampling frame and Census variables used in the NHES:2016 screener-level unit nonresponse bias analysis 229
Table 10-2. Screener and sampling frame variables used in the NHES:2016 topical-level unit nonresponse bias analysis 230
Table 10-3. Summary of bias in NHES:2016 sampling frame characteristics, before and after weighting adjustments for nonresponse 233
Table 10-4. Estimates of unit nonresponse bias for various sample characteristics from the NHES:2016 screener 234
Table 10-5. Estimates of unit nonresponse bias for various sample characteristics from the NHES:2016 PFI Parent and Family Involvement in Education topical survey 241
Table 10-6. Estimates of unit nonresponse bias for various sample characteristics from the NHES:2016 Early Childhood Program Participation topical survey 249
Table 10-7. Estimates of unit nonresponse bias for various sample characteristics from the NHES:2016 Adult Training and Education Survey 256
Table 10-8. Summary of differences in NHES:2016 estimates by mailing wave 265
Table 10-9. Parent and Family Involvement in Education child and household demographic characteristics and key survey estimates by mailing wave returned 265
Table 10-10. Early Childhood Program Participation child and household demographic characteristics and key survey estimates by mailing wave returned 269
Table 10-11. Adult Training and Education Survey demographic characteristics and key survey estimates by mailing wave returned 273
Table 10-12. Summary of changes in NHES:2016 estimates from use of nonresponse-adjusted weights 276
Table 10-13. Parent and Family Involvement in Education child and household demographic characteristics and key survey estimates by race/ethnicity of child and weighting type 278
Table 10-14. Early Childhood Program Participation child and household demographic characteristics and key survey estimates by race/ethnicity of child and weighting type 288
Table 10-15. Adult Training and Education Survey respondent demographic characteristics and key survey estimates by race/ethnicity and weighting type 297
Table 10-16. Percentage distribution or mean of Parent and Family Involvement in Education NHES:2016 variables with item response rates less than 85 percent, original estimate versus estimate with extreme imputed values 309
Table 10-17. Percentage distribution or mean of Early Childhood Program Participation NHES:2016 variables with item response rates less than 85 percent, original estimate versus estimate with extreme imputed values 312
Table 10-18. Percentage distribution of Adult Training and Education Survey NHES:2016 variables with item response rates less than 85 percent, original estimate versus estimate with extreme imputed values 313
Table 10-19. Percentage distribution or mean of Parent and Family Involvement in Education NHES:2016 variables with response rates less than 85 percent, original imputed estimate versus estimate with imputed values excluded 317
Table 10-20. Percentage distribution or mean of Early Childhood Program Participation NHES:2016 variables with response rates than 85 percent, original imputed estimate versus estimate with imputed values excluded 320
Table 10-21. Percentage distribution of Adult Training and Education Survey NHES:2016 variables with response rates less than 85 percent, original imputed estimate versus estimate with imputed values excluded 321
Figure 3-1. Screener mail operations 57
Figure 3-2. Early Childhood Program Participation Survey (ECPP) and Parent and Family Involvement in Education Survey (PFI) mail operations 58
Figure 3-3. Adult Training and Education Survey (ATES) mail operations 59
Figure 3-4. Screener web operations 60
Figure 3-5. Topical web operations 61
Exhibits
Exhibit D-1. Definitions of column headings for Screener interview adjustment cells table 507
Exhibit E-1. Definitions of column headings for ECPP nonresponse adjustment cells table 512
Exhibit F-1. Definitions of column headings for PFI nonresponse adjustment cells table 515
Exhibit G-1. Definitions of column headings for ATES nonresponse adjustment cells table 518
Exhibit H-1. Summary of weighting and sample variance estimation variables: 1991-2016 521
Table B-1. Restricted-Use Data file Layout in Position Order, ECPP:2016 421
Table B-2. Restricted-Use Data file Layout in Position Order, PFI:2016 431
Table B-3. Restricted-Use Data file Layout in Position Order, ATES:2016 443
Table B-4. Public-Use Data file Layout in Position Order, ECPP:2016 449
Table B-5. Public-Use Data file Layout in Position Order, PFI:2016 459
Table B-6. Public-Use Data file Layout in Position Order, ATES:2016 470
Table C-1. Percentage distribution for household size, place of birth, age, and number of children in the household: ECPP-NHES:2016, PFI-NHES:2016, and CPS:2015 476
Table C-2A. Percentage distribution of children ages 3 through 20 not enrolled in school or enrolled in kindergarten through grade 12: ECPP-NHES:2016 and PFI-NHES:2016 477
Table C-2B. Standard errors of the percentage distribution of children ages 3 through 20 not enrolled in school or enrolled in kindergarten through grade 12: ECPP-NHES:2016 and PFI-NHES:2016 478
Table C-2C. Percentage distribution of children ages 3 through 20 not enrolled in school or enrolled in kindergarten through grade 12: CPS:2015 479
Table C-2D. Standard errors of the percentage distribution of children ages 3 through 20 not enrolled in school or enrolled in kindergarten through grade 12: CPS:2015 480
Table C-2E. Difference in percentage distribution of children ages 3 through 20 not enrolled in school or enrolled in kindergarten through grade 12: CPS:2015 vs. NHES:2016 481
Table C-2F. Standard errors of difference in the percentage distribution of children ages 3 through 20 not enrolled in school or enrolled in kindergarten through grade 12: CPS:2015 vs. NHES:2016 482
Table C-3. Number of children in kindergarten through grade 12, by school type and by student grade level: PFI-NHES:2016 and CPS:2015 483
Table C-4. Number and percentage of children in kindergarten through grade 12 enrolled in public and private schools: PFI-NHES:2016 and CPS:2015 484
Table C-5. Percentage of children enrolled in kindergarten through grade 12 enrolled in public and private schools, by race/ethnicity: PFI-NHES:2016 and CPS:2015 486
Table C-6. Percentage of children in kindergarten through grade 12, by household income: PFI-NHES:2016 and ACS: 2015 487
Table C-7. Percentage of children in kindergarten through grade 12, by household income and race/ethnicity: PFI-NHES:2016 and ACS:2015 488
Table C-8. Percentage of students in kindergarten through grade 12, by parents' highest level of education and race/ethnicity: PFI-NHES:2016, PFI-NHES:2012 489
Table C-9. Percentage of children in kindergarten through grade 12 by family structure and parents' highest level of education, and mean number of siblings: PFI-NHES:2016, PFI-NHES:2012 490
Table C-10. Percentage of students enrolled in kindergarten through grade 12, by selected characteristics: PFI-NHES:2016, PFI-NHES:2012 491
Table C-11. Homeschooling rate among students ages 5-17: PFI-NHES:2016, PFI-NHES:2012 492
Table C-12. Percentage of children from birth through age 6 and not enrolled in school, by household income: ECPP-NHES:2016 and ACS: 2015 493
Table C-13. Percentage of children ages 0 through 6 and not enrolled in school, by household income and race/ethnicity: ECPP-NHES:2016 and ACS:2015 494
Table C-14. Percentage of children ages 0 through 6 not yet in kindergarten, by parents' highest level of education and race/ethnicity: ECPP-NHES:2016, ECPP-NHES:2012 495
Table C-15. Percentage of children ages 0 through 6 not yet in kindergarten by family characteristics, and mean number of siblings: ECPP-NHES:2016, ECPP-NHES:2012 496
Table C-16. Percentage of children ages 0 through 6 not yet in kindergarten participating in different care arrangements, by race/ethnicity: ECPP-NHES:2016, ECPP-NHES:2012 497
Table C-17. Percentage of children (ages 0 through 6 not yet in kindergarten) participating in relative, nonrelative, or center- or school-based care who participate in the care arrangement at least once each week, by race/ethnicity:... 498
Table C-18. Percentage of children ages 0 through 6 not yet in kindergarten participating in center-based programs, by high and low income: ECPP-NHES:2016, ECPP-NHES:2012 499
Table C-19. Percentage of children ages 0 through 6 not yet in kindergarten, by frequency read to per week, disability status, and pretending to read: ECPP-NHES:2016 and ECPP-NHES:2012 500
Table C-20. Percentage distribution of adults age 16 through 65, by age, race/ethnicity, and educational attainment: ATES-NHES:2016 and CPS:2015 501
Table C-21. Percentage distribution of adults age 16 through 65, by educational attainment and race/ethnicity: ATES-NHES:2016 and CPS:2015 502
Table C-22. Percentage of adults ages 16 through 65, by total annual earnings: ATES-NHES:2016 and ACS: 2015 503
Table C-23. Percentage of adults ages 16-65, by total annual earnings and race/ethnicity: ATES-NHES:2016 and ACS:2015 504
Table C-24. Percentage of adults ages 16-65 reporting a certification or license, by race/ethnicity: ATES-NHES:2016 and CPS:2015 505
Table D-1. Screener nonresponse adjustment cells, NHES:2016 508
Table E-1. ECPP nonresponse adjustment cells, NHES:2016 513
Table F-1. PFI nonresponse adjustment cells, NHES:2016 516
Table G-1. ATES nonresponse adjustment cells, NHES:2016 519
Table 1. Taking Notes When Coding: Examples 546
Table 2/Table 1. Taxonomy Summary 550