Title page
Contents
1. Overview of the Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS) 9
1.1. Introduction 10
1.2. PISA 2012 10
1.3. What PISA YAFS Measures 11
1.4. PISA YAFS Administration 13
1.5. Organization of This Document 16
2. Sampling 17
2.1. Overview of the PISA and PISA YAFS Sample 17
2.2. Target Population 17
2.3. Respondent Definition 18
3. Data Collection Activities 19
3.1. Data Collection Instruments 19
3.1.1. Cognitive Instruments 20
3.1.2. Noncognitive Instruments 21
3.2. Data Collection Activities 22
3.2.1. Tracing and Tracking 22
3.2.2. Recruitment 24
3.2.3. Data Collection 25
3.3. Quality Control 27
4. Weighting and Nonresponse Adjustment 29
4.1. Justification 29
4.2. Response Propensity Scores 29
4.3. Nonresponse Adjusted Weights 30
5. Nonresponse Bias Analysis 31
5.1. Introduction 31
5.2. Methodology 31
5.3. Participating Sample 35
5.3.1. Categorical Variables 35
5.3.2. Continuous Variables 40
5.3.3. Logistic Regression Model 41
5.4. Nonresponse Adjusted Sample 43
5.4.1. Categorical Variables 43
5.4.2. Continuous Variables 47
5.5. Summary 47
6. Data Analysis and Scaling 49
6.1. Data Handling and Data Quantity for ESO-modified Items 50
6.1.1. Handling of Missing Data 50
6.1.2. Data Quantity 51
6.1.3. Data Completion - Treatment of Respondents With Fewer Than 5 Cognitive Item Responses 51
6.1.4. Classical Test Theory Statistics: Item Analyses 51
6.2. IRT Scaling: Evaluation and Estimation of Item Parameters 52
6.3. Population Modeling and Generating Plausible Values 55
7. Data Management 57
7.1. Data Cleaning and Validation 57
7.2. Reducing the Risk of Data Disclosure 57
8. The PISA-YAFS 2016 Data 59
8.1. PISA YAFS Dataset 59
8.2. Variable Names 60
8.3. Derived Variables 60
8.3.1. Skill Use 60
8.3.2. Career Interest and Intentionality Items and Related Indices 62
8.3.3. Subjective Well-Being and Health 64
8.3.4. Adapted Variable 67
8.4. Missing Value Codes 68
8.5. Special Considerations 68
References 71
Appendix A. Learning Experience Questionnaire (LEQ) 73
Appendix B. ESO Noncognitive Items 77
Appendix C. List of Variables for Population Modeling 86
Appendix D. PISA YAFS 2016 Recruiting Materials 89
Table 1. Contact forms returned from PISA 2012: PISA YAFS 2016 13
Table 2. Overall summary of PISA YAFS 2016 tracing effort 23
Table 3. Status of sampled participants selected for the PISA YAFS field test: 2016 26
Table 4. Status of sampled participants selected for the PISA YAFS main study: 2016 27
Table 5. Percentage distribution of eligible and participating young adults in the PISA YAFS participating sample, by derived race/ethnicity, gender, attended preschool or kindergarten, and attended... 37
Table 6. Percentage distribution of eligible and participating young adults in the PISA YAFS participating sample, by truancy and highest grade expected to complete: 2016 38
Table 7. Percentage distribution of eligible and participating young adults in the PISA YAFS participating sample, by highest level of parental educational attainment, father's current job status,... 39
Table 8. Mean scores of plausible value 1 for eligible and participating young adults in the PISA YAFS participating sample: 2016 40
Table 9. Mean values of openness for problem solving for eligible and participating young adults in the PISA YAFS participating sample: 2016 40
Table 10. Logistic regression model parameters using the PISA YAFS young adult sample: 2016 42
Table 11. Percentage distribution of eligible and participating young adults in the PISA YAFS nonresponse-adjusted sample, by derived race/ethnicity, gender, attended preschool or kindergarten,... 44
Table 12. Percentage distribution of eligible and participating young adults in the PISA YAFS nonresponse-adjusted sample, by truancy and highest grade expected to complete: 2016 45
Table 13. Percentage distribution of eligible and participating young adults in the PISA YAFS nonresponse-adjusted sample, by highest level of parental educational attainment, father's current job... 46
Table 14. Mean scores of plausible values for eligible and participating young adults in the PISA YAFS nonresponse-adjusted sample: 2016 47
Table 15. Mean values of openness for problem solving for eligible and participating young adults in the YAFS nonresponse-adjusted sample: 2016 47
Table 16. Items with insufficient number of responses for the IRT parameters evaluation: PISA YAFS 2016 54
Table 17. Interest area description 62
Table 18. Career Intentionality score cutoffs 64
Exhibits
Exhibit 1. Components of PISA YAFS 12
Exhibit 2. Timeline of contact activities and administration of PISA YAFS 2016 15
Table A-1. Learning Experience Questions (LEQ) - Additional Background Questions 73
Table B-1. ESO Noncognitive Battery - Core Background Questionnaire 77
Table B-2. ESO Noncognitive Battery - Career Interest and Intentionality (CII) 79
Table B-3. ESO Noncognitive Battery - Subjective Well-Being and Health (SWBH) 82
Table B-4. ESO Noncognitive Battery - Skill Use (WST) 84
Appendix Exhibits
Exhibit D-1. Text of PISA Young Adult Follow-up Study NCES initial information letter 89
Exhibit D-2. Text of PISA Young Adult Follow-up Study NCES contact information update form 90
Exhibit D-3. PISA Young Adult Follow-up Study information flyer 91
Exhibit D-4. Text of PISA Young Adult Follow-up Study NCES participant website registration letter 92
Exhibit D-5. Text of PISA Young Adult Follow-up Study participant website registration confirmation 93
Exhibit D-6. Text of PISA Young Adult Follow-up Study NCES primer e-mail: pilot and main study 94
Exhibit D-7. PISA Young Adult Follow-up Study NCES invitation letter 95
Exhibit D-8. PISA Young Adult Follow-up Study respondent reminder 96
Exhibit D-9. PISA Young Adult Follow-up Study announcement that survey is open 97
Exhibit D-10. PISA Young Adult Follow-up Study nonresponse e-mail/hardcopy - credentials sent 98
Exhibit D-11. PISA Young Adult Follow-up Study partial complete 99
Exhibit D-12. PISA Young Adult Follow-up Study notification of survey closing 100
Exhibit D-13. PISA Young Adult Follow-up Study thank you e-mail for completing the survey 101