Title page
Contents
ACKNOWLEDGMENTS 4
SUMMARY 8
I. INTRODUCTION 9
II. BACKGROUND ON STATES' TQRIS 12
III. COMPONENTS OF VALIDATION STUDIES 17
IV. DATA AND METHODS 19
V. APPROACHES FOR STATES' VALIDATION STUDIES 21
VI. ASSOCIATIONS BETWEEN RATINGS AND INDEPENDENT MEASURES OF PROGRAM QUALITY 25
VII. ASSOCIATIONS BETWEEN RATINGS AND CHILDREN'S DEVELOPMENT OUTCOMES 27
VIII. CHALLENGES IN CONDUCTING VALIDATION STUDIES 34
IX. DISCUSSION 36
REFERENCES 38
APPENDIX A. OUTCOME DOMAINS AND MEASURES 43
APPENDIX B. APPROACH TO SYNTHESIZING RESULTS OF VALIDATION STUDIES 48
APPENDIX C. DETAILED STATE-SPECIFIC FINDINGS 54
APPENDIX D. DETAILED FINDINGS ABOUT CHALLENGES 80
Table II.1. TQRIS characteristics, by state 15
Table V.1. Summary of approaches states used to examine measures of program quality and children's outcomes 24
Table VIII.1. Common challenges reported by researchers who conducted validation studies 34
Figure II.1. Rating structures in TQRIS 14
Figure VI.1. Difference in program quality between higher- and lower-rated programs 26
Figure VII.1. Differences in child development outcomes between children attending higher- and lower-rated programs, by domain 28
Figure VII.2. Difference in child development outcomes between children attending higher- and lower-rated programs, by state 30
Figure VII.3. Differences in child development outcomes between low-income children attending higher- and lower-rated programs, by domain 31
Boxes
Box 1. RTT-ELC's five TQRIS objectives 9
Box 2. Data from state validation studies and interviews with researchers 19
Box 3. Methods for synthesizing results of validation studies 20
Table A.1. Outcome domains used for establishing validity 45
Table A.2. Outcome measures used for establishing validity 46
Table A.3. Common program quality outcome measures 47
Table B.1. Contrasts for supplemental analysis of highest and lowest possible ratings that could be examined in each state 53
Table B.2. Findings from the supplemental analysis of highest and lowest possible ratings that could be examined in each state 53
Table C.1. California findings for child development outcomes 56
Table C.2. California findings for program quality outcomes 56
Table C.3. Delaware findings for child development outcomes 58
Table C.4. Delaware findings for child development outcomes for low-income children 59
Table C.5. Delaware findings for program quality outcomes 60
Table C.6. Massachusetts findings for child development outcomes 62
Table C.7. Massachusetts findings for program quality outcomes 62
Table C.8. Minnesota findings for child development outcomes 64
Table C.9. Minnesota findings for child development outcomes for low-income children 65
Table C.10. Minnesota findings for program quality outcomes 66
Table C.11. Ohio findings for child development outcomes 67
Table C.12. Ohio findings for program quality outcomes 68
Table C.13. Oregon findings for program quality outcomes 69
Table C.14. Rhode Island findings for child development outcomes 71
Table C.15. Rhode Island findings for child development outcomes for low-income children 72
Table C.16. Rhode Island findings for program quality outcomes 73
Table C.17. Washington findings for child development outcomes 75
Table C.18. Wisconsin findings for child development outcomes 77
Table C.19. Wisconsin findings for child development outcomes for low-income children 78
Table C.20. Wisconsin findings for program quality outcomes 79
Table D.1. Detailed findings about challenges 81