Title page
Contents
ACKNOWLEDGMENTS 3
A. DESIGN OF THE EXPERIMENT 7
1. Overview of the experiment and its outcomes 7
2. Design of the information display and its variations 9
3. School information to display 13
B. METHODS 17
1. Bayesian analysis 17
a. Statistical model 17
b. Posterior probabilities 20
c. Subsample analyses 21
d. Sensitivity tests 22
2. Approach to measuring outcomes 22
a. Background on the survey items used for outcome measurement 22
b. Selection of outcome scales 24
3. Construction of outcome measures 24
a. Understanding 24
b. Usability 25
c. Effects on choices 28
4. Sample screening, survey protocol, and randomization 28
a. Screening criteria and survey integrity 28
b. Characteristics of the sample 31
c. Results from randomization 34
C. RESULTS 38
1. Impact results 38
a. Main effects 38
b. Subsamples 39
2. Sensitivity analyses 55
a. Alternative outcome measures 55
b. Alternative model specifications and sample definitions 57
REFERENCES 59
Table A.1. Factors and display strategies tested in the experiment 11
Table A.2. Variations in the school measures appearing in tested displays 14
Table B.1. Criteria for determining if subsample results were consistent with the full sample, for a given factor and outcome 22
Table B.2. Descriptive statistics for the understanding outcome 25
Table B.3. Items included in satisfaction and ease-of-use scales 26
Table B.4. Correlations, means, and standard deviations of usability items on a 0-1 scale 27
Table B.5. Descriptive statistics for usability measures 27
Table B.6. Fit of alternative factor structures 27
Table B.7. Flow of participants out of the sample 30
Table B.8. Demographic characteristics of the analytic sample 33
Table B.9. Income and prior choice exposure of urban and non-urban sample members 34
Table B.10. Parent demographic characteristics, by display strategy 36
Table B.11. Demographic characteristics of parent's youngest child, by display strategy 37
Table B.12. Community characteristics of parents, by display strategy 37
Table C.1. Impacts on understanding, ease of use, and satisfaction, for the full sample 41
Table C.2. Impacts on choices (characteristics of schools selected by parents), for the full sample 42
Table C.3. Predicted knowledge and attitudinal outcomes for best and worst factor combinations 43
Table C.4. Predicted effects on choice for best and worst factor combinations 45
Table C.5. Comparison of subsample results to results for the full sample 46
Table C.6. Impacts on understanding, ease of use, and satisfaction, for parents with school choice exposure 47
Table C.7. Impacts on choices (characteristics of schools selected by parents), for parents with school choice exposure 48
Table C.8. Impacts on understanding, ease of use, and satisfaction, for parents with lower education levels 49
Table C.9. Impacts on choices (characteristics of schools selected by parents), for parents with lower education levels 50
Table C.10. Impacts on understanding, ease of use, and satisfaction, for parents with lower income levels 51
Table C.11. Impacts on choices (characteristics of schools selected by parents), for parents with lower income levels 52
Table C.12. Impacts on understanding, ease of use, and satisfaction, for parents with non-intensive Internet use 53
Table C.13. Impacts on choices (characteristics of schools selected by parents), for parents with non-intensive Internet use 54
Table C.14. Comparison of effect sizes of original understanding, ease of use, satisfaction, and effects on choices outcomes to alternatives 56
Table C.15. Comparison of effect sizes for the understanding outcome to effect sizes with alternative model specifications and sample definitions 58
Figure A.1. How the study worked 9
Figure B.1. Model estimation traceplots for the understanding outcome 20
Figure B.2. Education level by screen-out status 30
Figure B.3. Number of participants from each state 31
Figure B.4. Attrition, by display strategy 35
Figure C.1. Most understandable display 43
Figure C.2. Display with the greatest predicted ease of use 44
Figure C.3. Display with the greatest predicted level of satisfaction 44
Figure C.4. Display that leads parents to select the highest academically-rated schools 45