Title page
Contents
Acknowledgments 2
Executive Summary 4
Introduction 6
PART I: BACKGROUND 8
Cost and Access to AI Models 9
Implications for Policymakers 10
Understanding Cost and Compute Growth 11
Related Work 12
Current Best Estimates and Assumptions 15
PART II: ANALYSIS 18
Cost and Compute Projections 19
Limits on Cost Growth 22
Limits on Hardware Improvements 24
Proliferation 28
Costs for Hardware-Restricted Actors 29
Compute Regulatory Threshold 33
Conclusion 36
Appendix A: Additional Limitations on Compute Growth 38
Appendix B: Observed Growth Rates 40
Appendix C: Estimating Compute Costs 41
Appendix D: Uncertainty in Cost Projections 42
Appendix E: Tapered Cost Growth Projections 44
Selected Bibliography 45
TABLE 1.1. TRAINING COST AND COMPUTE OVER TIME (STRAIGHTFORWARD PROJECTION OF CURRENT TRENDS) 21
TABLE 1.2. TRAINING COST AND EFFECTIVE COMPUTE OVER TIME (STRAIGHTFORWARD PROJECTION OF CURRENT TRENDS) 21
TABLE 2.1. FRONTIER MODEL TRAINING COST AND COMPUTE OVER TIME (ASSUMING HARDWARE LIMITS AROUND 2031 AND TAPERED COST... 27
TABLE 2.2. FRONTIER MODEL TRAINING COST AND EFFECTIVE COMPUTE OVER TIME (ASSUMING HARDWARE LIMITS AROUND 2031 AND... 27
FIGURE 1.1. COMPUTE GROWS DUE TO INCREASE SPENDING AND HARDWARE IMPROVEMENTS 13
FIGURE 1.2. PERFORMANCE IMPROVES DUE TO INCREASED COMPUTE AND ALGORITHMIC IMPROVEMENTS 13
FIGURE 2.1. THE COST TO TRAIN A FRONTIER AI MODEL RISES OVER TIME (STRAIGHTFORWARD PROJECTION OF CURRENT TRENDS) 19
FIGURE 2.2. COMPUTE USED TO TRAIN A FRONTIER AI MODEL RISES OVER TIME (STRAIGHTFORWARD PROJECTION OF CURRENT TRENDS) 20
FIGURE 2.3. EFFECTIVE COMPUTE RISES OVER TIME (STRAIGHTFORWARD PROJECTION OF CURRENT TRENDS) 20
FIGURE 3.1. FRONTIER MODEL TRAINING COSTS (ARBITRARY TAPERED COST GROWTH PROJECTION) 23
FIGURE 3.2. TRAINING COMPUTE (TAPERED COST GROWTH PROJECTION) 23
FIGURE 4.1. COMPUTE GROWTH DUE TO HARDWARE IMPROVEMENTS ALONE (HARDWARE PERFORMANCE LIMIT AROUND 2031) 26
FIGURE 4.2. FRONTIER MODEL TRAINING COSTS (ARBITRARY TAPERED COST GROWTH PROJECTION) 26
FIGURE 4.3. TRAINING COMPUTE (TAPERED COST PROJECTION AND HARDWARE LIMITS AROUND 2031) 26
FIGURE 4.4. EFFECTIVE COMPUTE (TAPERED COST PROJECTION AND HARDWARE LIMITS AROUND 2031) 26
FIGURE 5.1/FIGURE 5. CAPABILITIES OF ONCE-FRONTIER MODELS BECOME MORE ACCESSIBLE OVER TIME AS COSTS DECREASE 28
FIGURE 6.1. RESTRICTIONS ON HARDWARE MAKE IT UNAFFORDABLE TO KEEP PACE WITH FRONTIER MODELS 31
FIGURE 6.2. CAPABILITIES PROLIFERATE TO HARDWARE-RESTRICTED ACTORS DUE TO ALGORITHMIC IMPROVEMENTS 32
FIGURE 7.1. THE COST TO TRAIN A 1026 FLOP REGULATORY THRESHOLD MODEL DECREASES OVER TIME 34
FIGURE 7.2. THE COST TO TRAIN A MODEL WITH CAPABILITIES EQUIVALENT TO A 1026 FLOP MODEL IN 2024 DECREASES OVER TIME 34
TABLE A.1. OBSERVED RATES OF GROWTH IN RELEVANT METRICS 40
TABLE A.2. MEDIAN, HIGH, AND LOW ESTIMATES FOR TRAINING COST OVER TIME IN 2022 DOLLARS (STRAIGHTFORWARD PROJECTION) 43
TABLE A.3. MEDIAN, HIGH, AND LOW ESTIMATES FOR TRAINING COMPUTE OVER TIME IN FLOP (STRAIGHTFORWARD PROJECTION) 43
TABLE A.4. YEAR TO REACH TRAINING COST MILESTONE FOR MEDIAN, HIGH, AND LOW ESTIMATES FOR TRAINING COST GROWTH... 43
TABLE A.5. YEAR TO REACH TRAINING COMPUTE MILESTONE FOR MEDIAN, HIGH, AND LOW ESTIMATES FOR TRAINING COMPUTE GROWTH... 43
TABLE A.6. MODEL TRAINING COST AND COST DOUBLING RATE UNDER VARIOUS SCENARIOS OF TAPERED COST GROWTH 44
FIGURE A.1. FRONTIER MODEL TRAINING COST OVER TIME (SENSITIVITY ANALYSIS) 42
FIGURE A.2. FRONTIER MODEL TRAINING COST OVER TIME (TAPERED COST GROWTH PROJECTIONS) 44