It is important to know the differences among related groups-experts, residents live in old downtown and outside of downtown- and identify the relationship among decline factors for the old downtown regeneration project. This study identified the cognitive differences among participants and classified the decline impact factors using x² test, F-test and factor analysis. Also, we analyzed mutual relationship between decline factors and a negative image factor using SEM(Structural Equation Model). For this, we selected old downtown of Daejeon city, Korea, as a case study area which is ongoing decline. We collected data through interview on participants related with the regeneration project. The results show that there are statistical difference on decline factors by visit purpose, number of visit and related group of project. Also, the decline factors are classified as five such as economic activity, education and welfare facilities, ageing of appearance, social problem and easiness of life. Lastly, SEM shows that economic factor and social factor(rate of crime, rate of kids with no parents) impacted on the making of negative slum image of old downtown regardless of the real facts.