Objective: This study was conducted for the purpose of proposing a method to evaluate user experience for a system using a multi-criteria decision making model based on the evaluation factors of user experience.
Background: User experience refers to a direct or indirect experience that a user obtains in the process of using a product or service. As various digital products and services become popular and competition intensifies in modern life, user experience is becoming more and more important. Various studies have been attempted to evaluate user experience according to such necessity. Since the user experience that an individual has for the same system can be different, it is important how the user experience evaluation of the system incorporates this subjectivity.
Method: In this study, a method to obtain the user experience score was proposed by using importance weights of the user experience evaluation criteria and averaging the scores for each evaluation criterion of the system. To this end, we first determined the criteria for evaluating user experience based on the honeycomb model. AHP was used as a method to derive the relative importance weight of evaluation criteria. In addition, a Likert scale from 1 to 5 was used to give a rating for the user experience evaluation criteria.
Results: In order to examine the applicability of the method proposed in this study, it was applied to the user experience evaluation problem for car rental websites.
In this study, 'useful, usable, findable, credible, desirable, accessible, and valuable' presented in the honeycomb model were used as user experience evaluation criteria.
Both the UX score and the SUS score showed a statistically significant difference at the significance level of 0.05 for each of the three car rental websites. A similar pattern was also observed in the average score. And the UX score showed a significant correlation with the SUS score of 0.509.
Conclusion: User experience evaluation using the multi-criteria decision-making method showed a high correlation with SUS, which has been used as an effective usability evaluation method in many studies. From this fact, it can be seen that the user experience evaluation method proposed in this study can be effectively applied to the user experience evaluation problem of a system.
Application: The results of this study could be used as a method to evaluate user experience on a digital system.