• ISSN: 2301-3567 (Print), 2972-3981 (Online)
    • Abbreviated Title: J. Econ. Bus. Manag.
    • Frequency: Quarterly
    • DOI: 10.18178/JOEBM
    • Editor-in-Chief: Prof. Eunjin Hwang
    • Executive Editor: Ms. Fiona Chu
    • Abstracting/ Indexing:  CNKI, Google Scholar, Crossref
    • Article Processing Charge (APC): 500 USD
    • E-mail: joebm.editor@gmail.com
HOME > Archive > 2026 > Volume 14 Number 2 (2026) >
JOEBM 2026 Vol.14(2): 91-96
DOI: 10.18178/joebm.2026.14.2.915

Evaluating User Experience of a Retrieval-Augmented Generation-Based Customer Service Chatbot

Syuan-Yu Chen and Chang-Yi Kao
Syuan-Yu Chen and Chang-Yi Kao
Department of Computer Science & Information Management, Soochow University, Taiwan
Email: 104102042g@gmail.com (S.Y.C.), edenkao@scu.edu.tw (C.Y.K.)

Manuscript received December 15, 2025; accepted January 29, 2026; published May 22, 2026.

Abstract—As digital services become increasingly widespread, online real-time customer support has become a key way for consumers to seek help. A good service experience can boost customer satisfaction and loyalty, particularly in the financial sector. However, traditional keyword-based chatbots often fail to capture users’ intent, limiting the scenarios in which they can be used. This study introduces a system based on Retrieval-Augmented Generation (RAG) and Large Language Models (LLMs), aiming to better understand user queries and provide more helpful responses. Through interviews and sentiment analysis, we explored user reactions and what influenced their emotional responses. Findings revealed that traditional bots often triggered frustration and confusion due to rigid responses and poor understanding. In contrast, the
RAG-based system demonstrated stronger natural language handling and was perceived as more empathetic. Some users even reported feeling emotionally supported. However, slow response times and difficulty in handling complex queries remained challenges. Based on these insights, we suggest improving response speed, adding voice interaction, and enhancing response reliability to guide future development of intelligent customer support tools.
 
Keywords—retrieval-augmented generation, intelligent customer service, chatbot, sentiment analysis

Cite: Syuan-Yu Chen and Chang-Yi Kao, "Evaluating User Experience of a Retrieval-Augmented Generation-Based Customer Service Chatbot," Journal of Economics, Business and Management, vol. 14, no. 2, pp. 91-96, 2026.

Copyright © 2026 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).


PREVIOUS PAPER
A Framework Detecting Risk of Geopolitical Competitions over Critical Minerals and a Case Study of Copper
NEXT PAPER
Structuring for Acceptance: How Cultural Legitimacy Drives Organizational Innovation in Theme Parks
Copyright © 2013-2026. Journal of Economics, Business and Management. Unless otherwise stated.
E-mail: joebm.editor@gmail.com