PaperlayerView exact revision on arXiv

Source-linked AI summary

Automating Customer Service using LangChain: Building custom open-source GPT Chatbot for organizations

Keivalya Pandya, Mehfuza Holia

arXiv:2310.05421v1cs.CLcs.CYcs.LG

TL;DR

The paper argues that traditional support methods, particularly FAQs, are becoming obsolete, and presents Sahaay, an open-source chatbot framework using organizational website information and conversational context. In the reported task, Flan T5 XXL outperforms the Base and Small models; the authors describe the approach as supporting efficient, personalized customer service.

  • Problem

    The paper argues that traditional customer-support methods, including FAQs, are becoming obsolete.

    Source: §I · p. 1

  • Method

    Sahaay is an open-source chatbot approach that scrapes organizational website information and uses Flan T5 XXL with prior conversational context to answer queries.

    Source: §I · p. 1, A. Data Collection · p. 2, C. Language Model · p. 2

  • Results

    The Flan T5 XXL model outperforms the Base and Small models on the provided task when the models are posed the same questions.

    Source: A. Evaluating the Performance of LLMs · p. 2, §V · p. 4

  • Takeaways & Limitations

    The authors present Sahaay as supporting efficient, personalized, and responsive service, with customer retention, customer value, and brand image as intended benefits.

    Source: §V · pp. 3–4

Abstract

from arXiv · show

In the digital age, the dynamics of customer service are evolving, driven by technological advancements and the integration of Large Language Models (LLMs). This research paper introduces a groundbreaking approach to automating customer service using LangChain, a custom LLM tailored for organizations. The paper explores the obsolescence of traditional customer support techniques, particularly Frequently Asked Questions (FAQs), and proposes a paradigm shift towards responsive, context-aware, and personalized customer interactions. The heart of this innovation lies in the fusion of open-source methodologies, web scraping, fine-tuning, and the seamless integration of LangChain into customer service platforms. This open-source state-of-the-art framework, presented as "Sahaay," demonstrates the ability to scale across industries and organizations, offering real-time support and query resolution. Key elements of this research encompass data collection via web scraping, the role of embeddings, the utilization of Google's Flan T5 XXL, Base and Small language models for knowledge retrieval, and the integration of the chatbot into customer service platforms. The results section provides insights into their performance and use cases, here particularly within an educational institution. This research heralds a new era in customer service, where technology is harnessed to create efficient, personalized, and responsive interactions. Sahaay, powered by LangChain, redefines the customer-company relationship, elevating customer retention, value extraction, and brand image. As organizations embrace LLMs, customer service becomes a dynamic and customer-centric ecosystem.

I. INTRODUCTION

The paper frames customer service as a frontline of business engagement and argues that traditional support methods are becoming obsolete. It proposes an open-source framework for scalable, rapid query resolution, demonstrated with information from an engineering college website.

I. INTRODUCTION

  • Customer service is described as the frontline where businesses answer queries, resolve problems, and build trust.Source: §I · p. 1
  • The paper presents LangChain as an example of integrating LLMs into customer service to support automation, personalization, and responsiveness.Source: §I · p. 1
  • The authors describe traditional customer-support methods as gradually becoming obsolete as customer-service automation advances.The introduction discusses FAQs and traditional support mechanisms as established repositories for customer queries.Source: §I · p. 1
  • The proposed open-source framework is intended to scale across industries and organizations and resolve support queries within seconds.Source: §I · p. 1
  • The demonstration uses Birla Vishvakarma Mahavidyalaya’s website as chatbot context for prospective students, current students, and researchers seeking information.Source: §I · p. 1

II. LITERATURE SURVEY

The paper proposes an open-source chatbot approach that uses custom website knowledge and is intended to complement FAQs with a more interactive interface. It also reports a comparative study of models against expected responses.

II. LITERATURE SURVEY

  • Prior work reports distilling dual-encoder and cross-encoder models to asymmetric students one-tenth their size while retaining 95–97% of teacher performance.Another cited work describes MS MARCO’s role in enabling zero-shot transfer across tasks.Source: §II · p. 1
  • The paper characterizes its contribution as an open-source approach to building chatbots from custom website knowledge.Source: §II · p. 1
  • The paper proposes an open-source chatbot framework designed to handle datasets available on webpages or linked pages.Source: §II · p. 1
  • The proposed interface is intended to complement FAQs with a more interactive and user-friendly experience.Source: §II · p. 1
  • The paper compares model performance on provided data relative to expected LLM responses.Source: §II · p. 1

III. METHODOLOGY

The methodology section covers data collection, model selection, fine-tuning, and integration with Gradio APIs for web deployment.

III. METHODOLOGY

  • The methodology covers data collection as part of the project workflow.The passage lists data collection alongside model selection, fine-tuning, and web deployment.Source: §III · p. 2
  • The methodology includes selecting a model and fine-tuning it.Source: §III · p. 2
  • The chatbot is integrated with Gradio APIs for web deployment.Source: §III · p. 2

A. Data Collection

The project scrapes publicly accessible organizational homepage information with BeautifulSoup, collecting material that serves as context for the LLM.

A. Data Collection

  • BeautifulSoup web scraping retrieves publicly accessible information from an organization’s homepage.The authors note that the homepage is often linked to relevant information for users or visitors.Source: A. Data Collection · p. 2
  • The collected material includes FAQs, product manuals, support forums, chat logs, and associated-institution information.Source: A. Data Collection · p. 2
  • The scraped data serves as context for the LLM.Source: A. Data Collection · p. 2

B. Embeddings

The section describes embeddings as semantic and contextual vector representations and identifies the embedding model and vector-search library used in Sahaay.

B. Embeddings

  • Embeddings represent words or phrases as vectors that capture semantic and contextual information for responding meaningfully to queries.The passage also describes them as helping the model understand relationships between words.Source: B. Embeddings · p. 2
  • Sahaay uses HuggingFace Instructor Embeddings, “hkunlp/instructor-large,” for tasks including retrieval, without additional fine-tuning.The passage also lists classification, clustering, and text evaluation as suitable tasks.Source: B. Embeddings · p. 2
  • The resulting embeddings are stored with FAISS, which supports searching for similar multimedia-document embeddings.Source: B. Embeddings · p. 2

C. Language Model

The paper selects Flan-T5-XXL to retrieve knowledge from the vector space and chat history, then demonstrates a Gradio interface for user interaction.

C. Language Model

  • Flan-T5-XXL is selected over other Flan-T5 distributions to retrieve knowledge from the vector space and chat history.Source: C. Language Model · p. 2
  • Using previous messages as context, the model predicts answers to subsequent questions to support interactive conversation.Source: C. Language Model · p. 2

D. Integration with Customer Service Platforms

IV. RESULTS

The results section presents comparison metrics, comparative analysis, and use cases associated with an educational institution.

IV. RESULTS

  • The section presents metrics for comparing the models.Source: §IV · p. 2
  • The section provides a comparative analysis of model performance.Source: §IV · p. 2
  • The reported use cases are associated with an educational institution.Source: §IV · p. 2

A. Evaluating the Performance of LLMs

The section reports that Flan-T5-XXL outperforms BASE and SMALL, while noting that evaluating LangChain agents on large retrieval contexts is difficult.

A. Evaluating the Performance of LLMs

B. Applications

The paper presents LLM-powered customer service as a way to provide context-aware, personalized support, with a Gradio interface used for the demonstration.

B. Applications

  • Fine-tuning LangChain on organizational knowledge and using conversation context can support personalized responses to customer inquiries.The paper says the model can use conversation history and operate around the clock in multiple languages.Source: B. Applications · p. 3
  • The demonstration interface is identified as using the Gradio framework.Source: Figure 2 · p. 3

V. CONCLUSION

On the same questions, GOOGLE/FLAN-T5-XXL outperforms the Base and Small models in response quality. The conclusion presents Sahaay as an open-source LangChain-based approach to customer support, with multimodal retrieval described as future potential.

V. CONCLUSION

  • Sahaay’s open-source Large Language architecture leveraging LangChain is presented as enabling efficient, personalized, responsive support and benefits including retention, customer value, and brand image.Source: §V · pp. 3–4
  • GOOGLE/FLAN-T5-XXL outperforms GOOGLE/FLAN-T5-BASE and GOOGLE/FLAN-T5-SMALL on response quality when all models receive the same questions.Source: §V · p. 4
  • Future Sahaay development could access PDFs, videos, audio, and other files to retrieve information about student activities, research, and innovation at BVM.The passage describes this multimodal capability as having the potential to change website interaction and information retrieval time.Source: §V · p. 4
Loading 2310.05421v1…