AI Based ChatBot | Clara

As part of the "AI Based ChatBot | Clara" project, b-nova supported Helvetia Insurance Switzerland in the conception and technical implementation of an AI-powered assistant system based on Large Language Models (LLMs). Using Azure OpenAI, LangChain, and Retrieval Augmented Generation (RAG), a chatbot was developed that understands domain-specific queries in context, retrieves relevant information from a knowledge base, and generates precise answers. In addition to the architecture and implementation of the RAG pipeline, we guided testing, quality assurance, prompt engineering, and the iterative optimization of user interaction through to production.

Biggest challenge

Context-aware answering of domain-specific queries through LLM-powered retrieval architecture with consistent response quality

What we did

Architecture and implementation of a RAG-based AI chatbot with Azure OpenAI, LangChain, and vector database including integration, testing, and operational support

Main tools we used

Azure OpenAI, LangChain, LangGraph, RAG, Vector DB, Python, OpenShift

Tasks

Analysis of business requirements and close coordination with stakeholders to define chatbot goals and expectations

Identification and prioritization of relevant use cases for AI-powered customer dialogue

Design of the overall architecture including RAG pipeline, LLM integration, and retrieval strategy

Setup and maintenance of the knowledge base with embedding creation and integration of a vector database for semantic search

Systematic prompt engineering to optimize dialogue quality, tone of voice, and domain accuracy

Design and implementation of dialogue flows and response structures considering various user intents

Development of chatbot agents using LangChain and LangGraph for multi-step conversational logic and tool orchestration

Integration of Azure OpenAI Service as the central LLM backend for text generation and embedding creation

Connection to existing backend and third-party systems via REST APIs for real-time data retrieval during conversations

Implementation of authentication and authorization using OIDC for secure user interactions

Setup of automated testing and evaluation processes for continuous measurement of response quality

Introduction of guardrails and content filtering to ensure domain-accurate and brand-compliant responses

Containerization and deployment of the solution on OpenShift for scalable and highly available provisioning

Setup and maintenance of CI/CD pipelines with GitHub Actions for automated builds, tests, and deployments

Implementation of monitoring, logging, and observability to track LLM performance and usage patterns

Iterative improvement of response quality and user experience based on feedback and usage analytics

Comprehensive documentation of architecture, operational processes, and knowledge transfer to internal teams

Technologies

Azure OpenAI Service LangChain / LangGraph Retrieval Augmented Generation (RAG) Vector Databases (Embeddings) Python Prompt Engineering REST APIs / FastAPI OpenShift GitHub Actions Redis / Caching OIDC / Authentication Monitoring / Observability