Bridging the Gap: A Transformer-Based Chatbot Architecture for Low-Resource Brahui via Embedding Fusion and Transfer Learning

Abstract

Large language models have transformed conversational artificial intelligence; however, their performance remains weak for languages with extreme resource scarcity. Brahui is a typologically distinct Dravidian language with limited annotated data and minimal representation in multilingual resources. This study presents a transformer-based chatbot architecture designed for Brahui and introduces an embedding fusion mechanism to address data scarcity. The mechanism combines multilingual embeddings with Brahui-specific embeddings, enabling the model to retain cross-lingual knowledge while learning language-specific semantic and morphological features. Experiments show substantial improvement over the baseline: semantic similarity increases from 79.91 to 98.23, perplexity decreases from 94.27 to 14.64, and the system achieves 76% accuracy in correctly answering questions. The proposed framework provides a robust and parameter-efficient approach for developing fluent generative dialogue systems in severely under-resourced linguistic contexts.