A Secure LLM for Enhanced Information Sharing

Objective

The primary purpose of the project was to develop an analog for ChatGPT by using an open-source language model framework. This venture provides employees with a very comfortable communication tool and at the same time keeps the confidentiality of the corporate data. The efficiency of our communication is improved; at the same time, the company is prepared to meet the new demands of the present-day business environment in a better way.

Сhallenges

Process

1. Choosing a Suitable Language Model (LLM):
   The process began with a review of open-source language models considering factors like performance, accuracy, and suitability for corporate use. Popular models such as BERT and GPT 2, among others, were evaluated to determine which one aligns best with the project objectives.
2. Fine-tuning using LoRA (Low-Rank Adaptation):
   After the base LLM is chosen, domain-tuning is the next step. This is accomplished via the expansion of the model to satisfy the needs of the application setting which include terminology, business needs, and other business characteristics.
3. Quantization:
   After fine-tuning, the quantization — an optimization technique that reduces the model’s memory footprint — was made. It was important since in an enterprise setting the computational resources might be very limited.
4. Enhancing the Retrieval-Augmented Generation (RAG) Approach:

   To optimize interactions enhancements were made to the RAG approach to methods:

   • Addressing questions: Enhancing response generation for queries.
   • Multiquering: Expanding capabilities to handle queries simultaneously.
   • Parent retrieval: Improving retrieving parent queries efficiently.
   • Hypothetical questions: Introducing questions to enrich content creation.
   • Keyword and topic extraction: Enhancing keyword and topic extraction procedures.
   • HyDE (Hybrid Data Enhancer): Employing a technique to enhance data quality.

   These measures not only tailored the model to fit requirements but also boosted its performance in query handling and content generation.

Results

As part of this initiative, the models Mixtral 8x7B and Mistral 7B were deployed locally. This process entailed installing and configuring these models on servers within the company’s infrastructure. Local deployment offers control over data processing. Delivers superior performance compared to cloud-based alternatives.

This approach enhances security by processing data, thus reducing risks associated with transmitting information through channels.

Deploying models locally provides flexibility for tailoring the system to suit the organization’s requirements, leading to improved efficiency in using advanced models in a corporate setting.

Now