Homepage > Portfolio > A Secure LLM for Enhanced Information Sharing

A Secure LLM for Enhanced Information Sharing

Azati successfully developed a local alternative to ChatGPT, effectively addressing resource challenges and bolstering data security. The implementation marks a significant advancement in elevating standards for corporate communication within the organization.


The artificial intelligence technology advancements, combined with the need for communication in organizations, inspires us to develop a modern and accessible platform for information exchange. There is a need for a ChatGPT equivalent for our company because it will create a solution that greatly improves our company’s communication methods and data security.


The primary purpose of the project was to develop an analog for ChatGPT by using an open-source language model framework. This venture provides employees with a very comfortable communication tool and at the same time keeps the confidentiality of the corporate data. The efficiency of our communication is improved; at the same time, the company is prepared to meet the new demands of the present-day business environment in a better way.



Challenge#1. High computational resource requirements:


A key challenge encountered in developing an alternative to GPT was meeting the computational resource demands.

Developing language models, like OpenAI GPT’s, demands an amount of computational power for training and optimal performance. Securing computing resources can pose a challenge for businesses operating on tight budgets, which may impede the smooth execution of the project.

Challenge#2. Identifying optimal models comparable in quality to OpenAI GPT’s:


It required research and testing of open-source LLMs, not just focusing on text generation quality but also on understanding context and user interaction in a corporate setting. This process could take some time due to the need to compare models.


Challenge#3. Ensuring the security of corporate data:


Creating a secure analog to the GPT also raised concerns about the security of corporate data. Considering the sensitivity of information exchanged in a corporate setting, mechanisms for data encryption and other security measures needed to be developed. Ensuring the security of employee communication and protecting corporate secrets became an integral part of the process of creating such a system.


  1. Choosing a Suitable Language Model (LLM):
    The process began with a review of open-source language models considering factors like performance, accuracy, and suitability for corporate use. Popular models such as BERT and GPT 2, among others, were evaluated to determine which one aligns best with the project objectives.
  2. Fine-tuning using LoRA (Low-Rank Adaptation):
    After the base LLM is chosen, domain-tuning is the next step. This is accomplished via the expansion of the model to satisfy the needs of the application setting which include terminology, business needs, and other business characteristics.
  3. Quantization:
    After fine-tuning, the quantization — an optimization technique that reduces the model’s memory footprint — was made. It was important since in an enterprise setting the computational resources might be very limited.
  4. Enhancing the Retrieval-Augmented Generation (RAG) Approach:

    To optimize interactions enhancements were made to the RAG approach to methods:

    • Addressing questions: Enhancing response generation for queries.
    • Multiquering: Expanding capabilities to handle queries simultaneously.
    • Parent retrieval: Improving retrieving parent queries efficiently.
    • Hypothetical questions: Introducing questions to enrich content creation.
    • Keyword and topic extraction: Enhancing keyword and topic extraction procedures.
    • HyDE (Hybrid Data Enhancer): Employing a technique to enhance data quality.

    These measures not only tailored the model to fit requirements but also boosted its performance in query handling and content generation.


The solution to the task involved the following key steps:

Development of an Independent Internal Service:
By choosing an LLM and using techniques, like tuning, quantization, and an enhanced RAG method, an independent internal service was successfully developed. This platform offered a smart way of sharing information designed to meet the needs of the business world.

Ensuring the Security of Corporate Data:
To safeguard information, security protocols were established across different tiers. These measures encompassed encryption techniques, stringent access controls, and additional technologies aimed at guaranteeing defense against data breaches.

Integration with Confidential Data:
In the end, the developed service was smoothly connected with the organization’s databases allowing easy data exchange among staff members. The main goal in the meantime was to customize the service according to the organization’s data structures and specific needs.

Testing and Performance Evaluation:
Following the development and integration of the service, various tests were carried out to evaluate its performance and efficiency. These assessments included real-world usage scenarios, analysis of response times, and validation of adherence to security protocols.

Employee Training and Implementation:
To ensure an implementation of the service, comprehensive training sessions were conducted for employees. This involved getting acquainted with the interface, understanding its functionalities, and receiving guidance on usage practices.

Because of these steps, the successful creation and implementation of an independent internal service were achieved, completely replacing ChatGPT and ensuring the secure handling of confidential corporate data.


As part of this initiative, the models Mixtral 8x7B and Mistral 7B were deployed locally. This process entailed installing and configuring these models on servers within the company’s infrastructure. Local deployment offers control over data processing. Delivers superior performance compared to cloud-based alternatives.

This approach enhances security by processing data, thus reducing risks associated with transmitting information through channels.

Deploying models locally provides flexibility for tailoring the system to suit the organization’s requirements, leading to improved efficiency in using advanced models in a corporate setting.


Besides expanding the system’s functionality, there are plans to create a universal assistant with access to all accumulated knowledge within the company.

Drop us a line

If you are interested in the development of a custom solution — send us the message and we'll schedule a talk about it.