Drop us a line
If you are interested in the development of a custom solution — send us the message and we'll schedule a talk about it.
Azati Software
Recommendation system for banking industry
Azati helped a well-known banking company develop an automated system able to calculate and analyze financial metrics to build recommendations for an employee on how to improve their efficiency.
Customer
Today, recommendation systems are ubiquitous: marketing, finance, insurance, bioinformatics, and other business industries.
Initially, recommendation systems were used primarily to attract an external client and increase the company’s profits. But now it’s actively used for the internal client — employees. For example, training courses recommendations for staff development or even the performance analysis.
A banking employee faces a huge number of different indicators: sales growth, current ratio, transfers, and much more, depending on the particular department specifics. Thus, it is sometimes difficult for a manager to track down weaknesses and help staff enhance the result.
The customer turned to Azati with the idea of automating the analysis of incoming data from indicators and metrics and, based on this, to form proper recommendations that will help employees upgrade their workflow.
Objective
Recommendation systems are a great example of a successful IT tool for understanding personal achievement based on the available initial information.
Azati’s main goal on the project was to develop a server API to calculate metrics and build recommendations for banking staff, setting it up and creating a DAG Airflow.
Briefly about Airflow: it is a library for developing, scheduling and monitoring workflows. Data processing, or pipelines, are described using DAG (Directed Acyclic Graph). This is a semantic association of tasks that must be completed in a strictly defined sequence according to a specified schedule.
Challenges
The key issue we faced is the data variability. Typically, there are hundreds of thousands of users and objects with different properties. For high-quality recommendations, it is necessary to take all of them into account.
Our end solution involves the continuous analysis of over 500 recommendations that are built based on over 1000 metrics (e.g. gross margin, net burn, net profit, revenue per employee).
The system is shared between several company’s departments using specific data, so such metrics diversity and different performance evaluation criteria created some difficulties in the accuracy of calculations and subsequent analysis.
However, our team successfully coped with this task, accurately working under each criteria.
Process
To build a cutting-edge recommendation system, it was necessary to go through 5 main stages.
1. Understand the Business
An extremely obvious and important first step is defining the primary goals and key parameters. The step certainly includes discussions and data entry between both the data team and business groups (product managers, operations teams, even affiliate or advertising teams, depending on your product).
Here, we answer paramount questions:
- What is the end goal of the product?
- What metrics should be used to build recommendations?
- How to visualize recommendations to users?
- Do all metrics have the same priority?
2. Data collection
The best recommendation systems use terabytes of data. So when it comes to data collection — the more the better.
Here it is important to understand what kind of incoming data we have and how related recommendations can be calculated based on them.
3. Explore, Clean, and Augment the Data
Consider only looking at features that are more likely to represent the employee current results and removing older data that might no longer be relevant or adding a weight factor to give more importance to recent actions compared to older ones.
4. Rank and recommend
Further, the metrics are read, scores are calculated and recommendations are made according to certain indicators.
5. Visualize the Data
Here we need to clearly provide the end result to the user, in a form that can easily and accurately reflect the main problem areas, as well as suggest ways to improve.
Solution
Our team developed from scratch a server-side API for calculating metrics and making recommendations to bank employees.
To create such a system, the Azati team used Airflow.
We can define a workflow as any sequence of steps you take to achieve a specific goal. Airflow using Directed Acyclic Graphs (DAGs), which comprise of tasks to be performed along with their associated dependencies, which is quite suitable for performance estimation.
This robust and scalable workflow planning platform has four main components:
- Scheduler: Scheduler keeps track of all DAGs and their associated tasks;
- Web Server: The Web Server is the user interface of Airflow. It shows the tasks status and allows the user to configure DAGs;
- Database: The state of the DAGs and their associated tasks is stored in the database so that the schedule remembers the metadata information;
- Performer: The performer defines how the work will be done. There are different types of executors that can be used for different use cases.
Result
Our team successfully created the system from scratch and developed it from the initial model to end implementation.
Technologies
Now
At the moment, our solution has gone into production and is successfully functioning and being used by employees and managers of the customer’s company. In the future, we plan to integrate with Hadoop and enhance the system.
Featured case studies:
JavaScript, Ruby
HR Planning SoftwareThe customer asked Azati to audit the existing solution in terms of general performance to create a roadmap of future improvements. Our team also increased application performance and delivered several new features.
Python
Stock Market Trend Discovery with Machine LearningAt Azati Labs, our engineers developed an AI-powered prototype of a tool that can spot a stock market trend. Online trading applications may use this information to calculate the actual stock market price change.
Python
Semantic Search Engine for Bioinformatics CompanyAzati designed and developed a semantic search engine powered by machine learning. It extracts the actual meaning from the search query and looks for the most relevant results across huge scientific datasets.
Java, JavaScript
E-health Web Portal for International Software IntegratorAzati helped a well-known software integrator to eliminate legacy code, rebuild a complex web application, and fix the majority of mission-critical bugs.
JavaScript, Ruby
Custom Platform for Logistics and Goods TransportationAzati helped a European startup to create a custom logistics platform. It helps shippers to track goods in a real-time, as well as guarantees that the buyer will receive the product in a perfect condition.
