Automated Data Collection
Azati’s solution automated the phone call data collection process, drastically reducing manual labor and ensuring that the customer receives daily, accurate reports.
Technologies used
PostgreSQL
Ruby
Redis
Selenium
Sinatra
Motivation
Managing a large call center with hundreds of phone numbers across multiple providers was extremely time-consuming, with employees manually collecting daily call statistics. Existing solutions were slow, partially functional, and required constant maintenance. We built a secure, automated scraping platform to reduce manual work, improve data accuracy, optimize daily reporting, and free employees to focus on higher-value tasks.
Main challenges
Challenge 01
Constant Breakages Due to Interface Changes
The existing scripts frequently stopped working whenever providers updated their interfaces or logic, requiring immediate fixes to avoid downtime. Since web scraping is highly sensitive to UI changes, the system needed daily monitoring to ensure stability. We addressed this by continuously adjusting the scraping and authentication logic to match evolving provider environments.
#1
Challenge 02
Performance Degradation as Data Volume Grew
As the number of processed records increased, the system slowed down due to rising demands on computation, storage, and page-processing capacity. This required infrastructure upgrades and more efficient resource orchestration. We tackled this by moving the solution to high-performance servers and optimizing resource usage to sustain growth.
#2
Challenge 03
Delivering Fast, Daily Reports Under Growing Load
The customer relied on daily statistics, but the expanding dataset made it difficult to process everything within strict time limits. Full coverage wasn’t always guaranteed, yet maintaining accuracy and speed remained critical. We improved this through ongoing performance tuning and algorithm refinements that steadily increased throughput and reporting reliability.
#3
Our approach
Authentication and Authorization
We built a secure module to handle login for each account and automatically bypass two-factor authentication, intercepting SMS codes in the background to save employee time and ensure seamless access.
Scraper Development
Our engineers rebuilt and optimized scrapers for each provider, enabling efficient extraction of large volumes of phone call data, including call duration, acceptance, callbacks, recordings, and traffic usage.
Data Matching and Report Generation
Extracted data was matched to the corresponding phone numbers in an internal database, and daily CSV reports were automatically generated for management, ensuring accuracy and completeness.
Facing the same challenge?
Bring your complexity. We'll bring the plan. Select a convenient slot to start a conversation with our experts.
Schedule a callSolution
01
Authentication Module
This module handles secure logins for all accounts and automatically bypasses two-factor authentication, saving time and reducing manual effort. It works in the background to intercept and forward SMS codes without interrupting operations.
Key capabilities:
- Automatic SMS code interception
- Secure and reliable login for multiple accounts
- Reduces manual effort for employees
02
Five Provider-Specific Scrapers
Each scraper is customized for a specific telecommunications provider to efficiently extract large volumes of call data. They are optimized to handle growing numbers of phone numbers and ensure high reliability across different interfaces.
Key capabilities:
- Collects call duration, acceptance, callbacks, recordings, and traffic usage
- Optimized for large-scale data extraction
- Handles daily growth in the number of phone numbers
03
Data Matching and Reporting
This module aligns extracted data with the corresponding phone numbers and generates daily CSV reports for management. It ensures accurate, actionable insights without manual post-processing.
Key capabilities:
- Accurate data alignment
- Automatic report generation
- Daily delivery to management
Results & business impact
Operational Efficiency
The system has significantly improved the operational efficiency of the call center, allowing for better tracking of employee performance and call quality.
Reduced Manual Labor
Automation of the data collection process freed up employees from routine work, allowing them to focus on more value-added tasks.
Improved Performance
The solution delivers daily, accurate statistics on phone calls, including quality and traffic consumption data.
Scalability
The system is designed to handle an increasing number of phone numbers, with optimizations in place to maintain performance as the call center expands.
Cost Savings
Automation and the reduction of manual maintenance have helped reduce ongoing operational costs for the customer.
Platform Optimization
Rebuilt and optimized the scraping platform, significantly improving stability and performance.
Accurate Reporting
Enabled daily reporting with accurate phone call statistics, including missed calls, call duration, and operator performance.
Time Efficiency
Implemented an automatic two-factor authentication bypass module to save time and ensure smooth operation.
Data Processing Performance
The solution is now able to process data for approximately 96% of the phone numbers required by the customer, with continued performance improvements.
