Drop us a line
If you are interested in the development of a custom solution — send us the message and we'll schedule a talk about it.
Azati Software
Road Pothole Detection With Machine Learning And Computer Vision
As the part of Azati Labs, our data scientists have successfully built a prototype of the system, that can detect road defects analyzing images and videos. This prototype can be useful to municipal government to simplify roads defect detection to calculate road repairing costs automatically. The information extracted by the prototype can also be used by automotive manufacturers to help smart cars avoid potholes and decrease overall repairing costs.
PROJECT IDEA:
Road defects are one of the most common reasons for suspension repairs and tire replacements. According to general statistics, even the most careful drivers face suspension repairing every three years or so. Bad roads are a well-known issue of all east-European countries.
The goal of this project was to train computer vision to determine road defects, especially potholes. There are a vast number of pre-made machine learning models for object detection and classification trained by Google and Facebook. Unfortunately, there were no pre-made models for pothole detection. This way we decided to train a custom model from scratch.
MAIN CHALLENGES:
The machine learning process was exciting and worrying at the same time: we were among the first who decided to train CV for such purposes. While developing a prototype, there were several challenges.
CHALLENGE #1:
The very first challenge our team faced was a lack of data. Data Scientists require huge cleansed datasets for successful machine learning model training. In our case, there were no high-quality images of potholes and other road defects on the Internet. Our engineers tried to use pictures extracted from the open sources, but the results of the trainings were quite disappointing. This way, our team decided to use the live-data.
CHALLENGE #2:
We spent some time driving and capturing on video the roads, and after that, we faced another issue – low data quality. Potholes have different shapes and look differently in sunny and cloudy weather. Also, footages captured from different vehicles had different fields of view due to the different camera mounting points. We could not use the collected data “as is” due to its inconsistency. The only way was to map all the footages manually.
CHALLENGE #3:
As mapping the entire video is a quite tricky process, we split videos on a sets keyframes. If the camera recorded video in 30 frames per second, it took about an hour to map all the keyframes in minute footage. During data mapping, we also considered the footage quality. If a clip was recorded in poor quality or low resolution, it made a clip unusable for a model training. The usage of low-quality clips does more harm distorting the data.
PROTOTYPE DESCRIPTION:
While solving these challenges we developed small script written in Python. The prototype takes an image or video clip as the input and returns a set of frames where the potential potholes and other road defects are outlined with squares. If the script takes a video, it splits it into a set of frames and examines each frame separately. When a script processed all the data, it joins all frames into a one video. Here are how fancy clips about computer vision are made. As a result, we get an image or a video, where the potential potholes and other road defects are outlined with squares. Check out the screenshots below to see how the results look like.
TECHNOLOGY STACK:
RESULTS:
The development process was quite challenging for our data scientists. We made it is possible to find potholes and other road defects using machine learning and computer vision and delivered the proof of concept prototype. The model finds the road defects quite accurately, but there are some issues with the classification of defects. It is quite hard to identify pothole if it looks similar to roadhatch. For complex object classification, the model requires additional data and extra training. The more data we provide, the more accurately a model classifies road defects. But as the data requires manual mapping, it takes a lot of time and makes data processing and cleansing quite expensive.
WAYS OF IMPROVEMENT:
The prototype uses a single model to find and classify road defects. It is simple to understand how it works, and why it provides predictable results. When data is processed using a single model, we know where and when everything went wrong. According to our calculations, to make this model to classify defects accurately it will take about 12 million of manually mapped images, what is entirely unaffordable for the majority of companies. This way, our scientists suggest processing the data in several steps and using multiple methods:combining traditional object classification algorithms and machine learning. We found several ways to improve the prototype, so if you want to learn more — contact us, and we will have a chat about that!
Featured case studies:
JavaScript, Ruby
HR Planning SoftwareThe customer asked Azati to audit the existing solution in terms of general performance to create a roadmap of future improvements. Our team also increased application performance and delivered several new features.
Python
Stock Market Trend Discovery with Machine LearningAt Azati Labs, our engineers developed an AI-powered prototype of a tool that can spot a stock market trend. Online trading applications may use this information to calculate the actual stock market price change.
Python
Semantic Search Engine for Bioinformatics CompanyAzati designed and developed a semantic search engine powered by machine learning. It extracts the actual meaning from the search query and looks for the most relevant results across huge scientific datasets.
Java, JavaScript
E-health Web Portal for International Software IntegratorAzati helped a well-known software integrator to eliminate legacy code, rebuild a complex web application, and fix the majority of mission-critical bugs.
JavaScript, Ruby
Custom Platform for Logistics and Goods TransportationAzati helped a European startup to create a custom logistics platform. It helps shippers to track goods in a real-time, as well as guarantees that the buyer will receive the product in a perfect condition.
