AI Calorie Calculator and Food Recognition
Our Data Scientists have successfully implemented a prototype system into an already functioning calorie counting application, that can instantly estimate the calorie content of complex dishes by images analysis. Such a solution can be useful in such domains as agriculture, catering, sports or even for everyday life.
Objective
It’s not a secret that at least once we were interested in how many calories we eat per day. Many who are fond of a healthy lifestyle know how important it is to maintain a balance of nutrients.
Today there are a huge number of applications which help calculate how many calories we consume per day. But in such products there is one common issue — they are rather a personal diary with a primitive calculator, where the user must enter data manually. Cooking a variety of dishes with many ingredients is even more confusing, making it difficult to determine the exact calorie count.
In recent years, it has become possible to use deep learning to recognize objects in images with high accuracy. We realized Azati could apply new technologies to the problem of food quality estimation to simplify the process and provide the user with the fastest and most efficient result.
Challenges
#1 Challenge
When working with artificial intelligence and computer vision algorithms, often the main problem is to provide enough data for training. Moreover, these data must be prepared for subsequent processing. But as you know, there are a huge number of products which in various combinations and cooking methods, can provide different amounts of calories. And all these nuances had to be considered.
For example, the calorie content of a boiled egg per 100 grams will be 155 calories, and a fried one — about 175.
Therefore, our specialists had to prepare colossal examples of high-quality potential dishes images. Some of them we took from open sources, and some were made manually due to the lack of appropriate pictures.
#2 Challenge
The second issue the Azati team faced was the quality of the processed data. At different angles or lighting, the algorithm could not see the full picture. It is also worth considering the different dish sizes.
For example, we had to take into account the fact that only half of the steak could lie on the plate, but in the database we stored data for 100 grams of each product. So we had to compare and calculate it accurately.
The only way to overcome it was to match the frames manually. But our team did a good job and we got a vast database for further use.
Process
Step 1.
First of all, our specialists analyzed existing similar software, found out the basic work principles and found points that could be improved.
Step 2.
Our team studied all the necessary documentation for development, chose the appropriate tools, and created the algorithm itself.
Step 3.
Next, as it was described above — we needed to collect, prepare and provide high-quality images and data for training our algorithm, which was the sufficient part of the project.
Step 4.
Azati engineers trained the machine learning model to recognize and automatically divide the image into several categories. Our experts looked through these categories to determine groups that are later used to create templates.
Step 5.
We created a template for each group, and after that, this template is used to process all pictures in this group. To achieve maximum accuracy, our specialists manually mapped the areas. For example, separate the garnish from the meat.
Step 6.
We processed each document multiple times to provide maximum quality and accuracy. As a result, the system displays data to the user’s screen.
Solution
While solving these challenges we developed a small script written in Python. The prototype takes an image as the input and returns a set of frames where each component is circled with a square in which calories are indicated, and the total result of the whole dish is displayed. Check out the screenshots below to see how the results look.
SCREENSHOTS
Technologies
Results
We made it possible to process images of compound dishes and calculate their calorie content using machine learning and computer vision. The model quite accurately recognizes each product and distinguishes one component from another. For more complex object classification, the model requires additional data and extra training.The more data we provide, the more accurately a model classifies road defects.
Featured case studies:
JavaScript, Ruby
The customer asked Azati to audit the existing solution in terms of general performance to create a roadmap of future improvements. Our team also increased application performance and delivered several new features.
Python
Stock Market Trend Discovery with Machine Learning
At Azati Labs, our engineers developed an AI-powered prototype of a tool that can spot a stock market trend. Online trading applications may use this information to calculate the actual stock market price change.
Python
Semantic Search Engine for Bioinformatics Company
Azati designed and developed a semantic search engine powered by machine learning. It extracts the actual meaning from the search query and looks for the most relevant results across huge scientific datasets.
Java, JavaScript
E-health Web Portal for International Software Integrator
Azati helped a well-known software integrator to eliminate legacy code, rebuild a complex web application, and fix the majority of mission-critical bugs.
JavaScript, Ruby
Custom Platform for Logistics and Goods Transportation
Azati helped a European startup to create a custom logistics platform. It helps shippers to track goods in a real-time, as well as guarantees that the buyer will receive the product in a perfect condition.