Homepage > Portfolio > AI Calorie Calculator and Food Recognition

AI Calorie Calculator and Food Recognition

Our Data Scientists have successfully implemented a prototype system into an already functioning calorie counting application, that can instantly estimate the calorie content of complex dishes by images analysis. Such a solution can be useful in such domains as agriculture, catering, sports or even for everyday life.

Objective

It’s not a secret that at least once we were interested in how many calories we eat per day. Many who are fond of a healthy lifestyle know how important it is to maintain a balance of nutrients.

Today there is a huge number of applications which helps to calculate how many calories we consume per day. But in such products there is one common issue — they are rather a personal diary with a primitive calculator, where the user must enter data manually. Cooking a variety of dishes with many ingredients is even more confusing, making it difficult to determine the exact calorie count.

In recent years, it has become possible to use deep learning to recognize objects in images with high accuracy. We realized Azati could apply new technologies to the problem of food quality estimation to simplify the process and provide the user with the fastest and most efficient result.

Challenges

#1 Challenge

When working with artificial intelligence and computer vision algorithms, often the main problem is to provide enough data for training. Moreover, this data must be prepared for subsequent processing. But as you know, there is a huge number of products which in various combinations and cooking methods, can provide different amounts of calories. And all these nuances had to be considered.

For example, the calorie content of a boiled egg per 100 grams will be 155 calories, and a fried one — about 175.

Therefore, our specialists had to prepare colossal examples of high-quality potential dishes images. Some of them we took from open sources, and some were made manually due to the lack of appropriate pictures.

#2 Challenge

The second issue the Azati team faced was the quality of the processed data. At different angles or lighting, the algorithm could not see the full picture. Also different dish sizes were a problem as well.

For example, we had to take into account the fact that only half of the steak could lie on the plate, but in the database we stored data for 100 grams of each product. So we had to compare and calculate it accurately.

The only way to overcome it was to match the frames manually. But our team did a good job and we got a vast database for further use.

Process

Step 1.

First of all, our specialists analyzed existing similar software, found out the basic work principles and found points that could be improved.

Step 2.

Our team studied all the necessary documentation for development, chose the appropriate tools, and created the algorithm.

Step 3.

Next, as it was described above — we needed to collect, prepare and provide high-quality images and data for training our algorithm, which was a sufficient part of the project.

Step 4.

Azati engineers trained the machine learning model to recognize and automatically divide the image into several categories. Our experts looked through these categories to determine groups that are later used to create templates.

Step 5.

We created a template for each group, and after that, this template is used to process all pictures in this group. To achieve maximum accuracy, our specialists manually mapped the areas. For example, separate the garnish from the meat.

Step 6.

We processed each document multiple times to provide maximum quality and accuracy. As a result, the system displays data to the user’s screen.

Solution

While solving these challenges we developed a small script written in Python. The prototype takes an image as the input and returns a set of frames where each component is circled with a square in which calories are indicated, and the total result of the whole dish is displayed. Check out the screenshots below to see how the results look.

SCREENSHOTS

Technologies

 
 
 

Results

We made it possible to process images of compound dishes and calculate their calorie content using machine learning and computer vision. The model recognizes each product quite accurately and distinguishes one component from another. For more complex object classification, the model requires additional data and extra training.

Drop us a line

If you are interested in the development of a custom solution — send us the message and we'll schedule a talk about it.