Kroger Uses AI to Match a Picture of Your Fridge Shelf with a Recipe

Kroger Uses AI to Match a Picture of Your Fridge Shelf with a Recipe

Kroger uses visual recognition AI to help people make better use of the leftovers in their fridge.

Grocery chain and agency 360i recently set up a Twitter account called Chefbot, which can identify up to three foods in user-submitted images and search for the best match in a database of recipes. The creators claim that the bot is trained to recognize 2,000 ingredients and to pull them from a log of around 20,000 recipes.

While the tool is currently quite rudimentary, Kroger has big plans to develop it into something more sophisticated over time, together with 360i and technology partners Coffee Labs and Clarifai. The developers plan to include every interaction the bot makes in its training data in order to refine future iterations in identifying food and adapting recipes.

“Over time, Chefbot will continue to learn and improve. We look forward to seeing how the experience evolves and ultimately integrates with Kroger’s mobile app,” said Menno Kluin, chief creative officer at 360i. “This tool makes so much sense for Kroger.”

The tool may still take some time before it’s ready for such app integration and can be used beyond a fun Twitter redirect. The current iteration of the bot can rarely guess all three ingredients in one picture – although it can usually identify at least one.

Asking the Chefbot to confirm the ingredients after trying to identify them can also provide Kroger with valuable labeled data as he trains future iterations of the project.

Kroger also bills the project to reduce food waste. This is part of his social health campaign in the area of ​​sustainability and food accessibility. “It is in line with their mission to reduce food waste while delivering much-needed benefits to people who spend more time than ever cooking at home and trying to maximize their purchases,” said Kluin.

Other companies are also investigating how AI and computer vision in particular can be used in the kitchen. McCormick Spices teamed up with IBM last year on a project that used AI to invent new flavor combinations. Facebook’s research laboratory has been working on a tool that can use AI image recognition to guess recipe ingredients for a meal.

At the moment, brands have to think about the customer journey.