Why Multimodal Embeddings Could Change Enterprise Search

 


Why Multimodal Embeddings Could Change Enterprise Search

Imagine this.

A field technician sees a machine component on a factory floor, takes a photo of it, and the system instantly retrieves the relevant manual or maintenance documentation.

No part numbers.
No exact keywords.
Just a photo.

This kind of search experience is becoming possible with multimodal embeddings.

In simple terms, embeddings convert things like text or images into numerical representations that capture their meaning. When two things are conceptually similar, their embeddings end up closer to each other in vector space.

Recently, Google released Gemini Embedding 2, which creates embeddings for multiple types of data such as text, images, audio, and video, all in the same shared space.

This means that a photo, a written description, or even a video frame of the same object can be understood as related by a search system. I ran a quick experiment using the Gemini API.

I compared a sports car image with two text queries:

  •  “cycle” → similarity ≈ 0.28

  • “red sports car” → similarity ≈ 0.40


The results are not very strong yet, but the relative ranking is correct. The system understands that the sports car image is closer to “red sports car” than to “cycle.” This is probably where the technology stands today: early but promising. The performance will likely improve as models and training data evolve. But the important thing is that the foundation of multimodal embeddings has now been laid.

For businesses, this opens up some interesting possibilities.

  • In manufacturing and field service: Technicians could take a picture of a component and instantly find manuals, troubleshooting guides, or spare part catalogs.

  • In industrial supply chains: Teams could search large parts catalogs using photos, making it easier to identify compatible components.

  • In media and content organizations: Large archives of images and videos could be searched using natural language descriptions instead of manual tagging.

  • In enterprise knowledge systems: Employees could search using whatever they have available, a description, a screenshot, or a photo and still retrieve relevant information.

We’ve already seen the shift from keyword search → semantic search. 

With models like Gemini Embedding 2, we are starting to see the next step: multimodal search.

Comments

Post a Comment

Popular posts from this blog

5 Skills to Work Better with AI

Image
  AI is everywhere, and it’s here to stay. From virtual assistants to powerful decision-making tools, AI is shaping the way we work and interact with technology. But simply using AI isn’t enough—you need to understand how to make the most of it. Many people, even in everyday interactions with AI-powered tools like ChatGPT, Gemini, or DeepSeek, struggle to get the results they expect. This isn’t because AI is flawed, but because working effectively with it requires a certain set of skills. Knowing how to phrase your queries, provide the right context, and interpret AI’s responses can make all the difference. Whether you use AI for work, creativity, or problem-solving, keeping these five key skills in mind will help you unlock its true potential. 1. Know Where to Use AI AI isn’t a one-size-fits-all solution. While it excels at processing vast amounts of data and analyzing lengthy documents, it’s not always necessary—or even useful—in every situation. It’s not a magic wand, and unders...
Read more

AI learns language with the help of a child

Image
  Today I want to talk to you about an experiment conducted by some AI researchers to see if AI can learn language the way human babies do. With this in mind, they put a head mounted camera on an infant and tried to understand how human babies comprehend language with the help of the visual and auditory cues it gets from its environment and can AI learn on the same lines. We’ll discuss that experiment in a bit. If you come to think of it, learning a language is such a complex task and yet human babies do it pretty easily! Contrary to the belief that parents or caretakers teach a child to learn a language, children acquire language through interaction — not only with their parents and other adults, but also with other children. All normal children who grow up in normal households, surrounded by conversation, will acquire the language that is being used around them. And it is just as easy for a child to acquire two or more languages at the same time, as long as they are regularly int...
Read more

Part 1: Prompt engineering: The Art of Delegating to Generative AI

Image
Ever wondered why some people get spot-on, helpful responses from AI, while others end up with vague or confusing answers? It’s not just luck—it’s the power of good prompting. The way you ask the AI to do something matters—a lot. This is part one of a three-part series where we’ll explore what prompt engineering is and the different techniques you can use to guide AI effectively. Part two will cover the constituents of a prompt. In part thre, we’ll dive into how to create great prompts—with practical tips and examples to help you get the most out of AI. What is prompt engineering? So, what exactly is prompt engineering? It’s the art of telling an AI model what you want it to do. But why is it called an art—and yes, also a science? Because the way you frame your request can completely change the quality, accuracy, and usefulness of the response. To understand why prompting is such a big deal, you first need to know how these AI models actually work. The popular chat companions we’ve all...
Read more