AI Angel: Solving the Precision Gap in AI Business Search

One of the first architectural decisions was how to connect the AI assistant to the community database.

Figure 1. Fine tuning VS RAG

Fine-tuning was not the right fit for this use case. Community data changes constantly: new members join, profiles are updated, events are added, and business interests evolve. Training a model on this data would be expensive, slow to update, and difficult to keep accurate.

Instead, we chose a RAG architecture — Retrieval-Augmented Generation.

In simple terms, RAG allows an AI model to answer questions using an external knowledge base. When a user asks something, the system first retrieves relevant information from the database. Only then does the language model generate a response based on that retrieved data.

Figure 2. RAG diagram

This approach helped us keep answers connected to real, current community information rather than relying only on the model’s general knowledge.

In the first iteration, we used a standard vector database to search for matches based on semantic similarity.

This was a logical starting point. Vector search is good at understanding meaning beyond exact keywords. For example, it can understand that “startup funding” and “investment for early-stage companies” are related ideas.

But business search has a precision problem.

In a business community, a result can be semantically similar and still be practically irrelevant. A member, event, or company may match the topic of a query but still be wrong because of location, timing, role, industry focus, or business context.

During manual testing, we found several important limitations.

The first issue was location awareness.

The system could recommend an event in Prague to a user in Kyiv simply because the event topic was relevant. From a semantic search perspective, this made sense. From a user experience perspective, it was not helpful.

Business recommendations often depend on practical constraints. Location, market, region, and proximity matter. A good AI concierge needs to understand not only what the user is asking about, but also where the answer is relevant.

The second issue was business context.

A user looking for mentors, founders or industry experts should not receive a generic list of semantically similar profiles. The system needs to understand which role is useful for the specific intent behind the query.

For example, two people may both be connected to “healthcare startups,” but one may be a mentor in a community and another may be a member. Depending on the user’s question, only one of those profiles may be the right recommendation.

We also found that search by specific business names was not precise enough. In business communities, exact names matter. Users often search for a particular company, brand, project, or person. A good business search system needs to handle both broad semantic intent and exact business terms.

The third issue was how the language model handled too much raw data.

When the system passed many retrieved records to the LLM, the model sometimes failed to use all of the important information. Some relevant data points were ignored, while other results received too much attention.

This created inconsistency. Even when the correct information had been retrieved, the final response could still miss useful matches.

For a business concierge, this is a serious issue. The assistant should not only sound natural it must also reliably consider the available data and explain recommendations in a useful way.

Another issue was recommendation diversity.

During testing, the same members could appear repeatedly across different queries, even when other relevant people existed in the community. This created a narrow experience. Users expect discovery, not the same shortlist every time.

For an AI concierge, relevance is not only about finding the strongest match once. It is also about surfacing useful alternatives and helping users explore the community more effectively.

To solve these issues, we moved beyond simple semantic similarity and built a hybrid search workflow.

First, we added metadata extraction. This allowed the system to detect useful structured information from the user’s query, such as locations, dates, business categories, roles, and other keyword-like signals.

Then we switched the search layer to Elasticsearch.

Elasticsearch gave us more control over ranking. Instead of relying only on semantic similarity, we could combine different relevance signals and assign weights to them. This made it possible to prioritize results based on both meaning and business context.

For example, if a user asks for relevant events in Kyiv, the system can now treat location as a priority rather than just another piece of text. If a user searches for a specific company or business name, the system can handle exact terms more reliably. If timing matters, upcoming events can be ranked higher than outdated ones.

We also improved how retrieved data was prepared for the final AI response. Instead of sending too many raw records directly to the LLM, the system now prepares a more compact and structured context before final generation. This helps the assistant use retrieved information more consistently and reduces the chance of important results being ignored.

To make recommendations more personal, the workflow can also use current user data where relevant. This allows the assistant to consider the user’s own profile, region, and context instead of treating every query as isolated.

The result is a more practical and precise search experience.

In manual testing, this hybrid approach consistently produced more useful results than the initial vector-only version, especially for queries involving location, timing, roles, and specific business names. While we have not yet completed a formal quantitative evaluation, these tests confirmed that structured ranking signals are critical for business search quality.

Figure 3. Current workflow

By moving from simple vector similarity to a structured search workflow with Elasticsearch, we transformed AI Angel from a basic chatbot into a more reliable business concierge — one that can better understand user intent, work with real-time community data, and deliver recommendations that are not only relevant in theory, but useful in practice.

The main lesson from AI Angel is that the quality of an AI assistant depends heavily on the data workflow behind it.

A good prompt is not enough.

For a business search assistant, the system needs to understand the query, extract the right signals, search across live data, rank results intelligently, prepare the retrieved context, and only then generate a natural response.

The “magic” of AI chat is not just in the conversation layer. It is in the infrastructure that decides what information the model sees and how that information is prioritized.