Enter­prise inform­a­tion retrieval with Azure AI Search

Gen­er­at­ive AI is already hav­ing a massive impact across all indus­tries, and at synvert we’ve been act­ively learn­ing, test­ing and devel­op­ing real solu­tions within this new land­scape.
Most recently, we developed an AI-powered pilot for an enter­prise cli­ent all within the Azure ecosystem.

In this art­icle we’ll dive into the solu­tion we built for this first iter­a­tion of a chat applic­a­tion that allows users to talk with tech­nical manu­als and data sheets.

This art­icle will focus on the fol­low­ing topics:

• The prob­lem we need to solve
• The Azure AI solu­tions we used
• Our approach
• Future enhance­ments

Chal­lenge

This par­tic­u­lar com­pany has a sup­port team that handles calls and emails from cus­tom­ers ask­ing ques­tions about their products. To be able to answer, they need to find the right doc­u­ment for the product, as each model can have doc­u­ment­a­tion in dif­fer­ent lan­guages, and some with oper­a­tion instruc­tions and oth­ers that are data sheets. After they find the right source, they need to find the inform­a­tion that the cus­tomer needs, that can be present in text, image or table formats.

The chal­lenge here is to stream­line and auto­mate as much as pos­sible all these steps, to reduce the feed­back loop between the sup­port team and the customer.

Our idea was to build an AI-powered solu­tion to increase the respons­ive­ness of their sup­port team when look­ing for inform­a­tion in oper­at­ing instruc­tions manu­als and product data sheets.
The ini­tial iter­a­tion would con­sist in a chat applic­a­tion where the sup­port team could ask ques­tions about any product they have to quickly find the inform­a­tion they needed.

The app needs to be able to:

• Under­stand what product model the user is ask­ing about.
• Get the cor­res­pond­ing files for that model in Ger­man or Eng­lish, depend­ing on what lan­guage the ques­tion is asked.
• Answer the question.
• Render the pdf file sec­tion from where the inform­a­tion came from.

Our Approach

To build our AI-powered chat, we need:

• A place to store doc­u­ments, so we can then show where did we get the inform­a­tion to gen­er­ate our answer.
• A search engine, able to con­vert our text into vec­tors, able to save extra metadata with page num­bers, product name, type of file and lan­guage, able to fil­ter for these para­met­ers and retrieve rel­ev­ant context.
• A lan­guage model, that can receive con­text and provide cor­rect answers, without hallucinating.
• This solu­tion would need to work within the cli­ent’s Azure ecosystem.

Stor­age

Azure Blob Stor­age is a stor­age solu­tion for the cloud that let us save a large amount of unstruc­tured data. In this spe­cific case, we did­n’t need a reg­u­lar data­base, we just needed a place to store and load our doc­u­ments (pdf files). These doc­u­ments were ini­tially being saved in the cli­ent’s Share­Point, but we needed a place where the search engine and our fron­tend could eas­ily access it, so this was the best option for that purpose.

Search Engine

Azure AI Search is an inform­a­tion retrieval plat­form, powered by AI. It has an index­ing engine, so it’s cap­able of trans­form­ing our data into vec­tors. After that, it stores the data in vec­tors and plain text and also allows to add some metadata into an index. It was our choice because it allows us to save our texts, embed­dings and metadata all in the same place, while hav­ing great search cap­ab­il­it­ies, provid­ing fast and accur­ate res­ults, a good match for our requirements.

LLM

The Azure OpenAI Ser­vice provides us access to OpenAI’s lan­guage mod­els. We can access all mod­els avail­able from within Azure. This ser­vice is more suit­able for enter­prise applic­a­tions, offer­ing super­ior secur­ity cap­ab­il­it­ies, LLM data pri­vacy, while also sup­port­ing VNETs and private end­points, all under Azure Cog­nit­ive Ser­vices SLAs.

Solu­tion

Stor­ing Data

To have our data ready to be quer­ied, we first had to upload all of the files in the blob stor­age ser­vice. We decided to upload one file per page, so in the fron­tend we could show the page that was used to answer the user question.

Index­ing Data

After upload­ing data, we relied on Azure AI Search to cre­ate the embed­dings using text-embed­ding-ada-002 and store them in an index. We did this pro­cess of stor­ing in the index by page as we did when stor­ing files. To have addi­tional inform­a­tion stored, so we could then fil­ter our doc­u­ments, we did­n’t only save the con­tent of the files, but we also saved the file­name (title), the num­ber of the page (page) and also, the model of each product, lan­guage and type of the file based on the file­name (cat­egory).

Ana­lys­ing questions

Everything starts with the user’s ques­tion. This ques­tion trig­gers a request to our API. The first thing we do is identi­fy­ing the product model, the lan­guage, the type of request and the type of doc­u­ment. For each of these para­met­ers, we do a call to the LLM to identify them and then use it to fil­ter the documents.

The type of request can be “search” if the user asks to retrieve a doc­u­ment (e.g. “Can you send me the doc­u­ment for product model X”?) or it can be “ques­tion and answer” if the user asks some­thing spe­cific about a product (e.g. “Can you provide me the tem­per­at­ure ranges for model Y”?).

Depend­ing on the type of request, the pro­cess can con­tinue one of two ways:

• If the type of the request was “search”, we used the inform­a­tion that we got from the ques­tion to fil­ter our search with AI Search and get the required doc­u­ment. In this search we used hybrid search to return us the top 5 res­ults, in this case the top 5 pages, that we use to fetch the file­names of the doc­u­ments we need to send back to the user.
• If the type of the request was “ques­tion and answer”, we searched using semantic hybrid search with score and rerank, to return us the top 5 pages related to the ques­tion, also filtered by the iden­ti­fied para­met­ers. Then, we would send to gpt-4-turbo model as con­text, to get a proper answer. After that, we would send the answer back to the user.

Future Enhance­ments

With this first iter­a­tion, we already provided a solu­tion that improves con­sid­er­ably the search and response time of the sup­port team. Non­ethe­less, we can already identify some areas for improve­ment for a sub­sequent iteration:

• To optim­ize per­form­ance we could use cach­ing, in a way that, when a user asks a ques­tion that someone or even him­self already made, instead of doing another call to the LLM, we would just use the cache to get the answer, pos­sibly also stor­ing them as embed­dings in a vec­tor store.
• We can also try to optim­ise cost and speed by mix­ing dif­fer­ent mod­els for spe­cific tasks where cheaper and faster mod­els are good enough for the task.
• Fur­ther­more, we could train and fine-tune an LLM model using the cli­ent’s data which in turn will save us on token con­sump­tion, request latency and out­put quality.

Final thoughts

Azure AI ser­vices, spe­cially Azure AI Search and OpenAI, are an excel­lent entry point for enter­prises that want to add AI cap­ab­il­it­ies to their applic­a­tions and data. Many enter­prises already trust Azure with that data, enabling us to build a robust proof of concept all within this eco­sys­tem in a short amount of time.

At synvert we are already enabling com­pan­ies to take advant­age of the AI revolu­tion, so stay tuned for more insights and break­throughs in the near future!