When ChatGPT 3.5 launched in 2022, it was the greatest sensation in the world of business. Open AI has been launching GPTs since before 2020, but the launch of GPT 3.5 stood apart. Firstly because the quality of responses of the model were levels beyond what was existing, and secondly because they decided to make it a B2C launch instead of B2B. Launching this as a B2C application democratized the knowledge about where AI had reached and what it could do.

Now, it has been close to 1.5 years since the launch of GPT 3.5, and direct consumers couldn’t be happier. They have been using it to compose Emails, complete School projects, write love letters and poems, improve their coding speed, and so much more. But what about the enterprises? Enterprise adoption of Gen AI has been painfully slow, which I have covered in another blog. Of course, employees are using it here and there for completing their tasks, but enterprises have not deeply integrated Gen AI within their workflows to completely automate them.

The challenges are many, Gen AI cannot be trusted to always provide the expected response and data being leaked outside the environment of the company being the greatest of them. The area where Gen AI has seen some light, is in Chat Bots. Almost every company has built Chatbots in a few areas. They may still be PoCs or being used internally, but what matters is they are there. So what decides whether an enterprise should be building a chatbot or not?

What are the primary use cases for a Chat Bot?

To put it in brief, a chatbot is simply an interface for a user to interact with Information Systems. Although significant, Gen AI’s integration in the backend infrastructure of Information Systems has not been dramatic . Where it has made a difference is in the User Experience or Interface.

Just to give an example, an E-Commerce company integrating a chatbot within their application intends the bot to communicate with the user and recommend great items. However, this isn’t vastly different than a user entering the details in the search box of the e-commerce site and seeing the results on the page. In fact, that is exactly what is happening in the backend of the chatbot, just that the information presentation will be in Natural Language instead of looking at a list of items on the website. This is a use case of information retrieval, which is one of the top use cases for chat bots. Let’s have a look at the major ones.

[1] Data Retrieval

(a) Unstructured Data Retrieval

Enterprises have a massive amount of data being created every day. Unstructured Data usually comes in from sources like knowledge or strategy documents, meeting transcripts, tutorials, SoPs, etc. It is created by humans from scratch and stored in Word or PDF documents on the cloud. Such data is so vast, that it is almost impossible to find specific pieces of information quickly.

Companies build internal knowledge bases and search engines to retrieve this data quickly, but the traditional search interface is not fast enough. Users must still go through the first 4 to 5 links before they find the right piece of information.

With Gen AI, this changes completely.  A typical Gen AI application will include feeding the information from all of the top 5 links to Gen AI, and asking it to condense it in a way that answers the user’s questions with precision. Now, the user only needs to read 4-5 lines without lifting a finger instead of reading 4-5 pages that include multiple clicks.

This is the same use case that is adding value in customer success functions in the company. Instead of retrieving data from knowledge bases, the chatbots retrieve it from SoPs and answer as good or better than a human would.

(b) Structured Data Retrieval

If amount of unstructured data is large, the amount of structured data is orders of magnitude larger. Once processes are set up, organizations set up monitoring systems to log every action. Be it core operations, admin activities, or software, everything is monitored closely to derive insights that might save a couple of millions here and there. This data is structured, and can be viewed on dashboards by the relevant decision makers.

Now with Generative AI powered chatbots, dashboards are getting replaced with chat interfaces. If we compare the user journey in both cases, it is not very different in terms of number of steps.

‍

Instead of clicking, the user is able to type the same things in natural language. For someone handling a lot of dashboards at once, it may be easier to type than to find the correct buttons, but for people handling 2-3 dashboards, it is easier to just use the dashboard.

Typically, organization leaders using a lot of dashboards infrequently (since this makes them less familiar with the UI) are the ideal target market for a chat based structured data retrieval system. However, if you add what will be discussed in the next use case, Gen AI chatbots become far more superior in dealing with structure data.

For both the data retrieval use cases, the primary value that Generative AI adds is more than anything, the presentation or summarization of data.

[2] Data Update

Up till now, what we have seen is Gen AI retrieving existing data for the users. This is how the responses of the bot are generated. Similarly, it also can also perform write operations on a database using user responses. The updated database can trigger actions that automate workflows. Currently, all organizations approach this cautiously, as Generative AI is unpredictable and has lower security protocols. A prompt injection attack could potentially flood an organization’s database with millions of fraudulent, undetectable rows in no time.

However, with a strong security layer on top of the Gen AI application, a Gen AI powered chatbot can be a blessing. The way this happens is through application integrations. Imagine every application on your device connected to your chat interface, and anything that you want to do can simply be typed (or spoken) in the chat box. This is how every company visualizes work in the future, and we are not far from there.

Currently, simple applications like payroll, taxation, Email, etc. have been connected with chatbots. Tailored AI is working with multiple customers to consolidate multiple touchpoints into one by offering such integrations. The reduction in number of touchpoints, and the ease of use makes this a powerful tool. If not for this, an average employee must be trained on the use of multiple tools, and follow a learning curve before they are able to use the tools well. But with this, you just type in whatever your decision is based on the information that the chatbot has retrieved, and the agent will take care of the rest.  There’s still a long way till ALL of the applications an employee uses are linked with the chatbot, but we are getting there.

The use cases once you master the art of writing databases are endless. This would include building new applications, forms, dashboards etc., in natural language, performing data analytics on previously unseen data, etc., all in natural language. Another interesting place where enterprises are using this blessing is to collect user feedback from open ended conversations as opposed to traditional, structured surveys.

This will eventually unlock much higher productivities for employees, as this allows the company to collect a high volume of data on human decision making. With sufficient data, many of these decisions can be automated.

For this use case, the primary value addition of Generative AI is converting natural language to commands that can be interpreted by a computer.

What is the ROI for these use cases?

[1] Data Retrieval

(a) Unstructured Data Retrieval

An extremely useful feature for research, this has the potential to save millions of dollars for a company depending on how many people use it. Generally, for a company of the size of 10000+ people, you are able to save 100-200 hours of researching efforts of high salaried individuals. The cost of implementing such a solution ranges from 10K to a 100K USD depending on the size and structure of data.

If you are planning to use this feature in the customer support function, then the calculations are a bit different. A customer success executive typically earns 1/10th of a high salaried individual, and you would need that much more scale to get a positive ROI.

For most cases, enterprises see an ROI of 10X. A rule of thumb to consider investing in a data retrieval bot is to check how many people are using the same sources of data. For every 10 people, a one time spend of $1000 makes economic sense.

(b) Structured Data Retrieval

Without analytical capabilities, this is a useful feature for top executives willing to monitor the company. When people with millions in salaries save a few minutes a day and a few hours in turnaround time, it is a considerable impact. Moreover, this can easily evolve into a data analytics tool as well, which not only fetches data, but insights for the decision makers.

For every 4 dashboards that you connect to the data retrieval bot, you save around 5 minutes per day. For an executive, that is $40-50 a day. Considering this, an ROI of 10X can be achieved if # of dashboards times the # of executives who will use this Bot is higher than 100.

[2] Updating Data

For this, it is difficult to quantify the value since it can vary a lot with the use cases at hand. The complexity of the tools being integrated, number of tools being integrated, level of accuracy that can be realistically achieved in performing write operations, number of employees who use these tools frequently, all play a significant role in determining value. A complex tool like Adobe Photoshop integrated with a chat interface unlocks tremendous value for designers, whereas integrating Email might unlock relatively lower value. The curve of number of tools integrated with the chat interface and the value unlocked is not linear. Higher the number of tools, exponentially will the value increase. This is because the learning curve for multiple tools is much longer than for a few tools due to human memory limitations.

The recommended method to take on investments in these fields is to sit with your AI team or AI partner, and figure out the possibilities, conduct a detailed ROI analysis for each possibility, and then take it forward.

Concluding Remarks

Investing in Gen AI chatbots right now, makes sense. Although it is not as big of a value add as you might imagine due to the high operating costs of a Large Language Model, those are bound to go down and down as better and better processors come into the market.

It is extremely easy to replace existing models with more efficient models if you have built the chatbot on a modular infrastructure. If you are using 3P tools to build such bots, you may not have as much control as you would like, and you might lose out on the value created to the Chatbot service provider.

If you are anticipating thousands of users, tens of databases, and tens of thousands of documents to be integrated into the database, then consider choosing an AI partner to help you build a continuously improving chatbot tailored for your needs. If you are just trying the waters, consider doing PoCs (proof of concept)with the ready made solutions available in the market.

‍