What do other ISV do with Generative AI?

Table of Contents

The most common question I get from customers about GenAI is “What are other customers doing about it?” And it’s not a big surprise- Gartner’s study on what are the biggest challenges of implementing AI revealed that most executives find the following challenges: 1. Estimating and demonstrating value. In short- They just don’t know what to do with it. and 2. knowledge gap- It is very hard to find the people that have the knowledge, not just to do a POC, but also take it to production.

So to those who are still struggling I wanted to summarize the use cases and the target audience for each.

But first- The basics- What is Generative AI, and what is the difference from any other AI/ML? (Those who are familiar, are welcome to skip to the next section)

What is Generative AI?

Let’s start from the beginning.

AI- Artificial intelligence (AI) encompasses methods and technologies that enable computers to emulate human-like intelligence through the application of logical reasoning, conditional statements (if-then rules), and machine learning techniques.

Machine Learning- a branch of artificial intelligence that involves developing algorithms and statistical models that allow systems to learn from data and make predictions or decisions without being explicitly programmed. These algorithms have the capability to identify patterns and insights within data sets and construct models automatically, enabling machines to exhibit intelligent behavior and improve their performance on specific tasks over time. With Machine learning we can use simple input and create simple output. For example, taking metrics and using these metrics to predict sales. Simple input, simple output.

Deep learning (DL) is a branch of machine learning that employs artificial neural networks with multiple hidden layers to process and extract intricate patterns from structured and unstructured data. These multi-layered neural networks are designed to mimic the human brain’s ability to learn and make intelligent decisions. Deep learning algorithms excel at tasks such as speech recognition, image classification, and object detection, enabling computers to interpret and comprehend complex data like human speech and visual information. With deep learning we can use complex input and produce simple output. For example- take images and provide labels if it’s hot dog or not hot dog.

Which leads us to Generative AI- Generative AI refers to artificial intelligence systems that possess the capability to generate novel content and ideas, such as engaging in conversations, crafting stories, creating images, producing videos, and composing music. This form of AI is driven by large-scale models that have undergone extensive pre-training on vast datasets, often referred to as foundation models (FMs) or large language models (LLMs). These models leverage their exposure to massive corpora of data to identify patterns, extract insights, and generate original content by combining and recombining the information they have learned in innovative ways. Finally we can take complex input and produce complex output such as images, books and music.

How FM differ from other ML models?

Foundation models (FMs) differ from traditional machine learning models in their versatility and ability to adapt to multiple tasks. Conventional machine learning models are typically trained for a specific task, requiring separate models to be developed and trained for each individual application. In contrast, foundation models leverage their extensive pre-training on vast amounts of data, enabling them to be fine-tuned and applied to a wide range of tasks without the need for task-specific training from scratch. This transferability is a significant advantage of foundation models, as they can be adapted and customized for various use cases, eliminating the need to create and train separate models for each task, thereby saving time and computational resources.

Types of GenAI use cases

Since everyone is talking about GenAI, it can be very confusing. So I grouped the different use cases to make it a bit simpler:

Model Consumers

This is everyone - we all use Generative AI, whether it’s in chats with the models, in Google searches, within apps that help us create content, and so on.

Prompt Engineering

It sounds impressive, but all it means is formatting the ask (Prompt) when chatting to the model to get better results. I taught it in my son’s school to 11-year-old children. If they can catch on in less than one hour - anyone can. Just note that prompt engineering is different for different models - make sure you are using best practices of the model you are using.

RAG - Retrieval Augmented Generation

Now this is where things get a bit more technical. It’s like taking prompt engineering to the next level by programmatically augmenting the model’s responses with your own stored data. To do this efficiently, you store that data as vectors in a vector database. The cool part? You can customize GenAI to your specific needs without needing data scientists or retraining the model. I’ll go into more detail later, but this RAG approach is generally the most cost-effective way for companies to implement GenAI. Any business with an IT team can do it, not just software companies. Just keep in mind, RAG doesn’t actually train the model - it just augments the responses. If you want the model to truly learn, you’ll need to fine-tune it.

Fine-tuning

This is where you’re actually customizing and retraining the model itself, teaching it new things. But fair warning, this requires having data scientists on board, along with the time, compute power, and knowledge to properly train and test the model. It’s why it is reserved for companies that can afford to fine-tune models.

Model Producing

And finally, we have the big dogs - companies like OpenAI, Meta, Amazon, Anthropic, Google, etc. These are the ones with the massive resources to create brand new generative AI models from scratch. For the rest of us mere mortals, that’s just not feasible.

GenAI for Software companies

Alright, let’s talk about how this generative AI thing can impact software companies in a bunch of different ways.

First up, you’ve got the developers themselves using GenAI as like a super-powered coding assistant. They can have it crank out code, refactor existing stuff, upgrade applications, and even document their infrastructure as code. Talk about a productivity boost, right?

But it doesn’t stop there. Software companies are also finding ways to integrate GenAI directly into their products for their customers to use. Like, let’s say you’ve got a company that provides a real estate rental website. They could give their customers the ability to generate slick listing descriptions, create virtual tours from images and videos, and all sorts of cool stuff like that, all powered by GenAI.

Now, even companies that don’t directly deal with GenAI are feeling the ripple effects. Project management tool makers might need to add support for managing these massive language models, which they call FMOps. And cybersecurity firms? They’ll probably need to start covering potential risks and vulnerabilities that come with using GenAI.

Let’s say you’ve got customers who want to train their own retrieval-augmented generation (RAG) models using their data. If you’ve been providing them with regular reports up until now, they might start asking for that data in vector databases instead, so they can use it to train their GenAI models more effectively.

What can we do?

Still not sure how GenAI impacts your company? AWS have a use case portal you can look to see how companies implement GenAI, but in general- Think of those use cases that you have complex input and need to produce complex output, or when your customers might need to produce complex output.

It’s a whole new world out there, and GenAI is shaking things up left and right. Software companies that get on board early with this tech could seriously gain a competitive edge. It’s an exciting time to be in this space.