Removing Friction: Using Gen AI to Find, Hone and Generate Data

Damian Olthoff & Paul Lacey talk gen AI use cases for fast-growing tech companies and removing friction from information access. This is an abridged transcript of our webinar with Damian Olthoff.

Paul Lacey: Today we’re going to talk about gen AI use cases for fast-growing tech companies. How can folks use gen AI to get value quickly, both for expert teams as well as the wider organization? I’m excited to have Damian, GC at PROS, with us here. Since working together, we’ve had a lot of pretty long discussions on AI, so I thought it would be great to get you onto a webinar. I think we’ll start by asking Damian to introduce himself. Damian, you’ve got a career steeped in tech and AI. PROS is an AI company that’s been doing AI since long before Legal OS [now Flank] was even thought about. From your own career background, your tech history goes back to the dotcom era. 

Damian Olthoff: To date myself even further back than that, I started tinkering with computers before we could even afford a hard drive. I coded from high school all the way through law school, and even helped fund tuition by doing coding work. Post law school I ended up joining a fortune 100 tech company, got recruited away by a smaller tech company, and then for the past almost 20 years now I’ve been GC of various companies, mostly public, one in the pharma space that had a technology arm, and then for a number of years here at PROS. That’s everything you might expect from IPOs, to selling public companies, M&A, and everything in between. As far as me personally, and I think this will resonate for those that love tech like I do, I’ve always hated busywork. What I hated the most about school was when the sole purpose of the work was not productive, but just to keep you busy. I’ve kept that hatred of busywork throughout my professional career. Anytime I’m given tasks that I know could be done cheaper, better, or faster, I’m always one of the early adopters to try it. Fast forward to generative AI. Having done this for a while, it’s the thing I’m probably most excited about, because it has the opportunity to democratize access to AI. Despite having been around for many years, a lot of AI, just like the AI PROS sells, is not that easy for individuals to access in their daily lives. I think we’re on the verge of a lot of change in that regard, even more than we’ve seen in the last year. 

"Fast forward to generative AI. Having done this for a while, it’s the thing I’m probably most excited about, because it has the opportunity to democratize access to AI."

PL: The pace of change in the last year has really been absolutely crazy. It would be great to go back to when your team first started thinking about using generative AI. Could you describe the tangible pain you were feeling, not just in the team but also the wider organization? 

DO: The original genesis was just the growth of the business, not only in the volume of legal work, but also in the sheer headcount growth. When I joined we had around 300 employees, now we’ve got north of 1400. If you think about, not just new deals, because that certainly drives volume, but onboarding new people, how do you get them access to the data they need, particularly for those of us with teams that support customer-facing work? By the time a sales rep gets to the closed cycle process, the content they would have consumed during onboarding will be lost in their brain, so to speak. So they turn to the trusted advisors with experience, which is often the legal team. It’s easier to phone a friend than to go back through the original training material, because we need answers now. We want business to move faster, not slower. That was the original genesis of it.

Then there’s obviously some fundamental technologies you have to get in place to manage contracts and so on. Once you’ve got that baseline to be able to have the data, then you really start to see the trends. And that’s where we started to double click further into generative AI. We were seeing a lot of the same types of high volume, not even legal requests, but just requests for information. And we thought there’s got to be a way to leverage what we’re learning about generative AI, to make that information accessible to our sales reps, so they can answer questions in real time, particularly with a geographically dispersed business. We’ve got folks on the ground in many countries all over the world, and you simply can’t wait on an answer because of time zone for something that’s low-hanging fruit. So even if the answer is well documented, if that person isn’t steeped in the nuance of that specific question, it could be a delay for no reason. So we started saying, how do we document what these things are? That’s where generative AI came in, as a tool that had the ability to get us context quickly. When I talk about context, it’s really about being able to discern, if you’re asking a contract question, are you asking sell-side or buy-side? Because you may have different answers for that.

So much of what we’ve learned about gen AI is that, if you get the context right, all these tools are incredible. And if you don’t get the context right, they’re equally as frustrating with the hallucinations and gobbledygook they give you. 

PL: Absolutely. I use OpenAI and ChatGPT almost hourly. But for some expert stuff, where context and expertise are really important, that’s where it’s going to fall down; you need that extra context. So what I’m hearing is you’ve got high growth, a sales team that needs answers straight away, and you want to drive growth as well, not slow things down. Queries are coming in by email, by phone, by MS Teams, by Slack. Can you talk us through some of the kind of questions you get, just to really ground that?

DO: Yeah, so it’s anything and everything. As we started, we thought it was going to be one thing. Then, as we continue to watch it, how it’s expanded has been really interesting, because we really take an organic approach. We started with what are the questions we know we’ve had historically. We have an intake tool we can go through and analyze, and then we can very quickly create standard generalized answers for high repeat questions. Where do I get a copy of this? What are the terms within an SLA? Where do I find a copy of a support guide? Anybody on the legal team can tell you these things off the top of their head. But if you’re new in sales, or new to selling this specific skew, you may not understand all the nuance within that. So being able to get that tacked down was where we started.

"We have an intake tool we can go through and analyze, and then we can very quickly create standard generalized answers for high repeat questions."

Then we pretty quickly evolved to all the other things. Security is certainly a big area where you don’t need to be an expert, you just need to be able to point to all the great content that the security team has spent many years building and maturing, to effectively be able to get credit quickly for the work that’s already been done. Oftentimes, especially when you’re customer-facing, if you can give a thoughtful and clear answer quickly, you can get past objections that otherwise may bog down negotiations. I’ve been on the other side of those negotiations. If I ask a question of a vendor, and it takes them three days to get back to me for something that should be pretty run of the mill, you start to question what they’re not doing right. So that is where we saw the value, and then we built on that.

Something that generative AI can do is help with negotiation strategies. If you haven’t tried already, go into your favorite gen AI tool and, for a specific company, list out the common objections you hear. I think you’ll be surprised as to the answers the generative AI can give you. Are they going to be 100%? No, but we never went into using this technology assuming that it was going to be 100%. Our initial view was, if we can get it to do 60%, wouldn’t that be incredible? What we’re finding is that it does a lot better than 60%, as long as you stay within the context for which the data was trained.

Naturally, at some point people want to test the bounds of what these chatbots are trained on. So we’ve seen all sorts of questions, and each of those questions is an opportunity to not only learn, but also retrain the content upon which the bots have been trained, such that over time they provide better answers. They’re like real-time FAQs, so if you don’t see the answer, you get to have a second bite of that apple. And if one person has asked the question, human nature means there’s probably ten other people that have the question as well; they just haven’t asked it yet. So it’s been really interesting to see the organic nature of it take off. 

"We’ve got folks on the ground in many countries all over the world, and you simply can’t wait on an answer because of time zone for something that’s low-hanging fruit.(...) That’s where generative AI came in..."

PL: That’s really interesting. There’s a couple of things I want to loop back to. The chart is called Gen AI’s Superpower: Commercial Copilot, which I know is internally the name you’ve given it. You mentioned that you’re dealing with not just legal questions, but also non-legal questions. You’re seen as trusted experts; if Legal says it, then it’s good. That’s kind of the vibe, right?

DO: That’s right. And we can get you to the answer quickly, because a sales rep may close five deals within a longer period, whereas one of the commercial attorneys may do that many in a week. Commercial attorneys have a lot of experience in negotiating contracts. If you actually look at any of the contracts that we’re signing in our businesses, the overwhelming majority of the contents are not legal terms. Just because it’s written down on a piece of paper doesn’t make it a legal term, but some sales reps think that makes it the purview of Legal. We can certainly help and advise, but payment terms, for example, are not legal terms per se. 

PL: Exactly. We’ve listed a few of the sales enablement use cases, or what you call Commercial Copilot. These use cases and things like privacy and infosec are really low hanging fruit for gen AI, because almost by definition they’re already well documented; the content is already there. Similarly for negotiation support, maybe not to the same extent, but a lot of companies have pretty extensive playbooks, and gen AI handles this really well.

DO: Even if you don't have an extensive playbook, you can actually use gen AI to create one. You can ask any of the tools to give you a list of ten questions and sample answers, and have something to work with that you can then tweak into a playbook really quickly.

PL: Yeah, precisely. You can use your existing negotiations to generate that playbook. Looking through this list of use cases, some of it is very much within the realm of Legal, some of it isn’t either. Did anything not work that well? Did you try anything that you now know just isn’t for gen AI?

DO: Yeah. We learned really early on that whether or not your generative AI can consume it, depends on how your data is stored. At its simplest, if it’s text based, it can work really well. If it’s an infographic, the system doesn’t have the ability to process it. Because at the end of the day, the generative AI is connecting words and context to say what is the next word that should be predicted. And if, instead of words, you’re using colors or images to be able to connect concepts, the technology just can’t consume the data. As an example, we’ve got certain content in PowerPoint format, and one of the things that we’ll be doing going forward is maintain not only the PowerPoint, so that humans can consume it, but have text in the notes section of the PowerPoint, that we can then input into the generative AI technology, to allow people to consume that content in both ways. We have a SharePoint site internally, so if folks want to, they can do it the old fashioned way. And if they want to use the chatbot, they can do that as well. But we’re being intentional about having the data in a format that’s easily consumable, because then we can iterate on it over time, see where there’s edge cases and questions we didn’t answer. Personally, I get almost more out of the questions than I get from the tool, because people are much more apt to ask questions; they may actually be afraid to ask someone directly. Being able to aggregate what those questions are and thinking through whether there is a better answer to the one we’ve done, this iterative process is really interesting. Lawyers are really good at being able to come up with arguments. All we need to do now is just write them down, and then we can meet about them as a team offline. What do we want the default answers to be? Not to provide legal advice, but to give a list of ideas that our business people may not have thought about. Because there's often five ways to solve a problem.

PL: I remember you saying this on a call, and it really resonated, how every moment where data is being created, you’re trying to capture it. And you don’t save it for another day, you don’t kick the can down the road. But every moment where there’s a learning or potentially a new negotiation play, you document it. That makes it a lot easier, because you don’t have to generate it all from zero. 

DO: That’s right, and that’s something we started years ago. We were growing so fast, we knew that we had to be able to harden down processes. Taking an almost pedantic look on a weekly basis, saying what came in, why, how did it change, and what was the outcome? When you think about negotiating master services agreements, every negotiation is an opportunity to say, did I have the right assets? Did I have the right arguments? Or do I need to be rethinking what the positions or fallbacks should be? If you’re just thinking about it as a transaction, and the transaction is done, when it comes time to scale your business, all the learnings that have been happening across the team members, they never get shared, they never get improved, and you don’t have any better data at the end of a year of closing deals than you did at the beginning. One of the things that I’m excited to get to, and I know I’m jumping ahead a little, is how do I use this technology to onboard a new lawyer? What are the company positions and fallbacks? We're not going to make that broadly available. For a well trained lawyer on a call to be able to go to their private chatbot and get real-time contextual suggestions is an incredibly powerful potential. Now we’ve got to build it; who knows if it’ll work. But I think that’s one that will pay dividends in the very near future. 

PL: What’s so powerful about it is that it doesn’t have to be delivered in one hit, like training; it’s on demand. 

DO: That’s the key differentiator of this technology. We may give you six hours of training, but you probably won’t remember what happened at four hours and thirteen minutes. I’m probably glazed over and need to get some coffee by that point. The technology allows you to go precisely to what the question is. So imagine this: You’re onboarding an employee, and you have two versions of this technology, one that asks them a list of questions, and another that allows them to experiment with what the answers are. And you can capture all of that. 

PL: If an organization has the ability to get contextual information on demand, when and where it’s needed, the competitive advantage there is absolutely enormous; the agility will be huge. I want to loop back to something you mentioned earlier, about 60% being incredible. This is something we say to prospects, that it won’t automate everything, but even 30 or 40% is enormous. 

"If an organization has the ability to get contextual information on demand, when and where it’s needed, the competitive advantage there is absolutely enormous; the agility will be huge."

DO: Especially with high volume work.

PL: Exactly. If someone offered to take away 40% of my work, where do I sign? I think a lot of people get anxious about an AI being imperfect and unable to cover everything. 

DO: It’s a legitimate concern.

PL: I'd love to hear your take on that and how you approach that. What happens when it can’t answer a question or gives out incorrect information?

DO: That was one of the critical objections we had to get past, before we could launch this thing into the wild. There’s a tool that stops the AI from hallucinating. We’ve been at it for six months now, and in every single instance where we thought we caught a hallucination, it wasn’t a hallucination; it was the trading data that was actually off. So we do a couple of things. First, before anyone uses the tool, we’re very clear that this isn’t legal advice; these are best practices in general information. When you have a specific question that absolutely needs a lawyer, please still call the legal team. We still want to hear from you. But there’s a ton of content that it can provide, and the tool won’t hallucinate; by default it will say, I’m not able to answer that question. In some instances, I thought it should have been able to answer a question, and that’s where we realized we didn’t have our data stored in the proper way. We had the words in there, but they weren’t linked in such a way that the technology could consume it. 

"What we’ve learned is that, the more we’re able to consolidate our data across teams and have specific systems of record, it really starts unlocking some of the more advanced use cases that we hadn’t even thought about when we started down this path."

PL: From our perspective, that’s obviously great to hear. I think the work required to implement a tool like this is really that work on the content. I remember Andy, GC at TravelPerk, at one of our webinars said that he didn’t expect that; he didn’t expect to upload content and then find the gaps and inconsistencies in it. 

DO: It’s incredible how good it is at finding those inconsistencies. We may publish a 50-page document and think it’s all fine, until the AI reveals one little nuance. The root cause of it may be that some other team or department is the source owner of that data, and currently we don’t have connected systems that update in real time. They may have updated their data, and then it’s a telephone game until our data element gets updated. What we’ve learned is that, the more we’re able to consolidate our data across teams and have specific systems of record, it really starts unlocking some of the more advanced use cases that we hadn’t even thought about when we started down this path. 

Interested in seeing GenAI in action? Make a request here.