to search out in companies proper now — there’s a proposed product or characteristic that might contain utilizing AI, resembling an LLM-based agent, and discussions start about the best way to scope the venture and construct it. Product and Engineering could have nice concepts for a way this device may be helpful, and the way a lot pleasure it could generate for the enterprise. Nonetheless, if I’m in that room, the very first thing I need to know after the venture is proposed is “how are we going to guage this?” Typically it will lead to questions on whether or not AI analysis is actually vital or mandatory, or whether or not this may wait till later (or by no means).
Right here’s the reality: you solely want AI evaluations if you wish to know if it really works. When you’re comfy constructing and delivery with out realizing the affect on what you are promoting or your prospects, then you possibly can skip evaluation — nonetheless, most companies wouldn’t really be okay with that. No person needs to consider themselves as constructing issues with out being positive whether or not they work.
So, let’s speak about what you want earlier than you begin constructing AI, so that you just’re prepared to guage it.
The Goal
This will sound apparent, however what’s your AI imagined to do? What’s the objective of it, and what’s going to it appear like when it’s working?
You may be stunned how many individuals enterprise into constructing AI merchandise with out a solution to this query. However it actually issues that we cease and assume exhausting about this, as a result of realizing what we’re picturing after we envision the success of a venture is important to know the best way to arrange measurements of that success.
It is usually vital to spend time on this query earlier than you start, as a result of chances are you’ll uncover that you just and your colleagues/leaders don’t really agree concerning the reply. Too usually organizations resolve so as to add AI to their product in some vogue, with out clearly defining the scope of the venture, as a result of AI is perceived as precious by itself phrases. Then, because the venture proceeds, the inner battle about what success is comes out when one individual’s expectations are met, and one other’s aren’t. This generally is a actual mess, and can solely come out after a ton of time, power, and energy have been dedicated. The one technique to repair that is to agree forward of time, explicitly, about what you’re attempting to realize.
KPIs
It’s not only a matter of arising with a psychological picture of a situation the place this AI product or characteristic is working, nonetheless. This imaginative and prescient must be damaged down into measurable kinds, resembling KPIs, to ensure that us to later construct the analysis tooling required to calculate them. Whereas qualitative or advert hoc information generally is a nice assist for getting colour or doing a “sniff check”, having individuals check out the AI device advert hoc, with out a systematic plan and course of, just isn’t going to supply sufficient of the proper info to generalize about product success.
When we rely on vibes, “it seems ok”, or “nobody’s complaining”, to assess the results of a project, it’s both lazy and ineffective. Collecting the data to get a statistically significant picture of the project’s outcomes can sometimes be costly and time consuming, but the alternative is pseudoscientific guessing about how things worked. You can’t trust that the spot checks or feedback that is volunteered are truly representative of the broad experiences people will have. People routinely don’t bother to reach out about their experiences, good or bad, so you need to ask them in a systematic way. Furthermore, your test cases of an LLM based tool can’t just be made up on the fly — you need to determine what scenarios you care about, define tests that will capture those, and run them enough times to be confident about the range of results. Defining and running the tests will come later, but you need to identify usage scenarios and start to plan that now.
Set the Goalposts Before the Game
It’s also important to think about assessment and measurement before you begin so that you and your teams are not tempted, explicitly or implicitly, to game the numbers. Figuring out your KPIs after the project is built, or after it’s deployed, may naturally lead to choosing metrics that are easier to measure, easier to achieve, or both. In social science research, there’s a concept that differentiates between what you can measure, and what actually matters, known as “measurement validity”.
For example, if you want to measure people’s health for a research study, and determine if your intervention improved their health, you need to define what you mean by “health” in this context, break it down, and take quite a few measurements of the different components that health includes. If, instead of doing all that work and spending the time and money, you just measured height and weight and calculated BMI, you would not have measurement validity. BMI may, depending on your perspective, have some relationship to health, but it certainly isn’t a comprehensive measure of the concept. Health cannot be measured with something like BMI alone, even though it’s cheap and easy to get people’s height and weight.
For this reason, after you’ve figured out what your vision of success is in practical terms, you need to formalize this and break down your vision into measurable objectives. The KPIs you define may later need to be broken down more, or made more granular, but until the development work of creating your AI tool begins, there’s going to be a certain amount of information you won’t be able to know. Before you begin, do your best to set the goalposts you’re shooting for and stick with them.
Think About Risk
Particular to using LLM based technology, I think having a very honest conversation among your organization about risk tolerance is extremely important before setting out. I recommend putting the risk conversation at the beginning of the process because just like defining success, this may reveal differences in thinking among people involved in the project, and those differences need to be resolved for an AI project to proceed. This can even influence how you define success, and it will also affect the types of tests you create later in the process.
LLMs are nondeterministic, which means that given the same input they may respond differently in different situations. For a business, this means that you are accepting the risk that the way an LLM responds to a particular input may be novel, undesirable, or just plain weird from time to time. You can’t always, for sure, guarantee that an AI agent or LLM will behave the way you expect. Even if it does behave as you expect 99 times out of 100, you need to figure out what the character of that hundredth case will be, understand the failure or error modes, and decide if you can accept the risk that constitutes — this is part of what AI assessment is for.
Conclusion
This might feel like a lot, I realize. I’m giving you a whole to-do list before anyone’s written a line of code! However, evaluation for AI projects is more important than for many other types of software project because of the inherent nondeterministic character of LLMs I described. Producing an AI project that generates value and makes the business better requires close scrutiny, planning, and honest self-assessment about what you hope to achieve and how you will handle the unexpected. As you proceed with constructing AI assessments, you’ll get to think about what kind of problems may occur (hallucinations, tool misuse, etc) and how to nail down when these are happening, both so you can reduce their frequency and be prepared for them when they do occur.
Read more of my work at www.stephaniekirmer.com
