“It is all about execution, stupid!”, said a wise man. And it is nowhere truer than in AI implementations these days. A vast majority of AI inspired projects start with a lot of fanfare: budgets are lined up, a prototype is funded, a few data scientists go to town with the latest and coolest AI technique and then just as quickly, reality bites. The project makes it way to the corporate detritus – and adds to the frustration of the executive leadership.
The story doesn’t need to unfold this way. What we need is a solid execution mindset – one that is committed to ‘cross the river by feeling the stones’, to use a term coined by Deng Xiaoping.
Here are some lessons that I have learnt over the years:
- Start with outcomes: a specific, measurable business KPI needs to be the guiding north star. At all stages, keep going back and asking the hard questions:
- How does the project impact the KPI? This has to be a measurable improvement. It is not enough to create feel-good qualitative outcomes
- What are the intermediate goals? This is much harder than you would think – more than anything else, it requires an upfront definition of measurable goals.
- Is there a measurement process defined? A quantified method (e.g. Pre-post/Test and Control) needs to be administered at the process level to rigorously keep track of the improvement, if any.
- Process changes: To deliver the outcome, identify the processes that need to be re-engineered or new ones that need to be introduced. This is a step that is, sadly enough, ignored too often. Remember that at the desired outcomes will only come from process improvements – and unless there is clarity around embedding the AI solution into the business process, the AI solution is doomed. This is where the Art and Science come together – more on this later.
- Embrace an MVP mindset: Analytics solutions need to operate the edges of the business – which is where the volatility and uncertainty is the highest. This demands a rapid prototyping and fail-fast mindset. The quest to develop the ‘most accurate’ model often becomes an end in itself – a trap that data scientists fall into all the time. Instead, it is far more important to operate with an MVP mindset:
- Define a lower threshold for the model accuracy metric. This is usually driven by the business context.
- Implement a POC/POP (Proof of Concept/Proof of Production) with the model. Field test it.
- Setup a continuous improvement process – essentially, tweaking around the model (either by adding more features, feature engineering; or try out new model techniques).
- Always follow a champion/challenger approach to constantly evaluate a portfolio of models. In every sprint, push the best performing model into production. Measure the lift – rinse and repeat.
Needless to say, there is lots more to it – but if you start with these guidelines, you are on your way. And you have reduced the chances to driving the AI spend to the ground. Will talk about the importance of process improvements later.