#49: Problem Solving: Figuring out when to stop

Imagine this: you are going to the theater for a movie (in a post-covid world!) and you enter the parking lot. You have to make a choice as you drive towards the theater: do you pick the first open spot that you see (which means you have to walk longer and might miss the opening credits?!) or do you keep driving, looking for a closer spot (note that you don’t really know if there is a spot that is closer to the theater) with the risk being that if you don’t find a better spot, you will have to turn around and come back, losing more time and burning more gas? In other words, when do you stop looking and settle on a decision? Turns out that this is not a trivial problem. I have lost count of how many times we end up finding a spot closer to the destination after parking – Murphy’s Law is universally applicable in parking situations!

The ‘Optimal Stopping problem’¹ – i.e. choosing a point in time to stop looking for a solution and making a decision has been studied in mathematics. There are several examples (most famously, the Secretary Problem²) – in fact, we run into this almost every day. From not-so-consequential choices like the parking one to more important decisions. If you are selling a house: do you take the first bidder that comes along? Or do you wait? If you do wait, how many bids do you want to turn down before, knowing that the bidders are not likely to come back? Or in work context: if you are trying to hire someone for a role in your team, how long do you keep interviewing people? At what point are you better off to make the offer once you have interviewed the best candidate so far and thereby, losing out on a better candidate that you have not yet interviewed? And just as importantly, do you have an objective way of understanding the trade-offs? There are several dimensions to think about: in the case of selling the house: what is the economic cost of a delayed sale; how is the market sentiment moving; what if the pool of buyers is shrinking and so on. Similarly, in case of hiring, how long can you afford to wait to fill the position; how do you objectively rank the different candidates; what if the candidates you have already interviewed are no longer available and so on.

Problem Solving: goal seeking

A couple of weeks ago, I had talked about ‘perfect being the enemy of the good’. Problem solving usually involves a similar dilemma:

1. How do you know if you are progressively moving towards the right solution? In other words, what is the goal you are seeking?
2. How long do you iterate and look for the right answer? In other words, what is the optimal stopping point when you go ahead and pick the solution recommendation and run with it?

As we know, these questions are even more important in the current business environment where problems usually start with a very fuzzy question and as you navigate through the problem space: the question itself evolves, and along with it, the goal. And almost always, there is a time constraint – you are operating under a time-constraint and have to make a move sooner than later.

Say you want to achieve a BHAG (Big Hairy Audacious Goal – one of the many concepts from Jim Collins³) – e.g. one of the banks we have worked with wanted to ramp-up their digital sales from 2% to 15% in a span of 2 years. This calls for a series of programs – and for each one of them, you will need to ask the above questions.

How do we go about doing this?

As always, this requires a combination of art and science and I don’t purport to give a prescriptive checklist or any formula of sorts: what I do submit are a few pointers that can help teams navigate:

1. Understand that you are dealing with a feedback loop ecosystem: This is so intuitive that we almost always miss it: too often, teams tend to start with the goal of defining an optimal solution, without taking into account that they are dealing with an adaptive system (some have even called an organization as an organism). Every decision that this taken influences the environment (e.g. customers, competition etc.), which in turn impacts the system itself. A Systems Thinking mindset opens up the teams to the reality that in many cases, there is no pre-defined equilibrium ‘end-state
2. Don’t get sucked into the optimization loop: it is all about incremental improvement: This follows naturally from Systems Thinking: for a constantly evolving problem space, it is not the best idea to look for an optimization function and instead, challenge teams to derive solutions that drive incremental improvements and can ‘learn’. Thus, instead of designing a digital campaign to maximize the customer response rate to a new credit product, it may be better to design a series of campaigns with multiple levers – e.g. frequency and timing of outreach by customer segment so that the teams can adapt based on how the segments are responding to the campaigns.
3. Incrementally learn: define small steps, test and iterate: Approach every customer interaction as an opportunity to learn: design and execute a series of experiments to test out alternative strategies and learn from them. This feedback loop mechanism serves to take away the pressure of divining a program that achieves the optimization goal during the design stage. And more importantly, offers a framework to iteratively seek ‘intermediate optimal stopping’ points.
4. Use early iterations to calibrate: For the brave of the heart, do take a look at the solution to the optimal stopping problem⁵. The solution framework is as follows: use the first ‘N’ iterations to calibrate and get a baseline of what ‘good’ looks like. And from the’N+1’ iteration onwards, pick the first solution that performs the best on the calibration metric. Likewise, if you are running experiments or trying out model iterations, use the first ‘few’ to really understand the right evaluation metrics. And once you have done that, you might want to go with the first one that performs the best (trying to figure out what the ‘few’ is a different matter)

Side note: there is a classic example for the Optimal Stopping problem: the marriage problem⁴ – should you propose to the first person you meet or keep looking? And if so, how long? Anyone who has seen the Indian arranged marriage system in action should relate to this problem!

Further Reading:

1. The Optimal stopping problem: https://en.wikipedia.org/wiki/Optimal_stopping
2. The Secretary Problem: https://en.wikipedia.org/wiki/Secretary_problem
3. Jim Collins BHAG: https://www.jimcollins.com/concepts/bhag.html (I wonder if he had someone from India in his research team who came up with this acronym!)
4. Knowing when to stop: https://www.americanscientist.org/article/knowing-when-to-stop