Problem Solving: Learning from other disciplines

Richard Feynman (he doesn’t need an introduction) was a consummate problem solver. When asked about his problem-solving techniques, his colleague Murray Gell-Mann (a Nobel Laureate himself) defined the ‘Feynman Problem Solving Algorithm’:

1. Write down the problem
2. Think very hard
3. Write down the answer

While this was partly in jest, this to me captures why it is important to look to disciplines like physics for inspiration when it comes to building the capability to solve the hard problems. As someone once said, ‘All problems in physics start off as word problems and end up with mathematical solutions’.  Physics – as we all have learnt in high school – deals with the very large and the very small. And so, students of large scale phenomena (e.g. astrophysics) are taught to deal with vast amounts of data with a very poor signal to noise ratio. On the other end, students of quantum physics need to deal with very small amounts (if at all) data.

Needless to say, it is more about the mindset than anything else. Let’s take a look at each of these steps.

Write down the Problem

This is by far, the most important step. We all have been in situations where we have chased a problem down a rabbit hole without even first getting clarity and alignment on the question itself. And so, the absolute first step is to well and truly imagine the problem, write it down and then noodle on it until you have nailed down the exact question. Here’s an excerpt from Albert Einstein¹ on how he started thinking about special relativity. You cannot help but marvel at his ability to ask a bold question – the rest, as we know, is history.

“…a paradox upon which I had already hit at the age of sixteen: If I pursue a beam of light with the velocity c (velocity of light in a vacuum), I should observe such a beam of light as an electromagnetic field at rest though spatially oscillating. There seems to be no such thing, however, neither on the basis of experience nor according to Maxwell’s equations. From the very beginning it appeared to me intuitively clear that, judged from the standpoint of such an observer, everything would have to happen according to the same laws as for an observer who, relative to the earth, was at rest. For how should the first observer know or be able to determine, that he is in a state of fast uniform motion? One sees in this paradox the germ of the special relativity theory is already contained.”

Think very hard

This sounds simple – even facetious. In reality, this is the essence of problem solving. For the truly wicked problems, we are going to need think very hard: and not just try to get away with the standard methods and sub-optimal solutions. If you need inspiration, look no further than quantum physics. One of the more storied examples from recent times is the search for the Higgs boson and the experiments at CERN’s LHC (there is a fantastic documentary, ‘Particle Fever’ that chronicles the journey). While I don’t pretend to remotely understand the physics, what stood out in the body of work is a key problem that these (very, very) clever people were trying to answer: how do you estimate smooth probability distributions from a limited number of samples? Formally, this is called the ‘density estimation’ problem – which seeks to answer 2 fundamental questions in a world of small data sets, where the standard large data-set approximations do not apply:

1. What is the best estimate for the underlying probability distribution? Can you estimate with a limited number of observations?
2. What do other plausible distributions look like? Or, how does one distribution stack up against all other distributions?

If this sounds familiar to the arc of the narrative of the past few weeks, your suspicion is well founded (!): this is exactly what I have been coming back to again and again: if you are responsible for analyzing clinical trials and come out with insights based on a limited set of observations trickling in or if you are a risk officer in a bank who has been tasked to predict the changing risk profile of your customers as the post Covid-19 unravels on a daily basis, you would probably thinking about these very problems. You might want to explore an implementation of DEFT (Density Estimation using Field Theory) that came out from the quantum physics world (https://suftware.readthedocs.io/en/latest/)

And so, the point that I have been coming back to time and again: step outside the narrow lanes of the standard analytical techniques and talk to your physicist friend. You are likely to learn a trick or two from them on how to build mathematical models from first principles (example a couple of blogs ago) to figure out how to make sense from say, small data.

Write down the answer

It is not good enough to just building a model – either a statistical or a mathematical model. What is just as important is to be able to quantify why the recommended model is the best one under the circumstances. And as Einstein famously said, a theorist can go wrong in two ways²:

1. The devil leads him by the nose with a false hypothesis (for this, he deserves our pity)
2. His arguments are erroneous and sloppy (for this, he deserves a beating)

The same holds true for the empiricists as well. Define a confidence interval and as data comes in, be open to challenging your models and refining them. In other words, as any scientist will tell you: ‘there is never THE answer. There is an answer that is good only until data disproves it’.

And as for the solution itself, Richard Feynman added this critical condition: you should able to explain your solution to a 6^th grade. If you cannot, either the solution is not right or not good enough. And in our world, we can modify this to: can you convert all the math gobbledygook into insights and recommendations that are actionable and consumed by business? And so, while it is all good to play around with different algorithms to improve the quality of customer risk, you should be able to take the solution and make them really useful.

Further reading:

1. A highly readable biography of Einstein by Walter Isaacson. I am sure there are several of them
2. Brilliant Blunders by Mario Livio. Fascinating book that covers the blunders made some of the biggest scientists of our times. Great insights into how the best can make mistakes and how individual egos come in the way.
3. Several videos on Richard Feynman’s lectures on Youtube. You need to watch a few to realize how special he was: there have been (and will be) great physicists but few had the sheer enthusiasm and the ability to explain concepts like Feynman