Continuing from my previous post, this is an attempt to define metrics for measuring the quality of Agentic AI Systems
#81: How do we evaluate Agentic AI Systems? Part-1
I recently met with the Head of Data Science at one of the largest media conglomerates and as often happens these days, evaluation and quality of AI Agents came up. One of the most important problems he has been trying to solve is how to evaluate the quality of AI agents, and once in production,... Continue Reading →
#80: Measuring Human and AI Agents
Since 2023, we have been hearing doomsayers talking about the coming AI Apocalypse that will take our jobs, run economies etc. And they keep pointing to the impressive performance gains and the reasoning capabilities of the frontier models. Meanwhile, those of us in Enterprise AI continue to be frustrated by the temptation to treat these... Continue Reading →
#79: Demand Forecasting and Language Models
Demand Forecasting is a widespread application for Machine Learning. Almost every enterprise, across various sectors, keeps investing in this area. None more so than industries with physical Supply Chains, where it is estimated that a 10-20% improvement in forecast accuracy can translate to a 5% reduction in inventory costs and 2%-3% increase in revenues (McKinsey).... Continue Reading →
#78: Model-free vs. Model-based learning
Over the holidays, I read A Brief History of Intelligence by Max Bennett, a fantastic book that traces the evolution of intelligence from worm-like nematodes (microscopic worms) all the way to us humans. The author does an absolutely brilliant job of the journey by breaking it into five key breakthroughs starting from the minute organisms... Continue Reading →
#77 Systems or Models
It was exactly 2 years ago (Nov,2022) that I had written about Software 2.0 (term coined by Andrej Karpathy) and I had argued that we should expect a rapid and meaningful shift from systems performing deterministic tasks (think ERP, RPA etc.) to learning, adaptive systems where the "source code comprises of a 1/neural net architecture... Continue Reading →
#73: The Value of Data
In enterprise data circles, it is a well-known statistic that less than 30% of companies claim to have been successful in becoming a data driven organization. What is not so easy is to figure out why companies have such a hard time getting actionable insights from data – in my experience consulting with many companies,... Continue Reading →
#72: Software 2.0 in the Enterprise
In 2020, Gartner identified Composability as a key factor to be resilient and agile in an uncertain and rapidly changing environment. Fast forward to 2022 and as most companies are bracing down an economic slowdown, it is becoming important for CIOs and IT leaders to accelerate the adoption of technologies that help business functions to:... Continue Reading →
#70: Time as a fundamental feature
Last week, I got into a discussion on Forecasting with a friend. He works with operational data flowing from server farms and in his line of work, he is interested in use-cases that go beyond the standard use-cases like load forecasting, what-if scenario analysis for failures to more nuanced (yet critical from an SLA point... Continue Reading →
#69: Small and wide data
Those of us who have spent enough years in Data and Analytics know that this industry, more than most in tech, is rife with jargon and ever so often, the irresistible urge to spin some new buzzword (to some, this is how the industry continues to maintain the hype and they wouldn’t be wrong. But... Continue Reading →
#78: Model-free vs. Model-based learning
Over the holidays, I read A Brief History of Intelligence by Max Bennett, a fantastic book that traces the evolution of intelligence from worm-like nematodes (microscopic worms) all the way to us humans. The author does an absolutely brilliant job of the journey by breaking it into five key breakthroughs starting from the minute organisms... Continue Reading →
Celebrating the learning and curious mindset 