Designing and Executing Valuable Strategic Experiments
Part 3 of the "Navigating a Path to Success in the Age of AI" series
This series focuses on Collaborative Intelligence technologies, techniques, and their business applications. Strategic Experiments can be used in any situation characterized by medium to high strategic uncertainty.
Whether company executives realize it or not, most companies’ approaches to strategic management are derived from Operations Research (OR) techniques that first appeared in the business arena in the late 1940s1 and were popularized by academics and management consultants through the 1990s.
OR creates mathematical models of processes, such as strategy development and execution, to optimize process design and process-related decisions and outputs. It has led to the “one best way” approach to strategy-making and informed the development of valuation models (such as Net Present Value and Internal Rate of Return) used to make strategic investment decisions.
Optimization-driven approaches only work in business environments with little uncertainty. To optimize a decision, decision-makers must be able to identify, in advance, all viable prospective courses of action as well as the probability of success and expected payoff of each alternative.
If strategists think only with their spreadsheets, how much use of imagination can we expect?
From “Possibility Thinking: Lessons from Breakthrough Engineering,” by Robert Friedel and Jeanne Liedtka, Journal of Business Strategy, Vol. 28, No. 4, pp. 30-37.
In highly uncertain business environments, this is not possible (even though many executive teams and their advisors fool themselves into thinking it is.)
In business environments dominated by novelty (such as those faced by many entrepreneurs and those impacted by “game-changing” technologies such as AI and collaborative robotics), traditional approaches to planning are effectively useless. Plans can be optimized and valued only if companies have the experience and data required to inform their optimization efforts. This experience and data do not exist if companies have never done something before.
Highly uncertain or novel business environments are risky. Managing this uncertainty and novelty requires a strategic management approach that reduces uncertainty and risk over time and begins to capture the experience and data required to guide decision-making in these environments.
Strategy Development and Execution Under Uncertainty
Such an approach exists. A high-level overview is depicted in Figure 1.2
Organizations facing a high degree of uncertainty should iteratively design and execute a set of experiments (described below) to discover promising strategies and to gather information that reduces environmental uncertainty.
Once they have identified a prospective strategy, they must iteratively design and execute another set of experiments to gather evidence that this course of action is promising. If they find evidence invalidating the strategy, they must return to the Discovery stage.
The Discovery and Validation stages comprise an experiment-driven search for answers that reduce uncertainty and risk exposure. Once a valid path forward has been found, the organization moves from searching to executing.
Initially, the organization should proceed with a small-scale implementation since more will be learned as the implementation proceeds. Once the organization has successfully implemented its strategy on a small scale, it should have reduced its uncertainty and risk exposure enough to warrant large-scale investment in the strategy. (If it cannot implement the strategy, it must begin searching for a different or modified strategy.) It can then scale the implementation across the entire organization (or the applicable scope) in ways that ensure profitability and effectiveness.
Designing, Executing, and Learning from Strategic Experiments
The practice of strategic experimentation is deep and broad. It can be applied to strategy development and execution, business model design, product/service/process design, and other business disciplines facing medium to high uncertainty or novelty. As such, it would take one or more books, not a few paragraphs, to present the entire body of knowledge.
Figure 2 depicts a high-level overview of our approach. (Please note that the descriptions use the word “idea” as a generic representation of any concept related to one of the many business disciplines to which experimentation can be applied.)
The iterative process proceeds as follows:
Ideate: Creation must precede experimental testing. This step identifies things such as prospective business strategies, pricing strategies, or product concepts.
Prototype: In this step, the idea is prototyped. For a product concept, the initial prototype may be only a brief description and some key perceived benefits for a prospective target market. Subsequent iterations of the same idea would produce more realistic and complete prototypes.
Assess: In this step, the idea/prototype is tested with prospective customers or other stakeholders to determine whether the idea represented by the prototype is valuable.
Decide: Based on the data obtained in Step 3, the idea can be modified (by running through the Design loop again, with the Ideate step focused on improving the original idea), killed, or tested by continuing to the Test loop.
Hypothesize: In this step, a hypothesis is developed to test a key uncertainty associated with the idea. For example, when Reed Hastings was developing the idea of Netflix, he hypothesized that the DVDs the company planned to mail to customers wouldn’t break so often as to render this distribution channel ineffective.
Experiment: In this step, an experiment is designed to test the hypothesis developed in Step 5. Continuing the example, Mr. Hastings drove around the country and mailed DVDs to himself.
Learn: In this step, the experimental results are examined to gain insights that reduce uncertainties or risks associated with the idea. In Mr. Hastings’ case, almost all the DVDs arrived in good condition. He gained the insight that the US Mail was a viable distribution channel for Netflix.
Decide: Based on the insights gained in Step 7, additional experiments can be run (execute the Test loop again), the idea can be modified (execute the Design loop again) or killed, or the idea can proceed from experimentation to small-scale implementation.
Operations Research groups were formed first by the British military during World War II. Academic courses applying Operations Research to business first appeared in 1948.
Adapted from The Four Steps to the Epiphany, by Steve Blank.