Agile Know-How Magazine • Fall 2017
Hypotheses to be validated
Customer Hypotheses Do we truly understand the motivation, needs, and pains of our customers?
Solution Hypotheses Does our solution solve the problem? Is it used? Is it accepted?
Channel / Pricing Hypotheses Is the business model viable? Have the projections and the distribution hypotheses been validated?
Demand / Creation Hypotheses Have the messages and the marketing assumptions been validated?
Market and Competition Hypotheses Is the market reacting according to your expectations? What about your competition? Any legal challenges?
Only through short cycles of hypotheses and validation do we really understand the market, the product’ s value, and its users. They allow teams to make better product management decisions. Organizations need to support Product Owners in validating their hypotheses as quickly as possible.
It may sound like common sense, yet many organizations find it difficult to track the impact of their team’ s work. And as it is easier to track the cost of a team, it becomes natural to make investment decisions based on cost more than ROI.
To efficiently validate hypotheses, organizations must help their Product Owners.
Tracking the impact of work done For any type of product, Product Owners will want to validate if and how it is used, and by whom. Mainly, they will want to know if the work done produces the projected value( the hypothesis) and what impact the work had on the organization’ s metrics.
Several metrics models can help organizations get started. Scrum. org suggests a series of metrics for its evidence-based management practice. The metrics are intended to measure the impact of the transformation, including Product Agility. They can be used to validate what is most important for your organization and define ways to measure the impact of work done.
Another model introduced by 500 Startups is often used in Lean startups: AARRR or the Pirate Metrics! The model suggests to track every product touch point, as per the customer’ s journey. For example, using this model, you could measure the impact of a new press release on SEO, and follow it through the full customer life cycle.
The organization may have to adapt some accounting and customer relationship management practices to allow monitoring and be transparent about its numbers.
Validating each hypothesis in itself To gain the knowledge we seek through hypothesis validation, Product Owners must be able to validate one hypothesis at a time. This poses a few organizational challenges.
Traditionally, organizations aimed to create“ big bangs” where, in a single day, they would do press releases, press conferences, and interviews, start ad campaigns, and deliver a large product increment. The goal was to get word of mouth in a nondigital environment. Today, you may want to consider a more granular approach where you can validate the impact of each hypothesis. You can find that a message works better in a market and less in another, or you can find that a feature is used for a different reason than what is shown in an ad. Each finding is a hint toward a better market / product fit.
To achieve short validation cycles, teams need to have their organization’ s support regarding investment in enabling technology and technical debt payment. Organizations should review those costs with the end benefits in mind. For instance, to receive rapid market feedback, some teams may want to invest in continuous delivery technology, while others might want to invest in architecture refactoring.
Product Owners may propose new tools, such as A / B tests and feature toggles to shorten the feedback loop or to limit the test to a small portion of your market. As an organization, you may want to review your team’ s needs and support upscaling of such tools.
Finally, you can use prototyping to test some hypotheses. They can be quick, using only paper and pen, or more complex using a semi-functional interface. While prototyping is a great way to quickly validate hypotheses, it is recommended to remove the prototype once the test is completed. If the solution is validated, build it according to your quality standards. Should you make additional tests or add new prototyping elements over the existing prototype, you may run into some of the following issues:
• As the prototype still carries assumptions that can be wrong, building new tests over it increases the risk of not validating your hypotheses correctly.
44 agileknowhow. com