Frameworks for Rapid Piloting

After a decade of designing, testing and scaling new products, I’ve learned what it takes to deliver a successful pilot (also sometimes called a “test” or “experiment”)— one that resonates with customers enough to build an organisation’s confidence to deliver new and novel concepts to market.

A well-executed pilot requires a well-calibrated team that understands the subject-matter, market, their customers, and their executive stakeholders.

A successful pilot requires a team to strike a delicate balance, to nimbly and cost-effectively deliver a “believable” experience to customers, and quickly deliver “believable” evidence to executives. Off-kilter pilots often yield data that is too poor quality to make effective design and strategy decisions, or end up fatiguing the organisation by sucking up resources due to over-engineering. Pilots that drag on simply lose executive interest and miss the market opportunity.

After a fair bit of trial and error, I’ve found the following two frameworks useful to plan the pilot and calibrate the team.

Framework 1: The Customer Line of Believability

This requires working with a team, internal stakeholders and external partners to deliver to the threshold of “believability” to conduct an effective market pilot. Discerning what can be done with “smoke and mirrors” manually versus what must look and feel like the “real thing” depends entirely on what is being tested, timelines, resources and other factors.

The Customer Line of Believability: A threshold, along the axes of Fidelity x Risk

Framework 2: The Executive Evidence-to-Intuition Ratio

Launching a suboptimal product can cost an organisation millions of dollars , hurt the brand and kill careers — there’s a lot at stake. Most executives rely on a blend of intuition (based on wisdom built up over years) and hard evidence when it comes time to make decisions— even in organisations that are reputed to be data-driven.

A successful pilot recognises the balance between evidence and intuition, and requires the team have a keen ability to adjust on-the-fly based on what the pilot reveals day to day. Exactly how much data and evidence is required to make the leadership comfortable with a decision to go/no-go dictates how long the pilot should run, how to report, which measurements must be taken, and who must be involved. When the ratio is too high in either direction, it’s a signal that piloting is likely not the right approach.

The Executive Evidence-to-Intuition Ratio (E:IR)

If you’re interested in running better pilots, and want to meet folks from the the community facing similar challenges to you, I’ll be in San Francisco on May 15th to run a Rapid Piloting Masterclass. It draws on ten years of working with fast-paced clients from my time at frog, Studio D and most recently as the Head of Service Design at Yoma Bank. Look forward to seeing you there.

Finally, if you’re interested in how other people run pilots, and are willing to share some of your own experience, please take the Rapid Piloting Survey.

Cross-posted on the Studio D Radiodurans Medium Channel. Thanks, as always, to Jan Chipchase.

Introducing Yoma Bank's Design Team, Our First Annual Report

The financial services sector in Myanmar is nascent. Estimates of the population who are banked clock in at around 25%, with fewer than 50% served formally by groups such as MFIs. With a population that is still >60% living in rural areas, serving more than just the urban, top of the pyramid requires a drastically different approach to banking. I’ve been fortunate to work with one of the nation’s leading commercial banks as they tackle this challenge, aiming to redefine banking to truly serve the mass market.

A typical sight: A customer waiting to make a deposit of cash into his bank account. Though much of Myanmar is still cash-based and systems are closed / siloed, interoperability between institutions is in-progress.

My work with Yoma Bank began in late 2017, our budding team tasked to create an end-to-end design and product management practice. Our efforts have focussed on creating new products around the unique needs of the aforementioned group of unserved / underserved customers, in addition to improving efficiency and construct of existing bank products that meet this remit.

In 2 years, our team has grown exponentially. We have brought half a dozen products to market and piloted countless others. We’ve each deconstructed and rebuilt our own understanding of what financial services are, through the brilliant hypothesis generated by our BI practice, an increasing wealth of accessible data on the market, and a regular cadence of engaging in-person with our customers (and prospective customers).

Our team member Ko Maung using Zapya to receive an .apk from research participant. Photo by Kyle Becker.

The below report is a snapshot of who we are, how we work and what we’ve made to date. It highlights some of the challenges we’ve faced along the way, good habits we’ve built and glimpses into our culture. We hope it proves a nice introduction to where we are, what we’ve accomplished, and where we still have room to grow.

Part of our team, bright and early, during a team trip to Shan State, March 2019.

Special thanks to those who have shaped my views on and practice of leadership.

100 Questions for the Young Creative

Last year, we at Studio D spoke with creatives across the design spectrum in order to better understand the dynamics of our industry and the kind of studio we wanted to create. Agencies large and small, corporate design teams, freelancers, consultants, veterans and college students all shared their perspectives. As part of this process we generated an in-house list of over 100 questions that we wish we’d thought to ask earlier in our careers.

Designed both for self reflection and to cut through hiring process gloss, they start out innocently enough but quickly cut to the chase. There are eight categories:

  1. Who are you, and what are you bringing to the table?
  2. Why do you want to work at ______?
  3. What does the company stand for?
  4. Who’s really in charge, and how is the business run?
  5. Why are people joining or leaving?
  6. What kind of work do they (and will you) do?
  7. What is the company culture?
  8. What are you worth?

We also cued up a number of Wild Card Questions. Deliver one of these out of left field, and cut through the theatre of an interview.

Submit your own questions via the Studio D Site.

Shared under a Creative Commons Non-commercial License.