Pivotal Labs applies the Google Ventures Sprint process with a remote design team and suggests how the framework can be improved.
At Pivotal Labs, if Pivots are not allocated to a project, they can work on developing their skills in an area relevant to their job. Similar to other companies such as Zappos, we call this off-project time “The Beach.” It is a great opportunity to learn a new programing language, reflect on past projects, or explore new tools and techniques in product design and product management. Recently, four Tokyo Pivots decided to use their beach time to test the Google Ventures Design Sprint to learn another approach to rapid and collaborative digital product design.
A Design Sprint is a week-long series of activities aiming at gathering insight from the team around a problem in a particular market, prototyping a solution, and testing that prototype with real customers. The desired outcome is to de-risk a costly idea. Instead of building a full-featured product, you test a facade to evaluate how useful it will be once released. You can find an exhaustive set of instructions on how to run one in the book Sprint published by the fine folks at GV (like Jake Knapp) who pioneered the format. All you need to start is an organization with a bold, risky idea in mind for solving a particular problem and a small, committed team with stakeholder support.
We found these ingredients in a team of volunteers at Dell EMC (Pivotal’s primary investor). For the past 3 years, Patricia Florissi, the Vice-President and Global CTO of Sales has been spreading the word about her project the World Wide Herd (WWH). WWH is a distributed analytics solution that helps data scientists analyze hard to move data scattered in various locations. By “hard to move,” we mean situations in which bandwidth cost is too great to regularly stream petabytes of data or when such data is legally required to remain in its country of origin for privacy reasons.
The WWH team was delighted by the amount of learning that could be achieved within just five days of work. At Labs, we now have a better opinion of remote work and found that a Design Sprint could be an interesting approach for some teams.
“It was such a pleasure and learning-intense activity that has literally re-steered the direction of the project. We truly got some very very tangible outcome and we are planning how to best follow through on all the ideas and options that came out of it.”
Patricia Florissi, Global CTO & VP of Sales — Dell EMC
What we learned about remote work
Key Takeaways:
- Focus is what matters for both co-located teams and remote teams
- You can leverage technology in creative ways to increase remote engagement
- A remote sprint might make more sense for distributed teams and products
WWH is currently being built by a talented team of volunteers, with people located around the world, including Israel, Egypt, India, Canada, and the US. This distribution of personnel was a key obstacle to achieving shared understanding. The Design Sprint process advocates for co-location for just this reason, and it is a core Pivotal Labs practice as well. This experience helped us understand that communication shorthand and focus were the two ingredients needed to maintain speed of learning. Of those two, focus is the most important factor and the good news is that it is possible to achieve a comparable level of focus in a remote project.
Before the sprint started, the biggest hurdle we envisioned was whiteboarding. Indeed, the whiteboard is the star of several activities in a design sprint (“The Map” and “Storyboard” in particular). Since we use the idea of “draw and tell” all the time at Labs, we thought we needed something better than Google Slides.
After a bit of research, we decided to purchase a Kaptivo. Kaptivo is a Kickstarter-funded whiteboard-mounted camera that enables remote participants to a conference call to see the whiteboard through the web, almost as if they were in the room. While it’s a rather young technology, we found Kaptivo to be highly effective at delivering what it promises. We even started saying “write it on Kaptivo” instead of “write it on the whiteboard”. Kaptivo was designed to be wall-mounted, which unfortunately did not fit our need for our whiteboard to be mobile; as you can see in this series of photos, our hacked-together mounting solution grew pretty elaborate. We also feel this wonderful product would benefit tremendously from being more accurate at reading sticky-notes and paper drawings taped to the whiteboard.
Our biggest learnings were about the parameters that enable continuous remote work to make sense. Companies such as Automattic and Github are famous for their remote team structure; they even actually consider it a competitive advantage. Since the WWH team is split across several countries, VOIP and time difference are already part of their day-to-day operations. So while doing an off-site for a Design Sprint might have been valuable, demonstrating that some of the activities are manageable remotely made it easier to apply afterward on a normal day. We now feel more strongly that a team that is culturally structured around a distributed product can benefit from this kind of remote workshop.
A word of caution though: while our particular experience with remote collaboration far exceeded our expectations, there were still many hurdles in our way. Dropped calls, voice clarity issues, and the absence of video made our interactions less fluid and enjoyable. Our remote participants lamented their inability to join our famous ping-pong breaks.
What we learned about running an effective design sprint
There are four main things we think the Sprint book could do a better job at explaining. In particular these criticisms should be relevant to practitioners of Lean Startup and user-centered design. While the Design Sprint is a battle-tested framework that has been run hundreds of time by the inventors as well as other teams within and outside Google, we think it is biased towards teams who are not familiar with lean and agile. This is a great example of designing several levels below the expected proficiency of your users. We are confident this is overall a very successful method but we believe it could do a better job at showcasing generative research, collaborative sketching iterations, balanced product teams, and sustainability.
1. Research about who your user is beforehand will make your tests more valuable
“I haven’t run into that problem, but some of my data scientists were working with something like that recently.”
We heard similar statements from three out of our six total user interviews. What this told us was that our interviewees had only second-hand experience, and thus lacked the context needed to draw meaningful conclusions. It showed us that — while our target user may exist — they weren’t the people we were talking to. We needed people who touched the problem directly, but the client (WWH) did not yet know who had this problem. In short, we needed to first identify who the actual user of the product was going to be, before we could test any solutions.
This led us to believe that in order to get the most out of a design sprint, which is inherently evaluative in its focus, we would first need to do generative research to discover clearly who our user might be and what problem we might be solving for them.
This is the case with many new projects: either the user persona is not yet validated, or the problem is not yet validated. In our case, we had a clear data-science problem to solve, but only a fuzzy grasp of who might need such a solution.
We think this quote from Google Venture’s Daniel Burka highlights one of the ways the inventors of the framework handle this problem:
“Without the right problem, you are unlikely to be successful in a design sprint […] sometimes we get really sophisticated and do a bunch of user research first and try to identify where the problems are so the masterclass, if you’ve got a full-time user researcher on your team, is to go deep on research and really know the problem space extremely well […]”
Daniel Burka, Google Ventures — High Resolution, Episode 7 (30:28)
We can see how with a validated user and problem a Design Sprint yields better results than with just a rough idea for each. If you are working on a nascent and complex technology, we highly encourage you to start validating your underlying assumptions with generative user interviews a few weeks prior. When starting a new Pivotal Labs project, we start by understanding the target audience, the problem we are trying to solve for them, and the underlying value we are generating for the business.
Daniel goes on to describe how to test leadership’s definition of the problem, and ensure that they can find users who really experience it. We found that this step can still be risky, but suggest starting by asking the following questions:
- In detail, what is the concrete scenario in which the problem expresses itself?
- Who is suffering from the problem directly, today?
- Who is the most impacted by this problem?
If you can find a problem and a user that satisfy these criteria then you might be in business. Otherwise you may need to dig a little more first. Either way, the more confident you are in this foundation, the more confident you can be in Google Sprint’s results.
2. Not talking to each other and making many unvalidated decisions individually is draining
At Pivotal, we use a series of workshops — including scenario writing and quick design prototyping — in order to come up with collaborative solutions rapidly. These activities involve writing, drawing, and critiquing together, just like day two and three of a design sprint. Spending the better part of day two drawing on our own was very frustrating and tiresome, though. We believe day two is designed this way in order to address organizations or teams that don’t create space for each other to properly surface different opinions effectively. This way, people can feel their voices are heard, and the highest paid person doesn’t immediately kill what could be a good idea. This separation makes sense, but inherently forcing people to come up with a full-fledged solution in isolation creates greater risks for miscommunication along the way:
- You might go down your own rabbit hole
- It creates more difficult conversations later on (deciding is hard)
- It pits peoples’ solutions against one another
- It is very tiring
We would suggest instead starting with a user scenario, make sure everyone on the team agrees with it and treats it as a contract. We would then do multiple design studios based on this scenario, where everyone could iterate a solution on their own. After each iteration, we could review, which would de-risk people going down their own rabbit hole, and create more collaboration between the team. This also makes it easier to take more breaks, which is essential to not burning out by the end of the day.
A proposed alternative agenda for the second Sprint day:
3. Agree on a more collaborative alternative to the Super Vote
On day three of Google’s Sprint process, before moving onto the storyboard, the team is asked to dot vote on the best ideas across all designs and then select the best individual sketch. This creates a heat map of the team’s collective opinion as to which is the best solution. The Decider is then tasked to allocate “Super Votes” that will decide what the winners of the sprint are. The three super votes can be divided across multiple sketches or combined.
This type of techniques can useful because it ensures the person who has the most power over the product is responsible for choosing the what solution to test. This helps prevent misalignment later on as the executive sponsor might be more tempted to kill a solution they have not chosen. The most important benefit is to allow the team to move forward with an experiment without spending too much time deliberating.
As a team, we did not like this process because it gives the opportunity for conflict when the Decider obviously goes against the will of the team. Instead, we would prefer giving that power only in certain situations. For example, if the votes are evenly split, it would fall on the Decider to make a hard choice in service of moving the team forward, the same way a PM might sometimes do. We recognize this might not be possible in organizations that do not practice Balanced Team. However, we think Sprint is a great platform to promote how each role contributes to positive long-term outcomes.
4. Find patterns and decide what to learn next
Sprint is rather fuzzy on how the data for user interviews should be collected and the patterns found. The idea is that by having the whole team observing the interviews and going through the sprint questions at the end it should be easy to build consensus. We would rather have a more disciplined combination of note taking and research synthesis. This enforces more discipline and maintains objectivity across the team.
If you combine the lack of structured learning with the exhaustion from the week, it is easy to imagine the paralysis at the beginning of the following week. One way to address this is to perform an agile retrospective during the last hour of the sprint when the restitution of the interviews is done and the patterns have been identified. The “retro” format will enable the team to celebrate victories and discuss openly the things that need to be improved. The outcome should be an action list for the following week with clear owners of each action.
Conclusion
Remote Design Sprints are a great way for distributed teams to be exposed to user-centered design and lean experiments for the first time. They provide tools to prioritize and test assumptions in a timely fashion. If you want to incorporate its benefits in a more sustainable way to your product and design practice, we recommend tweaking the formula a bit by:
- Adding generative research to understand the problem better upfront and avoid waste
- Making the research outcomes clearer and actionable
- Allowing some activities to be more collaborative (sketching) and less confrontational (no super vote) while still moving fast
- Using the right technical tools to ensure a smoother communication like Kaptivo
This article was written by Canyon Boak (Product Designer) and Hadrien Raffalli (Product Manager) from Pivotal Labs Tokyo. Huge thanks to Yui Anzai, Doug Blumeyer, and Joanna Beltowska for the time they spent editing it.
Change is the only constant, so individuals, institutions, and businesses must be Built to Adapt. At Pivotal, we believe change should be expected, embraced and incorporated continuously through development and innovation, because good software is never finished.
Lessons Learned Running Our First Remote Design Sprint was originally published in Built to Adapt on Medium, where people are continuing the conversation by highlighting and responding to this story.
