This post was co-written by Alex Basson, Eric Cipra, Michele Molino, Liam Morley, and Peter Piazza.
Since the No Child Left Behind Act was signed into law in the United States in 2001, states have assessed the performance of public schools largely on the basis of standardized test scores, with the results having significant consequences for students and teachers in schools that receive low ratings. Unfortunately, standardized testing offers a very limited and often biased view into school quality. To address this shortcoming, the Massachusetts Consortium for Innovative Education Assessment (MCIEA) spent years developing an evidence-based School Quality Measures (SQM) framework and an accompanying dashboard to provide a fair and comprehensive picture of school performance.
Over the course of several months, MCIEA and VMware Tanzu Labs partnered together to reimagine the existing dashboard that leverages this SQM framework to offer district leaders, instructional leaders, and other groups a fairer and actionable view into how their schools are performing. Notably, MCIEA partnered with VMware after a previous effort with another company did not ultimately generate a minimum viable product. Throughout the engagement with VMware, we took advantage of parallels we noticed between educational, agile, and extreme programming (XP) practices. A few of the most impactful moments will be covered in this post, including building trust, approaching feedback with a growth mindset, and delivering value quickly—all of which helped the Tanzu Labs product team succeed where the previous software team had failed.
Building trust
Without trust, it’s difficult for a group of people to feel safe and comfortable around each other. This is true both in the classroom, where a lack of trust can lead to behavioral problems, and the workplace, where that same lack of trust can reduce collaboration and erode progress. Accordingly, it’s crucial to actively work to nurture trust in any team environment. This team was no exception. During the engagement, trust was built by working through a Discovery and Framing (D&F) process, collaborating frequently, and pairing.
Discovery and Framing
Discovery and Framing (D&F) is a process in which the product team has open discussions to align on the key direction, explore possible problems, prioritize those problems, brainstorm solutions, and prioritize solutions. Throughout this engagement, everyone on the team was engaged in the D&F process. The team became what is known as a balanced team—a multidisciplinary and autonomous team made up of product management, engineering, and design—ensuring that multiple perspectives were included for each critical early decision.
After aligning on goals, the Tanzu Labs product team prioritized creating personas so the team could focus efforts. They decided to prioritize district leaders, who manage multiple schools in a district, as the primary persona since they were already key users of the existing dashboard and found value in it. Through a number of user interviews, the team learned that district leaders really wanted to see the School Quality Measures data displayed in a clearer way with more data disaggregation opportunities over time (e.g., by student subgroups, such as grade level). In addition, they wanted to be able to easily share the data with their colleagues and school leaders for inclusion in school improvement plans.
Our team’s primary persona
In parallel, the Tanzu Labs engineers conducted a technical investigation to determine whether to leverage the existing dashboard’s codebase, build a new dashboard from scratch, or go with an off-the-shelf solution. Technical information in hand, the team came together to have a frank discussion about which path forward made the most sense. Ultimately, they decided to build on top of the existing codebase since off-the-shelf solutions were not sufficiently customizable, and they saw value in reusing the domain models that had been built as part of the existing codebase.
This entire D&F process allowed the team to build a transparent and common understanding of what to build and why, because everyone on the team was engaged every step of the way. The team leaned on every balanced team discipline to make sure we made the best possible decisions with the information available.
Frequent collaboration
At the end of the D&F, the Tanzu Labs product team held collaborative prioritization sessions, which allowed everyone to share their prioritization perspectives and to gain insight into what to build and why. In one prioritization session, for example, an MCIEA team member called out, “If something felt difficult, [our previous partner] would try to convince us not to do it. It doesn’t feel that way with Tanzu Labs.” By building trust through transparency into product decisions, Tanzu Labs enabled this team member to feel confident that they could tackle the hard problems.
It’s worth noting that the team, though made up of folks from Tanzu Labs and MCIEA, operated as one during this engagement rather than as client and vendor—it’s one reason why we’re writing this post from a joint perspective. There was never a moment where the Tanzu Labs team came up with recommendations in a silo; rather, both Tanzu Labs and MCIEA worked in tandem to incorporate everyone’s perspectives to make the best decisions.
Our team’s road map
Pairing
As we began building software in earnest post-D&F, the team continued this model of transparency through the XP practice of pairing, wherein a Tanzu Labs person collaborates directly with an MCIEA person for the day and solves problems together, whether that be writing code, designing features, or writing user stories. Think of this as akin to pairs of students working together on one assignment during class.
Partway through the engagement, MCIEA hired a new engineer, whom the team had the opportunity to onboard. After experiencing pairing, which was new to him, our new engineer remarked, “I could tell that this is a healthy team when I joined. I’m so fortunate to pair and learn alongside this team of engineers.” Not only does this dedicated collaboration lead to better learning outcomes, but after working together, it goes a long way in building real working relationships and trust.
Trust is an excellent place to start, but it wasn’t the only thing we were after. The confidence to make hard decisions—and to embrace change if we uncover new information that contradicts what we thought to be true—is just as important. And without a growth mindset, gaining confidence can be very difficult.
Side-by-side views of our developers pairing on work remotely
Approaching feedback with a growth mindset
Growth mindset is not only one of the measures within MCIEA’s SQM framework, it is also one of the key ways Tanzu Labs teams, in particular, approach work. A growth mindset is the belief that skills can be nurtured through learning. In the classroom, this can manifest in such ways as celebrating successes, embracing the word “yet” (e.g., “You’re not a math person yet), and normalizing productive struggle. During our engagement, we practiced a growth mindset most notably with our feedback opportunities such as design critiques, experimentation, and speedback.
Design critique
Periodically we conducted design critique, in which the whole team gathered to provide feedback on upcoming design prototypes. We hosted sessions where the product designer would present a prototype design of a feature and ask the team to provide feedback in the form of roses, buds, and thorns. Roses were parts of the design we liked, buds were new ideas the design sparked in us, and thorns were areas that we’d consider changing. This collaborative approach allowed us to instill confidence in our designs by getting fast feedback from different perspectives. And it also aligned with our growth mindset values: we are not our work, and we have the ability to make our work better.
Example of a design critique
Experimentation
Additionally, we embraced “learning by doing” by treating mistakes as learning opportunities. One of the ways we encouraged this practice was by having separate staging and production environments. The staging environment was where engineers could deliver code that the rest of the team can see before reaching end users. Having this space gave engineers the ability to move quickly and make changes to the code without fear of inadvertently impacting real users, and it allowed product managers and designers the opportunity to provide fast feedback loops on working software.
Speedback
Ongoing feedback loops are important not only for products but also for our personal and professional development. Our team regularly practiced speedback, where we set time aside to give peer-to-peer feedback in a speed dating format. It was an excellent opportunity for us to very quickly learn about our strengths and areas for growth. And on an interpersonal level, it also allowed us to quickly address those habits that maybe rubbed someone the wrong way before they impacted our overall team health.
Finally, while building and maintaining trust, growth, and confidence are absolutely important, by not having a focus on delivering value, we run the risk of missing the point of what it means to be a product team entirely. It’s no surprise, then, that with our trust and feedback loops in place, we were able to get into a groove of quick and sustainable delivery.
Delivering value quickly
The concept of “delivering value quickly” can have many readings and, in some cases, be corrupted into “delivery above all.” For Tanzu Labs, delivering value quickly is something we embrace in order to get feedback from users (so we can iterate), reach working software as soon as possible while recognizing that work is not life (and therefore not something we do at all hours), and improve the lives of our users. There isn’t a perfect analogue in education, but the flipped classroom—where students get valuable hands-on time with materials in classes and engage with the lecture outside the classroom—is similar, as is the amazing agility teachers have demonstrated in making remote learning work during the ongoing COVID-19 pandemic. During our engagement, we delivered value quickly with agility, user stories, and recurring “collab time” (short for collaboration time).
Agility
There were numerous moments when we had to adjust course from a planned direction in order to chase value. Agility allowed us to make the right decisions because we embraced change and flexibility. We were never stuck with our decisions or our path forward.
For example, the dashboard requires administrative data about each school to provide the most robust quality insights, but this data is something each participating district could provide only through time-consuming manual data extraction. As we learned more about how to automate this process, we realized it could be very difficult to do—something that initially led us to prioritize other work above it. However, through open discussion, we realized that having this data was so impactful that we decided to steer away from “easier” work in order to make incremental movement toward direct data access.
Peter Piazza, project director for the MCIEA SQM work, said, “The [agility] of our workstyle in software mirrors the kind of 'flexibility but with structure' kind of approach that often is necessary in work with educators.” With this common working style, we were able to seamlessly reprioritize the backlog as needed, get and act on feedback from educators, and handle new information in stride.
User stories
The ease of adapting to change was also supported by the way we write user stories. User stories are small and testable. By breaking up requirements into small pieces of work that deliver value to users, we were able to deliver more quickly but also pivot quickly. Our engineers were never working on large, unchangeable buckets of work; that approach not only creates waste but also slows the pace at which we can deliver value to users.
It’s worth noting that while a team is generally more likely to deliver more stories if the stories themselves are smaller, we explicitly avoided using “number of stories” or “story points delivered” as a metric because it is very misleading. This “delivery above all” mindset mirrors a learning environment that is so focused on test scores it may overlook the equally important socio-emotional elements of learning. MCIEA wants a different environment for schools—one that values process and collaboration above simple numerical measurement—and we didn’t want to replicate that system in our work together.
Scheduled “collab time”
One particularly interesting way we set ourselves up to deliver quickly was by blocking out time on our calendars for at least one hour per day of “collab time.” This block was our virtual equivalent of rounding up the team for an in-person mob session on any important or new topics on which we needed to achieve a fast decision. In several cases, this time allowed us to learn about something the previous evening, discuss it at the team stand-up the next morning, offer potential solutions, decide, and move on.
Conclusion
The workplace, like the classroom, can be an interpersonal minefield, and there are many ways it can descend into an environment of distrust, little feedback, and limited value. Through focused attention on building trust, improving feedback by being curious and open to learning, and delivering value quickly, it is possible to avoid these pitfalls. Throughout this engagement, we demonstrated the importance of these healthy team practices. Although the specific strategies look different depending on the context, these practices reflect both positive vendor-client relationships and also replicate the kinds of meaningful learning experiences that can happen when schools are free of the narrow limitations of test-based measurement.
Learn more about MCIEA and its mission, and be sure to follow them on Twitter.
