This post was authored by Caroline Hane-Weijman, David Edwards and Francisco Hui
Through APIs, we can access the power of billions of dollars of functionality built by others to radically accelerate our productivity, business expansion, and customer offerings. API-driven development is understandably a primary focus for today’s enterprises and start-ups to drive innovation and speed to market.
Application Program Interfaces (APIs) define how software communicates with other software systems. This is similar to how user interfaces define how a user accesses functionality of an application, but the end user for an API is a computer system, not a human. APIs create lego blocks of software functionality. They can be used by developers internal to your company to leverage discrete, common functionality as well as by developers external to your company to use for their own business and user needs. Often, development teams creating an API assume that because their system does not have a graphical interface, development won't require the kind of product and design practices you would expect to use to create an application for human users. In fact, that's not the case at all! When developing APIs, the same practices & tools for lean product development apply but there are inherent differences that need to be considered for how you apply these practices.
We are Caroline, David, Francisco—a cross-functional product development team that worked together at Pivotal Labs, a product development consulting company. Over the course of the next four blogs we will share tactical learnings we encountered when designing & developing a product offering where the interface was a set of APIs for external developers to consume in order to meet the business and user needs of their company.
Key Takeaways
We’ll start with the TL;DR. We’ll elaborate on these and other learnings throughout the blogs.
-
Treat product and design roles and their practices with as much significance as you would a product with a presentation layer. The same lean and user-centered design disciplines apply to ensure you are iteratively building a product valued by business & users
-
Don’t assume things take less time because you don’t have a presentation layer. They don’t.
-
Consider all levels of users when defining features and designing the API experience. The business and their users that the API features provide value to/for, the developers working for said business that are using your APIs and the computer system that ultimately consumes your APIs.
-
The ultimate consumer of your API will be a computer system, not a human. This means that some usual intuitions like minimizing the number of steps in a workflow don’t apply; it’s more important to make the building blocks as intuitive as possible for developers to build this computer system.
-
Define your versioning strategy carefully as developing APIs is like developing a language; changing APIs can break the computer system that is built on top of it
-
Consider creating either a client library for the API, or a dummy application that consumes the API, in parallel with the API itself. This allows you to create quicker feedback loops to test the usability of your APIs.
Importance of a Balanced Team
We’ve come across many teams working on APIs where staffing a Product Manager and Designer has been deemed unnecessary. The same lean and user-centered design principles apply to products without a presentation layer as one with a user interface; API products are still solutions to business and end-user needs. A Product Manager and Designer will help you validate the priority of the business problem you are solving and whether your API Design is going to solve this problem in a desirable experience while minimizing waste and keeping the team productive.
A Product Manager (PM) should lead the team to discover and deliver an API product that creates meaningful value for the business and users, with minimal waste using lean principles. For APIs, the set of businesses & users may also be very complex and PMs will need to understand the needs & business impacts to facilitate decision-making in service of shipping successful features. Like with all products, PMs should articulate the product vision & strategy, establish an outcome-oriented product roadmap that translates to a prioritized backlog, establish and track measurable objectives for your API usage, de-risk product direction, and help ship software. All in favor of effective communication and keeping the team productive & motivated!
Designers are too often pegged in a design agency role to make things “pretty”. However, designers contribute a set of capabilities far beyond visual design in order to develop a desirable, useful & useable experience. These user-centered design (UCD) capabilities include conducting exploratory user research, ethnographics, pattern mapping, prototyping, conducting user validation sessions, mapping information architecture, etc. UCD activities are just as valuable for APIs as for other products with presentation layers and designers play a critical role in focusing on these essential UCD capabilities for an API team.
Since the direct users of APIs are developers, we found the developers on our API team could easily slip into the habit of making decisions based on what they would want themselves. We quickly discovered that every engineer on our team had a different take on what a “great” RESTful API should be and we were stuck in never-ending design discussions. Having a Product Manager & Designer lead the team through objective processes of user exploratory interviews, design validation sessions, and facilitated team discussions, we were able to move forward more quickly with more confidence that our decisions reflected our users’ needs and not our own assumptions.
API Design
While an API benefits from the same user-centered design practices as any other system, the form and focal points of design are different. In designing a graphical user interface, your team would consider things like visual language, consistency, information layout, intuitiveness of controls, and usability. All of those have analogs in an API. Additionally, there are design considerations that are unique to the nature of APIs (like versioning) and principles from graphical interface design that do not apply (like minimizing the number of steps in a workflow).
We followed a RESTful API architecture. REST is an architecture style for designing networked applications that provides a convenient and consistent approach to requesting and modifying data. REST APIs use HTTP requests to GET (retrieve), PUT (update), POST (create), and DELETE data. It simplifies each call into a “single-action lego block”. While this approach has become standardized, we interestingly discovered that our team members had different ideas for how to design RESTful endpoints. We ended up having discussion fatigue after endless back and forth on what a “proper” RESTful solution would be. Based on our experience, we suggest quickly choosing an endpoint design principle, creating mocks, getting feedback from developers, and then iterating, using user feedback to break design deadlocks.
In summary, the key considerations of API design that determined the usability of our product were:
-
Resources, Requests & Responses
-
Endpoints
-
Error Responses
-
Versioning
We will explain these in more detail. To help anchor the discussion, we will use the following business use case: A product team working for a ‘Payment Company’ is building an application that processes payments, developing features that easily allows online retailers (e.g., ‘Sock Company’) to take recurring payments from customers (‘Sock buyer’). When the Sock Buyer purchases a pair of socks online, they provide personal information and payment information to Sock Company. The APIs we developed are used to communicate the relevant information from Sock Company’s software system to Payment Company’s software system in order to process the payment.
Resources, Requests, and Responses
In order to communicate between software systems, an API essentially functions as a collection of questions/commands (requests) and answers (responses) that allow you to access the data (resources) to accomplish what you need.
For example, Sock Company needs to interact with our API to do tasks like collecting a consumer’s payment information on sign-up, and updating that consumer’s subscription information so they can be billed on the appropriate interval. Our API would then include resources like consumer, subscription, and payment method.
It isn’t always obvious what resources you should include in your API. Is it better for our API to have a subscription resource or a payment resource? Both? Is consumer or customer the better name for the person placing a subscription? Your resources make up the language available for users to ask questions and send commands, so you need that language to be clear, unambiguous, and powerful enough to accomplish their needs.
When a user requests information about a resource, your API sends them a response. For example, if Sock Company sends us a request asking about one of their consumers, we might answer with this response:
{
“username”: “johndoe”,
“email”: “[email protected]”,
“emailConfirmed”: true,
“subscribedOn”: “2019-01-01”
}
How much information do you put in a response? Does the same information occur in different kinds of responses? For example, the response above doesn’t say anything about johndoe’s registered payment methods. Should it, or should users have to ask about payment methods specifically?
There are costs and benefits to both, so the right answer will depend on your users’ specific needs. Regardless, the responses your API sends comprise the information layout of your interface, so they need to be consistent, intuitive to your users, and structured in a way that helps them accomplish their goals.
Key learnings when designing resources, requests and responses:
-
Consistency is key.
-
When users can both create and later fetch a particular resource, responses should echo all the original inputs along with additional system generated fields. (such as its creation date, etc.)
-
For future iterations, taking away fields is worse than adding fields, as it hampers backwards compatibility. When in doubt, start with less fields.
Endpoints
An endpoint is an address to which users send requests to retrieve information. Most of the time, an endpoint corresponds to a particular resource. The way you design your endpoints also indicates the hierarchy & structure of your resources and data, similar to how you choose to organize folders & files in a file system. For example, Sock Company might ask for a list of John Doe’s registered payment methods by sending a “GET” request to this endpoint:
https://our-payment-api.com/consumer/johndoe/payment-methods
The consumer resource is the “top-level” resource and payment-method is “nested” under consumer/johndoe in order to access John Doe’s payment information.
However, this isn’t the only way to structure such an endpoint. You could instead provide an endpoint, where payment-method is “non-nested” and is a “top-level” resource:
https://our-payment-api.com/payment-methods/johndoe
Here, users would indicate whose payment methods they want by providing the consumer’s username as a separate parameter in the request, rather than making it part of the endpoint itself.
Again, there are costs and benefits to both approaches. Embedding information in the endpoint shapes how users think about the structure of your data by forcing them to think about certain resources in terms of others. If done well, this can help users build an effective mental model of your system—but in other situations, the constraints it introduces result in inconsistencies that hinder user understanding. In our project, we ended up using the non-nested approach, because our users often needed to query payment method data in different ways. (For instance, rather than getting all payment methods for a given consumer, they might need to get all confirmed payment methods regardless of who they belonged to.) The non-nested endpoint allowed us to accommodate all of these queries in a consistent way.
Just like a button on a web page, it needs to be intuitive to a user what will happen when they send a particular request to a particular endpoint—and you shouldn’t trust that your own intuition matches that of your users! The only way to know for sure what a user expects an endpoint to do is to ask—hence the need for user research.
Key considerations for designing endpoints were:
-
Top-level resources vs nesting sub-level resources.
-
# of nested resources for one endpoint.
-
Consistency in endpoint paths.
-
Whether the endpoint is intuitive, flexible, and accessible.
Error Responses
Error responses arise when you’ve made a request where receiver is unable to provide you with the response you are expecting. The error response includes an HTTP status code, which specifies the type off error, and the response body. Error response code design can become intricate as different error codes signify both different types of errors and different severity levels of each error. It is important to keep in mind that a computer system will be reading the error code initially and therefore the error code should signal the behavior of the system and how you wish to respond as a business. For example, is there something wrong with the system (urgent) or with the format of user input (not urgent and can be handled with messaging?) We opted to start with generic error code reponses and return everything as a 400 (‘Bad Request’ the server cannot or will not process the request due to an apparent client error) except for authorization errors which we specified as 403. The intent was to gather feedback from developers and iterate on the granularity of the error response codes.
Versioning
Humans are adaptable—if you decide to roll out some changes to a user interface, as long as they’re intuitive and clear, human users will figure out what to do in a minute or so. But humans are not the direct users of an API—computer systems are. And computer systems are not nearly that smart.
This means that in terms of versioning/product iteration, developing APIs is like developing a language. On the spectrum of developing a language (versioning extremely important) to internal app for internal users (versioning less important), APIs are closer to that of developing a language.
Developers will be using APIs like lego blocks to build their own computer system. Changing a block in the structure they’ve built could break the structure.
For example, if you release a version of your API that includes an emailConfirmed field in the response for customers, one of your users might build a system that pulls that field from that response and uses it for some logic. If you then decide to put that field in a different response and remove it, your user’s system will continue looking for it in the original location, and break when it isn’t there.
On the other hand, systems don’t usually break if there are fields in a response besides the ones they use, so adding fields (starting to provide something that, worst case, no one uses) poses a lower risk for breaking changes than removing fields (taking away something that a developer might be relying on).
In conclusion, start with less and add more as feedback dictates. Collapsing two endpoints is easier than breaking apart a single endpoint and adding fields from responses is easier than removing fields. Start with fewer fields in responses and highly focused endpoints.
Putting It All Together For a Useable API
Just like any other product, making the product easy to use is a key consideration of design. But since the direct “user” of your API will be a computer and not a person, the nature of “usability” is a little different.
For example, consider this workflow: A customer visits the Sock Company website. They select socks, check out, create an account, save profile, and payment information, and then click buy.
The API calls involved in this workflow likely do not correspond directly to the customer’s actions; “Sock Company” developers may use multiple API calls to create a profile, save payment information, and trigger a transaction with the payment API business. That’s fine.
The developer is not the one clicking on every step for every customer—they build a system that automates this. Therefore, while you might try to minimize the number of clicks needed for a particular workflow in a human-driven interface, that intuition doesn’t necessarily apply to an API—having to make multiple calls is not necessarily slower. It’s more important to make the building blocks as intuitive as possible to help developers to build their automated system.
Overall, developers agreed that focused, single-action, intuitive, consistently designed endpoints were desired. As mentioned above, APIs are like lego blocks for developers. Small, focused endpoints give developers more flexibility for how they want to design their own system/interface. We originally had an endpoint that consolidated creating a consumer profile, payment, subscription, and triggering a transaction all in one call. This was poorly received. So we broke each action up into separate endpoints.
In this blog post we shared our learnings for foundational principles & concepts of designing and developing API products. In the next blog, we will share in further detail the process we took to design the APIs to meet business & user needs.
Hello! We are Caroline, David, Francisco – a cross-functional product development team that worked together at Pivotal Labs, a product development consulting company. Pivotal Labs work with our clients as an integrated team, sitting side-by-side, to build & deploy digital products while enabling our clients to learn lean, user-centered, agile software practices that they can use as key capabilities on an ongoing basis within their organizations.
This blog is one of our 4-part blog series on Designing & Developing APIs:
