Ask most developers to explain how to create an API, and they’ll probably give you an answer that focuses on writing a specification and/or implementation.
That answer wouldn’t be wrong, exactly. Specifications and implementations are certainly among the core components of APIs.
However, designing and implementing modern APIs requires attention to much more than these components alone. Due to the way in which APIs have evolved over the past decade – as well as the importance APIs have assumed not just for software development, but also for functions that span all parts of the business – today’s developers need to think far beyond the most basic API components to build and manage APIs that are optimized for the needs of all stakeholders.
Allow me to explain by walking through the various components and processes that go into building a successful API today.
What is an API?
You could spend some time debating what an API even is. Some developers think of APIs as a specification, or a set of rules, that govern how applications or services can interact. Others focus more on the implementation of the API, meaning the code that drives the interactions.
From my view, however, the simplest way to think about APIs is to treat them as a way to perform an action on a remote application. APIs let you list, create, update, and remove data between applications, remotely over the network.
From this perspective, it doesn’t matter much which API protocol you are dealing with, how you define your API specifications or how you implement the API. Those factors can all vary from one API to another. But what all APIs have in common is that they enable interactions between applications, remotely. The better your API can do that, the better the API is.
What goes into a modern API?
Historically, APIs were mainly a technical resource used by developers to expose the functionalities of an application or service, so that other applications could connect to them.
Today, however, APIs play a much broader role within organizations. They’re still technical, of course, but they drive business functions – such as delivering and monetizing applications and services, allowing B2B service integrations with partners and customers, and enabling the interoperability of different lines of businesses – to name just a few common business use cases for APIs today. Note, too, that there are businesses – commonly known as “API-first” companies – whose sole product and business model is based on APIs.
Given the absolutely vital role that APIs play in modern businesses, there is much more at stake surrounding the security, performance, reliability, and usability of APIs than mere remote procedure calls (RPCs). When your API is the lifeblood of your business as opposed to a tool your developers use, you need to invest much more heavily in optimizing the API in all respects, not simply ensuring connections between applications.
What that means in practice is thinking holistically and comprehensively about everything that your API needs to function optimally at all stages of the lifecycle – from design and code through to production and operations.
So, what, exactly, should you be thinking about when creating an API? Let’s dig deeper.
API design, development, and testing
Designing and building an API should never be an ad hoc process. Instead, businesses should manage the design, development, and testing processes systematically and deliberately.
They should start by determining which API standards they want to follow. Popular options today include the Open API Specification (OAS) and the Async API specification. They can then define API contracts and API specification by talking to the various consumers of the APIs.
Next comes API linting, validation and testing. These processes should focus not just on ensuring that the specification conforms with the standard, but also on validating the API specification for security and performance. Although these considerations can be managed at later stages of the API lifecycle as well, developers should begin addressing them during the design and development phases so that APIs are secure and reliable from the start.
Planning for API documentation, versioning and deprecation should be part of the design process, too. Although documentation, version management and deprecation are not essential to the development of API specifications and implementations, they are critical for effective API management over the long term.
Discovery, cataloging and API portals
Reusability should be a priority for every API. When a team within an organization builds an analytics API, for example, different lines-of-business within the organization should not need to rebuild the same analytics API; instead, they should be able to reuse and enhance it.
For that reason, discovery and cataloging of APIs is key when creating APIs. Furthermore, this cataloging should result in a Universal API Catalog, which displays all the APIs available to a business, regardless of the lifecycle stage at which each API currently stands. For example, the catalog should include APIs that are in the development/testing phase and pre-prod/prod phase. It should also list APIs that are fully configured, plus brownfield APIs that are discovered based on network traffic. And it should delineate APIs that are internal only from those that are external or partner-facing.
The Universal API Catalog also allows – through role-based access management – a mechanism to expose each API to external stakeholders using multiple API portals, which customers and partners can access to view all the APIs. In addition, developers should consult and collaborate with other external stakeholders, like their user communities, to gain their perspective on how the API should work and be used. A key point to note is that when promoting an API from internal to external consumption, developers should be careful to ensure that there are policies in place to make APIs performant, secure, and reliable in any context ( more on that later). Collaboration with both internal and external stakeholders of APIs is an essential component of modern APIs, and API portals play a central role in this collaboration.
API runtime planning
Developers and API owners have a plethora of choices, and hence decisions, to make when it comes to how their APIs will run within runtime environments. Beyond choosing a protocol (like REST, GraphQL or gRPC), they must plan for API translation and routing and caching, and they must determine which combination of API gateway form-factors they want to use. API gateways come in different form factors, such as Edge API gateways, which sits at the edge of an enterprise and primarily offers an API translation layer; Distributed API gateways, which are more developer-friendly and are placed at the ingress of an application; or Micro-API gateways, which are also known as service meshes, and sit right next to each workload as a sidecar.
In addition to deciding which form factor to use for runtime purposes, developers must choose how to manage identity and API access control, how to perform single sign-on and whether to have end-to-end encryption and mutual TLS.
Deciding how to deploy APIs to production environments is a complex but important consideration, too. How will you perform progressive rollouts of APIs? What about canary releases? How will you push out updates to APIs after releasing the initial version? These are all questions that developers and API owners need to answer before they can implement an effective API runtime.
API policies and compliance objectives
After addressing runtime considerations, organizations and API owners next have to ensure that modern APIs align with a variety of internal and external policies in order to be acceptable for the business.
They need to set and meet API Service Level Objectives (SLO) policies and determine what corrective enforcement action (such as auto-scaling) needs to be adopted if the SLOs are violated. They must also determine whether there are any data compliance, data leakage, data privacy and/or data sovereignty concerns at stake for their APIs, and how to address them using API security, segmentation, and Data Loss Prevention (DLP) policies. Conforming with API security best practices, including but not limited to those detailed by organizations like OWASP, should also be a priority for modern API development.
Even API monetization policies, which define how the business can generate revenue from an API, is a factor that API teams and owners will need to assess and react to. They need to decide who should have access to the API, under which terms and whether access policies will have to evolve over time (due, for instance, to changes in pricing or revenue models).
API observability, metrics, and analytics
It’s impossible to guarantee the success of your API if you can’t measure how well the API meets core requirements surrounding operations, performance, and security. That’s why it’s critical to collect and analyze metrics about API operational patterns, usage, and security risks.
Developers and API owners need first to define which metrics they want to make available, then ensure that the necessary data is exposed within their API specification and implementation. They must also determine exactly how to collect and analyze observability and security metrics based on factors like the API protocol and gateway they use.
It’s important, too, to have a plan for reacting to API performance and security issues through logs, events, and alerts for diagnostics and troubleshooting. Having the data to know that something is wrong is not very valuable if there is no actionable plan for responding to the problem.
Multiple stakeholders require close collaboration
APIs cannot be implemented in isolation, by a single person or team. Instead, there are numerous stakeholders who should be involved in the entire end-to-end development process. For that reason, there is a need for close collaboration across the entire lifecycle of APIs for all stakeholders.
Collaboration should include, at a minimum, engaging with other technical stakeholders within the business – like API architects who can assist in API design governance, the Platform Ops teams who will help to support applications that depend on APIs and the DevSecOps teams who will help secure them.
The entire process starts with the developer experience. The priority should be to ensure that developers have the full perspective and feedback they need to go through the iterative cycle of discover → design → develop → test → document → share → deploy → observe → fix with the most effective and efficient experience possible. And as this API development lifecycle continues, there are architects who need to participate by creating and collaborating on design governance rules and SLO templates.
Beyond this, there are also Platform and API Operations teams who should collaborate at various stages of the process by managing and monitoring the actual API runtime environments and ensuring that all the required policies are enforced. All this needs to happen with as little disruption to the current behavior of the various stakeholders as possible – which means there has to be a proper workflow and dev and ops tool integrations.
The bottom line: In modern API implementations collaboration between all relevant stakeholders is of paramount importance. Otherwise, you risk creating an API that may make sense to the developers who implement it, but that other stakeholders struggle to deploy, support, and use in an optimal fashion.
API-design-first versus code-first approaches
Any discussion about APIs is not complete if we do not talk about the two approaches to API development that are popular today: The API design-first approach and the code-first approach.
The API design-first technique is a more recently introduced approach, and it’s gaining popularity especially amongst API-first companies that offer API-as-a-product, as well as companies that develop internal APIs for reusability by multiple internal lines of businesses. This method centers on ensuring that APIs and contracts are discussed and agreed upon at the very start of the API development process. Only after the API has been designed according to these discussions, is the API implemented.
The code-first strategy is a more traditional approach to implementing APIs typically for applications that have already been written. It’s a way to expose application functionality using APIs in situations where the application was not originally written with APIs in mind. In such a scenario, the API is often an afterthought. That said, although this approach is not ideal, this is often the most straightforward solution for organizations and teams that have legacy applications to expose functionality to partners and customers through APIs. This approach is also adopted by developers who want to quickly bring their applications to market, and think about exposing APIs later.
Both approaches have their respective merits and use-cases. When choosing to implement your API, it’s important to consider which of the two approaches makes most sense for your situation.
Still, regardless of which approach you take, the performance, security, reliability, and usability of the API should remain your top priority. That doesn’t change, regardless of how you go about implementing the API.
Conclusion: The brave new world of end-to-end API Lifecycle management
Historically, creating an API was as simple as writing a specification and implementation and calling it done.
Clearly, that approach is not sufficient in today’s API-first world. APIs that excel in all respects, and that drive success for the modern business, are subject to a long and complex set of end-to-end requirements – ranging from choosing the right specifications based on stakeholder needs, to planning the optimal runtime environment, to ensuring that APIs are easy to observe and secure.
Without heeding these requirements, you will end up with an API that either doesn’t serve the needs of its consumers in an ideal way, or that is not usable, reliable or performant. Worse yet, you might produce an API that exposes the organization to data exfiltration, compliance violations or other serious risks. Given that there are more than 24,000 individual external APIs (and several times that number of internal APIs) available today, the result is that your API will probably end up being ignored – which defeats the purpose of creating an API in the first place.
