As cloud native concepts and adoption take hold, many enterprises are now considering and implementing ways to achieve the primary objective of cloud native technology: enabling engineers to make significant changes to systems easily, frequently, and confidently.
More and more enterprises are recognizing that cloud native technologies, such as Kubernetes, can indeed serve as the foundational infrastructure for building their own in-house platforms, greatly empowering their operations teams. However, they are also aware that Kubernetes is still too complex for many developers and comes with a steep learning curve.
These days, it's easier to find a developer who is familiar with the use of Kubernetes, but it is still difficult to find one who is proficient. This is why more organizations are recognizing the need to invest in developer experience.
So what is the typical developer experience? Imagine a new developer joins the team and frequently asks questions such as:
- What are our technology and architecture standards?
- What development languages, frameworks, and open source software can I use? Which versions? What coding standards and code style do we use?
- Where can I find the project file? What about the architecture and topology diagrams?
- What microservice APIs can I invoke?
- Are there any ready-made project templates that I can reuse?
- Where do I submit my code? How often do I submit? How much test coverage is required?
- How do I create a container image?
- How do I trigger the pipeline execution to deploy the app?
- Where can I view telemetry data—such as running logs, metrics, and distributed tracing data—for my app?
With any luck, they'll find the right people to answer their questions. But then what? They will also need to apply for multiple system accounts, open multiple tool interfaces (such as Wiki to view information about the file, Jira to view user stories or bugs, Jenkins to view the execution status of the pipeline, Kubernetes to view the workload running state, etc.), familiarize themselves with their respective operations, occasionally switch between different systems and copy and paste information from one to the other, and manually string together the entire workflow, which is time-consuming and error prone.
A study published by McKinsey suggests that companies with high developer efficiency grow their revenue four to five times faster than companies with low developer efficiency. They also have larger operating profit margins and are 55 times more innovative. Companies in the top quartile also had 60 percent higher total shareholder returns and 20 percent higher operating profit margins. In addition, in VMware’s quarterly Executive Pulse survey, half of the enterprise technology executives polled said that an improved developer experience would have the greatest potential to increase revenue for their organization.
This indicates that developer efficiency is one of the key factors for companies to shape their core competencies. So how do we bridge these gaps in the development experience?
Let's look at a recent Gartner® report on developer experience (Innovation Insight for Internal Developer Portals, by Manjunath Bhat, Mark O'Neill, Oleksandr Matvitskyy, 1 February 2022), which recommends a number of key practices, including:
• "Improve developer experience and effectiveness by establishing internal developer portals to streamline the software delivery lifecycle and support reuse, sharing, and collaboration
• Enable governance without sacrificing agility by using developer portals that provide self-service cloud access through built-in guardrails while still enabling rapid delivery and innovation
• Continuously innovate portal capabilities by appointing a platform owner for the developer portal to manage its road map, gather feedback, and market its capabilities"
This report also says, “By 2025, 75 percent of organizations with platform teams will provide self-service developer portals to improve developer experience and accelerate product innovation.” The ideal developer portal provides a one-stop service covering the three major phases of Day 0 discovery and creation, Day 1 integration and deployment, and Day 2 operation and improvement.
Backstage is the quintessential developer portal. Having long recognized the importance of developer experience to building a streamlined, cloud native business, VMware in 2021 released VMware Tanzu Application Platform, a state-of-the-art application platform that includes a developer portal delivering a consistent GUI, constructed and extended based on Backstage. VMware is also an active contributor to the Backstage community, providing enterprise-level commercial support.
What is Backstage?
Backstage was founded by music streaming giant Spotify, and its vision is to be the "Kubernetes for developer experience."
The general timeline of Backstage's development is as follows:
- An internal project launched in 2016 to build Spotify's internal developer portal. It reduced the onboarding time for Spotify's developers by 55 percent and is widely used internally.
- Launched as open source during "Hack Week" in March 2020
- Began incubation in the Cloud Native Computing Foundation Sandbox in September 2020
- Moved from the Sandbox to the official incubation stage in March 2022
- Official GA, version 1.0, released on March 17, 2022
More than 100 renowned companies and organizations have publicly deployed this technology, including Netflix, Expedia, Splunk, American Airlines, and VMware.
Backstage provides a unified user experience and an extensible core framework. Key components include:
- Software catalog – Software catalog, unified management of the various components of the software system.
- Software templates – Software templates to quickly create projects and scaffolding, ensuring compliance with technical standards and specifications.
- TechDocs – Technical documentation, unified publishing, search, and use of technical documentation (Markdown format).
- Kubernetes – Visualization of resources running on Kubernetes such as Deployment, Pod, etc.
- Search – Search software directories and technical documentation based on Lunr, Elasticsearch, or Postgres, etc.
Backstage's system model
Backstage's software catalog defines a set of models for describing software systems.
The main types of model include:
- Domain – A business area, such as e-commerce
- System – Each application system of the Domain, such as product catalog, shopping cart, order system, etc.
- Component – Components in the system, such as the order front-end SPA and the order microservices that make up the order system
- Resource – Resources in the system, such as order database, order fulfillment system based on messaging middleware, etc.
- API – Components can both publish APIs (in line with OpenAPI, AsyncAPI, GraphSQL, etc.) for the user to invoke, and use APIs provided by other components, such as the order microservice component, to publish order APIs for the frontend to invoke.
Each project needs to be provided with metadata to describe itself (catalog-info.yaml), which is stored with the source code in a version control system, such as Git. Metadata mainly includes:
- Category (domain/system/component/resource, etc.)
- Name, description
- Labels, annotations
- Type (service/website/library)
- Affiliation system/component
- Responsible team (group/user)
- Service lifecycle (production/experimental/deprecated)
- Dependencies (component/resource)
- APIs provided, APIs used, etc.
Sample YAML defining a Backstage component
See the full list of configuration items.
Backstage's ecosystem
Backstage seamlessly integrates with major source code management systems, such as GitHub, GitLab, and Amazon Web Services S3, and it supports the use of third-party authentication including GitHub, GitLab, Okta, Auth0, and Atlassian.
The Backstage open source community is very active, and the ecosystem has about 60 ready-made plugins covering the software development lifecycle, which can be broadly categorized as follows:
- Agile management – Jira
- Source code management – GitHub Pull Requests, GitHub Insights
- Quality – SonarQube, FOSSA, Google Lighthouse
- Discovery – API Docs, Harbor, Home, Tech Radar, TODO, Cost Insights
- CI/CD – Argo CD, Azure Pipelines, CircleCI, GitHub Actions, GitLab, Jenkins, Travis CI
- Monitoring – Datadog, Grafana, Kafka, New Relic, PagerDuty, Prometheus
- Security – Snyk, Security Insights
- Infrastructure – AWS CloudFormation, AWS Lambda
As you can see, there is already a large number of plugins, and the ecosystem is continuously being improved and enriched, so many of the typical needs of enterprises can be met. Even if they are not fully met yet, enterprises can and should develop their own plugins and contribute back to the community so that the community ecosystem can get better and better!
Tanzu Application Platform extensions
In addition to the core Backstage components, Tanzu Application Platform has developed the following additional components based on Backstage's flexible plugin architecture:
Tanzu Application Platform uses plugin architecture
Workload visibility: Dynamic view of the application after deployment, including detailed information about objects at each level of the resource tree such as Knative Service, Kubernetes Deployment, ReplicaSet, Pods, etc., with one-click access to the app via Knative Route.
Tanzu Application Platform workload visibility
App Live View: Graphical display of the app's health status based on the Spring Boot app's Actuator mechanism, as well as real-time monitoring of CPU/memory, details of recent HTTP requests, dynamic adjustment of log levels, and more.
Tanzu Application Platform application live view
App Accelerator: A predefined project template, which is made available to the development team. It automatically generates the overall project directory structure and basic configuration files to meet enterprise technology standards and architecture specifications, making it easy for the project team to get started quickly.
Tanzu Application Platform application accelerator
Supply chain visibility: Graphical display of the supply chain execution path and status, allowing further viewing of the execution details of each step.
Tanzu Application Platform supply chain visibility
Summary and outlook
Building a developer portal is an ongoing process rather than a one-time turnkey project, and not everything will be ready once Backstage and Tanzu Application Platform are installed. On top of this, organizations need to adapt to their own specific needs and integrate existing tools to continuously improve their current development experience.
At the same time, Tanzu Application Platform is a relatively new product that is still in continuous iteration and development. Learn more about Tanzu Application Platform, and see the new features that were delivered in the recent version 1.2 release.
Related reading
Open Source FOMO? Not with Tanzu Application Platform
Elevating the Developer Experience: A Forrester Study on How (and Why) to Improve DevX
