App Dev Best Practices Case Studies Customers digital transformation Modernization Best Practices Tanzu Labs

How Rapid Iteration with GraphQL Helped Reenvision a Government Payments Platform

When embarking on digital transformation, success often comes down to using the right tools for the job. 

Emerging technologies have the ability to enable organizations to deliver better customer experiences more efficiently. This truism can serve as a forcing function for engineering teams to routinely reevaluate their tech choices and make sure they aren't missing out on a better solution.

When adopting new tools or developing new products, teams must factor in the nature of their organization, the available skills for building and maintaining their product, and the business problem driving the need for change. Such technology changes become even trickier when made to a live product. The key to giving teams the flexibility to make these changes at various stages of the product is to adopt an evolutionary architecture. 

In one recent engagement, VMware Tanzu Labs took an agile approach toward building an adaptable architecture that allowed evolving from REST to GraphQL in an already-live platform. This made the entire solution much easier to iterate on and maintain. The success is yet another testimony for the Tanzu Labs approach: start small, experiment, deliver ROI, and continuously learn and evolve the architecture to add value.

The Tanzu Labs team worked with an Australian state government agency to transform its payments function. We helped the client envision and deliver an omnichannel, scalable, and highly-resilient payments platform. The platform was created to enable people to pay for availing government services and is also used by a finance team to do due diligence and reconciliation, as well as agencies that need to collect payments. The platform now saves the government more than 100 million AUD per year through consolidation of the payments functions across agencies. The product achieves a straight-through processing rate of 99.99 percent, enabling a small team to monitor more than 10 million AUD each day.

Hon. Minister for Customer Service Victor Dominello visits the VMware Tanzu Labs office.

The payments platform is composed of transaction systems that were built for specific channels, such as digital and over the counter, with more channels in the roadmap. These channel transaction systems send their payment data to the back-office solution, which serves as the system of record. 

Below is a simplified view of the platform’s architecture. The Apigee layer provides the nuts and bolts for the authentication and authorization mechanism along with other security features we built into the app.

Evolution of the payments portal

The first phase of the back-office solution was used purely for bookkeeping purposes, generating reconciliation reports for the accounting and general ledger systems. Soon, there was a need for doing diagnostics on payments from both the internal finance team and the agencies using the product. In the beginning, the product team acted as a concierge for any request for information from the agencies and the finance team. However, as the number of payment transactions increased, the concierge process became unsustainable. In order to automate this process, we decided to create a self-serve payment back-office portal that enabled the agencies and finance team to do their due diligence independently.     

Initially, the team created a simple front-end application built in React and served by an Express server. We also added a simple Express back-end system to support login and session management for portal users. The updated architecture predominantly served the needs of payments made through the digital channel and looked something like what is shown here.

The challenge

While this pattern would suffice for the first milestone, we knew that we would need to iterate based on what we learned from observing the customer journey. For the first few iterations, we were using the REST API to connect to our bookkeeping Spring Boot back end, which worked fine until we needed data from an additional back-end application. The search criteria advanced to have logic where a field itself might not be sufficient on its own and would require another mandatory field to be present. Additional logic was required based on varying user inputs, and a different result was presented for different combinations of search field selections. This started the bending and twisting of the existing REST APIs to fit in optional checks, combining unrelated data into one model and merging models to produce results. The pattern deviated from the normal REST convention of “operations on a model,” and modifications started to feel contrived. The front end then needed to connect to multiple back-end applications to create search results based on the user input.

Pivot to GraphQL  

At this juncture, we knew it was time to rethink the way we were approaching the problem. So, the team reconvened to give GraphQL a try. GraphQL is ideal for scenarios where your input and output can vary and are controlled by the user selections on the page. We can also control the exact fields that are required without extra fields being returned, making it simpler for the front end to just read the result and display it. The front end was modified to create a GraphQL query with the fields needed just enough to surface the result based on user input. The decision to pivot to GraphQL was made easier by the fact that REST and GraphQL can coexist in harmony. Otherwise, it would have called for a major refactor, which was not only time-consuming but also largely unnecessary. REST was serving us well in other areas, and changing it seems impertinent at the time. To lower the risk of the change, we did a small spike first. As part of the spike, we created a separate endpoint serving results through GraphQL and, once confident, we swapped the REST out.

For the front end, we used the Apollo client and the Spring Boot starter GraphQL library for the back end. Like pretty much any layer that goes on top of a React app, the APP needed to be wrapped in an ApolloProvider, which also has an equivalent MockProvider for unit tests. On the back-end testing, similar to Rest Test Templates, GraphQL has GraphQLTestTemplate.

This chart compares the before and after solutions in the front end and back end.

 

Before

After

Front end

If user entered value available for search in App1 then: 

-Call App1, provide all the other data as null

-Pick the values which are applicable for the current result and ignore the others

-Render the page

 

If user entered value available for search in App2 then: 

-Call App2, provide all the other data as null

-Pick the values which are applicable for the current result and ignore the others

-Render the page

Create a query call with what the user types, the rest of the fields do not need to be set. 

 

Based on the user input, alter the output fields, populate query for only what is needed to render the page.

Back end

Create a REST Endpoint over a custom model with every input as nullable.

 

The custom model then internally calls multiple repository calls to consolidate data from multiple DB models.

 

A layer in between to combine and stitch together those models to form the final output state.

Create a GraphQL server with accepting Query for the GraphQL resource. 

 

Make a DB call with the search inputs and provide it to the output fields as per every request.

 

Call another app only if the search inputs are something that needs to be retrieved from another app.

With the newer approach, responsibilities became clearly defined and closely associated with the data source. The front end holds the knowledge and decision on what is entered and what is required to render. The back end knows about data retrieval and can make complex queries to retrieve that data. The refined interaction between the apps now looked like this.

We did encounter one issue in GraphQL: our setup duplicated the contract at both the front end and the back end. If a change was required in the interface, it then had to be synced in multiple places that felt trivial and redundant.

When to use GraphQL

Here are some use cases where GraphQL might be a good fit:

  1. If input/output is dynamic and changes with the path a user can take in the app

  2. If data needs to come from multiple sources

  3. If an app demands high response times and needs to be light (think mobile apps), then GraphQL can cut off extra objects being returned in the response

  4. If the data structure or requirement does not feel like a natural fit to the conventions of REST

Finding the right tool for the job

In the world of technology, there is no magic wand to solve every problem. So, teams should assess and experiment in their projects to find the right candidate for the job and not hesitate to pivot if continuing on one path complicates development. It is easy to get carried away with all the choices available today. It is, therefore, critical to follow business priorities and let the architecture emerge from there.

Related Articles