Funding Service: How we built a new funding platform for MHCLG

Over the past year, the Funding Service has been building a new funding platform for the Ministry of Housing, Communities and Local Government (MHCLG) based on an entirely new single codebase. ‘Deliver grant funding’ and ‘Access grant funding’ deliver enhanced capabilities whilst being simpler to maintain and iterate on.

This blog post explains why we decided to do this and how we approached it from a technical standpoint, as part of our mission to deliver grants faster, better and cheaper.

Why we built a new platform

We thought long and hard about writing a new platform versus making incremental improvements to our previous microservices. This is not a decision to take lightly as it involves a major upfront commitment and is often regarded a last resort once other re-engineering options have been considered.

Learning from our previous architecture

Previously we used a microservices architecture, slicing by both frontend and backend functionality, as well as by different stages of the funding lifecycle. To paint the picture, our microservices included:

Authenticator Account Store Fund Store Form Designer Form Runner Apply Assess Post-Award

Whilst this delivered some initial benefits, for example being able to speed up time to go-live by using existing capabilities and components from the XGov Form Builder as part of the system, there were many downsides caused by early design decisions:

Whilst each transaction was important (with grants distributing millions of pounds in total), actual user numbers in web-scale terms were very low. This meant we were running a large amount of (costly to run and maintain) infrastructure relative to the number of actual users. A lack of API versioning, combined with tight service coupling, meant co-ordinating the rollout of new features often required a complex orchestration of releases and might cause unexpected regressions or downtime.  We had numerous end-to-end tests which were slow and flaky creating error fatigue, false positives and making the wall time from code push to release sometimes take hours. The use of traditional Server-Side rendering combined with many backend services meant one webpage often called multiple microservices synchronously, meaning any failure in a single microservice could cause entire sections of the site to fail to render. There was a distinct lack of fault isolation and swim lanes. Feature development was slower as developers had to maintain the mental model of all these services in their head and build all the REST HTTP interfaces between them to change any major end user functionality. Furthermore, the XGov Form Builder being written in TypeScript while the rest of the system was Python, meant developers had to be fluent in both of these languages, or hand work off. This combined complexity meant the time to onboard a new developer, and for them to be confident in making changes, was months. Configuration for each grant was hardcoded in multiple microservices. Whilst initially a pragmatic decision to get a first grant minimum viable product (MVP) live, this was never fully revisited, meaning that developer time and multiple app deployments were needed to add each grant and grant round, involving a hugely complex runbook to perform. This process also meant grants were configured 'just in time' so only available for policy teams to preview just before go-live, meaning any issues spotted in user acceptance testing (UAT) could cause a delay in the grant go-live date.

We realised what we had built was closer to a Distributed Monolith. We had all the pain of maintaining microservices without many of the benefits.

These issues impacted all our user groups in different ways, grant applicants and recipients were more likely to experience errors and outages, it was difficult for developers to work at pace (without impacting quality) and feature development was slow for our product and service owners.

Another factor in play was a pivot in policy through government’s commitment to simplify funding, moving away from competitive grants towards allocative grants, which required different capabilities. This meant some of what the previous microservices did would be less useful and new capabilities were needed, making it an opportune moment for change.

Testing our hypothesis with a prototype

To prove and de-risk the single platform approach, a small team of a couple of developers spent a fortnight building a prototype. This was much higher fidelity than you might ordinarily build using the GOV.UK Prototype Kit. But taking this approach enabled us to prove and demonstrate high-level hypotheses the team had, including:

development would be faster and simpler using a two-tier architecture (app and database) in a single codebase we could re-create most of the capabilities offered by the XGov Form Builder in Python we could provide a 'self-service' offering where non-technical users could configure a grant within the service

Following a really intense effort by the team, by the end of the fortnight we had a prototype with a real data model, database backend and full WYSIWYG (What You See is What You Get) preview capabilities. This felt like a huge step up from the status quo and the demo was enough to convince stakeholders including our product manager and service owner of the value of the project.

How we approached the build

The team were well aware of the possible pitfalls designing and scaling a Monolith. To try and avoid those, we put some guardrails in place which would provide clear boundaries within the system, such as:

all grant configuration is performed in the web interface and stored in the database (no Git commits or developer scripts are required to onboard a grant) all communication with the database is through a Data Access Layer in a separate Python Module. We enforce this contract with a linter which runs every time the code is updated all functionality is in a single codebase and deployable application all functions have Python type hints which are enforced by a type checker

Although we were changing architectural pattern, we retained many of the tried and tested building blocks the team were already familiar with, including:

Python Flask SQLAlchemy AWS Aurora Postgres Cloud Native Buildpacks GitHub Actions

This brought a level of familiarity and helped team members quickly become productive on the new codebase.

Now to deliver it!

What we achieved

A screenshot showing the new platform interface which includes full preview capabilities

We have recently gone live with our new 'Deliver grant funding' and 'Access grant funding' services, and 2 grants have already been onboarded to the new system.

Although 'Deliver' and 'Access' appear as separate services, they are essentially 2 sides of the same coin, a form building/CMS side and an applicant/recipient-facing side. Working in a single codebase and database made it much easier to deliver features like WYSIWYG preview, as they are essentially just different views on the same data.

Furthermore, having these self-service capabilities means that content designers can design, share live previews with policy teams and launch a whole grant without any developer involvement. This will be hugely beneficial in freeing up developers to work on new functionality, rather than doing grant configuration as code.

A screenshot showing a review reporting page in Access grant funding (fictional example)

As I move on to an equally exciting digital programme within MHCLG (Elections Digital), I'm proud to have left our civil servant-led team working in the open with (after a few iterations) a simple, right-sized architecture which enables them to build out features at pace, empower colleagues with self-service capabilities, and a platform to continue to extend across the funding lifecycle.

To learn more about the future direction of the service, check out this blog post or email This email address is being protected from spambots. You need JavaScript enabled to view it..