Platform Engineering – The Evolution of the Cloud Landing Zone

Over the last few years, many financial services firms have embraced and adopted the public cloud.

There have been some challenges along the way, with regulation, compliance, and integration with on-premises data centres being just a few.

One of the first steps to adopting a public cloud has been the Cloud Landing Zone. For some firms, this has taken a few iterations to get right – but now that in 2024 most have managed to clear this hurdle, they’re asking:

“OK, so I have this wonderful cloud environment with a landing zone which meets our security, regulatory and compliance requirements – what’s next?”

The cloud journey has seen firms pushing DevSecOps with the mindset of "You build it, you run it.”

However, this presents several challenges, such as:

• The “run” part also includes the “provisioning” of the infrastructure, meaning that software engineers must learn core infrastructure concepts, including networking and security, which they have either never done or have had minimal exposure to.
• Designing and building comprehensive CI/CD pipelines with the appropriate steps for SAST, DAST, etc, involves a steep learning curve.
• Confusion due to the sheer number of services available.
   

So, what can be done?

Enter Platform Engineering

Platform engineering seems to be the next “thing”; however, as always, this raises several questions:

• What is platform engineering?
• How can financial services firms adopt this effectively and get the best ROI?
• How can teams best utilise their team members without adding extra responsibility, especially responsibilities they must be equipped to take on?
   

Self Service Platform

One way to do this is to build a centrally managed, self-service platform allowing application teams to get cloud services up and running quickly via approved services and reference architectures, which can be rapidly provisioned.

An example of what goes into a self-service platform is described in the diagram below. If you wish to dive into more detail in this diagram, then this blog written earlier in the year will be helpful.

Using a self-service platform offers a host of advantages to software teams:

• The Risk Engine can guide software engineers to the right services and architectures, reducing the challenge of selecting the right service.
• The platform only provides approved services and ensures they’re configured to meet the organisation's security requirements.
• Application teams no longer rely on the InfraSec or Cloud Operations team to provision their infrastructure.
   

However, as with most things, there are also disadvantages to this approach:

• Implementing a self-service platform for a highly-regulated industry such as financial services can be very expensive. More than likely, a dedicated team of highly skilled polyglot engineers will be required.
• Application teams must still “run” the infrastructure themselves again; this may not be something they have done before.
• There is less flexibility as teams rely on the risk engine to decide what infrastructure is right for them.
   

A centrally managed self-service platform can act as a tool to help application teams get the best out of the public cloud; however, as mentioned, it has disadvantages, with investment and flexibility being two.

Is there another option to use as an alternative or with a self-service platform to deliver a better developer experience?

The Open Application Model

One of the outcomes of firms moving to the cloud and adopting DevSecOps has been creating a new “DevOps Engineer” role to help maintain the cloud services required to run the application.

However, most teams – typically consisting of software engineers - have had to absorb this role. As mentioned before, they may not be skilled in aspects such as security, networking, etc, which are essential to maintaining cloud services.

So, how can we ensure that application team members are focused on what they do best and are not distracted by having to do a role they’re either not equipped - or may not want - to do?"

One way is to adopt the Open Application Model (OAM), which divides the responsibilities for a software product into three roles and ensures that team members are focused on what they’re most skilled at:

Software engineers – Build the application.

Developing software is complicated, from understanding the requirements to translating that into code, from code structure and unit testing.

Adding additional responsibilities, such as maintaining and learning the infrastructure and deployment strategies, prevents them from doing what they do best – developing software to help the organisation.

This is where the next role comes in.

Platform engineers – Run the Application

Platform engineers work with the software engineers to decide what is the ideal platform for the application code to run. They decide whether it should be Kubernetes, Serverless, or perhaps on traditional virtual machines; they’re well versed in features such as scaling, networking, security and even cost management.

The platform the code runs on should be treated as a product and follow similar practices to the software engineers, such as version control, unit, and integration testing.

While software engineers spend time improving their application code and adding additional features, platform teams spend time doing the same. By treating it as a product, they can spend time and effort improving the platform and making it better. This can be done by taking feedback from both InfraSec and application teams or taking advantage of improvements and additional features released by the CSPs.

However, they’re limited to the policies, security practices, and components made available by the InfraSec team via a self-service platform. They can’t bypass the controls and gates critical to organisational security.

And this takes us to the next role.

InfraSec engineers

The InfraSec team are responsible for setting the security practices, deciding what cloud services should be made available to the firm and how each component should be configured to meet the firm's security and compliance standards.

Each component should be “codified”, version controlled, stored in a software catalogue, and easily consumed by the platform team via a self-service platform.

In other words:

The software team writes the application code, which runs on a platform created and maintained by the platform team and underpinned by components produced by the InfraSec team.

In conclusion

Platform engineers should treat their platform as a product and make every effort to improve it — just as software engineers improve their applications.

The software application's platform is an integral part of the whole system. It should therefore be treated as such, which is not something that has historically been done in financial services. You could argue that this is where SRE teams fit in; however, platform engineering teams have a wider remit than just improving reliability; they work closely with software engineers to ensure that platform and the code run in harmony, helping software engineers better understand the services involved.