Opencast devs rise to the Scala challenge

11 February 2022

UK government departments are using Scala developers to help them deliver the transformational change they need. The experience of our people shows that not having Scala in your skillset is no major obstacle if you’re a great developer able to learn along the way, says Julian Blake.

Back in 2018, not long after joining Opencast as a senior consultant, Andy Chilton was asked to join the team helping Her Majesty’s Revenue and Customs (HMRC) to revamp a key public-facing digital service. As with much of its digital work, HMRC was using Scala to build and deliver the service.

HMRC invited Andy to lead the Opencast team contributing to the project, led by an HMRC product manager. The challenge that Andy faced was he had had no direct experience of using Scala. 

But Andy was far from a beginner, bringing over 15 years of software engineering and programming experience to Opencast, working on cloud and web solutions for big corporates and consultancies. He arrived with extensive programming experience – including Java, Ruby, CSS and SQL – with additional expertise across agile/scrum, leadership and project management. 

Scala logo in full colour
"UK government departments including HMRC and also the Home Office want Scala developers, and Opencast has included them in teams working on their projects"

No major hurdle

This broad experience meant Andy was well positioned to help meet many of the challenges of the project. Addressing its large service-based architecture and complex user journeys would draw on his wider expertise.

Not having Scala in his specific skillset at that point was no major hurdle. “Given that I did not have a Scala background, the new project was challenging,” Andy says. “But I saw it as an opportunity to better the service and welcomed the role with open arms.

“One of the first things I did was to help address some outstanding coding issues and look to better the service’s error-handling capability. As a result, we learned a great deal about the types of issues our users were experiencing and used the enhanced logging to retrospectively fix them.

“The result was that the service became much more robust, easier to work with and provided a better journey and experience for users.”

Learning and collaborating

By using his expertise, learning from Opencast colleagues and collaborating with other project members, Andy was able to learn Scala along the way. He used the language’s constructs and patterns to help improve the HMRC service in terms of readability, maintainability and extensibility. 

“We helped break up the monolith, adding any new logic as a new integrated front end in the micro service architecture, thus reducing the complexity of the service,” he explains. “The work met GDS standards and a live assessment and became an exemplar service. All this of course was with the help of a fantastic team.”

Another OC developer, Jack Heslop, also found himself at HMRC with no prior Scala knowledge. He arrived at Opencast as a graduate developer with intern experience in big corporates. 

“I really had to push myself to get up to speed as quickly as I could and become a valued member of the team,” Jack recalls. “I was also able to learn more about Scala at the same time. We had great feedback from both the team and product owner”.

Since moving to a team working on another HMRC project, Jack has used Scala in developing a feature to help service users see their information more clearly, as well as tackle outstanding bugs and technical debt, ahead of having the service ready for handover.

Challenge finding Scala devs

Of the major programming languages out there, Scala is used by relatively small numbers of developers and organisations worldwide. Compared to Python which is used by 52%, and Java used by 49%, Scala is used by just 3%, with 2% of organisations planning to adopt or migrate to it in the next 12 months.

Given the relatively low use of Scala globally, it’s unsurprising that there are relatively few developers out there who are specifically skilled in using it – which would explain the challenges that some have in finding lead Scala devs

As its name suggests, Scala is about scale and language – it is designed to grow with the demands of its users. Its usage ranges from writing small scripts to building massive systems, and is used primarily in data processing, distributed computing and web development – and it powers the data engineering infrastructure of many organisations.

Those wanting to understand Scala better can attend conferences like Scalabase, a super-practical workshop event featuring experts from the Scala community. An open source foundation, The Scala Center, was established by Scala itself to help developers learn the language through online courses and seminars.

Capacity to grow

Within Opencast, developers are taken through a four-stage process for training in Scala as the company builds its capacity overall to respond to client needs. Today the company is one of the biggest providers of Scala developers in the North East region.

UK government departments including HMRC and also the Home Office want Scala developers, and Opencast has included them in teams working on their projects.

Tam Mageean, a senior OC scrum master with years of experience working on government digital projects, explains the Scala attraction. “From an operational perspective, the big appeal for standardisation of Scala/functional programming for government departments is that it allows them to operate on a mostly micro service architecture approach to building their services. 

“This makes the services more maintainable, supportable and reusable overall, due to them all being built of mostly the same things and in similar ways. This also allows people to move from team to team more fluidly and even allow services to move from centre to centre with relatively less onboarding needed.”

Opencast’s developers are helping to deliver great work for several UK government partners. They may use technologies like Scala, but because we support our people in their learning and development, not having Scala in their skillset at the outset has not proved an insurmountable hurdle to us helping to deliver great results.

If developers arrive at Opencast with valuable experience and knowledge of agile and XP practices, with the right level of training alongside on-the-job experience, there is almost always capacity for them to grow.

Opencast is deploying up to 30 people into new Scala roles in 2022. For more information, check out our careers page.