In an increasingly data-driven market, businesses need confidence in the quality of their data to make effective decisions.
New companies may take a clean-sheet approach when it comes to collecting the data they need, but existing businesses
often have legacy systems containing data of mixed quality and value, and any low-quality data lurking within them can
jeopardise any decisions made on the whole dataset. Altus Group wanted to target and correct issues across their data
more effectively – to improve the quality of their existing information and to ensure new data arriving from external sources
was of acceptable quality before it was loaded into their systems.
The Hartree Centre team worked closely with Altus Group to investigate the existing quality of the data systems in scope
and understand issues faced, then went on to propose and develop a new data extraction, validation and transformation
pipeline. Rather than hard-coding business logic and risk limiting the shelf-life of a potential solution, we created a flexible
framework that can be reconfigured according to application. By delivering a prototype quickly and refining through
iterative testing, feedback and development cycles, the solution was effectively tailored to address the challenge. Altus
Group technical staff were simultaneously trained to develop their own configurations and were comfortable taking
ownership of the project outputs going forwards to maintain the system in production.
Altus Group are now continually validating their internal data and generating weekly reports highlighting
relevant issues to data owners for review. This lightweight system can run at laptop scale and is capable
of screening hundreds of thousands of records within minutes, freeing up valuable staff time and
effort, with resulting data quality improvements underpinning core business activities. The
flexibility and extensibility built into the project from the outset has enabled the Altus Group
team to apply the new system to new datasets. In a single day, the team established a
new data ingestion and validation workflow, effectively automating a previously
manual data ingestion process.
“Altus Group have benefited from
immediate efficiency gains using
the Hartree Centre’s solution, one
task that previously took several
months can now be
completed in hours.”
“Our experience of working with the Hartree Centre
has been outstanding, providing a robust and
scalable solution that Altus Group can embed into
our data processes delivering fast and more accurate
outcomes leading to significant commercial benefits.”