Contract Software Data Engineer - Chester - Inside IR35 - Investment Banking - Investigo

Role overview

Software / Data Engineer - Compliance Assurance & Streaming Data Location:  Chester (Hybrid) A leading global financial institution is seeking a Senior Data Platform Engineer to help build and enhance a strategic enterprise platform responsible for ingesting, processing and assuring large-scale communication, collaboration and transactional data.  We're looking for an engineer who can own the entire data lifecycle, from source-system integration and streaming ingestion through to transformation, storage and the delivery of production-grade compliance and assurance controls. You'll work at the intersection of software engineering, data engineering and regulatory technology, building scalable solutions used across a highly regulated environment. The role offers exposure to modern data-platform technologies, complex distributed systems, enterprise-scale datasets and emerging AI-enabled engineering practices. What You'll Be Doing End-to-End Platform Engineering Design and develop resilient enterprise-scale data pipelines Build integrations across vendor APIs, streaming technologies and distributed data platforms Engineer high-volume batch and near real-time processing solutions Develop reusable ingestion and processing frameworks rather than one-off solutions Implement robust data handling patterns including replay, backfill, recoverability and schema evolution Define and enhance operational telemetry, monitoring and platform observability Streaming & Distributed Data Processing Build and optimise solutions using: Apache Kafka Apache NiFi Apache Spark / PySpark Hadoop ecosystem technologies Hive and large-scale data storage platforms Develop scalable matching, enrichment and reconciliation services Create canonical data models spanning multiple source systems Ensure data lineage, quality and governance requirements are met Control & Assurance Engineering Translate regulatory and operational requirements into deterministic control logic Develop reconciliation, completeness, configuration and assurance controls Produce auditable outputs, exception reporting and evidence frameworks Build monitoring and alerting capabilities for compliance-critical processes Work closely with engineering, architecture, risk and operational stakeholders Software Engineering & Delivery Write clean, maintainable and testable Python and PySpark code Contribute to CI/CD pipelines and automated deployment processes Participate in code reviews and engineering standards initiatives Deliver solutions through Development, UAT and Production environments Support troubleshooting and root-cause analysis across distributed systems Required Experience We're interested in engineers who can demonstrate strong expertise across several of the following areas: Core Technical Skills Python development in enterprise environments Apache Spark / PySpark Apache Kafka Apache NiFi Hadoop and Hive Advanced SQL REST API integration and development Data modelling and data lineage Distributed systems engineering CI/CD and modern software delivery practices Git and collaborative development workflows Additional Desirable Experience Microsoft Azure Microsoft Graph Entra ID Service principals and secure identity patterns Grafana, Splunk, Tableau or Power BI Java or another strongly typed language AI-assisted development tools including GitHub Copilot Exposure to agentic AI, orchestration frameworks or automation tooling Ideal Background Candidates are likely to come from: Financial services Banking and capital markets RegTech Large-scale enterprise data engineering environments Telecommunications or communications platforms Highly regulated industries Experience working with voice, communications or collaboration technologies would be advantageous, as would any exposure to assurance, reconciliation or control-focused platforms. What Success Looks Like New data sources are onboarded through reusable engineering patterns Pipelines are resilient, observable and fully recoverable Control logic is transparent, auditable and trusted Data quality and lineage are clearly understood Solutions are delivered through a disciplined SDLC into production environments The platform scales to support new technologies, controls and business requirements Source: Reed Original listing: https://www.reed.co.uk/jobs/contract-software-data-engineer-chester-inside-ir35-investment-banking-investigo/57522507