Contract Software Data Engineer - Chester - Inside IR35 - Investment Banking - Investigo
other jobs eFinancialCareers
Added before 1 Days
- England,North West,Cheshire,Cheshire West and Chester
- Full Time, Permanent
- Competitive salary
Job Description:
Full job description Software / Data Engineer - Compliance Assurance & Streaming Data
Location: Chester (Hybrid)
A leading global financial institution is seeking a Senior Data Platform Engineer to help build and enhance a strategic enterprise platform responsible for ingesting, processing and assuring large-scale communication, collaboration and transactional data.
We’re looking for an engineer who can own the entire data lifecycle, from source-system integration and streaming ingestion through to transformation, storage and the delivery of production-grade compliance and assurance controls. You’ll work at the intersection of software engineering, data engineering and regulatory technology, building scalable solutions used across a highly regulated environment.
The role offers exposure to modern data-platform technologies, complex distributed systems, enterprise-scale datasets and emerging AI-enabled engineering practices.
What You’ll Be Doing End-to-End Platform Engineering
*Design and develop resilient enterprise-scale data pipelines
*Build integrations across vendor APIs, streaming technologies and distributed data platforms
*Engineer high-volume batch and near real-time processing solutions
*Develop reusable ingestion and processing frameworks rather than one-off solutions
*Implement robust data handling patterns including replay, backfill, recoverability and schema evolution
*Define and enhance operational telemetry, monitoring and platform observability
Streaming & Distributed Data Processing *Build and optimise solutions using: *Apache Kafka
*Apache NiFi
*Apache Spark / PySpark
*Hadoop ecosystem technologies
*Hive and large-scale data storage platforms
*Develop scalable matching, enrichment and reconciliation services
*Create canonical data models spanning multiple source systems
*Ensure data lineage, quality and governance requirements are met
Control & Assurance Engineering *Translate regulatory and operational requirements into deterministic control logic
*Develop reconciliation, completeness, configuration and assurance controls
*Produce auditable outputs, exception reporting and evidence frameworks
*Build monitoring and alerting capabilities for compliance-critical processes
*Work closely with engineering, architecture, risk and operational stakeholders
Software Engineering & Delivery *Write clean, maintainable and testable Python and PySpark code
*Contribute to CI/CD pipelines and automated deployment processes
*Participate in code reviews and engineering standards initiatives
*Deliver solutions through Development, UAT and Production environments
*Support troubleshooting and root-cause analysis across distributed systems
Required Experience We’re interested in engineers who can demonstrate strong expertise across several of the following areas:
Core Technical Skills *Python development in enterprise environments
*Apache Spark / PySpark
*Apache Kafka
*Apache NiFi
*Hadoop and Hive
*Advanced SQL
*REST API integration and development
*Data modelling and data lineage
*Distributed systems engineering
*CI/CD and modern software delivery practices
*Git and collaborative development workflows
Additional Desirable Experience *Microsoft Azure
*Microsoft Graph
*Entra ID
*Service principals and secure identity patterns
*Grafana, Splunk, Tableau or Power BI
*Java or another strongly typed language
*AI-assisted development tools including GitHub Copilot
*Exposure to agentic AI, orchestration frameworks or automation tooling
Ideal Background Candidates are likely to come from:
*Financial services
*Banking and capital markets
*RegTech
*Large-scale enterprise data engineering environments
*Telecommunications or communications platforms
*Highly regulated industries
Experience working with voice, communications or collaboration technologies would be advantageous, as would any exposure to assurance, reconciliation or control-focused platforms.
What Success Looks Like *New data sources are onboarded through reusable engineering patterns
*Pipelines are resilient, observable and fully recoverable
*Control logic is transparent, auditable and trusted
*Data quality and lineage are clearly understood
*Solutions are delivered through a disciplined SDLC into production environments
*The platform scales to support new technologies, controls and business requirements
Location: Chester (Hybrid)
A leading global financial institution is seeking a Senior Data Platform Engineer to help build and enhance a strategic enterprise platform responsible for ingesting, processing and assuring large-scale communication, collaboration and transactional data.
We’re looking for an engineer who can own the entire data lifecycle, from source-system integration and streaming ingestion through to transformation, storage and the delivery of production-grade compliance and assurance controls. You’ll work at the intersection of software engineering, data engineering and regulatory technology, building scalable solutions used across a highly regulated environment.
The role offers exposure to modern data-platform technologies, complex distributed systems, enterprise-scale datasets and emerging AI-enabled engineering practices.
What You’ll Be Doing End-to-End Platform Engineering
*Design and develop resilient enterprise-scale data pipelines
*Build integrations across vendor APIs, streaming technologies and distributed data platforms
*Engineer high-volume batch and near real-time processing solutions
*Develop reusable ingestion and processing frameworks rather than one-off solutions
*Implement robust data handling patterns including replay, backfill, recoverability and schema evolution
*Define and enhance operational telemetry, monitoring and platform observability
Streaming & Distributed Data Processing *Build and optimise solutions using: *Apache Kafka
*Apache NiFi
*Apache Spark / PySpark
*Hadoop ecosystem technologies
*Hive and large-scale data storage platforms
*Develop scalable matching, enrichment and reconciliation services
*Create canonical data models spanning multiple source systems
*Ensure data lineage, quality and governance requirements are met
Control & Assurance Engineering *Translate regulatory and operational requirements into deterministic control logic
*Develop reconciliation, completeness, configuration and assurance controls
*Produce auditable outputs, exception reporting and evidence frameworks
*Build monitoring and alerting capabilities for compliance-critical processes
*Work closely with engineering, architecture, risk and operational stakeholders
Software Engineering & Delivery *Write clean, maintainable and testable Python and PySpark code
*Contribute to CI/CD pipelines and automated deployment processes
*Participate in code reviews and engineering standards initiatives
*Deliver solutions through Development, UAT and Production environments
*Support troubleshooting and root-cause analysis across distributed systems
Required Experience We’re interested in engineers who can demonstrate strong expertise across several of the following areas:
Core Technical Skills *Python development in enterprise environments
*Apache Spark / PySpark
*Apache Kafka
*Apache NiFi
*Hadoop and Hive
*Advanced SQL
*REST API integration and development
*Data modelling and data lineage
*Distributed systems engineering
*CI/CD and modern software delivery practices
*Git and collaborative development workflows
Additional Desirable Experience *Microsoft Azure
*Microsoft Graph
*Entra ID
*Service principals and secure identity patterns
*Grafana, Splunk, Tableau or Power BI
*Java or another strongly typed language
*AI-assisted development tools including GitHub Copilot
*Exposure to agentic AI, orchestration frameworks or automation tooling
Ideal Background Candidates are likely to come from:
*Financial services
*Banking and capital markets
*RegTech
*Large-scale enterprise data engineering environments
*Telecommunications or communications platforms
*Highly regulated industries
Experience working with voice, communications or collaboration technologies would be advantageous, as would any exposure to assurance, reconciliation or control-focused platforms.
What Success Looks Like *New data sources are onboarded through reusable engineering patterns
*Pipelines are resilient, observable and fully recoverable
*Control logic is transparent, auditable and trusted
*Data quality and lineage are clearly understood
*Solutions are delivered through a disciplined SDLC into production environments
*The platform scales to support new technologies, controls and business requirements
Job number 4337881
The jobs on site are for both men and women