Contract Software Data Engineer - Chester - Inside IR35 - Investment Banking - Investigo

eFinancialCareers; Chester; Hybride; GBP 90 000 - 120 000; Plein temps

Résumé du poste

eFinancialCareers seeks a Software / Data Engineer for Compliance Assurance & Streaming Data in Chester on a hybrid basis. You will own end-to-end data platform engineering—ingestion, processing, and auditable controls for large-scale financial data.

Responsibilities include building resilient pipelines, integrating vendor APIs, and delivering production-grade, compliant solutions across distributed systems.

Qualifications

• Strong Python development in enterprise environments.
• Experience with Spark/PySpark for large-scale data processing.
• Proficiency with Apache Kafka and real-time streaming patterns.
• Experience with NiFi, Hadoop, Hive and SQL for data processing and storage.

Responsabilités

• Design and build resilient, enterprise-scale data pipelines.
• Integrate across vendor APIs, streaming tech and distributed data platforms.
• Engineer high-volume batch and near real-time processing solutions.
• Develop reusable ingestion and processing frameworks.
• Develop robust data handling patterns including replay, backfill, recoverability and schema evolution
• Define and enhance operational telemetry, monitoring and platform observability
• Deliver solutions through CI/CD pipelines and production environments

Connaissances

Python development

Apache Spark / PySpark

Apache Kafka

Apache NiFi

Hadoop

Advanced SQL

REST API integration

Data modelling

Distributed systems engineering

CI/CD

Git workflows

Outils

Hive

Spark

Kafka ecosystem tooling

NiFi

Python tooling

Description du poste

Software / Data Engineer - Compliance Assurance & Streaming Data

Location: Chester (Hybrid)

A leading global financial institution is seeking a Senior Data Platform Engineer to help build and enhance a strategic enterprise platform responsible for ingesting, processing and assuring large-scale communication, collaboration and transactional data.

We're looking for an engineer who can own the entire data lifecycle, from source-system integration and streaming ingestion through to transformation, storage and the delivery of production-grade compliance and assurance controls. You'll work at the intersection of software engineering, data engineering and regulatory technology, building scalable solutions used across a highly regulated environment.

The role offers exposure to modern data-platform technologies, complex distributed systems, enterprise-scale datasets and emerging AI-enabled engineering practices.

What You'll Be Doing End-to-End Platform Engineering

• Design and develop resilient enterprise-scale data pipelines
• Build integrations across vendor APIs, streaming technologies and distributed data platforms
• Engineer high-volume batch and near real-time processing solutions
• Develop reusable ingestion and processing frameworks rather than one-off solutions
• Implement robust data handling patterns including replay, backfill, recoverability and schema evolution
• Define and enhance operational telemetry, monitoring and platform observability

Streaming & Distributed Data Processing

Build and optimise solutions using:

• Apache Kafka
• Apache NiFi
• Apache Spark / PySpark
• Hadoop ecosystem technologies
• Hive and large-scale data storage platforms

Control & Assurance Engineering

Translate regulatory and operational requirements into deterministic control logic

• Develop reconciliation, completeness, configuration and assurance controls
• Produce auditable outputs, exception reporting and evidence frameworks
• Build monitoring and alerting capabilities for compliance-critical processes
• Work closely with engineering, architecture, risk and operational stakeholders

Software Engineering & Delivery

Write clean, maintainable and testable Python and PySpark code

• Contribute to CI/CD pipelines and automated deployment processes
• Participate in code reviews and engineering standards initiatives
• Deliver solutions through Development, UAT and Production environments
• Support troubleshooting and root-cause analysis across distributed systems

Required Experience

We're interested in engineers who can demonstrate strong expertise across several of the following areas:

• Core Technical Skills Python development in enterprise environments
• Apache Spark / PySpark
• Apache Kafka
• Apache NiFi
• Hadoop and Hive
• Advanced SQL
• REST API integration and development
• Data modelling and data lineage
• Distributed systems engineering
• CI/CD and modern software delivery practices
• Git and collaborative development workflows

Additional Desirable Experience

• Microsoft Azure
• Microsoft Graph
• Entra ID
• Service principals and secure identity patterns
• Grafana, Splunk, Tableau or Power BI
• Java or another strongly typed language
• AI-assisted development tools including GitHub Copilot
• Exposure to agentic AI, orchestration frameworks or automation tooling

Ideal Background Candidates are likely to come from:

• Financial services
• Banking and capital markets
• RegTech
• Large-scale enterprise data engineering environments
• Telecommunications or communications platforms
• Highly regulated industries

Experience working with voice, communications or collaboration technologies would be advantageous, as would any exposure to assurance, reconciliation or control-focused platforms.

What Success Looks Like

• New data sources are onboarded through reusable engineering patterns
• Pipelines are resilient, observable and fully recoverable
• Control logic is transparent, auditable and trusted
• Data quality and lineage are clearly understood
• Solutions are delivered through a disciplined SDLC into production environments
• The platform scales to support new technologies, controls and business requirements