Senior Python Data Engineer - ETL/Big Data, Flexible

Solirius Consulting; Greater London; On-site; GBP 70,000 - 110,000; Full time

Benefits offered by this job

Competitive Salary

Bonus Scheme

Private Healthcare Insurance

25 Days Annual Leave + Bank Holidays

Development training allowance

Job summary

Solirius Reply, part of the Reply Group, is seeking an experienced Python Developer with significant data engineering experience to join a high-performing engineering team. You will design, build and maintain robust data pipelines and services within secure, data-intensive environments.

Applicants should have a strong background in Python, ETL/ELT pipelines, and data processing, with experience handling large datasets and collaborating with cross-functional teams.

Qualifications

• Significant commercial experience building Python apps and services.
• Experience designing scalable ETL/ELT pipelines.
• Proficient with SQL and large datasets.
• Experience with PySpark, Spark or Pandas.
• Knowledge of data quality, transformation and integration.
• Experience with CI/CD and automated testing.
• Able to work in Linux and secure development environments.
• Active UK Security Clearance (SC/DV) required.

Responsibilities

• Design, develop and maintain high-quality Python applications, services and data processing solutions.
• Build and optimise scalable ETL/ELT pipelines for the ingestion, transformation and delivery of structured and unstructured data.
• Develop robust data processing workflows using Python and modern data engineering technologies.
• Work with large and complex datasets, ensuring data quality, integrity, availability and performance.
• Integrate data from multiple sources, including APIs, databases, files and streaming systems.
• Develop reusable, well-tested and maintainable software components following modern engineering practices.
• Design and implement automated testing, monitoring, logging and error-handling mechanisms.
• Collaborate with data engineers, software engineers, architects and other technical stakeholders to deliver end-to-end data solutions.
• Contribute to the design and evolution of data platforms, pipelines and supporting infrastructure.
• Identify opportunities to improve system performance, scalability, reliability and automation.
• Participate in code reviews and contribute to engineering standards and best practices.
• Work within secure development environments and adhere to relevant security, governance and compliance requirements.
• Produce clear technical documentation and contribute to architectural and engineering decisions.

Skills

Python development

Data engineering

ETL/ELT pipelines

SQL & relational databases

PySpark / Spark / Pandas

Data quality & governance

Git CI/CD

Linux / secure environments

UK Security Clearance (SC/DV)

Job description

Solirius Reply, part of the Reply Group, is seeking an experienced Python Developer with significant data engineering experience to join a high-performing engineering team. You will design, build and maintain robust data pipelines and services within secure, data-intensive environments.

Applicants should have a strong background in Python, ETL/ELT pipelines, and data processing, with experience handling large datasets and collaborating with cross-functional teams.