Senior Data Engineer | Python, PySpark & SQL

Capgemini; Greater London; Hybrid; GBP 60,000 - 90,000; Full time

Job summary

Capgemini in Northampton is seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines using Python, PySpark, and SQL. You will work with large datasets and distributed computing frameworks, delivering data-driven solutions for business needs.

The role requires 7–9 years in data engineering, strong Python, PySpark and SQL skills, and exposure to Scala; cloud platform experience and DevOps practices are a plus.

Qualifications

• 7–9 years of experience in Data Engineering or related roles.
• Strong hands-on Python development experience.
• Proven PySpark experience for large-scale data processing.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL processes.
• Develop and optimize data processing solutions using Python and PySpark.
• Write complex SQL queries for data extraction, transformation, and analysis.
• Work with large datasets and distributed computing frameworks.
• Collaborate with architects, analysts, and stakeholders to deliver data-driven solutions.
• Troubleshoot performance, scalability, and data quality issues.
• Contribute to data platform modernization and best practices.

Skills

Python development

PySpark

SQL

Scala exposure

Cloud platforms

DevOps/CI/CD

Job description

Capgemini in Northampton is seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines using Python, PySpark, and SQL. You will work with large datasets and distributed computing frameworks, delivering data-driven solutions for business needs.

The role requires 7–9 years in data engineering, strong Python, PySpark and SQL skills, and exposure to Scala; cloud platform experience and DevOps practices are a plus.