Remote Lead Data Engineer - Databricks Lakehouse Leader

Stott and May Inc.; Greater London; A distancia; GBP 120.000 - 166.000; A tiempo parcial

Descripción de la vacante

Stott and May is seeking a Lead Data Engineer to provide technical leadership and hands-on delivery for a Databricks-centric data platform in the insurance/fintech space.

You will own architecture, strategy and roadmap for scalable data pipelines using Python, PySpark and SQL, while guiding a team of Data Engineers and collaborating with Risk, Quantitative Finance, Compliance and Data Science teams. Strong governance and stakeholder communication are essential.

Formación

• 7+ years of data engineering experience.
• Strong Databricks experience with large-scale implementations.
• Advanced Python, PySpark and SQL skills.
• Experience leading or mentoring data engineering teams.
• Knowledge of data governance and regulated environments.

Responsabilidades

• Own the Databricks architecture, strategy and roadmap.
• Design, build and operationalise scalable ETL/ELT pipelines using Databricks, Python and SQL.
• Develop and optimise Databricks Lakehouse and Delta Lake solutions.
• Implement data governance, lineage, controls and audit frameworks in a regulated environment.
• Lead, mentor and upskill a team of Data Engineers with technical standards and code reviews.
• Provide technical direction across projects and promote engineering best practices.
• Collaborate with Risk, Quantitative Finance, Compliance and Data Science teams to translate requirements into solutions.
• Communicate complex concepts to senior and executive stakeholders.

Conocimientos

Data engineering

Databricks

Python

PySpark

SQL

Databricks Lakehouse

Delta Lake

Cloud platforms

Leadership

Descripción del empleo

Stott and May is seeking a Lead Data Engineer to provide technical leadership and hands-on delivery for a Databricks-centric data platform in the insurance/fintech space.

You will own architecture, strategy and roadmap for scalable data pipelines using Python, PySpark and SQL, while guiding a team of Data Engineers and collaborating with Risk, Quantitative Finance, Compliance and Data Science teams. Strong governance and stakeholder communication are essential.