Senior Data Engineer – Cloud Lakehouse

Jobtailor; London; Sur place; EUR 90 000 - 130 000; Plein temps

Résumé du poste

Jobtailor is seeking an experienced Data Engineer to design, build, and modernize data platforms that empower analytics and reporting. You will work in an agile data team, crafting scalable data lake/lakehouse solutions and real-time pipelines in AWS/Azure environments.

You will implement transformations with dbt, orchestrate pipelines with Airflow, and contribute to AI-enabled analytics with Power BI and QuickSight. Strong focus on data quality and deployment automation.

Qualifications

• 5+ years of experience in data engineering and scalable data architectures.
• Experience building modern data lake/lakehouse architectures in cloud environments.
• Strong knowledge of AWS or Azure data services and ETL/ELT processes.
• Proficiency with data transformation tools like dbt and orchestration with Airflow.

Responsabilités

• Design and build innovative data solutions within an agile data engineering team.
• Modernize the data platform to enhance reporting, analytics, and decision making.
• Develop and maintain data products across on-premises and cloud environments.
• Build scalable data lake/lakehouse architectures and real-time processing pipelines.
• Automate data engineering processes with CI/CD and optimize ETL/ELT flows.

Connaissances

Data Engineering

Data Lake/Lakehouse Architecture

Cloud Data Services

ETL/ELT

Dbt

Apache Airflow

CI/CD

Data Quality

Generative AI

Java

Outils

Apache Iceberg

AWS S3

AWS Redshift

Azure Data Services

Open Metadata

Power BI

Amazon QuickSight

Description du poste

• Design and build innovative data solutions within an agile data engineering team
• Modernize the data platform to enhance product offerings and maximize the value of data for reporting, analytics, and decision-making
• Develop and maintain data solutions and products, including data transformation, data modelling, and reporting across on-premises and cloud environments
• Build scalable, flexible data lake/lakehouse architectures
• Leverage domain events for efficient, real-time data processing
• Adopt generative AI features such as Amazon Q and Copilot for analytics and self-service capabilities
• Build performant, scalable, and accessible solutions delivering actionable insights
• Automate data engineering processes, implement CI/CD pipelines, and optimize ETL/ELT flows
• Translate business requirements into technical solutions, document GAPs, and align with Architects
• Facilitate data discovery and data management using platforms such as Open Metadata
• Maintain KPI accuracy and documentation; support data quality and compliance standards
• Experiment with new approaches supporting the A&I department strategy
• Support and maintain data products, including release management, incident troubleshooting, and on-call support as necessary

Requirements

• 5+ years of experience in data engineering, with demonstrated expertise in designing and implementing scalable, flexible modern data architectures
• Experience building modern data lake/lakehouse architectures (e.g., Apache Iceberg) in cloud environments
• Strong knowledge of AWS (S3, Redshift) or Azure data services
• Experience with ETL/ELT processes and data ingestion
• Proven expertise using transformation tools such as dbt
• Experience with data pipeline orchestration tools such as Apache Airflow
• Experience with best practices applied to data, including testing and CI/CD
• Experience developing dashboards and reporting solutions using data visualization tools such as Power BI or Amazon QuickSight is advantageous
• Experience with generative AI applications in data analytics is advantageous
• Java experience is nice to have

Core Competencies

Demonstrates expertise in designing and implementing scalable data architectures, leveraging cloud services and modern data solutions to enhance analytics and decision-making. Proficient in automating data engineering processes and ensuring data quality and compliance standards.

Highest-signal resume keywords

• Data Engineering
• Data Lake/Lakehouse Architecture
• AWS Data Services
• ETL/ELT Processes
• Data Pipeline Orchestration

Hard Skills

• Data Transformation
• Data Modelling
• Data Reporting
• CI/CD Implementation
• KPI Documentation
• Data Quality Management
• Generative AI Applications
• Apache Iceberg
• Dbt
• Apache Airflow

Industry Keywords

• Agile Data Engineering
• Data Solutions
• Data Discovery
• Data Management
• Incident Troubleshooting

Tools & Technologies

• AWS S3
• AWS Redshift
• Power BI
• Amazon QuickSight
• Open Metadata