Principal Data Engineering : Databricks

Eaglecliff; Greater London; On-site; GBP 110,000 - 150,000; Full time

Job summary

Eaglecliff seeks a Principal Data Engineer to lead, design and build data foundations and end-to-end solutions for the business to maximize value from data. You will mentor senior and junior engineers while translating Vision and Data Strategy into IT solutions and driving delivery across programs.

The role requires hands-on work with Databricks, PySpark, Python and SQL, with a focus on data modeling, integration and cloud data services.

Qualifications

• Extensive experience designing and building data platforms.
• Hands-on with Databricks, PySpark, Python and SQL.
• Experience with data modeling, data integration and ETL.
• Knowledge of cloud data services (Azure, Synapse) and data tooling.

Responsibilities

• Lead the design and build of data foundations and end-to-end solutions.
• Mentor senior and junior engineers.
• Translate Vision and Data Strategy into IT solutions and deliver them.
• Drive roll-out of Data Foundation initiatives across key programs.
• Coordinate change, incident and problem management.
• Present reports and insights to stakeholders.
• Improve delivery speed and value through efficiency.
• Contribute to Centers of Excellence and Communities of Practice.

Skills

Azure Stack

ELT Experience

Data Modeling

Data Integration

Data Manipulation

GitHub Actions

Azure DevOps

Data Factory

Databricks

Databricks

PySpark

Python

SQL

Synapse

Stream Analytics

Glue

Airflow

Kinesis

Redshift

SonarQube

PyTest

Tools

Databricks

PySpark

Python

SQL

Azure Data Factory

Synapse

Airflow

Kinesis

Redshift

Glue

GitHub

GitHub Actions

Azure DevOps

Data Factory

Job description

We are looking for a Principal Data Engineer to join a world-leading Energy Trading company to lead, design and build data foundations and end-to-end solutions for the Business to maximize value from data.

The candidate is expected to be a subject matter expert who designs and builds data solutions while also mentoring senior and junior engineers. They are also a key driver in translating Vision and Data Strategy into IT solutions and delivering on them.

Strong hands-on expertise in Databricks, PySpark, Python and SQL is central to this role.

Key Skills:

• Azure Stack
• ELT Experience
• Data Modeling
• Data Integration & Ingestion
• Data Manipulation and Processing
• GitHub, GitHub Actions, Azure DevOps
• Data Factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest

Key Characteristics

• Technology expert who constantly pursues knowledge enhancement and has an inherent curiosity to understand work from multiple dimensions.
• Deep data focus with expertise across relevant technology domains.
• A skilled communicator capable of engaging both technical developers and business managers; respected and trusted by leaders and staff.
• Actively delivers the roll-out and embedding of Data Foundation initiatives in support of key business programs.
• Coordinates the change management, incident management, and problem management processes.
• Presents reports and findings to key stakeholders and acts as the subject matter expert on Data Analysis & Design.
• Drives implementation efficiency and effectiveness across pilots and future projects to minimize cost, increase speed of delivery, and maximize value.
• Contributes to community-building initiatives such as Centers of Excellence (CoE) and Communities of Practice (CoP)