Senior Data Engineer : Databricks

Eaglecliff; Greater London; On-site; GBP 90,000 - 130,000; Full time

Job summary

We are looking for a Senior Data Engineer to join Eaglecliff, a world-leading Energy Trading company, to design and build data foundations and end-to-end solutions for the Business to maximize value from data. The candidate is expected to be a subject matter expert who designs and builds data solutions while mentoring junior engineers.

They are a key driver in translating Vision and Data Strategy into IT solutions and delivering on them, with strong hands-on expertise in Databricks, PySpark,

Qualifications

• Strong hands-on expertise in Databricks, PySpark, Python and SQL is central to this role.
• Mentor junior engineers and translate Vision and Data Strategy into IT solutions and delivering on them.
• Presents reports and findings to key stakeholders and acts as the subject matter expert on Data Analysis & Design.

Responsibilities

• Design and build data foundations and end-to-end data solutions for the Business to maximize value from data.
• Mentor junior engineers and actively drive roll-out and embedding of Data Foundation initiatives.
• Coordinate change, incident and problem management processes and report to stakeholders.

Skills

Databricks

PySpark

Python

SQL

Data Modeling

Data Integration & Ingestion

Data Manipulation & Processing

GitHub

GitHub Actions

Azure DevOps

Tools

Azure Stack

Data Factory

Databricks

SQL DB

Synapse

Stream Analytics

Glue

Airflow

Kinesis

Redshift

SonarQube

PyTest

Job description

We are looking for a Senior Data Engineer to join a world-leading Energy Trading company to design and build data foundations and end-to-end solutions for the Business to maximize value from data.

The candidate is expected to be a subject matter expert who designs and builds data solutions while also mentoring junior engineers. They are also a key driver in translating Vision and Data Strategy into IT solutions and delivering on them.

Strong hands-on expertise in Databricks, PySpark, Python and SQL is central to this role.

Key Skills

• Azure Stack
• ELT Experience
• Data Modeling
• Data Integration & Ingestion
• Data Manipulation and Processing
• GitHub, GitHub Actions, Azure DevOps
• Data Factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest

Key Characteristics

• Technology expert who constantly pursues knowledge enhancement and has an inherent curiosity to understand work from multiple dimensions.
• Deep data focus with expertise across relevant technology domains.
• A skilled communicator capable of engaging both technical developers and business managers; respected and trusted by leaders and staff.
• Actively delivers the roll-out and embedding of Data Foundation initiatives in support of key business programs.
• Coordinates the change management, incident management, and problem management processes.
• Presents reports and findings to key stakeholders and acts as the subject matter expert on Data Analysis & Design.
• Drives implementation efficiency and effectiveness across pilots and future projects to minimize cost, increase speed of delivery, and maximize value.
• Contributes to community-building initiatives such as Centers of Excellence (CoE) and Communities of Practice (CoP).