Senior Data Engineer : Azure

Eaglecliff; Greater London; On-site; GBP 70,000 - 110,000; Full time

Job summary

Eaglecliff is seeking a Senior Data Engineer with Azure experience to join a world-leading Energy Trading company on a Data Flow project for their Front Office. You will design and implement robust data pipelines and contribute to data foundation initiatives.

You will collaborate with technical teams and business managers, present reports, and drive efficiency across pilots to maximize value. The role includes building a Data Lake and enabling faster, data-driven trading decisions.

Qualifications

• Proven mastery of AWS, Azure, or SAP in data environments.
• Expert ELT design and data modeling.
• Strong data integration and ingestion capabilities.
• Advanced data manipulation and processing skills.
• Proficient in GitHub, Azure DevOps and CI/CD.
• Hands-on with Data Factory, Databricks, Synapse and related tools.

Responsibilities

• Lead data foundation roll-out and embedding initiatives.
• Coordinate change, incident and problem management processes.
• Present reports to stakeholders and act as SME on data analysis and design.
• Drive efficiency across pilots to maximize value delivery.
• Contribute to CoE and CoP communities.
• Tackle low hanging fruit: move from Excel to modern visualization tools.
• Support analysis of refinery maintenance and fuel-supply data.
• Create a Data Lake (not a Data Swamp) to improve decision making.

Skills

AWS Master

Azure Master

SAP Master

ELT Master

Data Modeling Master

Data Integration

Data Ingestion

Data Manipulation

Data Processing

GitHub

Azure DevOps

Data Factory

Databricks

SQL DB

Synapse

Stream Analytics

Glue

Kinesis

Redshift

PyTest

SonarQube

Tools

GitHub

GitHub Actions

Azure DevOps

Azure Data Factory

Databricks

Azure Synapse

Kinesis

Redshift

Job description

Our latest opportunity is for a Senior Data Engineer with Azure experience to join a world-leading Energy Trading company as part of a Data Flow project for their Front Office.

Key Characteristics

• Technology expert who constantly pursues knowledge enhancement and has inherent curiosity to understand work from multiple dimensions.
• Deep data focus with expertise in technology domain
• A skilled communicator capable of speaking to both technical developers and business managers. Respected and trusted by leaders and staff.
• Actively delivers the roll‑out and embedding of Data Foundation initiatives in support of the key business programmers.
• Coordinate the change management process, incident management and problem management process.
• Present reports and findings to key stakeholders and be the subject matter expert on DataAnalysis & Design.
• Drive implementation efficiency and effectiveness across the pilots and future projects to minimize cost, increase speed of implementation and maximize value delivery.
• Contributes to community building initiatives like CoE, CoP.

Mandatory skills

• AWS/Azure/SAP Master
• ELT Master
• Data Modeling Master
• Data Integration & Ingestion Skill
• Data Manipulation and Processing Skill
• GITHUB, Action, Azure DevOps Skill
• Data factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest Skill

Optional skills

• Experience in project management, running a scrum team.
• Experience working with BPC, Planning.
• Exposure to working with external technical ecosystem.
• MKDocs documentation

Project Initiatives

To leverage trading information flow advantage across RDS to improve decision making and trading bias based on data insights.The project will increase value creation through collaboration, creating new cross commodity trading opportunities (in both Crude and Products). It will provide better investment in analysis capabilities, tools and infrastructure and more rapidly deploy near term value creation for trading desks.

Deliverables

• Aim to tackle the low hanging fruit, getting things out of Excel, unlocking the power of new visualisation tools, restructuring and cleaning up existing models.
• Support better analysis of Refinery maintenance, Crude supply & demand and Gasoline supply & demand amongst others.
• Creation of a Data Lake and not a Data Swamp and will improve decision making & trading bias via insights from data
• An Asset Capacity & Maintenance deployment will give T&S the summary of planned and unplanned Refinery Maintenance plus Capacity information