Senior Data Engineer

E1 Edf Trading Ltd; England; Hybrid; GBP 90,000 - 130,000; Full time

Job summary

E1 Edf Trading Ltd in London seeks a hands-on Senior Data Engineer to accelerate delivery of high-value data products for the front office. You will work with Python fundamentals, data orchestration, and real-time visualisations to support trading decisions and critical business processes.

The role requires strong Python/SQL, experience with Airflow, Dagster, Prefect, and UI tooling like Plotly Dash or Streamlit, plus DevOps exposure. Hybrid work with 3 days in the office is expected.

Qualifications

• Strong Python and SQL experience.
• Experience with data orchestration tools such as Airflow, Dagster, or Prefect.
• Experience designing database schemas and ORM models (SQLAlchemy).
• SQL performance tuning and optimisation.
• Experience with Python UI/dashboard tools (Plotly Dash or Streamlit).
• DevOps experience (CI/CD, Azure DevOps or similar).
• Strong communication skills with business and technical stakeholders.

Responsibilities

• Build and maintain data pipelines integrated with trading workflows and monitor data quality.
• Collaborate with front office stakeholders to translate needs into technical tasks.
• Contribute to architectural decisions for pipelines and systems.
• Design and optimise database schemas and ORM models.
• Create real-time visualisations using streaming tech and dashboards.

Skills

Python

SQL

Data orchestration

UI dashboards

Communication

Tools

Airflow

Dagster

Prefect

SQLAlchemy

Plotly Dash

Streamlit

Docker

Kubernetes

Azure DevOps

Job description

Description

:

Department: Front Office Data Products

The Front Office Data Products team serves a variety of stakeholders across the Front Office, including traders, risk and senior management. We work with lots of different internal and external data sources, building and maintaining data pipelines, data models, and data products to support trading decisions and critical business processes. The team is currently expanding its capabilities to support real-time use cases alongside existing batch processing workflows, and is also expanding its Python data visualisation capabilities.

Position purpose

We are looking for a hands-on Senior Data Engineer to join our team to accelerate the delivery of high-value data products to the business. This person will be someone with strong Python fundamentals, experience with orchestration tools and Python data visualisation/UI tooling (ideally Plotly Dash) and will be someone who can communicate effectively with both business stakeholders and technical colleagues.

Main responsibilities

• Build and maintain data pipelines that integrate with existing trading workflows and business processes, including operational ownership (monitoring, alerting, incident support) and embedding data quality controls (validation, anomaly detection).
• Collaborate with stakeholders across different teams within the front office to understand requirements, translate business needs into technical tasks, and deliver high-quality data products. Explain trade-offs clearly and align on practical delivery plans.
• Contribute to architectural decisions on how to implement pipelines and systems
• Design database schemas and SQLAlchemy ORM models to support our data needs and cater for data quality and data governance requirements.
• Utilise streaming technologies and event driven architecture to build real time visualisations
• Develop dashboards and data products using Python UI tools (Plotly Dash, Streamlit or similar)
• Participate in peer reviews and uphold engineering standards across the team (coding conventions, testing, CI/CD, observability, security-by-design).
• Tune and optimise SQL queries for performance
• Work with DevOps processes (Azure DevOps, CI/CD pipelines, Docker, Kubernetes)
• Take ownership of work end-to-end: prioritise effectively, communicate with stakeholders, plan out the work, manage risks and dependencies, and drive tasks through to completion.
• Be a role model for Python (and data engineering) best practices in the team

Experience and technical requirements

Essential

• Strong Python and SQL experience
• Data orchestration tools (Airflow, Dagster, Prefect)
• Designing database schemas and ORM models (e.g. SQLAlchemy)
• SQL query performance tuning and optimisation
• Python UI/dashboard tools (Plotly Dash, Streamlit or similar)
• DevOps experience (CI/CD pipelines, Azure DevOps or similar)
• Strong communication skills – ability to work with both technical colleagues and business stakeholders.

Strongly Preferred

• Kafka or other streaming technologies (e.g. Azure Event Hubs)
• Kubernetes
• Docker and containerisation
• Azure Cloud services (Blob Storage, Application Insights, KeyVault, Azure DevOps)
• Commodities trading or financial services background

Nice to have

• Dagster
• Tableau experience

Hours of work:

40 hours per week (Mon-Fri 08:30 - 17:30)

Hybrid (min 3 days per week at the office)