Senior Enterprise Technology Engineer

bp; Greater London; Hybrid; GBP 110,000 - 150,000; Full time

Benefits offered by this job

Flexible working options

Retirement benefits

Job summary

BP is seeking a Senior Enterprise Tech Engineer to establish and lead the Data Engineering squad within the TA&I organisation of the T&S entity. You will design and implement enterprise-scale data pipelines, mentor 8–10 engineers, and drive AI-enabled data capabilities across global environments.

You will leverage Python, SQL, Airflow, and AWS to deliver robust data platforms, while championing automation, DevOps practices, and clear technical documentation in a dynamic trading-focused setting.

Qualifications

• Bachelor/Master’s degree or equivalent in CS/Engineering or numerate field.
• 10 years as a Data Engineer with mentoring leadership.
• Experience building market data solutions in trading/financial services.
• Expertise in Python (Pandas, Selenium/Beautiful Soup, Requests) and ML libs.
• Generative AI techniques including agents, RAG, AI SDLC, context engineering.
• Mastery of SQL and PostgreSQL; data modelling and design.
• Extensive experience with large-scale data pipelines and time-series data.
• Apache Airflow for data orchestration at scale.
• AWS cloud services (S3, Redshift, EC2, Lambda).
• SDLC design using Git and ADO/Jenkins pipelines.
• Linux environment and Git proficiency.
• Excellent communication and global collaboration skills.
• Desire to build high-performing teams and foster learning.

Responsibilities

• Establish and lead a global data engineering squad and define technical direction.
• Design and implement scalable data pipelines using Python, SQL and Airflow.
• Lead ETL of large datasets from diverse sources including time-series data.
• Leverage Generative AI to deliver novel capabilities for trading org.
• Utilize AWS to build and manage secure, cost-effective pipelines.
• Mentor 8–10 data engineers and promote knowledge sharing.
• Collaborate across borders in a global, cross-functional team.
• Drive Agile adoption (Kanban/Scrum) and manage workloads via ADO/CI-CD.
• Create data visualisations with Tableau/Power BI for diverse audiences.
• Maintain clear technical documentation for pipelines and procedures.
• Foster continuous improvement through sprints, RCAs and design reviews.

Skills

Python

SQL

Apache Airflow

AWS

Generative AI

CI/CD/DevOps

Data pipelines

Leadership

Time series data

Education

Bachelor’s or Master’s degree in CS/Engineering

Tools

Databricks

Tableau/Power BI

Git/Azure DevOps/Jenkins

Job description

Entity:

Technology

Job Family Group:

IT&S Group

Job Description:

We are looking for a Senior Enterprise Tech Engineer to establish and lead the Data Engineering squad within our global data team, supporting the Trading Analytics and Insights (TA&I) organisation within the Trading and Shipping (T&S) Entity.

You will play a pivotal role in building and leading a high-performing, global team of engineers responsible for designing and implementing robust, enterprise-scale data pipelines, while mentoring and coaching a team of 8‑10 engineers. Operating at true enterprise scale across multiple environments and subject area communities, you will bring deep expertise in Python and SQL data engineering, ETL, cloud data platforms and orchestration at scale. Working with TA&I sponsors and Technology product owners, you will define, design and implement the data engineering roadmap, balancing the demands of safe and reliable daily operation with the need to continually explore and bring new capabilities — including Generative AI — to market.

This role requires expertise across: data engineering, platform engineering, software engineering, capability analysis and design and forensic problem diagnosis. You will be able to define and apply meticulous principles of automation, CI/CD and DevOps, and to lead a demanding front-office trading organisation in fully exploiting the potential of its data platform.

In This Role You Will

• Strategic Leadership: Establish and lead a high-performing, global data engineering squad, fostering a collaborative, results-oriented environment and owning the technical direction for the team’s deliverables.
• Design, develop and implement highly scalable and efficient data pipelines using advanced Python (Pandas, web scraping and machine learning libraries), SQL (including advanced optimisation techniques) and Apache Airflow.
• Lead the extraction, transformation and loading (ETL) of large and complex datasets, including time series data, from various sources (web APIs, databases, real-time streams).
• Leverage Generative AI platforms and frameworks — agents, RAG, prompt and context engineering, evaluations and AI SDLC — to deliver novel capabilities for the trading organisation.
• Tap into expertise in AWS cloud services (S3, Redshift, EC2, Lambda, etc.) to build, deploy and manage data pipelines in a secure and cost-effective manner, with particular focus on cloud cost optimisation and Total Cost of Ownership (TCO).
• Coach and mentor a team of 8‑10 data engineers, providing guidance on standard processes, technical development and career growth, and fostering a culture of knowledge sharing and continuous learning.
• Lead and collaborate effectively within a global, cross-functional team, building strong relationships across borders and time zones.
• Drive the adoption of Agile methodologies (Kanban/Scrum) and use Azure DevOps (ADO) to plan, track and manage team workload efficiently, refining operational and capability backlogs.
• Create compelling data visualisations using advanced tools (e.g. Tableau, Power BI) to communicate insights to both technical and non-technical audiences.
• Champion automation tools and standard processes to deliver reliable, repeatable results, ensuring all configuration is maintained under source control with automated CI/CD deployment and test.
• Develop and maintain clear, concise technical documentation for data pipelines and procedures to ensure knowledge transfer and maintainability.
• Embodies the spirit of continuous improvement through agile sprint retros, RCAs and design review workshops; coaching junior team members and using data to drive where the team invests.

Essential Experience and Requirements

• Bachelor/Master’s degree or equivalent experience in Computer Science, Engineering, Information Systems or other numerate fields.
• Demonstrated ability of 10 years as a Data Engineer with a proven track record of success in leading and mentoring teams.
• Experience of building market data solutions within a trading or financial services environment.
• In-depth expertise in Python (Pandas, web scraping libraries such as Selenium, Beautiful Soup and Requests, and machine learning libraries such as TensorFlow and PyTorch).
• Generative AI techniques including agents, skills, prompt engineering, RAG, AI SDLC, context engineering and evaluations.
• Mastery of SQL (including advanced optimisation techniques) and experience with relational databases (PostgreSQL preferred), including data modelling and design skills.
• Extensive experience designing and building large-scale, efficient data pipelines for complex and time series data.
• Proven experience with Apache Airflow for data orchestration and scheduling at scale.
• Expertise in AWS cloud services (S3, Redshift, EC2, Lambda, etc.) is a must.
• Definition and design of the full SDLC (standards and tooling): development, test and deployment, using Git source control and ADO (or Jenkins) pipelines.
• Experience working in a Linux environment and proficiency with Git version control.
• Excellent written and verbal communication skills, with the ability to present complex technical concepts clearly to both technical and non-technical audiences.
• Demonstrated experience working in a global environment and building strong relationships with teams across borders.
• A passion for building high-performing teams and fostering a culture of innovation and continuous learning.

Desirable Criteria

• Experience with Databricks for large-scale data processing.
• Experience with advanced data visualisation tools (e.g. Tableau, Power BI) with a focus on insightful storytelling.
• Knowledge of Microsoft SQL Server administration and query performance tuning.
• Familiarity with core cloud concepts: subscriptions, resource groups, IAM.
• Knowledge of the commodities front office domain e.g. financial and physical instruments, market data prices.

Why Join Us?

At BP, we support our people to grow in a diverse and exciting environment. We believe that our team is strengthened by diversity. There are many aspects of our employees’ lives that are significant, so we offer benefits to enable your work to fit with your life. These benefits can include flexible working options, a generous paid parental leave policy, excellent retirement benefits, among others.

We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform crucial job functions, and to receive other benefits and privileges of employment.

Please contact us to request accommodation. Reinvent your career as you help our business meet the challenges of the future.

Travel Requirement

No travel is expected with this role

Relocation Assistance:

This role is not eligible for relocation

Remote Type:

This position is a hybrid of office/remote working

Skills:

Cloud Platforms, Cloud Platforms, Collaboration, Communication, Configuration management and release, Continuous deployment and release, Creating a high performing team, Database Design, Digital Project Management, Documentation and knowledge sharing, Emerging technology monitoring, Facilitation, Information Security, Mentoring, Metrics definition and instrumentation, NoSql data modelling, Problem Solving, Relational Data Modelling, Risk Management, Scripting, Secure development, Service operations and resiliency, Software Design and Development, Solution Architecture, Source control and code management {+ 5 more}

.

Legal Disclaimer:

We are an equal opportunity employer. We do not discriminate on the basis of protected characteristics like race, religion, color, sex, national origin, sexual orientation, veteran status or disability status.Individuals with an accessibility need may request an adjustment/accommodation related to bp’s recruiting process (e.g., accessing the job application, completing required assessments, participating in telephone screenings or interviews, etc.). If you would like to request an adjustment/accommodation related to the recruitment process,please contact us.

If you are selected for a position and depending upon your role, your employment may be contingent upon adherence to local policy. This may include pre-placement drug screening, medical review of physical fitness for the role, and background checks.