Enterprise Technology Engineer

BP p.l.c.; United States; Remote; USD 180,000 - 240,000; Full time

Benefits offered by this job

Flexible working options

Global team

Retirement benefits

Job summary

BP p.l.c. is seeking a Senior Enterprise Tech Engineer to establish and lead the Data Engineering squad within the global TA&I data team.

You will build and guide a high-performing group responsible for enterprise-scale data pipelines, mentoring 8–10 engineers and shaping the data engineering roadmap across multiple environments. You will apply deep expertise in Python, SQL, ETL, cloud platforms, and orchestration at scale, using Generative AI and modern DevOps to empower front-office trading

Qualifications

• 10+ years as a Data Engineer with leadership experience.
• Experience building market data solutions in trading/financial services.
• Strong Python (Pandas, web scraping) and SQL with optimization skills.

Responsibilities

• Establish and lead a global data engineering squad and set the technical direction.
• Design, develop and deploy scalable data pipelines with Python, SQL and Airflow.
• Oversee ETL of large time-series datasets from diverse sources (APIs, DBs, streams).
• Leverage Generative AI platforms to deliver new capabilities for trading.
• Work with AWS services to build secure, cost-efficient pipelines and optimise TCO.
• Mentor 8–10 engineers and promote knowledge sharing and learning.
• Collaborate across global teams and manage workload with Agile and ADO.

Skills

Python

SQL

Apache Airflow

AWS

Generative AI

CI/CD

Data pipelines

Education

Bachelor’s or Master’s degree in CS or related field

Tools

Tableau

Power BI

Databricks

PostgreSQL

Git

Azure DevOps

S3/Redshift

Job description

We are looking for a Senior Enterprise Tech Engineer to establish and lead the Data Engineering squad within our global data team, supporting the Trading Analytics and Insights (TA&I) organisation within the Trading and Shipping (T&S) Entity. You will play a pivotal role in building and leading a high-performing, global team of engineers responsible for designing and implementing robust, enterprise-scale data pipelines, while mentoring and coaching a team of 8-10 engineers. Operating at true enterprise scale across multiple environments and subject area communities, you will bring deep expertise in Python and SQL data engineering, ETL, cloud data platforms and orchestration at scale. Working with TA&I sponsors and Technology product owners, you will define, design and implement the data engineering roadmap, balancing the demands of safe and reliable daily operation with the need to continually explore and bring new capabilities - including Generative AI - to market. This role requires expertise across: data engineering, platform engineering, software engineering, capability analysis and design and forensic problem diagnosis. You will be able to define and apply meticulous principles of automation, CI/CD and DevOps, and to lead a demanding front-office trading organisation in fully exploiting the potential of its data platform. In This Role You Will

• Strategic Leadership: Establish and lead a high-performing, global data engineering squad, fostering a collaborative, results-oriented environment and owning the technical direction for the team's deliverables.
• Design, develop and implement highly scalable and efficient data pipelines using advanced Python (Pandas, web scraping and machine learning libraries), SQL (including advanced optimisation techniques) and Apache Airflow.
• Lead the extraction, transformation and loading (ETL) of large and complex datasets, including time series data, from various sources (web APIs, databases, real-time streams).
• Leverage Generative AI platforms and frameworks - agents, RAG, prompt and context engineering, evaluations and AI SDLC - to deliver novel capabilities for the trading organisation.
• Tap into expertise in AWS cloud services (S3, Redshift, EC2, Lambda, etc.) to build, deploy and manage data pipelines in a secure and cost-effective manner, with particular focus on cloud cost optimisation and Total Cost of Ownership (TCO).
• Coach and mentor a team of 8-10 data engineers, providing guidance on standard processes, technical development and career growth, and fostering a culture of knowledge sharing and continuous learning.
• Lead and collaborate effectively within a global, cross-functional team, building strong relationships across borders and time zones.
• Drive the adoption of Agile methodologies (Kanban/Scrum) and use Azure DevOps (ADO) to plan, track and manage team workload efficiently, refining operational and capability backlogs.
• Create compelling data visualisations using advanced tools (e.g. Tableau, Power BI) to communicate insights to both technical and non-technical audiences.
• Champion automation tools and standard processes to deliver reliable, repeatable results, ensuring all configuration is maintained under source control with automated CI/CD deployment and test.
• Develop and maintain clear, concise technical documentation for data pipelines and procedures to ensure knowledge transfer and maintainability.
• Embody the spirit of continuous improvement through agile sprint retros, RCAs and design review workshops; coaching junior team members and using data to drive where the team invests.
  Bachelor/Master's degree or equivalent experience in Computer Science, Engineering, Information Systems or other numerate fields.
• Demonstrated ability of 10 years as a Data Engineer with a proven track record of success in leading and mentoring teams.
• Experience of building market data solutions within a trading or financial services environment.
• In-depth expertise in Python (Pandas, web scraping libraries such as Selenium, Beautiful Soup and Requests, and machine learning libraries such as TensorFlow and PyTorch).
• Generative AI techniques including agents, skills, prompt engineering, RAG, AI SDLC, context engineering and evaluations.
• Mastery of SQL (including advanced optimisation techniques) and experience with relational databases (PostgreSQL preferred), including data modelling and design skills.
• Extensive experience designing and building large-scale, efficient data pipelines for complex and time series data.
• Proven experience with Apache Airflow for data orchestration and scheduling at scale.
• Expertise in AWS cloud services (S3, Redshift, EC2, Lambda, etc.) is a must.
• Definition and design of the full SDLC (standards and tooling): development, test and deployment, using Git source control and ADO (or Jenkins) pipelines.
• Experience working in a Linux environment and proficiency with Git version control.
• Excellent written and verbal communication skills, with the ability to present complex technical concepts clearly to both technical and non-technical audiences.
• Demonstrated experience working in a global environment and building strong relationships with teams across borders.
• A passion for building high-performing teams and fostering a culture of innovation and continuous learning., Experience with Databricks for large-scale data processing.
• Experience with advanced data visualisation tools (e.g. Tableau, Power BI) with a focus on insightful storytelling.
• Knowledge of Microsoft SQL Server administration and query performance tuning.
• Familiarity with core cloud concepts: subscriptions, resource groups, IAM.
• Knowledge of the commodities front office domain e.g. financial and physical instruments, market data prices., Cloud Platforms, Cloud Platforms, Collaboration, Communication, Configuration management and release, Continuous deployment and release, Creating a high performing team, Database Design, Digital Project Management, Documentation and knowledge sharing, Emerging technology monitoring, Facilitation, Information Security, Mentoring, Metrics definition and instrumentation, NoSql data modelling, Problem Solving, Relational Data Modeling, Risk Management, Scripting, Secure development, Service operations and resiliency, Software Design and Development, Solution Architecture, Source control and code management {+ 5 more}

At BP, we support our people to grow in a diverse and exciting environment. We believe that our team is strengthened by diversity. There are many aspects of our employees' lives that are significant, so we offer benefits to enable your work to fit with your life. These benefits can include flexible working options, a generous paid parental leave policy, excellent retirement benefits, among others.