Senior Data Engineer

BP PLC; Pune District; Hybrid; INR 2,500,000 - 5,000,000; Full time

Benefits offered by this job

Competitive salary and bonus

Pension plan

Private healthcare

Flexible working

Relocation within country

Job summary

BP plc is seeking a Senior Data Engineer in Pune to design, build and run scalable data pipelines on Databricks and AWS. You will model data, ensure governance, write secure code, and contribute to CI/CD and IaC practices.

You will mentor engineers, collaborate with cross-functional teams, and shape data products that meet privacy and compliance needs in a hybrid work environment.

Qualifications

• Hands-on experience building and running reliable data pipelines.
• Proficient with Databricks, PySpark, Spark SQL and Delta Lake.
• Experience with AWS data services and serverless options.
• Strong Python and advanced SQL skills.
• Strong data modelling skills in production.
• Experience with infrastructure as code and CI/CD.
• Mentoring engineers and helping raise quality.
• Ability to influence across business units.

Responsibilities

• Design, build and run reliable, scalable data pipelines on Databricks and AWS.
• Model data for analysis, keeping it accurate and well-governed.
• Write secure, well-tested software to build, integrate and quality-assure data.
• Follow and encourage good engineering practice: design, testing, code review, source control and documentation.
• Build and improve CI/CD pipelines and manage infrastructure as code.
• Look after pipelines and services: monitoring, alerting and business-hours support.
• Mentor other engineers and share knowledge to lift team quality.
• Work with business to translate needs into dependable data products.

Skills

Databricks

PySpark

Spark SQL

Delta Lake

Unity Catalog

AWS

Python

Advanced SQL

Data modelling

Infrastructure as code

CI/CD

Mentoring

Cross-functional influence

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

Delta Lake

Unity Catalog

Airflow

Terraform

dbt

Job description

## Senior Data EngineerApply: This position is a hybrid of office/remote working: India - Pune: Full time: Posted Today: End Date: October 16, 2026 (8 days left to apply): RQ116229**Entity:**Technology**Job Family Group:**IT&S Group**Job Description:**## Let me tell you about the roleOur data engineers build and run the data pipelines and platforms that bp pulse relies on every day. The team builds on Databricks and AWS.As a Senior Data Engineer, you will design and build data pipelines, model data for analysis, and keep our platforms dependable and well-governed. It's a hands-on, senior role: setting a high engineering standard, mentoring other engineers, and acting as an informal technical lead within the squad leading through experience rather than a formal management line.## What you will deliver* Design, build and run reliable, scalable, distributed data pipelines on Databricks and AWS.* Model data for analysis, keeping it accurate, well-governed and easy to use.* Write secure, well-tested software to build, integrate and quality-assure data across bp pulse, meeting privacy and compliance needs.* Follow and encourage good engineering practice: technical design and review, testing, code review, source control and documentation.* Build and improve CI/CD pipelines and manage infrastructure as code.* Look after the pipelines and services the team builds monitoring, alerting and business-hours support and keep improving their reliability.* Mentor other engineers and share knowledge to lift the quality of the team's work.* Work with people across the business to understand what they need and turn it into dependable data products.## What you will need to be successful## Essential* Strong, hands-on experience building and running reliable, scalable, distributed data pipelines and data products in complex environments.* Hands-on experience with Databricks, PySpark, Spark SQL, Delta Lake, Unity Catalog and Workflows (or similar lakehouse experience, with a real appetite to grow on Databricks).* Experience with AWS data and serverless services (e.g. Lambda, Glue, S3, Redshift, Kinesis, SNS/SQS).* Strong Python and advanced SQL.* Good data modelling skills, applied in production.* Experience with infrastructure as code and CI/CD.* Experience mentoring engineers and helping raise the quality bar.* Comfortable working with people across the business and shaping work through technical influence.* A curious, continuous-improvement mindset.* A degree in computer science or a related field, or equivalent knowledge and experience.## Desired* Experience with dbt or Apache Airflow.* Familiarity with open table formats such as Delta, Apache Hudi or Apache Iceberg.* Experience with Terraform or another infrastructure-as-code tool.* No prior experience in the energy industry needed.## What we offerIn return, bp offers a competitive salary and bonus, a strong pension, private healthcare, flexible working and real support for your learning and career growth. [P&C to confirm country-specific benefits.]## About bpOur purpose is to deliver energy to the world, today and tomorrow. For over 100 years, bp has focused on discovering, developing and producing oil and gas in the nations where we operate. We are one of the few companies globally that can provide governments and customers with an integrated energy offering. Delivering our strategy sustainably is fundamental to achieving our ambition to be a net zero company by 2050 or sooner!We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform crucial job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.**Travel Requirement**No travel is expected with this role**Relocation Assistance:**This role is eligible for relocation within country**Remote Type:**This position is a hybrid of office/remote working**Skills:**Cloud Platforms, Cloud Platforms, Collaboration, Communication, Configuration management and release, Continuous deployment and release, Creating a high performing team, Database Design, Digital Project Management, Documentation and knowledge sharing, Emerging technology monitoring, Facilitation, Information Security, Mentoring, Metrics definition and instrumentation, NoSql data modelling, Problem Solving, Relational Data Modeling, Risk Management, Scripting, Secure development, Service operations and resiliency, Software Design and Development, Solution Architecture, Source control and code management {+ 5 more}