Lead solution architect - Data engineering

BP PLC; Pune District; Hybrid; INR 4,000,000 - 6,000,000; Full time

Benefits offered by this job

Hybrid work model

Relocation assistance

Health insurance

Life insurance

Job summary

BP plc in Pune, India, seeks a Lead Solution Architect for Data Engineering to design and optimize cloud-native data platforms using Azure Databricks, Delta Lake, and Synapse. The role involves leading architecture, governance, and scalable data solutions across teams in a hybrid work model.

The candidate will drive data strategy, implement secure data pipelines, and mentor engineers while collaborating with product owners and analysts to meet business goals.

Qualifications

• 7+ years in Data Architecture or Data Engineering roles.
• Strong expertise in Azure Databricks (Spark, Delta Lake), Azure Data Lake Gen2, Azure Data Factory / Synapse Pipelines, Azure SQL.

Responsibilities

• Architect end-to-end cloud-native data platforms using Azure services.
• Design Delta Lake architectures and ETL/ELT pipelines with Databricks.
• Lead governance using Unity Catalog, implement data quality and metadata management.

Education

Bachelor's degree in Computer Science

Job description

## Lead solution architect - Data engineeringApply: This position is a hybrid of office/remote working: India - Pune: Full time: Posted Today: End Date: September 24, 2026 (5 days left to apply): RQ115966**Entity:**Technology**Job Family Group:**IT&S Group**Job Description:**## **Role Overview**We are a global energy business involved in every aspect of the energy system. We are working towards delivering light, heat and mobility to millions of people, every day. We are one of the very few companies equipped to solve some of the big complex challenges that matter for the future. We have a real contribution to make to the world's ambition of a low carbon future. Join us and be part of what we can accomplish together. You can participate in our new ambition to become a net zero company by 2050 or sooner, and to help the world get to net zero.Critical to achieving bp’s digital ambitions is the delivery of our high value data and analytics initiatives, and the enablement of the technologies and platforms that will support those objectives. We are developing deep data capability to transform the access, supply, control and quality to our vast and ever-growing reserves of data.We are looking for an outstanding Data Architect with deep expertise in **Microsoft Azure** and **Databricks** to design, build, and optimize modern data platforms. The ideal candidate will have strong architectural leadership, hands-on technical proficiency, and the ability to influence internal customers to drive data strategy, governance, and scalable data solutions.. These use cases would drive significant operational efficiency, safety and environmental aims that are key to the transformation of bp.## ## **Key Responsibilities****Data Architecture & Platform Design*** Architect end-to-end **cloud-native data platforms** using Azure services, including Azure Data Lake Storage Gen2, Azure Databricks, Azure Synapse, Azure SQL, Azure Data Factory, and Event/Service Bus.* Define and implement **data modeling standards**, including dimensional, canonical, and domain-driven designs.* Design **Delta Lake architectures**, including medallion (Bronze–Silver–Gold) frameworks.* Develop scalable and secure **ETL/ELT pipelines** using Databricks (PySpark/Spark SQL).## **2. Databricks & Delta Lake Engineering*** Implement **Delta Lake best practices**, including OPTIMIZE, ZORDER, Change Data Feed (CDF), schema evolution, and table versioning.* Optimize Spark workloads for performance, cost, and cluster utilization.* Lead governance implementation using **Unity Catalog**, including permissions, data lineage, and metadata modeling.## **3. Azure Data Ecosystem Integration*** Integrate solutions across Azure Data Factory, Synapse Pipelines, Event Hubs, Functions, Key Vault, Azure DevOps, and Purview.* Drive CI/CD and Infrastructure-as-Code using Terraform, Bicep, or ARM templates.* Build APIs and microservices supporting data consumption.## **4. Data Governance & Security*** Define policies for **data quality**, **master data management**, **metadata**, and **data lifecycle management**.* Ensure GDPR-compliant data design, encryption (KMS/Key Vault), RBAC/ABAC, and secure network architecture.* Establish frameworks for observability, including logging, monitoring, and alerting.## **5. Stakeholder Leadership*** Collaborate with business, product owners, engineers, and analysts to gather data requirements.* Lead architectural reviews and provide long-term technology strategy.* Mentor engineering teams on Azure, Databricks, and scalable data design principles.## **Required Skills & Qualifications****Technical Skills*** 7+ years in Data Architecture or Data Engineering roles.* Expert-level knowledge of:* **Azure Databricks (Spark, Delta Lake, MLflow), Agent Bricks*** **Azure Data Lake Gen2*** **Azure Data Factory / Synapse Pipelines*** **Azure SQL / Synapse Dedicated SQL / Cosmos DB*** **Unity Catalog & Delta Live Tables*** Strong programming experience in **PySpark**, **SQL**, and **Python**.* Hands-on with **CI/CD (Azure DevOps / GitHub Actions)**.* Deep understanding of **distributed systems & big data performance tuning**.## **Soft Skills*** Ability to translate business requirements into technical architecture.* Strong documentation, presentation, and interpersonal skills.* Leadership in multi-disciplinary teams and sophisticated enterprise environments.## Formal education* A Bachelor's (or higher) degree, ideally in Computer Science, MIS/IT, Mathematics, chartered accountancy, engineering, or equivalent work experience.## **Preferred Qualifications and Skills*** Azure certifications (e.g., **DP-203**, **AZ-305**, **Databricks Data Engineer Professional/Associate**).* Courses on Generative AI in Data engineering* Experience with **data mesh**, **domain-driven design**, and **event-driven architecture**.* Familiarity with Power BI or other BI tools.* Experience integrating governance tools such as **Azure Purview**.* TOGAF 9 certification## ## **Why join us?**At bp, we provide an **excellent working environment and employee benefits!** We offer an open and inclusive culture and a great work-life balance. Tremendous learning and development opportunities are available to craft a career path, life and health insurance, medical care package and many others!We support our people to learn and grow in a diverse and exciting environment. We believe that our team is strengthened by diversity. We are committed to crafting an inclusive environment in which everyone is respected and treated fairly. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform crucial job functions, and receive other benefits and privileges of employment.**Travel Requirement**Up to 10% travel should be expected with this role**Relocation Assistance:**This role is eligible for relocation within country**Remote Type:**This position is a hybrid of office/remote working**Skills:**Agility core practices, Business modelling, Creativity and Innovation, Data Analysis, Data design, Digital Analytics, Emerging technology monitoring, Information Assurance, Integration, IT Management, IT strategy and planning, Network Design, Requirements definition and management, Risk Management, Service operations and resiliency, Solution Architecture, Systems Development Management, Technical specialism, User Experience Analysis.**Legal Disclaimer:**We are an equal opportunity employer. We do not discriminate on the basis of protected characteristics like race, religion, color, sex, national origin, sexual orientation, veteran status or disability status. Individuals with an accessibility need may request an adjustment/accommodation related to bp’s recruiting process (e.g., accessing the job application, completing required assessments, participating in telephone screenings or interviews, etc.). If you would like to request an adjustment/accommodation related to the recruitment process, please contact us. If you are selected for a position and depending upon your role, your employment may be contingent upon adherence to local policy. This may include pre-placement drug screening, medical review of physical fitness for the role, and background checks.