Lead solution architect - Data engineering
bp
A complete application in a minute — tailored resume and cover letter, ready to send.
Benefits offered by this job
Job summary
bp in India seeks a Data Architect with deep Azure and Databricks expertise to design, build, and optimize modern data platforms. You will lead architectural design, govern data assets, and influence internal customers to drive data strategy and scalable solutions that support bp's net-zero ambition.
This role requires 7+ years in data architecture or engineering, strong PySpark/SQL/Python, and hands-on CI/CD with Azure DevOps or GitHub Actions.
Qualifications
- 7+ years in Data Architecture or Data Engineering roles.
- Expert-level knowledge of Azure Databricks, Delta Lake, and Unity Catalog.
- Strong programming in PySpark, SQL, and Python.
Responsibilities
- Architect end-to-end cloud-native data platforms on Azure.
- Define data modeling standards (dimensional, canonical, domain-driven).
- Design Delta Lake architectures (Bronze-Silver-Gold).
- Develop scalable ETL/ELT pipelines with Databricks.
- Lead governance using Unity Catalog and data lineage.
- Drive CI/CD and IaC with Terraform/Bicep/ARM.
Skills
Education
Tools
Job description
We are a global energy business involved in every aspect of the energy system. We are working towards delivering light, heat and mobility to millions of people, every day. We are one of the very few companies equipped to solve some of the big complex challenges that matter for the future. We have a real contribution to make to the world's ambition of a low carbon future. Join us and be part of what we can accomplish together. You can participate in our new ambition to become a net zero company by 2050 or sooner, and to help the world get to net zero.
Technology
IT&S Group
We are a global energy business involved in every aspect of the energy system. We are working towards delivering light, heat and mobility to millions of people, every day. We are one of the very few companies equipped to solve some of the big complex challenges that matter for the future. We have a real contribution to make to the world's ambition of a low carbon future. Join us and be part of what we can accomplish together. You can participate in our new ambition to become a net zero company by 2050 or sooner, and to help the world get to net zero.
Critical to achieving bp’s digital ambitions is the delivery of our high value data and analytics initiatives, and the enablement of the technologies and platforms that will support those objectives. We are developing deep data capability to transform the access, supply, control and quality to our vast and ever-growing reserves of data.
We are looking for an outstanding Data Architect with deep expertise in Microsoft Azure and Databricks to design, build, and optimize modern data platforms. The ideal candidate will have strong architectural leadership, hands‑on technical proficiency, and the ability to influence internal customers to drive data strategy, governance, and scalable data solutions. These use cases would drive significant operational efficiency, safety and environmental aims that are key to the transformation of bp.
- Architect end-to-end cloud-native data platforms using Azure services, including Azure Data Lake Storage Gen2, Azure Databricks, Azure Synapse, Azure SQL, Azure Data Factory, and Event/Service Bus.
- Define and implement data modeling standards, including dimensional, canonical, and domain-driven designs.
- Design Delta Lake architectures, including medallion (Bronze-Silver-Gold) frameworks.
- Develop scalable and secure ETL/ELT pipelines using Databricks (PySpark/Spark SQL).
- Databricks & Delta Lake Engineering
- Implement Delta Lake best practices, including OPTIMIZE, ZORDER, Change Data Feed (CDF), schema evolution, and table versioning.
- Optimize Spark workloads for performance, cost, and cluster utilization.
- Lead governance implementation using Unity Catalog, including permissions, data lineage, and metadata modeling.
- Azure Data Ecosystem Integration
- Integrate solutions across Azure Data Factory, Synapse Pipelines, Event Hubs, Functions, Key Vault, Azure DevOps, and Purview.
- Drive CI/CD and Infrastructure-as-Code using Terraform, Bicep, or ARM templates.
- Build APIs and microservices supporting data consumption.
- Data Governance & Security
- Define policies for data quality, master data management, metadata, and data lifecycle management.
- Ensure GDPR-compliant data design, encryption (KMS/Key Vault), RBAC/ABAC, and secure network architecture.
- Establish frameworks for observability, including logging, monitoring, and alerting.
- Stakeholder Leadership
- Collaborate with business, product owners, engineers, and analysts to gather data requirements.
- Lead architectural reviews and provide long‑term technology strategy.
- Mentor engineering teams on Azure, Databricks, and scalable data design principles.
- 7+ years in Data Architecture or Data Engineering roles.
- Expert‑level knowledge of:
- Azure Databricks (Spark, Delta Lake, MLflow), Agent Bricks
- Azure Data Lake Gen2
- Azure Data Factory / Synapse Pipelines
- Azure SQL / Synapse Dedicated SQL / Cosmos DB
- Unity Catalog & Delta Live Tables
- Strong programming experience in PySpark, SQL, and Python.
- Hands‑on with CI/CD (Azure DevOps / GitHub Actions).
- Deep understanding of distributed systems & big data performance tuning.
- Ability to translate business requirements into technical architecture.
- Strong documentation, presentation, and interpersonal skills.
- Leadership in multi‑disciplinary teams and sophisticated enterprise environments.
Formal education
- A Bachelor's (or higher) degree, ideally in Computer Science, MIS/IT, Mathematics, chartered accountancy, engineering, or equivalent work experience.
- Azure certifications (e.g., DP-203, AZ-305, Databricks Data Engineer Professional/Associate).
- Courses on Generative AI in Data engineering
- Experience with data mesh, domain-driven design, and event-driven architecture.
- Familiarity with Power BI or other BI tools.
- Experience integrating governance tools such as Azure Purview.
- TOGAF 9 certification
At bp, we provide an excellent working environment and employee benefits! We offer an open and inclusive culture and a great work‑life balance. Tremendous learning and development opportunities are available to craft a career path, life and health insurance, medical care package and many others! We support our people to learn and grow in a diverse and exciting environment. We believe that our team is strengthened by diversity. We are committed to crafting an inclusive environment in which everyone is respected and treated fairly. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform crucial job functions, and receive other benefits and privileges of employment.
Up to 10% travel should be expected with this role
This role is eligible for relocation within country
This position is a hybrid of office/remote working
Agility core practices, Business modelling, Creativity and Innovation, Data Analysis, Data design, Digital Analytics, Emerging technology monitoring, Information Assurance, Integration, IT Management, IT strategy and planning, Network Design, Requirements definition and management, Risk Management, Service operations and resiliency, Solution Architecture, Systems Development Management, Technical specialism, User Experience Analysis
We are an equal opportunity employer. We do not discriminate on the basis of protected characteristics like race, religion, color, sex, national origin, sexual orientation, veteran status or disability status. Individuals with an accessibility need may request an adjustment/accommodation related to bp’s recruiting process (e.g., accessing the job application, completing required assessments, participating in telephone screenings or interviews, etc.). If you would like to request an adjustment/accommodation related to the recruitment process, please contact us. If you are selected for a position and depending upon your role, your employment may be contingent upon adherence to local policy. This may include pre-placement drug screening, medical review of physical fitness for the role, and background checks.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
