Data Engineering Lead

Biocon; Bengaluru; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Biocon seeks an Azure Data & Integration Architect to lead enterprise-scale data initiatives across Azure Databricks, Unity Catalog, and data governance. You will enable analytics, self-service reporting with Power BI, and secure data flows between SAP and non-SAP systems across hybrid environments.

You will drive standards for data models, security, and performance, mentor engineers, and collaborate with IT, data, and business teams to deliver scalable solutions.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or related field.
• MBA is a plus.

Responsibilities

• Lead enterprise-wide data and integration initiatives using Microsoft Azure and Databricks.
• Architect scalable data pipelines and a unified lakehouse platform for analytics.
• Enable governance, security, and data quality with Unity Catalog and governance tooling.
• Design enterprise data models and analytical layers on Azure.
• Integrate SAP and non-SAP systems using SAP CPI, REST/SOAP APIs, and Azure API Management.
• Collaborate with data scientists and business teams for ML and predictive analytics use cases.
• Oversee CI/CD pipelines with Azure DevOps for automated deployment and testing.
• Mentor developers and ensure architectural standards are followed.
• Communicate complex technical concepts to senior management and stakeholders.

Skills

Azure Databricks

PySpark

SQL

Power BI

Azure Data Factory

Azure Synapse Analytics

Unity Catalog

Azure Entra ID

OAuth 2.0 / JWT / SAML

S/4HANA / SAP integrations

Python

Education

Bachelor's or Master’s in CS/IT

MBA (plus)

Tools

Azure Databricks

SAP CPI

Azure API Management

REST / SOAP APIs

Azure DevOps

Terraform / ARM templates

Job description

Purpose of the Role

The Azure Data & Integration Architect will lead enterprise-wide data and integration initiatives, leveraging Microsoft Azure and Databricks to drive digital transformation, operational efficiency, and data-driven decision-making. This role focuses on architecting scalable data pipelines, designing a unified lakehouse platform, enabling advanced analytics, and integrating enterprise systems across SAP and non-SAP landscapes.

Key Responsibilities

Data Architecture & Analytics

• Implement Unity Catalog for governance, data lineage, and access control.
• Design and deploy enterprise data models and analytical layers using Azure.
• Enable enterprise reporting and self-service analytics through Power BI integration.
• Manage both structured and unstructured data using Azure Blob Storage and Azure Data Lake Storage Gen2.
• Collaborate with data scientists and business teams to enable machine learning and predictive analytics use cases.
• Define and enforce data architecture standards, naming conventions, and performance best practices.

Integration & Enterprise Connectivity

• Design and implement integrations between SAP and non-SAP systems using SAP CPI, Azure API Management, and REST/SOAP APIs.
• Ensure secure and scalable data flow across on-premises, cloud, and hybrid environments.
• Implement API authentication and security protocols (OAuth 2.0, JWT, SAML).
• Integrate enterprise applications with Azure Entra ID for identity and access management.
• Establish reusable integration patterns and frameworks for enterprise-wide adoption.

Governance, Security & Compliance

• Ensure data quality, lineage, and governance across the analytics ecosystem.
• Implement security and compliance controls aligned with GxP, 21 CFR Part 11, and corporate IT policies.
• Monitor cost and performance optimization across Azure services.
• Oversee DevOps and CI/CD pipelines for automated deployment and testing using Azure DevOps.

Leadership & Collaboration

• Collaborate closely with cross-functional IT, data, and business teams.
• Conduct technical design reviews, mentor developers, and ensure adherence to architectural standards.
• Support vendor evaluations, solution assessments, and proof-of-concept initiatives.
• Communicate complex technical concepts effectively to senior management and business stakeholders.

Education & Qualifications

• Bachelors or Masters degree in Computer Science, Information Technology, or related field.
• MBA is a plus.

Mandatory Requirements

• Minimum 6+ years of experience in data analytics, engineering, and enterprise integration.
• Proven experience architecting enterprise-scale data platforms on Azure.
• Strong hands on expertise in Azure Databricks (PySpark, Delta Lake, Unity Catalog).
• Proficiency in Azure Data Factory, Azure Synapse Analytics, Power BI, Blob Storage, and Data Lake Gen2.
• Experience integrating SAP systems (S/4HANA, SuccessFactors, MES, LIMS, CRM) with Azure and other platforms.
• Proficient in integration technologies (SAP CPI, Azure API Management, middleware platforms).
• Deep understanding of data security and identity protocols (OAuth, JWT, SAML) and Azure Entra ID.
• Familiarity with GxP-compliant data handling and governance practices.
• Strong proficiency in Python, PySpark, and SQL, hands on experience in writing clean, efficient and scalable code

Desirable Requirements

• Experience in pharmaceutical manufacturing and regulatory environments.
  Exposure to Azure DevOps, Terraform, or ARM templates for infrastructure automation.
• Knowledge of modern data architecture principles (Lakehouse, Data Mesh, ELT).
• Experience with Agentic AI, preferably using Microsoft AI technologies

Preferred Certifications

• Microsoft Certified: Azure Solutions Architect Expert
• Microsoft Certified: Data Engineer Associate
• Microsoft Certified: Identity and Access Administrator Associate
• Preferred Industry Experience
• Pharmaceutical Manufacturing
• Regulated Environments (GxP, FDA, ISO)