Data Engineering Lead

Biocon Biologics; Bengaluru; On-site; INR 4,000,000 - 6,500,000; Full time

Job summary

Biocon Biologics seeks an Azure Data & Integration Architect to lead enterprise data initiatives on Azure, crafting scalable data pipelines and a unified lakehouse platform across SAP and non-SAP landscapes. You will drive governance, secure data flows, and enable predictive analytics in collaboration with data scientists and business teams.

The role emphasizes hands-on design, adherence to industry standards, and mentorship of developers, with focus on Unity Catalog, OT/ETL best practices, and

Qualifications

• Minimum 6+ years of experience in data analytics, engineering, and enterprise integration.
• Proven experience architecting enterprise-scale data platforms on Azure.
• Strong hands-on expertise in Azure Databricks (PySpark, Delta Lake, Unity Catalog).
• Proficiency in Azure Data Factory, Azure Synapse Analytics, Power BI, Blob Storage, and Data Lake Gen2.
• Experience integrating SAP systems (S/4HANA, SuccessFactors, MES, LIMS, CRM) with Azure and other platforms.
• Proficient in integration technologies (SAP CPI, Azure API Management, middleware platforms).
• Deep understanding of data security and identity protocols (OAuth, JWT, SAML) and Azure Entra ID.
• Familiarity with GxP-compliant data handling and governance practices.
• Strong proficiency in Python, PySpark, and SQL, hands on experience in writing clean, efficient and scalable code
• Experience in pharmaceutical manufacturing and regulatory environments. Exposure to Azure DevOps, Terraform, or ARM templates for infrastructure automation.
• Knowledge of modern data architecture principles (Lakehouse, Data Mesh, ELT).
• Experience with Agentic AI, preferably using Microsoft AI technologies.

Responsibilities

• Implement Unity Catalog for governance, data lineage, and access control.
• Design and deploy enterprise data models and analytical layers using Azure.
• Enable enterprise reporting and self-service analytics through Power BI integration.
• Manage both structured and unstructured data using Azure Blob Storage and Azure Data Lake Storage Gen2.
• Collaborate with data scientists and business teams to enable machine learning and predictive analytics use cases.
• Define and enforce data architecture standards, naming conventions, and performance best practices.
• Design and implement integrations between SAP and non-SAP systems using SAP CPI, Azure API Management, and REST/SOAP APIs.
• Ensure secure and scalable data flow across on-premises, cloud, and hybrid environments.
• Implement API authentication and security protocols (OAuth 2.0, JWT, SAML).
• Integrate enterprise applications with Azure Entra ID for identity and access management.
• Establish reusable integration patterns and frameworks for enterprise-wide adoption.

Skills

Azure Databricks

Python

PySpark

SQL

Unity Catalog

Azure Data Factory

Azure Synapse

Power BI

Blob Storage

Data Lake Gen2

SAP CPI

Azure API Management

OAuth 2.0

JWT

SAML

Azure Entra ID

DevOps

Terraform

ARM templates

Lakehouse

Data Mesh

ELT

Agentic AI

Education

Bachelors in Computer Science / IT

Masters in Computer Science / IT

MBA (plus)

Tools

SAP CPI

Azure API Management

Azure DevOps

Terraform

ARM templates

Delta Lake

Unity Catalog

Job description

The Azure Data & Integration Architect will lead enterprise-wide data and integration initiatives, leveraging Microsoft Azure and Databricks to drive digital transformation, operational efficiency, and data-driven decision-making. This role focuses on architecting scalable data pipelines, designing a unified lakehouse platform, enabling advanced analytics, and integrating enterprise systems across SAP and non-SAP landscapes.

Key Responsibilities

• Implement Unity Catalog for governance, data lineage, and access control.
• Design and deploy enterprise data models and analytical layers using Azure.
• Enable enterprise reporting and self-service analytics through Power BI integration.
• Manage both structured and unstructured data using Azure Blob Storage and Azure Data Lake Storage Gen2.
• Collaborate with data scientists and business teams to enable machine learning and predictive analytics use cases.
• Define and enforce data architecture standards, naming conventions, and performance best practices.
• Design and implement integrations between SAP and non-SAP systems using SAP CPI, Azure API Management, and REST/SOAP APIs.
• Ensure secure and scalable data flow across on-premises, cloud, and hybrid environments.
• Implement API authentication and security protocols (OAuth 2.0, JWT, SAML).
• Integrate enterprise applications with Azure Entra ID for identity and access management.
• Establish reusable integration patterns and frameworks for enterprise-wide adoption.

Governance, Security & Compliance

• Ensure data quality, lineage, and governance across the analytics ecosystem.
• Implement security and compliance controls aligned with GxP, 21 CFR Part 11, and corporate IT policies.
• Monitor cost and performance optimization across Azure services.
• Oversee DevOps and CI/CD pipelines for automated deployment and testing using Azure DevOps.
• Collaborate closely with cross-functional IT, data, and business teams.
• Conduct technical design reviews, mentor developers, and ensure adherence to architectural standards.
• Support vendor evaluations, solution assessments, and proof-of-concept initiatives.
• Communicate complex technical concepts effectively to senior management and business stakeholders.

Education & Qualifications

• Bachelors or Masters degree in Computer Science, Information Technology, or related field.
• MBA is a plus.

Mandatory Requirements

• Minimum 6+ years of experience in data analytics, engineering, and enterprise integration.
• Proven experience architecting enterprise-scale data platforms on Azure.
• Strong hands‑on expertise in Azure Databricks (PySpark, Delta Lake, Unity Catalog).
• Proficiency in Azure Data Factory, Azure Synapse Analytics, Power BI, Blob Storage, and Data Lake Gen2.
• Experience integrating SAP systems (S/4HANA, SuccessFactors, MES, LIMS, CRM) with Azure and other platforms.
• Proficient in integration technologies (SAP CPI, Azure API Management, middleware platforms).
• Deep understanding of data security and identity protocols (OAuth, JWT, SAML) and Azure Entra ID.
• Familiarity with GxP‑compliant data handling and governance practices.
• Strong proficiency in Python, PySpark, and SQL, hands on experience in writing clean, efficient and scalable code
• Experience in pharmaceutical manufacturing and regulatory environments. Exposure to Azure DevOps, Terraform, or ARM templates for infrastructure automation.
• Knowledge of modern data architecture principles (Lakehouse, Data Mesh, ELT).
• Experience with Agentic AI, preferably using Microsoft AI technologies

Preferred Certifications

• Microsoft Certified: Azure Solutions Architect Expert
• Microsoft Certified: Identity and Access Administrator Associate