Azure Data Engineer – Informatica / ADF / Databricks

Papigen; India; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Papigen in India is seeking an experienced Azure Data Engineer to design, develop, and optimize enterprise data integration pipelines using ADF, Databricks, ADLS, Synapse, and Informatica PowerCenter. The role requires strong hands-on Azure data engineering, complex SQL, data modelling, governance, quality and end-to-end warehousing.

You'll build ETL/ELT pipelines, load data into lakes and warehouses, and support real-time data integration.

Qualifications

• 8–10 years of data engineering, data modelling, or related data platform roles.
• Hands-on Azure Data Factory for ETL/ELT pipeline development.
• Experience with Azure Databricks, ADLS, and Azure Synapse Analytics.

Responsibilities

• Build and maintain ETL/ELT pipelines and integration feeds between enterprise applications.
• Design data pipelines to load data into Azure Data Lakes, Azure Synapse, Azure Databricks, and Azure SQL Database.
• Integrate data from XML, flat files, SAP, relational databases, and other enterprise sources.
• Develop and optimize ADF pipelines, Databricks jobs, and Spark/PySpark transformations.
• Design data models, metadata structures, data architecture, and raw/enriched/curated data layers.
• Implement data-quality checks, governance standards, and master data management practices.
• Optimize SQL queries, data pipelines, and processing performance.
• Support real‑time and near‑real‑time data integration and warehousing.
• Use Azure DevOps, Azure Automation, PowerShell, AzCopy, and ARM Templates for deployment and automation.
• Prepare technical specifications, design documents, and project artifacts.
• Collaborate with project teams in an Agile SDLC environment.

Skills

Azure Data Factory

Azure Databricks

Azure Synapse Analytics

SQL development

Python

PySpark

Data modelling

Data governance & quality

Agile/SDLC

Big Data concepts

Education

BSc in Computer Science, Engineering, Management Information Systems, or a related field

Tools

ADLS

Informatica PowerCenter

SSIS

Azure DevOps

PowerShell

ARM Templates

AzCopy

Job description

Role Overview

We are looking for an experienced Azure Data Engineer to design, develop, and optimize enterprise data integration pipelines using Azure Data Factory (ADF), Azure Databricks, ADLS, Azure Synapse, and Informatica PowerCenter.

Role Overview

We are looking for an experienced Azure Data Engineer to design, develop, and optimize enterprise data integration pipelines using Azure Data Factory (ADF), Azure Databricks, ADLS, Azure Synapse, and Informatica PowerCenter.

The candidate must have strong hands‑on Azure data engineering skills, complex SQL expertise, and experience in data modelling, data governance, data quality, and end‑to‑end data warehousing solutions.

Key Responsibilities

• Build and maintain ETL/ELT pipelines and integration feeds between enterprise applications.
• Design data pipelines to load data into Azure Data Lakes, Azure Synapse, Azure Databricks, and Azure SQL Database.
• Integrate data from XML, flat files, SAP, relational databases, and other enterprise sources.
• Develop and optimize ADF pipelines, Databricks jobs, and Spark/PySpark transformations.
• Design data models, metadata structures, data architecture, and raw/enriched/curated data layers.
• Implement data-quality checks, governance standards, and master data management practices.
• Optimize SQL queries, data pipelines, and processing performance.
• Support real‑time and near‑real‑time data integration and warehousing.
• Use Azure DevOps, Azure Automation, PowerShell, AzCopy, and ARM Templates for deployment and automation.
• Prepare technical specifications, design documents, and project artifacts.
• Collaborate with project teams in an Agile SDLC environment.

Mandatory Skills & Experience

• 8–10 years of experience in data engineering, data modelling, information architecture, or related data platform roles.
• Strong hands‑on expertise in Azure Data Factory (ADF) for ETL/ELT pipeline development.
• Mandatory experience with Azure Databricks, ADLS, and Azure Synapse Analytics.
• Strong knowledge of Azure SQL Database, Azure Functions, Logic Apps, Spark, Python, and PySpark.
• Experience with Informatica PowerCenter; SSIS is an additional skill.
• Strong SQL development skills, including complex queries, preferably across SQL Server and Oracle.
• Experience with data modelling, metadata management, data governance, data quality, and master data management.
• Understanding of Big Data architecture, data ingestion, curation, and storage.
• Experience with data lake architecture and raw, enriched, and curated layers.
• End‑to‑end data warehousing, analytics, and BI/reporting delivery experience.
• Experience with Azure DevOps, Azure PowerShell, Azure Automation, AzCopy, or ARM Templates.
• Ability to work in Agile teams and coordinate technical deliverables.

Education: BSc in Computer Science, Engineering, Management Information Systems, or a related field.

Preferred / Additional Skills

• Informatica PowerCenter and SSIS for enterprise data integration.
• Real‑time and near‑real‑time data processing.
• Azure Functions and Logic Apps for event‑driven integrations.
• Azure Automation and infrastructure deployment using ARM Templates.
• Experience with enterprise‑scale analytics and reporting platforms.

Skills: azure data factory,powershell,adls,informatica powercenter,databricks,azure,azure devops,azure synapse,azure sql