Senior Azure Data Engineer (ADF / Databricks / Informatica)

Papigen; India; On-site; INR 2,500,000 - 4,200,000; Full time

Job summary

Papigen is seeking an experienced Azure Data Engineer with 8 years of data engineering and data warehousing expertise in the Azure data platform. You will design and build scalable ETL/ELT pipelines using ADF, Azure Databricks, Synapse, and ADLS to support analytics and reporting across multiple sources.

You will work with business and analytics teams in an Agile environment, implement data lake architectures, perform data governance and quality activities, and document technical designs.

Qualifications

• 8 years of Data Engineering / Data Warehousing experience.
• Strong expertise in Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, and ADLS.
• Experience building and optimizing ETL/ELT pipelines.
• Proficiency in PySpark, Python, SQL, and Informatica PowerCenter.

Responsibilities

• Design, develop, and maintain ETL/ELT pipelines using Azure Data Factory and Informatica PowerCenter.
• Build data integration solutions across multiple sources including databases, SAP, XML, and flat files.
• Develop and optimize data pipelines using Azure Databricks, PySpark, and Azure Synapse.
• Design and implement data lake architectures (Raw, Enriched, Curated layers).
• Perform data modeling, data governance, metadata management, and data quality activities.
• Optimize data processing performance and troubleshoot pipeline failures.
• Develop technical design documents and support project deliverables.
• Work closely with business, analytics, and project teams in an Agile environment.

Skills

Azure Data Factory (ADF)

Azure Databricks

Azure Synapse Analytics

Azure Data Lake Storage (ADLS)

PySpark

Python

Informatica PowerCenter

SQL Server / Oracle

ETL / ELT Development

Data Modeling

Data Warehousing

Data Governance

Metadata Management

Master Data Management (MDM)

Data Quality

Azure DevOps

ARM Templates

PowerShell

Azure Automation

Job description

We are looking for an experienced Azure Data Engineer with 8 years of experience in Data Engineering, ETL/ELT development, Data Warehousing, and Azure Data Platform services. The ideal candidate should have strong expertise in Azure Data Factory (ADF), Azure Databricks, Azure Synapse, ADLS, Informatica PowerCenter, PySpark, and SQL to build scalable data integration and analytics solutions.

Key Responsibilities

• Design, develop, and maintain ETL/ELT pipelines using Azure Data Factory and Informatica PowerCenter.
• Build data integration solutions across multiple sources including databases, SAP, XML, and flat files.
• Develop and optimize data pipelines using Azure Databricks, PySpark, and Azure Synapse.
• Design and implement data lake architectures (Raw, Enriched, Curated layers).
• Perform data modeling, data governance, metadata management, and data quality activities.
• Optimize data processing performance and troubleshoot pipeline failures.
• Develop technical design documents and support project deliverables.
• Work closely with business, analytics, and project teams in an Agile environment.

Required Skills

• 8 years of Data Engineering / Data Warehousing experience.

• Strong expertise in:

• Azure Data Factory (ADF)

• Azure Databricks

• Azure Synapse Analytics

• Azure Data Lake Storage (ADLS)

• PySpark / Spark

• Python

• Informatica PowerCenter

• SQL Server / Oracle

• ETL / ELT Development

• Strong understanding of:

• Data Modeling

• Data Warehousing

• Data Governance

• Metadata Management

• Master Data Management (MDM)

• Data Quality

• Experience with Azure DevOps, ARM Templates, PowerShell, Azure Automation.

Preferred Skills

• SSIS
• Real-time Data Processing
• Big Data Architecture
• Enterprise Analytics & BI Solutions

Qualifications

[Some qualifications you may want to include are Skills, Education, Experience, or Certifications.]

Example: Excellent verbal and written communication skills