Azure Datafactory

Infosys Limited; Bengaluru; On-site; INR 900,000 - 1,300,000; Full time

Job summary

Infosys Limited in Bengaluru seeks a data engineering professional to design and maintain end-to-end data pipelines using Azure Data Factory and Databricks. You will orchestrate transformations, ensure data quality, and optimize performance while collaborating with cross‑functional teams.

The role requires 3–5 years of relevant experience and hands-on expertise with ADF, Databricks, and related cloud data tools in a fast-paced analytics environment.

Qualifications

• Education: BTECH, MTECH, MCA, MSC.
• 3-5 years in data engineering / data integration roles.
• Hands-on with Azure Data Factory: pipeline development, scheduling, triggers.
• Hands-on with Databricks for data processing and transformation.

Responsibilities

• Design, develop, and maintain end-to-end data pipelines using Azure Data Factory for batch workloads.
• Build and orchestrate data transformations with Databricks per requirements.
• Implement pipeline monitoring, alerting, and failure handling for reliability.
• Perform data validation and reconciliation across sources and targets.
• Optimize pipeline performance by tuning ADF activities and Databricks transformations.
• Collaborate with cross-functional teams to gather requirements and deliver on time.
• Maintain technical documentation for pipelines, datasets, schedules, dependencies, and runbooks.
• Support deployments and environment promotions following release and change practices.

Skills

Azure Data Factory

Databricks

PySpark

CI/CD for data pipelines

Azure Synapse Analytics

Delta Lake

Education

B.Tech / M.Tech / MCA / MSc

Tools

Azure Data Factory

Databricks

Azure Synapse Analytics

ADLS

Job description

Job ID/Reference Code INFSYS-INDEED1-252643

Work Experience 3 - 5 Years

Educational Requirements MCA,MSc,MTech,Bachelor of Engineering

Service Line Data & Analytics Unit

Responsibilities Key Responsibilities:

• Design, develop, and maintain end-to-end data pipelines using Azure Data Factory for batch and scheduled workloads.
• Build and orchestrate data transformations and processing workflows using Databricks aligned to business requirements.
• Implement robust pipeline monitoring, alerting, and failure handling to ensure reliable and repeatable executions.
• Perform data validation and reconciliation checks to ensure accuracy, completeness, and consistency across sources and targets.
• Optimize pipeline performance by tuning ADF activities, improving orchestration logic, and streamlining transformations in Databricks.
• Collaborate with cross-functional teams to gather requirements, estimate effort, and deliver solutions within timelines.
• Maintain technical documentation for pipelines, datasets, schedules, dependencies, and operational runbooks.
• Support deployments and environment promotions by following structured release and change management practices.

Minimum Qualifications:

• Education: BTECH, MTECH, MCA, MSC.
• Overall experience: 3-5 years in data engineering / data integration roles.
• Hands-on experience with Azure Data Factory including pipeline development, scheduling, triggers, and integration patterns.
• Hands-on experience with Databricks for data processing and transformation workflows.
• Ability to troubleshoot pipeline failures, analyze logs, and implement corrective actions to improve stability.

Additional Responsibilities:

• Experience designing scalable orchestration patterns in ADF (parameterization, reusable components, dependency handling).
• Strong understanding of data transformation best practices and implementing efficient processing in Databricks.
• Exposure to building data quality checks and operational dashboards for pipeline health and SLA tracking.
• Experience collaborating with stakeholders to translate business requirements into technical pipeline designs and delivery plans.
• Familiarity with performance tuning approaches for cloud data pipelines and distributed processing workloads.

Technical and Professional Requirements: Azure Synapse Analytics, Azure Data Lake Storage (ADLS), PySpark, Delta Lake, CI/CD for data pipelines, AZURE DATAFACTORY

Preferred Skills: Technology->Cloud Integration->Azure Data Factory (ADF)