Azure Datafactory

Infosys; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Infosys seeks a data engineer to design, build, and optimize cloud-based data pipelines using Azure Data Factory and Databricks in Bengaluru. You will collaborate with data engineers and platform teams to translate business needs into scalable, reliable data solutions.

You will be responsible for transforming raw data into meaningful insights, validating data quality, and ensuring smooth data movement across systems with robust monitoring and CI/CD practices.

Qualifications

• Hands-on experience with Azure Data Factory including pipeline development and scheduling.
• Databricks for data processing and transformation workflows.
• Experience with Azure Synapse Analytics and data lake storage.
• Ability to troubleshoot pipelines and optimize performance.

Responsibilities

• Design, develop, and maintain end-to-end data pipelines using Azure Data Factory.
• Build and orchestrate data transformations with Databricks.
• Implement monitoring, alerting, and failure handling for reliable executions.
• Perform data validation and reconciliation across sources and targets.
• Optimize pipeline performance and orchestration logic.
• Collaborate with stakeholders to translate requirements into technical designs.
• Maintain documentation for pipelines, datasets, and runbooks.
• Support deployments and environment promotions via change management practices.

Skills

Azure Data Factory

Databricks

PySpark

Delta Lake

Azure Synapse Analytics

Azure Data Lake Storage

CI/CD

Data pipelines

Education

B.Tech

M.Tech

MCA

MSc

Tools

Azure Data Factory

Databricks

Azure Synapse Analytics

Azure Data Lake Storage

Delta Lake

PySpark

Job description

Job DescriptionJoin a collaborative data engineering team where your work directly powers trusted analytics and smarter business decisions. In this role, youll build and optimize cloud-based data pipelines using Azure Data Factory and Databricks, helping teams move from raw data to meaningful insights with speed and reliability. Youll partner closely with stakeholders, data engineers, and platform teams to understand data needs, design scalable solutions, and ensure smooth, automated data movement across systems. If you enjoy solving real-world data challenges, improving performance, and bringing structure to complex datasetswhile working in a supportive, learning-focused environmentthis opportunity will help you grow your impact and deepen your expertise in modern Azure data platforms.

Roles ResponsibilitiesKey Responsibilities:

• Design, develop, and maintain end-to-end data pipelines using Azure Data Factory for batch and scheduled workloads.
  
• Build and orchestrate data transformations and processing workflows using Databricks aligned to business requirements.
  
• Implement robust pipeline monitoring, alerting, and failure handling to ensure reliable and repeatable executions.
  
• Perform data validation and reconciliation checks to ensure accuracy, completeness, and consistency across sources and targets.
  
• Optimize pipeline performance by tuning ADF activities, improving orchestration logic, and streamlining transformations in Databricks.
  
• Collaborate with cross-functional teams to gather requirements, estimate effort, and deliver solutions within timelines.
  
• Maintain technical documentation for pipelines, datasets, schedules, dependencies, and operational runbooks.
  
• Support deployments and environment promotions by following structured release and change management practices.
  Minimum Qualifications:
  
• Education: BTECH, MTECH, MCA, MSC.
  
• Overall experience: 35 years in data engineering / data integration roles.
  
• Hands-on experience with Azure Data Factory including pipeline development, scheduling, triggers, and integration patterns.
  
• Hands-on experience with Databricks for data processing and transformation workflows.
  
• Ability to troubleshoot pipeline failures, analyze logs, and implement corrective actions to improve stability.
  Technical RequirementAzure Synapse Analytics, Azure Data Lake Storage (ADLS), PySpark, Delta Lake, CI/CD for data pipelines, AZURE DATAFACTORY
  Additional Responsibility
• Experience designing scalable orchestration patterns in ADF (parameterization, reusable components, dependency handling).
  
• Strong understanding of data transformation best practices and implementing efficient processing in Databricks.
  
• Exposure to building data quality checks and operational dashboards for pipeline health and SLA tracking.
  
• Experience collaborating with stakeholders to translate business requirements into technical pipeline designs and delivery plans.
  
• Familiarity with performance tuning approaches for cloud data pipelines and distributed processing workloads.
  Educational RequirementMCA,MSc,MTech,Bachelor of Engineering
  Preferred SkillsTechnology->Cloud Integration->Azure Data Factory (ADF)
  Service LineData Analytics Unit