Azure Data Factory Developer

Infosys; Hyderabad, Pune District, Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Infosys is seeking an ADF, Databricks, PySpark Engineer to design and build modern data pipelines in Bangalore with opportunities in Trivandrum. The role focuses on end-to-end data integration across Azure data services and scalable processing using Spark.

You will optimize workloads, implement data quality checks, and collaborate with stakeholders to deliver robust data solutions supporting analytics initiatives.

Qualifications

• Design, develop, and maintain data pipelines using ADF.
• Build scalable data processing with Azure Databricks, Python, and PySpark.
• Develop ETL/ELT workflows for data ingestion, transformation, and loading.
• Collaborate with business and data teams to define data requirements.

Responsibilities

• Work with ADLS, Azure SQL, and other Azure data services.
• Optimize Spark jobs and Databricks notebooks for performance and scalability.
• Implement data quality checks, validation rules, and monitoring processes.
• Create and manage end-to-end data integration solutions across multiple data sources.
• Collaborate with business stakeholders and data teams to understand data requirements.
• Troubleshoot and resolve data pipeline and performance issues.

Skills

ADF

Databricks

PySpark

Python

Azure

ETL/ELT

Azure Data Lake

CI/CD

Tools

Azure Databricks

Job description

Job Description:

Position: ADF, Databricks, PySpark Engineer

Location: Bangalore, Trivandrum

Roles & Responsibilities

• Design, develop, and maintain data pipelines using Azure Data Factory (ADF).
• Build scalable data processing solutions using Azure Databricks, Python, and PySpark.
• Develop ETL/ELT workflows for data ingestion, transformation, and loading.
• Work with Azure Data Lake Storage (ADLS), Azure SQL, and other Azure data services.
• Optimize Spark jobs and Databricks notebooks for performance and scalability.
• Implement data quality checks, validation rules, and monitoring processes.
• Create and manage end-to-end data integration solutions across multiple data sources.
• Collaborate with business stakeholders and data teams to understand data requirements.
• Troubleshoot and resolve data pipeline and performance issues.
• Follow coding standards, version control, and CI/CD best practices.
• Support data warehousing and analytics initiatives.
• Prepare technical documentation and maintain deployment artifacts.