ADF, Databrick

Infosys; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Infosys in Bengaluru, India is seeking a data engineering professional to design and develop ETL/ELT pipelines using Azure Data Factory and Databricks, and to integrate data from SQL Server, ADLS, APIs, and third-party sources into scalable data lakes and Azure data warehouses.

You will monitor pipelines, troubleshoot failures and performance bottlenecks, enforce data quality and governance, and document technical solutions to support deployment and operations.

Qualifications

• Experience designing ETL/ELT pipelines on Azure.
• Hands-on with Azure Data Factory and Azure Databricks.
• Experience integrating data from SQL Server, ADLS, APIs.

Responsibilities

• Develop and optimize data processing workflows on Azure.
• Build data lakes and data warehouse solutions on Azure.
• Monitor pipelines and troubleshoot failures and data quality.

Tools

Azure Services

Azure Data Factory (ADF)

Azure Databricks

Azure Data Lake Storage (ADLS Gen2)

Azure SQL Database / SQL Server

Azure Synapse Analytics (preferred)

Job description

Design and develop ETL/ELT pipelines using Azure Data Factory (ADF). Build and optimize data processing workflows using Azure Databricks (PySpark/Spark SQL). Integrate data from various sources such as SQL Server, Azure SQL, ADLS, APIs, and third-party applications. Implement data transformation, cleansing, and validation processes. Develop and maintain data lakes and data warehouse solutions on Azure. Monitor and troubleshoot pipeline failures, performance bottlenecks, and data quality issues. Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. Implement security, governance, and best practices for data management. Create technical documentation and support deployment activities.

Required Skills

• Azure Services
• Azure Data Factory (ADF)
• Azure Databricks
• Azure Data Lake Storage (ADLS Gen2)
• Azure SQL Database / SQL Server
• Azure Synapse Analytics (preferred)