ADF (Azure Data Factory) + Databricks+ Snowflake

Infosys Limited; Bengaluru; On-site; INR 1,800,000 - 2,400,000; Full time

Job summary

Infosys Limited in Bengaluru is seeking a Senior Data Engineer with 7–9 years of hands-on experience to lead data ingestion and orchestration using Azure Data Factory and Databricks. You will design scalable pipelines for batch and incremental loads and drive best practices across teams.

The role requires strong SQL skills, experience with Snowflake, and a track record of reliable ETL/ELT workflows, including monitoring and incident response.

Qualifications

• 7–9 years of hands-on experience with ADF and Databricks for production data pipelines.
• Proven ETL/ELT design, deployment, monitoring, and failure-recovery patterns.
• Strong SQL skills and experience with large datasets and data quality controls.
• Experience leading technical delivery and collaborating with cross-functional teams.

Responsibilities

• Lead end-to-end data ingestion and orchestration workflows using ADF, including scheduling and error handling.
• Design scalable Databricks pipelines using Spark-based transformations for batch and incremental loads.
• Build and optimize ELT/ETL pipelines with Snowflake as target/source, focusing on performance and cost.
• Define data pipeline standards and enforce engineering best practices across the team.
• Implement monitoring, alerting, and runbooks; drive incident triage and root-cause analysis.
• Collaborate with stakeholders to translate requirements into designs, estimates and delivery plans.

Skills

ADF

Databricks

Snowflake

SQL

ETL/ELT

Data pipelines

Spark

CI/CD for data engineering

Education

BTECH

MTECH

MCA

MSC

Tools

Azure Databricks

Snowflake

ADF

SQL tooling

Job description

Job ID/Reference Code INFSYS-INDEED1-252687

Work Experience 7 - 9 Years

Educational Requirements MCA,MSc,MTech,Bachelor of Engineering,BTech

Service Line Data & Analytics Unit

Responsibilities

• Lead end-to-end implementation of data ingestion and orchestration workflows using ADF, including scheduling, dependency management, parameterization, and error handling.
• Design and develop scalable data processing pipelines in Databricks using Spark-based transformations for batch and incremental loads.
• Build and optimize ELT/ETL patterns integrating Snowflake as a target/source, ensuring performance, reliability, and cost efficiency.
• Define data pipeline standards (naming, modularity, reusability) and enforce engineering best practices across the team.
• Implement monitoring, alerting, and operational runbooks for production pipelines; drive incident triage and root-cause analysis.
• Collaborate with stakeholders to translate requirements into technical designs, estimates, and delivery plans; manage risks and dependencies.
• Conduct code reviews, mentor engineers, and guide technical decisions to ensure maintainable and secure solutions.
• Improve pipeline performance through tuning, partitioning strategies, and efficient data layout/processing approaches.

Minimum Qualifications

• BTECH, MTECH, MCA, or MSC.
• 7–9 years of overall experience with strong hands-on expertise in ADF and Databricks for building production-grade data pipelines.
• Proven experience designing and supporting reliable ETL/ELT workflows, including scheduling, retries, and failure recovery patterns.
• Strong SQL skills and experience working with large datasets and data quality considerations.
• Experience collaborating with cross-functional teams and leading technical delivery with ownership mindset.

Additional Responsibilities

• Strong experience integrating and optimizing workloads with Snowflake, including loading strategies and performance tuning.
• Experience implementing medallion/lakehouse-style architectures and scalable data modeling patterns for analytics consumption.
• Familiarity with CI/CD practices for data engineering workflows and automated testing approaches for pipelines.
• Experience with production observability practices (pipeline metrics, logging, alerting) and operational excellence.
• Demonstrated ability to mentor team members, drive design discussions, and influence engineering standards across projects.

Technical and Professional Requirements

Technology->Cloud Integration->Azure Data Factory (ADF)Technology->Data Engineering->DatabricksTechnology->Data on Cloud->Snowflake

Preferred Skills

Technology->Cloud Integration->Azure Data Factory (ADF) Technology->Data on Cloud-DataStore->Snowflake Technology->Data Engineering->Databricks