Databricks Data Engineer

Infosys; Dadri, Hyderabad, Mumbai; Hybrid; INR 600,000 - 1,200,000; Full time

Job summary

Infosys in India is seeking a Data Engineer to design, develop, and maintain data pipelines using Databricks and PySpark. You will build scalable ETL/ELT solutions for processing large datasets and work with Delta Lake on cloud-based platforms.

Responsibilities include optimizing data processing, integrating data from multiple sources, and collaborating with cross-functional teams to deliver business requirements, leveraging SQL and Spark SQL skills.

Qualifications

• 2-5 years of experience in Data Engineering.
• Hands-on experience with Databricks and PySpark.
• Strong SQL and Spark SQL knowledge.
• Experience in ETL development and data integration.
• Exposure to Azure Data Factory and cloud environments.
• Strong analytical and problem-solving skills.

Responsibilities

• Design, develop, and maintain data pipelines using Databricks.
• Build ETL/ELT solutions for processing large datasets.
• Develop scalable data transformation workflows using PySpark and Spark SQL.
• Work with Delta Lake and cloud-based data platforms.
• Integrate data from multiple source systems.
• Optimize data processing performance and reliability.
• Collaborate with cross-functional teams to deliver business requirements.

Skills

SQL

Analytical thinking

Problem-solving

Data engineering concepts

Tools

Databricks

PySpark

Spark SQL

Azure Data Factory

Job description

Role & Responsibilities

• Design, develop, and maintain data pipelines using Databricks.
• Build ETL/ELT solutions for processing large datasets.
• Develop scalable data transformation workflows using PySpark and Spark SQL.
• Work with Delta Lake and cloud-based data platforms.
• Integrate data from multiple source systems.
• Optimize data processing performance and reliability.
• Collaborate with cross-functional teams to deliver business requirements.

Preferred Candidate Profile

• 2-5 years of experience in Data Engineering.
• Hands-on experience with Databricks and PySpark.
• Strong SQL and Spark SQL knowledge.
• Experience in ETL development and data integration.
• Exposure to Azure Data Factory and cloud environments.
• Strong analytical and problem-solving skills.