Databricks Data Engineer

Infosys; Ahmedabad District, Chennai District, Bengaluru; Hybrid; INR 800,000 - 1,400,000; Full time

Job summary

Infosys is seeking a data engineer to design, develop, and optimize data pipelines using Databricks. You will build ETL/ELT solutions, implement PySpark based transformations, and work with Delta Lake on cloud platforms.

The role requires strong SQL/Spark SQL skills, data integration experience, and collaboration with cross-functional teams to meet business requirements.

Qualifications

• 2-5 years of experience in Data Engineering.
• Hands-on experience with Databricks and PySpark.
• Strong SQL and Spark SQL knowledge.
• Experience in ETL development and data integration.
• Exposure to Azure Data Factory and cloud environments.
• Strong analytical and problem-solving skills.

Responsibilities

• Design, develop, and maintain data pipelines using Databricks.
• Build ETL/ELT solutions for processing large datasets.
• Develop scalable data transformation workflows using PySpark and Spark SQL.
• Work with Delta Lake and cloud-based data platforms.
• Integrate data from multiple source systems.
• Optimize data processing performance and reliability.
• Collaborate with cross-functional teams to deliver business requirements.

Skills

Databricks

PySpark

Spark SQL

SQL

ETL development

Data integration

Azure Data Factory

Cloud platforms

Analytical thinking

Tools

Databricks

Job description

Role & Responsibilities

• Design, develop, and maintain data pipelines using Databricks.


• Build ETL/ELT solutions for processing large datasets.


• Develop scalable data transformation workflows using PySpark and Spark SQL.


• Work with Delta Lake and cloud-based data platforms.


• Integrate data from multiple source systems.


• Optimize data processing performance and reliability.


• Collaborate with cross-functional teams to deliver business requirements.

Preferred Candidate Profile

• 2-5 years of experience in Data Engineering.


• Hands-on experience with Databricks and PySpark.


• Strong SQL and Spark SQL knowledge.


• Experience in ETL development and data integration.


• Exposure to Azure Data Factory and cloud environments.


• Strong analytical and problem-solving skills.