Azure Databricks Data Engineer

Infosys; Hyderabad, Pune District, Bengaluru; On-site; INR 850,000 - 1,200,000; Full time

Job summary

Infosys in Hyderabad is hiring a Data Bricks-Data Engineer with 2 to 5 years of experience. The role focuses on building scalable data pipelines, Spark applications, and Delta Lake-based Lakehouse architecture to support data modernization initiatives.

The candidate will collaborate with data scientists, BI developers, and stakeholders to ensure data quality, governance, and efficient deployment pipelines in a cloud-friendly environment.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks.
• Build and optimize ETL/ELT processes for large-scale data ingestion and transformation.
• Develop Spark and PySpark applications for batch and real-time processing.
• Implement Lakehouse architecture using Delta Lake.
• Create and maintain data models, tables, and data warehouses.
• Optimize Databricks jobs, Spark performance, and query execution.
• Integrate data from multiple cloud and on-premises sources.
• Ensure data quality, governance, and security standards.
• Work with Unity Catalog for data governance and access management.
• Automate deployments using CI/CD and Infrastructure as Code (Terraform).
• Monitor and troubleshoot production data pipelines.
• Collaborate with Data Scientists, BI Developers, and Business Stakeholders.
• Support cloud migration and data modernization initiatives.

Job description

Position: Data Bricks-Data Engineer

Experience: 2 to 5 years

Location: Hyderabad

Employment Type: Full-Time

Roles & Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks.
• Build and optimize ETL/ELT processes for large-scale data ingestion and transformation.
• Develop Spark and PySpark applications for batch and real-time processing.
• Implement Lakehouse architecture using Delta Lake.
• Create and maintain data models, tables, and data warehouses.
• Optimize Databricks jobs, Spark performance, and query execution.
• Integrate data from multiple cloud and on-premises sources.
• Ensure data quality, governance, and security standards.
• Work with Unity Catalog for data governance and access management.
• Automate deployments using CI/CD and Infrastructure as Code (Terraform).
• Monitor and troubleshoot production data pipelines.
• Collaborate with Data Scientists, BI Developers, and Business Stakeholders.
• Support cloud migration and data modernization initiatives.