Azure Data Engineer | 7+ Years | Bangalore

Apexon; Bengaluru; On-site; INR 1,500,000 - 2,800,000; Full time

Job summary

Apexon in Bengaluru is seeking an Azure Data Engineer with 6-8 years of experience to design and deliver scalable data pipelines on Azure Databricks, PySpark, Delta Lake, and Lakehouse architectures. You will work with Hadoop ecosystems and Talend for ETL, while integrating data from diverse sources to support analytics and AI/ML initiatives.

The role involves optimizing pipelines, ensuring data quality and governance, and collaborating with data architects and business stakeholders to implement

Qualifications

• 6–8 years of experience in Data Engineering / Big Data Engineering.
• Strong hands-on experience with Azure Databricks and PySpark.
• Strong experience with Hadoop and its ecosystem.
• Hands-on experience with Talend for ETL/data integration.
• Good experience with Delta Lake / Lakehouse architecture.
• Strong experience with Azure Data Factory (ADF).
• Strong SQL and data transformation skills.
• Experience working with large-scale datasets.

Responsibilities

• Develop and maintain scalable data pipelines using Azure Databricks and PySpark.
• Build data transformation and processing solutions using Delta Lake and Lakehouse architecture.
• Develop and optimize ETL/ELT pipelines using Talend for large-scale data ingestion and transformation.
• Work with Hadoop ecosystem technologies for distributed data processing and storage.
• Integrate data from multiple sources including databases, APIs, files, and streaming platforms.
• Build and manage data orchestration workflows using Azure Data Factory (ADF).
• Optimize Spark jobs, Databricks clusters, and data pipelines for performance and cost efficiency.
• Implement data quality, validation, security, and governance practices.
• Collaborate with architects, business stakeholders, and data teams to understand requirements and deliver technical solutions.
• Participate in code reviews, technical discussions, and implementation of engineering best practices.
• Work with DevOps teams on CI/CD and deployment automation for data engineering solutions.
• Troubleshoot and resolve data pipeline and production issues.

Skills

Azure Databricks

PySpark

Hadoop

Delta Lake

Lakehouse

Azure Data Factory

SQL

ETL/ELT

Tools

Talend

Job description

Requirement details:

Role: Azure Data Engineer

Location: Bengaluru / 5 days WFO

Interview Process

• Round 1: Face-to-Face Interview


• Round 2: Virtual Techno Manager

Job Summary

We are looking for a Data Engineer with 6to 8 years of experience in building and supporting scalable data engineering solutions across Azure and Big Data platforms. The ideal candidate should have strong hands-on experience with Azure Databricks, PySpark, Hadoop, Talend, Delta Lake, and Azure Data Factory (ADF).

The role will involve developing, integrating, and optimizing data pipelines that support analytics, reporting, and AI/ML initiatives, while working closely with data architects, engineers, and business stakeholders.

Key Responsibilities

• Develop and maintain scalable data pipelines using Azure Databricks and PySpark.


• Build data transformation and processing solutions using Delta Lake and Lakehouse architecture.


• Develop and optimize ETL/ELT pipelines using Talend for large-scale data ingestion and transformation.


• Work with Hadoop ecosystem technologies for distributed data processing and storage.


• Integrate data from multiple sources including databases, APIs, files, and streaming platforms.


• Build and manage data orchestration workflows using Azure Data Factory (ADF).


• Optimize Spark jobs, Databricks clusters, and data pipelines for performance and cost efficiency.


• Implement data quality, validation, security, and governance practices.


• Collaborate with architects, business stakeholders, and data teams to understand requirements and deliver technical solutions.


• Participate in code reviews, technical discussions, and implementation of engineering best practices.


• Work with DevOps teams on CI/CD and deployment automation for data engineering solutions.


• Troubleshoot and resolve data pipeline and production issues.

Must-Have Skills

• 6to 8 years of experience in Data Engineering / Big Data Engineering.


• Strong hands-on experience with Azure Databricks and PySpark.


• Strong experience with Hadoop and its ecosystem.


• Hands-on experience with Talend for ETL/data integration.


• Good experience with Delta Lake / Lakehouse architecture.


• Strong experience with Azure Data Factory (ADF).


• Strong SQL and data transformation skills.


• Experience working with large-scale datasets and distributed data processing.


• Understanding of cloud-based data engineering and data integration concepts.


• Exposure to CI/CD and DevOps practices.

Good to Have

• Experience with Azure Data Lake Storage (ADLS).


• Experience with streaming technologies.


• Knowledge of data governance and security.


• Exposure to data engineering solutions supporting AI/ML workloads.


• Experience with Agile development methodologies.