Azure Data Engineer | 6+ Years | Bangalore

Apexon; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Apexon is seeking a Data Engineer with extensive experience to design and support scalable data solutions on Azure and Big Data platforms. You will work with Azure Databricks, PySpark, Hadoop, Talend, Delta Lake, and Azure Data Factory to build robust data pipelines for analytics, reporting, and AI initiatives.

You will collaborate with data architects, engineers, and business stakeholders to develop, optimize, and govern data pipelines, ensuring performance, reliability, and cost efficiency

Qualifications

• 68 years of experience in Data Engineering / Big Data Engineering.
• Hands-on experience with Azure Databricks and PySpark.
• Experience with Hadoop ecosystem.
• Experience with Talend for ETL/data integration.
• Experience with Delta Lake / Lakehouse architecture.
• Experience with Azure Data Factory (ADF).
• Strong SQL and data transformation skills.
• Experience with large-scale datasets and distributed data processing.
• Knowledge of cloud-based data engineering concepts.
• Exposure to CI/CD and DevOps practices.

Responsibilities

• Develop and maintain scalable data pipelines using Azure Databricks and PySpark.
• Build data transformation and processing solutions using Delta Lake and Lakehouse architecture.
• Develop and optimize ETL/ELT pipelines using Talend for large-scale data ingestion and transformation.
• Work with Hadoop ecosystem technologies for distributed data processing and storage.
• Integrate data from multiple sources including databases, APIs, files, and streaming platforms.
• Build and manage data orchestration workflows using Azure Data Factory (ADF).
• Optimize Spark jobs, Databricks clusters, and data pipelines for performance and cost efficiency.
• Implement data quality, validation, security, and governance practices.
• Collaborate with architects, business stakeholders, and data teams to understand requirements and deliver technical solutions.
• Participate in code reviews, technical discussions, and implementation of engineering best practices.
• Work with DevOps teams on CI/CD and deployment automation for data engineering solutions.
• Troubleshoot and resolve data pipeline and production issues.

Skills

Data engineering experience

SQL

Distributed data processing

Cloud data integration concepts

CI/CD and DevOps exposure

Tools

Azure Databricks

PySpark

Hadoop ecosystem

Talend

Delta Lake

Azure Data Factory (ADF)

Spark

Job description

Job Summary

We are looking for a Data Engineer with 68 years of experience in building and supporting scalable data engineering solutions across Azure and Big Data platforms. The ideal candidate should have strong hands‑on experience with Azure Databricks, PySpark, Hadoop, Talend, Delta Lake, and Azure Data Factory (ADF).

The role will involve developing, integrating, and optimizing data pipelines that support analytics, reporting, and AI/ML initiatives, while working closely with data architects, engineers, and business stakeholders.

Key Responsibilities

• Develop and maintain scalable data pipelines using Azure Databricks and PySpark.
• Build data transformation and processing solutions using Delta Lake and Lakehouse architecture.
• Develop and optimize ETL/ELT pipelines using Talend for large‑scale data ingestion and transformation.
• Work with Hadoop ecosystem technologies for distributed data processing and storage.
• Integrate data from multiple sources including databases, APIs, files, and streaming platforms.
• Build and manage data orchestration workflows using Azure Data Factory (ADF).
• Optimize Spark jobs, Databricks clusters, and data pipelines for performance and cost efficiency.
• Implement data quality, validation, security, and governance practices.
• Collaborate with architects, business stakeholders, and data teams to understand requirements and deliver technical solutions.
• Participate in code reviews, technical discussions, and implementation of engineering best practices.
• Work with DevOps teams on CI/CD and deployment automation for data engineering solutions.
• Troubleshoot and resolve data pipeline and production issues.

Must-Have Skills

• 68 years of experience in Data Engineering / Big Data Engineering.
• Strong hands‑on experience with Azure Databricks and PySpark.
• Strong experience with Hadoop and its ecosystem.
• Hands‑on experience with Talend for ETL/data integration.
• Good experience with Delta Lake / Lakehouse architecture.
• Strong experience with Azure Data Factory (ADF).
• Strong SQL and data transformation skills.
• Experience working with large‑scale datasets and distributed data processing.
• Understanding of cloud‑based data engineering and data integration concepts.
• Exposure to CI/CD and DevOps practices.

Good to Have

• Experience with Azure Data Lake Storage (ADLS).
• Experience with streaming technologies.
• Knowledge of data governance and security.
• Exposure to data engineering solutions supporting AI/ML workloads.
• Experience with Agile development methodologies.