Data Engineer-Data Platforms-Azure

IBM; Bengaluru; On-site; INR 1,400,000 - 2,400,000; Full time

Job summary

IBM Bengaluru is seeking a Data Engineer specializing in Data Platforms on Azure. You will design, build, and operate batch and real-time data pipelines, and implement the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB.

You will apply Open Source tools like Apache Airflow and dbt, with Spark and Python/Scala, to support robust data engineering solutions. This role emphasizes Azure data platform components and seamless processing.

Qualifications

• Proficiency with Azure services for batch and real-time data pipelines.
• Experience with ADLS2, Blob Storage, SQLDW, Synapse Spark/SQL, Python functions.
• Open source tech exposure: Apache Airflow, dbt, Spark Python/Scala.
• Design, build, and operate Azure data pipelines.
• Create data platform components and data layer.

Responsibilities

• Design, build, and operate batch and real-time data pipelines using Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub.
• Design, build, and operate the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB.
• Implement and maintain Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics with Spark and SQL, Azure functions with Python, Azure Purview, and Cosmos DB.
• Utilize open source technologies like Apache Airflow and dbt, Spark/Python, or Spark/Scala to support data engineering solutions.
• Operate and maintain data engineering solutions to ensure seamless batch and real-time data processing.

Skills

Azure Synapse Analytics

Azure Data Factory

Azure DataBricks

Event Hub

Apache Airflow

dbt

Spark

Python

Spark/Scala

Cosmos DB

Education

Master's Degree

Job description

Introduction

A career in IBM Consulting is built on long‑term client relationships and close collaboration worldwide. You’ll work with leading companies across industries, helping them shape their hybrid cloud and AI journeys. With support from our strategic partners, robust IBM technology, and Red Hat, you’ll have the tools to drive meaningful change and accelerate client impact. At IBM Consulting, curiosity fuels success. You’ll be encouraged to challenge the norm, explore new ideas, and create innovative solutions that deliver real results. Our culture of growth and empathy focuses on your long‑term career development while valuing your unique skills and experiences.

Introduction

A career in IBM Consulting is built on long‑term client relationships and close collaboration worldwide. You’ll work with leading companies across industries, helping them shape their hybrid cloud and AI journeys. With support from our strategic partners, robust IBM technology, and Red Hat, you’ll have the tools to drive meaningful change and accelerate client impact. At IBM Consulting, curiosity fuels success. You’ll be encouraged to challenge the norm, explore new ideas, and create innovative solutions that deliver real results. Our culture of growth and empathy focuses on your long‑term career development while valuing your unique skills and experiences.

Your Role And Responsibilities

As a Data Engineer specializing in Data Platforms on Azure, you will advise on, develop, and maintain data engineering solutions on the Azure Cloud ecosystem. You will design, build, and operate batch and real‑time data pipelines using various Azure services. Your primary responsibilities will include:

• Design and Build Data Pipelines: Design, build, and operate batch and real‑time data pipelines using Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub.
• Develop Data Layer: Design, build, and operate the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB.
• Implement Azure Data Platform: Implement and maintain Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics with Spark and SQL, Azure functions with Python, Azure Purview, and Cosmos DB.
• Utilize Open Source Technologies: Apply proficiency in open source technologies like Apache Airflow and dbt, Spark / Python, or Spark / Scala to support data engineering solutions.
• Ensure Seamless Operations: Operate and maintain data engineering solutions to ensure seamless batch and real‑time data processing.

Preferred Education

Master's Degree

Required Technical And Professional Expertise

• Exposure to Azure Toolset: Proficiency in Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub to design, build, and operate batch and real‑time data pipelines.
• Data Engineering on Azure: Experience working with Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics with Spark and SQL, Azure functions with Python, Azure Purview, and Cosmos DB.
• Open Source Technologies: Exposure to open source technologies like Apache Airflow and dbt, Spark / Python, or Spark / Scala to support data engineering solutions.
• Data Pipeline Development: Experience developing and maintaining data engineering solutions on the Azure Cloud ecosystem, including designing, building, and operating batch and real‑time data pipelines.
• Azure Data Layer Design: Experience designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB.

Preferred Technical And Professional Experience

• Proficiency in Azure Services: Exposure to Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub to design, build, and operate batch and real‑time data pipelines.
• Open Source Technologies: Exposure to open source technologies like Apache Airflow and dbt, Spark / Python, or Spark / Scala to support data engineering solutions.
• Data Engineering on Azure: Experience working with Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics with Spark and SQL, Azure functions with Python, Azure Purview, and Cosmos DB.