Senior Data Engineer Azure Data Engineering

Quess; Bengaluru; On-site; INR 2,400,000 - 4,200,000; Full time

Job summary

Quess is seeking a Senior Data Engineer in Bengaluru to design and implement scalable data and AI solutions using Azure Data Engineering, Databricks, Python, and AI Foundry.

The role involves building end-to-end data pipelines, applying Medallion Architecture, and working closely with business stakeholders to translate requirements into technical solutions. Collaboration, governance, and CI/CD with GitHub are key aspects of the position.

Qualifications

• Senior Data Engineer with Azure Data Engineering and AI focus.
• Strong Python development experience.
• Hands-on with Azure Databricks, ADF and SQL databases.

Responsibilities

• Design scalable, end-to-end data pipelines and AI solutions.
• Implement Medallion Architecture across data platforms.
• Develop AI applications and agents using Azure AI Foundry.
• Collaborate with stakeholders to translate requirements into technical solutions.
• Lead CI/CD practices using GitHub and maintain governance.

Skills

Azure Data Engineering

Azure Databricks

Python

Azure Data Factory

Azure SQL DB

Databricks Apps

CI/CD

GitHub

Medallion Architecture

Azure AI Foundry

LLMs

Prompt Engineering

Job description

JOB DESCRIPTION

Senior Data Engineer Azure Data Engineering & AI

Experience: 5 to 11 Years

Notice Period: Immediate Joiners / Candidates serving up to 15 days notice period

Location:Begalore

Role Overview

We are looking for a highly skilled Senior Data Engineer with strong expertise in Azure Data Engineering, Databricks, and Azure AI Foundry. The candidate will be responsible for designing, developing, and implementing scalable end-to-end data and AI solutions.

The role requires a technology expert who can work closely with technical teams and business stakeholders, translate business requirements into effective data solutions, and drive the implementation of modern data and AI capabilities.

Mandatory Technical Skills

• Strong expertise in Azure Data Engineering
• Hands-on experience with Azure Databricks
• Strong programming skills in Python
• Hands-on experience with Azure Data Factory (ADF)
• Experience with Azure SQL Database
• Experience working with Databricks Apps
• Strong understanding of CI/CD and GitHub
• Strong understanding and implementation experience of Medallion Architecture
• Strong hands-on experience with Azure AI Foundry
• Experience designing, developing, and deploying AI applications and AI agents using Azure AI Foundry
• Strong experience working with Large Language Models (LLMs)
• Strong knowledge of Prompt Engineering
• Ability to build scalable, secure, and governed AI solutions

Nice to Have

• Experience with Databricks Mosaic AI
• Knowledge of new and emerging Databricks features and capabilities
• Experience integrating AI/ML capabilities with modern data platforms

Key Responsibilities

• Design and build scalable, secure, and high-performance end-to-end data engineering solutions.
• Develop data pipelines and transformation workflows using Azure Databricks, Python, ADF, and Azure SQL.
• Implement and maintain data solutions based on Medallion Architecture.
• Design, develop, and deploy AI applications and agents using Azure AI Foundry.
• Work with LLMs and develop effective prompt engineering strategies for AI-powered solutions.
• Integrate data engineering and AI capabilities to deliver business-focused solutions.
• Implement CI/CD pipelines and development practices using GitHub.
• Drive data foundation initiatives and support the organization broader data strategy.
• Work closely with business stakeholders to understand requirements and translate them into technical solutions.
• Act as a Subject Matter Expert (SME) across data engineering and AI technologies.
• Mentor junior data engineers and contribute to technical knowledge-sharing initiatives.
• Drive implementation efficiency, reduce development and operational costs, and improve speed of delivery.
• Support incident management, problem management, and change management activities.
• Present technical solutions, findings, and recommendations to senior stakeholders.
• Contribute to Center of Excellence (CoE) and Community of Practice (CoP) initiatives.
• Stay current with emerging capabilities across Azure, Databricks, AI, and LLM technologies.