Azure Data Engineer

Acme Services Private Limited; Bengaluru; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Acme Services Private Limited is seeking a senior data engineer to design, build, and maintain Azure data solutions in Bengaluru. You will develop end-to-end ELT/ETL pipelines, process large datasets with PySpark, Python, and SQL, and optimize data models for analytics and reporting.

You’ll work with Data Factory, Databricks, Synapse, and engage with stakeholders to deliver reliable data platforms. You’ll also manage versioned deployments, CI/CD workflows, and documentation using MKDocs, while

Qualifications

• Experience designing and implementing end-to-end Azure data platforms.
• Proficiency with PySpark, Python, SQL and data modeling for analytics.
• Hands-on with ETL/ELT pipelines, data ingestion and integration.
• Familiarity with CI/CD for data platforms and versioned deployments.
• Ability to collaborate with stakeholders and document technical designs.

Responsibilities

• Design, develop, and maintain end-to-end Azure data engineering solutions.
• Build scalable ELT/ETL pipelines for data ingestion and transformation.
• Develop data processing workflows using PySpark, Python, and SQL.
• Design data models optimized for analytics and reporting.
• Manage data integration frameworks using Azure Data Factory and Synapse.
• Collaborate with teams to translate requirements into technical solutions.
• Present design approaches, updates, and findings to stakeholders.
• Maintain source control, CI/CD pipelines, and deployment processes.
• Mentor junior data engineers and contribute to best practices.

Skills

PySpark

Python

SQL

Data modeling

ETL pipelines

CI/CD

Agile/Scrum

Stakeholder communication

Documentation

Tools

Azure Data Factory

Azure Databricks

Azure Data Lake Store

Azure Synapse Analytics

GitHub

GitHub Actions

Azure DevOps

MKDocs

Job description

Job Description:

• Design, develop, and maintain end-to-end Azure data engineering solutions using Azure Data Factory, Azure Databricks, and Azure Data Lake Store.
• Build scalable and high-performance ELT/ETL pipelines for data ingestion, transformation, and integration from multiple source systems.
• Develop robust data processing workflows using PySpark, Python, and SQL within Azure Databricks.
• Design and implement data models optimized for analytics, reporting, and enterprise data platforms.
• Build and manage data integration and ingestion frameworks using Azure Data Factory and Synapse Analytics.
• Work with structured and unstructured datasets to deliver reliable and high-quality data solutions.
• Optimize data pipelines for performance, scalability, reliability, and cost efficiency.
• Collaborate with business stakeholders, architects, analysts, and engineering teams to translate business requirements into technical solutions.
• Present data design approaches, implementation updates, and technical findings to business and IT stakeholders.
• Manage source code, CI/CD pipelines, and deployment processes using GitHub, GitHub Actions, and Azure DevOps.
• Support incident management, problem management, and change management activities for data platforms.
• Mentor junior data engineers and contribute to engineering best practices, CoE, and knowledge-sharing initiatives.
• Ensure proper documentation of data solutions using MKDocs and maintain technical design artifacts.
• Participate in Agile/Scrum ceremonies including sprint planning, backlog refinement, stand-ups, reviews, and retrospectives.