Data Engineer – Azure

Codecraft; Bengaluru; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

A leading data solutions company is seeking a Data Engineer to design and implement enterprise-scale data pipelines using Azure services. The ideal candidate will have 4-7 years of experience with skills in Azure Databricks, microservices architecture, and data quality management. This role requires collaboration with various stakeholders to ensure data integrity and availability across multiple business functions. A strong background in CI/CD automation and problem-solving is essential for success in this position.

Qualifications

• 4-7 years of experience in data engineering.
• Strong experience with Azure Databricks and Service Fabric.
• Proficiency in Python, Scala, SQL.

Responsibilities

• Design and operate scalable data pipelines and microservices.
• Develop artifacts as version-controlled code with CI/CD automation.
• Ensure data availability, integrity, and quality.

Skills

Azure Databricks

Azure Service Fabric

Python

SQL

CI/CD Automation

Data Modeling

Problem-solving

Tools

Git

Docker

Kubernetes

Job description

Experience: 4-7 Years

Location: Bangalore

No. of Positions: 01

Role Description

As a Data Engineer, you will be responsible for end-to-end design and operation of enterprise-scale data pipelines and infrastructure using Azure Databricks, Azure Service Fabric, and related Azure services. You will ensure data availability, reliability, and quality across HR, Procurement, and Finance functions while applying software engineering best practices such as modular design, TDD, and CI/CD automation.

You will collaborate with business stakeholders, architects, ML/AI engineers, Fabric service teams, and Power BI developers to align delivery with business outcomes, operating within an agile team structure.

Key Responsibilities

• Data Pipeline & Microservices Engineering: Design and operate scalable, high-performance data pipelines and service Fabric, and other Azure ecosystem services.
• Code as Assets: Develop all artifacts as version-controlled code (Git) with CI/CD automation.
• Data Reliability & Quality: Ensure availability, integrity, and quality of data supporting analytics in HR, Procurement, and Finance.
• Analytical Assets: Build data products and assets leveraging software engineering best practices.
• Engineering Practices: Apply modular design, TDD, reusable code patterns, and performance optimization.
• Fabric maintain microservices on Azure Service Fabric, ensuring high availability, resiliency, and scalability.
• Collaboration: Work closely with architects, ML/AI engineers, functional IT, Power BI developers, Fabric service teams, and business stakeholders to align solutions with requirements.
• Agile Contribution: Actively participate in sprint planning, retrospectives, and demos within an enterprise agile product team.
• Monitoring & Optimization: Monitor and improve data, failure points, and optimization opportunities.

Required Skills & Qualifications

• Strong experience with Azure Databricks, Azure Service Fabric, and the broader Azure Data ecosystem (ADF, Synapse, Data Lake, Event Hub).
• Proficiency in Python / Scala / SQL for data engineering.
• Hands-on experience in CI/CD automation, Git version control, and test-driven development (TDD).
• Deep understanding of data modeling, ETL/ELT pipelines, modular design patterns.
• Experience in building, deploying, and maintaining microservices in Azure Service Fabric (or equivalent orchestrated service mesh / containerized services).
• Ability to build analytical data products using engineering-first principles.
• Experience working in agile, enterprise-scale environments.
• Strong problem-solving and debugging skills.

Preferred Qualifications

• Exposure to AI/ML pipelines integration in data workflows.
• Experience with Power BI or equivalent BI platforms.
• Knowledge of DevOps practices in a cloud-first environment.
• Microsoft Azure Data Engineer Associate certification (or equivalent).
• Familiarity with containerization technologies (e.g. Docker), orchestration (e.g. Kubernetes), in addition to Service Fabric.

Desired Profile

• A pragmatic engineer passionate about building reliable, reusable, scalable data and microservice solutions.
• Collaborative and able to work seamlessly with business, IT, analytics, and infrastructure (service fabric) stakeholders.
• Self-driven with strong ownership, accountability, and problem-solving.