Data Engineer

Aanseacore; Pune District; On-site; INR 1,200,000 - 2,100,000; Full time

Benefits offered by this job

Competitive Market Rate

Job summary

Aanseacore is seeking a skilled Data Engineer with hands-on experience in Microsoft Fabric and Azure Data Services to build scalable data pipelines and integration solutions. You will collaborate with cross-functional teams to develop reliable, high-performance data platforms that support analytics and enterprise reporting, using PySpark, Python, SQL and Azure technologies.

The role requires strong problem solving, data modeling expertise, and a passion for data quality and security within cloud

Qualifications

• Bachelor’s degree in a technical field as listed in the qualifications.
• Hands-on experience with Microsoft Fabric and Azure Data Services.
• Proficiency in PySpark, Python, and SQL for data transformation.
• Experience building scalable ETL/ELT pipelines for structured data.
• Experience integrating enterprise apps via REST/SOAP APIs and cloud data services.
• Strong data modeling, warehousing, and governance understanding.

Responsibilities

• Design, develop and implement scalable data engineering solutions using Fabric and Azure Data Services.
• Build and maintain ingestion and transformation pipelines with PySpark, Python and SQL.
• Design storage architecture and data models to support analytics and reporting.
• Develop API-driven and scheduled data ingestion workflows.
• Collaborate with cross-functional teams and participate in Agile ceremonies.
• Monitor pipelines with Spark UI and Azure Log Analytics; optimize performance.

Skills

Microsoft Fabric

Azure Data Services

PySpark

Python

SQL

ETL/ELT design

REST APIs

SOAP APIs

Agile/Scrum

Problem solving

Communication

Education

Bachelor's degree in Computer Science/IT/Data Engineering/Software Engineering

Tools

Azure DevOps

Git

CI/CD

Terraform

Spark UI

Azure Purview

Azure Active Directory

Job description

Job Description

We are seeking a skilled Data Engineer with hands-on experience in Microsoft Fabric and Azure Data Services to build scalable data pipelines and integration solutions. The ideal candidate will have expertise in data ingestion, transformation, and modeling using PySpark, Python, SQL, and Azure technologies to deliver secure, high-quality datasets for analytics and business applications. The role requires close collaboration with cross-functional teams to develop reliable, high-performance data platforms that support enterprise reporting and digital transformation.

Skill / Qualifications

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, Software Engineering, or a related technical field.
• Strong hands-on experience with Microsoft Fabric, including Workspaces, OneLake, Fabric Pipelines, Data Factory in Fabric, Lakehouse, and Warehouse.
• Experience with Azure Data Services including Azure Data Factory (ADF), Azure Synapse Analytics, Azure Databricks, ADLS Gen2, and Azure Blob Storage.
• Strong proficiency in PySpark, Python, SQL (T-SQL), and data transformation techniques.
• Experience building scalable ETL/ELT pipelines for structured and unstructured data.
• Hands-on experience integrating enterprise applications using REST APIs, SOAP APIs, and cloud-based data services.
• Strong understanding of data modeling, data warehousing, and enterprise data integration principles.
• Experience implementing data quality frameworks, validation processes, reconciliation, and monitoring solutions.
• Knowledge of cloud security concepts including Azure Active Directory, Managed Identities, RBAC, and secure data access.
• Experience with Azure DevOps, Git, CI/CD pipelines, and Infrastructure as Code tools such as Terraform.
• Experience using Azure Log Analytics and Spark UI for monitoring and performance optimization.
• Familiarity with Azure Purview or other enterprise data governance platforms.
• Exposure to AI productivity tools such as Microsoft Copilot and Claude.
• Experience working in Agile/Scrum environments.
• Strong analytical, problem-solving, debugging, and troubleshooting skills.
• Excellent communication, collaboration, and stakeholder management abilities.

Job Responsibilities

• Design, develop, and implement scalable data engineering solutions using Microsoft Fabric and Azure Data Services.
• Build, optimize, and maintain data ingestion and transformation pipelines using Microsoft Fabric Pipelines, Azure Data Factory, Azure Synapse Pipelines, PySpark, Python, and SQL.
• Design and manage OneLake storage architecture and implement efficient data storage and processing solutions.
• Develop API-driven and scheduled data ingestion workflows supporting multiple daily processing cycles.
• Integrate data from cloud platforms, on-premises databases, third-party applications, files, and REST/SOAP APIs.
• Implement robust business rules, validation logic, incremental processing, and batch processing strategies.
• Design curated data layers and consumption-ready data models for analytics and reporting.
• Develop and optimize SQL queries, stored procedures, views, and datasets to improve reporting performance.
• Perform performance tuning for Spark jobs, SQL workloads, data pipelines, and cloud resources to improve scalability and reduce operational costs.
• Implement automated data quality validation, reconciliation, monitoring, and audit logging frameworks.
• Ensure secure data access through Azure Active Directory, Managed Identities, Role-Based Access Control (RBAC), and secure connectivity mechanisms.
• Build CI/CD pipelines using Azure DevOps and Git to automate deployments across multiple environments.
• Monitor data pipelines using Spark UI and Azure Log Analytics, troubleshoot failures, and perform root cause analysis.
• Collaborate with cross-functional teams including solution architects, platform engineers, DevOps teams, business analysts, and data scientists.
• Participate in Agile/Scrum ceremonies, technical design discussions, code reviews, and continuous improvement initiatives.

Benefits

Competitive Market Rate (Depending on Experience)