Senior Data Engineer

Datavail Infotech; Mumbai; Hybrid; INR 2,500,000 - 4,000,000; Full time

Benefits offered by this job

Datavail Infotech Pvt. Ltd. perks

Job summary

Datavail Infotech Pvt. Ltd. in Mumbai is seeking an experienced Senior Data Engineer to design, build, and govern enterprise data platforms with a focus on reliability, scalability, and data quality.

You will work across Databricks and Microsoft Fabric to shape data pipelines, semantic models, and analytics solutions. You will collaborate with cross-functional teams to implement CI/CD, automation, and data governance best practices while ensuring secure and compliant data operations for diverse

Qualifications

• 5+ years of experience in Data Engineering, Analytics Engineering, or Data Platform Development.
• Hands-on with Databricks and Apache Spark (PySpark).
• Strong SQL skills and experience with Data Warehousing.
• Proficiency with Microsoft Fabric and its components (Data Factory, Lakehouse, Warehouse, Semantic Models).
• Experience designing and implementing Data Pipelines, ETL/ELT, and Lakehouse Architectures.
• Experience with Databricks Workflows, Jobs, Automation, and Orchestration.
• CI/CD pipelines using Azure DevOps, GitHub, or equivalent tools.
• Knowledge of data quality, reconciliation, metadata management, and governance practices.
• Performance tuning of Spark workloads.
• Familiarity with Power BI and its semantic models for analytics.

Responsibilities

• Design, develop, and maintain scalable data solutions using Databricks, Microsoft Fabric, SQL, Spark, and Delta Lake.
• Build and optimize data ingestion, transformation, and orchestration pipelines.
• Develop automation frameworks for data validation, testing, monitoring, deployment, and support.
• Lead data engineering activities across the full lifecycle from requirements to deployment.
• Create automated QA and reconciliation frameworks for analytics solutions.
• Ensure data quality, governance, and security across enterprise datasets.
• Collaborate with stakeholders to deliver enterprise-grade data platforms.
• Support CI/CD, DevOps, IaC, and deployment automation initiatives.

Skills

Databricks

Spark

SQL

Microsoft Fabric

Data Warehousing

Data Pipelines

ETL/ELT

Automation

CI/CD

Tools

Azure DevOps

GitHub

Terraform

Bicep

ARM Templates

Power BI

Job description

Datavail is a leading provider of data management, application development, analytics, and cloud services, with more than 1,000 professionals helping clients build and manage applications and data via a world-class tech-enabled delivery platform and software solutions across all leading technologies. For more than 17 years, Datavail has worked with thousands of companies spanning different industries and sizes, and is an AWS Advanced Tier Consulting Partner, a Microsoft Solutions Partner for Data & AI and Digital & App Innovation (Azure), an Oracle Partner, and a MySQL Partner.

Job Description

Job Description

We are seeking an experienced Senior Data Engineer with strong expertise in Databricks, Microsoft Fabric, Data Engineering, Automation, and Data Quality Engineering. The ideal candidate will play a key role in designing, building, automating, and governing enterprise data platforms while ensuring the reliability, accuracy, and security of analytics solutions.

The role will support strategic data modernization initiatives across multiple clients and industry domains, focusing on scalable data pipelines, data quality frameworks, automation, and analytics platform engineering.

Key Responsibilities

• Design, develop, and maintain scalable data solutions using Databricks, Microsoft Fabric, SQL, Spark, and Delta Lake.
• Build and optimize data ingestion, transformation, and orchestration pipelines for enterprise data platforms.
• Develop and implement automation frameworks for data validation, testing, monitoring, deployment, and operational support.
• Lead data engineering activities across the full lifecycle, including requirements, design, development, testing, deployment, and support.
• Build and maintain automated QA and reconciliation frameworks for analytical and reporting solutions.
• Validate data quality, completeness, consistency, and business rule compliance across enterprise datasets.
• Develop and maintain semantic models, lakehouses, warehouses, and data marts within Microsoft Fabric.
• Implement and validate security controls, including Row-Level Security (RLS) and data governance standards.
• Collaborate with business stakeholders, architects, analytics teams, and platform engineers to deliver enterprise-grade solutions.
• Support CI/CD, DevOps, Infrastructure-as-Code, and deployment automation initiatives.
• Drive performance optimization and cost efficiency across Databricks and Fabric environments.
• Establish engineering best practices, coding standards, reusable frameworks, and governance processes.

Roles and Responsibilities

Required Skills

• 5+ years of experience in Data Engineering, Analytics Engineering, or Data Platform Development.
• Databricks
• Apache Spark (PySpark)
• SQL
• Microsoft Fabric
• Data Warehousing
• Experience designing and implementing:
  • Data Pipelines
  • ETL/ELT Solutions
  • Lakehouse Architectures
  • Data Integration Frameworks
• Strong experience with Databricks Workflows, Jobs, Automation, and Orchestration.
• Experience developing automated testing and validation frameworks for data platforms.
• Hands-on experience implementing CI/CD pipelines using Azure DevOps, GitHub, or equivalent tools.
• Strong understanding of data quality, reconciliation, metadata management, and governance practices.
• Experience with performance tuning and optimization of Spark workloads.
• Experience working with Microsoft Fabric components, including:
  • Data Factory
  • Lakehouse
  • Warehouse
  • Semantic Models
  • Power BI

Preferred Skills

• Experience with Infrastructure as Code (Terraform, Bicep, ARM Templates).
• Experience with DataOps and MLOps processes.
• Knowledge of Power BI semantic models, DAX, and analytical reporting solutions.
• Experience with enterprise data governance and security frameworks.
• Familiarity with cloud platforms such as Azure, AWS, or GCP.
• Experience leading technical discussions, mentoring engineers, and driving engineering best practices.

Soft Skills

• Strong analytical and problem-solving skills.
• Excellent communication and stakeholder management abilities.
• Ability to lead initiatives independently and work effectively in cross-functional teams.
• Strategic mindset with the ability to align technical solutions with business objectives.

About the Team

Datavail’s Data Management and Analytics practice is made up of experts who provide a variety of data services including initial consulting and development, designing and building complete data systems, as well as ongoing support and management of database, data warehouse, data lake, data integration, and virtualization and reporting environments. Datavail’s team is comprised of not just excellent BI & analytics consultants, but great people as well. Datavail’s data intelligence consultants are experienced, knowledgeable and certified in the best in breed BI and analytics software applications and technologies. We ascertain your business objectives, goals and requirements, assess your environment, and recommend the tools which best fit your unique situation. Our proven methodology can help your project succeed, regardless of stage. With the combination of a proven delivery model and top-notch experience ensures that Datavail will remain the Data Management experts on demand you desire. Datavail’s flexible and client focused services always add value to your organization.

Job Info

• Job Identification 1945
• Job Category Technical
• Posting Date 09/10/2026, 04:17 PM
• Locations A-902, Supreme Business Park, Mumbai, MH, 400076, IN Ground floor, South Wing, Maruthi Chambers, Bengaluru, KA, 560068, IN 1002 - 1003, Manjeera Trinity Corporate, JNTU-Hitech City Road, Hyderabad, TG, 500072, IN (Hybrid)
• Job Schedule Full time
• Worker Type Employee
• Number of Openings 1
• Legal Employer Datavail Infotech Pvt. Ltd.