Data Engineer

ScaleupAlly; Uttar Pradesh; On-site; INR 1,500,000 - 2,300,000; Full time

Job summary

ScaleupAlly is seeking a Senior Data Engineer to own the design and delivery of production data platforms from requirements to production support. You will collaborate with clients and our engineering team, leveraging Python, PySpark, and Azure Databricks or Fabric to build robust data pipelines.

The role requires strong SQL, data modeling (dimensional, Medallion/Lakehouse), and experience with Azure Data Factory.

Qualifications

• Bachelor's degree in any discipline.
• Information Systems, or a related field.
• 5-8 years in data engineering.

Responsibilities

• Own end-to-end data pipelines design, build, and production support.
• Design models (dimensional, Medallion/Lakehouse) that reflect business use.
• Build scalable ETL/ELT pipelines using Python, SQL, PySpark and Azure Data Factory.
• Implement batch, incremental/CDC and streaming loads and optimize performance and cost.
• Establish data quality checks, monitoring, governance, and lineage.

Skills

SQL

Python

PySpark

Dimensional modeling

Lakehouse

Data quality

Git

Education

Bachelors degree

Information Systems or related field

Tools

Azure Databricks

Microsoft Fabric

Azure Data Factory

Delta Lake

Power BI

CI/CD tooling

Job description

Job Information

• Date Opened 26/10/2026
• Job Type Full time
• Industry IT Services
• Work Experience 6+ - 15 years
• Salary As per company norms
• City Noida
• Province Uttar Pradesh
• Country India
• Postal Code 201301

About Us

At ScaleupAlly, we believe that technology improves all aspects of lives. That is why we are geared towards becoming a tech ally of great ideas. The aim is to be the remote in-house tech team of these ideas. We work with Start-ups, Founders, and Leaders around the globe with ideas of Web and App Development, Business Intelligence and Data Visualization. We achieve our aim by building the long-lasting dynamic teams made of the top talent across the globe managed by experienced in-house Technical Leads. At ScaleupAlly, we build, manage and care our distributed teams like no one else. It’s a whole new approach to make you rethink what your idea is capable of.

Purpose of Job

We are looking for a Senior Data Engineer who is equally strong on the data side (modelling, data quality, SQL, understanding the business) and the engineering side (Python, PySpark, CI/CD, performance, reliability). You will own the design and delivery of production data platforms from requirements to production support, work directly with clients, and share your knowledge with the wider engineering team.

Your stack will include Python, SQL, PySpark, Azure Databricks or Microsoft Fabric, Azure Data Factory, Delta Lake, and Lakehouse and Data Warehouse designs. Essential Qualifications

• Bachelor's degree in any discipline
• Information Systems, or a related field.
• Holds at least 5-8 years in data engineering

Skills ADF l Microsoft Fabric l SQL l ETL/ELT l Power BI | Python

Duties

• Own the design and delivery of data pipelines and platforms end to end: requirements, architecture, build, testing, deployment and production support.
• Design data models (dimensional, SCD, Medallion Bronze/Silver/Gold) that match how the business actually uses the data.
• Build and maintain scalable ETL/ELT pipelines using Python, SQL, PySpark and Azure Data Factory on Azure Databricks or Microsoft Fabric.
• Implement batch, incremental/CDC and streaming loads, and tune Spark and SQL performance and cost.
• Build data quality checks, monitoring and alerting into every pipeline, and apply governance, lineage and access controls.
• Automate testing and deployment with Git and CI/CD, and follow engineering best practices in code reviews.
• Work directly with client stakeholders to gather requirements, explain design choices and agree acceptance criteria.
• Integrate data from APIs, transactional systems and external platforms.
• Investigate and resolve complex production issues and document root causes.
• Document your solutions and share knowledge with the team, including peer code reviews.

Required Qualifications

• Data side: strong SQL, dimensional and Medallion/Lakehouse modelling, data quality practices, and the ability to understand business requirements.
• Engineering side: strong Python and PySpark, Git and CI/CD, automated testing, and performance tuning of high-volume workloads.
• Hands-on experience with Azure Databricks or Microsoft Fabric, and Azure Data Factory or an equivalent orchestration tool.
• Proven delivery of batch and incremental/CDC pipelines on Delta Lake or similar.
• Client-facing experience: requirement gathering, stakeholder communication and clear technical documentation.
• Measurable impact you can explain, such as runtime, cost or data quality improvements.

Good to Have

• Streaming experience (Spark Structured Streaming, Kafka).
• Governance and lineage tools (Unity Catalog, Azure Purview).
• Infrastructure as code (Terraform).
• Power BI and semantic-layer knowledge, to work effectively with BI teams.
• Microsoft certifications such as DP-203 or DP-600.