Data Engineer

ScaleupAlly; Dadri; Remote; INR 1.200.000 - 2.400.000; Vollzeit

Benefits dieser Stelle

Medical Insurance coverage

Quarterly parties

Yearly team outings

Regular leadership check-ins

Recognition awards

Zusammenfassung

ScaleupAlly is seeking a Senior Data Engineer to own design and delivery of production data platforms. You will work with clients directly and share knowledge with the engineering team.

Your stack includes Python, SQL, PySpark, Azure Databricks or Microsoft Fabric, Azure Data Factory, Delta Lake, and data warehouse designs.

Qualifikationen

• Bachelor's degree in Information Systems or a related field.
• Strong SQL with data modeling and data quality practices.
• Experience delivering end-to-end data pipelines and platforms.

Aufgaben

• Own the design and delivery of data pipelines end to end: requirements, architecture, build, testing, deployment and production support.
• Design data models (dimensional, SCD, Medallion Bronze/Silver/Gold) aligned with business data usage.
• Build scalable ETL/ELT pipelines using Python, SQL, PySpark on Azure Databricks or Fabric.
• Implement batch, incremental/CDC and streaming loads; tune Spark and SQL performance and cost.
• Establish data quality checks, monitoring, governance, lineage and access controls.
• Automate testing and deployment with Git and CI/CD; conduct code reviews.

Kenntnisse

Python

SQL

ETL/ELT

Power BI

Azure Databricks

Ausbildung

Bachelor's degree in Information Systems or related field

Tools

Azure Data Factory

Delta Lake

Microsoft Fabric

CI/CD

Jobbeschreibung

At ScaleupAlly, we believe that technology improves all aspects of lives. That is why we are geared towards becoming a tech ally of great ideas. The aim is to be the remote in-house tech team of these ideas. We work with Start-ups, Founders, and Leaders around the globe with ideas of Web and App Development, Business Intelligence and Data Visualization. We achieve our aim by building the long-lasting dynamic teams made of the top talent across the globe managed by experienced in-house Technical Leads. At ScaleupAlly, we build, manage and care our distributed teams like no one else. It’s a whole new approach to make you rethink what your idea is capable of.

Job Description

Purpose of Job

We are looking for a Senior Data Engineer who is equally strong on the data side (modelling, data quality, SQL, understanding the business) and the engineering side (Python, PySpark, CI/CD, performance, reliability). You will own the design and delivery of production data platforms from requirements to production support, work directly with clients, and share your knowledge with the wider engineering team.

Your stack will include Python, SQL, PySpark, Azure Databricks or Microsoft Fabric, Azure Data Factory, Delta Lake, and Lakehouse and Data Warehouse designs. Essential Qualifications

Bachelor's degree in any discipline

Information Systems, or a related field.

Holds at least 5-8 years in data engineering

Skills ADF l Microsoft Fabric l SQL l ETL/ELT l Power BI | Python

Duties

Own the design and delivery of data pipelines and platforms end to end: requirements, architecture, build, testing, deployment and production support.

Design data models (dimensional, SCD, Medallion Bronze/Silver/Gold) that match how the business actually uses the data.

Build and maintain scalable ETL/ELT pipelines using Python, SQL, PySpark and Azure Data Factory on Azure Databricks or Microsoft Fabric.

Implement batch, incremental/CDC and streaming loads, and tune Spark and SQL performance and cost.

Build data quality checks, monitoring and alerting into every pipeline, and apply governance, lineage and access controls.

Automate testing and deployment with Git and CI/CD, and follow engineering best practices in code reviews.

Work directly with client stakeholders to gather requirements, explain design choices and agree acceptance criteria.

Integrate data from APIs, transactional systems and external platforms.

Investigate and resolve complex production issues and document root causes.

Document your solutions and share knowledge with the team, including peer code reviews.

Requirements

Required Qualifications

Data side: strong SQL, dimensional and Medallion/Lakehouse modelling, data quality practices, and the ability to understand business requirements.

Engineering side: strong Python and PySpark, Git and CI/CD, automated testing, and performance tuning of high-volume workloads.

Hands-on experience with Azure Databricks or Microsoft Fabric, and Azure Data Factory or an equivalent orchestration tool.

Proven delivery of batch and incremental/CDC pipelines on Delta Lake or similar.

Client-facing experience: requirement gathering, stakeholder communication and clear technical documentation.

Measurable impact you can explain, such as runtime, cost or data quality improvements.

Good to Have

Streaming experience (Spark Structured Streaming, Kafka).

Governance and lineage tools (Unity Catalog, Azure Purview).

Infrastructure as code (Terraform).

Power BI and semantic-layer knowledge, to work effectively with BI teams.

Microsoft certifications such as DP-203 or DP-600.

Benefits and Additional Information

Working hours: 10:00 AM – 7:00 PM

Working days: 5 days a week (plus 1st & 3rd Saturdays working)

Medical Insurance coverage for employees

Quarterly parties and yearly outings/trips for team bonding

Regular check-ins with leadership for growth and feedback

Recognition awards to celebrate high performance

Fun activities and team engagement sessions throughout the year