Data Engineer

Calfus Inc.; Pune District; On-site; INR 3,000,000 - 5,200,000; Full time

Benefits offered by this job

Medical insurance

Parental insurance

Birthday leave

Gratuity

Provident fund

Job summary

Calfus Inc. is seeking a Senior / Lead Data Engineer to own the design and delivery of cloud data platforms on AWS and Azure. You will architect lakehouse solutions, set data modelling standards, and lead a team of data engineers delivering reliable, scalable pipelines.

This is a hands‑on leadership role with SQL and PySpark coding. With 8–12 years of experience, you will craft architecture, guide stakeholders, and ensure high-quality data for analytics and business insights.

Qualifications

• Bachelor's degree required in CS/IS/Data Science or related field.
• 6–12 years in data engineering with leadership experience.
• Strong AWS or Azure experience demonstrated through delivered projects.
• Medallion Architecture experience on Databricks or Delta Lake.
• Proficiency in advanced SQL and data modelling.
• ETL/ELT orchestration with SSIS, Azure Data Factory, or Airflow.
• Python for data engineering (Pandas, PySpark) and automation.
• Experience with Snowflake, Postgres, Redshift, and MongoDB.
• Familiarity with JSON, CSV, Parquet, Pickle formats.
• Git, CI/CD, and data quality practices.

Responsibilities

• Design scalable data lake, lakehouse, and warehouse architectures on AWS and Azure.
• Implement Medallion Architecture (bronze, silver, gold) on Databricks/Delta Lake.
• Lead data engineers to gather requirements and translate into technical specs and architecture designs.
• Build and oversee ETL/ELT pipelines using Azure Data Factory, Airflow, SSIS, or equivalents.
• Use Azure Databricks and PySpark for large-scale data processing.
• Own pipeline reliability — orchestration, monitoring, failure handling, and automation.
• Design and maintain dimensional and relational data models (star, snowflake) for analytics.
• Own schema design decisions and modelling standards across the platform.
• Write complex SQL queries and transformations; optimise pipelines and Spark jobs.
• Mentor the team and foster continuous learning and improvement.

Skills

Team leadership

Data architecture

SQL proficiency

Python programming

ETL/ELT design

Cloud data platforms

Education

Bachelor's degree in Computer Science / Information Systems / Data Science

Tools

Databricks

Delta Lake

Azure Data Factory

Azure Databricks

Airflow

SSIS

S3

Snowflake

Redshift

Postgres

MongoDB

Job description

At Calfus, we are known for delivering cutting-edge AI agents and products that transform businesses in ways previously unimaginable. We empower companies to harness the full potential of AI, unlocking opportunities they never imagined possible before the AI era. Our software engineering teams are highly valued by customers, whether start-ups or established enterprises, because we consistently deliver solutions that drive revenue growth. Our ERP solution teams have successfully implemented cloud solutions and developed tools that seamlessly integrate with ERP systems, reducing manual work so teams can focus on high-impact tasks.

None of this would be possible without talent like you! Our global teams thrive on collaboration, and we’re actively looking for skilled professionals to strengthen our in-house expertise and help us deliver exceptional AI, software engineering, and solutions using enterprise applications.

As one of the fastest-growing companies in our industry, we take pride in fostering a culture of innovation where new ideas are always welcomed—without hesitation. We are driven and expect the same dedication from our team members. Our speed, agility, and dedication set us apart, and we perform best when surrounded by high-energy, driven individuals.

To continue our rapid growth and deliver an even greater impact, we invite you to apply for our open positions and become part of our journey!

About the Role

As a Senior / Lead Data Engineer, you'll own the design and delivery of our cloud data platforms on AWS and Azure. You'll architect Lakehouse solutions built on Medallion Architecture, set the data modelling standards, and lead a team of data engineers delivering reliable, scalable pipelines. This is a hands‑on leadership role: you'll make the architecture decisions and still be close enough to the code to write SQL and PySpark yourself. With 8–12 years of experience, you've designed data platforms, not just built pipelines on them.

What You’ll Do

• Design scalable data lake, lakehouse, and warehouse architectures on AWS and Azure that meet business requirements
• Implement Medallion Architecture (bronze, silver, gold) using Databricks and Delta Lake
• Work with stakeholders to gather requirements and translate them into technical specifications and architecture designs
• Build and oversee ETL/ELT pipelines using Azure Data Factory, Airflow, SSIS, or equivalent tools
• Use Azure Databricks and PySpark for large-scale data processing
• Own pipeline reliability — orchestration, monitoring, failure handling, and automation
• Design and maintain dimensional and relational data models (star, snowflake) for analytical workloads
• Own schema design decisions and modelling standards across the platform
• Write complex SQL queries, stored procedures, and transformations
• Optimise pipelines, queries, and Spark jobs for performance, scalability, reliability, and cost

Data Quality & Governance

• Implement data quality checks, validation, and governance best practices to ensure accurate and compliant data
• Lead, mentor, and guide data engineers, review code, and build a culture of continuous learning and improvement

What You’ll Need — Must Have

• 6-12 years in data engineering, including experience leading data engineering teams
• Strong AWS or Azure experience (S3, Lambda, Glue / ADF, ADLS, Databricks) demonstrated through delivered projects
• Medallion Architecture — you've designed and implemented bronze/silver/gold layers on Databricks or Delta Lake
• Data Architecture — you've designed end-to-end data platforms, not only built pipelines on them
• Data Modelling — you've built dimensional and relational models, not only queried them
• Advanced SQL — SQL Server, Postgres, Snowflake, or Redshift, including stored procedures and query tuning
• ETL/ELT and orchestration tools such as SSIS, Azure Data Factory, or Airflow
• Python for data engineering — Pandas, NumPy, PySpark, scripting, and automation
• Experience with databases like Snowflake, Postgres, Redshift, and MongoDB
• Familiarity with data formats like JSON, CSV, Parquet, and Pickle
• Version control (Git), CI/CD, and code quality practices for data engineering projects
• Bachelor's degree in Computer Science, Information Systems, Data Science, or a related field

Good to Have

• Hands-on experience across both AWS and Azure
• Working with REST APIs and web scraping
• Unity Catalog or other data governance and cataloguing tools

At Calfus, we value our employees and offer a strong benefits package. This includes medical, group, and parental insurance, coupled with gratuity and provident fund options. Further, we support employee wellness and provide birthday leave as a valued benefit.

Calfus is an Equal Opportunity Employer.

We believe diversity drives innovation. We’re committed to creating an inclusive workplace where everyone—regardless of background, identity, or experience—has the opportunity to thrive. We welcome all applicants!