Data Engineer

Weekday AI (YC W21); India; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

A fast-growing data solutions company in India is seeking an experienced Data Engineer to build scalable data pipelines and manage distributed data systems. The ideal candidate will have at least 4 years of experience, particularly with Databricks and other modern data engineering practices. Key responsibilities include designing ETL/ELT pipelines, collaborating with teams, and ensuring data accuracy across workflows. This role requires strong skills in SQL, Python, and familiarity with cloud-based platforms.

Qualifications

• 4+ years of experience in data engineering and distributed data processing.
• Strong hands-on experience with Databricks or equivalent technologies.
• Solid understanding of distributed storage systems and formats.

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines for large-scale datasets.
• Collaborate with cross‑functional teams to translate requirements into technical solutions.
• Monitor and optimize data workflows for performance, scalability, and reliability.

Skills

Databricks

Python

SQL

Spark

Data governance

Education

Bachelor's degree in IT, Computer Science, or related field

Tools

Informatica

Talend

SSIS

Snowflake

AWS

Azure

Airflow

Job description

This role is for one of the Weekday's clients.

Min Experience: 4 years
Location: India
JobType: full-time

We are seeking an experienced Data Engineer with strong expertise in Databricks and modern data engineering practices. The ideal candidate will have 4+ years of hands‑on experience in developing scalable data pipelines, managing distributed data systems, and supporting end-to-end CI/CD processes. This role involves architecting and optimizing data workflows that enable seamless data‑driven decision‑making across the organization.

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines for large-scale datasets using Spark, Hive, or Glue.
• Develop and optimize data integration workflows using ETL tools such as Informatica, Talend, or SSIS.
• Write, optimize, and maintain complex SQL queries for data transformation and analytics.
• Collaborate with cross‑functional teams including data scientists, analysts, and product stakeholders to translate requirements into technical solutions.
• Deploy data workflows using CI/CD pipelines and ensure smooth automated releases.
• Monitor and optimize data workflows for performance, scalability, and reliability.
• Ensure data accuracy, governance, security, and compliance across pipelines.
• Work with cloud-based data platforms such as Azure (ADF, Synapse, Databricks) or AWS (EMR, Glue, S3, Athena).
• Maintain clear documentation of data systems, architectures, and processes.
• Provide mentorship and technical guidance to junior team members.
• Stay current with emerging data engineering tools, technologies, and best practices.

Requirements

• Bachelor's degree in IT, Computer Science, or related field.
• 4+ years of experience in data engineering and distributed data processing.
• Strong hands-on experience with Databricks or equivalent technologies (Spark, EMR, Hadoop).
• Proficiency in Python or Scala.
• Experience with modern data warehouses (Snowflake, Redshift, Oracle).
• Solid understanding of distributed storage systems (HDFS, ADLS, S3) and formats such as Parquet and ORC.
• Familiarity with orchestration tools such as ADF, Airflow, or Step Functions.
• Databricks Data Engineering Professional certification (preferred / required as needed).
• Experience in multi-cloud or migration-based projects is a plus.