Lead Assistant Manager

Ex; Pune District; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Ex is seeking a Senior Data Engineer to join our growing data engineering team in Pune. You will build services that integrate with client products and external vendors, leveraging Hadoop, Spark, and modern data platforms on AWS. You will automate data delivery, monitor data quality and governance, and support production operations with RCA-driven issue resolution.

You will write tests, adopt TDD, and contribute to engineering wikis and design docs, collaborating across distributed teams.

Qualifications

• Bachelor’s degree or equivalent practical experience in CS/Engineering or related field.
• 7+ years of experience in Data Engineering or related roles.
• Self-driven and proactive mindset.
• Strong ownership and accountability and communication skills.
• Passion for building scalable and maintainable data systems.
• Ability to work independently while collaborating effectively with distributed teams and client.

Responsibilities

• In this role, you will build services that will integrate directly with client’s products and with external vendors.
• You will work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc., modern data platforms such as Databricks and Snowflake and cloud technologies across AWS stack.
• Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc.
• Implement processes and systems to monitor Data Quality, Observability, Governance and Lineage.
• Support operations to manage the production environment and help in resolving production issues with RCA
• Write unit/integration tests, adopt Test-driven development, contribute to engineering wiki, and document design/implementation etc.

Skills

Hadoop

Spark

Hive

DBT

Airflow

Dagster

AWS

Databricks

Snowflake

Git

Jira

Jenkins

Shell scripting

Python

Agile

TDD

Cross-functional teamwork

Data quality

Observability

Education

Bachelor’s degree in Computer Science, Engineering, or a related field

7+ years of experience in Data Engineering or related roles

Tools

Git

JIRA

Jenkins

Job description

Role Overview:

We are looking for a highly skilled and motivated Senior Data Engineer to join our growing data engineering team.

Key Responsibilities:

• In this role, you will build services that will integrate directly with client’s products and with external vendors. you
• You will work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc., modern data platforms such as Databricks and Snowflake and cloud technologies across AWS stack
• Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc.
• Implement processes and systems to monitor Data Quality, Observability, Governance and Lineage.
• Support operations to manage the production environment and help in resolving production issues with RCA
• Write unit/integration tests, adopt Test-driven development, contribute to engineering wiki, and document design/implementation etc.

Skills required:

• Strong hands-on experience with distributed systems and big data technologies, including Hadoop, Spark, Hive, DBT, and Airflow/Dagster.
• Minimum 7 years of solid production-quality coding experience in Python focused on data pipeline development.
• Experience working with cloud platforms, preferably AWS.
• Experience with Databricks and/or Snowflake.
• Proficiency in Git, JIRA, Jenkins, and shell scripting.
• Good understanding of Agile methodologies, source control management, test automation, and Test-Driven Development.
• Experience working with cross-functional teams in fast-paced and dynamic environments.
• Strong analytical, collaborative problem-solving, and communication skills.
• Proven track record of building fast, scalable, reliable, and high-quality data pipelines.

Nice to Have :

Experience building data pipelines for Digital Marketing use cases.

Required Qualifications :

• Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
• 7+ years of experience in Data Engineering or related roles
• Self-driven and proactive mindset.
• Strong ownership and accountability and communication skills.
• Passion for building scalable and maintainable data systems.
• Ability to work independently while collaborating effectively with distributed teams and client.