Manager

EXL; Gurugram District; On-site; INR 2,400,000 - 4,200,000; Full time

Job summary

EXL is seeking a highly skilled Senior Data Engineer to join its growing data engineering team. You will build services that integrate with Lending Club's products and external vendors, using modern data technologies and AWS cloud stack.

Key requirements include 6+ years in data engineering, hands-on experience with Hadoop, Spark, DBT, Airflow/Dagster, Databricks, Snowflake, and strong Python, Git, and CI/CD tooling. Collaboration with distributed teams is essential.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
• 6+ years of experience in Data Engineering or related roles
• Self-driven and proactive mindset.
• Strong ownership and accountability and communication skills. Passion for building scalable and maintainable data systems.
• Ability to work independently while collaborating effectively with distributed teams and client.

Responsibilities

• In this role, you will build services that will integrate directly with Lending Club's products and with external vendors.
• You will work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc., modern data platforms such as Databricks and Snowflake and cloud technologies across AWS stack
• Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc.
• Implement processes and systems to monitor Data Quality, Observability, Governance and Lineage.
• Support operations to manage the production environment and help in resolving production issues with RCA
• Write unit/integration tests, adopt Test-driven development, contribute to engineering wiki, and document design/implementation etc.

Skills

Hadoop

Spark

Hive

DBT

Dagster

Airflow

Atlan

Trino

Databricks

Snowflake

AWS

Python

Git

JIRA

Jenkins

Shell scripting

Agile methodologies

Test-driven development

Data pipelines

Data quality

Observability

Governance

Lineage

Education

Bachelor's degree in Computer Science, Engineering, or related field

Equivalent practical experience

Tools

Airflow

Dagster

Git

JIRA

Jenkins

Databricks

Snowflake

DBT

Hadoop

Spark

Job description

Job Description:

Role Overview:

We are looking for a highly skilled and motivated Senior Data Engineer to join our growing data engineering team.

Key Responsibilities:

• In this role, you will build services that will integrate directly with Lending Club's products and with external vendors. you
• You will work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc., modern data platforms such as Databricks and Snowflake and cloud technologies across AWS stack
• Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc.
• Implement processes and systems to monitor Data Quality, Observability, Governance and Lineage.
• Support operations to manage the production environment and help in resolving production issues with RCA
• Write unit/integration tests, adopt Test-driven development, contribute to engineering wiki, and document design/implementation etc.

Skills required:

• Strong hands-on experience with distributed systems and big data technologies, including Hadoop, Spark, Hive, DBT, and Airflow/Dagster.
• Minimum 4 years of solid production-quality coding experience in Python focused on data pipeline development.
• Experience working with cloud platforms, preferably AWS.
• Experience with Databricks and/or Snowflake.
• Proficiency in Git, JIRA, Jenkins, and shell scripting.
• Good understanding of Agile methodologies, source control management, test automation, and Test-Driven Development.
• Experience working with cross-functional teams in fast-paced and dynamic environments.
• Strong analytical, collaborative problem-solving, and communication skills.
• Proven track record of building fast, scalable, reliable, and high-quality data pipelines.

Nice to Have:

Experience building data pipelines for Digital Marketing use cases.

Required Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
• 6+ years of experience in Data Engineering or related roles
• Self-driven and proactive mindset.
• Strong ownership and accountability and communication skills. Passion for building scalable and maintainable data systems.
• Ability to work independently while collaborating effectively with distributed teams and client.

Responsibilities

• In this role, you will build services that will integrate directly with Lending Club's products and with external vendors. you
• You will work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc., modern data platforms such as Databricks and Snowflake and cloud technologies across AWS stack
• Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc.
• Implement processes and systems to monitor Data Quality, Observability, Governance and Lineage.
• Support operations to manage the production environment and help in resolving production issues with RCA
• Write unit/integration tests, adopt Test-driven development, contribute to engineering wiki, and document design/implementation etc.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
• 6+ years of experience in Data Engineering or related roles
• Self-driven and proactive mindset.
• Strong ownership and accountability and communication skills. Passion for building scalable and maintainable data systems.
• Ability to work independently while collaborating effectively with distributed teams and client.

Requirements