Data Team Lead

Recro; Chennai; On-site; INR 1,400,000 - 2,100,000; Full time

Job summary

Recro seeks a Lead Data Engineer to architect and deliver high-throughput data pipelines for BFSI applications. You will lead a team of data engineers, shape data architecture, and ensure robust ETL/ELT delivery aligned with financial services release cycles.

The role demands 8+ years of data engineering experience, expertise in Databricks, Delta Lake, Unity Catalog, PySpark, and SQL, plus a strong focus on data quality and governance for core banking workloads.

Qualifications

• Minimum 8+ years of hands-on data engineering experience, including at least 2 years in a technical leadership or module lead capacity.
• Deep expertise in Databricks Lakehouse architecture, Delta Lake features (Time Travel, ACID transactions, vacuuming, compaction), and Unity Catalog governance.
• Advanced proficiency with Apache Spark / PySpark (dataframes, RDDs, partitioning strategies, memory management, shuffle tuning).
• Strong software engineering fundamentals in Python (OOP, error handling, unit testing) and expert-level SQL (window functions, query tuning, complex aggregations).
• Proven track record designing enterprise-scale ETL pipelines integrating diverse legacy systems, core banking databases, and modern data warehouses.
• Solid familiarity with banking data models (e.g., core transactions, cards, loans, AML, or risk reporting).

Responsibilities

• Architect, build, and optimize high-throughput, fault-tolerant batch and streaming data pipelines capable of handling mission-critical financial transactions and accounting workflows.
• Lead a pod of data engineers, conduct rigorous code reviews, enforce design best practices, and ensure timely delivery aligned with banking release cycles.
• Implement automated data validation, integrity checks, and ledger reconciliation frameworks to adhere to strict banking accuracy standards.
• Build reusable, modular ETL/ELT frameworks; tune distributed cluster compute, query execution plans, and storage layout to meet aggressive SLAs.
• Work closely with delivery heads, enterprise architects, product owners, and risk/compliance officers to translate banking business requirements into scalable technical implementations.

Skills

Databricks Lakehouse

Delta Lake

Unity Catalog

PySpark

Python

SQL

ETL / ELT pipelines

Distributed Computing

Data quality & reconciliation

Banking domain knowledge

Tools

Databricks Unity Catalog governance

Job description

Job Description: Lead Data Engineer (Banking & Financial Services)

• Position: Lead Data Engineer
• Total Experience: 8+ years
• Domain: Banking, Financial Services, and Insurance (BFSI)
• Employment Type: Full-time

Key Responsibilities

• Pipeline Architecture & Strategy: Architect, build, and optimize high-throughput, fault-tolerant batch and streaming data pipelines capable of handling mission-critical financial transactions and accounting workflows.
• Technical Leadership & Delivery: Lead a pod of data engineers, conduct rigorous code reviews, enforce design best practices, and ensure timely delivery aligned with banking release cycles.
• Data Quality & Reconciliation: Implement automated data validation, integrity checks, and ledger reconciliation frameworks to adhere to strict banking accuracy standards.
• ETL Frameworks & Optimization: Build reusable, modular ETL/ELT frameworks; tune distributed cluster compute, query execution plans, and storage layout to meet aggressive SLAs.
• Stakeholder Management: Work closely with delivery heads, enterprise architects, product owners, and risk/compliance officers to translate banking business requirements into scalable technical implementations.

Mandatory Skills & Qualifications

• Experience: Minimum 8+ years of hands-on data engineering experience, including at least 2 years in a technical leadership or module lead capacity.
• Databricks: Deep expertise in Databricks Lakehouse architecture, Delta Lake features (Time Travel, ACID transactions, vacuuming, compaction), and Unity Catalog governance.
• PySpark & Distributed Computing: Advanced proficiency with Apache Spark / PySpark (dataframes, RDDs, partitioning strategies, memory management, shuffle tuning).
• Python & SQL: Strong software engineering fundamentals in Python (OOP, error handling, unit testing) and expert-level SQL (window functions, query tuning, complex aggregations).
• ETL & Data Integration: Proven track record designing enterprise-scale ETL pipelines integrating diverse legacy systems, core banking databases, and modern data warehouses.
• BFSI Domain Knowledge: Solid familiarity with banking data models (e.g., core transactions, cards, loans, AML, or risk reporting).