Lead Data Engineer

Recro; Bengaluru; Presencial; INR 2.500.000 - 4.200.000; Jornada completa

Descripción de la vacante

Recro in Bengaluru, India is seeking a Senior / Lead Data Engineer to own end-to-end data pipelines feeding analytics and ML models. You will ensure massive data volumes arrive reliably, scale gracefully, and stay trustworthy across the enterprise.

You will architect and maintain scalable batch and real-time pipelines with Spark, Kafka, Python/Scala; manage Snowflake/Databricks data warehouses, and implement orchestration with Airflow or Prefect.

Formación

• 4–10 years of software engineering and data pipeline experience.
• Strong expertise in data engineering and large-scale data processing.
• Experience with batch and real-time pipelines is required.

Responsabilidades

• Own end-to-end data pipelines for analytics and ML models.
• Build scalable batch and real-time pipelines using Spark, Kafka, Python/Scala.
• Design cloud data warehouses/lakehouses (Snowflake, Databricks).
• Implement pipeline orchestration with Airflow or Prefect.

Conocimientos

Python

Scala

SQL

Spark

Kafka

Airflow

Snowflake

Databricks

Kinesis

Docker

Herramientas

Docker

Git

CI/CD

Descripción del empleo

. Job Role : Senior / Lead Data Engineer

As a Senior Data Engineer, you will own the end-to-end data pipelines that feed our analytics and production machine learning models. You are responsible for ensuring that massive volumes of raw data arrive reliably, scale gracefully, and can be trusted across the entire enterprise.

Key Responsibilities

• Architect, implement, and maintain scalable batch and real-time streaming data pipelines using Apache Spark, Kafka, and Python/Scala.
• Design and manage cloud data warehouse and lakehouse architectures (Snowflake, Databricks, BigQuery).
• Implement robust pipeline orchestration using Apache Airflow or Prefect.
• Enforce data governance, quality frameworks, schema evolution, and automated unit/integration testing on data assets.
• Partner with Data Scientists and ML Engineers to supply reliable, low-latency features and ingestion workflows.

• Relevant Experience: 4–10 years of dedicated software engineering and data pipeline development experience.
• Core Technical Stack: Advanced Python and/or Scala, expert SQL, Spark (PySpark), Kafka/Kinesis, Airflow, and Snowflake/Databricks.
• Proven Track Record: Tangible history of processing multi-terabyte/petabyte workloads and handling millions of events/day in production.
• Engineering Standards: Deep understanding of containerization (Docker), version control (Git), and CI/CD for data pipelines.