Data Engineer

Recro; Bengaluru; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Recro in Bengaluru is seeking a Data Engineer to design and manage scalable data pipelines powering AI-driven SaaS applications for the CPG sector. This role focuses on building reliable data infrastructure to enable insights into consumer behavior and supply chain performance.

The ideal candidate has 3+ years in data engineering, strong Python, PySpark, SQL skills, and hands-on Spark in production. You will develop ETL workflows, orchestrate with Airflow/Kubeflow, and collaborate with AI/ML,

Qualifications

• 3+ years of experience in Data Engineering.
• Strong coding in Python, PySpark, and SQL.
• Hands-on experience working with Apache Spark in production environments.
• Experience building scalable ETL pipelines and data workflows.
• Familiarity with Airflow / Kubeflow orchestration tools.
• Experience with AWS cloud services for data engineering workloads.
• Strong understanding of data modeling, big data infrastructure, and performance optimization.
• Experience with GitHub, Docker, and modern development tools.

Responsibilities

• Build, maintain, and optimize scalable data pipelines (ETL/ELT) for AI and Predictions-as-a-Service platforms.
• Develop and orchestrate pipelines using Airflow / Kubeflow.
• Write efficient Python, PySpark, and SQL code for large-scale data processing.
• Own end-to-end pipeline lifecycle including development, monitoring, debugging, and optimization.
• Design and document data architectures, HLDs/LLDs, and technical solutions.
• Collaborate with AI/ML, MLOps, DevOps, and Product teams to implement data-driven solutions.
• Optimize data ingestion, transformation, and pipeline performance.

Skills

Data Engineering

Python

PySpark

SQL

Apache Spark

ETL pipelines

Airflow

Kubeflow

AWS

data modeling

GitHub

Docker

modern development tools

Tools

Apache Spark

Airflow

Kubeflow

AWS

GitHub

Docker

Job description

Non-negotiable: Python; Pyspark, Experienced in processing raw data and transforming it into actionable insights, working with data from multiple sources and handling large volumes of data, with the ability to build and deploy data pipelines.

Role Overview

We are looking for a Data Engineer to design and manage scalable data pipelines powering AI-driven SaaS applications for the Consumer Packaged Goods (CPG) sector. This role focuses on building reliable data infrastructure to enable insights into consumer behavior and supply chain performance.

Required Skills

• 3+ years of experience in Data Engineering.
• Strong coding in Python, PySpark, and SQL.
• Hands-on experience working with Apache Spark in production environments.
• Experience building scalable ETL pipelines and data workflows.
• Familiarity with Airflow / Kubeflow orchestration tools.
• Experience with AWS cloud services for data engineering workloads.
• Strong understanding of data modeling, big data infrastructure, and performance optimization.
• Experience with GitHub, Docker, and modern development tools.

Key Responsibilities

• Build, maintain, and optimize scalable data pipelines (ETL/ELT) for AI and Predictions-as-a-Service platforms.
• Develop and orchestrate pipelines using Airflow / Kubeflow.
• Write efficient Python, PySpark, and SQL code for large-scale data processing.
• Own end-to-end pipeline lifecycle including development, monitoring, debugging, and optimization.
• Design and document data architectures, HLDs/LLDs, and technical solutions.
• Collaborate with AI/ML, MLOps, DevOps, and Product teams to implement data-driven solutions.
• Optimize data ingestion, transformation, and pipeline performance.

Nice to Have

• Experience supporting AI/ML data pipelines.
• Exposure to large-scale data platforms and real-time monitoring systems.