Data Engineer

Recro; Bengaluru; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

A technology company is seeking a Data Engineer in Bengaluru, India. The role involves designing and managing scalable data pipelines for AI-driven SaaS applications in the Consumer Packaged Goods sector. Candidates should have over 3 years of experience, strong skills in Python, PySpark, and SQL, and hands-on experience with AWS services. The job also requires developing data workflows, collaborating with various teams, and optimizing pipeline performance. Comprehensive knowledge of data modeling and ETL processes is essential.

Qualifications

• 3+ years of experience in Data Engineering.
• Strong coding in Python, PySpark, and SQL.
• Hands-on experience with Apache Spark in production environments.
• Experience building scalable ETL pipelines.
• Familiarity with Airflow/Kubeflow orchestration tools.
• Experience with AWS cloud services.

Responsibilities

• Build and optimize scalable data pipelines for AI platforms.
• Develop and orchestrate data pipelines using Airflow/Kubeflow.
• Write Python, PySpark, and SQL for data processing.
• Own the end-to-end pipeline lifecycle including monitoring.
• Design and document data architectures and technical solutions.
• Collaborate with AI/ML, MLOps, DevOps, and Product teams.
• Optimize data ingestion and pipeline performance.

Skills

Data Engineering

Python

PySpark

SQL

Apache Spark

ETL pipelines

Airflow/Kubeflow

AWS cloud services

GitHub

Docker

Job description

Role Overview

We are looking for a Data Engineer to design and manage scalable data pipelines powering AI-driven SaaS applications for the Consumer Packaged Goods (CPG) sector. This role focuses on building reliable data infrastructure to enable insights into consumer behavior and supply chain performance.

Required Skills

• 3+ years of experience in Data Engineering.
• Strong coding in Python, PySpark, and SQL.
• Hands‑on experience working with Apache Spark in production environments.
• Experience building scalable ETL pipelines and data workflows.
• Familiarity with Airflow / Kubeflow orchestration tools.
• Experience with AWS cloud services for data engineering workloads.
• Strong understanding of data modeling, big data infrastructure, and performance optimization.
• Experience with GitHub, Docker, and modern development tools.

Key Responsibilities

• Build, maintain, and optimize scalable data pipelines (ETL/ELT) for AI and Predictions-as-a-Service platforms.
• Develop and orchestrate pipelines using Airflow / Kubeflow.
• Write efficient Python, PySpark, and SQL code for large‑scale data processing.
• Own end‑to‑end pipeline lifecycle including development, monitoring, debugging, and optimization.
• Design and document data architectures, HLDs/LLDs, and technical solutions.
• Collaborate with AI/ML, MLOps, DevOps, and Product teams to implement data‑driven solutions.
• Optimize data ingestion, transformation, and pipeline performance.

Nice to Have

• Experience supporting AI/ML data pipelines.
• Exposure to large-scale data platforms and real‑time monitoring systems.