SDE - 2 - Data Engineering

Pluang; Gurugram District; On-site; INR 4,200,000 - 6,400,000; Full time

Job summary

Pluang is seeking a passionate Sr. Software Engineer (SDE-2) - Data Engineer to develop scalable data pipelines and analytics solutions.

You will solve complex data challenges, optimize systems for performance, and collaborate with teams to drive data-driven innovations fueling growth. You will design and implement data platforms using Spark, Kafka, BigQuery, and cloud functions, build ETL pipelines, and help enable analysts to derive timely business insights.

Qualifications

• Bachelor's degree in Computer Science with expertise in Spark/Kafka and cloud data platforms.
• Proficient in Python/Scala and ETL tooling such as Airflow.
• Experience with real-time data processing and data warehousing.
• Fintech experience a plus.

Responsibilities

• Contribute to design and architecture of Pluang's Data Platform, leveraging Apache Spark, HDFS, Kafka Streams, and ETL pipelines to sync data to the Data Lake and Data Warehouse.
• Optimize data mart ETLs to be scalable and efficient, collaborating with Data Analytics teams to ensure timely insights.
• Build frameworks/infrastructure in GCP enabling data analysts to deploy reporting efficiently.
• Build/maintain a data catalog platform to capture metadata and documentation of datasets.
• Contribute to foundation infrastructure for AI agents to maximize business value.
• Implement monitoring and alerting frameworks to ensure scalability, availability, and performance of the Data Platform.

Skills

Apache Spark

Apache Kafka

Google BigQuery

Python

Scala

Airflow

dbt

ETL pipelines

GCP

Education

Bachelor's in Computer Science

Tools

AWS Lambda

GCP Cloud Functions

Kafka Streams

HDFS

Job description

We're looking for a passionate Sr. Software Engineer (SDE-2) - Data Engineer to develop scalable data pipelines and analytics solutions. You'll solve complex data challenges, optimize systems for performance, and collaborate with teams to drive data-driven innovation that fuels our growth.

Responsibilities

• Contribute to the design and architecture of Pluang's Data Platform, leveraging Apache Spark, HDFS, Kafka Streams, and ETL pipelines to sync data to the Data Lake and Data Warehouse.
• Optimize data mart ETLs to make them scalable and efficient, collaborating with the Data Analytics teams to ensure business insights are delivered timely and accurately.
• Build frameworks/infrastructure in GCP that enable data analysts to be able to deploy reporting more efficiently.
• Build/maintain a data catalog platform to capture all the metadata and documentation of Pluang's vast datasets.
• Contribute to the foundation infrastructure for implementing AI agents to maximize business value.
• Implement monitoring and alerting frameworks to ensure the scalability, availability, and performance of the Data Platform, driving Pluang's data-driven future.

Requirements

• Bachelor's in Computer Science with expertise in Apache Spark, Apache Kafka, Google BigQuery, and serverless tech like AWS Lambda and GCP Cloud Functions.
• Proficient in Python/Scala, building ETL pipelines with Apache Airflow/dbt, and working with data streaming technologies for real-time processing.
• Strong analytical and problem-solving skills, with excellent communication abilities to collaborate across teams.
• Fintech experience is a plus, with an understanding of the unique challenges in data engineering for the industry.