Associate Data Engineer

Wibmo Inc.; Gurugram District; On-site; INR 1,000,000 - 1,400,000; Full time

Job summary

PayU is hiring an Associate Data Engineer / Data Engineer to design, build, and maintain scalable data pipelines and processing solutions across batch and real-time platforms.

You will enable reliable data movement from operational systems to data warehouses, lakes, and analytics platforms, collaborating with stakeholders to meet evolving business needs. Your role evolves in a cloud-native, open-source tech stack at scale.

Qualifications

• 2–4 years of professional experience in Data Engineering or related fields.
• Proficiency in Python and SQL with strong programming skills.
• Understanding of distributed data processing and large-scale architectures.
• Experience building ETL/ELT pipelines and data warehouses or lakehouses.
• Experience with cloud platforms, preferably AWS.
• Knowledge of data modeling and DataMart development.
• Familiarity with batch processing; streaming is a plus.
• Experience with Git, testing, and CI/CD practices.
• Strong communication and collaboration skills.

Responsibilities

• Understand data requirements with stakeholders and deliver reliable technical solutions.
• Participate in requirements gathering, design, development, testing, deployment, and support for data engineering initiatives.
• Design, develop, and maintain reliable near-real-time (NRT) data pipelines using Kafka, Flink, Python, and SQL.
• Build and support NRT data processing workflows for analytics, monitoring, reporting, and downstream use cases.
• Develop and maintain Flink-based data lake solutions for high-volume data.
• Work with Apache Iceberg for scalable lakehouse platforms and data lifecycle management.
• Contribute to data ingestion, transformation, schema evolution, and partitioning.

Skills

Python

SQL

Analytics

Communication

Problem solving

Distributed systems

ETL/ELT

Data warehousing

Data modeling

Batch processing

Tools

Apache Spark

Apache Kafka

Apache Flink

Apache Iceberg

Trino

StarRocks

ClickHouse

Hadoop

Airflow

Git

Job description

Select how often (in days) to receive an alert:

Our Data Engineering team is the backbone of PayU’s data-driven organization. We design, build, and operate scalable data platforms that support critical business decisions across multiple domains.

Working with large and diverse data volumes, the team enables reliable data movement from operational systems to data warehouses, data lakes, lakehouses, and analytics platforms. We collaborate closely with business and technology stakeholders to build fit-for-purpose data solutions that address evolving organizational needs.

About the Role

As an Associate Data Engineer / Data Engineer, you will contribute to the design, development, and maintenance of scalable data pipelines and processing solutions. You will work with batch and real-time data platforms, support data warehouse and lakehouse initiatives, and help ensure the performance, quality, reliability, and accessibility of enterprise data.

This role provides an opportunity to work with modern cloud and open-source technologies while developing expertise across data warehousing, data lakes, lakehouses, and analytical serving platforms.

Key Responsibilities

• Work with data engineers, architects, product teams, and business stakeholders to understand data requirements and deliver reliable technical solutions.
• Participate in requirements gathering, solution design, development, testing, deployment, and support for data engineering initiatives.
• Design, develop, and maintain reliablenear-real-time (NRT) data pipelinesusing technologies such as Apache Kafka, Apache Flink, Python, and SQL.
• Build and support NRT data processing workflows that enable timely analytics, monitoring, reporting, and downstream business use cases.
• Develop and maintainFlink-based data lake solutionsfor processing high-volume, continuously changing data.
• Work withApache Iceberg as the table and data formatfor building scalable, reliable, and queryable data lakehouse platforms.
• Contribute to data ingestion, transformation, schema evolution, partitioning, compaction, and data lifecycle management within Iceberg-based data lakes.
• Develop and maintain analytical datasets, data models, and DataMart solutions for reporting and business intelligence.
• Build and optimize data processing and query workloads using open-source technologies such asApache Spark, Trino, and StarRocks.
• Support low-latency analytical serving and interactive query use cases using open-source query engines such asStarRocks and Trino.
• Use Apache Spark for distributed data processing, transformations, data preparation, and large-scale analytical workloads.
• Write efficient SQL queries and contribute to query optimization, data partitioning, and performance improvements across data platforms.
• Implement data validation, quality checks, monitoring, alerting, and observability across NRT pipelines and data lake workloads.
• Troubleshoot data pipeline and production issues, perform initial root-cause analysis, and support timely incident resolution.
• Write maintainable, testable, and well-documented code.
• Participate in code reviews and follow engineering standards and best practices.
• Contribute to technical documentation, operational runbooks, and continuous improvement initiatives.
• Work effectively in an Agile and collaborative engineering environment.

Required Qualifications and Experience

• 2–4 years of professional experience in Data Engineering, Software Engineering, Analytics Engineering, or a related discipline.
• Strong programming and analytical skills, with hands-on experience inPython and SQL.
• Good understanding of data structures, algorithms, relational databases, and software development principles.
• Understanding of distributed systems, data processing, and large-scale data architectures.
• Experience building, maintaining, or supporting ETL and ELT data pipelines.
• Experience working with data warehouses, data lakes, or lakehouse platforms.
• Understanding of data modeling, dimensional modeling, and DataMart development.
• Experience working with batch processing; exposure to streaming or event-driven data pipelines is an advantage.
• Understanding of SQL query optimization, data partitioning, indexing, and performance tuning.
• Experience working with relational databases and familiarity with NoSQL databases.
• Hands-on experience with cloud platforms, preferably AWS, including one or more of the following:
  • Amazon S3
  • Amazon EMR
  • Amazon Redshift
  • Amazon MSK
• Practical experience with some of the following technologies:
  • Apache Spark / PySpark
  • Apache Airflow
  • Apache Kafka
  • Debezium
  • Apache Flink
  • Apache Iceberg
  • Trino
  • StarRocks
  • ClickHouse
  • Hadoop
• Experience with Git, code reviews, testing, and CI/CD practices.
• Ability to troubleshoot issues, analyze problems, and communicate solutions clearly.
• Strong communication and collaboration skills.

Good to Have

• Experience with Change Data Capture, database replication, or incremental data processing.
• Familiarity with data quality frameworks, pipeline monitoring, and observability practices.
• Exposure to containerization technologies such as Docker and Kubernetes.
• Familiarity with infrastructure automation and DevOps practices.
• Experience working with on-premises or hybrid data platforms.
• Exposure to Apache Iceberg, lakehouse architecture, or open table formats.
• Experience working in fintech, payments, banking, or other high-volume transaction environments.
• Familiarity with NoSQL databases such as Cassandra, MongoDB, or Redis.
• Understanding of Agile development methodologies and engineering best practices.

What We Offer

• A positive, collaborative, and get-things-done workplace.
• A dynamic and constantly evolving environment where adaptability is valued.
• An inclusive culture that encourages diverse perspectives and open communication.
• Opportunities to work on modern data technologies at global scale.
• The opportunity to learn and innovate in an agile fintech environment.
• Access to 5,000+ training courses available anytime and anywhere through leading learning partners such as Harvard, Coursera, and Udacity.
• Opportunities to work closely with experienced engineers and grow across data engineering, cloud, and distributed systems.

About PayU

At PayU, we are a global fintech investor with a vision to build a world without financial borders, where everyone can prosper. We provide people in high-growth markets with the financial services and products they need to thrive.

Our expertise across 18+ high-growth markets enables us to expand access to financial services. This drives everything we do—from investing in technology entrepreneurs and offering credit to underserved individuals, to helping merchants buy, sell, and operate online.

As part of Prosus, one of the world’s largest technology investors, PayU combines global reach, deep expertise, and the ability to create meaningful impact. Learn more at www.payu.com .

Our Commitment to Diversity and Inclusion

PayU is committed to building a diverse, inclusive, and safe workplace where every individual feels respected, valued, heard, and empowered to succeed.

As a global, multicultural organization, we welcome people from diverse backgrounds, identities, experiences, and perspectives. Our leaders are committed to fostering a transparent, flexible, and equitable work culture where everyone has the opportunity to grow and contribute.

PayU has zero tolerance for discrimination or prejudice based on race, ethnicity, gender, color, religion, disability, sexual orientation, gender identity, or any other personal characteristic.

Copyright 2025 PayU. All rights reserved / Privacy / Terms of use