Senior Data Engineer

PayU; India; On-site; INR 1,800,000 - 3,000,000; Full time

Benefits offered by this job

Training courses

Global exposure

Inclusive culture

Learning & development

Job summary

PayU is seeking a Senior Data Engineer to design data infrastructure and pipelines in support of ML and analytics.

Role focuses on building scalable data systems, integrating diverse data sources, and enabling high-performance data processing for merchant and customer applications.

Qualifications

• Proficiency in Python, Pandas, ETL, and data modelling is required.
• Experience with scalable compute like Kafka and Snowflake is essential.
• Familiarity with Airflow and AWS Step Functions is expected.
• Understanding data ingestion pipelines and ETL practices is necessary.
• Knowledge of ML concepts, pipelines, and testing methods is a plus.
• Educational background in CS/Engineering/Math/Physics is preferred.

Responsibilities

• Design data infrastructure for ML consumption and analytics.
• Define architecture to integrate data from multiple sources with integrity.
• Ensure performance of data systems across services and applications.
• Work with large, complex datasets to solve non-routine analyses.
• Build and maintain data pipelines and infrastructure components.
• Collaborate with Data Engineers, ML Engineers and SREs to deploy data science solutions.

Skills

Python

Pandas

Data wrangling

ETL processes

Spark statistics

Data visualisation

Data Modelling

Informatica

Kafka

Snowflake

Airflow

AWS Step Functions

Data ingestion pipelines

Machine learning concepts

Education

B.Tech / BE /MEng/MSc/PhD in CS/Engineering/Math/Physics

Tools

Kafka

Snowflake

Airflow

AWS Step Functions

Informatica

Job description

About Company

PayU is the payments and fintech business of Prosus, a global consumer internet group and one of the largest technology investors in the world. Operating and investing globally in markets with long-term growth potential, Prosus builds leading consumer internet companies that empower people and enrich communities.

Role: Senior Data Engineer

PayU PAYMENTS PRIVATE LIMITED

Company: PayU

Location: Bangalore/ Mumbai / Gurgoan

About Company

PayU is the payments and fintech business of Prosus, a global consumer internet group and one of the largest technology investors in the world. Operating and investing globally in markets with long-term growth potential, Prosus builds leading consumer internet companies that empower people and enrich communities. The leading online payment service provider in 36 countries, PayU is dedicated to creating a fast, simple and efficient payment process for merchants and buyers. Focused on empowering people through financial services and creating a world without financial borders where everyone can prosper, PayU is one of the biggest investors in the fintech space globally, with investments totalling $700 million- to date. PayU also specializes in credit products and services for emerging markets across the globe. We are dedicated to removing risks to merchants, allowing consumers to use credit in ways that suit them and enabling a greater number of global citizens to access credit services.

Our local operations in Asia, Central and Eastern Europe, Latin America, the Middle East, Africa and South East Asia enable us to combine the expertise of high growth companies with our own unique local knowledge and technology to ensure that our customers have access to the best financial services.

India is the biggest market for PayU globally and the company has already invested $400 million in this region in last 4 years. PayU in its next phase of growth is developing a full regional fintech ecosystem providing multiple digital financial services in one integrated experience. We are going to do this through 3 mechanisms: build, co-build/partner; select strategic investments.

PayU supports over 350,000+ merchants and millions of consumers making payments online with over 250 payment methods and 1,800+ payment specialists. The markets in which PayU operates represent a potential consumer base of nearly 2.3 billion people and a huge growth potential for merchants.

Job Responsibilities

• Design infrastructure for data, especially for but not limited to consumption in machine learning applications
• Define database architecture needed to combine and link data, and ensure integrity across different sources
• Ensure performance of data systems for machine learning to customer-facing web and mobile applications using cutting-edge open source frameworks, to highly available RESTful services, to back-end Java based systems
• Work with large, fast, complex data sets to solve difficult, non-routine analysis problems, applying advanced data handling techniques if needed
• Build data pipelines, includes implementing, testing, and maintaining infrastructural components related to the data engineering stack.
• Work closely with Data Engineers, ML Engineers and SREs to gather data engineering requirements to prototype, develop, validate and deploy data science and machine learning solutions

Requirements To Be Successful In This Role

• Strong knowledge and experience in Python, Pandas, Data wrangling, ETL processes, Spark statistics, data visualisation, Data Modelling and Informatica.
• Strong experience with scalable compute solutions such as in Kafka, Snowflake
• Strong experience with workflow management libraries and tools such as Airflow, AWS Step Functions etc.
• Strong experience with data engineering practices (i.e. data ingestion pipelines and ETL)
• A good understanding of machine learning methods, algorithms, pipelines, testing practices and frameworks
• B. Tech / BE /MEng/MSc/PhD degree in computer science, engineering, mathematics, physics, or equivalent (preference: DS/ AI)
• Experience with designing and implementing tools that support sharing of data, code, practices across organizations at scale

What we offer?

• A positive, get-things-done workplace
• A dynamic, constantly evolving space (change is par for the course – important you are comfortable with this)
• An inclusive environment that ensures we listen to a diverse range of voices when making decisions.
• Ability to learn cutting edge concepts and innovation in an agile start-up environment with a global scale
• Access to 5000+ training courses accessible anytime/anywhere to support your growth and development (Corporate with top learning partners like Harvard, Coursera, Udacity)

About Us

At PayU, we are a global fintech investor and our vision is to build a world without financial borders where everyone can prosper. We give people in high growth markets the financial services and products they need to thrive. Our expertise in 18+ high-growth markets enables us to extend the reach of financial services. This drives everything we do, from investing in technology entrepreneurs to offering credit to underserved individuals, to helping merchants buy, sell, and operate online. Being part of Prosus, one of the largest technology investors in the world, gives us the presence and expertise to make a real impact. Find out more at www.payu.com

Our Commitment to Building A Diverse and Inclusive Workforce

As a global and multi-cultural organization with varied ethnicities thriving across locations, we realize that our responsibility towards fulfilling the D&I commitment is huge. Therefore, we continuously strive to create a diverse, inclusive, and safe environment, for all our people, communities, and customers. Our leaders are committed to create an inclusive work culture which enables transparency, flexibility, and unbiased attention to every PayUneer so they can succeed, irrespective of gender, color, or personal faith. An environment where every person feels they belong, that they are listened to, and where they are empowered to speak up. At PayU we have zero tolerance towards any form of prejudice whether a specific race, ethnicity, or of persons with disabilities, or the LGBTQ communities.