Data Engineer II

V2 Solutions; Pune District; On-site; INR 1,500,000 - 2,800,000; Full time

Job summary

Mastercard Data Platform Orchestration team is seeking a Data Engineer II to design and build cloud-native data platforms supporting Mastercards global data ecosystem.

You will lead scalable batch and real-time pipelines, spanning Data Lakes and Data Warehouses, using Java, Spark, and cloud services. This hands-on role emphasizes code quality, mentoring, and engineering excellence.

Qualifications

• Strong proficiency in Java (JDK 8+) and OOAD principles.
• Experience with Spring Boot, REST APIs, Spring Security, Hibernate.
• Hands-on experience with distributed systems, multithreading, and messaging (Kafka).
• Hands-on experience with Apache Spark (Core, SQL, or Structured Streaming).
• Experience with Kafka or similar messaging/streaming platforms.
• Understanding of ETL/ELT pipelines, batch and streaming architectures.

Responsibilities

• Design and build scalable data pipelines and microservices using Java (Spring Boot), Spark, and cloud-native technologies.
• Develop high-throughput, low-latency data processing systems for real-time and batch workloads.
• Design and develop batch and near real-time data pipelines using Spark, Kafka, and Java-based frameworks.
• Build and maintain ETL/ELT pipelines for structured and unstructured data.
• Develop data processing solutions across Data Lakes and Data Warehouse environments.
• Contribute to stream processing use cases (Kafka, and optionally Flink or Spark Structured Streaming).
• Ensure data quality, validation, and reliability across pipelines.
• Optimize data processing workloads for performance and scalability.
• Develop and deploy data pipelines on AWS, Azure, or GCP.
• Leverage cloud-native services such as S3/ADLS/GCS, EMR/Databricks, BigQuery/Redshift/Snowflake.
• Contribute to Infrastructure as Code (Terraform, CloudFormation, or equivalent).
• Build solutions with high availability, fault tolerance, and scalability.
• Follow best practices for secure data processing and cloud resource utilization.
• Follow best practices in coding, testing, and CI/CD pipelines.
• Contribute to automation, monitoring, and observability of data pipelines.
• Develop reusable components to improve engineering efficiency and consistency.
• Participate in code reviews and design discussions.
• Collaborate with architects, product teams, and cross-functional stakeholders.
• Support production deployments and troubleshoot issues in distributed systems.
• Contribute to a culture of continuous improvement and technical excellence.
• Mentor junior engineers and share knowledge within the team.

Skills

Java (JDK 8+)

Spring Boot

Kafka

Spark

Cloud (AWS/Azure/GCP)

SQL

CI/CD

Python

Distributed systems

Education

Bachelor's degree in Computer Science / IT / Engineering

Tools

Docker

Kubernetes

Airflow

Terraform

Databricks

Job description

Title and Summary Data Engineer II Overview

Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, were helping build asustainableeconomy where everyone can prosper. We support a wide range of digital payments choices, making transactionssecure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.

Who is Mastercard

Mastercard is a global technology company in the payments industry. Our mission is to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart, and accessible. Using secure data and networks, partnerships, and passion, our innovations help individuals, financial institutions, governments, and businesses realize their greatest potential. The Mastercard Services organization is a key differentiator, delivering cutting-edge solutions used by some of the worlds largest organizations to make critical business decisions.

Team Overview

As part of Mastercards Data Platform Orchestration team, you will contribute to building next-generation data platforms that are critical to our global data ecosystem.

• Cloud-native infrastructure and application provisioning
• Standardized CI/CD pipelines and engineering tooling
• Reusable frameworks and data platform components

These platforms enable teams to build, deploy, and operate data-driven solutions efficiently and securely at scale.

Role Overview

Data Platform Orchestration is seeking a Data Engineer II to design and build next-generation, cloud-native data platforms supporting Mastercards global data ecosystem. In this role, you will lead the development of scalable batch and real-time data pipelines, enabling efficient data processing across Data Lakes and Data Warehouses. You will work at the intersection of data engineering, cloud platforms, and distributed systems, contributing to high-impact initiatives and driving engineering excellence. This role is ideal for someone who thrives in a fast-paced, collaborative environment, enjoys solving complex data challenges, and is passionate about building resilient, high-performance systems at scale.

As a Data Engineer II, you will design and develop scalable batch and near real-time data pipelines that power Mastercards analytics and operational systems. You will work across data engineering, cloud platforms, and distributed systems, building robust data solutions on top of Data Lakes and Data Warehouses. This role is highly hands-on and requires strong engineering fundamentals, with opportunities to influence design decisions and mentor junior engineers. You will contribute to building cloud-native data platforms, enabling reliable, high-performance data processing at scale.

Key Responsibilities

• Design and build scalable data pipelines and microservices using Java (Spring Boot), Spark, and cloud-native technologies
• Develop high-throughput, low-latency data processing systems for real-time and batch workloads
• Design and develop batch and near real-time data pipelines using Spark, Kafka, and Java-based frameworks
• Build and maintain ETL/ELT pipelines for structured and unstructured data
• Develop data processing solutions across Data Lakes and Data Warehouse environments
• Contribute to stream processing use cases (Kafka, and optionally Flink or Spark Structured Streaming)
• Ensure data quality, validation, and reliability across pipelines
• Optimize data processing workloads for performance and scalability
• Develop and deploy data pipelines on AWS, Azure, or GCP
• Leverage cloud-native services such as S3/ADLS/GCS, EMR/Databricks, BigQuery/Redshift/Snowflake
• Contribute to Infrastructure as Code (Terraform, CloudFormation, or equivalent)
• Build solutions with high availability, fault tolerance, and scalability
• Follow best practices for secure data processing and cloud resource utilization
• Follow best practices in coding, testing, and CI/CD pipelines
• Contribute to automation, monitoring, and observability of data pipelines
• Develop reusable components to improve engineering efficiency and consistency
• Participate in code reviews and design discussions
• Collaborate with architects, product teams, and cross-functional stakeholders
• Support production deployments and troubleshoot issues in distributed systems
• Contribute to a culture of continuous improvement and technical excellence
• Mentor junior engineers and share knowledge within the team

Required Skills Qualifications

• Strong proficiency in Java (JDK 8+), OOP/OOAD principles, Python is a plus
• Experience with Spring Boot, REST APIs, Spring Security, Hibernate
• Hands-on experience with distributed systems, multithreading, and messaging systems (Kafka preferred)
• Hands-on experience with Apache Spark (Core, SQL, or Structured Streaming)
• Experience with Kafka or similar messaging/streaming platforms
• Understanding of ETL/ELT pipelines, batch and streaming architectures
• Familiarity with data formats (Parquet, Avro, ORC)
• Basic understanding of data modeling (star/snowflake schemas)
• Understanding of distributed systems and multithreading concepts
• Hands-on experience with at least one cloud platform: AWS, Azure, or GCP
• Experience using cloud storage and compute services (e.g., S3, ADLS, Databricks, EMR)
• Familiarity with containerization (Docker) and basic Kubernetes concepts
• Exposure to Infrastructure as Code tools is a plus
• Strong SQL skills and experience working with relational and analytical databases
• Exposure to Data Lakes and Data Warehousing platforms
• Familiarity with workflow orchestration tools (Airflow or similar)
• Experience with CI/CD tools (Jenkins, GitHub Actions, or similar)
• Good understanding of unit testing (JUnit or equivalent)
• Familiarity with monitoring tools (Splunk, Prometheus, Grafana, etc.)
• Awareness of secure development practices
• Experience with real-time processing frameworks -Spark Streaming, good to have knowledge on Apache Flink
• Exposure to performance testing tools (JMeter, Gatling)
• Familiarity with DevSecOps or SRE concepts
• Experience improving automation and developer productivity
• Strong problem-solving and analytical skills
• Ownership mindset with ability to deliver independently
• Good communication and collaboration skills
• Passion for learning new technologies and improving engineering practices
• Ability to work effectively in a fast-paced, global environment

Education

Bachelors degree in Computer Science, Information Technology, Engineering, or a related field

Corporate Security Responsibility

All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must: Abide by Mastercards security policies and practices; Ensure the confidentiality and integrity of the information being accessed; Report any suspected information security violation or breach, and Complete all periodic mandatory security trainings in accordance with Mastercards guidelines.