Senior Data Platform Engineer: Spark Kafka Flink on Azure & Kubernetes

NTT DATA North America; Cupertino (CA); On-site; USD 83,000 - 103,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

401k plan

AD&D insurance

Job summary

NTT DATA North America is seeking a Data Engineer to join its Cupertino team, building next-generation cloud-native data platforms for real-time and batch processing. You will craft Spark/Flink/Kafka-based pipelines, design metadata-driven ingestion connectors, and deploy on Kubernetes in Azure cloud environments.

The role emphasizes distributed systems expertise, platform engineering, automation, and data modernization across enterprise-scale workloads.

Qualifications

• 8+ years of software or data engineering experience.
• 5+ years with Apache Spark for large-scale batch and streaming data processing.
• 5+ years with Apache Kafka including event-driven architectures and real-time data streaming solutions.
• 3+ years with Apache Flink for stream processing and real-time analytics workloads.
• 5+ years developing applications using Java and/or Scala.
• 5+ years writing complex SQL queries and optimizing database performance.
• 3+ years with Kubernetes and containerized application deployment.
• 3+ years designing and implementing solutions on Microsoft Azure Cloud.
• 3+ years building and supporting distributed data platforms using Cassandra, YugabyteDB, PostgreSQL, or similar databases.
• 2+ years implementing CI/CD pipelines via GitLab, GitHub, or similar tools.
• 2+ years cloud-native deployment patterns, containerization, and platform automation.

Responsibilities

• Design, build, and support scalable real-time and batch data pipelines using Spark, Flink, Kafka, and Airflow.
• Develop metadata-driven self-service data integration platforms and connectors.
• Create and maintain connectors for relational DBs, file systems, Kafka, Cassandra, YugabyteDB, and other data stores.
• Deploy and operate Kubernetes-based streaming/batch platforms.
• Lead cloud migration initiatives to Microsoft Azure.
• Tune performance, improve scalability and reliability of large-scale data workloads.
• Develop cloud-native solutions for enterprise data movement and processing.
• Collaborate with product owners, architects, cloud engineering, and business stakeholders to deliver enterprise-scale data solutions.
• Contribute to platform modernization, CI/CD, and engineering best practices.
• Provide troubleshooting and ensure production stability of critical data platforms.

Skills

Apache Spark

Apache Kafka

Apache Flink

Java/Scala

SQL

Kubernetes

Azure

Airflow

Education

Bachelor's degree in Computer Science or related field

Tools

Git

GitHub

GitLab

Docker

Job description

NTT DATA North America is seeking a Data Engineer to join its Cupertino team, building next-generation cloud-native data platforms for real-time and batch processing. You will craft Spark/Flink/Kafka-based pipelines, design metadata-driven ingestion connectors, and deploy on Kubernetes in Azure cloud environments.

The role emphasizes distributed systems expertise, platform engineering, automation, and data modernization across enterprise-scale workloads.