Senior Spark Engineer - Real-Time Data & Cloud Migration

Randstad USA; Jersey City (NJ); On-site; USD 171,208,000 - 199,862,000; Full time

Job summary

Randstad Digital in Jersey City, NJ seeks a Senior Apache Spark Developer to design and implement distributed data processing solutions using Spark and Kafka. You will support cloud adoption, Databricks and AWS migrations, and scale real-time and batch workloads across large-scale data platforms.

The role emphasizes building reusable data engineering components, optimizing Spark jobs, and collaborating with platform engineers to deliver cloud-native data solutions.

Qualifications

• 5+ years of hands-on development experience with Apache Spark.
• Experience integrating Spark with Apache Kafka.
• Bachelor's Degree in a related field or equivalent.
• Strong knowledge of Java and Python development and distributed systems.

Responsibilities

• Design, develop, and optimize Spark-based applications for large-scale data processing.
• Build streaming and batch data solutions leveraging Kafka and Spark.
• Develop reusable frameworks and components for enterprise data engineering.
• Support migration from on-premises platforms to Databricks and AWS.
• Collaborate with architects to implement cloud-native data solutions.
• Optimize Spark jobs for performance, scalability and reliability.
• Participate in design reviews, testing and production support.
• Implement CI/CD, monitoring and observability for distributed data apps.
• Troubleshoot issues across Spark, Kafka, Databricks and cloud environments.

Skills

Java

Python

Spark

Kafka

Spark SQL

Structured Streaming

DataFrames/Datasets

System design

Performance tuning

CI/CD

Observability

Education

Bachelor's Degree

Tools

Databricks

AWS

Job description

Randstad Digital in Jersey City, NJ seeks a Senior Apache Spark Developer to design and implement distributed data processing solutions using Spark and Kafka. You will support cloud adoption, Databricks and AWS migrations, and scale real-time and batch workloads across large-scale data platforms.

The role emphasizes building reusable data engineering components, optimizing Spark jobs, and collaborating with platform engineers to deliver cloud-native data solutions.