TSenior Apache Spark Developer

Randstad USA; Jersey City (NJ); On-site; USD 171,208,000 - 199,862,000; Full time

Job summary

Randstad Digital in Jersey City, NJ seeks a Senior Apache Spark Developer to design and implement distributed data processing solutions using Spark and Kafka. You will support cloud adoption, Databricks and AWS migrations, and scale real-time and batch workloads across large-scale data platforms.

The role emphasizes building reusable data engineering components, optimizing Spark jobs, and collaborating with platform engineers to deliver cloud-native data solutions.

Qualifications

• 5+ years of hands-on development experience with Apache Spark.
• Experience integrating Spark with Apache Kafka.
• Bachelor's Degree in a related field or equivalent.
• Strong knowledge of Java and Python development and distributed systems.

Responsibilities

• Design, develop, and optimize Spark-based applications for large-scale data processing.
• Build streaming and batch data solutions leveraging Kafka and Spark.
• Develop reusable frameworks and components for enterprise data engineering.
• Support migration from on-premises platforms to Databricks and AWS.
• Collaborate with architects to implement cloud-native data solutions.
• Optimize Spark jobs for performance, scalability and reliability.
• Participate in design reviews, testing and production support.
• Implement CI/CD, monitoring and observability for distributed data apps.
• Troubleshoot issues across Spark, Kafka, Databricks and cloud environments.

Skills

Java

Python

Spark

Kafka

Spark SQL

Structured Streaming

DataFrames/Datasets

System design

Performance tuning

CI/CD

Observability

Education

Bachelor's Degree

Tools

Databricks

AWS

Job description

Job Summary

We are seeking a Senior Apache Spark Developer to help build and expand our next-generation data processing platform. This role will focus on designing and developing distributed data processing solutions using Apache Spark while leveraging an existing enterprise Kafka ecosystem.

The successful candidate will play a critical role in scaling our data processing capabilities, enabling cloud adoption, and supporting strategic migration initiatives involving Databricks and AWS. This position offers the opportunity to work on high-volume, mission-critical data platforms processing large-scale real-time and batch workloads.

Location

Jersey City, New Jersey

Job Type

Contract

Salary

$59.75 - 69.75 per hour

Work Hours

8am to 5pm

Education

Bachelors

Responsibilities

• Design, develop, and optimize Spark-based applications for large-scale data processing.
• Build streaming and batch data solutions leveraging Apache Kafka and Apache Spark.
• Develop reusable frameworks and components supporting enterprise data engineering initiatives.
• Support migration efforts from on-premises data platforms to Databricks and AWS.
• Collaborate with architects and platform engineers to implement cloud-native data solutions.
• Optimize Spark jobs for performance, scalability, reliability, and operational efficiency.
• Participate in system design, code reviews, testing, and production support activities.
• Implement CI/CD, monitoring, and observability practices for distributed data applications.
• Troubleshoot and resolve complex issues across Spark, Kafka, Databricks, and cloud environments and with other enterprise messaging systems.

Required Qualifications

• 5+ years of hands-on development experience with Apache Spark.
• Spark SQL
• Structured Streaming
• DataFrames/Datasets
• Performance tuning and optimization
• Experience integrating Spark with Apache Kafka.
• Strong development skills in Java
• Strong development skills in Python
• Experience designing and supporting large-scale distributed systems.
• Solid understanding of software engineering principles, object-oriented design, and testing practices.
• Experience working with relational and distributed data platforms.
• Bachelor's Degree

Preferred Qualifications

• Databricks experience (SQL).
• AWS experience, including services such as: S3, EMR, Glue, Lambda, ECS/EKS
• Experience supporting large-scale cloud migration initiatives.
• Experience with lakehouse architectures and modern data platform patterns.
• Experience handling multi-terabyte or petabyte-scale datasets.
• Financial services or regulated industry experience.

Skills

• S3
• AWS
• AWS experience
• Spark SQL
• Kafka
• Apache Kafka
• Apache Spark
• Spark
• Reusable frameworks
• Large-scale data processing
• Cloud environments
• Code reviews
• CI/CD
• Data platform
• Data platforms
• Data processing
• Streaming
• Data solutions
• Databricks
• Distributed data processing
• Distributed data
• Large-scale distributed systems
• Enterprise messaging systems
• Java
• Object-oriented design
• Performance tuning and optimization
• Python
• SQL
• Software engineering principles
• Troubleshoot
• Reliability
• Operational efficiency
• Financial services
• Observability practices
• Production support
• Scalability
• System design
• Testing
• Testing practices

Equal Opportunity Employer

Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.

Accommodation Statement

At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact [e-mail].

Benefits

Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including: medical, prescription, dental, vision, AD&D, and life insurance offerings, short-term disability, and a 401K plan (all benefits are based on eligibility).

Posting Duration

This posting is open for thirty (30) days.