Senior Spark Developer — Cloud Data Platform Lead

The Matlen Silver Group, Inc.; Jersey City (NJ); On-site; USD 90,000 - 101,000; Full time

Job summary

The Matlen Silver Group, Inc. is seeking a Senior Apache Spark Developer in Jersey City, NJ, on a 18-month W2 contract. You will design and build distributed data processing solutions using Spark, Kafka, and cloud services, with emphasis on Databricks and AWS migration work.

You will optimize Spark jobs, implement CI/CD, and collaborate with architects and platform engineers to deliver scalable, reliable data platforms handling real-time and batch workloads.

Qualifications

• 5+ years of hands-on development experience with Apache Spark.
• Strong expertise in Spark SQL, Structured Streaming, DataFrames/Datasets, Performance tuning.
• Experience integrating Spark with Apache Kafka.
• Strong development skills in Java and Python.
• Experience designing and supporting large-scale distributed systems.
• Solid understanding of software engineering principles, object-oriented design, and testing practices.
• Experience working with relational and distributed data platforms.
• Bachelor’s Degree

Responsibilities

• Design, develop, and optimize Spark-based applications for large-scale data processing.
• Build streaming and batch data solutions leveraging Apache Kafka and Apache Spark.
• Develop reusable frameworks and components supporting enterprise data engineering initiatives.
• Support migration efforts from on-premises data platforms to Databricks and AWS.
• Collaborate with architects and platform engineers to implement cloud-native data solutions.
• Optimize Spark jobs for performance, scalability, reliability, and operational efficiency.
• Participate in system design, code reviews, testing, and production support activities.
• Implement CI/CD, monitoring, and observability practices for distributed data applications.
• Troubleshoot and resolve complex issues across Spark, Kafka, Databricks, and cloud environments and with other enterprise messaging systems.

Skills

Spark

Java

Python

Kafka

Distributed systems

Spark SQL

Structured Streaming

DataFrames/Datasets

Performance tuning

Education

Bachelor's degree

Tools

Databricks

AWS

S3

EMR

Glue

Lambda

ECS/EKS

Job description

The Matlen Silver Group, Inc. is seeking a Senior Apache Spark Developer in Jersey City, NJ, on a 18-month W2 contract. You will design and build distributed data processing solutions using Spark, Kafka, and cloud services, with emphasis on Databricks and AWS migration work.

You will optimize Spark jobs, implement CI/CD, and collaborate with architects and platform engineers to deliver scalable, reliable data platforms handling real-time and batch workloads.