Senior Spark Developer — Hybrid, Banking, Databricks & AWS

Collabera; Jersey City (NJ); Hybrid; USD 83,000 - 90,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

401(k) retirement plan

Life insurance

Long-term disability insurance

Short-term disability insurance

Paid parking/public transportation

Paid time off

Paid sick and safe time

Paid vacation time

Paid parental leave

Paid holidays

Job summary

Collabera is seeking a Senior Apache Spark Developer in Jersey City, NJ for a 12-month contract, hybrid in-person interview for local candidates. You will design and optimize Spark-based data processing, streaming and batch solutions, and support migration to Databricks and AWS. Strong Java/Python skills and experience with Kafka are essential.

The role focuses on large-scale distributed systems, performance tuning, and enterprise data platforms, with a mix of on-prem and cloud components.

Qualifications

• 5+ years of hands-on Apache Spark development.
• Strong skills in Spark SQL, Structured Streaming, and DataFrames/Datasets.
• Experience with performance tuning and optimization.
• Experience integrating Spark with Kafka.
• Proficient in Java and Python.
• Experience designing large-scale distributed systems.
• Solid understanding of OO design and testing practices.
• Experience with relational and distributed data platforms.
• Bachelor's degree required.

Responsibilities

• Design, develop, and optimize Spark-based applications for large-scale data processing.
• Build streaming and batch data solutions leveraging Apache Kafka and Apache Spark.
• Develop reusable frameworks and components supporting enterprise data engineering initiatives.
• Support migration efforts from on-premises data platforms to Databricks and AWS.
• Collaborate with architects and platform engineers to implement cloud-native data solutions.
• Optimize Spark jobs for performance, scalability, reliability, and operational efficiency.
• Participate in system design, code reviews, testing, and production support activities.
• Implement CI/CD, monitoring, and observability practices for distributed data applications.
• Troubleshoot and resolve complex issues across Spark, Kafka, Databricks, and cloud environments and with other enterprise messaging systems.

Skills

Spark SQL

Structured Streaming

DataFrames/Datasets

Performance tuning

Kafka integration

Java

Python

Distributed systems design

Object-oriented design

Testing practices

Relational/distributed data platforms

Education

Bachelor's Degree

Tools

Databricks

AWS

S3

EMR

Glue

Lambda

ECS/EKS

Job description

Collabera is seeking a Senior Apache Spark Developer in Jersey City, NJ for a 12-month contract, hybrid in-person interview for local candidates. You will design and optimize Spark-based data processing, streaming and batch solutions, and support migration to Databricks and AWS. Strong Java/Python skills and experience with Kafka are essential.

The role focuses on large-scale distributed systems, performance tuning, and enterprise data platforms, with a mix of on-prem and cloud components.