Senior Spark Architect | Data Platform & Cloud Migration

Talentify; Jersey City (NJ); On-site; USD 88,000 - 99,000; Full time

Benefits offered by this job

Medical

Dental

Vision

Health Savings Account

401K

Sick Pay

Commuter Benefits

Behavioral Health Platform

Life Insurance

Job summary

Genesis10 is seeking a Senior Apache Spark Developer to design and develop distributed data processing solutions using Spark while leveraging an enterprise Kafka ecosystem. The role focuses on scaling data processing capabilities, enabling cloud adoption, and supporting migration initiatives involving Databricks and AWS.

The candidate will design and deploy Spark-based applications, build streaming/batch solutions, and collaborate with architects and platform engineers to ensure performance,

Qualifications

• 5+ years of hands-on development experience with Apache Spark.
• Strong expertise in Spark SQL, Structured Streaming, DataFrames/Datasets, performance tuning and optimization.
• Experience integrating Spark with Apache Kafka.
• Strong development skills in Java and Python.
• Experience designing and supporting large-scale distributed systems.
• Solid understanding of software engineering principles, object-oriented design, and testing practices.
• Experience working with relational and distributed data platforms.

Responsibilities

• Design, develop, and optimize Spark-based applications for large-scale data processing.
• Build streaming and batch data solutions leveraging Apache Kafka and Apache Spark.
• Develop reusable frameworks and components supporting enterprise data engineering initiatives.
• Support migration efforts from on-premises data platforms to Databricks and AWS.
• Collaborate with architects and platform engineers to implement cloud-native data solutions.
• Optimize Spark jobs for performance, scalability, reliability, and operational efficiency.
• Participate in system design, code reviews, testing, and production support activities.
• Implement CI/CD, monitoring, and observability practices for distributed data applications.
• Troubleshoot and resolve complex issues across Spark, Kafka, Databricks, and cloud environments and with other enterprise messaging systems.

Skills

Apache Spark

Spark SQL

Structured Streaming

DataFrames/Datasets

Kafka

Java

Python

Distributed systems

Testing practices

Education

Bachelor's Degree

Tools

Databricks

AWS

Job description

Genesis10 is seeking a Senior Apache Spark Developer to design and develop distributed data processing solutions using Spark while leveraging an enterprise Kafka ecosystem. The role focuses on scaling data processing capabilities, enabling cloud adoption, and supporting migration initiatives involving Databricks and AWS.

The candidate will design and deploy Spark-based applications, build streaming/batch solutions, and collaborate with architects and platform engineers to ensure performance,