Spark, HDFS & Kafka Engineer — Hybrid (Jersey City)

Yoh; Jersey City (NJ); Hybrid; USD 68,000 - 96,000; Full time

Benefits offered by this job

Medical Benefits

401K Retirement Savings Plan

Life & Disability Insurance

Employee Assistance Program (EAP)

Direct Deposit & weekly epayroll

Certification and training

Job summary

Yoh is seeking an experienced Spark, HDFS, and Kafka Administrator/Engineer to manage and optimize enterprise-grade big data platforms. The role focuses on Spark, HDFS, and Kafka within a Cloudera environment, with broader platform duties as needed.

The candidate will deploy, configure, monitor, and tune Spark, HDFS, and Kafka workloads, ensure security and compliance, and participate in on-call support. Hybrid onsite model in Jersey City with competitive benefits.

Qualifications

• Excellent problem-solving, analytical, and communication skills.
• Hybrid onsite 4 days a week.
• Minimum 3 years of hands-on production experience administering and supporting Apache Spark, HDFS, and Kafka.
• Strong, demonstrable expertise in Spark

Responsibilities

• Spark Administration: Deploy, configure, monitor, tune, and troubleshoot Spark environments and workloads.
• HDFS Administration: Manage storage, capacity, performance, permissions, replication, and troubleshooting.
• Kafka Administration: Deploy, configure, manage, monitor, and troubleshoot Kafka clusters, topics, producers, and consumers.
• Monitoring and Performance Tuning: Monitor connectivity, health, capacity, and performance using Grafana and Splunk.
• Automation and Scripting: Develop automation using Python, Bash, or Perl for routine tasks.

Skills

Communication

Problem-solving

Analytical thinking

Hands-on

Tools

Grafana

Splunk

Python

Bash

Perl

Job description

Yoh is seeking an experienced Spark, HDFS, and Kafka Administrator/Engineer to manage and optimize enterprise-grade big data platforms. The role focuses on Spark, HDFS, and Kafka within a Cloudera environment, with broader platform duties as needed.

The candidate will deploy, configure, monitor, and tune Spark, HDFS, and Kafka workloads, ensure security and compliance, and participate in on-call support. Hybrid onsite model in Jersey City with competitive benefits.