Big Data Platform Administrator

Seaglass Technology Partners, LLC; Jersey City (NJ); Hybrid; USD 96,000 - 110,000; Full time

Job summary

Seaglass Technology Partners, LLC is seeking an experienced Spark, HDFS, and Kafka Administrator/Engineer for a hybrid role in Jersey City, NJ.

You will maintain and optimize enterprise big data environments, manage Spark workloads, HDFS storage, and Kafka clusters, while leveraging Grafana or Splunk for monitoring.

Qualifications

• 3+ years hands-on with Spark, HDFS and Kafka in production.
• Experience managing Spark environments: setup, tuning, monitoring.
• Strong HDFS background: storage, access, availability.
• Kafka cluster operations: security, topic config, troubleshooting.
• Proficient in Linux administration and scripting basics.

Responsibilities

• Maintain and support enterprise big data environments.
• Optimize Spark workloads and monitor performance.
• Manage HDFS storage, access controls and availability.
• Operate Kafka clusters, including security and throughput.
• Troubleshoot with Grafana, Splunk and other tools.

Skills

Spark admin

HDFS admin

Kafka admin

Linux sysadmin

Monitoring tools

Tools

Grafana

Splunk

Cloudera CDP

Cloudera Manager

Job description

Title: Spark, HDFS, and Kafka Administrator/Engineer

Location: Hybrid (4 days on site - Jersey City, NJ)

Duration: 6-12 Month + contract

Pay: 70-80 H/R W2

• Minimum of 3 years of hands-on experience maintaining and supporting enterprise big data environments utilizing Apache Spark, HDFS, and Kafka in production.
• Proven ability to manage Spark environments, including system setup, workload optimization, performance monitoring, and resolving technical issues.
• Strong background supporting HDFS, with experience managing distributed storage, access controls, system availability, and performance.
• Practical experience administering Kafka, including cluster operations, topic configuration, message processing, security, and troubleshooting.
• Proficiency working in Linux environments, including executing commands, investigating system issues, and performing administrative tasks.
• Familiarity with monitoring and troubleshooting platforms such as Grafana, Splunk, or comparable enterprise tools.
• Working knowledge of Cloudera CDP, Hadoop, and Cloudera Manager is a plus, although direct expertise with Spark, HDFS, and Kafka takes priority.
• Relevant certifications in Cloudera or other big data technologies are beneficial but not mandatory.
• Understanding of distributed platform security, including Kerberos, SSL/TLS, IAM, and LDAP authentication and access management.