Big Data Platform Engineer (Spark/HDFS/Kafka) – Chicago

Ethereum Technologies

Chicago (IL)

On-site

USD 81,000 - 115,000

Part time

6 hours ago

Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Ethereum Technologies is seeking a Big Data Platform Engineer in Chicago to administer Spark, HDFS, and Kafka environments. You will monitor, tune, and secure a distributed data platform, integrating Kerberos, SSL/TLS, and IAM/LDAP.

The role requires strong scripting skills and experience with Grafana/Splunk for performance visibility. Responsibilities include platform lifecycle support, on-call rotation, and collaboration across teams to ensure reliable data processing, storage, and analytics

Qualifications

Minimum 3–5 years of hands-on production administration with Spark, HDFS, and Kafka.
Solid Linux administration experience and command-line proficiency.
Experience with monitoring/observability tools (Grafana, Splunk).
Strong understanding of Kerberos, SSL/TLS, IAM/LDAP integration and security concepts.
Excellent communication and collaboration skills.
Bachelor’s degree in Computer Science, Information Systems, or related field.

Responsibilities

Deploy, configure, monitor, tune, and troubleshoot Spark environments and workloads.
Manage HDFS storage, capacity, performance, and replication.
Deploy, configure, monitor, and troubleshoot Kafka clusters, topics, producers, and consumers.
Monitor and optimize platform performance using Grafana, Splunk, and related tools.
Implement security configurations including Kerberos and IAM/LDAP integrations.
Develop automation with Python, Bash, or Perl for admin tasks.
Provide on-call support and troubleshoot end-to-end big data stack.
Support Cloudera Hadoop/CDP installations, patching, upgrades, and migrations.