Lead Big Data Platform Engineer (Remote/Hadoop/Kafka)

Publicis Groupe; Irving (TX); Remote; USD 139,000 - 182,000; Full time

Benefits offered by this job

Remote work (US)

Job summary

Epsilon Data Management LLC in Irving, Texas is seeking a Lead, Big Data Administrator to deploy and manage large-scale distributed systems including Hadoop and Kafka. The role requires strong expertise in security, monitoring, and data pipelines, with remote work options nationwide.

Ideal candidates will have extensive experience with Hadoop components, Kafka, and automation using Bash and Python. Join a global team delivering scalable data solutions for the Publicis Groupe ecosystem.

Qualifications

• Bachelor's degree in Computer Applications or related field with eight years of progressive post-baccalaureate experience.
• Proficiency with core Hadoop components and managing a cluster (HDFS, Yarn, Hive, Spark) and Kafka clusters.
• Kerberos Security-Authentication, LDAP integration; Encryption: SSL/TLS; Audit tools: Ranger/Atlas.

Responsibilities

• Deploy and manage large-scale distributed systems with high availability and scalability.
• Install, configure, and maintain Hadoop ecosystems (HDFS, Yarn, Hive, Spark) and Kafka clusters.
• Implement disaster recovery and high-availability strategies for Hadoop and Kafka environments.
• Performance tuning with resource allocation, cluster optimization, and monitoring.
• Data pipeline orchestration and integration between Kafka producers, consumers, and downstream processing frameworks.
• Automate routines with Ansible and shell scripting to improve operational efficiency.

Skills

Hadoop stack

ZooKeeper

Kerberos/LDAP

Prometheus Grafana

Troubleshooting

Cluster tuning

Bash

Python

Education

Bachelor's degree in Computer Applications

Tools

Kafka

Ansible

Shell scripting

Job description

Epsilon Data Management LLC in Irving, Texas is seeking a Lead, Big Data Administrator to deploy and manage large-scale distributed systems including Hadoop and Kafka. The role requires strong expertise in security, monitoring, and data pipelines, with remote work options nationwide.

Ideal candidates will have extensive experience with Hadoop components, Kafka, and automation using Bash and Python. Join a global team delivering scalable data solutions for the Publicis Groupe ecosystem.