Hadoop Platform Engineer

Dahl Consulting; United States; Remote; USD 113,000 - 165,000; Full time

Job summary

Dahl Consulting is seeking a Hadoop Platform Engineer for a remote, 6-month contract with a Fortune 100 retail technology organization. You will design, build, and optimize a large-scale Hadoop ecosystem, focusing on reliability, performance, and observability.

The role requires deep Hadoop, HDFS, Hive, Spark, Java, Linux, and automation skills, plus experience with Ansible and Chef. You will collaborate with data engineering and platform teams to keep the data platform secure and scalable.

Qualifications

• Bachelor's degree or equivalent practical experience.
• Experience supporting and administering enterprise Hadoop platforms.
• Hands-on Apache Hadoop stack experience.
• Experience with HDFS, Hive, Spark, YARN, and MapReduce.
• Proficiency in Java and Linux administration.
• Experience with Ansible and Chef for automation.
• Ability to diagnose and optimize Hadoop cluster performance.
• Knowledge of monitoring, observability, and alerting frameworks.
• Experience with infrastructure automation and runbooks.
• Strong communication and collaboration skills.

Responsibilities

• Design, build, and maintain a scalable Hadoop platform.
• Administer core Hadoop components and optimize performance.
• Troubleshoot cluster issues and perform root cause analysis.
• Develop and standardize monitoring and alerting frameworks.
• Automate operational tasks with scripting and automation tools.
• Collaborate with data engineering and infrastructure teams.
• Ensure security, governance, and compliance across the platform.

Skills

Apache Hadoop

HDFS

Hive

Spark

Java

Linux administration

Ansible

Chef

Monitoring & observability

Scripting (Python/Bash)

Education

Bachelor's degree in Computer Science or related field

Tools

Grafana

Python

Trino

Bash/Scala scripting

Job description

Title: Hadoop Platform Engineer

Location: Remote

Job Type: Contract (6 months)

Compensation: $82.00 - $120.00 per hour (W2)

Industry: Retail

About the Role
Our staffing firm is partnering with a leading Fortune 100 retail and consumer technology organization to identify an experienced Hadoop Platform Engineer. This company operates at enterprise scale, supporting large volumes of data that drive critical business, analytics, and customer-focused initiatives.
We are seeking a highly skilled engineer to design, optimize, and support a large-scale Hadoop ecosystem. The ideal candidate will combine deep technical expertise across the Apache Hadoop stack with strong Linux and automation experience to ensure platform reliability, performance, scalability, and observability.
This role offers the opportunity to work on a mission-critical data platform supporting enterprise analytics and data processing workloads while collaborating closely with data engineering, infrastructure, and platform teams.

Job Description
As a Hadoop Platform Engineer, you will be responsible for building, maintaining, and optimizing a high-performance Hadoop platform that supports large-scale data processing and analytics initiatives.

Responsibilities

• Design, build, and maintain a reliable, scalable, and high-performing Hadoop platform.
• Administer and optimize core Hadoop ecosystem components, including HDFS, Hive, Spark, YARN, and MapReduce.
• Troubleshoot and resolve cluster-level performance issues through deep analysis of Hadoop and Java-based systems.
• Develop and standardize monitoring, observability, and platform health frameworks.
• Implement proactive monitoring, automated health checks, and alerting capabilities to identify and resolve issues before they affect end users.
• Analyze and optimize Linux-based infrastructure to improve performance, resource utilization, and platform stability.
• Collaborate with data engineering, platform, and infrastructure teams to improve workload efficiency and cluster performance.
• Establish best practices for platform management, capacity planning, performance tuning, and upgrade strategies.
• Automate operational processes and reduce manual effort through scripting and infrastructure automation.
• Support data security, governance, and compliance requirements across the Hadoop ecosystem.
• Drive root cause analysis efforts for production incidents and implement preventative solutions.
• Create and maintain operational documentation, configuration standards, and support runbooks.
• Manage large-scale environments using configuration management and automation tools.

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent practical experience.

• Professional experience supporting and administering enterprise Hadoop platforms.

• Strong hands-on experience with:

• Apache Hadoop

• Hadoop Distributed File System (HDFS)

• Apache Hive

• Apache Spark

• Java

• Linux administration

• Ansible

• Chef

• Experience diagnosing and optimizing Hadoop cluster performance.

• Knowledge of monitoring, observability, and alerting frameworks.

• Experience managing large-scale distributed systems and data platforms.

• Strong troubleshooting and root cause analysis skills.

• Experience with infrastructure automation and operational process improvement.

• Ability to collaborate effectively across engineering and infrastructure teams.

• Strong written and verbal communication skills.

Preferred Qualifications

• Experience with Grafana for monitoring and visualization.
• Experience developing automation or tooling using Python.
• Experience with Trino and distributed query engines.
• Experience with Bash or Scala scripting.
• Experience supporting enterprise-scale analytics and data engineering environments.
• Knowledge of data governance, security, and compliance practices within big data ecosystems.
• Experience with capacity planning and platform modernization initiatives.

Benefits

Benefits
Dahl Consulting is proud to offer a comprehensive benefits package to eligible employees that will allow you to choose the best coverage to meet your family's needs. For details, please review the DAHL Benefits Summary: https://www.dahlconsulting.com/benefits-w2fta/.

Equal Opportunity Statement

Equal Opportunity Statement
As an equal opportunity employer, Dahl Consulting welcomes candidates of all backgrounds and experiences to apply. If this position sounds like the right opportunity for you, we encourage you to take the next step and connect with us. We look forward to meeting you!