Senior Hadoop Platform Engineer

Veriipro; Charlotte (NC); On-site; USD 120,000 - 150,000; Full time

Job summary

Veriipro is seeking a senior data platform engineer to own and optimize on-premise Hadoop ecosystems in the Charlotte area. The role focuses on maintaining HDFS, Hive Metastore, Spark and Ozone, and ensuring data lifecycle governance.

You will lead troubleshooting, capacity planning, data ingestion, ETL processes, and automation using Shell and Python, collaborating with cross-functional teams to keep production systems reliable and scalable.

Qualifications

• 10+ years of experience in Hadoop ecosystem and large-scale distributed data platforms.
• Strong hands-on experience with HDFS, Hive, Spark and Ozone.
• Experience with Hive Metastore administration and table/partition maintenance.
• Experience with Hadoop data lifecycle management, storage management, and data retention.
• SQL, Shell scripting, and Python automation proficiency.
• Experience with CDP or similar Hadoop distributions.
• Strong data analysis, troubleshooting, and root cause analysis skills.

Responsibilities

• Monitor, troubleshoot, and resolve HDFS and Apache Ozone issues.
• Investigate Hadoop cluster performance, storage utilization, capacity, and availability.
• Manage storage allocation, quota requests, capacity planning, and storage forecasting.
• Perform Hive Metastore administration, including table and partition maintenance.
• Support Hadoop platform maintenance, upgrades, integrations, onboarding, and decommissioning.
• Support data ingestion, ETL workflows, and data validation processes.
• Develop and maintain Shell and Python automation scripts for operational activities.
• Collaborate with application, data engineering, and platform teams during incident, problem, and change management activities.
• Perform root cause analysis and implement corrective and preventive actions.
• Create and maintain runbooks, operational documentation, knowledge articles, and support procedures.
• Ensure effective Hadoop data lifecycle, storage, and quota governance.

Tools

HDFS

Apache Hive

Apache Spark

Apache Ozone

Hive Metastore

Data lifecycle management

Storage management

Data retention

SQL

Shell scripting

Python automation

Cloudera Data Platform

CDP or Hadoop distributions

Monitoring/Observability tools

Job description

Veriipro is seeking a senior data platform engineer to own and optimize on-premise Hadoop ecosystems in the Charlotte area. The role focuses on maintaining HDFS, Hive Metastore, Spark and Ozone, and ensuring data lifecycle governance.

You will lead troubleshooting, capacity planning, data ingestion, ETL processes, and automation using Shell and Python, collaborating with cross-functional teams to keep production systems reliable and scalable.