Hadoop Support with Devops-6

Realign Llc; Plano (TX); On-site; USD 110,000 - 150,000; Full time

Job summary

Realign Llc in Plano, TX seeks an experienced Hadoop Support with DevOps to provide L1–L3 platform support and optimize cluster performance. You will manage storage, quotas, and capacity while coordinating incidents with application teams.

Ideal candidates have deep Hadoop ecosystem knowledge (HDFS, Ozone, Hive, Spark) and strong Linux/DevOps skills, including automation, scripting, and monitoring tools. This is a full-time on-site role in Plano.

Qualifications

• Experience on L1, L2 & L3 level support in Hadoop Platform.
• Experience in Yarn, Spark and Impala job debugging and troubleshooting.
• Experience in addressing issues around NameNode, HDFS Space, File/Folder Permission & backups.
• Understanding and experience in query performance tuning and resource utilization tuning.
• Technical knowledge on relational databases, data warehousing and SQL skills.
• Hadoop, Kafka, Spark, Impala, Hive, HBase, Ozone etc.
• Strong knowledge of Hadoop Architecture, HDFS, Hadoop Cluster and Hadoop Administrator's role.
• Intimate knowledge of fully integrated AD/Kerberos authentication.
• Experience setting up optimum cluster configurations.
• Debugging knowledge of YARN.
• Hands-on with analyzing various Hadoop log files, compression, encoding, & file formats.
• Expert level knowledge of Cloudera Hadoop components such as HDFS, Sentry, HBase, Kafka, Impala, SOLR, Hue, Spark, Hive, YARN, Zookeeper and Postgres.
• Strong technical knowledge: Unix/Linux; Database (Sybase/SQL/Oracle), Java, Python, Perl, Shell scripting, Infrastructure.
• Experience in Monitoring & Alerting, and Job Scheduling Systems.
• Being comfortable with frequent, incremental code testing and deployment.
• Strong grasp of automation / DevOps tools - Ansible, Jenkins, SVN, Bitbucket.

Responsibilities

• L1–L3 level support in Hadoop Platform.
• Monitor and resolve HDFS and Ozone related issues.
• Investigate cluster performance, storage utilization, and capacity concerns.
• Manage quota requests, storage allocation, and capacity planning.
• Support platform maintenance and integration activities.
• Coordinate with application teams during incident and problem management.
• Create and maintain operational documentation, runbooks, and knowledge articles.

Skills

Hadoop Platform

Linux

DevOps

YARN

Spark

Impala

Hive

HBase

Ozone

SQL

Databases

Python

Shell scripting

Java

Jenkins

Ansible

Communication

Education

Bachelor's degree in Computer Science

Tools

Autosys

Tableau

Hadoop Stack

Job description

Primary - Hadoop, Secondary Linux with Devops

Key Responsibilities

• L1, L2 & L3 level support in Hadoop Platform
• Monitor and resolve HDFS and Ozone related issues.
• Investigate cluster performance, storage utilization, and capacity concerns.
• Manage quota requests, storage allocation, and capacity planning.
• Support platform maintenance and perform integration activities.
• Coordinate with application teams during incident and problem management.
• Create and maintain operational documentation, runbooks, and knowledge articles.

Required Qualifications

• Experience on L1, L2 & L3 level support in Hadoop Platform.
• Experience in Yarn, Spark and Impala job debugging and troubleshooting.
• Experience in addressing issues around NameNode, HDFS Space, File/Folder Permission & backups.
• Understanding and experience in query performance tuning and resource utilization tuning.
• Technical knowledge on relational databases, data warehousing and SQL skills.
• Hadoop, Kafka, Spark, Impala, Hive, HBase, Ozone etc.
• Strong knowledge of Hadoop Architecture, HDFS, Hadoop Cluster and Hadoop Administrator's role.
• Intimate knowledge of fully integrated AD/Kerberos authentication.
• Experience setting up optimum cluster configurations.
• Debugging knowledge of YARN.
• Hands-on with analyzing various Hadoop log files, compression, encoding, & file formats.
• Expert level knowledge of Cloudera Hadoop components such as HDFS, Sentry, HBase, Kafka, Impala, SOLR, Hue, Spark, Hive, YARN, Zookeeper and Postgres.
• Strong technical knowledge: Unix/Linux; Database (Sybase/SQL/Oracle), Java, Python, Perl, Shell scripting, Infrastructure.
• Experience in Monitoring & Alerting, and Job Scheduling Systems.
• Being comfortable with frequent, incremental code testing and deployment.
• Strong grasp of automation / DevOps tools - Ansible, Jenkins, SVN, Bitbucket.

Tools Involved

• ETL tools: Hadoop Stack
• Job Scheduling tools: Autosys
• BI tools: Tableau; Tableau dashboard building connecting Hadoop-Hive Tables is equally preferred.

Required Skills

Job Type: Full Time

Job Category: IT

Job Description

Hadoop Support with Devops

Location - Plano, TX

Full Time

Must Have Technical/Functional Skills

• Primary Skill: Hadoop
• Secondary: Linux, DevOps
• Experience: Minimum10 years

Roles & Responsibilities

• L1, L2 & L3 level support in Hadoop Platform
• Monitor and resolve HDFS and Ozone related issues.
• Investigate cluster performance, storage utilization, and capacity concerns.
• Manage quota requests, storage allocation, and capacity planning.
• Support platform maintenance and perform integration activities.
• Coordinate with application teams during incident and problem management.
• Create and maintain operational documentation, runbooks, and knowledge articles.

Required Qualifications

• Experience on L1, L2 & L3 level support in Hadoop Platform.
• Experience in Yarn, Spark and Impala job debugging and troubleshooting.
• Experience in addressing issues around NameNode, HDFS Space, File/Folder Permission & backups.
• Understanding and experience in query performance tuning and resource utilization tuning.
• Technical knowledge on relational databases, data warehousing and SQL skills.
• Hadoop, Kafka, Spark, Impala, Hive, HBase, Ozone etc.
• Strong knowledge of Hadoop Architecture, HDFS, Hadoop Cluster and Hadoop Administrator's role.
• Intimate knowledge of fully integrated AD/Kerberos authentication.
• Experience setting up optimum cluster configurations.
• Debugging knowledge of YARN.
• Hands-on with analyzing various Hadoop log files, compression, encoding, & file formats.
• Expert level knowledge of Cloudera Hadoop components such as HDFS, Sentry, HBase, Kafka, Impala, SOLR, Hue, Spark, Hive, YARN, Zookeeper and Postgres.
• Strong technical knowledge: Unix/Linux; Database (Sybase/SQL/Oracle), Java, Python, Perl, Shell scripting, Infrastructure.
• Experience in Monitoring & Alerting, and Job Scheduling Systems.
• Being comfortable with frequent, incremental code testing and deployment.
• Strong grasp of automation / DevOps tools - Ansible, Jenkins, SVN, Bitbucket.

Tools Involved

• ETL tools: Hadoop Stack
• Job Scheduling tools: Autosys
• BI tools: Tableau; Tableau dashboard building connecting Hadoop-Hive Tables is equally preferred.

Required Skills

Hadoop Developer