Hadoop Engineer SME (Chicago, CLT)

The Matlen Silver Group, Inc.; Chicago (IL); On-site; USD 120,000 - 180,000; Full time

Job summary

The Matlen Silver Group, Inc. seeks a Hadoop Engineer SME to support NextGen Platforms built around Big Data technologies in Chicago.

The role involves managing Hadoop clusters, including Cloudera components, YARN, Impala, Kafka, and related tooling, with collaboration across clients, dev teams, and vendors. Candidates should have strong UNIX/Linux, Java, Python, SQL skills and experience with OpenShift, Docker, Ansible, and data science environments.

Qualifications

• Experience managing Hadoop/Data Science platforms end-to-end.
• Strong knowledge of Cloudera Hadoop components (HDFS, Hue, YARN, Impala).
• Proficient in UNIX/Linux, Java, Python, and SQL.
• Familiarity with containerization (Docker) and DevOps tools (Ansible, Jenkins).
• Ability to communicate complex technical solutions to clients and teams.

Responsibilities

• Provide SME leadership for Hadoop stack and data platform operations.
• Manage cluster upgrades, monitoring, and capacity planning across environments.
• Collaborate with clients, vendors, and project teams to deliver solutions.
• Troubleshoot Hadoop, Spark, and Docker-based deployments and performance issues.

Skills

Hadoop

Spark

Kafka

Impala

HBase

Docker

Ansible

YARN

Cloudera

Unix/Linux

Java

Python

SQL

OpenShift

Jupyter Notebook

Tableau

Tools

OpenShift

Jupyter Notebook

Autosys

Sentry

Hue

Postgres

Job description

Hadoop Engineer SME (Chicago, CLT) (BH-110632)

Location Chicago, United States Sector Banking

Hadoop Engineer (SME) role supporting NextGen Platforms built around Big Data Technologies (Hadoop, Spark, Kafka, Impala, HBase, Docker-Container, Ansible and many more). Requires experience in cluster management of vendor-based Hadoop and Data Science (AI/ML) products like Cloudera, Databricks. Hadoop Engineer is involved in the full life cycle of an application and part of an agile development process. They require the ability to interact, develop, engineer, and communicate collaboratively at the highest technical levels with clients, development teams, vendors, and other partners. The following section is intended to serve as a general guideline for each relative dimension of project complexity, responsibility, and education/experience within this role.

• - Works on complex, major or highly visible tasks in support of multiple projects that require multiple areas of expertise
• - Team member will be expected to provide subject matter expertise in managing Hadoop and Data Science Platform operations with focus around Cloudera Hadoop, Jupyter Notebook, OpenShift, Docker-Container Cluster Management and Administration
• - Integrates solutions with other applications and platforms outside the framework
• - He / She will be responsible for managing platform operations across all environments which includes upgrades, bug fixes, deployments, metrics / monitoring for resolution and forecasting, disaster recovery, incident / problem / capacity management
• - Serves as a liaison between client partners and vendors in coordination with project managers to provide technical solutions that address user needs
• Experience on L1, L2 L3 level support in Hadoop Platform
• Experience in Yarn , Spark and Impala job debugging and troubleshooting
• Experience in addressing issues around Name node , HDFS Space , File/Folder Permission backups
• Understanding and experience in experience in query performance tuning and resource utilization tuning
• Technical knowledge on relational databases, data warehousing and SQL skills
• Hadoop, Kafka, Spark, Impala, Hive, HBase, Ozone etc.
• Strong knowledge of Hadoop Architecture, HDFS, Hadoop Cluster and Hadoop Administrator's role
• Intimate knowledge of fully integrated AD/Kerberos authentication
• Experience setting up optimum cluster configurations
• Debugging knowledge of YARN.
• Hands-on with analyzing various Hadoop log files, compression, encoding, file formats
• Expert level knowledge of Cloudera Hadoop components such as HDFS, Sentry, HBase, Kafka, Impala, SOLR, Hue, Spark, Hive, YARN, Zookeeper and Postgres
• Strong technical knowledge: Unix/Linux; Database (Sybase/SQL/Oracle), Java, Python, Perl, Shell scripting, Infrastructure.
• Experience in Monitoring Alerting, and Job Scheduling Systems
• Being comfortable with frequent, incremental code testing and deployment
• Strong grasp of automation / DevOps tools - Ansible, Jenkins, SVN, Bitbucket

Tools Involved -

• ETL tools: Hadoop Stack
• Job Scheduling tools: Autosys
• BI tools: Tableau ; Tableau dashboard building connecting Hadoop-Hive Tables is equally preferred