Lead Hadoop Infrastructure Engineer - Remote

Pearson Carter; United States; Remote; USD 100,000 - 125,000; Full time

Job summary

A leading company is seeking a highly skilled Hadoop Infrastructure Lead Engineer to join their team. The ideal candidate will have extensive experience in migrating data centers, upgrading driver versions, and implementing a lift & shift approach. This role involves collaborating with teams to resolve issues and standardizing processes during migration. Proficiency in Hive, Impala, Spark, and PySpark is essential.

Benefits

Pension plan

Medical insurance

Vision insurance

Paid maternity leave

401(k)

Qualifications

• Proven experience in migrating data centers for Hadoop infrastructure.
• Strong knowledge of Hadoop ecosystem, including Hive/Impala, Spark, and PySpark.

Responsibilities

• Lead the migration of data centers for multiple physical Hadoop servers.
• Collaborate with cross-functional teams to ensure successful migration.
• Document and standardize support processes throughout the migration.

Skills

Hadoop

Hive

Impala

Spark

PySpark

Problem Solving

Analytical Skills

Job description

This range is provided by Pearson Carter. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

Base pay range

$60.00/hr - $80.00/hr

Direct message the job poster from Pearson Carter

Job Title: Hadoop Infrastructure Lead Engineer

Job Summary: We are seeking a highly skilled Hadoop Infrastructure Subject Matter Expert to join our client team. The ideal candidate will have extensive experience in migrating data centers for multiple physical Hadoop servers, upgrading driver versions, and implementing a lift & shift approach. This role involves replacing hardware with fully supported configurations, documenting processes, and standardizing support throughout the migration to the target data center. Proficiency in Hive/Impala, Spark, and PySpark is essential.

Key Responsibilities:

• Lead the migration of data centers for multiple physical Hadoop servers.
• Work closely with data center migration team and project manager.
• Provide on-time infrastructure assessments, analysis, impacts, and mitigate challenges & risks.
• Collaborate with data center migration team to resolve bugs, defects, and support infrastructure issues.
• Upgrade drivers to supported versions and ensure compatibility.
• Implement a lift & shift approach for seamless migration.
• Replace hardware with fully supported configurations.
• Document and standardize support processes throughout the migration.
• Collaborate with cross-functional teams to ensure successful migration.
• Provide expertise in Hive/Impala, Spark, and PySpark.

Qualifications:

• Proven experience in migrating data centers for Hadoop infrastructure.
• Strong knowledge of Hadoop ecosystem, including Hive/Impala, Spark, and PySpark.
• Experience with upgrading drivers and implementing lift & shift migrations.
• Ability to replace hardware and ensure fully supported configurations.
• Excellent documentation and process standardization skills.
• Strong problem-solving and analytical skills.
• Ability to work collaboratively with cross-functional teams.

Preferred Qualifications:

• Experience in IT operations and infrastructure management.
• Familiarity with cloud-based Hadoop solutions.
• Strong communication and leadership skills.

Seniority level

• Mid-Senior level

Employment type

• Full-time

Job function

• Information Technology

Industries

• IT Services and IT Consulting

Referrals increase your chances of interviewing at Pearson Carter by 2x.

Inferred from the description for this job

Pension plan

Medical insurance

Vision insurance

Paid maternity leave

401(k)

Get notified when a new job is posted.

Sign in to set job alerts for “Infrastructure Engineer” roles.

San Francisco, CA $93,000.00-$104,000.00 9 hours ago

Seattle, WA $85,000.00-$95,000.00 3 weeks ago

San Francisco, CA $150,000.00-$250,000.00 1 year ago

Denver, CO $85,000.00-$95,000.00 1 day ago

Austin, TX $85,000.00-$95,000.00 1 day ago

Boston, MA $85,000.00-$95,000.00 9 hours ago

United States $72.00-$82,000.00 1 month ago

Infrastructure Engineer - Dragonfly Portfolio

Site Reliability Engineer (SRE) - Platform Infrastructure team (100% Remote - USA)

Infrastructure Engineer: Deployments (Early career level)

United States $100,000.00-$100,000.00 3 months ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.