Hadoop developer-6

Realign Llc; Charlotte (NC); On-site; USD 120,000 - 180,000; Full time

Job summary

Realign Llc is seeking a highly experienced Hadoop Spark Developer to design, develop, and maintain large-scale data processing solutions. The role emphasizes PySpark, Hive, Python, and SQL, with strong distributed-data experience.

You will work closely with data architects, engineers, and business stakeholders to optimize workloads, ensure data quality, and deliver scalable analytics capabilities across batch and streaming pipelines.

Qualifications

• 10+ years of experience with Big Data technologies and distributed processing.
• Bachelor's or Master's degree in a relevant field is required or preferred.
• Proven ability to design, develop, and optimize large-scale data solutions.

Responsibilities

• Design, develop, and optimize large-scale data processing solutions using Hadoop and Spark.
• Build and optimize ETL/ELT pipelines with PySpark, Hive, and Python.
• Develop high-performance Spark jobs and manage data ingestion frameworks.
• Collaborate with architects, data scientists, and DevOps teams.

Skills

PySpark

Hive

Python

SQL

Unix

Kafka

Java

Education

Bachelor's degree

Master's degree

Tools

Hadoop

Spark

Kafka

Hive

Job description

Charlotte, North Carolina 28105 Posted October 1st, 2026

Primary Skill: PySpark, Hive, Python, SQL

Secondary: Unix, Kafka, Java

Experience: 10+ years

We are seeking a highly experienced Hadoop Spark Developer with 10+ years of expertise in Big Data technologies, including PySpark, Hadoop Ecosystem, Hive, and Python. The ideal candidate will be responsible for designing, developing, optimizing, and maintaining large-scale data processing solutions.

Experience with Microsoft Copilot for AI-assisted development and productivity enhancement is highly desirable. The developer should hold a bachelor's or master's degree.

The candidate should possess strong analytical skills, hands‑on experience in distributed data processing, and the ability to work closely with business stakeholders, architects, and data engineering teams.

Required Skills

• Design, develop, and maintain scalable Big Data solutions using Hadoop and Spark.
• Build and optimize ETL/ELT pipelines using PySpark, Hive, and Python.
• Process and analyze large datasets in distributed environments.
• Develop high-performance Spark jobs and optimize existing workloads.
• Create and manage Hive tables, partitions, views, and complex queries.
• Implement data quality, data validation, and reconciliation frameworks.
• Perform code reviews and ensure adherence to coding standards and best practices.
• Utilize Microsoft Copilot to accelerate development, automate code generation, troubleshooting, documentation, and testing activities.
• Strong experience building both batch and real-time streaming applications with Kafka.
• Collaborate with Data Architects, Data Scientists, Business Analysts, and DevOps teams.
• Troubleshoot production issues and perform root cause analysis.
• Design data ingestion frameworks for structured, semi-structured, and unstructured data.
• Mentor junior developers and provide technical leadership.