Hadoop Data engineer

ETEAM Inc.

Maryland Heights (MO)

Hybrid

USD 120,000 - 180,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

ETEAM Inc. is seeking a Hadoop Data Engineer to design and optimize large-scale data pipelines.

The role focuses on Spark/Scala/Python development, Java, and Kafka, with a strong emphasis on AWS S3 and Iceberg-based lakehouse architectures. You will work in a hybrid environment in Maryland Heights, collaborating with architects, analysts, and platform engineers to translate business needs into scalable data solutions.

Qualifications

  • Must have hands-on experience with Spark (Scala/Python) and core Java.
  • Strong SQL skills including CDC concepts across Oracle/SQL Server is expected.
  • Proficiency with Unix shell scripting and Python for automation.

Responsibilities

  • Design, build, and optimize scalable data pipelines (batch and real-time) using Spark, Java, and Kafka.
  • Maintain data lakehouse architectures on AWS and Cloudera, utilizing S3, Iceberg, and Hive.
  • Migrate on-prem Hadoop workloads to cloud-native architectures and ensure governance/compliance.

Skills

Scala/Python (Spark)
Core Java
Unix scripting
Python
SQL

Education

Bachelor's degree in CS/Engineering or related field

Tools

Apache Spark
Apache Kafka
Hadoop
AWS S3
Iceberg
Hive

Job description

Job Description

Job DescriptionJob Title: Hadoop Data engineer

Work Location: Maryland Heights MO 63043 (hybrid )

Contract duration: 12

F2F interview - Yes

Job Details

Must Have skills: AWS, Hadoop

Key Responsibilities
  • Design, build, and optimize scalable batch and real-time data pipelines using Apache Spark (Scala/Python), Core Java, and Apache Kafka to process high-volume data.
  • Manage and maintain enterprise data Lakehouse architectures spanning AWS and Cloudera environments, leveraging S3, Apache Iceberg, and Hive for scalable storage and querying.
  • Implement Change Data Capture (CDC) and write complex SQL to integrate data from relational databases (Oracle, SQL Server) into dimensional data warehouses.
  • Automate and orchestrate pipeline workflows using Unix shell scripting and Python.
  • Lead and support migration of legacy Hadoop workloads to AWS cloud-native architectures.
  • Monitor, troubleshoot, and tune JVM and Spark job performance to ensure reliability and efficiency at scale.
  • Ensure all data pipelines and storage practices adhere to regulatory compliance and data governance standards.
  • Collaborate with cross-functional teams (data architects, analysts, platform engineers) to translate business requirements into robust data solutions.
Required Skills & Technology
  • Strong programming experience in Scala/Python (Spark) and Core Java
  • Hands-on experience with Apache Kafka for streaming data pipelines
  • Working knowledge of AWS services (S3) and Cloudera platform tools (Hive, Iceberg)
  • Strong SQL skills, including CDC concepts, across Oracle and SQL Server
  • Experience with dimensional data warehouse modeling
  • Proficiency in Unix shell scripting and Python for automation
  • Experience migrating on-prem Hadoop workloads to cloud platforms
  • Understanding of JVM internals and Spark performance tuning techniques
Preferred Qualifications
  • Prior experience in the Communications/Telecom domain (CDRs, network telemetry data)
  • Familiarity with data governance and regulatory compliance frameworks
  • Bachelor s degree in computer science, Engineering, or a related field, or equivalent practical experience
Top 3 responsibilities
  • Manage and maintain enterprise data lakehouse architectures spanning AWS and Cloudera environments, leveraging S3, Apache Iceberg, and Hive for scalable storage and querying. Implement Change Data Capture CDC and write complex SQL to integrate data from relational databases Oracle, SQL Server into dimensional data warehouses.
  • Automate and orchestrate pipeline workflows using Unix shell scripting and Python. Lead and support migration of legacy Hadoop workloads to AWS cloudnative architectures.
  • M
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Hadoop Developer
Hadoop Developer

BinTech Group • Atlanta (GA)

Hybrid
USD 110,000 - 160,000
Hadoop / Bigdata Developer
Hadoop / Bigdata Developer

Envision Technology Solutions • Charlotte (NC)

On-site
USD 140,000 - 180,000
AWS Data Engineer
AWS Data Engineer

Tata Consultancy Services Limited • Irving (TX)

On-site
USD 90,000 - 110,000
Hadoop Developer
Hadoop Developer

Tata Consultancy Services Limited • Charlotte (NC)

On-site
USD 100,000 - 120,000
Discretionary Annual Incentive
Medical Coverage: Medical & Health, 9/
Dental & Vision
+3
Data Engineer
Data Engineer

Tata Consultancy Services Limited • Irving (TX)

On-site
USD 125,000 - 140,000
Sr. Hadoop Data Engineer
Sr. Hadoop Data Engineer

Sonoma Consulting Inc. • St. Louis (MO)

On-site
USD 100,000 - 130,000
Hadoop Developer
Hadoop Developer

Quantum Technologies LLC • Bentonville (AR)

On-site
USD 110,000 - 150,000
Hadoop Developer
Hadoop Developer

Veriipro • Addison (TX)

On-site
USD 120,000 - 150,000
Hadoop Developer
Hadoop Developer

Robotic Process Automation LLC • Arizona

On-site
USD 120,000 - 160,000
Hadoop Developer
Hadoop Developer

Valiant Technologies LLC • Bentonville (AR)

On-site
USD 110,000 - 150,000