Onsite Hadoop Data Engineer — Real-Time & Batch Pipelines

Collabera; Charlotte (NC); On-site; USD 82,656 - 89,544; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

401(k) retirement plan

Paid time off

Paid sick leave

Job summary

Collabera is seeking an experienced Hadoop Data Engineer in Charlotte, NC. The ideal candidate will excel in designing and maintaining data pipelines with expertise in Hadoop, Spark, and SQL. Responsibilities include data ingestion from various sources and processing large datasets. Compensation ranges from $60 to $65 per hour, with benefits including medical, dental, and 401(k). This is an onsite role with a potential transition to full-time.

Qualifications

• Strong hands-on experience with Hadoop and Big Data ecosystems.
• Expertise in Spark Structured Streaming and Apache Spark.
• Strong SQL skills with Hive, Impala, MySQL, or Spark SQL.

Responsibilities

• Design, develop, and maintain batch and near real-time data pipelines using Spark.
• Ingest data from multiple sources including Kafka, REST APIs, and databases.
• Transform, validate, and process large datasets using Hive and Impala.

Skills

Hadoop and Big Data ecosystems

Spark Structured Streaming

SQL skills (Hive, Impala, MySQL, Spark SQL)

Kafka

Scala

Distributed systems experience

Tools

Cloudera/Hortonworks

Elastic Search

Kibana

Job description

Collabera is seeking an experienced Hadoop Data Engineer in Charlotte, NC. The ideal candidate will excel in designing and maintaining data pipelines with expertise in Hadoop, Spark, and SQL. Responsibilities include data ingestion from various sources and processing large datasets. Compensation ranges from $60 to $65 per hour, with benefits including medical, dental, and 401(k). This is an onsite role with a potential transition to full-time.