Big Data Engineer: Spark, ETL & Data Quality

eNcloud Services LLC

Katy (TX)

On-site

USD 100,000 - 170,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

eNcloud Services LLC is seeking a data engineer to build scalable ETL pipelines and big data processing solutions. You will design data ingestion, transformation and curation processes using Spark, Hive, HDFS, and related tools.

The role requires strong SQL, data analysis skills, and experience with Git and CI/CD workflows. The position involves handling large data volumes across client locations in the United States, with travel as needed and a commitment to data quality and reliable pipeline

Qualifications

  • Bachelor’s degree in Computer Science, IT, Engineering or related field with at least 60 months of experience.
  • Travel and relocation to various client locations throughout the United States may be required.

Responsibilities

  • Build processes that support data transformation, data structures, metadata, dependency and workload management.
  • Build and implement data ingestion and curation processes developed using Big data tools such as Spark (Scala/Python), Hive, HDFS, Sqoop, HBase, Kerberos, Sentry and Impala.
  • Handle ingestion of large volumes of data from various platforms for analytics needs and write high-performance, reliable and maintainable ETL code.
  • Demonstrate strong SQL knowledge and data analysis skills for data anomaly detection and data quality assurance.
  • Write shell scripts along with complex SQL queries, Hadoop commands and use Git.
  • Create database schemas and Hive tables (external and managed) with various file formats (Orc, Parquet, Avro, Text, etc.).
  • Monitor performance of production jobs and advise on necessary infrastructure changes.
  • Work on code versioning using Bitbucket and CI/CD pipeline.

Skills

Data transformation
Data ingestion
Data quality assurance
SQL knowledge
Data analysis
CI/CD

Education

Bachelor's degree in Computer Science/IT/Engineering or related field

Tools

Spark
Hive
HDFS
Sqoop
HBase
Kerberos
Sentry
Impala
Bitbucket
Git
CI/CD pipeline

Job description

eNcloud Services LLC is seeking a data engineer to build scalable ETL pipelines and big data processing solutions. You will design data ingestion, transformation and curation processes using Spark, Hive, HDFS, and related tools.

The role requires strong SQL, data analysis skills, and experience with Git and CI/CD workflows. The position involves handling large data volumes across client locations in the United States, with travel as needed and a commitment to data quality and reliable pipeline

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer ETL
Data Engineer ETL

Compunnel Inc. • Durham (NC)

On-site
USD 100,000 - 130,000
Big Data Engineer: ETL, Spark & Data Pipelines
Big Data Engineer: ETL, Spark & Data Pipelines

Synechron • Charlotte (NC)

On-site
USD 100,000 - 110,000
Highly competitive compensation and benefits package
10 days of paid annual leave
Comprehensive insurance plan
+2
Cloud Data Engineer: Spark & Hive Data Pipelines
Cloud Data Engineer: Spark & Hive Data Pipelines

Tata Consultancy Services Limited • Irving (TX)

On-site
USD 90,000 - 110,000
Big Data Engineer: Cloud, Spark & Real-Time Pipelines
Big Data Engineer: Cloud, Spark & Real-Time Pipelines

EdgeQ Technologies Private Limited • Chicago (IL)

Hybrid
USD 85,000 - 160,000
Health insurance
Paid time off
Professional development
+1
Remote Data Engineer: Cloud & Big Data Pipelines
Remote Data Engineer: Cloud & Big Data Pipelines

TheHiveCareers • United States

Remote
USD 30,000 - 60,000
Big Data Engineer
Big Data Engineer

TechDigital Group • Jersey City (NJ)

On-site
USD 90,000 - 150,000
Senior Spark ETL Engineer for Enterprise Data Solutions
Senior Spark ETL Engineer for Enterprise Data Solutions

Compunnel Inc. • Durham (NC)

On-site
USD 100,000 - 130,000
Senior Data Engineer
Senior Data Engineer

Veriipro • Atlanta (GA)

On-site
USD 140,000 - 180,000
Data Engineer
Data Engineer

Tata Consultancy Services Limited • Irving (TX)

On-site
USD 125,000 - 140,000
Data Engineer
Data Engineer

Valiant Technologies LLC • Jersey City (NJ)

On-site
USD 110,000 - 150,000