Senior Spark ETL Engineer for Enterprise Data Solutions

Compunnel, Inc.; Durham (NC); On-site; USD 100,000 - 130,000; Full time

Job summary

Compunnel, Inc. is seeking a highly motivated Data Engineer (ETL) to design, develop, and maintain scalable data engineering solutions. The role includes building high-performance ETL pipelines, developing applications using Apache Spark, and processing large-scale datasets with cloud-native technologies.

Ideal candidates have strong expertise in Spark, Scala, and Java, while collaborating with cross-functional teams. The company values innovation and provides an environment for continuous improvement in data engineering practices.

Qualifications

• 10+ years of experience developing Spark services for large-scale data movement.
• Experience scheduling, monitoring, debugging, and optimizing ETL Spark batch processes.
• Hands-on experience developing Java clients for REST and SOAP API integrations.

Responsibilities

• Design, develop, and maintain scalable ETL pipelines for enterprise data integration.
• Develop, test, deploy, and support ETL batch jobs using Apache Spark.
• Collaborate with cross-functional teams to design innovative data solutions.

Skills

Apache Spark

Scala

Java

ETL processes

Cloud platforms

Big data technologies

Agile methodologies

Education

Bachelor's degree in Computer Science or related field

Tools

Apache Spark

Amazon EMR

NoSQL databases (Cassandra/Yugabyte)

Snowflake

REST/SOAP APIs

Parquet/HDF5

Job description

Compunnel, Inc. is seeking a highly motivated Data Engineer (ETL) to design, develop, and maintain scalable data engineering solutions. The role includes building high-performance ETL pipelines, developing applications using Apache Spark, and processing large-scale datasets with cloud-native technologies.

Ideal candidates have strong expertise in Spark, Scala, and Java, while collaborating with cross-functional teams. The company values innovation and provides an environment for continuous improvement in data engineering practices.