Big Data Data Engineer - Spark/Scala & Hadoop

Computer Task Group; Piscataway Township (NJ); On-site; USD 69,000 - 83,000; Full time

Job summary

CTG is seeking a Data Engineer – Big Data for a 12‑month assignment in Piscataway, NJ. The role focuses on designing and implementing scalable data-processing pipelines using Spark/Scala and Hadoop-based tools, with strong SQL and Python capabilities.

You will collaborate with developers, analysts, and stakeholders in an Agile environment, contribute to API development, and ensure data quality and integrity across enterprise platforms. Banking or financial services experience is a plus.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent combination of education and experience.

Responsibilities

• Design, develop, test, and maintain scalable data-processing applications using Apache Spark and Scala.
• Develop and support Big Data applications and data pipelines utilizing Hadoop, Hive, and Impala.
• Write complex SQL queries for data extraction, transformation, reconciliation, analysis, and performance optimization.
• Develop Python scripts and utilities to support data processing, automation, and data engineering activities.
• Design, build, and maintain scalable ETL/data pipelines for large-volume datasets.
• Analyze, troubleshoot, and debug complex application and data-processing code.
• Support data quality, validation, reconciliation, and integrity across enterprise data platforms.
• Collaborate with application developers, data engineers, analysts, and business stakeholders in an Agile environment.
• Contribute to API development and application solutions supporting Big Data platforms and analytics.
• Apply knowledge of analytics libraries, open-source technologies, statistical computing, and Big Data processing frameworks as appropriate.

Skills

Spark/Scala

Hadoop

Hive

Impala

SQL

Python

API development

CI/CD

Data quality

Agile

Education

Bachelor's degree

Job description

CTG is seeking a Data Engineer – Big Data for a 12‑month assignment in Piscataway, NJ. The role focuses on designing and implementing scalable data-processing pipelines using Spark/Scala and Hadoop-based tools, with strong SQL and Python capabilities.

You will collaborate with developers, analysts, and stakeholders in an Agile environment, contribute to API development, and ensure data quality and integrity across enterprise platforms. Banking or financial services experience is a plus.