Data Engineer – Big Data (Spark/Scala, Hadoop, SQL, Python)

Computer Task Group; Piscataway Township (NJ); On-site; USD 69,000 - 83,000; Full time

Job summary

CTG is seeking a Data Engineer – Big Data for a 12‑month assignment in Piscataway, NJ. The role focuses on designing and implementing scalable data-processing pipelines using Spark/Scala and Hadoop-based tools, with strong SQL and Python capabilities.

You will collaborate with developers, analysts, and stakeholders in an Agile environment, contribute to API development, and ensure data quality and integrity across enterprise platforms. Banking or financial services experience is a plus.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent combination of education and experience.

Responsibilities

• Design, develop, test, and maintain scalable data-processing applications using Apache Spark and Scala.
• Develop and support Big Data applications and data pipelines utilizing Hadoop, Hive, and Impala.
• Write complex SQL queries for data extraction, transformation, reconciliation, analysis, and performance optimization.
• Develop Python scripts and utilities to support data processing, automation, and data engineering activities.
• Design, build, and maintain scalable ETL/data pipelines for large-volume datasets.
• Analyze, troubleshoot, and debug complex application and data-processing code.
• Support data quality, validation, reconciliation, and integrity across enterprise data platforms.
• Collaborate with application developers, data engineers, analysts, and business stakeholders in an Agile environment.
• Contribute to API development and application solutions supporting Big Data platforms and analytics.
• Apply knowledge of analytics libraries, open-source technologies, statistical computing, and Big Data processing frameworks as appropriate.

Skills

Spark/Scala

Hadoop

Hive

Impala

SQL

Python

API development

CI/CD

Data quality

Agile

Education

Bachelor's degree

Job description

CTG is seeking to fill a Data Engineer – Big Data (Spark/Scala, Hadoop, SQL, Python) position for our client.

Location: Piscataway, NJ

Duration: 12 months

Duties:

• Design, develop, test, and maintain scalable data-processing applications using Apache Spark and Scala.

• Develop and support Big Data applications and data pipelines utilizing Hadoop, Hive, and Impala.

• Write complex SQL queries for data extraction, transformation, reconciliation, analysis, and performance optimization.

• Develop Python scripts and utilities to support data processing, automation, and data engineering activities.

• Design, build, and maintain scalable ETL/data pipelines for large-volume datasets.

• Analyze, troubleshoot, and debug complex application and data-processing code.

• Support data quality, validation, reconciliation, and integrity across enterprise data platforms.

• Collaborate with application developers, data engineers, analysts, and business stakeholders in an Agile environment.

• Contribute to API development and application solutions supporting Big Data platforms and analytics.

• Apply knowledge of analytics libraries, open-source technologies, statistical computing, and Big Data processing frameworks as appropriate.

Skills:

• Strong hands-on experience with Apache Spark and Scala for distributed data processing.

• Strong experience with Hadoop, Hive, and Impala and enterprise Big Data platforms.

• Advanced SQL skills, including complex queries, joins, transformations, reconciliation, and performance tuning.

• Strong Python scripting and data-processing experience.

• Strong application development, coding, debugging, and problem-solving skills.

• Knowledge of ETL/data pipelines, data quality, and data reconciliation.

• Understanding of analytics libraries, statistical computing, and open-source data-processing technologies.

• Experience with Agile development methodologies and CI/CD practices is preferred.

Experience:

• Demonstrated experience developing and supporting applications in enterprise Big Data environments.

• Hands-on experience working with large datasets and distributed data-processing technologies.

• Experience designing, developing, testing, and debugging complex code.

• Experience with large-volume transactional or financial datasets is preferred.

• Banking or financial services experience is preferred.

• Experience supporting production data applications and resolving technical issues is a plus.

Education:

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent combination of education and experience.

Excellent verbal and written English communication skills and the ability to interact professionally with a diverse group are required.

CTG does not accept unsolicited resumes from headhunters, recruitment agencies, or fee based recruitment services for this role.

The expected base salary for this position ranges from $50.00 to $60.00/hour. Salary offers are based on a wide range of factors including relevant skills, training, experience, education, market factors, and where applicable, licensure or certifications obtained. In addition to salary, a competitive benefit package is also offered.

About CTG

CTG, a Cegeka company, delivers IT and business solutions that enhance clients’ digital agility, empowering them to seize new opportunities and overcome any challenge. Backed by more than 60 years’ experience and a commitment to being a reliable, results-driven partner, we work shoulder to shoulder with clients to shape digital together. Our vision is to be an indispensable partner to our clients and the preferred career destination for digital and technology experts. With more than 9,000 team members in over 15 countries, we combine global expertise with local insight to deliver innovative solutions. We operate across the Americas, Europe, and India, working with over 3,000 clients in many of today's highest-growth industries.

Together, we shape what’s next—working shoulder to shoulder to deliver impactful solutions for our clients and society. Our culture is built by the people who work at CTG, the values we hold, and the actions we take. It's a living, breathing thing that is renewed every day through the ways we engage with each other, our clients, and our communities. At CTG, you’ll find a workplace where you are encouraged to grow, supported in your ambitions, and empowered to shape your own career journey. For more information, visit www.ctg.com .

CTG will consider for employment all qualified applicants including those with criminal histories in a manner consistent with the requirements of all applicable local, state, and federal laws.

CTG is an Equal Opportunity Employer. CTG will assure equal opportunity and consideration to all applicants and employees in recruitment, selection, placement, training, benefits, compensation, promotion, transfer, and release of individuals without regard to race, creed, religion, color, national origin, sex, sexual orientation, gender identity and gender expression, age, disability, marital or veteran status, citizenship status, or any other discriminatory factors as required by law. CTG is fully committed to promoting employment opportunities for members of protected classes.