Big Data Engineer

Princeton IT Services, Inc; Mount Laurel Township (NJ); On-site; USD 120,000 - 150,000; Full time

Job summary

A technology services company in Mt. Laurel, NJ is seeking a skilled Big Data Engineer with over 10 years of experience. The role involves designing and optimizing data pipelines using frameworks like Spark and Flink, with a strong emphasis on AWS cloud services. The ideal candidate will possess strong programming skills in Java, Python, and Scala, and have expertise in real-time data streaming technologies like Kafka. This position offers the opportunity to work in a collaborative Agile environment.

Qualifications

• 10+ years of experience as a Data Engineer or Big Data Engineer.
• Hands-on expertise in cloud platforms, preferably AWS.
• Strong experience with Big Data tools including Flink and Hadoop.

Responsibilities

• Design, develop, and optimize scalable data pipelines.
• Work with streaming frameworks to handle large event-driven data flows.
• Implement data lake and data warehouse solutions on AWS.

Skills

Spark (PySpark, Spark SQL, Spark Streaming)

Java

Python

Scala

Kafka

AWS

CI/CD tools (Jenkins, Git, Bitbucket, Terraform, Docker, Kubernetes)

SQL

NoSQL databases

Data quality and governance

Education

Bachelor's degree in Computer Science, Engineering, or related field

Job description

Overview

We are seeking a highly skilled Big Data Engineer to join our team in Mt. Laurel, NJ. The ideal candidate will have hands-on experience with Big Data frameworks (Spark, Flink, Databricks, Hadoop ecosystem), streaming technologies (Kafka, Kinesis, Flink Streaming), and Cloud platforms (AWS preferred). The role involves designing and optimizing data pipelines, enabling large-scale data ingestion, transformation, and analytics to support enterprise applications and reporting.

Responsibilities

• Design, develop, and optimize scalable data pipelines for batch and real-time processing.


• Work with streaming frameworks (Flink, Spark Streaming, Kafka) to handle large event-driven data flows.


• Build and maintain ETL/ELT processes using tools like Apache Airflow, AWS Glue, and EMR.


• Implement data lake and data warehouse solutions on AWS (S3, Redshift, DynamoDB, Athena, EMR, Lambda, Step Functions, Kinesis, MSK).


• Collaborate with data scientists, analysts, and application developers to deliver curated data sets for analytics and reporting.


• Ensure data quality, governance, and security standards across systems.


• Work with Java, Python, and Scala for data transformation, API integration, and microservices.


• Participate in Agile ceremonies and contribute to sprint planning, demos, and retrospectives.


• Troubleshoot and optimize performance of big data pipelines in production environments.

Required Skills & Experience

• Bachelor’s degree in Computer Science, Engineering, or related field.


• 10+ years of experience as a Data Engineer or Big Data Engineer.


• Strong experience with Big Data tools: Spark (PySpark, Spark SQL, Spark Streaming), Flink, Hadoop ecosystem (Hive, HDFS, HBase, Sqoop).


• Hands-on expertise in cloud platforms (preferably AWS – EMR, Glue, S3, Kinesis, Lambda, Step Functions, DynamoDB).


• Strong programming skills in Java, Python, or Scala.


• Experience with Kafka (MSK or equivalent) for real-time data streaming.


• Proficiency with SQL and NoSQL databases (Redshift, Oracle, MongoDB, Cassandra, DynamoDB).


• Knowledge of CI/CD tools (Jenkins, Git, Bitbucket, Terraform, Docker, Kubernetes).


• Strong problem-solving, analytical, and communication skills.