Data Engineer - Java/Scala | Spark & AWS Pipelines

PINNACLE GROUP (Pinnacle Technical Resources); Wilmington, Northern (DE, KY); Hybrid; USD 103,000 - 110,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

401K

Job summary

PTR Global is seeking a Data Engineer with strong Java, Scala, and Spark expertise to build robust back-end data pipelines and govern data quality. The role emphasizes AWS pipeline tooling, event-driven processing, and Glue job management, with potential onsite work in Wilmington, DE and other East Coast locations in a hybrid setup.

The ideal candidate will design multi-stage data pipelines using S3, Glue, Step Functions, and Lambda, while validating data quality and managing schema drift and

Qualifications

• Back-end development with Java, Scala, Spark.
• Hands-on experience with AWS tools such as S3, Glue, Step Functions, Lambda.
• Strong understanding of event-based architecture and event-driven triggers.
• Experience with Glue job configuration, scaling, and addressing limitations.
• Knowledge of data quality validation, schema drift, and backfill handling.
• Ability to work in a hybrid environment with occasional onsite presence.

Responsibilities

• Develop and maintain back‑end systems using Java, Scala, and Spark.
• Design and implement AWS pipelines, including S3, Glue, Step Functions, and Lambda for multi‑stage data pipelines.
• Work with event‑based architecture, including EventBridge rules and S3 event notifications, and handle reruns from event‑based triggers.
• Configure and scale Glue jobs, addressing limitations such as Glue 4.0 Iceberg.
• Build and maintain data validation components to ensure data security and handle schema drift and backfill processes.
• Understand and manage state machine scheduling and downstream job synchronization using Step Functions.

Skills

Java

Scala

Spark

AWS

Glue

Step Functions

Lambda

Event‑driven architecture

Data quality validation

Tools

S3

Glue

Step Functions

Lambda

Job description

PTR Global is seeking a Data Engineer with strong Java, Scala, and Spark expertise to build robust back-end data pipelines and govern data quality. The role emphasizes AWS pipeline tooling, event-driven processing, and Glue job management, with potential onsite work in Wilmington, DE and other East Coast locations in a hybrid setup.

The ideal candidate will design multi-stage data pipelines using S3, Glue, Step Functions, and Lambda, while validating data quality and managing schema drift and