Senior Data Engineer — AWS Pipelines & Spark (Hybrid)

Eliassen Group; McLean (VA); Hybrid; USD 96,000 - 110,000; Full time

Benefits offered by this job

Medical, Dental, and Vision benefits

401k with company matching

Life insurance

Job summary

Eliassen Group in McLean, VA (hybrid) is seeking a Data Engineer to support a large-scale migration and data-load initiative. The role will build and optimize AWS-based data pipelines using Java, Scala, Spark, PySpark, and AWS Glue across multi-stage workflows.

You'll implement event-driven architectures, ensure data quality and resilience, and contribute to CI/CD and Spring Boot repositories while collaborating with stakeholders across financial services clients.

Qualifications

• Hands-on Java with data engineering background.
• Proficiency with Scala, Spark and PySpark.
• Strong AWS experience (S3, Glue, Step Functions, Lambda, and EventBridge).
• Knowledge of data quality controls, schema evolution, and backfill management.
• CI/CD pipeline implementation and operations.

Responsibilities

• Design, build, and maintain AWS data pipelines using S3, Glue, Step Functions, and Lambda.
• Develop back-end data processing services in Java and Scala with Spark and PySpark.
• Implement event-based architectures leveraging EventBridge and S3 event notifications.
• Configure and scale AWS Glue jobs, addressing version and table format considerations.
• Establish data validation, security checks, schema drift handling, and backfill strategies.
• Integrate external API calls within data pipelines where required.
• Contribute to CI/CD pipelines and Spring Boot based repositories.
• Support production runbooks, reruns from event triggers, and downstream job synchronization.

Skills

Java

Python

Scala

Spark

PySpark

AWS

Event-driven

CI/CD

Spring Boot

Tools

AWS Glue

S3

Step Functions

Lambda

EventBridge

Spring Boot

Job description

Eliassen Group in McLean, VA (hybrid) is seeking a Data Engineer to support a large-scale migration and data-load initiative. The role will build and optimize AWS-based data pipelines using Java, Scala, Spark, PySpark, and AWS Glue across multi-stage workflows.

You'll implement event-driven architectures, ensure data quality and resilience, and contribute to CI/CD and Spring Boot repositories while collaborating with stakeholders across financial services clients.