Sr. Data Engineer

Eliassen Group; McLean (VA); Hybrid; USD 96,000 - 110,000; Full time

Benefits offered by this job

Medical, Dental, and Vision benefits

401k with company matching

Life insurance

Job summary

Eliassen Group in McLean, VA (hybrid) is seeking a Data Engineer to support a large-scale migration and data-load initiative. The role will build and optimize AWS-based data pipelines using Java, Scala, Spark, PySpark, and AWS Glue across multi-stage workflows.

You'll implement event-driven architectures, ensure data quality and resilience, and contribute to CI/CD and Spring Boot repositories while collaborating with stakeholders across financial services clients.

Qualifications

• Hands-on Java with data engineering background.
• Proficiency with Scala, Spark and PySpark.
• Strong AWS experience (S3, Glue, Step Functions, Lambda, and EventBridge).
• Knowledge of data quality controls, schema evolution, and backfill management.
• CI/CD pipeline implementation and operations.

Responsibilities

• Design, build, and maintain AWS data pipelines using S3, Glue, Step Functions, and Lambda.
• Develop back-end data processing services in Java and Scala with Spark and PySpark.
• Implement event-based architectures leveraging EventBridge and S3 event notifications.
• Configure and scale AWS Glue jobs, addressing version and table format considerations.
• Establish data validation, security checks, schema drift handling, and backfill strategies.
• Integrate external API calls within data pipelines where required.
• Contribute to CI/CD pipelines and Spring Boot based repositories.
• Support production runbooks, reruns from event triggers, and downstream job synchronization.

Skills

Java

Python

Scala

Spark

PySpark

AWS

Event-driven

CI/CD

Spring Boot

Tools

AWS Glue

S3

Step Functions

Lambda

EventBridge

Spring Boot

Job description

Description:

May sit hybrid onsite in Wilmington, DE, McLean, VA, Richmond, VA or NYC

Our client seeks a Data Engineer professional with strong ability to support a large-scale customer migration and data load initiative. The role will build and optimize AWS-based data pipelines using Java, Scala, and Spark, with PySpark and AWS Glue across multi-stage workflows. The engagement will emphasize event-driven patterns, data quality, and operational resilience.

Due to client requirements, applicants must be willing and able to work on a w2 basis. For our w2 consultants, we offer a great benefits package that includes Medical, Dental, and Vision benefits, 401k with company matching, and life insurance.

Rate: $70.00 to $80.00/hr. w2

Responsibilities:

• Design, build, and maintain AWS data pipelines using S3, Glue, Step Functions, and Lambda.
• Develop back-end data processing services in Java and Scala with Spark and PySpark.
• Implement event-based architectures leveraging EventBridge and S3 event notifications.
• Configure and scale AWS Glue jobs, addressing version and table format considerations.
• Establish data validation, security checks, schema drift handling, and backfill strategies.
• Integrate external API calls within data pipelines where required.
• Contribute to CI/CD pipelines and Spring Boot based repositories.
• Support production runbooks, reruns from event triggers, and downstream job synchronization.

Experience Requirements:

• Hands-on Java with data engineering background. Python considered, Java preferred.
• Proficiency with Scala, Spark, and PySpark.
• Strong AWS experience: S3, Glue, Step Functions, Lambda, and EventBridge.
• Knowledge of state machine scheduling and multi-stage pipeline orchestration.
• Experience with data quality controls, schema evolution, and backfill management.
• CI/CD pipeline implementation and operations.
• Spring Boot repository familiarity.
• Plus: external API integration in pipelines and AI automation exposure.
• Prior experience supporting financial services data migrations preferred. Prior Capital One experience preferred.