Remote Data Engineer - AWS/PySpark Pipelines

Eliassen Group; Sacramento (CA); Remote; USD 69,000 - 79,000; Full time

Job summary

Eliassen Group is seeking a Data Engineer to design and implement high-performance data pipelines at scale in a remote setting for a Financial Services client. You will focus on reliable ingestion, transformation, data quality, and lineage across AWS services.

The role requires Python and PySpark expertise, hands-on work with AWS Glue and EMR, and experience building scalable pipelines to terabyte-scale data, with real-time processing using Kinesis or Kafka.

Qualifications

• Expert-level proficiency in Python and PySpark.
• Deep hands-on experience with AWS Glue and Amazon EMR.
• Proven track record building scalable pipelines for large-scale data, preferably at terabyte scale.
• Experience implementing frameworks for data lineage and data quality.
• Strong experience with S3-based data lakes and relational databases such as RDS, PostgreSQL, or MySQL.
• Experience with real-time processing using Kinesis or Kafka.
• Familiarity with CI/CD for data pipelines and Infrastructure as Code.

Responsibilities

• Design and implement high-performance data pipelines that handle large volumes of data.
• Ensure data integrity through advanced quality checks and lineage tracking.

Skills

Python

PySpark

AWS Glue

Amazon EMR

Data pipelines

Data lineage

Data quality

S3 data lakes

RDS

PostgreSQL

MySQL

Kinesis

Kafka

CI/CD

IaC

Tools

Python

PySpark

Job description

Eliassen Group is seeking a Data Engineer to design and implement high-performance data pipelines at scale in a remote setting for a Financial Services client. You will focus on reliable ingestion, transformation, data quality, and lineage across AWS services.

The role requires Python and PySpark expertise, hands-on work with AWS Glue and EMR, and experience building scalable pipelines to terabyte-scale data, with real-time processing using Kinesis or Kafka.