Senior Data Engineer, AWS Infrastructure Supply Chain Automation

D8 LooP; Seattle (WA); On-site; USD 150,000 - 190,000; Full time

Job summary

D8 LooP is seeking a Senior Data Engineer to design, implement, and operate AWS-based data pipelines for its infrastructure and data lake. The role combines data modeling, warehousing, and streaming technologies to support scalable data warehousing and analytics.

You will work on a serverless architecture using Redshift Spectrum, Athena, S3, Lambda, Glue, EMR, Kinesis, and CloudWatch, ensuring data quality, governance, and security across the data platform.

Qualifications

• Experience with data modeling, warehousing and ETL pipelines.
• Proficiency in Python and/or Java/Scala/NodeJS.
• SQL expertise required.
• Experience with Spark and big data tech (Hadoop/Hive/Spark/EMR).
• Experience operating large data warehouses.
• Familiarity with AWS data services (S3/Glue/Lambda/SNS/SQS/KMS).
• Knowledge of data lake/warehouse governance and security.

Responsibilities

• Develop and maintain automated ETL pipelines with monitoring using Python, Spark, SQL and AWS services.
• Build and grow a serverless data lake/warehousing stack (Redshift Spectrum, Athena, S3, Lambda, Glue, EMR, Kinesis).
• Collaborate across teams to ensure data quality, performance and cost efficiency.

Skills

Data modeling

ETL pipelines

Python

SQL

Spark

Big data

Warehousing

Tools

S3

Glue

Lambda

SNS

SQS

KMS

Redshift Spectrum

Athena

EMR

Kinesis

CloudWatch

Job description

Senior Data Engineer, AWS Infrastructure Supply Chain Automation

Seattle, Washington, USA Senior 5+ yrs Data Engineer posted 1 d ago

AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we’re the people who keep the cloud running. We support all AWS data centers and all of the servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they rely on.

Skills, with evidence

• Experience with data modeling, warehousing and building ETL pipelines must have
• Python Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS must have
• SQL Experience with SQL must have
• Spark Experience with big data technologies such as: Hadoop, Hive, Spark, EMR must have
• Warehousing Experience operating large data warehouses must have
• AWS AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. not practised here
• Cost & performance You’ll collaborate with people across AWS to help us deliver the highest standards for safety and security while providing seemingly infinite capacity at the lowest possible cost for our customers.
• - Develop and maintain automated ETL pipelines (with monitoring) using scripting languages such as Python, Spark, SQL and AWS services such as S3, Glue, Lambda, SNS, SQS, KMS.
• Streaming We’re looking for Senior Data Engineer to help us grow our Data Lake and Data Warehouse Systems, which is being built using a serverless architecture, with 100% native AWS components including Redshift Spectrum, Athena, S3, Lambda, Glue, EMR, Kinesis, SNS, CloudWatch and more!
• Governance & security - Develop and maintain data warehouse and data lake metadata, data catalog, and user documentation for internal business customers. not practised here
• Java - Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS not practised here
• Scala - Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS not practised here