AWS Data Engineer

System One, Inc.; McLean (VA); Remote; USD 150,000 - 170,000; Full time

Benefits offered by this job

Health benefits

401(k) plan

Remote work

Job summary

System One, Inc. seeks an experienced AWS Data Engineer for a remote contract-to-hire role.

The candidate will build and operate batch and streaming data pipelines, design ETL/ELT processes with Python and PySpark, and implement CDC, data contracts, and governance. You will develop a scalable lakehouse on Amazon S3 with Iceberg, enable fast analytics with Athena/EMR/Glue/Redshift, and ensure reliability and cost efficiency across environments.

Qualifications

• Bachelor's degree in Engineering, IT, CS, Data Engineering or equivalent practical experience.
• 6+ years of hands-on data engineering experience.
• Extensive experience with AWS-native data lake/lakehouse architectures (S3, Glue, Athena, EMR, Lake Formation, Redshift).
• Proven ability to develop production ETL/ELT pipelines using Python and PySpark.
• Hands-on experience with Apache Iceberg (ACID, snapshots, schema evolution).
• Advanced SQL skills for analytical workloads.
• Experience with data governance, cataloging, lineage, access controls.
• IaC-driven provisioning and multi-environment platforms.
• CI/CD for data workflows with testing and rollback.

Responsibilities

• Build and operate data pipelines (batch and streaming) from APIs, databases, file drops, streams, and partners.
• Design and optimize ETL/ELT pipelines with Python and PySpark for analytics-ready datasets.
• Implement CDC, data contracts, schema validation, and reusable transforms.
• Improve reliability via testing, orchestration, monitoring, retries and runbooks.
• Create scalable lakehouse on S3 using Apache Iceberg and Parquet formats.
• Enable fast querying with AWS services (Athena, EMR, Glue, Redshift).
• Optimize partitioning, file sizing, caching, lifecycle policies for cost efficiency.
• Establish environments with consistent config across stages; promote safely.
• Enforce data governance, cataloging, lineage, ownership, and access controls.

Skills

Python

PySpark

AWS

ETL/ELT

SQL

CI/CD

IaC

Iceberg

Data lake

Data governance

Education

Bachelor's degree or 4 years experience

Tools

AWS S3

Athena

EMR

Glue

Redshift

Iceberg

Parquet

IAM

Job description

Job Title: AWS Data Engineer
Location: McLean, VA
Type: Contract-to-Hire
Compensation: $150,000.00 - $170,000.00
Work Model: Remote - offsite
Security Clearance: US Citizen with the ability to obtain a public trust

System One is looking for a strong AWS Data Engineer for a remote contract-to-hire position. While this position is remote, preference is given to those who are located in the Washington, DC Metro in case the team needs to meet in person.

Responsibilities

• Build and operate data pipelines (batch and streaming) from various sources including APIs, relational databases, file drops, event streams, and external partners.
• Design, implement, and optimize ETL/ELT pipelines using Python and PySpark to produce analytics-ready datasets for reporting, visualization, and machine learning.
• Implement incremental processing, change data capture (CDC), data contracts, schema validation, and reusable transformation frameworks.
• Improve pipeline reliability through automated testing, orchestration, monitoring, retry handling, and operational runbooks.
• Build and manage a scalable lakehouse on Amazon S3 using Apache Iceberg and open columnar formats such as Apache Parquet.
• Implement SQL-like table reliability features including ACID transactions, schema evolution, partition evolution, snapshot isolation, time travel, and rollback capabilities.
• Enable fast, interactive querying of lakehouse data using AWS-native query and compute services such as Amazon Athena, Amazon EMR, AWS Glue, and Amazon Redshift.
• Optimize performance and cost efficiency through partitioning, compaction, file sizing, caching, lifecycle policies, and efficient compute/storage separation.
• Establish standardized development, testing, and production environments with consistent configuration and controlled promotion across stages.
• Implement data governance and fine-grained access control utilizing AWS-native services like AWS Lake Formation, AWS Glue Data Catalog, IAM, KMS, and related security tools.
• Create a managed metadata repository for dataset cataloging, ownership, tagging, classification, and discoverability.
• Support end-to-end data lineage for source, transformation, and consumption to facilitate auditability and impact analysis.
• Apply security policies such as least privilege access, data classification, encryption, retention, and secure data handling.
• Build operational data quality checks for metrics such as freshness, completeness, validity, and anomaly detection, along with publishing SLAs/SLOs.
• Implement automated AWS provisioning through Infrastructure as Code (IaC) to ensure consistent, secure environments.
• Enhance CI/CD pipelines for data workflows and lakehouse components, including automated testing, security validation, packaging, deployment, promotion, and rollback.
• Maintain observability with centralized metrics, logs, traces, alerts, dashboards, runbooks, and incident response procedures.
• Continually evaluate platform performance, scalability, reliability, security, and cost, and implement measurable improvements.
• Collaborate closely with data, application, analytics, AI/ML, security, networking, and cloud platform teams to support mission-critical requirements.
• Maintain high-quality documentation including architecture diagrams, SOPs, data models, interface specs, and operational runbooks.
• Present technical findings, trade-offs, risks, and recommendations clearly to stakeholders.

Requirements

• Bachelor's degree in Engineering, Information Technology, Computer Science, Data Engineering, or a related field, or four (4) years of equivalent practical experience.
• Six (6) years of relevant hands‑on experience in data engineering.
• Extensive experience designing, implementing, and operating AWS-native data lake or lakehouse architectures with Amazon S3, AWS Glue, Amazon Athena, Amazon EMR, AWS Lake Formation, and Amazon Redshift.
• Proven ability developing production ETL/ELT pipelines using Python and PySpark, including data modeling, transformation, tuning, and error handling.
• Hands‑on experience with Apache Iceberg covering ACID transactions, snapshots, schema evolution, and query optimization.
• Advanced SQL skills supporting analytical queries, reporting, and data visualization workloads.
• Demonstrated experience with data governance, cataloging, lineage, ownership, classification, and access controls.
• Knowledge of AWS security fundamentals including IAM, encryption (KMS), secrets management, and network security.
• Experience provisioning resources via Infrastructure as Code (IaC) and managing multi-environment platforms.
• Skilled in building and maintaining CI/CD pipelines for data workflows with automation, testing, and rollback strategies.
• Strong troubleshooting skills for distributed data workloads with focus on performance, reliability, and cost management.

System One, and its subsidiaries including Joulé and Mountain Ltd., are leaders in delivering outsourced services and workforce solutions across North America. We help clients get work done more efficiently and economically, without compromising quality. System One not only serves as a valued partner for our clients, but we offer eligible employees health and welfare benefits coverage options including medical, dental, vision, spending accounts, life insurance, voluntary plans, as well as participation in a 401(k) plan.

System One is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, age, national origin, disability, family care or medical leave status, genetic information, veteran status, marital status, or any other characteristic protected by applicable federal, state, or local law.

#M-2
#LI-CK1
Ref: #851-Rockville-S1