Senior Full-Stack AWS Data Engineer

Virtue America; Pennsylvania; On-site; USD 140,000 - 190,000; Full time

Job summary

Virtue America in Pennsylvania seeks a Senior Full-Stack AWS Data Engineer to design and maintain scalable cloud-based data platforms and modern pipelines. Expertise in Python, PySpark, SQL, AWS services, and data lake architecture is required to build reliable, production-ready data solutions.

The candidate will drive ETL/ELT development, implement IaC with Terraform, and ensure observability, security, and performance across data workflows in a fast-paced environment.

Qualifications

• Strong experience in designing and maintaining scalable data platforms.
• Proficient in Python, PySpark, and SQL for ETL/ELT.
• Hands-on with AWS services (S3, Glue, IAM, Kinesis) and data lake/warehouse tech.

Responsibilities

• Design, develop, and optimize ETL/ELT pipelines using Python, PySpark, and SQL.
• Build and maintain cloud-based data solutions with AWS services (S3, Glue, IAM, Kinesis, ECS, EKS).
• Implement data warehouses/queries with Snowflake, Redshift, and Athena.
• Develop streaming/in real-time data integration using Kinesis and Confluent.
• Create and manage data lakes with Iceberg and related tech.
• Automate infrastructure with Terraform and IaC practices.
• Establish CI/CD pipelines using GitLab/GitHub; support DevOps tooling.
• Manage containerized workloads with Docker and Kubernetes; support scalable deployments.
• Monitor pipelines, troubleshoot production issues, and implement observability.
• Apply DataOps/DevOps practices to improve data quality, reliability, and security.
• Collaborate with architects, analysts, and engineers to deliver enterprise data solutions.

Skills

Python

PySpark

SQL

Data engineering

Cloud data architecture

CI/CD

Observability

DevOps practices

Communication

Tools

AWS

S3

Glue

IAM

Kinesis

ECS

EKS

Snowflake

Redshift

Athena

Terraform

Docker

Kubernetes

Confluent

Iceberg

dbt Core

GitLab

GitHub

Job description

Job Title: Senior Full-Stack AWS Data Engineer

Duration: Long-Term

Job Summary:

We are seeking an experienced Senior Full-Stack AWS Data Engineer to design, develop, and maintain scalable cloud-based data platforms and modern data pipelines. The ideal candidate will have strong expertise in Python, PySpark, SQL, AWS services, data lake architecture, and ETL/ELT development, along with experience in infrastructure automation, streaming data integration, and production support.

Key Responsibilities:

• Design, develop, and optimize scalable ETL/ELT pipelines using Python, PySpark, and SQL.
• Build and maintain cloud-based data solutions using AWS S3, Glue, IAM, Kinesis, ECS, and EKS.
• Implement and optimize data warehouses and query solutions using Snowflake, Amazon Redshift, and Athena.
• Design streaming and real-time data integration solutions using Amazon Kinesis and Confluent.
• Build and manage modern data lakes using Apache Iceberg and related technologies.
• Automate cloud infrastructure provisioning using Terraform and Infrastructure as Code (IaC) practices.
• Implement CI/CD pipelines using GitLab, GitHub, and DevOps tools.
• Manage containerized workloads and support scalable deployments using Docker and Kubernetes.
• Monitor data pipeline performance, troubleshoot production issues, and implement observability solutions.
• Apply DataOps and DevOps practices to improve data quality, reliability, security, and deployment efficiency.
• Collaborate with data architects, analysts, and engineering teams to deliver reliable enterprise data solutions.

Required Qualifications:

• Strong experience in data engineering and scalable data architecture.
• Advanced proficiency in Python, PySpark, and SQL.
• Hands-on experience with AWS cloud services, including S3, Glue, IAM, and Kinesis.
• Experience building and maintaining scalable ETL/ELT data pipelines.
• Knowledge of ECS/EKS, containerized workloads, and Kubernetes.
• Experience with Terraform and infrastructure automation.
• Hands-on experience with dbt Core and modern data transformation practices.
• Experience with Snowflake, Redshift, and Athena.
• Knowledge of streaming data platforms such as Kinesis and Confluent.
• Experience with GitLab/GitHub, CI/CD pipelines, and version control.
• Understanding of Apache Iceberg and modern data lake architectures.
• Strong knowledge of DataOps, DevOps, monitoring, and production troubleshooting.

Preferred Qualifications:

• Knowledge of data governance, data quality, and security best practices.
• Experience with automated data testing and pipeline orchestration.
• Strong analytical, problem-solving, and communication skills.