Data Engineer: Secure AWS Data Pipelines (Python)

Compunnel Inc.

Richmond (VA)

On-site

USD 95,000 - 140,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Compunnel, Inc. seeks a Data Engineer to design, build, and optimize a Python-based, AWS-native data ingestion platform. You will move financial data from source systems into a third-party app, while ensuring secure, scalable pipelines and strict governance.

You will implement testing strategies and partner with teams to maintain data quality and platform reliability in a regulated environment.

Qualifications

  • Proficiency in building production-grade data processing applications.
  • Hands-on experience with AWS services and modern data engineering frameworks.
  • Experience with event-driven and streaming architectures.
  • Ability to design and develop data ingestion and transformation pipelines.
  • Experience implementing CI/CD pipelines and deployment automation.
  • Strong understanding of data security, governance, and compliance.

Responsibilities

  • Design, develop, and maintain data pipeline components across the full data lifecycle.
  • Build scalable data processing solutions using AWS-native services.
  • Develop unit, integration, and end-to-end testing strategies.
  • Collaborate with cross-functional teams for data quality and platform reliability.
  • Ensure compliance with data governance, security, and regulatory requirements.
  • Identify and remediate security findings across applications and infrastructure.
  • Contribute to CI/CD best practices and deployment automation.
  • Monitor and optimize cloud-native data pipelines and platforms.
  • Support secure, scalable, and reliable data engineering solutions.

Skills

Python
Data pipelines
AWS services
CI/CD pipelines
Testing methodologies
Data security
Regulatory compliance
Cloud-native troubleshooting
Event-driven architectures
Streaming architectures

Tools

AWS Lambda
AWS ECS
AWS S3
DynamoDB
AWS Flink

Job description

Job Summary
We are seeking a Data Engineer to support the development and enhancement of a Python-based, AWS-native data ingestion platform responsible for moving financial data from source systems into a third-party application. This role focuses on building scalable, secure, and highly reliable data pipelines while ensuring compliance with enterprise data governance, security, and regulatory requirements.

Key Responsibilities
• Design, develop, and maintain data pipeline components across the full data lifecycle, including ingestion, validation, tokenization, transformation, and publishing.
• Build and support scalable data processing solutions using AWS-native services and modern data engineering frameworks.
• Develop and execute unit, integration, end-to-end, and system testing strategies.
• Partner with cross-functional teams during system and user acceptance testing to ensure data quality and platform reliability.
• Ensure compliance with enterprise data governance, security, and regulatory requirements, including the secure handling of sensitive and highly sensitive data.
• Identify, troubleshoot, and remediate application vulnerabilities and security findings across applications, infrastructure, and third-party dependencies.
• Contribute to CI/CD best practices, source control management, and deployment automation.
• Monitor, troubleshoot, and optimize cloud-native data pipelines and processing platforms.
• Support secure, scalable, and reliable cloud-based data engineering solutions.

Required Qualifications
• Strong proficiency in Python.
• Experience building and maintaining production-grade data processing applications.
• Hands-on experience with AWS services, including Lambda, ECS, S3, and DynamoDB.
• Experience with event-driven and streaming architectures.
• Experience designing and developing data ingestion and transformation pipelines.
• Experience implementing CI/CD pipelines and deployment automation.
• Experience with unit, integration, and end-to-end testing methodologies.
• Strong understanding of data security, governance, and compliance requirements.
• Experience supporting the secure handling of sensitive financial or regulated data.
• Ability to troubleshoot complex cloud-native data pipeline and platform issues.

Preferred Qualifications
• Experience with AWS Flink.
• Experience building event-driven architectures and large-scale streaming platforms.
• Experience supporting financial services or other highly regulated environments.
• Familiarity with Infrastructure as Code (IaC) and cloud-native application development.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

Compunnel Inc. • Richmond (VA)

On-site
USD 95,000 - 140,000
Senior Software Data Engineer (Python)
Senior Software Data Engineer (Python)

Pointwest-North America • Atlanta (GA)

On-site
USD 110,000 - 140,000
Senior Data Engineer: Cloud Pipelines, Analytics & ML Data
Senior Data Engineer: Cloud Pipelines, Analytics & ML Data

Compunnel Inc. • Charlotte (NC)

On-site
USD 120,000 - 150,000
AWS Data Engineer
AWS Data Engineer

Qode • United States

On-site
USD 150,000 - 210,000
AWS Data Engineer: Scalable Pipelines & APIs
AWS Data Engineer: Scalable Pipelines & APIs

Compunnel Inc. • Pennsylvania

On-site
USD 120,000 - 150,000
Senior Data Engineer: Real-Time Pipelines on AWS
Senior Data Engineer: Real-Time Pipelines on AWS

eOne Infotech • Fort Mill (SC)

On-site
USD 90,000 - 120,000
Data Engineer
Data Engineer

The Value Maximizer • South Carolina

On-site
USD 90,000 - 120,000
Data Engineer: Scalable ETL & Analytics Pipelines
Data Engineer: Scalable ETL & Analytics Pipelines

Compunnel Inc. • Pennsylvania

Hybrid
USD 85,000 - 125,000
Senior Software Data Engineer (Python)
Senior Software Data Engineer (Python)

Pointwest Technologies Corp • Atlanta (GA)

On-site
USD 110,000 - 130,000
Cloud Data Engineer(Python)
Cloud Data Engineer(Python)

Premier Inn Hotels LLC (UAE) • Plano (TX)

On-site
USD 130,000 - 190,000