Data Engineering Lead: Cloud-Native Pipelines & Real-Time Data

Quantum Technologies. LLC

Irving (TX)

On-site

USD 84,000 - 120,000

Full time

2 hours ago

Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Quantum Technologies LLC is seeking a senior Data Engineer to lead the design and optimization of large-scale data pipelines and microservices in Python, partnering with Principal Software Engineers and Data Architects to define scalable solution architectures.

You will drive cloud-native data ingestion and streaming using AWS services (Kinesis, S3, DynamoDB, EventBridge) and ensure CI/CD delivery with Azure DevOps, Jira, and Jenkins across real-time and batch data processing.

Qualifications

Bachelor's degree required in CS/CE or related field
8+ years of experience in data engineering or related disciplines
Strong Python production experience
Extensive AWS data services experience (Kinesis, S3, DynamoDB, EventBridge) at scale
Experience with CI/CD tooling (Azure DevOps, Jira, Jenkins)
Experience with data integrity, quality controls and observability

Responsibilities

Define solution architecture with Principal Engineers and Data Architects
Lead design and optimization of scalable data pipelines and microservices in Python
Drive cloud-native ingestion and streaming solutions using AWS services
Own data integration frameworks and source data pipelines for CI Autonomy initiatives
Translate complex requirements into scalable data architectures and workflows
Establish automated testing and data quality validation across distributed systems
Lead monitoring, performance tuning and root-cause analysis using CloudWatch