Cloud Data Engineering Lead — Real-Time Pipelines & AWS

Valiant Technologies LLC; Irving (TX); In loco; USD 84.000 - 120.000; Tempo pieno

Descrizione del lavoro

Valiant Technologies LLC in Irving, TX is seeking a senior Data Engineer to design and optimize scalable data pipelines and microservices using Python. You will collaborate with Principal Software Engineers and Data Architects to define solution architecture and align data platforms with enterprise goals.

The role focuses on cloud-native ingestion and streaming solutions leveraging AWS services (Kinesis, S3, DynamoDB, EventBridge) and on ensuring data quality, automated testing, and

Competenze

• Bachelor's degree in Computer Science, Computer Engineering, or related field.
• 8+ years of experience in data engineering or related disciplines with increasing responsibility.
• Extensive experience on large scale data platforms such as Helios Data Platform.
• Strong Python production experience and data pipeline development.
• Strong SQL skills with data quality and validation practices.
• Experience with CI/CD tools such as Azure DevOps, Jira, Jenkins.
• Experience with AWS data services at scale (Kinesis, S3, DynamoDB, EventBridge).

Mansioni

• Actively collaborate with Principal Software Engineers and Data Architects to define solution architecture
• Lead the solution design and optimization of scalable data pipelines and microservices in Python, enabling both real-time and batch data processing across enterprise platforms
• Drive the development of cloud-native data ingestion and streaming solutions leveraging AWS services including Kinesis, S3, DynamoDB, EventBridge, and related technologies
• Own the design, implementation, and operational excellence of data integration frameworks and source data pipelines supporting CI Autonomy initiatives
• Partner with business, product, and engineering stakeholders to translate complex requirements into scalable data architectures, workflows, mappings, and system designs
• Establish and enforce automated testing, data quality controls, and validation frameworks to ensure integrity, reliability, and compliance across distributed data ecosystems
• Lead operational monitoring, performance tuning, and root-cause analysis of production data platforms using observability tools such as CloudWatch to maintain high availability and service reliability

Conoscenze

Python programming

Data engineering

Leadership

SQL

Data quality

Formazione

Bachelor's degree in Computer Science/Engineering

Strumenti

AWS Kinesis

AWS S3

DynamoDB

EventBridge

CloudWatch

Azure DevOps

Jira

Jenkins

Descrizione del lavoro

Valiant Technologies LLC in Irving, TX is seeking a senior Data Engineer to design and optimize scalable data pipelines and microservices using Python. You will collaborate with Principal Software Engineers and Data Architects to define solution architecture and align data platforms with enterprise goals.

The role focuses on cloud-native ingestion and streaming solutions leveraging AWS services (Kinesis, S3, DynamoDB, EventBridge) and on ensuring data quality, automated testing, and