Lead Data Engineer - Real-Time Cloud Pipelines

Robotics Technologies LLC; Irving (TX); In loco; USD 84.000 - 120.000; Tempo pieno

Descrizione del lavoro

Robotics Technologies LLC in Irving, TX is seeking an experienced Senior Data Engineer to architect and build scalable data pipelines and microservices in Python. You will partner with principal engineers and data architects to define solution architecture and deliver real-time and batch processing across enterprise platforms.

You will lead cloud-native data ingestion leveraging AWS Kinesis, S3, DynamoDB, EventBridge, and related services, ensuring data quality, observability, and CI/CD

Competenze

• Bachelor's degree in Computer Science, Computer Engineering, or related field.
• 8+ years of experience in data engineering or related disciplines.
• Experience with large-scale Caterpillar data platforms such as Helios Data Platform.
• Strong Python production experience and data pipeline development.
• Expertise with AWS data services (Kinesis, S3, DynamoDB, EventBridge) at scale.
• Proven ability to lead teams building high-throughput, scalable data pipelines.
• Experience deploying software using CI/CD tools (Azure DevOps, Jira, Jenkins).
• Strong SQL skills with data quality and validation practices.
• Experience with distributed and streaming systems data integrity.
• Familiarity with observability and monitoring (e.g., CloudWatch).

Mansioni

• Collaborate with principal engineers and data architects to define solution architecture.
• Lead design and optimization of scalable data pipelines and microservices in Python.
• Develop cloud-native data ingestion and streaming solutions using AWS services.
• Own data integration frameworks and source data pipelines for CI Autonomy initiatives.
• Translate complex requirements into scalable data architectures and workflows.
• Institute automated testing, data quality controls, and validation frameworks.
• Lead production monitoring, performance tuning, and root-cause analysis using observability tools.

Conoscenze

Python

Data pipelines

Data engineering

SQL

AWS

Microservices

Cloud architecture

Team leadership

Formazione

Bachelor's degree in Computer Science/Engineering

Strumenti

AWS Kinesis

Amazon S3

DynamoDB

EventBridge

CloudWatch

Descrizione del lavoro

Robotics Technologies LLC in Irving, TX is seeking an experienced Senior Data Engineer to architect and build scalable data pipelines and microservices in Python. You will partner with principal engineers and data architects to define solution architecture and deliver real-time and batch processing across enterprise platforms.

You will lead cloud-native data ingestion leveraging AWS Kinesis, S3, DynamoDB, EventBridge, and related services, ensuring data quality, observability, and CI/CD