Lead Data Engineer — AWS & PySpark Data Pipelines

MaziCTools; Los Angeles (CA); Hybrid; USD 34,000 - 50,000; Full time

Job summary

United Airlines is seeking a Lead Data Engineer to drive data platform initiatives and scalable pipelines. You will design and implement streaming data applications on AWS, lead a team, and collaborate with analytics, ML, and software groups to enable business capabilities.

We value diverse perspectives and offer a broad set of opportunities, including open roles across cloud-based data services and innovative architectures.

Qualifications

• Four year college education and 8+ years of professional experience.
• Experience managing infrastructure and data pipelines using Cloud Development, Terraform, PySpark, Wind stream, or similar tech.
• Experience leading a team that designs, architects, or supports solutions on AWS.
• Hands-on experience with AWS services such as EC2, S3, IAM and Lambda.
• Working knowledge of CI/CD tools, SQL, Python, and Git.

Responsibilities

• Collaborate with advancement groups and other division leaders to deliver innovative technical solutions enabling business capabilities.
• Lead plan and develop creative streaming data applications using AWS; support large-scale data pipelines in a distributed, scalable environment.
• Set up containers and Serverless environments with cloud infrastructure.
• Provide technical direction and guide the team in problem solving and incident resolution.
• Value diverse perspectives and invite all qualified candidates to apply; seek fresh ideas and inclusive collaboration.

Skills

AWS

Terraform

PySpark

Streaming

OpenTelemetry

OpenLineage

Marquez

CI/CD

Python

Education

Bachelor Degree

Tools

AWS

Terraform

PySpark

Streaming

Wind stream

OpenTelemetry

OpenLineage

Marquez

EC2

S3

IAM

Lambda

Job description

United Airlines is seeking a Lead Data Engineer to drive data platform initiatives and scalable pipelines. You will design and implement streaming data applications on AWS, lead a team, and collaborate with analytics, ML, and software groups to enable business capabilities.

We value diverse perspectives and offer a broad set of opportunities, including open roles across cloud-based data services and innovative architectures.