Lead Data Engineer: Cloud Data Pipelines

The Phoenix Group; New York (NY); In loco; USD 140.000 - 190.000; Tempo pieno

Descrizione del lavoro

The Phoenix Group is seeking a Lead Data Engineer to own design, development, and maintenance of enterprise-scale data pipelines and architecture. You will collaborate with cross-functional teams to enable analytics, AI/ML initiatives, and governance across cloud environments.

Responsibilities include building scalable pipelines with Snowflake, Airflow, and dbt, implementing medallion data models, and driving CI/CD for data solutions while ensuring security and observability.

Competenze

• 5+ years in data or software engineering with progressive responsibilities.
• Experience with Snowflake, AWS, and data orchestration tools.
• Designing data architectures and medallion models.
• Migration of production workflows to cloud orchestration.
• Experience with IAM, S3, Lambda, and CloudWatch.
• Data quality, lineage, and observability practices.
• Strong communication with technical and business stakeholders.

Mansioni

• Design, build, test, deploy, and support scalable data pipelines using Airflow, dbt, Snowflake.
• Optimize cloud data architecture, schemas, security roles, and performance.
• Implement medallion architecture (raw/bronze to gold) for data.
• Migrate legacy workflows to cloud orchestration with reusable patterns.
• Maintain orchestration runtimes: schedulers, workers, metadata, queues, autoscaling.
• Collaborate with AWS services like S3, IAM, EKS, Lambda, CloudWatch.
• Build secure integrations across APIs, SFTP, databases, files, and event systems.
• Develop ingestion pipelines for structured and unstructured data incl OCR.
• Embed data quality, lineage, observability into workflows.
• Partner with governance and security teams on data ownership and retention.
• Build CI/CD for workflows and infrastructure code with testing and rollback.
• Provide on-call support and incident management.
• Establish engineering standards and reusable frameworks.
• Lead technical workstreams and code reviews.

Conoscenze

Snowflake

Airflow

dbt

Python

SQL

Kubernetes

CI/CD

Data governance

Strumenti

Git workflows

Docker

Kubernetes

Descrizione del lavoro

The Phoenix Group is seeking a Lead Data Engineer to own design, development, and maintenance of enterprise-scale data pipelines and architecture. You will collaborate with cross-functional teams to enable analytics, AI/ML initiatives, and governance across cloud environments.

Responsibilities include building scalable pipelines with Snowflake, Airflow, and dbt, implementing medallion data models, and driving CI/CD for data solutions while ensuring security and observability.