Principal Data Engineer — Lead Data Platform & Pipelines

Watershed Ventures; United States; Presencial; USD 160 000 - 230 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Competitive health benefits

Matched 401k and pension plans

PTO

Parental leave

Gym subsidies

Education reimbursements

Resumo da oferta

Octus, a leading global credit intelligence provider, seeks a Principal Data Engineer in the United States. This hands-on IC role includes mentoring, architecture guidance, and technical leadership to design scalable data ingestion and transformation pipelines powering data platforms and automation initiatives.

Expect expert Python and SQL skills and deep AWS experience across MWAA, Lambda, ECS, and SQS. The role emphasizes data modeling, quality, lineage, and IaC with Terraform, collaborating

Qualificações

• 8+ years of experience in data engineering or backend development.
• Strong proficiency in Python for data pipelines and automation.
• Advanced SQL skills and experience with complex data models.
• Experience designing cloud-native data pipelines on AWS (MWAA, Lambda, ECS, SQS, Glue, S3, Redshift).

Responsabilidades

• Lead the overarching technical strategy for the data platform.
• Design and develop data ingestion and transformation pipelines for multiple sources.
• Mentor senior engineers and guide architecture decisions.
• Architect and manage data pipelines and orchestration with AWS services like MWAA, Lambda, ECS, and SQS.
• Implement IaC using Terraform to ensure reproducibility and compliance.
• Collaborate with analysts and backend engineers to ensure data reliability and discoverability.
• Promote continuous improvement through code reviews and observability practices.

Conhecimentos

Python

SQL

Data pipelines

AWS

Terraform

Data modeling

CI/CD

Leadership

Docker

Airflow

Ferramentas

MWAA (Airflow)

Lambda

ECS

SQS

Redshift

Snowflake

Databricks

Descrição da oferta de emprego

Octus, a leading global credit intelligence provider, seeks a Principal Data Engineer in the United States. This hands-on IC role includes mentoring, architecture guidance, and technical leadership to design scalable data ingestion and transformation pipelines powering data platforms and automation initiatives.

Expect expert Python and SQL skills and deep AWS experience across MWAA, Lambda, ECS, and SQS. The role emphasizes data modeling, quality, lineage, and IaC with Terraform, collaborating