Principal Data Engineer — Lead Data Platform & Pipelines

Watershed Ventures; United States; On-site; USD 160,000 - 230,000; Full time

Benefits offered by this job

Competitive health benefits

Matched 401k and pension plans

PTO

Parental leave

Gym subsidies

Education reimbursements

Job summary

Octus, a leading global credit intelligence provider, seeks a Principal Data Engineer in the United States. This hands-on IC role includes mentoring, architecture guidance, and technical leadership to design scalable data ingestion and transformation pipelines powering data platforms and automation initiatives.

Expect expert Python and SQL skills and deep AWS experience across MWAA, Lambda, ECS, and SQS. The role emphasizes data modeling, quality, lineage, and IaC with Terraform, collaborating

Qualifications

• 8+ years of experience in data engineering or backend development.
• Strong proficiency in Python for data pipelines and automation.
• Advanced SQL skills and experience with complex data models.
• Experience designing cloud-native data pipelines on AWS (MWAA, Lambda, ECS, SQS, Glue, S3, Redshift).

Responsibilities

• Lead the overarching technical strategy for the data platform.
• Design and develop data ingestion and transformation pipelines for multiple sources.
• Mentor senior engineers and guide architecture decisions.
• Architect and manage data pipelines and orchestration with AWS services like MWAA, Lambda, ECS, and SQS.
• Implement IaC using Terraform to ensure reproducibility and compliance.
• Collaborate with analysts and backend engineers to ensure data reliability and discoverability.
• Promote continuous improvement through code reviews and observability practices.

Skills

Python

SQL

Data pipelines

AWS

Terraform

Data modeling

CI/CD

Leadership

Docker

Airflow

Tools

MWAA (Airflow)

Lambda

ECS

SQS

Redshift

Snowflake

Databricks

Job description

Octus, a leading global credit intelligence provider, seeks a Principal Data Engineer in the United States. This hands-on IC role includes mentoring, architecture guidance, and technical leadership to design scalable data ingestion and transformation pipelines powering data platforms and automation initiatives.

Expect expert Python and SQL skills and deep AWS experience across MWAA, Lambda, ECS, and SQS. The role emphasizes data modeling, quality, lineage, and IaC with Terraform, collaborating