Principal Data Engineer: Lead Data Pipelines & Architecture

Socket.dev; United States; On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Health benefits

401k match

Paid time off

Parental leave

Gym subsidies

Education reimbursement

Recognition programs

Job summary

Octus is seeking a Principal Data Engineer to design, develop, and lead scalable data ingestion and transformation pipelines. This hands-on IC role includes mentorship, architecture guidance, and technical leadership across the data platform.

The ideal candidate is an expert Python and SQL developer with deep AWS experience, IaC, and data workflow orchestration for high‑volume data. Join a global team driving data innovation.

Qualifications

• Strong software engineering principles (SOLID, modularity, scalability).
• Expert proficiency in Python for data pipelines and automation.
• Advanced SQL skills and data modeling expertise.
• Experience building cloud-native data pipelines on AWS (MWAA, Lambda, ECS, SQS, Glue, S3, Redshift).
• Experience with data warehousing/lakehouse tech (Redshift, Snowflake, Databricks).
• Terraform or IaC framework experience.
• Knowledge of data ingestion, transformation, orchestration incl. AI/ML pipelines.
• Familiarity with CI/CD, automated testing, DevOps practices.
• 8+ years in data engineering or backend development with leadership experience.
• Leadership in complex data infra projects and mentoring.
• Experience with Docker and workflow orchestration.

Responsibilities

• Set technical strategy for the data platform aligned with business goals.
• Design and build scalable data ingestion and transformation pipelines.
• Mentor senior engineers and guide architecture decisions.
• Architect data pipelines and orchestration using AWS services (MWAA, Lambda, ECS, SQS).
• Implement infrastructure as code (Terraform) for reproducibility and compliance.
• Collaborate with analysts and engineers to ensure data reliability and discoverability.
• Apply best practices in data modeling, schema design, and ETL/ELT processes.
• Ensure data quality with automated tests, monitoring, and alerts.
• Drive continuous improvement through code reviews and observability.

Skills

Python

SQL

Data modeling

Cloud native

ETL/ELT

CI/CD

Terraform

Docker

Airflow

Mentoring

Tools

MWAA (Airflow)

Lambda

ECS

SQS

Glue

Redshift

Job description

Octus is seeking a Principal Data Engineer to design, develop, and lead scalable data ingestion and transformation pipelines. This hands-on IC role includes mentorship, architecture guidance, and technical leadership across the data platform.

The ideal candidate is an expert Python and SQL developer with deep AWS experience, IaC, and data workflow orchestration for high‑volume data. Join a global team driving data innovation.