Principal Data Engineer

Unknown Company; United States; Hybrid; USD 150,000 - 190,000; Full time

Benefits offered by this job

Health care plan

401k

Life Insurance

Unlimited PTO

9 paid holidays

Family leave

Work From Home

Free food & snacks

Job summary

Worth AI is seeking a Principal Data Engineer to own the company-wide data architecture and platform. You will design scalable ELT/ETL pipelines, establish data quality and governance, and enable analytics and ML through secure, cost-efficient systems.

You will partner with engineering, product, analytics, and security to turn business needs into durable data products, leading architecture decisions and setting standards across pipelines, storage, and semantic models.

Qualifications

• 10+ years in data engineering (including staff/principal scope)
• Proven leadership of company-wide data architecture and platform initiatives
• Deep experience with at least one cloud (AWS) and a modern warehouse/lakehouse (Snowflake/Redshift/Databricks)
• Strong SQL and programming in Python or Scala/Java
• Orchestration (Airflow/Dagster/Prefect), transformations (dbt), streaming (Kafka/Kinesis)
• Data modeling (3NF, star, data vault) and semantic/metrics layers
• Data quality, lineage, observability in production
• Security: RBAC/ABAC, encryption, audits
• Nice-to-have: feature stores, ML data ops, governance tools
• Nice-to-have: cost optimization, multi-tenant architectures

Responsibilities

• Define end-to-end data architecture for lakehouse, batch/streaming, and metadata
• Publish roadmaps, ADRs/RFCs, and north star target states
• Design and build scalable ELT/ETL pipelines and event pipelines
• Provide self-service tooling for analysts/scientists (dbt/notebooks/catalogs)
• Define data SLAs/SLOs, lineage, and governance protocols
• Drive cost optimization and FinOps for storage/compute
• Ensure security, encryption, and compliance with privacy regulations
• Deliver versioned datasets/features for BI and ML
• Mentor senior engineers and lead design reviews
• Translate business goals into data products with cross-functional teams

Skills

Data architecture

ELT/ETL pipelines

Snowflake

Redshift

Databricks

SQL

Python

Scala/Java

Airflow

dbt

Data quality

Governance

Security best practices

Leadership

Tools

Dagster

Prefect

Kafka

Kinesis

Job description

Worth AI, a leader in the computer software industry, is looking for a talented and experienced Principal Data Engineer to join their innovative team. At Worth AI, we are on a mission to revolutionize decision-making with the power of artificial intelligence while fostering an environment of collaboration, and adaptability, aiming to make a meaningful impact in the tech landscape. Our team values include extreme ownership, one team and creating reaving fans both for our employees and customers. Worth is looking for a Principal Data Engineer to own the company-wide data architecture and platform. Design and scale reliable batch/streaming pipelines, institute data quality and governance, and enable analytics/ML with secure, cost-efficient systems. Partner with engineering, product, analytics, and security to turn business needs into durable data products.

Responsibilities

Architecture & Strategy

• Define end-to-end data architecture (lake/lakehouse/warehouse, batch/streaming, CDC, metadata)
• Set standards for schemas, contracts, orchestration, storage layers, and semantic/metrics models
• Publish roadmaps, ADRs/RFCs, and "north star" target states; guide build vs. buy decisions

Platform & Pipelines

• Design and build scalable, observable ELT/ETL and event pipelines
• Establish ingestion patterns (CDC, file, API, message bus) and schema-evolution policies
• Provide self-service tooling for analysts/scientists (dbt, notebooks, catalogs, feature stores)
• Ensure workflow reliability (idempotency, retries, backfills, SLAs)

Data Quality & Governance

• Define dataset SLAs/SLOs, freshness, lineage, and data certification tiers
• Enforce contracts and validation tests; deploy anomaly detection and incident runbooks
• Partner with governance on cataloging, PII handling, retention, and access policies

Reliability, Performance & Cost

• Lead capacity planning, partitioning/clustering, and query optimization
• Introduce SRE-style practices for data (error budgets, postmortems)
• Drive FinOps for storage/compute; monitor and reduce cost per TB/query/job

Security & Compliance

• Implement encryption, tokenization, and row/column-level security; manage secrets and audits
• Align with SOC 2 and privacy regulations (e.g., GDPR/CCPA; HIPAA if applicable)

ML & Analytics Enablement

• Deliver versioned, documented datasets/features for BI and ML
• Operationalize training/serving data flows, drift signals, and feature-store governance
• Build and maintain the semantic layer and metrics consistency for experimentation/BI

Leadership & Collaboration

• Provide technical leadership across squads; mentor senior/staff engineers
• Run design reviews and drive consensus on complex trade-offs
• Translate business goals into data products with product/analytics leaders

Requirements

• 10+ years in data engineering (including 3+ years as staff/principal or equivalent scope)
• Proven leadership of company-wide data architecture and platform initiatives
• Deep experience with at least one cloud (AWS) and a modern warehouse or lakehouse (e.g., Snowflake, Redshift, Databricks)
• Strong SQL and one programming language (Python or Scala/Java)
• Orchestration (Airflow/Dagster/Prefect), transformations (dbt or equivalent), and streaming (Kafka/Kinesis/PubSub)
• Data modeling (3NF, star, data vault) and semantic/metrics layers
• Data quality testing, lineage, and observability in production environments
• Security best practices: RBAC/ABAC, encryption, key management, auditability
• Nice to Have: Feature stores and ML data ops; experimentation frameworks
• Nice to Have: Cost optimization at scale; multi-tenant architectures
• Nice to Have: Governance tools (DataHub/Collibra/Alation), OpenLineage, and testing frameworks (Great Expectations/Deequ)
• Nice to Have: Compliance exposure (SOC 2, GDPR/CCPA; HIPAA/PCI where relevant)
• Nice to Have: Model features sourced from complex 3rd-party data (KYB/KYC, credit bureaus, fraud detection APIs)

Benefits

• Health Care Plan (Medical, Dental & Vision)
• Retirement Plan (401k, IRA)
• Life Insurance
• Unlimited Paid Time Off
• 9 paid Holidays
• Family Leave
• Work From Home
• Free Food & Snacks (Access to Industrious Co-working Membership!)
• Wellness Resources

#J-18808-Ljbffr