Staff Data Engineer: Lead Data Platform & AI Pipelines

Cribl; Columbia (SC); Remote; USD 170,000 - 220,000; Full time

Benefits offered by this job

Health care benefits

Dental insurance

Vision insurance

401(k)

Equity

Job summary

Cribl, a remote-first data observability software company, seeks a Staff Data Engineer to lead Cribl’s Data Engineering team. You’ll shape architecture for analytics, data science, and operations, partnering with data analysts, SREs, and IT. You’ll own self-hosted orchestration, Snowflake, dbt, and cost/performance at scale, and mentor engineers to raise the bar.

The role emphasizes AWS, IaC with Terraform, and secure, compliant engineering practices. Base salary range $170,000-$220,000 USD.

Qualifications

• Deep expertise in Snowflake: advanced data modeling, query and warehouse performance tuning, cost management, and security/RBAC design.
• Demonstrated technical leadership: experience as a tech lead or staff-level engineer setting direction, driving architecture.
• Excellent communication and cross-functional collaboration: able to work across data, engineering, and business teams and lead through influence.
• A track record of mentoring engineers: growing junior and mid-level engineers through coaching, review, and pairing.
• Strong Python and SQL fundamentals, with solid ELT patterns and data modeling experience.
• Hands-on AWS and infrastructure-as-code experience: building cloud infrastructure with Terraform (or equivalent) using predictable, maintainable patterns.
• Experience building cloud applications or backend services: APIs, ingestion services, and event-driven workflows.

Responsibilities

• Serve as a technical lead for the Data Engineering team.
• Own architecture and design decisions for core data systems and drive reliable, production-grade outcomes.
• Partner with stakeholders across functions (Data Analysts, SREs, IT Engineers, and business teams).
• Clarify requirements, validate outputs, and communicate risks, tradeoffs, and timelines proactively.
• Own and operate our self-hosted orchestration platform.
• Run and evolve our self-managed Prefect deployment and its build/worker system.
• Own the operational hygiene, upgrade path, and cost/footprint tradeoffs of running orchestration on a hybrid, self-hosted model.
• Own Cribl's Snowflake environment.
• Serve as the subject-matter expert on data modeling, performance, and cost optimization, and own RBAC/permissions design, permissions tooling, and governance.
• Build, operate, and monitor the core data tech stack. This covers data pipelines, ingestion services, integrations, and the warehouse, ensuring data is accurate, timely, and trusted, with logging, alerting, and observability as first-class concerns.
• Advance the modern data stack.
• Drive our dbt practice forward, including the in-flight dbt Fusion migration, and own warehouse cost/performance at scale.
• Develop cloud-native services and infrastructure on AWS. This includes our ingest/API services (event-driven ingestion, upsert patterns, package reuse) and the IaC that supports them.
• Advance our infrastructure-as-code practice (Terraform or similar).
• Build reusable modules, clean deployment patterns, CI integration, and strong operational hygiene, including self-hosted/hybrid deployment patterns.
• Partner with the Data Engineering manager to set technical direction.
• Shape the platform roadmap, engineering standards, and build-vs-buy/architectural tradeoffs.
• Mentor and grow data engineers through code review, pairing, design feedback, and coaching that levels up junior and mid-level engineers.
• Support Cribl's growing data science and agentic initiatives.
• Prepare model-ready datasets, expose features, and integrate AI/LLM workflows into production.
• Contribute to secure, compliance-minded engineering practices in collaboration with IT/Security.

Skills

Snowflake

Python

SQL

Data Modeling

ELT patterns

AWS

Terraform

Prefect

Airflow

Dagster

Tools

dbt

Airflow

Dagster

Kubernetes

Job description

Cribl, a remote-first data observability software company, seeks a Staff Data Engineer to lead Cribl’s Data Engineering team. You’ll shape architecture for analytics, data science, and operations, partnering with data analysts, SREs, and IT. You’ll own self-hosted orchestration, Snowflake, dbt, and cost/performance at scale, and mentor engineers to raise the bar.

The role emphasizes AWS, IaC with Terraform, and secure, compliant engineering practices. Base salary range $170,000-$220,000 USD.