Senior Data Engineer

Gravie; United States; Remote; USD 130,000 - 175,000; Full time

Benefits offered by this job

Alternative medicine coverage

Flexible PTO

16 weeks parental leave

Paid holidays

401k program

Transportation perks

Education reimbursement

Paw-ternity leave

Job summary

Gravie is hiring a Senior Data Engineer to own a near real-time streaming platform for operational data. You will build, extend and operate pipelines ingesting CDC events, transform with Spark, and enforce PHI governance. This is a hands-on role requiring production ownership and collaboration across teams.

You will manage latency SLOs, implement observability, and drive cost-aware infrastructure as code on AWS. Strong SQL/Python and on-call experience are essential for success at Gravie.

Qualifications

• 6+ years building and operating production data systems.
• Production experience with Kafka including exactly-once semantics.
• Strong PySpark experience for streaming and batch transformation.
• AWS-native data engineering across streaming, processing, storage, and catalog services.
• Infrastructure-as-code with AWS CDK or Terraform and CI/CD for data pipelines.
• Excellent SQL and Python skills.
• Experience with data contracts, schema governance, and cataloging.
• Health data/PHI domain knowledge is a plus.

Responsibilities

• Run the platform in production, owning latency/throughput SLOs and monitoring.
• Build and extend streaming pipelines ingesting CDC events.
• Transform and enrich data using Spark and related tools.
• Enable secure, governed data access over PHI.
• Ensure reliability with idempotent, replayable pipelines and runbooks.
• Provide code-defined platform in AWS with CDK and cost-conscious sizing.
• Collaborate with upstream/downstream teams to meet data needs.

Skills

Kafka

Spark (PySpark)

SQL

Python

AWS CDK

REST APIs

Datadog / CloudWatch

Data governance

On-call / incident response

Education

Degree in CS/IS or quantitative field

Tools

MSK

EMR

Glue

S3

Athena

Debezium

Iceberg

Delta Lake

Schema Registry

Glue Data Catalog

Unity Catalog

Atlan

Alation

Collibra

Datadog

CloudWatch

Job description

Hi, we’re Gravie. Our mission is to create health benefits that actually benefit small and midsize businesses and their employees. Our innovative benefit solutions and services are developed and delivered by a diverse group of unique people. We encourage you to be your authentic self - we like you that way.

More About This Role:

We're building a near real-time streaming data platform for operational data across the business. We seek a Senior Data Engineer to both build/extend and operate it: you'll build the infrastructure and you'll own the platform in production - latency and throughput SLOs, backpressure under spiky load, replay and backfill, dead-letter triage, and connector failure recovery. This is a hands-on role for someone who wants to build, extend, and operate a live production platform, not just design it and hand it off.

You are self-driven, calm under production pressure, and comfortable owning streaming systems in a regulated environment.

You Will:

• Run the platform in production: own latency/throughput SLOs, monitoring and alerting (e.g. Datadog) - including replays, per-source backfills, connector-failure recovery, dead-letter triage, and tuning for spiky, batch-driven claims load.

• Build and extend streaming pipelines that ingest CDC events from operational databases and SaaS sources into canonical, contract-validated form.

• Transform and enrich in Spark (Structured Streaming or dbt-on-Spark micro-batch), including cross-stream joins that correlate events into unified lifecycle entities.

• Enable secure, governed data access over PHI: classification, row/column controls, and access policy applied as data is served to consumers (e.g. ABAC).

• Make it reliable and observable: idempotent/replayable pipeline design, data-quality validation, runbooks, and observability the broader data team can rely on.

• Provision as code: define the platform (streaming, processing, storage, and catalog services on AWS) in CDK with CI/CD for data pipelines, and right-size for cost against the latency SLO.

• Partner across teams: work with upstream producers on source changes and contracts, with downstream consumers on access and data needs, and with stakeholders to turn requirements into what the platform delivers.

• Demonstrate commitment to our core competencies of being authentic, curious, creative, empathetic and outcome oriented.

You Bring:

• 6+ years building and operating production data systems including demonstrated ownership of streaming or event-driven pipelines - on-call, incident response, SLOs, runbooks, and recovery, not just development.

• Deep, production experience with Apache Kafka - partitioning, consumer groups, consumer-lag and broker-health troubleshooting, exactly-once/idempotent semantics, schema registry, and replay/backfill under load.

• Strong, hands-on Apache Spark experience (PySpark) for streaming and batch transformation in Production.

• AWS-native data engineering across streaming, processing, storage, and catalog services (e.g. MSK, EMR, Glue, S3, Athena) , with infrastructure-as-code - AWS CDK (preferred) or Terraform - CI/CD for data pipelines, and cost awareness.

• Comfort debugging distributed data pipelines (consumer lag, data skew, backpressure, late/out-of-order events) with observability tooling (e.g. Datadog/Cloudwatch).

• Expert-level SQL and Python, and experience building and consuming REST APIs.

• An AI-forward engineering mindset, with demonstrated hands‑on use of AI‑assisted and agentic development tools, an opinion on where AI adds value (and where it doesn’t), and an understanding of agentic data consumption patterns and needs to act on trusted operational data—including context management, lineage, provenance, permissions, freshness, and low-latency access for agentic discovery.

• Change data capture and open table formats - CDC (e.g. Debezium) plus Iceberg or Delta Lake: schema evolution, partitioning, and table maintenance.

• Data contracts, schema governance, and cataloging - schema registries and compatibility rules with dead-letter handling; and familiarity with a technical metastore (e.g. Glue Data Catalog, Unity Catalog) and a governance/discovery catalog (e.g. Atlan, Alation, Collibra).

• Degree in Computer Science, Information Systems or another quantitative field, and comfort on the command line / a Unix-based OS (we are 100% Mac+Linux at Gravie).

• Health insurance domain knowledge - HIPAA Protected Health Information (PHI) and governing access to it.

• Excellent communication skills and demonstrated success in driving results through influence and collaboration.

Extra Credit:

• Experience with Apache Flink or other stateful stream processors - helpful context but not required.

• Knowledge of JVM-based languages like Kotlin or Java.

• Familiarity with serving data to AI and agentic consumers - exposing canonical data as low-latency context or inputs for automated/agentic workloads.

• Previous venture-backed start-up company experience.

A Little More About Us:

• We know healthcare. Our company was founded and is still led by industry veterans who have started and grown several market-leading companies in the space.

• We have raised money from top tier investors who share the same long-term vision as we do of building an industry defining company that will endure over the long run. We are well capitalized.

• Our clients love us. Customer satisfaction rates among employees using Gravie health plans consistently rank above 80% – nearly 40 points above the industry average.

• Our culture is unique. We tend to be non-hierarchical, merit-driven, opinionated but kind people who thrive working in a high-performance, fast-paced environment. People at Gravie care deeply about making a positive impact in the lives of the people we serve.

Benefits

Our unique benefits program is the gravy, i.e., the special sauce that sets our compensation package apart. In addition to standard health and wellness benefits, Gravie’s package includes alternative medicine coverage, flexible PTO, up to 16 weeks paid parental leave, paid holidays, a 401k program, transportation perks, education reimbursement, and 2 days of paid paw-ternity leave.

Job Applicants

If you apply for employment with Gravie, personal information collected via our applicant tracking vendor is subject to our standalone California Job Applicant Notice at Collection, accessible directly within the application workflow and separate from this Policy.