Remote Senior Data Platform Engineer

Kforce Inc; Carlsbad (CA); Remote; USD 140,000 - 210,000; Full time

Job summary

Kforce Inc. is seeking a Remote Senior Data Engineer to design and own end-to-end data architecture, including secure data landing zone, ingestion pipelines, and curated analytics layer for analytics and ML workloads.

You will build ingestion pipelines for partner data files and REST/SFTP integrations, enforce data quality, and implement data governance in a HIPAA-regulated environment. Collaboration with backend, ML, product, and clinical teams is essential.

Qualifications

• BS/BA degree or higher in Computer Science, Engineering, or a related technical field.
• 8+ years of professional data engineering experience, with a track record of shipping production data systems end-to-end.
• Mastery of PostgreSQL: schema design, indexing, query tuning, partitioning, logical replication, JSONB, extensions (pg_partman, pg_cron, pgvector, etc.), and operating Postgres at scale.
• Strong experience designing and operating data pipelines, including file-based ingestion (SFTP/object storage drops) and API-based ingestion (REST, webhooks).
• Strong experience with dbt (or equivalent SQL-based transformation framework) and modern data modeling patterns (Kimball dimensional, Data Vault, One Big Table).
• Hands-on experience with one or more cloud platforms (AWS preferred) and their data primitives: object storage (S3), managed Postgres.
• Experience designing data warehouses and/or data lakes and the judgment to know which one a given problem actually needs.
• Experience with at least one orchestration framework (Dagster, Prefect, or Airflow).
• Experience with data validation and data-quality frameworks (Great Expectations, Pandera, Soda, or equivalent).
• Experience with change-data-capture from Postgres (logical replication, or equivalent).
• Data governance experience in a HIPAA-regulated environment or strong instincts for PHI/PII protection; HITRUST/SOC 2 is a plus.
• Experience supporting ML workloads and using AI coding assistants (e.g., GitHub Copilot).
• Strong Python skills for ingestion, validation, and tooling.
• Comfortable with infrastructure-as-code and CI/CD for data systems.

Responsibilities

• Design and own end-to-end data architecture including secure landing zone, validated ingestion pipelines, and curated analytics layer.
• Build ingestion pipelines for partner data files (CSV/JSON/XML/HL7/X12) and REST/SFTP API integrations with schema validation and data lineage.
• Stand up and operate the curated data layer (data warehouse/lakehouse-lite) for analytics and ML models.
• Select and operate the minimal toolset: object storage, an orchestrator, dbt or equivalent, and a single data validation library.
• Design and enforce data governance for HIPAA-regulated environment including PHI/PII classification, encryption, access controls, and audit trails.
• Partner with Backend/ML/Product/Clinical teams to define data contracts and ensure data quality.
• Develop and maintain reliable feature data for ML models including embeddings (pgvector) and feature tables for risk/engagement/outcomes.
• Instrument the data platform for observability with SLA, data freshness, schema drift, and quality metrics.
• Participate fully in Agile processes: backlog grooming, sprint planning, demos, and retrospectives.

Skills

Python

Data modeling

Data governance

HIPAA compliance

ML workloads

AI coding aids

CI/CD

Cloud experience

PostgreSQL

Education

BS/BA in CS/Engineering

Tools

Dagster

Prefect

Airflow

dbt

Great Expectations

Pandera

Soda

PostgreSQL

pgvector

AWS

S3

Job description

Kforce Inc. is seeking a Remote Senior Data Engineer to design and own end-to-end data architecture, including secure data landing zone, ingestion pipelines, and curated analytics layer for analytics and ML workloads.

You will build ingestion pipelines for partner data files and REST/SFTP integrations, enforce data quality, and implement data governance in a HIPAA-regulated environment. Collaboration with backend, ML, product, and clinical teams is essential.