Remote Senior Data Engineer

Kforce Inc; Carlsbad (CA); Remote; USD 140,000 - 210,000; Full time

Job summary

Kforce Inc. is seeking a Remote Senior Data Engineer to design and own end-to-end data architecture, including secure data landing zone, ingestion pipelines, and curated analytics layer for analytics and ML workloads.

You will build ingestion pipelines for partner data files and REST/SFTP integrations, enforce data quality, and implement data governance in a HIPAA-regulated environment. Collaboration with backend, ML, product, and clinical teams is essential.

Qualifications

• BS/BA degree or higher in Computer Science, Engineering, or a related technical field.
• 8+ years of professional data engineering experience, with a track record of shipping production data systems end-to-end.
• Mastery of PostgreSQL: schema design, indexing, query tuning, partitioning, logical replication, JSONB, extensions (pg_partman, pg_cron, pgvector, etc.), and operating Postgres at scale.
• Strong experience designing and operating data pipelines, including file-based ingestion (SFTP/object storage drops) and API-based ingestion (REST, webhooks).
• Strong experience with dbt (or equivalent SQL-based transformation framework) and modern data modeling patterns (Kimball dimensional, Data Vault, One Big Table).
• Hands-on experience with one or more cloud platforms (AWS preferred) and their data primitives: object storage (S3), managed Postgres.
• Experience designing data warehouses and/or data lakes and the judgment to know which one a given problem actually needs.
• Experience with at least one orchestration framework (Dagster, Prefect, or Airflow).
• Experience with data validation and data-quality frameworks (Great Expectations, Pandera, Soda, or equivalent).
• Experience with change-data-capture from Postgres (logical replication, or equivalent).
• Data governance experience in a HIPAA-regulated environment or strong instincts for PHI/PII protection; HITRUST/SOC 2 is a plus.
• Experience supporting ML workloads and using AI coding assistants (e.g., GitHub Copilot).
• Strong Python skills for ingestion, validation, and tooling.
• Comfortable with infrastructure-as-code and CI/CD for data systems.

Responsibilities

• Design and own end-to-end data architecture including secure landing zone, validated ingestion pipelines, and curated analytics layer.
• Build ingestion pipelines for partner data files (CSV/JSON/XML/HL7/X12) and REST/SFTP API integrations with schema validation and data lineage.
• Stand up and operate the curated data layer (data warehouse/lakehouse-lite) for analytics and ML models.
• Select and operate the minimal toolset: object storage, an orchestrator, dbt or equivalent, and a single data validation library.
• Design and enforce data governance for HIPAA-regulated environment including PHI/PII classification, encryption, access controls, and audit trails.
• Partner with Backend/ML/Product/Clinical teams to define data contracts and ensure data quality.
• Develop and maintain reliable feature data for ML models including embeddings (pgvector) and feature tables for risk/engagement/outcomes.
• Instrument the data platform for observability with SLA, data freshness, schema drift, and quality metrics.
• Participate fully in Agile processes: backlog grooming, sprint planning, demos, and retrospectives.

Skills

Python

Data modeling

Data governance

HIPAA compliance

ML workloads

AI coding aids

CI/CD

Cloud experience

PostgreSQL

Education

BS/BA in CS/Engineering

Tools

Dagster

Prefect

Airflow

dbt

Great Expectations

Pandera

Soda

PostgreSQL

pgvector

AWS

S3

Job description

Responsibilities

Kforce has a client that is seeking a Remote Senior Data Engineer.

• Design and own end-to-end data architecture: a landing zone with secure cloud object storage for raw partner files and API payloads, validated ingestion pipelines into our transactional Postgres, and a curated analytics layer that decouples reporting and AI workloads from production


• Build ingestion pipelines for the data we receive today, including partner data files (CSV/JSON/XML/HL7/X12 as applicable) and REST/SFTP API integrations with schema validation, quarantine of bad records, and full lineage from raw bytes to curated row


• Stand up and operate the curated layer (data warehouse/lakehouse-lite) so analytics and ML models can consume data without slowing down the transactional system


• Choose, integrate, and operate the smallest set of tools needed, including object storage, an orchestrator (Dagster, Prefect, Airflow, etc.), dbt or similar for transformations, a single validation library (Great Expectations/Pandera/Soda)


• Design and enforce data governance for a HIPAA-regulated environment: PHI/PII classification, encryption in transit and at rest, role-based access, audit logging, retention and minimum-necessary policies, and de-identification where appropriate


• Partner with backend, ML, product, and clinical stakeholders to define data contracts with our health plan and ACO partners and hold the line on data quality


• Build and maintain reliable feature data for ML models, including embeddings (e.g., pgvector) and curated feature tables for risk stratification, engagement, and outcomes work


• Instrument the data platform for observability including pipeline SLAs, data freshness, schema drift, quality metrics, and act on what the data tells you


• Participate fully in our Agile process: backlog grooming, sprint planning, demos, and retrospectives

Requirements

• BS/BA degree or higher in Computer Science, Engineering, or a related technical field


• 8+ years of professional data engineering experience, with a track record of shipping production data systems end-to-end


• Mastery of PostgreSQL: schema design, indexing, query tuning, partitioning, logical replication, JSONB, extensions (pg_partman, pg_cron, pgvector, etc.), and operating Postgres at scale


• Strong experience designing and operating data pipelines, including file-based ingestion (SFTP/object storage drops) and API-based ingestion (REST, webhooks)


• Strong experience with dbt (or equivalent SQL-based transformation framework) and modern data modeling patterns (Kimball dimensional, Data Vault, One Big Table - and an opinion about when each is right)


• Hands-on experience with one or more cloud platforms (AWS preferred) and their data primitives: object storage (S3), managed Postgres


• Experience designing data warehouses and/or data lakes and the judgment to know which one a given problem actually needs


• Experience with at least one orchestration framework (Dagster, Prefect, or Airflow) and a clear point of view on which to use when


• Experience with data validation and data-quality frameworks (Great Expectations, Pandera, Soda, or equivalent)


• Experience with change-data-capture from Postgres (logical replication, or equivalent)


• Data governance experience in a HIPAA-regulated environment or, at minimum, demonstrated instincts for protecting PHI and PII (encryption, least privilege, audit, de-identification, BAA-aware vendor selection); HITRUST or SOC 2 experience is a strong plus


• Experience supporting ML workloads


• Experience using AI coding assistants (e.g., GitHub Copilot, Cursor, Claude) to accelerate development


• Strong Python skills for ingestion, validation, and tooling


• Comfortable with infrastructure-as-code and CI/CD for data systems

The pay range is the lowest to highest compensation we reasonably in good faith believe we would pay at posting for this role. We may ultimately pay more or less than this range. Employee pay is based on factors like relevant education, qualifications, certifications, experience, skills, seniority, location, performance, union contract and business needs. This range may be modified in the future.

We offer comprehensive benefits including medical/dental/vision insurance, HSA, FSA, 401(k), and life, disability & ADD insurance to eligible employees. Salaried personnel receive paid time off. Hourly employees are not eligible for paid time off unless required by law. Hourly employees on a Service Contract Act project are eligible for paid sick leave.

Note: Pay is not considered compensation until it is earned, vested and determinable. The amount and availability of any compensation remains in Kforce's sole discretion unless and until paid and may be modified in its discretion consistent with the law.

This job is not eligible for bonuses, incentives or commissions.

Kforce is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status, or disability status.