Remote Data Engineer — PySpark, Databricks & Data Quality

Sentara Health; United States; Remote; USD 102,000 - 156,000; Full time

Benefits offered by this job

Medical, Dental, Vision plans

401k/403B with Employer Match

Paid Time Off

Parental leave

Educational assistance

Student debt pay down

Job summary

Sentara Health is seeking a Data Engineer to support a modern data platform built on Databricks. This is a fully remote position with opportunities across multiple states.

You will build scalable data pipelines, support a metadata-driven ingestion framework, and ensure data quality and governance are embedded in the platform. Ideal candidates have 3–5 years of relevant experience, strong PySpark and Databricks skills, SQL proficiency, and familiarity with Delta Lake, cloud platforms (Azure

Qualifications

• 3+ years of relevant experience, or 5+ years without a degree.
• Hands-on experience with PySpark and Databricks.
• Strong SQL skills and experience building ETL/ELT pipelines.
• Understanding of Delta Lake concepts and schema evolution.
• Familiarity with cloud platforms (Azure preferred).
• Experience with data governance, metadata, and data quality.

Responsibilities

• Develop and maintain data pipelines using PySpark and Databricks.
• Work within a metadata-driven ingestion framework to onboard new datasets.
• Implement data quality checks and validation rules within pipelines.
• Support ingestion from file-based sources and ingestion tools (e.g., Fivetran).
• Handle schema changes, incremental loads, and file processing patterns.
• Contribute to data governance practices including tagging, metadata, and lineage.
• Troubleshoot and resolve pipeline failures and performance issues.
• Collaborate with architects and stakeholders on data onboarding and requirements.
• Follow and contribute to coding standards, reusable components, and best practices.

Skills

PySpark

Databricks

SQL

ETL/ELT

Delta Lake

Azure

Git

Data governance

Fivetran

Metadata-driven

Tools

Git

DataHub

Fivetran

Job description

Sentara Health is seeking a Data Engineer to support a modern data platform built on Databricks. This is a fully remote position with opportunities across multiple states.

You will build scalable data pipelines, support a metadata-driven ingestion framework, and ensure data quality and governance are embedded in the platform. Ideal candidates have 3–5 years of relevant experience, strong PySpark and Databricks skills, SQL proficiency, and familiarity with Delta Lake, cloud platforms (Azure