Remote Data Engineer - PySpark & Databricks Expert

Sentara Health; United States; Remote; USD 102,000 - 156,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision plan

401(k) match

Paid time off

Parental leave

Tuition assistance

Student debt payoff

Life insurance

Disability coverage

Job summary

Sentara Health is seeking a Data Engineer for a fully remote role. The position supports a remote-friendly model with opportunities across multiple states, focusing on building robust data pipelines using PySpark and Databricks.

You will work within a metadata-driven framework to onboard datasets, ensure data quality, and support ingestion from tools like Fivetran. The role demands strong SQL skills, experience with Delta Lake concepts, and collaboration with architects to meet data onboarding

Qualifications

• 3+ years of relevant experience; degree or equivalent
• Hands-on PySpark and Databricks experience
• Strong SQL skills
• Experience building ETL/ELT data pipelines
• Understanding of Delta Lake concepts such as merge, schema evolution
• Familiarity with cloud platforms (Azure preferred)
• Basic Git/version control experience
• Exposure to data catalog or governance tools (e.g., DataHub)
• Experience with Fivetran or similar ingestion tools
• Knowledge of data quality and validation concepts
• Experience with metadata-driven frameworks
• Willingness to learn in a fast-evolving data ecosystem

Responsibilities

• Develop and maintain data pipelines using PySpark and Databricks
• Work within a metadata-driven ingestion framework to onboard new datasets
• Implement data quality checks and validation rules within pipelines
• Support ingestion from file-based sources and ingestion tools (e.g., Fivetran)
• Handle schema changes, incremental loads, and file processing patterns
• Contribute to data governance practices including tagging, metadata, and lineage
• Troubleshoot and resolve pipeline failures and performance issues
• Collaborate with architects and stakeholders on data onboarding and requirements
• Follow and contribute to coding standards, reusable components, and best practices

Skills

PySpark

SQL

ETL/ELT pipelines

Data quality checks

Data governance

Schema evolution

Metadata-driven frameworks

Git

DataHub

Ingestion tools

Education

Bachelor's Degree Optional

3+ years relevant experience with degree

5+ years relevant experience without degree

Tools

Databricks

Azure

Fivetran

Git

Delta Lake

DataHub

Job description

Sentara Health is seeking a Data Engineer for a fully remote role. The position supports a remote-friendly model with opportunities across multiple states, focusing on building robust data pipelines using PySpark and Databricks.

You will work within a metadata-driven framework to onboard datasets, ensure data quality, and support ingestion from tools like Fivetran. The role demands strong SQL skills, experience with Delta Lake concepts, and collaboration with architects to meet data onboarding