Remote Data Engineer: Databricks Medallion Pipelines

GovCIO; Raleigh (NC); Remote; USD 130,000 - 160,000; Full time

Benefits offered by this job

Employee Assistance Program (EAP)

Learning & Development platform

Certification support

Referral Bonus

Job summary

GovCIO is hiring a Data Engineer to design and build analytics-ready datasets in Databricks, sourced from the VA CDW and other systems. You will craft datasets for Power BI semantic models and work closely with clients to gather requirements, track work in JIRA, and deliver well-documented pipelines.

This fully remote role requires experience with Databricks, PySpark/SQL, and strong SQL. Collaboration with Power BI developers and data scientists is expected to ensure downstream reporting needs

Qualifications

• Bachelor's degree and 12+ yrs of experience (or commensurate experience).
• Hands-on experience building data pipelines and transformations in Databricks (PySpark and/or SQL).
• Strong SQL skills, including complex joins, aggregations, and performance tuning.
• Experience working with Power BI, including preparing data for semantic models.
• Familiarity with GitHub for version control and collaborative development.
• Experience using JIRA or similar tools to track and manage work.
• Strong communication skills and comfort working directly with clients/stakeholders.
• Understanding of medallion (bronze/silver/gold) data architecture principles.

Responsibilities

• Design, build, and maintain enriched datasets following medallion architecture (bronze, silver, gold layers) in Databricks.
• Develop, orchestrate, and monitor Databricks jobs and pipelines.
• Write efficient, well-structured SQL to transform and model healthcare data.
• Prepare and optimize gold-layer datasets for consumption by Power BI semantic models.
• Manage code and version control through GitHub, following team branching and review practices.
• Log, track, and update work items in JIRA.
• Partner directly with clients and stakeholders to understand requirements, clarify data definitions, and validate outputs.
• Troubleshoot data quality issues and ensure accuracy and consistency across pipelines.
• Write ad hoc queries to support client requests, investigations, and one-off analyses.
• Perform data validation and quality checks on pipelines and datasets.

Skills

SQL

PySpark

Data pipelines

Stakeholder comms

Education

Bachelor's degree

Tools

Databricks

GitHub

JIRA

Power BI

Job description

GovCIO is hiring a Data Engineer to design and build analytics-ready datasets in Databricks, sourced from the VA CDW and other systems. You will craft datasets for Power BI semantic models and work closely with clients to gather requirements, track work in JIRA, and deliver well-documented pipelines.

This fully remote role requires experience with Databricks, PySpark/SQL, and strong SQL. Collaboration with Power BI developers and data scientists is expected to ensure downstream reporting needs