Remote Databricks Data Engineer — Medallion Data Pipelines

GovCIO; City of Albany (NY); Remote; USD 130,000 - 160,000; Full time

Benefits offered by this job

Employee Assistance Program (EAP)

Corporate Discounts

Learning & Development platform

Certification Assistance

Referral Bonus Program

Internal Mobility Program

Pet Insurance

Flexible Work Environment

Job summary

GovCIO is hiring a Data Engineer to design and build enriched datasets using medallion architecture in Databricks, sourcing data from VA CDW and internal systems. You will feed Power BI semantic models and work directly with clients to gather requirements, track work in JIRA, and deliver robust data pipelines.

Ideal candidates have 12+ years of data engineering experience, strong SQL, Databricks, and Power BI skills, with effective communication for stakeholder collaboration in a fully remote

Qualifications

• Bachelor's degree and 12+ years of experience or commensurate experience
• Hands-on experience building data pipelines in Databricks (PySpark and/or SQL)
• Strong SQL skills including complex joins, aggregations, and tuning
• Experience with Power BI and preparing data for semantic models
• Familiarity with GitHub for version control and collaboration
• Experience using JIRA to track work
• Strong communication skills with clients/stakeholders
• Understanding of medallion (bronze/silver/gold) data architecture principles

Responsibilities

• Design, build, and maintain enriched datasets following medallion architecture in Databricks
• Develop, orchestrate, and monitor Databricks jobs and pipelines
• Write efficient, well-structured SQL to transform and model healthcare data
• Prepare and optimize gold-layer datasets for Power BI semantic models
• Manage code and version control through GitHub with standard branching/review practices
• Log, track, and update work items in JIRA
• Collaborate with clients to understand requirements and validate outputs
• Troubleshoot data quality issues and ensure accuracy across pipelines
• Document data lineage, transformation logic, and business rules
• Collaborate with Power BI developers and data scientists to meet downstream needs

Skills

PySpark

SQL

Power BI

Databricks

GitHub

JIRA

Client communication

Medallion architecture

Education

Bachelor's degree

Tools

Databricks

Power BI

GitHub

JIRA

Job description

GovCIO is hiring a Data Engineer to design and build enriched datasets using medallion architecture in Databricks, sourcing data from VA CDW and internal systems. You will feed Power BI semantic models and work directly with clients to gather requirements, track work in JIRA, and deliver robust data pipelines.

Ideal candidates have 12+ years of data engineering experience, strong SQL, Databricks, and Power BI skills, with effective communication for stakeholder collaboration in a fully remote