Data Engineer: Azure Databricks Migration (Remote)

Medasource; BLOOMINGTON, Northern (MN, KY); Hybrid; USD 117,000 - 138,000; Full time

Job summary

Medasource is seeking a Data Engineer for a remote, 1-year contract to migrate a finance-focused application from on-prem Oracle/IBM DataStage to Databricks on Azure. This role offers high visibility and end-to-end ownership.

You will build ingestion pipelines, medallion architecture transformations, collaborate with stakeholders, govern data, and contribute to a growing data team as additional staff are hired.

Qualifications

• Hands-on data engineering with data modeling and multi-source integration.
• Strong SQL skills.
• Proficiency in Python and PySpark.
• Experience with Python-based orchestration tools.
• Git version control.
• Databricks experience preferred.
• Experience migrating on-prem data to cloud platforms.
• Excellent communication with business stakeholders.

Responsibilities

• Migrate data from on-premises Oracle and IBM DataStage to Databricks on Azure.
• Design and build data ingestion pipelines and medallion-layer transformations using Python and PySpark.
• Gather requirements from business users and translate them into data solutions.
• Analyze and model source data and determine data element relationships.
• Monitor, troubleshoot, and resolve issues in data pipelines and data quality.
• Support data governance across the new cloud platform.
• Manage code using Git and participate in team workflows.
• Contribute to building the new data team as staff are hired.
• Own problems end-to-end from stakeholder conversations to production pipelines.

Skills

SQL

Python

PySpark

Data modeling

Data governance

Git

Databricks

Airflow

Azure

Stakeholder communication

Tools

Databricks

Azure Databricks

Airflow

Geared to cloud migration tools

Job description

Medasource is seeking a Data Engineer for a remote, 1-year contract to migrate a finance-focused application from on-prem Oracle/IBM DataStage to Databricks on Azure. This role offers high visibility and end-to-end ownership.

You will build ingestion pipelines, medallion architecture transformations, collaborate with stakeholders, govern data, and contribute to a growing data team as additional staff are hired.