Senior Data Engineer: Spark, Databricks & Azure

Allina Health; United States; On-site; USD 75,000 - 104,000; Full time

Benefits offered by this job

Medical/Dental

PTO/Time Away

Retirement Savings Plans

Life Insurance

Disability Insurance

Tuition Reimbursement

Student Loan Support

Job summary

Allina Health is seeking a Principal Data Architect/Engineer to lead data warehouse migration projects across sources and marts. You will architect pipelines using Azure Data Factory, Databricks, and Unity Catalog, translating legacy SQL into efficient Spark SQL and Python while ensuring governance and security.

Responsibilities include building reusable code assets, defining infrastructure for hybrid/on-prem/cloud setups, and enabling analytics with Power BI.

Qualifications

• Bachelor's degree in computer science, engineering or related field; 5+ years data engineering or related role.
• 5+ years experience in SQL and Python.
• Preferred: 3+ years Azure Cloud Platform; Epic Cogito experience.

Responsibilities

• Architect data pipelines and data warehouse migrations using Azure Data Factory and Spark SQL.
• Translate legacy SQL to optimized Spark SQL and Python code for analytics.
• Ensure data governance, security and regulatory compliance (HIPAA/GDPR).
• Develop reusable code assets and environment infrastructure for data teams.
• Collaborate with stakeholders to validate data outputs and enable BI reporting (Power BI).

Skills

SQL

Python

Education

Bachelor's degree in computer science, engineering or related field

Master's degree in computer science, engineering or related field

Tools

Azure Data Factory

Databricks

Unity Catalog

Spark SQL

Power BI

Job description

Allina Health is seeking a Principal Data Architect/Engineer to lead data warehouse migration projects across sources and marts. You will architect pipelines using Azure Data Factory, Databricks, and Unity Catalog, translating legacy SQL into efficient Spark SQL and Python while ensuring governance and security.

Responsibilities include building reusable code assets, defining infrastructure for hybrid/on-prem/cloud setups, and enabling analytics with Power BI.