Azure Databricks & Synapse Data Engineer (Hybrid)

Info Origin Inc; Indianapolis (IN); Hybrid; USD 55,000 - 59,000; Full time

Job summary

Info Origin Inc. in Indianapolis is seeking an experienced Data Engineer to design and optimize data pipelines and lakehouse solutions using Azure Databricks and Azure Synapse Analytics.

You will handle batch and streaming data, implement medallion architecture, and ensure data governance for sensitive health data in HIPAA-aligned environments. The role requires strong PySpark, Delta Lake, and Synapse skills, plus experience with Data Factory, Unity Catalog, and security controls.

Qualifications

• 4+ years of relevant hands-on data engineering experience or equivalent education and experience.
• Hands-on experience with Unity Catalog, Delta Lake transactions, schema evolution, and Spark performance tuning.
• Proficiency in Azure Data Factory, Databricks Workflows, ADLS Gen2, and pipeline orchestration.

Responsibilities

• Build and maintain batch and streaming pipelines using Azure Databricks, PySpark, Spark SQL, and Delta Lake.
• Implement medallion architecture (Bronze/Silver/Gold) and incremental data ingestion.
• Develop and optimize data models, SQL queries, and warehouse workloads in Azure Synapse Analytics.
• Orchestrate workflows using Azure Data Factory and Databricks Workflows.
• Implement data governance, security, and access controls using Unity Catalog, Entra ID, and Key Vault.
• Manage CI/CD, source control, monitoring, troubleshooting, and pipeline performance optimization.
• Collaborate with architects, analysts, data scientists, and project managers on requirements and technical solutions.
• Maintain technical documentation and support data quality, governance, and operational improvements.

Skills

PySpark

Spark SQL

Delta Lake

Databricks notebooks

medallion architecture

Unity Catalog

Delta Lake transactions

schema evolution

Spark performance tuning

Azure Synapse Analytics

MPP architecture

dimensional modelling

star schemas

SCD Type 1/2

SQL optimization

Azure Data Factory

Databricks Workflows

ADLS Gen2

pipeline orchestration

Azure DevOps

GitHub

CI/CD

security controls

Azure monitoring tools

lakehouse patterns

Synapse-Databricks integration

Job description

Info Origin Inc. in Indianapolis is seeking an experienced Data Engineer to design and optimize data pipelines and lakehouse solutions using Azure Databricks and Azure Synapse Analytics.

You will handle batch and streaming data, implement medallion architecture, and ensure data governance for sensitive health data in HIPAA-aligned environments. The role requires strong PySpark, Delta Lake, and Synapse skills, plus experience with Data Factory, Unity Catalog, and security controls.