Senior Data Engineer: Databricks + Delta Lake (Azure, Hybrid)

MS Companies Inc.; Indianapolis (IN); Hybrid; USD 115,000 - 145,000; Full time

Benefits offered by this job

Medical, Dental, Vision

401(k) Match

PTO & Holidays

Hybrid work

Job summary

MS Companies Inc. in Indianapolis, IN is seeking a Senior Data Engineer to design and scale a modern enterprise cloud data platform.

This hands-on role focuses on production systems, Databricks, Delta Lake, and governance, collaborating with Data Strategy, Finance, Operations, and Analytics teams. You'll build and optimize pipelines, strengthen the Databricks environment, integrate with Azure services, and implement CI/CD for data workflows, while driving reliability, cost efficiency, and

Qualifications

• 5+ years of Data Engineering or Cloud Data Engineering experience.
• Strong production experience with Databricks.
• Hands-on experience with Snowflake.
• Advanced Python / PySpark and SQL.
• Experience building and supporting scalable ETL/ELT pipelines.
• Solid understanding of cloud data architecture (Azure preferred).
• Experience with data modeling, schema design, and performance optimization.
• Bachelor’s degree in a technical field.
• Must be authorized to work in the U.S. without sponsorship.

Responsibilities

• Design, build, and maintain scalable data pipelines in Databricks.
• Optimize Databricks clusters for performance, reliability, and cost.
• Implement and maintain Delta Lake architecture.
• Integrate Databricks with Azure services (ADF, ADLS, Synapse, DevOps).
• Build CI/CD pipelines and workflow orchestration.
• Support data governance through Unity Catalog.
• Partner with business stakeholders to turn requirements into scalable solutions.
• Help establish engineering standards and best practices.

Skills

Python

PySpark

SQL

ETL/ELT pipelines

Azure

Data modeling

Schema design

Performance tuning

Data governance

Stakeholder comms

Education

Bachelor's degree in a technical field

Tools

Databricks

Snowflake

Job description

MS Companies Inc. in Indianapolis, IN is seeking a Senior Data Engineer to design and scale a modern enterprise cloud data platform.

This hands-on role focuses on production systems, Databricks, Delta Lake, and governance, collaborating with Data Strategy, Finance, Operations, and Analytics teams. You'll build and optimize pipelines, strengthen the Databricks environment, integrate with Azure services, and implement CI/CD for data workflows, while driving reliability, cost efficiency, and