Data Engineer

Elite WorkForce Inc; Indianapolis (IN); On-site; USD 110,000 - 150,000; Full time

Job summary

Elite WorkForce Inc. seeks a Data Engineer to design, build, and maintain secure, scalable data solutions using Azure Databricks and Azure Synapse Lakehouse.

You will develop batch and streaming pipelines, optimize data warehouse workloads, and implement governance and security controls for PHI/PII compliant data. The role requires collaboration with data architects, scientists, analysts, and PMs to translate business needs into robust technical designs, with emphasis on security, modeling, and

Qualifications

• Master's degree or Bachelor's with 4+ years in data engineering
• Experience with Azure Databricks PySpark and Delta Lake
• Strong knowledge of Azure Synapse (dedicated/serverless)
• Experience with data governance, HIPAA/PHI security
• Proficient in SQL, data modeling, and ETL/ELT pipelines

Responsibilities

• Design, build, and maintain batch and streaming data pipelines in Azure Databricks and Synapse
• Optimize data warehouse workloads for reporting and analytics
• Implement data governance, RBAC, data masking and security controls
• Collaborate with architects, scientists, analysts, and PMs to translate business needs into robust designs

Skills

Azure Databricks

PySpark

Delta Lake

Azure Synapse Analytics

Azure Data Factory

Security & Compliance

Education

Master's degree in data-related field

Bachelor's degree + 4+ years experience

Tools

Azure Databricks

Azure Synapse

GitHub

Azure DevOps

Unity Catalog

Job description

Position: Data Engineer Azure Databricks & Synapse Lakehouse

Location: Indianapolis, IN 46204

Client: State of Indiana

Duration: 12+ Months

Job Description: Seeking a data engineer to design, build, and maintain secure, scalable data solutions using Azure Databricks and Azure Synapse. You'll develop batch and streaming pipelines, optimize data warehouse workloads, implement data governance, and collaborate with architects, scientists, analysts, and project managers to translate business needs into robust technical designs. Focus areas include data integration, data modeling, and documenting solutions with strong emphasis on security and compliance (PHI/PII).

Key Responsibilities:

• Design, build, and maintain batch and streaming data pipelines in Azure Databricks (PySpark, Spark SQL, Delta Lake) using a medallion lakehouse architecture.
• Develop and optimize dimensional models and SQL workloads in Azure Synapse Analytics (dedicated/serverless) for reporting and dashboards.
• Orchestrate pipelines with Databricks Workflows, Synapse, and Data Factory; implement scheduling, monitoring, alerting, and failure recovery.
• Apply data governance and security controls (Unity Catalog, RBAC, data masking) to protect sensitive health data per HIPAA and policy.
• Collaborate with data architects, scientists, analysts, and PMs; document solutions and provide technical support.

Required/Desired Skills:

• Master's degree in data-related field or Bachelor's + 4+ years of relevant experience (or 4+ years in a similar data engineering role)
• Databricks: PySpark, Spark SQL, R; medallion lakehouse, Auto Loader, Structured Streaming
• Unity Catalog, workspace catalogs, permissions, lineage, external locations
• ADLS Gen2, Parquet/Delta formats; OPENROWSET, external tables, views
• Star/schema design, SCD Types 1 & 2; COPY INTO, CETAS, Synapse/ADF pipelines
• Git/GitHub, CI/CD, Azure DevOps; Databricks Asset Bundles
• Monitoring with Azure Monitor/Log Analytics; troubleshooting failed jobs
• Data governance, PHI/PII protection, HIPAA compliance
• Strong communication skills

Nice-to-have:

• Databricks Certified Data Engineer
• Azure Data Engineer Associate (DP-203)
• Power BI integration with Synapse (Import vs. DirectQuery)

Note: This posting emphasizes hands-on data engineering with a focus on secure, scalable, and governed data solutions using Azure Databricks and Synapse.