Senior Azure Databricks & PySpark Data Architect - Onsite

ERPMARK INC; Greenfield (IN); On-site; USD 130,000 - 190,000; Full time

Job summary

ERPMARK INC. seeks a Senior Azure Databricks and PySpark Data Engineer/Architect to design, develop, and implement enterprise-scale Azure data solutions.

You will build scalable pipelines, modern Lakehouse architectures, and cloud-native data platforms in a healthcare-adjacent enterprise setting. The role requires hands-on expertise with Azure Databricks, PySpark, ADF, ADLS Gen2, and Azure SQL, plus strong DevOps practices and technical leadership.

Qualifications

• 10–20 years of overall IT experience.
• Strong hands-on Azure Data Engineering background.
• Proven experience developing enterprise-scale data pipelines and Lakehouse solutions.
• Ability to prepare low-level design documents and follow coding standards.
• Strong technical communication and collaboration skills.
• Ability to take ownership of solution design and technical decisions.

Responsibilities

• Design, develop, and maintain enterprise-scale Azure Data Engineering solutions.
• Build and optimize data pipelines using Azure Databricks, PySpark, and Azure Data Factory.
• Develop and maintain modern Lakehouse architectures using Azure Synapse and Databricks.
• Implement and manage Databricks workflows and Unity Catalog.
• Build scalable data processing solutions using ADLS Gen2 and Azure SQL.
• Develop Python packages and support package builds.
• Implement DevOps practices and CI/CD processes for data engineering solutions.
• Develop and maintain data architecture and low-level design documents.
• Collaborate with internal engineering teams and customers to deliver high-quality solutions.
• Follow standard coding principles, guidelines, and engineering best practices.
• Lead technical decisions, mentor team members, and drive engineering improvements.

Skills

Azure Databricks

PySpark

Azure Data Factory

Azure Data Lake Storage Gen2

Azure SQL

Python

Azure Cloud

Communication

Tools

Databricks Workflows

Unity Catalog

GitHub Actions

Airflow

Dagster

Job description

ERPMARK INC. seeks a Senior Azure Databricks and PySpark Data Engineer/Architect to design, develop, and implement enterprise-scale Azure data solutions.

You will build scalable pipelines, modern Lakehouse architectures, and cloud-native data platforms in a healthcare-adjacent enterprise setting. The role requires hands-on expertise with Azure Databricks, PySpark, ADF, ADLS Gen2, and Azure SQL, plus strong DevOps practices and technical leadership.