Databricks Data Engineer – DLT, ADF & PySpark Focus

Johnson Service Group, Inc.; Almont (CO); Hybrid; USD 62,000 - 76,000; Full time

Job summary

Johnson Service Group (JSG) is seeking a Data Engineer (Databricks) to design, develop, and optimize enterprise data pipelines in a capital markets environment. The role requires hands-on Databricks, ADF, SQL, PySpark, and Azure Synapse expertise.

The position is hybrid, with three days onsite per week across the continental United States, offering exposure to cloud-based data solutions and production support in a fast-paced setting.

Qualifications

• At least 3 years of data engineering experience.
• Hands-on Databricks experience including Delta Live Tables.
• Proficiency with Azure Data Factory for orchestrating pipelines.
• Strong SQL and PySpark for data transformation.
• Experience with Azure Synapse Analytics and Dedicated SQL Pools.
• Python for data engineering and pipeline development.
• Knowledge of Azure DevOps for CI/CD workflows.
• Experience with Azure Function Apps and Azure Logic Apps.

Responsibilities

• Develop, enhance, and maintain data pipelines using existing ingestion frameworks and tools.
• Orchestrate and manage data pipelines using Azure Data Factory.
• Design and implement data transformations to parse, process, and load data into Enterprise Data Lake, Delta Lake, and Enterprise Data Warehouse (Synapse Analytics).
• Develop and maintain Databricks-based data processing solutions using DLT, Python, and PySpark.
• Perform unit testing and coordinate integration testing and User Acceptance Testing (UAT).
• Prepare and maintain technical documentation, including High-Level Designs (HLDs), Detailed Designs (DDs), and operational runbooks.
• Configure compute resources, implement data quality rules, and maintain pipeline reliability.
• Conduct performance tuning and optimization of data pipelines and processing workloads.
• Provide production support, troubleshoot operational issues, and resolve data pipeline failures.

Skills

Databricks

Delta Live Tables

Azure Data Factory

SQL

PySpark

Azure Synapse Analytics

Dedicated SQL Pools

Python

Azure DevOps

Azure Function Apps

Azure Logic Apps

Job description

Johnson Service Group (JSG) is seeking a Data Engineer (Databricks) to design, develop, and optimize enterprise data pipelines in a capital markets environment. The role requires hands-on Databricks, ADF, SQL, PySpark, and Azure Synapse expertise.

The position is hybrid, with three days onsite per week across the continental United States, offering exposure to cloud-based data solutions and production support in a fast-paced setting.