Data Engineer

Holistic Partners, Inc; United States; On-site; USD 120,000 - 160,000; Full time

Job summary

Holistic Partners, Inc. is seeking a Data Engineer to design, develop, and support enterprise Data Lake solutions that drive critical reporting, analytics, and BI initiatives.

You will build scalable data pipelines and optimize processing across the data platform. Responsibilities include implementing pipelines with Azure Data Factory and Azure Databricks, building PySpark and Python ETL, and delivering production-ready CI/CD workflows with Azure DevOps and Git.

Qualifications

• 5+ years of Data Engineering, ETL/ELT, or application development experience.
• Strong hands-on experience with Azure Databricks, Azure Data Factory (ADF), PySpark, Python, and SQL.
• Experience building Databricks notebooks and workspace datasets.
• Azure DevOps, Git, and CI/CD pipeline implementation.
• Experience with enterprise Data Lake and Data Warehouse environments.
• Strong troubleshooting, performance tuning, and production support experience.
• Experience supporting Tableau reporting and analytics solutions.
• Familiarity with Snowflake is highly preferred.

Responsibilities

• Design, develop, and maintain data pipelines using Azure Data Factory and Azure Databricks.
• Build and optimize PySpark and Python solutions for data ingestion, transformation, and enrichment.
• Develop Databricks notebooks and workspace datasets.
• Support Data Lake refresh processes and troubleshoot data quality, performance, and operational issues.
• Develop and maintain CI/CD pipelines using Azure DevOps and Git.
• Collaborate with architects, analysts, engineers, and reporting teams to deliver scalable data solutions.
• Provide production support, monitoring, root cause analysis, and continuous improvement across the data platform.
• Build pipelines and tables that enable the Analytics team to develop dashboards and reports.

Skills

Azure Databricks

Azure Data Factory

PySpark

Python

SQL

CI/CD pipelines

Azure DevOps

Git

Data Lake

Data Warehouse

Tableau

Snowflake

Production support

Tools

Databricks notebooks

Workspace datasets

Job description

We are seeking a Data Engineer to design, develop, and support enterprise Data Lake solutions that drive critical reporting, analytics, and business intelligence initiatives. This role will focus on building scalable data pipelines, optimizing data processing frameworks, automating deployments, and ensuring reliable production operations across the organization's data platform.

Position Description

• Design, develop, and maintain data pipelines using Azure Data Factory and Azure Databricks.
• Build and optimize PySpark and Python solutions for data ingestion, transformation, and enrichment.
• Develop Databricks notebooks and workspace datasets.
• Support Data Lake refresh processes and troubleshoot data quality, performance, and operational issues.
• Develop and maintain CI/CD pipelines using Azure DevOps and Git.
• Collaborate with architects, analysts, engineers, and reporting teams to deliver scalable data solutions.
• Provide production support, monitoring, root cause analysis, and continuous improvement across the data platform.
• Build pipelines and tables that enable the Analytics team to develop dashboards and reports.

Required Skills

• 5+ years of Data Engineering, ETL/ELT, or application development experience.
• Strong hands-on experience with Azure Databricks, Azure Data Factory (ADF), PySpark, Python, and SQL.
• Experience building Databricks notebooks and workspace datasets.
• Azure DevOps, Git, and CI/CD pipeline implementation.
• Experience with enterprise Data Lake and Data Warehouse environments.
• Strong troubleshooting, performance tuning, and production support experience.
• Experience supporting Tableau reporting and analytics solutions.
• Familiarity with Snowflake is highly preferred.

Bonus Skills

• Knowledge of data governance, security, and compliance best practices.
• Experience working in Agile/Scrum environments.
• Familiarity with enterprise monitoring and scheduling tools.
• Experience with Infrastructure-as-Code (IaC) and DevOps methodologies.
• Leadership, mentoring, and technical coaching experience