Senior Data Engineer - Azure Databricks & PySpark

TechDigital Group; Fort Worth (TX); On-site; USD 100,000 - 130,000; Full time

Job summary

A leading tech consulting firm in Fort Worth, Texas, is seeking a skilled data engineer to design and optimize scalable data pipelines using Azure Databricks and PySpark. The role requires a solid background in SQL and experience with data ingestion and transformation processes. The ideal candidate will collaborate closely with data analysts and stakeholders, ensuring efficient data solutions are delivered. This position offers an exciting opportunity to work in a dynamic tech environment focused on enterprise-level data processing.

Qualifications

• 5+ years' experience in SQL.
• 4+ years of experience in Azure Databricks with PySpark.
• 2+ years of experience in Python programming & package builds.
• Strong understanding of ETL/ELT design patterns, data warehousing, and data lakehouse architectures.

Responsibilities

• Design, develop, and optimize scalable data pipelines leveraging Databricks.
• Write clean, maintainable, and efficient PySpark and Python code.
• Integrate Azure Databricks with various Azure data services.

Skills

SQL

Azure Databricks

PySpark

Azure Cloud

Data ingestion

Data transformation

Python

ETL/ELT design patterns

Data warehouses

GitHub Actions

Job description

A leading tech consulting firm in Fort Worth, Texas, is seeking a skilled data engineer to design and optimize scalable data pipelines using Azure Databricks and PySpark. The role requires a solid background in SQL and experience with data ingestion and transformation processes. The ideal candidate will collaborate closely with data analysts and stakeholders, ensuring efficient data solutions are delivered. This position offers an exciting opportunity to work in a dynamic tech environment focused on enterprise-level data processing.