Azure Databricks Data Engineer - Scalable Pipelines Lead

Artifintech

Houston (TX)

On-site

USD 130,000 - 170,000

Full time

13 hours ago

Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

ARTIFICIAL INTELLIGENCE TECHNOLOGIES LLC is seeking a Specialist in Houston, TX with 7 years of experience designing and developing scalable data solutions using Azure Databricks. You will design, develop, and optimize data pipelines to support enterprise-level analytics and data processing.

Collaborate with cross-functional teams to align data solutions with business objectives and ensure high data quality and reliability.

Qualifications

7+ years of experience designing and developing scalable data solutions with Azure Databricks.
Proficient in PySpark and Python for ETL, ingestion, transformation and analytics.
Experience designing scalable data pipelines and integrating Azure data services.
Ability to collaborate with cross-functional teams and align data solutions with business goals.

Responsibilities

Lead design, development and optimization of complex data pipelines on Azure Databricks.
Write clean PySpark and Python code for data ingestion, transformation and analytics.
Troubleshoot pipelines for minimal downtime and high availability.
Document data processes, architectures and workflows for knowledge sharing.
Mentor junior engineers and participate in design reviews and agile ceremonies.