Databricks Lead

Anblicks; Coppell (TX); On-site; USD 120,000 - 150,000; Full time

Job summary

Anblicks is seeking a data engineer to design and implement end-to-end data architecture on the Databricks Lakehouse platform. You will collaborate with stakeholders to translate business requirements into scalable data solutions and mentor teammates.

The role focuses on building robust ETL/ELT pipelines, applying Delta Lake governance, and optimizing performance and cost across cloud environments (Azure, AWS, or GCP).

Qualifications

• Strong hands-on experience with Databricks Platform.
• Expertise in Apache Spark (PySpark, Scala, SQL).
• Experience with Delta Lake, Unity Catalog, and Lakehouse architecture.
• Proficiency in SQL and data modeling.
• Hands-on experience with cloud platforms.

Responsibilities

• Design and implement end-to-end data architecture using Databricks Lakehouse platform.
• Define data engineering best practices and build scalable ETL/ELT pipelines.
• Architect solutions leveraging Apache Spark (PySpark/Scala) and Databricks workflows.
• Implement Delta Lake for data reliability, performance, and governance.
• Collaborate with stakeholders to translate business requirements into technical solutions.
• Optimize performance, cost, and scalability of data pipelines.
• Provide technical leadership, mentoring data engineers and developers.
• Ensure data security, governance, and compliance standards are met.
• Integrate Databricks with cloud services (Azure, AWS, or GCP) and other systems.
• Support real-time and batch data processing use cases.

Skills

Databricks Platform

Apache Spark (PySpark/Scala/SQL)

Delta Lake

Unity Catalog

Lakehouse architecture

ETL/ELT pipelines

SQL

Data modeling

Cloud platforms (Azure/AWS/GCP)

Job description

• Design and implement end-to-end data architecture using Databricks Lakehouse platform
• Define data engineering best practices and build scalable ETL/ELT pipelines
• Architect solutions leveraging Apache Spark (PySpark/Scala) and Databricks workflows
• Implement Delta Lake for data reliability, performance, and governance
• Collaborate with stakeholders to understand business requirements and translate them into technical solutions
• Optimize performance, cost, and scalability of data pipelines
• Provide technical leadership, mentoring data engineers and developers
• Ensure data security, governance, and compliance standards are met
• Integrate Databricks with cloud services (Azure, AWS, or GCP) and other enterprise systems
• Support real-time and batch data processing use cases

Required Skills & Qualifications:

• Strong hands-on experience with Databricks Platform
• Expertise in Apache Spark (PySpark / Scala / SQL)
• Experience with Delta Lake, Unity Catalog, and Lakehouse architecture
• Proficiency in data pipelines, ETL/ELT frameworks
• Strong experience in SQL and data modeling
• Hands-on experience with cloud platforms: