Databricks Lead

Anblicks; Dallas (TX); On-site; USD 150,000 - 190,000; Full time

Job summary

Anblicks is seeking a senior data engineer to design and implement end-to-end data architecture on the Databricks Lakehouse platform in Dallas, TX. You will define best practices and build scalable ETL/ELT pipelines, leveraging Spark and Databricks workflows to deliver robust data solutions.

Responsibilities include implementing Delta Lake for reliability and governance, collaborating with stakeholders, and mentoring a team of engineers while ensuring security and compliance with enterprise

Qualifications

• Strong hands-on experience with Databricks Platform.
• Expertise in Apache Spark (PySpark / Scala / SQL).
• Experience with Delta Lake, Unity Catalog, and Lakehouse architecture.
• Proficiency in data pipelines, ETL/ELT frameworks.
• Strong experience in SQL and data modeling.

Responsibilities

• Design and implement end-to-end data architecture on Databricks Lakehouse.
• Define data engineering best practices and build scalable ETL/ELT pipelines.
• Architect solutions leveraging Apache Spark and Databricks workflows.
• Implement Delta Lake for data reliability, performance, and governance.
• Collaborate with stakeholders to translate business requirements into technical solutions.
• Optimize performance, cost, and scalability of data pipelines.
• Provide technical leadership and mentorship to data engineers and developers.
• Ensure data security, governance, and compliance standards are met.
• Integrate Databricks with cloud services (Azure, AWS, or GCP) and other ecosystems.
• Support real-time and batch data processing use cases.

Skills

Databricks Platform

Apache Spark

Delta Lake

SQL & Data Modeling

ETL/ELT

Cloud Platforms

Job description

• Design and implement end-to-end data architecture using Databricks Lakehouse platform
• Define data engineering best practices and build scalable ETL/ELT pipelines
• Architect solutions leveraging Apache Spark (PySpark/Scala) and Databricks workflows
• Implement Delta Lake for data reliability, performance, and governance
• Collaborate with stakeholders to understand business requirements and translate them into technical solutions
• Optimize performance, cost, and scalability of data pipelines
• Provide technical leadership, mentoring data engineers and developers
• Ensure data security, governance, and compliance standards are met
• Integrate Databricks with cloud services (Azure, AWS, or GCP) and other enterprise systems
• Support real-time and batch data processing use cases

Required Skills & Qualifications

Technical Skills

• Strong hands-on experience with Databricks Platform
• Expertise in Apache Spark (PySpark / Scala / SQL)
• Experience with Delta Lake, Unity Catalog, and Lakehouse architecture
• Proficiency in data pipelines, ETL/ELT frameworks
• Strong experience in SQL and data modeling
• Hands-on experience with cloud platforms: