Databricks Engineer & Data Platform Lead (Phoenix Onsite)

Conquest Tech Solutions Inc; Arizona; On-site; USD 140,000 - 190,000; Full time

Job summary

Conquest Tech Solutions Inc in the United States is seeking a Databricks Engineer to lead the design, implementation, and optimization of scalable cloud-based data platforms. This role requires deep expertise in Lakehouse architecture, Databricks, Apache Spark, and Delta Lake, with a track record of delivering end-to-end enterprise data solutions.

You will design end-to-end pipelines, build real-time and batch processing, implement data quality and governance, and drive CI/CD automation.

Qualifications

• End-to-end data platform design and implementation experience.
• Hands-on expertise with Spark-based batch and streaming pipelines.
• Strong knowledge of data modeling and warehouse strategies.

Responsibilities

• Design and deliver end-to-end data engineering pipelines, including batch and real-time streaming.
• Lead implementation of cloud-based data lakehouse platforms.
• Develop ETL/ELT pipelines using PySpark, Scala, and SQL.
• Implement data quality frameworks and governance standards.
• Manage Unity Catalog and fine-grained security.
• Drive CI/CD automation and DevOps practices for pipelines.
• Collaborate with stakeholders to translate business requirements into scalable solutions.

Skills

Databricks

Apache Spark

Delta Lake

Lakehouse architecture

PySpark

Scala

SQL

Python

CI/CD

DevOps

Tools

Delta Live Tables

Autoloader

Structured Streaming

Databricks Workflows

Airflow

Unity Catalog

Job description

Conquest Tech Solutions Inc in the United States is seeking a Databricks Engineer to lead the design, implementation, and optimization of scalable cloud-based data platforms. This role requires deep expertise in Lakehouse architecture, Databricks, Apache Spark, and Delta Lake, with a track record of delivering end-to-end enterprise data solutions.

You will design end-to-end pipelines, build real-time and batch processing, implement data quality and governance, and drive CI/CD automation.