Databricks Engineer

Noblesoft Technologies; Phoenix (AZ); On-site; USD 120,000 - 180,000; Full time

Job summary

Noblesoft Technologies in Phoenix, AZ is hiring a Databricks Engineer to lead design, implementation and optimization of scalable cloud‑based data platforms on site. The role demands deep expertise in Lakehouse architecture, Databricks, Apache Spark and Delta Lake, plus a track record of delivering end‑to‑end enterprise data engineering solutions.

You will drive data engineering pipelines, engage with cross‑functional teams, and mentor junior engineers while optimizing performance and governance

Qualifications

• Experience designing cloud-based data platforms using Databricks.
• Strong knowledge of Lakehouse architecture and Delta Lake.
• Proven ability to deliver enterprise data engineering projects end-to-end.

Responsibilities

• Design end-to-end data pipelines for batch and streaming workloads.
• Lead lakehouse platform implementations integrating diverse data sources.
• Build real-time data processing pipelines.
• Develop scalable ETL/ELT pipelines using PySpark, Scala, and SQL.
• Establish data quality and governance standards; manage security controls.
• Collaborate with stakeholders to translate requirements into scalable solutions.

Skills

Databricks

Spark

Delta Lake

ETL/ELT pipelines

Data modeling

CI/CD

Data governance

Tools

Databricks

Delta Live Tables

Autoloader

Structured Streaming

Unity Catalog

Spark

PySpark

SQL

Airflow

Job description

Databricks Engineer

Pheonix AZ Day 1 Onsite min 3 days a week.

We are seeking a highly skilled Databricks Engineer to lead the design, implementation, and optimization of scalable cloud-based data platforms. This role requires deep expertise in Lakehouse architecture, Databricks, Apache Spark, and Delta Lake, along with proven experience delivering end-to-end enterprise data engineering solutions.

Key Responsibilities

• Design and deliver end-to-end data engineering pipelines, including batch and real-time streaming solutions.


• Lead implementation of:
  
  
  
  • Cloud-based data lakehouse platforms integrating diverse data sources.
  
  
  • Real-time data processing pipelines for operational and analytical use cases.
  
  
  • Develop scalable ETL/ELT pipelines using PySpark, Scala, and SQL.
  
  
  • Implement advanced data modeling solutions including 3NF, dimensional modeling, and enterprise data warehousing strategies.
  
  
  • Design and build incremental data loading frameworks and metadata-driven ingestion pipelines.
  
  
  • Establish data quality frameworks and governance standards.
  
  
  • Implement and manage Unity Catalog, including fine-grained security and access controls.
  
  
  • Leverage Databricks components such as:
    
    
    
    • Delta Live Tables
    
    
    • Autoloader
    
    
    • Structured Streaming
    
    
    • Databricks Workflows
    
    
    
    
  
  
  • Integration with orchestration tools (e.g., Apache Airflow)
  
  
  • Drive CI/CD automation, deployment strategies, and DevOps best practices.
  
  
  • Optimize performance of pipelines, Spark jobs, and compute resources.
  
  
  • Provide architectural guidance and technical leadership across cross-functional teams.
  
  
  • Engage with stakeholders and clients to translate business requirements into scalable technical solutions.