Senior Databricks Engineer: Spark, Delta Lake & Lakehouse

Valiant Technologies LLC

Louisville (KY)

On-site

USD 140,000 - 190,000

Full time

4 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Valiant Technologies LLC in Louisville, KY is seeking an experienced Databricks Developer to design, develop, and optimize scalable data processing solutions on the Databricks Lakehouse Platform. The ideal candidate will bring deep expertise in Spark, PySpark, Spark SQL, and Delta Lake to build enterprise-grade data pipelines.

You will design data architectures, implement governance and security measures, optimize Spark jobs for performance and cost, and collaborate with Data Scientists,

Qualifications

  • Bachelor's degree in CS/IS/Engineering or related field.
  • 8+ years of hands-on Databricks development experience.
  • Strong expertise in Apache Spark and the Databricks Lakehouse Platform.
  • Hands-on experience developing data pipelines using PySpark and Spark SQL.
  • Experience with Delta Lake and its capabilities.

Responsibilities

  • Design, develop, and maintain scalable batch and real-time data pipelines using Databricks.
  • Build robust data processing solutions using PySpark, Spark SQL, and Delta Lake.
  • Implement data validation, data quality checks, and performance-tuning strategies.
  • Design and manage data architecture within the Databricks Lakehouse Platform.
  • Implement and manage Delta Lake tables, partitioning, optimization, and schema evolution.
  • Develop data models to support analytics and reporting requirements.
  • Ensure data reliability, scalability, and governance.
  • Collaborate with Data Architects, Data Scientists, Business Analysts, and Application Teams.

Skills

Databricks development
Apache Spark
PySpark
Spark SQL
Delta Lake

Education

Bachelor's degree in Computer Science, Information Systems, Engineering, or related field

Tools

Databricks Lakehouse Platform
Delta Lake

Job description

Valiant Technologies LLC in Louisville, KY is seeking an experienced Databricks Developer to design, develop, and optimize scalable data processing solutions on the Databricks Lakehouse Platform. The ideal candidate will bring deep expertise in Spark, PySpark, Spark SQL, and Delta Lake to build enterprise-grade data pipelines.

You will design data architectures, implement governance and security measures, optimize Spark jobs for performance and cost, and collaborate with Data Scientists,

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Databricks Engineer – Lakehouse Pipelines & Spark
Senior Databricks Engineer – Lakehouse Pipelines & Spark

Valiant Technologies LLC • Austin (TX)

On-site
USD 130,000 - 190,000
Senior Data Engineer: Spark, Databricks & Data Lakehouse
Senior Data Engineer: Spark, Databricks & Data Lakehouse

Valiant Technologies LLC • Charlotte (NC)

On-site
USD 120,000 - 190,000
Senior Databricks Engineer: Spark, Delta Lake Pipelines
Senior Databricks Engineer: Spark, Delta Lake Pipelines

Quantum Technologies. LLC • Louisville (KY)

On-site
USD 140,000 - 180,000
Senior Databricks Engineer: Spark, Delta Lake & Pipelines
Senior Databricks Engineer: Spark, Delta Lake & Pipelines

Ethereum Technologies • Louisville (KY)

On-site
USD 140,000 - 190,000
Senior Databricks Data Engineer - Lakehouse & Streaming
Senior Databricks Data Engineer - Lakehouse & Streaming

Valiant Technologies LLC • Columbus (OH)

On-site
USD 120,000 - 190,000
Senior Databricks Architect - AWS Lakehouse Leader
Senior Databricks Architect - AWS Lakehouse Leader

Valiant Technologies LLC • Atlanta (GA)

On-site
USD 170,000 - 210,000
Senior Databricks Developer - Lakehouse & Spark Expert
Senior Databricks Developer - Lakehouse & Spark Expert

Robotic Technologies Automation • Louisville (KY)

On-site
USD 110,000 - 170,000
Databricks Data Engineer: Lakehouse & Spark Expert
Databricks Data Engineer: Lakehouse & Spark Expert

CloudBigD • Louisville (KY)

On-site
USD 120,000 - 180,000
Senior Databricks Lakehouse Architect (PySpark/ETL)
Senior Databricks Lakehouse Architect (PySpark/ETL)

VService IT Solutions • United States

Remote
USD 140,000 - 190,000
Senior Databricks Lakehouse Engineer
Senior Databricks Lakehouse Engineer

Robotics Technologies LLC • Louisville (KY)

On-site
USD 90,000 - 130,000