Databricks Developer

Valiant Technologies LLC

Louisville (KY)

On-site

USD 140,000 - 190,000

Full time

30 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Valiant Technologies LLC in Louisville, KY is seeking an experienced Databricks Developer to design, develop, and optimize scalable data processing solutions on the Databricks Lakehouse Platform. The ideal candidate will bring deep expertise in Spark, PySpark, Spark SQL, and Delta Lake to build enterprise-grade data pipelines.

You will design data architectures, implement governance and security measures, optimize Spark jobs for performance and cost, and collaborate with Data Scientists,

Qualifications

  • Bachelor's degree in CS/IS/Engineering or related field.
  • 8+ years of hands-on Databricks development experience.
  • Strong expertise in Apache Spark and the Databricks Lakehouse Platform.
  • Hands-on experience developing data pipelines using PySpark and Spark SQL.
  • Experience with Delta Lake and its capabilities.

Responsibilities

  • Design, develop, and maintain scalable batch and real-time data pipelines using Databricks.
  • Build robust data processing solutions using PySpark, Spark SQL, and Delta Lake.
  • Implement data validation, data quality checks, and performance-tuning strategies.
  • Design and manage data architecture within the Databricks Lakehouse Platform.
  • Implement and manage Delta Lake tables, partitioning, optimization, and schema evolution.
  • Develop data models to support analytics and reporting requirements.
  • Ensure data reliability, scalability, and governance.
  • Collaborate with Data Architects, Data Scientists, Business Analysts, and Application Teams.

Skills

Databricks development
Apache Spark
PySpark
Spark SQL
Delta Lake

Education

Bachelor's degree in Computer Science, Information Systems, Engineering, or related field

Tools

Databricks Lakehouse Platform
Delta Lake

Job description

US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other

Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other

Position Summary

We are seeking an experienced Databricks Developer to design, develop, and optimize scalable data processing solutions using the Databricks Lakehouse Platform. The ideal candidate will have strong expertise in Apache Spark, PySpark, Spark SQL, and Delta Lake, along with experience building high-performance data pipelines and supporting enterprise-scale data modernization and analytics initiatives.

  • Design, develop, and maintain scalable batch and real-time data pipelines using Databricks.
  • Build robust data processing solutions using PySpark, Spark SQL, and Delta Lake.
  • Implement data validation, data quality checks, and performance-tuning strategies.
Data Lakehouse Management
  • Design and manage data architecture within the Databricks Lakehouse Platform.
  • Implement and manage Delta Lake tables, partitioning, optimization, and schema evolution.
  • Develop data models to support analytics and reporting requirements.
  • Ensure data reliability, scalability, and governance.
Performance Optimization
  • Analyze and optimize Spark jobs for improved performance and cost efficiency.
  • Monitor cluster utilization and apply resource management best practices.
  • Identify and troubleshoot bottlenecks in data processing workflows.
  • Improve processing times through caching, partitioning, and query optimization.
Data Governance & Security
  • Implement role-based access controls and data security standards.
  • Support data governance initiatives, data lineage tracking, and compliance requirements.
  • Ensure adherence to enterprise data management policies.
  • Collaborate with Data Architects, Data Scientists, Business Analysts, and Application Teams.
  • Participate in Agile development processes and sprint ceremonies.
  • Prepare technical design documentation and support knowledge transfer activities.
Required Qualifications
  • Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field.
  • 8+ years of hands-on Databricks development experience.
  • Strong expertise in Apache Spark and the Databricks Lakehouse Platform.
  • Hands-on experience developing data pipelines using PySpark and Spark SQL.
  • Experience with Delta Lake and its capabilities.
Experience Requirements
  • Overall Experience:10+ years.
  • Databricks Development Experience: 5+ years.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Databricks Developer
Databricks Developer

Ethereum Technologies LLC • Louisville (KY)

On-site
USD 140,000 - 190,000
Databricks Developer
Databricks Developer

Quantum Technologies LLC • Louisville (KY)

On-site
USD 140,000 - 180,000
Databricks Developer
Databricks Developer

Robotics Technologies • Louisville (KY)

On-site
USD 90,000 - 130,000
Databricks Developer
Databricks Developer

CloudBigD • Louisville (KY)

On-site
USD 120,000 - 180,000
Databricks Developer
Databricks Developer

Robotic Process Automation LLC • Louisville (KY)

On-site
USD 110,000 - 170,000
Databricks Engineer
Databricks Engineer

Ethereum Technologies LLC • Austin (TX)

On-site
USD 130,000 - 185,000
Databricks Engineer
Databricks Engineer

Quantum Technologies LLC • Austin (TX)

On-site
USD 120,000 - 190,000
Databricks Engineer
Databricks Engineer

Valiant Technologies LLC • Austin (TX)

On-site
USD 130,000 - 190,000
Sr. Databricks Developer
Sr. Databricks Developer

Ethereum Technologies LLC • Columbus (OH)

On-site
USD 140,000 - 180,000
Databricks Data Engineer
Databricks Data Engineer

Ethereum Technologies LLC • Dallas (TX)

On-site
USD 120,000 - 170,000