Remote Databricks Lead & Data Platform Architect

KATBOTZ; United States; Remote; USD 150,000 - 190,000; Full time

Benefits offered by this job

Competitive compensation package

Remote work options

Professional development

Job summary

KATBOTZ is actively seeking an experienced Databricks Subject Matter Expert to drive enterprise data engineering and analytics initiatives in a fully remote role. You will design, optimize, and implement scalable Databricks-based data platforms for global clients, working across cross-functional teams to translate requirements into robust data solutions.

The ideal candidate brings 10+ years in data engineering, deep Databricks and Spark expertise, and strong Python/SQL/Scala skills, with

Qualifications

• 10+ years of experience in Data Engineering and Big Data technologies.
• Deep expertise in Databricks and Apache Spark.
• Proficiency in Python, PySpark, SQL and Scala.
• Expertise in Delta Lake, Unity Catalog, and Databricks Workflows.
• Strong understanding of modern lakehouse architecture and data warehousing.
• Experience with cloud platforms (Azure, AWS, GCP).
• Experience integrating Databricks with BI and reporting tools.
• Knowledge of CI/CD pipelines and DevOps practices.

Responsibilities

• Lead design, development, and optimization of Databricks-based data platforms and pipelines.
• Provide technical expertise on Databricks architecture, performance tuning, and best practices.
• Build scalable ETL/ELT workflows using Apache Spark and Databricks.
• Collaborate with stakeholders to gather requirements and deliver solutions.
• Implement Delta Lake, data governance, and data quality frameworks.
• Support migration of legacy data platforms to Databricks.
• Optimize data processing workflows for performance and cost efficiency.
• Mentor teams on Databricks, Spark, and cloud-native solutions.
• Troubleshoot production issues and provide timely resolutions.
• Ensure adherence to security, compliance, and coding standards.

Skills

Databricks

Spark

Python

SQL

Scala

Delta Lake

Unity Catalog

CI/CD

DevOps

Azure AWS GCP

Tools

Databricks Workflows

CI/CD Tools

Delta Lake Utilities

Job description

KATBOTZ is actively seeking an experienced Databricks Subject Matter Expert to drive enterprise data engineering and analytics initiatives in a fully remote role. You will design, optimize, and implement scalable Databricks-based data platforms for global clients, working across cross-functional teams to translate requirements into robust data solutions.

The ideal candidate brings 10+ years in data engineering, deep Databricks and Spark expertise, and strong Python/SQL/Scala skills, with