Remote Databricks Engineer - Delta Lake + PySpark Expert

StaffXpert LLC; Birmingham (AL); Hybrid; USD 120,000 - 180,000; Full time

Job summary

STAFFXPERT LLC is seeking a Databricks Engineer to design, develop, and optimize enterprise-grade data solutions on the Databricks platform. The role emphasizes PySpark, SQL, Delta Lake, and data quality frameworks, with production-grade delivery.

Remote/Hybrid arrangement supported; candidate will collaborate with cross-functional teams to modernize data workflows and migrate from legacy ETL tools, ensuring scalable, reliable data pipelines and dashboards using Databricks SQL.

Qualifications

• Bachelor's degree or equivalent experience in CS/IT/Engineering.
• Strong hands-on experience with Databricks, PySpark, SQL, and Delta Lake.
• Proven experience building, optimizing, and maintaining large-scale data pipelines.
• Expertise in data quality, validation, monitoring, and governance frameworks.
• Experience creating dashboards and reporting using Databricks SQL.

Responsibilities

• Design, develop, and optimize Delta Lake data pipelines using PySpark and SQL.
• Implement data transformation processes including cleansing, deduplication, validation and MERGE/upsert operations.
• Build and maintain data quality frameworks, monitoring solutions, and reporting mechanisms.
• Develop Databricks SQL dashboards and AI/BI reporting to track data quality metrics.
• Migrate workflows from legacy ETL tools to Databricks using PySpark, SQL, and Lakeflow Jobs.
• Collaborate with cross-functional teams to deliver high-quality data solutions.
• Perform performance tuning and troubleshooting to improve reliability and efficiency of data pipelines.
• Provide technical guidance and knowledge transfer to internal and client teams.

Skills

Databricks

PySpark

SQL

Delta Lake

Data quality

Data governance

Databricks SQL

Education

Bachelor's degree in Computer Science, Information Technology, Engineering

Tools

Lakeflow Jobs

Lakehouse tooling

Job description

STAFFXPERT LLC is seeking a Databricks Engineer to design, develop, and optimize enterprise-grade data solutions on the Databricks platform. The role emphasizes PySpark, SQL, Delta Lake, and data quality frameworks, with production-grade delivery.

Remote/Hybrid arrangement supported; candidate will collaborate with cross-functional teams to modernize data workflows and migrate from legacy ETL tools, ensuring scalable, reliable data pipelines and dashboards using Databricks SQL.