Remote Lead Data Engineer: Databricks & Data Lakes

INSPYR Solutions; Georgia; Hybrid; USD 96,432 - 117,096; Full time

Benefits offered by this job

Comprehensive medical benefits

401(k) retirement plan

Competitive pay

Job summary

A technology solutions firm is looking for a Lead Data Engineer to work remotely on building end-to-end data solutions. The ideal candidate will design and maintain scalable data pipelines and collaborate with AI/ML teams. The position requires strong skills in Databricks, Python, and SQL, along with relevant experience in cloud environments. Competitive pay and comprehensive benefits are offered.

Qualifications

• Strong hands-on experience with Databricks, including Spark, Delta Lake, and MLflow.
• Proficiency in Python, SQL, and distributed data processing.
• Experience with cloud-native data services.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks.
• Implement robust ETL workflows using Databricks notebooks, SQL, and Python.
• Develop CI/CD pipelines for data and ML workflows using Databricks Repos.

Skills

Hands-on experience with Databricks

Proficiency in Python

Proficiency in SQL

Experience with cloud-native data services

Excellent communication skills

Education

Bachelor's or Master's degree in Computer Science or related field

Tools

Databricks

Apache Spark

Delta Lake

Job description

A technology solutions firm is looking for a Lead Data Engineer to work remotely on building end-to-end data solutions. The ideal candidate will design and maintain scalable data pipelines and collaborate with AI/ML teams. The position requires strong skills in Databricks, Python, and SQL, along with relevant experience in cloud environments. Competitive pay and comprehensive benefits are offered.