Databricks Lakehouse Engineer: Migration & Optimization

Kforce Inc; Purchase (NY); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

401(k)

Job summary

Kforce Inc is seeking a mid-level Databricks Engineer in Purchase, NY to join our data engineering team. You will drive migration of Hadoop workloads to the Databricks Lakehouse, develop scalable data solutions, and optimize performance with cross-functional partners.

The ideal candidate has 5–8 years in data engineering, strong Spark (Scala or PySpark), and experience with Delta Lake and Databricks features. This role emphasizes collaboration, architecture, and governance across teams.

Qualifications

• Bachelor's degree in Computer Science, IT, or related field or equivalent experience.
• 5–8 years in data engineering with Databricks and Hadoop.
• Strong Spark (Scala or PySpark) and ETL/data migration skills.
• Experience with Lakehouse architecture and Databricks features a plus.

Responsibilities

• Identify Hadoop jobs and data sources to plan migrations.
• Select scalable migration use cases for MVP initiatives.
• Configure Databricks workspaces and Lakehouse architecture.
• Pilot migrations of Hadoop workloads and monitor post-migration performance.
• Enable advanced Databricks features to optimize data layout and pipelines.
• Collaborate with data teams to gather feedback and document learnings.
• Track migration metrics and contribute to a phased migration roadmap.

Skills

Cross-functional collaboration

Spark (Scala or PySpark)

Data migration

Databricks

Hadoop ecosystem

Azure cloud

Problem solving

Education

Bachelor's degree in CS/IT or related field

Tools

Databricks

Hadoop

Delta Lake

Unity Catalog

Databricks Migration Accelerator

Job description

Kforce Inc is seeking a mid-level Databricks Engineer in Purchase, NY to join our data engineering team. You will drive migration of Hadoop workloads to the Databricks Lakehouse, develop scalable data solutions, and optimize performance with cross-functional partners.

The ideal candidate has 5–8 years in data engineering, strong Spark (Scala or PySpark), and experience with Delta Lake and Databricks features. This role emphasizes collaboration, architecture, and governance across teams.