Lead Data Engineer - Cloud Databricks Migration & Lakehouse
TechDigital Group
Jersey City (NJ)
On-site
USD 100,000 - 130,000
Full time
14 days+
Get a reply from this recruiter
Your resume and cover letter, tailored to exactly what they’re hiring for.
Get past ATS filters
Job summary
TechDigital Group is looking for a skilled data engineer in Jersey City, NJ, to lead the migration of legacy data platforms to cloud-native solutions using Databricks. The ideal candidate will have 8–12 years of experience in data engineering and strong expertise in Apache Spark and PySpark. Responsibilities include designing Databricks architectures, ensuring data quality and governance, and implementing cloud security measures. This role offers the opportunity to work on large-scale transformative projects in a collaborative environment.
Qualifications
8–12+ years in data engineering/platform modernization.
Strong hands-on experience with Databricks in large-scale enterprises.
Deep expertise in Apache Spark, PySpark, Spark SQL.
Responsibilities
Lead and execute migration of legacy data platforms to Databricks.
Design and implement Databricks Lakehouse architecture.
Embed data quality and governance controls in migrations.
Implement cloud security and resilience measures.
Skills
Data engineering experience
Databricks expertise
Apache Spark proficiency
Cloud migration experience
Data quality controls
Job description
Role Overview This role is part of a multi‐year enterprise initiative to modernize data platforms by migrating from legacy and on‐prem environments to cloud‐native, governed, and scalable architectures . The role focuses on migrating enterprise data workloads to Databricks on strategic cloud platforms , enabling standardized data engineering, analytics, centralized reporting, reconciliation utilities, and AI/ML use cases—while adhering to controls, security, resilience, and regulatory compliance .
Key Responsibilities
Lead and execute migration of legacy data platforms (on‐prem / non‐standard tools) to Databricks on cloud under the Olympus program
Perform application, data, and pipeline refactoring to cloud‐native Databricks patterns
Drive migration planning including dependency analysis, sequencing, and cutover strategy
Support coexistence models and transition from dual‐run to cloud‐only execution