Lead Software Engineer - Databricks (Corporate Technology)

ARCHER; Jersey City (NJ); On-site; USD 140,000 - 210,000; Full time

Benefits offered by this job

Health Care Coverage

On-Site Health And Wellness Centers

Retirement Savings Plan

Backup Childcare

Tuition Reimbursement

Mental Health Support

Financial Coaching

Job summary

ARCHER is seeking an experienced Data Platform Engineer to architect and operate secure, scalable Databricks Lakehouse platforms on AWS. You will deliver batch and streaming pipelines, governance, and reliable production workloads while mentoring engineers and coordinating with security and platform teams.

The role requires leadership in engineering standards, design reviews, and cost-effective, maintainable data solutions across multi-team initiatives.

Qualifications

• 7+ years of data engineering experience.
• 2+ years leading delivery or architecture for multi-team data platforms.
• Hands-on Databricks, Delta Lake, Spark, and AWS expertise.
• Experience with governance, production reliability, CI/CD and Terraform, testing, performance and cost optimization, and technical leadership.

Responsibilities

• Architect and operate secure, scalable Databricks Lakehouse platforms on AWS, delivering batch and streaming data pipelines, governance, and reliable production workloads.
• Lead engineering standards and design reviews, mentor engineers, and coordinate with stakeholders and security and platform teams to deliver maintainable, cost-effective data solutions.

Skills

Data Engineering

Databricks

AWS

Delta Lake

Apache Spark

Structured Streaming

Data Pipeline Architecture

Data Governance

Unity Catalog

Terraform

CI/CD

Data Quality Testing

Incident Response

Performance And Cost Optimization

Technical Leadership

Mentoring

Job description

Architect and operate secure, scalable Databricks Lakehouse platforms on AWS, delivering batch and streaming data pipelines, governance, and reliable production workloads. Lead engineering standards and design reviews, mentor engineers, and coordinate with stakeholders and security and platform teams to deliver maintainable, cost-effective data solutions.

Requirements:

Requires 7+ years of data engineering experience, including 2+ years leading delivery or architecture for multi-team data platforms, with hands-on Databricks, Delta Lake, Spark, and AWS expertise. Candidates should also have experience with governance, production reliability, CI/CD and Terraform, testing, performance and cost optimization, and technical leadership.

Key Skills:

• Data Engineering
• Databricks
• AWS
• Delta Lake
• Apache Spark
• Structured Streaming
• Data Pipeline Architecture
• Data Governance
• Unity Catalog
• Terraform
• CI/CD
• Data Quality Testing
• Incident Response
• Performance And Cost Optimization
• Technical Leadership
• Mentoring

Benefits:

• Health Care Coverage
• On-Site Health And Wellness Centers
• Retirement Savings Plan
• Backup Childcare
• Tuition Reimbursement
• Mental Health Support
• Financial Coaching