Lead Software Engineer - DataBricks, Spark, Terraform (Corporate - Employee Platforms)

ARCHER

Chicago (IL)

On-site

USD 150,000 - 210,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health Care Coverage
On-Site Health And Wellness Centers
Retirement Savings Plan
Backup Childcare
Tuition Reimbursement
Mental Health Support
Financial Coaching

Job summary

ARCHER in Chicago seeks a senior data platform architect to lead the Databricks lakehouse roadmap, owning data ingestion, modeling, governance, security, reliability, performance and cost management.

You will establish engineering standards, mentor engineers and guide teams in the safe, effective use of AI-assisted development tools and Terraform-related features.

Qualifications

  • 7+ years of data or platform engineering experience
  • Production Databricks expertise
  • Strong Spark and SQL skills
  • Experience building, operating and optimizing data platforms
  • Demonstrated governance, observability and technical leadership
  • Experience with AI-assisted development practices; Terraform and platform features preferred

Responsibilities

  • Lead the technical roadmap and architecture for the Databricks lakehouse platform (ingestion, modeling, governance, security, reliability, performance, cost)
  • Establish engineering standards and operational practices
  • Mentor engineers and guide teams in safe use of AI-assisted development tools

Skills

Databricks
Apache Spark
PySpark
Scala
SQL
Terraform
Delta Lake
Unity Catalog
Data Engineering
Data Modeling
Data Governance
Performance Tuning
CI/CD
Automated Testing
Data Observability
AI-Assisted Development

Tools

Terraform

Job description

Lead the technical roadmap and architecture for the Databricks lakehouse platform, including data ingestion, modeling, governance, security, reliability, performance, and cost management. Establish engineering standards and operational practices, mentor engineers, and guide teams in the safe, effective use of approved AI-assisted development tools.

Requirements: Requires 7+ years of data or platform engineering experience, production Databricks expertise, strong Spark and SQL skills, and experience building, operating, and optimizing data platforms. Candidates should also demonstrate software engineering, governance, observability, technical leadership, and responsible AI-assisted development practices; Terraform and other listed platform features are preferred.

Key Skills: Databricks, Apache Spark, PySpark, Scala, SQL, Terraform, Delta Lake, Unity Catalog, Data Engineering, Data Modeling, Data Governance, Performance Tuning, CI/CD, Automated Testing, Data Observability, AI-Assisted Development

Benefits: Health Care Coverage, On-Site Health And Wellness Centers, Retirement Savings Plan, Backup Childcare, Tuition Reimbursement, Mental Health Support, Financial Coaching

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Software Engineer - Databricks (Corporate Technology)
Lead Software Engineer - Databricks (Corporate Technology)

ARCHER • Jersey City (NJ)

On-site
USD 140,000 - 210,000
Health Care Coverage
On-Site Health And Wellness Centers
Retirement Savings Plan
+4
Principal Software Engineer - Databricks (Chief Data Analytics Office - AIML Data Platforms)
Principal Software Engineer - Databricks (Chief Data Analytics Office - AIML Data Platforms)

ARCHER • Jersey City (NJ)

On-site
USD 180,000 - 260,000
Health Care Coverage
On-Site Health And Wellness Centers
Retirement Savings Plan
+4
Lead Software Engineer - Databricks/Snowflake/AWS (Corporate Technology)
Lead Software Engineer - Databricks/Snowflake/AWS (Corporate Technology)

ARCHER • Plano (TX)

On-site
USD 140,000 - 180,000
Health Care Coverage
On-Site Health And Wellness Centers
Retirement Savings Plan
+4
Lead Data Engineer
Lead Data Engineer

Astrana Health • United States

Remote
USD 140,000 - 190,000
Databricks Integration Engineer
Databricks Integration Engineer

Valiant Technologies LLC • Boston (MA)

On-site
USD 180,000 - 260,000
Databricks Integration Engineer
Databricks Integration Engineer

Quantum Technologies LLC • Boston (MA)

On-site
USD 180,000 - 260,000
Senior Software Engineer (Databricks Platform Engineer)
Senior Software Engineer (Databricks Platform Engineer)

Vizient • Irving (TX)

On-site
USD 102,400 - 179,000
Incentive eligible
Databricks Data Platform Architect
Databricks Data Platform Architect

UsefulBI Corporation • Raleigh (NC)

On-site
USD 130,000 - 180,000
Lead Databricks Data Engineer
Lead Databricks Data Engineer

Fa Ewjt Saasfaprod1 • New York (NY)

On-site
USD 140,000 - 170,000
Lead Databricks Data Engineer
Lead Databricks Data Engineer

EXL Talent Acquisition Team • New York (NY)

On-site
USD 180,000 - 240,000