Senior Databricks Engineer-AWS Lakehouse Architect (Hybrid)

SMBC; New York (NY); Hybrid; USD 97,000 - 156,000; Full time

Benefits offered by this job

Hybrid work model

Accommodations available

Job summary

SMBC seeks a Senior Databricks Engineer with deep AWS expertise to build and maintain enterprise data Lakehouse infrastructure. You will design high-performance data pipelines, implement regulatory and real-time reporting, and ensure seamless Databricks–AWS integration for real-time financial data consumption.

Responsibilities include designing robust pipelines, automating deployments, integrating with S3 and IAM, and collaborating with data scientists and risk managers to deliver clean data

Qualifications

• 4+ years of hands-on experience in architectural design and development within the Databricks platform.
• 3+ years working in an AWS cloud environment.
• Proficiency in automated deployments using DABs, Terraform, and Git pipelines.

Responsibilities

• Design and implement Lakehouse architecture with Databricks Medallion Architecture (Bronze/Silver/Gold).
• Develop, scale, and orchestrate data workflows using Databricks Jobs and Delta Live Tables.

Skills

Databricks

AWS

Python (PySpark)

SQL

CI/CD

Terraform

Git pipelines

Education

Databricks/AWS certification

Tools

Databricks Asset Bundles

Terraform Databricks provider

GitHub Actions

Job description

SMBC seeks a Senior Databricks Engineer with deep AWS expertise to build and maintain enterprise data Lakehouse infrastructure. You will design high-performance data pipelines, implement regulatory and real-time reporting, and ensure seamless Databricks–AWS integration for real-time financial data consumption.

Responsibilities include designing robust pipelines, automating deployments, integrating with S3 and IAM, and collaborating with data scientists and risk managers to deliver clean data