Director, Enterprise Data Architecture & Databricks Platform

Evercore Inc.; New York (NY); On-site; USD 210,000 - 230,000; Full time

Benefits offered by this job

Medical insurance

401(k)

Life insurance

Well-being programs

Family benefits

Parental leave

Commuter benefits

Paid holidays

Job summary

Evercore seeks a Director, Lead, Enterprise Data Architecture to serve as the technical authority for data engineering standards and platform architecture within Global Technology’s AI, Data & Analytics organization in New York. This high-impact role focuses on Databricks, Spark, enterprise data governance, and secure, scalable data pipelines.

The candidate will partner with business, security, and infrastructure to align with long-term enterprise objectives and ensure compliance across

Qualifications

• 10+ years in data engineering or enterprise data platform architecture
• 5+ years designing and scaling Databricks/Spark platforms in enterprise
• Deep Apache Spark expertise and tuning
• Experience with Azure cloud infrastructure
• Experience with event-driven streaming systems (Kafka, pub/sub)
• Designing ETL/ELT frameworks and internal SDKs
• Unity Catalog or similar data governance tooling
• Experience in regulated environments with compliance/audit readiness
• Bachelor’s degree in CS/Engineering; advanced degree preferred
• US work authorization (no visa sponsorship)

Responsibilities

• Define and govern enterprise architecture standards for batch and streaming data pipelines on Azure Databricks
• Design and maintain standardized internal SDKs for ETL development
• Establish data engineering SDLC, including schema evolution, versioning, CI/CD
• Architect and administer secure, scalable Databricks environments with RBAC and Unity Catalog
• Design and implement enterprise metadata, observability, and data quality frameworks
• Establish data product standards including schema contracts and versioning
• Define best practices for exploratory and non-production workloads
• Collaborate with Data Scientists and ML Engineers on lifecycle from experimentation to production

Skills

Databricks

Spark

Data governance

Azure cloud

Regulated environments

Education

Bachelor's in CS/Engineering

Tools

Unity Catalog

SDKs

CI/CD

Event Hub / Kafka

Job description

Evercore seeks a Director, Lead, Enterprise Data Architecture to serve as the technical authority for data engineering standards and platform architecture within Global Technology’s AI, Data & Analytics organization in New York. This high-impact role focuses on Databricks, Spark, enterprise data governance, and secure, scalable data pipelines.

The candidate will partner with business, security, and infrastructure to align with long-term enterprise objectives and ensure compliance across