Experienced Director, Lead, Enterprise Data Architecture

Evercore Inc.; New York (NY); On-site; USD 210,000 - 230,000; Full time

Benefits offered by this job

Medical insurance

401(k)

Life insurance

Well-being programs

Family benefits

Parental leave

Commuter benefits

Paid holidays

Job summary

Evercore seeks a Director, Lead, Enterprise Data Architecture to serve as the technical authority for data engineering standards and platform architecture within Global Technology’s AI, Data & Analytics organization in New York. This high-impact role focuses on Databricks, Spark, enterprise data governance, and secure, scalable data pipelines.

The candidate will partner with business, security, and infrastructure to align with long-term enterprise objectives and ensure compliance across

Qualifications

• 10+ years in data engineering or enterprise data platform architecture
• 5+ years designing and scaling Databricks/Spark platforms in enterprise
• Deep Apache Spark expertise and tuning
• Experience with Azure cloud infrastructure
• Experience with event-driven streaming systems (Kafka, pub/sub)
• Designing ETL/ELT frameworks and internal SDKs
• Unity Catalog or similar data governance tooling
• Experience in regulated environments with compliance/audit readiness
• Bachelor’s degree in CS/Engineering; advanced degree preferred
• US work authorization (no visa sponsorship)

Responsibilities

• Define and govern enterprise architecture standards for batch and streaming data pipelines on Azure Databricks
• Design and maintain standardized internal SDKs for ETL development
• Establish data engineering SDLC, including schema evolution, versioning, CI/CD
• Architect and administer secure, scalable Databricks environments with RBAC and Unity Catalog
• Design and implement enterprise metadata, observability, and data quality frameworks
• Establish data product standards including schema contracts and versioning
• Define best practices for exploratory and non-production workloads
• Collaborate with Data Scientists and ML Engineers on lifecycle from experimentation to production

Skills

Databricks

Spark

Data governance

Azure cloud

Regulated environments

Education

Bachelor's in CS/Engineering

Tools

Unity Catalog

SDKs

CI/CD

Event Hub / Kafka

Job description

Experienced Director, Lead, Enterprise Data Architecture

Region Americas

Location New York

Group Global Technology

What is the Position?
Evercore seeks to hire Director, Lead, Enterprise Data Architecture to serve as the technical authority for data engineering standards and platform architecture within Global Technology’s AI, Data & Analytics organization. This role is responsible for defining and governing how data pipelines, data products, and AI workloads are architected, built, secured, and operated on the firm’s Databricks platform.

This high-impact role sits at the intersection of platform engineering, distributed systems design, and developer experience. The ideal candidate will bring deep expertise in Databricks, Spark, enterprise data governance and event-driven architectures for regulated environments.

Primary Responsibilities:
We are seeking an experienced technical leader with deep Databricks expertise to define strategy and establish architectural standards across the platform. This role will partner closely with business, technology, security, and infrastructure leaders to ensure the platform is scalable, compliant, cost-efficient, and aligned to long-term enterprise objectives. Responsibilities include, but are not limited to the following:

• Define and govern enterprise architecture standards for batch and streaming data pipelines on Azure Databricks, including Spark optimization, structured streaming, and pub/sub integration (e.g., Event Hub, Kafka).
• Design and maintain standardized internal SDKs and reusable frameworks for ETL development.
• Establish the data engineering SDLC, including schema evolution policies, artifact versioning, environment promotion controls and CI/CD standards.
• Architect and administer secure, scalable Databricks environments, including workspace segmentation strategies, Unity Catalog design, role-based and attribute-based access controls, row/column-level security, and compute isolation policies.
• Design and implement enterprise metadata, observability, and data quality frameworks, including automated lineage capture, SLA monitoring, operational telemetry, and cost attribution models.
• Establish data product standards, including schema contract enforcement, versioning policies, backward compatibility requirements, domain ownership models, and publish/subscribe consumption patterns.
• Define best practices for exploratory and non-production workloads, including guardrails for data scientist experimentation, compute policy controls, secure data access boundaries, and promotion of validated assets into production pipelines.
• Partner with Data Scientists and Machine Learning Engineers to define the lifecycle from experimentation to production, including feature engineering standards, model registry governance, reproducibility controls, and approved deployment pathways.

Specific Qualifications:

• Minimum 10+ years of experience in data engineering, enterprise data platform architecture, or large-scale analytics platforms.
• Minimum 5+ years of hands-on experience designing and scaling Databricks and Spark-based platforms in enterprise environments.
• Deep expertise in Apache Spark (performance tuning, optimization strategies, structured streaming, distributed compute patterns).
• Experience architecting Azure cloud infrastructure.
• Proven experience architecting event-driven and streaming data systems using pub/sub technologies such as Kafka or equivalent platforms.
• Demonstrated experience designing enterprise-grade ETL/ELT frameworks and internal SDKs for standardized pipeline development and deployment.
• Strong experience implementing Unity Catalog or equivalent enterprise data governance tooling, including fine-grained access control models and regulatory audit readiness.
• Experience operating within regulated or highly controlled environments with formal compliance and audit requirements.
• Bachelor’s degree in Computer Science, Engineering, or related quantitative discipline required; advanced degree preferred.
• This position requires valid United States work authorization and is not eligible for employer visa sponsorship.

Expected Base Salary Range:$210,000 - $230,000

In addition to a competitive base salary, employees may be eligible to receive a discretionary bonus delivered in the form of cash and/or deferred equity. Evercore also offers a variety of benefits and programs, subject to eligibility. These include, but are not limited to:

• Medical, prescription, dental, and vision insurance, including healthcare savings and reimbursements accounts
• 401(k) Retirement Plan
• Life and disability insurance, including additional voluntary financial protection insurance
• Well-being resources and programs, including mental health and mindfulness programs, digital wellness platforms, well-being events, and targeted on-site health services
• Family-building and family-support benefits
• Paid parental, caregiver, marriage and bereavement leave
• Commuter benefits, health club membership discounts, and other corporate discounts
• Paid holidays, vacation days, personal days, sick days, and volunteer opportunities

About Evercore: Evercore (NYSE: EVR) is a premier global independent investment banking advisory firm. We are dedicated to helping our clients achieve superior results through trusted independent and innovative advice on matters of strategic significance to boards of directors, management teams, and shareholders - including mergers and acquisitions, shareholder advisory, restructurings, and capital structure. Evercore also assists clients in raising public, private capital, delivers equity research, equity sales and agency trading execution, in addition to providing wealth and investment management services to high-net-worth individuals. Founded in 1995, the Firm is headquartered in New York and maintains offices and affiliate offices in major financial centers in North America, Europe, the Middle East, and Asia. For more information, please visit www.evercore.com .

Inclusion and Equal Opportunity Employment :Evercore is an equal employment opportunity employer and does not discriminate against individuals because of actual or perceived race, color, creed, religion, sex, pregnancy, national origin or ancestry, mental or physical disability, age, veteran status, military status, citizenship status, sexual orientation, gender identity or expression, genetic information, or immigration or citizenship status, or any other characteristic protected by applicable law (referred to as “protected status”), in accordance with applicable federal, state and local laws.