Director, Data Engineering

Core Specialty Insurance Holdings, Inc.

Dallas (TX)

On-site

USD 180,000 - 240,000

Full time

40 hours ago

Be an early applicant

Application generator

Don’t send a generic resume — generate a resume and cover letter tailored to this exact role.

Get past ATS filters

Benefits offered by this job

Medical, dental, vision, and life ins.

Disability coverage

401(k) with company match

Employee assistance plan

Health Savings Account

Flexible Spending Account

Wellness program

Job summary

Core Specialty Insurance Holdings, Inc. seeks a Director, Data Engineering to lead a large-scale data platform. The role blends hands-on solution architecture with strategic leadership, guiding architecture, pipelines, and governance as AI/ML initiatives scale.

You will report to the VP-Data Engineering, AI & ML and shape the data engineering team's technical direction and delivery. You will drive modernization, ensure reliability, and mentor engineers while aligning with company data strategies

Qualifications

Bachelor's degree in Computer Science or related field or equivalent experience.
10+ years in data engineering with scale, optimization and evolution of data platforms.
3+ years in an architect role with ownership of solution designs.
3+ years in a leadership role managing data engineering teams.
Experience enabling AI/ML solutions via data platforms.

Responsibilities

Lead the technical evolution of a large-scale data platform with architecture and delivery responsibilities.
Design, prototype, and develop key platform components for high-risk initiatives.
Define technical direction and standards in collaboration with Enterprise Architecture.
Mentor data engineers and manage team resources and priorities.
Ensure AI/ML readiness and governance across data pipelines and platforms.

Skills

Data architecture

Leadership

Python

SQL

Scala

Cloud platforms

Data governance

Terraform

GitHub Actions

Education

Bachelor Degree in Computer Science or related field

Tools

Snowflake

Databricks

Azure Synapse Analytics

MS Fabric

Job description

We are looking for a Director, Data Engineering, to lead the continued optimization and maturation of our existing large‑scale data platform. This is hands‑on technical leadership role: roughly 60% of your time will be spent as a working solution architect – improving, rearchitecting, and scaling data pipelines, infrastructure, and models that power the business. The remaining 40% will be focused on strategic alignment, technical direction, and resource management for the Data Engineering team. A key part of the mandate is ensuring the platform remains AI/ML ready as it scales – well modeled and well governed so that advanced analytics teams can reliably and efficiently access high-quality data.

Reporting to the VP‑Data Engineering, AI & ML, you will lead a team of Data Engineers – setting the technical execution direction while staying close enough to the architecture and code to guide hard decision, unblock the team, and ensure the platform continues to scale reliably and security as usage and AI/ML initiatives grow.

Applicants must be authorized to work for any employer in the U.S. We are unable to sponsor or take over work authorization sponsorship now or in the future for this position.

Key Accountabilities/Deliverables

Solution Architecture & Hands-On Delivery

Own the technical evolution of our existing large-scale data platform, identifying and riving improvements to architecture, performance and scalability data volume and complexity use cases grow
Act as the lead solution architect for data engineering initiatives
Personally design, prototype, and development of frameworks and critical platform components – especially for high-risk/high complexity initiatives
Have a forward-thinking posture as it comes to creating an AI/ML ready data platform
Participate in the evaluation, selection and implementation of data platform tooling (i.e. – storage, orchestration, compute, streaming)
Reinforce existing data observability solutions/standards across pipelines and the platform
Diagnose and resolve complex production issues/technical debt across the data platform – partnering directly with other engineers to resolve.

Strategy, Technical Direction & Resource Management

Partner with VP‑Data Engineering, AI & ML to define and execute the roadmap for a fully optimized and scalable data platform – ensuring alignment to broader company and data strategies including AI/ML readiness assessments
Set technical direction and architectural standards for the team by working closely with Enterprise Architecture
Manage, mentor, and develop data engineers
Participate in resource planning and prioritization across the team’s product roadmap – balancing business requests with platform-based initiatives
Represent Data Engineering planning, budgeting, and resource management discussions as needed
Stay current with emerging trends in data engineering and data platform architecture – brining innovations into continued data maturity for the organization.

Technical Knowledge and Understanding

Technical Knowledge

Deep expertise in data pipeline design, optimization, and distributed data processing architectures with experience identifying and resolving performance and scalability bottlenecks
Platforms: Hands on experience with Snowflake or Databricks (experience in an architecture role with these platforms is a plus), Azure Synapse Analytics/MS Fabric
Strong knowledge of cloud data platforms (AWS, Azure, or GCP) and modern data warehouse/Lakehouse architectures (Delta Lake, Iceberg)
Working familiarity with AI/ML data requirements and tooling to support AI/ML enablement
Strong working proficiency with at least two programming languages (Python, Scala, SQL)
Solid understanding of data governance, security, and compliance practices in a regulated or enterprise environment
Experience with IAC and CI/CD practices for data systems (i.e. Terraform, GitHub Actions)

Leadership & Soft Skills

Proven ability to set technical direction while remaining hands‑on and credible at the architecture and code level
Strong stakeholder management skills – the ability to translate between technical and non-technical audiences
Excellent prioritization and resource management skills across competing technical and business demands
A coaching mindset with genuine investment in growing the technical and soft skills of direct reports

Requirements

Bachelor Degree in Computer Science or related field or equivalent work experience.
10+ years experience in data engineering with significant experience in optimizing, scaling, and evolving large-scale mature data platforms
3+ years in an architect role (Solution, Enterprise, Data, Platform) with direct ownership of solution designs
3+ years in a leadership role managing data engineering teams – including direct people management responsibilities
Working knowledge of how data platforms enable AI/ML solutions
Track record of balancing hands‑on technical delivery with people leadership and strategic planning

Nice to Have