Director of Software Engineering (Data)

Leadout Capital; San Francisco (CA); On-site; USD 210,000 - 320,000; Full time

Job summary

Leadout Capital is seeking a Director of Software Engineering to lead our Data Engineering team. You will own the technical strategy for enterprise-scale data ingestion platforms powering analytics, ML models, and business apps.

You will manage senior data engineers, drive architecture across Snowflake/Data360, and collaborate with cross-functional leaders to deliver scalable, secure, and high‑quality data platforms.

Qualifications

• 15+ years of data engineering experience with focus on data ingestion and cloud data warehouses.
• 8+ years of engineering leadership and people management experience.
• Proven experience architecting and operating high-volume data ingestion platforms.
• Deep hands-on with Snowflake and data architecture, performance, security.
• Strong experience with Apache Spark and Kafka for large-scale processing.

Responsibilities

• Define and execute the technical strategy and roadmap for data ingestion platforms.
• Architect, build, and operate high-volume batch and real-time ingestion systems.
• Lead data pipelines and ETL/ELT workflows with Spark, Iceberg, Kafka, Airflow.
• Design data models across Snowflake, Data360 and lakehouse environments.
• Ensure availability, security, observability, and disaster recovery.

Skills

Data ingestion leadership

Snowflake

Apache Spark

Apache Kafka

Airflow

Python

SQL

CI/CD

Data governance

Stakeholder management

Tools

MuleSoft

Informatica

dbt

Iceberg

Snowflake

Job description

We’re hiring a Director of Software Engineering to lead our Data Engineering team. You’ll own the technical strategy and execution for enterprise-scale data ingestion platforms that power analytics, operational workflows, ML models, AI agents and business applications across the organization.

This role will lead an established team of senior data engineers and drive the architecture of high-volume, high-throughput data ingestion systems. You’ll set engineering direction for data ingestion from 100+ enterprise systems into Snowflake and Data360 leveraging technologies like MuleSoft, Informatica, Spark, dbt, Iceberg, Kafka and Airflow while ensuring our data ingestion platform is scalable, reliable, governed, secure, and easy for internal teams to use.

This is a director-level role with direct people management responsibility for senior data engineers.

What You’ll Do

Technical Leadership

• Define and execute the technical strategy and roadmap for enterprise data ingestion platforms.
• Architect, build, and operate high-volume, high-throughput batch and real-time data ingestion systems.
• Lead the design of scalable data pipelines and ETL/ELT workflows using technologies such as MuleSoft, Informatica, Spark, dbt, Iceberg, Kafka and Airflow.
• Design and optimize data models and storage architectures across Snowflake, Data360 and lakehouse environments.
• Partner with the Data Platform team to establish Apache Iceberg-based patterns for open, scalable, and interoperable data storage.
• Ensure data ingestion platforms meet enterprise requirements for availability, performance, security, observability, data quality, and disaster recovery.
• Drive platform scalability through automation, reusable frameworks, standardized ingestion patterns, and self-service capabilities.
• Establish and enforce engineering best practices, including architecture reviews, testing, code reviews, CI/CD, documentation, and operational readiness.
• Evaluate and champion new data technologies, frameworks, and architectural patterns.
• Maintain a strong balance between strategic technical leadership and hands‑on involvement in critical architectural decisions.

People Management

• Directly manage senior data engineers, providing mentorship, coaching, and career development support.
• Set clear goals and expectations, conduct regular one-on-one meetings, and lead performance and talent reviews.
• Build a collaborative, inclusive, accountable, and high-performing engineering culture.
• Develop technical leaders and create growth paths for senior and staff-level engineers.
• Partner with recruiting and engineering leadership to grow the team as business needs evolve.
• Establish effective team structures, ownership models, and operating mechanisms.

Cross-Functional Collaboration

• Partner with Product, Data Platform, Architecture, Analytics, Data Science, Security, Governance, and business stakeholders to align priorities and technical direction.
• Translate complex business and data requirements into pragmatic technical strategies and execution plans.
• Represent Data Ingestion Platform in roadmap, investment, capacity-planning, and architecture discussions with senior leaders.
• Establish clear service-level objectives and operating processes for support and incident management.
• Collaborate with data producers, consumers, trust and governance teams to improve data contracts, governance, lineage, discoverability, quality, and usability.
• Communicate architectural decisions, tradeoffs, risks, and delivery progress to both technical and non-technical stakeholders.

What We’re Looking For

• 15+ years of data engineering experience with focus on data ingestion, including significant experience designing cloud-based data warehouse and lakehouse platforms.
• 8+ years of engineering leadership and people management experience, ideally managing senior and staff-level engineers.
• Proven experience architecting and operating high-volume, high-throughput enterprise data ingestion platforms.
• Deep hands‑on experience with Snowflake, including data architecture, performance optimization, security, governance, and cost management.
• Strong expertise with Apache Spark for large-scale distributed data processing.
• Hands‑on experience designing lakehouse architectures using Apache Iceberg or comparable open table formats.
• Strong experience with Apache Kafka and event-driven, streaming-data architectures.
• Deep proficiency in SQL and at least one general-purpose programming language, preferably Python.
• Strong understanding of data modeling, distributed systems, schema evolution, data contracts, and batch and streaming processing patterns.
• Proficiency with infrastructure as code and CI/CD for data workflows.
• Experience implementing enterprise data quality, metadata management, lineage, observability, access control, and governance practices.
• Track record of establishing engineering standards and delivering reliable platforms across multiple teams.
• Experience leading complex technical programs involving multiple systems, stakeholders, and engineering teams.
• Excellent communication and stakeholder-management skills, with the ability to drive alignment across engineering and business leadership.

Nice to Have

• Experience with Salesforce data ecosystems (Data360) within a complex enterprise data ecosystem.
• Experience developing AI agents or agentic workflows for data ingestion, transformation, data quality, metadata management, or platform operations.
• Experience with agentic data ingestion architectures that can discover sources, interpret schemas, generate pipelines, detect failures, or recommend remediation.
• Experience building self-service data platforms, including reusable ingestion frameworks, developer portals, APIs, templates, and paved-road workflows.
• Experience designing platform capabilities that allow teams to onboard data sources safely without ongoing involvement from the core Data Engineering team.