Data Platform Engineer

Diagram; United States; Remote; USD 140,000 - 190,000; Full time

Benefits offered by this job

Stock options

Health benefits

New hire home-office setup: USD 500

Monthly Brex card stipend: USD 150

Job summary

Alpaca, a US-headquartered leader in brokered infrastructure, seeks a Data Platform Engineer to design and operate the data platform powering transactions, analytics, and developer APIs. You will own projects from implementation to production in a 100% remote, distributed environment.

The role requires 4+ years building scalable data platforms, strong Python/SQL skills, and experience with Kubernetes, Terraform, and streaming/CDC tech creating robust, observable data services.

Qualifications

• 4+ years of experience building and operating large-scale data platforms.
• Proficiency in Python and SQL for data pipelines and tooling.
• Experience with distributed systems, cloud data infrastructure, and CI/CD for data workloads.

Responsibilities

• Design and operate core data platform components, including lakehouse storage and metadata cataloging.
• Manage lakehouse deployment workflows with IaC tools like Terraform and Ansible on Kubernetes.
• Build scalable streaming and CDC pipelines with Kafka/Redpanda and Debezium.
• Develop serving and BI infrastructure enabling self-service access to data.
• Ensure reliability via observability, on-call practices, and incident response.

Skills

Distributed systems

Python

SQL

Data pipelines

Observability & SRE

Production ownership

Tools

Terraform

Ansible

Kubernetes

Helm

Argo CD

Airflow

Airbyte

Kafka

Iceberg

Job description

Who We Are:

Alpaca is a US-headquartered, global leader in agent-first brokerage infrastructure for stocks, ETFs, options, crypto, fixed income, 24/5 trading, and more.

Amongst our subsidiaries, Alpaca is a licensed financial services company, serving hundreds of financial institutions across 40 countries with our institutional-grade APIs. This includes broker-dealers, investment advisors, wealth managers, hedge funds, and crypto exchanges, totalling over 10 million brokerage accounts.

Our global team is a diverse group of experienced engineers, traders, and brokerage professionals who are working to achieve our mission of opening financial services to everyone on the planet. We're deeply committed to open-source contributions and fostering a vibrant community, continuously enhancing our award-winning, developer-friendly API and the robust infrastructure behind it.

Alpaca is proudly backed by $400 million in funding from top-tier global investors including Portage Ventures, Spark Capital, Tribe Capital, Social Leverage, Horizons Ventures, Opera Tech Ventures, SBI Group, Derayah Financial, Unbound, Peak XV, Elefund, and Y Combinator.

Our Team Members:

We're a dynamic team of 400+ globally distributed members who thrive working from our favorite places around the world, with teammates spanning the USA, Canada, Japan, Hungary, Nigeria, Brazil, the UK, and beyond!

We're searching for passionate individuals eager to contribute to Alpaca's rapid growth. If you align with our core values—Stay Curious, Have Empathy, and Be Accountable—and are ready to make a significant impact, we encourage you to apply.

About the team & role

The Data Platform team builds and operates the shared infrastructure that powers how data is stored, moved, and consumed across Alpaca. Our platform supports financial transactions, customer data, API and system events, enriched datasets, and third-party sources that power critical decisions and products for internal teams and external stakeholders. We process hundreds of millions of events each day and that scale continues to grow as Alpaca serves more customers and launches new products.

As a Data Platform Engineer, you will build and operate reliable systems across Alpaca’s data platform. You’ll contribute to our distributed query, lakehouse, CDC, and real-time streaming infrastructure while collaborating with experienced engineers and partner teams. You’ll own projects from implementation through production operation and help improve the platform’s scalability, reliability, and developer experience.

Our team is 100% distributed and remote.

Responsibilities:

• Design, develop, and evolve Alpaca’s core data platform, including distributed query engines, orchestration, lakehouse storage, metadata, and cataloging.
• Own our lakehouse infrastructure as code, building reliable and repeatable deployment workflows with Terraform and Ansible on Kubernetes.
• Build and operate scalable, low-latency streaming and CDC pipelines alongside dependable batch ingestion paths into Apache Iceberg.
• Develop and scale our serving and BI infrastructure, giving downstream teams and agents performant, governed, self-service access to lakehouse data.
• Take end-to-end ownership of platform reliability through observability, actionable alerting, on-call practices, incident response, maintenance procedures, runbooks, and service-level objectives.
• Partner with DevOps, Analytics Engineering, and stakeholders across Alpaca to close infrastructure gaps, shape technical requirements, and deliver reusable platform capabilities.

Must-Haves:

• 4+ years of experience in software engineering, data infrastructure, or platform engineering, with demonstrated ownership of complex production systems and technical projects.
• Strong understanding of distributed systems and production experience with query engines such as Trino or Presto.
• Strong experience operating Kubernetes-based data infrastructure using infrastructure-as-code and deployment tools such as Terraform, Helm, Ansible, or Argo CD.
• Experience designing and operating compute platforms across multiple cloud regions, balancing workload locality, peak and bursty demand, resource isolation, workload prioritization, and cost-performance trade-offs.
• Hands-on experience designing and operating cloud lakehouse architectures on GCP, AWS, or Azure, including object storage, metastores, and open table formats, particularly Apache Iceberg.
• Experience operating large-scale streaming and CDC systems using technologies such as Kafka, Redpanda, and Debezium.
• Strong Python and SQL skills, with experience building reliable data pipelines and platform tooling using orchestration and ingestion frameworks such as Airflow and Airbyte.
• Strong operational judgment and the ability to make sound technical decisions, communicate trade-offs, and work effectively through ambiguity.

Nice to Haves:

• Experience with semantic or metrics layers such as Cube, dbt, or Looker.
• Experience deploying and operating infrastructure for AI agents.
• Experience building or integrating context layers that connect structured and unstructured organizational data.
• Experience with data governance and access-control frameworks such as Apache Ranger.

How We Take Care of You:

• Competitive Salary & Stock Options
• Health Benefits
• New Hire Home-Office Setup: One-time USD $500
• Monthly Stipend: USD $150 per month via a Brex Card

Alpaca is proud to be an equal opportunity workplace dedicated to pursuing and hiring a diverse workforce.

Recruitment Privacy Policy