Senior Cloud Infrastructure Engineer, AI Platform

Procore Technologies; Austin (TX); Hybrid; USD 141,000 - 194,000; Full time

Benefits offered by this job

Equity compensation

Bonus incentive compensation

Job summary

Procore Technologies in Austin, TX is seeking a Senior Cloud Infrastructure Engineer to build the core infra powering AI agents, handling multi-modal data ingestion and vectorization across 100+ sources, with a focus on cost efficiency and fast responses.

This role reports to Senior Manager, Software Engineering and is a hybrid position with 2 days per week in our Austin office. Immediate start.

Qualifications

• 5+ years of hands-on experience with cloud platforms, especially GCP and IaC (Terraform).
• Experience with VectorDBs, LLMs, and agent observability tools (Milvus, Arize).
• Track record of performance analysis, tuning, and cost optimization for SaaS infra.
• Experience building multi-tenant SaaS platforms with observability and isolation.

Responsibilities

• Build and optimize scalable, multi-tenant ingestion and vectorization pipelines.
• Create dashboards, alerts, and logging for system health and performance.
• Tune performance to reduce latency in data retrieval, model inference, and responses.
• Design secure, cost-effective multi-tenant architectures for SaaS products.

Skills

GCP

Terraform

VectorDBs

LLMs

Observability

Performance tuning

Multi-tenant SaaS

Tools

Prometheus

Grafana

Datadog

Milvus

Arize

GCP Operations

Job description

We are hiring a Senior Cloud Infrastructure Engineer to build the core infrastructure that powers the next generation of AI agents. Our agents must ingest, enrich, and vectorize massive, multi-modal datasets from over 100 customer sources. The core challenge is twofold: how do we do this in a way that is radically cost-efficient, while still allowing agents to deliver thorough responses in seconds?

This position reports to a Senior Manager, Software Engineering and will be 2 days per week hybrid role in our Austin office. We’re looking for someone to join us immediately.

What You’ll Do

• Build & Optimize for Scale & Cost: Implement and optimize our highly scalable, multi-tenant data ingestion and vectorization pipeline, with a relentless focus on improving cost and performance.
• Implement Robust Monitoring: Create and maintain dashboards, alerts, and logging to ensure system health, identify performance bottlenecks, and provide immediate visibility into production issues.
• Contribute to Millisecond Latency: Be a key contributor in performance tuning across the stack. You will help identify and eliminate bottlenecks in data retrieval, model inference, and agent response times to ensure a snappy, real-time user experience.
• Engineer Multi-Tenant Architectures: Design and implement scalable, secure, and cost-effective multi-tenant infrastructures for our SaaS-based AI products, ensuring strict tenant isolation and fair resource allocation.

Who We're Looking For:

• 5+ years of hands-on experience with cloud platforms, with strong expertise in Google Cloud Platform (GCP) and Infrastructure as Code (e.g., Terraform)
• Experience with VectorDBs, LLMs, and Agentic Observability tools (Datagrid uses Milvus for our VectorDB and Arize for agent tracing)
• A demonstrated history of performance analysis, tuning, and infrastructure cost optimization, with the ability to speak about trade-offs and quantified impact
• Experience building or working on multi-tenant SaaS platforms
• Experience setting up end-to-end observability, including logging, metrics, and alerting using tools like Prometheus, Grafana, Datadog, or GCP Operations suite

Preferred Experience:

• A fundamental understanding of the challenges in training and serving large machine learning models (e.g., memory constraints, computational complexity)
• Experience with the Gemini API, specifically managing LLM quotas and load balancing
• Hands-on experience with LLM serving frameworks and optimization techniques (quantization, tensor parallelism, FlashAttention)
• Experience designing multi-tenant SaaS architectures and implementing resource quotas and cost allocation
• High-level knowledge of agentic systems and best practices

Additional Information

Base Pay Range:

140,960.00 - 193,820.00 USD Annual

This role may also be eligible for Equity Compensation and/or Bonus Incentive Compensation. Procore is committed to offering competitive, fair, and commensurate compensation. Actual compensation will be based on a candidate’s job-related skills, experience, education or training, and location.

For Los Angeles County (unincorporated) Candidates:

Procore will consider for employment all qualified applicants, including those with arrest or conviction records, in accordance with the requirements of applicable federal, state, and local laws, including the City of Los Angeles’ Fair Chance Initiative for Hiring Ordinance, the Los Angeles County Fair Chance Ordinance for Employers, and the California Fair Chance Act.

A criminal history may have a direct, adverse, and negative relationship on the following job duties, potentially resulting in the withdrawal of the conditional offer of employment: 1. appropriately managing, accessing, and handling confidential information including proprietary and trade secret information, as well as accessing Procore's information technology systems and platforms; 2. interacting with and occasionally having unsupervised contact with internal/external customers, stakeholders, and/or colleagues; and 3. exercising sound judgment.