Senior Software Engineer - AI Operator

HiveWatch; Los Angeles (CA); On-site; USD 160,000 - 200,000; Full time

Benefits offered by this job

Health coverage

Equity incentives

Competitive compensation package

HQ on Main Street, El Segundo, CA

401(k) with company match

Flexible PTO

Inclusive culture

Job summary

HiveWatch is seeking a Senior Software Engineer for the App Experience team. You will design, build, and ship AI-powered features in the intelligence layer of our platform, handling alarms, incidents, and operator actions with frontier and self-hosted models.

You’ll own evaluation loops, test hypotheses with real data, and craft guardrails, human-in-the-loop controls, and observability for trustable agent decisions.

Qualifications

• 7+ years of software engineering experience with at least 3 years in senior roles.
• Strong experience with at least one compiled, object oriented programming language such as Rust, Kotlin, Java, C#, or C++.
• Strong proficiency with relational databases, SQL, data modeling, and query tuning.
• Experience building full-stack applications in microservice architectures.
• Demonstrated ability to work hypothesis-first with data: framing questions, writing the SQL or analysis code to answer them, and drawing sound conclusions.
• Experience with modern cloud services and cloud-native development.
• Experience with containerization using Docker and deploying applications to Kubernetes.
• Experience with CI/CD pipelines and modern development workflows.
• Experience using monitoring tools to measure and alert on performance and reliability issues.
• Experience with and enthusiasm for AI native software development practices.
• Strong system design skills and experience with distributed, high-scale applications.
• Ability to learn new technologies quickly and work in agile environments.
• Data-driven individual who will use metrics to influence decision making.

Responsibilities

• Design and build agentic product features that assess security scenarios and make decisions human security operators make today, such as whether to triage, elevate, dismiss, or dispatch.
• Form hypotheses about our data, then validate them: query production datasets, analyze results, and turn findings into product and model decisions
• Orchestrate calls to both self-hosted inference models and frontier models, including prompt design, tool use, structured outputs, routing across models, and cost/latency tradeoffs
• Ship end-to-end features: Python for LLM/inference orchestration and analysis, and our Kotlin / Spring Boot / GraphQL backend to integrate agent capabilities into the platform
• Work with product, customers, and security operations experts to encode operator judgment into agent behavior
• Reliably deliver features and services on AWS using Infrastructure as Code, CI/CD pipelines, and modern service delivery practices
• Mentor junior engineers and drive technical decisions that influence the team’s architecture and development practices
• Support production systems through incident response, troubleshooting, and break-fix activities to maintain service reliability and customer trust
• Participate in a 24/7 on call rotation approximately 1 week per month as needed.

Skills

Kotlin

TypeScript

Python

Rust

SQL

Distributed systems

Cloud AWS

CI/CD

Docker

Kubernetes

GraphQL

Mentoring

Tools

GitHub Actions

Terraform

Terragrunt

Helm

AWS

Kinesis

Serverless

RDS

EKS

Job description

HiveWatch is a tech-forward, inclusive organization fostering the evolution of the physical security industry. We are a diverse team of forward thinkers who empower each other to find creative and collaborative solutions in an industry ripe for modernization. We are passionate about the problems we're solving for our customers and equally passionate about the company we're building.

HiveWatch is here to help security teams pivot from chasing threats to preventing them. We protect organizations, people, and property through the intelligent orchestration of physical security programs. With better communication, more insights, and less "noise", we are modernizing what it means for businesses and their employees to truly feel safe.

THE ROLE:

We're seeking a Senior Software Engineer to join our App Experience team. You'll design, build, and ship AI-powered features in the intelligence layer of our platform: the agents that triage alarms, assess incidents, and recommend or take action the way an experienced security operator would. That means composing frontier models (language, vision, audio) with our own data, tooling, and retrieval into production-grade agent workflows, and working across the full stack to do it, from model orchestration in our Kotlin / Spring Boot / Python services to the React/TypeScript surfaces where operators see and act on the results.

You'll own the evaluation loop for what you build. You'll define what "correct" means for an agent decision, build the eval sets and harnesses to measure it, and use those results to drive changes to prompts, tools, and architecture. To get there, you'll dig into real customer data, including alarm streams, video, access control events, and operator actions, to form and test hypotheses about where agents get things right, where they drift from human judgment, and why. You'll build the guardrails, human-in-the-loop controls, and observability that let us trust agents with higher-stakes decisions over time, and that let customers understand why an agent did what it did.

You'll prototype quickly to learn what works, then harden the winners into reliable, cost-aware features. You'll partner closely with our product team and in-house security operations experts to turn operator intuition into something an agent can act on and to find the next problem worth automating.

WHAT YOU'LL DO :

• Design and build agentic product features that assess security scenarios and make decisions human security operators make today, such as whether to triage, elevate, dismiss, or dispatch.

• Form hypotheses about our data, then validate them: query production datasets, analyze results, and turn findings into product and model decisions

• Orchestrate calls to both self-hosted inference models and frontier models, including prompt design, tool use, structured outputs, routing across models, and cost/latency tradeoffs

• Ship end-to-end features: Python for LLM/inference orchestration and analysis, and our Kotlin / Spring Boot / GraphQL backend to integrate agent capabilities into the platform

• Work with product, customers, and security operations experts to encode operator judgment into agent behavior

• Reliably deliver features and services on AWS using Infrastructure as Code, CI/CD pipelines, and modern service delivery practices

• Mentor junior engineers and drive technical decisions that influence the team’s architecture and development practices

• Support production systems through incident response, troubleshooting, and break-fix activities to maintain service reliability and customer trust

• Participate in a 24/7 on call rotation approximately 1 week per month as needed.

OUR TECH STACK:

• Languages: Kotlin, TypeScript, Python, and Rust

• Deployments: GitHub Actions, Terraform, Terragrunt, and Helm

• Infrastructure: AWS (Kinesis, Serverless, RDS, EKS), Kubernetes, Docker, PostgreSQL

MINIMUM QUALIFICATIONS :

• 7+ years of software engineering experience with at least 3 years in senior roles

• Strong experience with at least one compiled, object oriented programming language such as Rust, Kotlin, Java, C#, or C++

• Strong proficiency with relational databases, SQL, data modeling, and query tuning

• Experience building full-stack applications in microservice architectures

• Demonstrated ability to work hypothesis-first with data: framing questions, writing the SQL or analysis code to answer them, and drawing sound conclusions

• Experience with modern cloud services and cloud-native development

• Experience with containerization using Docker and deploying applications to Kubernetes

• Experience with CI/CD pipelines and modern development workflows

• Experience using monitoring tools to measure and alert on performance and reliability issues

• Experience with and enthusiasm for AI native software development practices

• Strong system design skills and experience with distributed, high-scale applications

• Ability to learn new technologies quickly and work in agile environments

• Data-driven individual who will use metrics to influence decision making

PREFERRED QUALIFICATIONS :

• Strong experience with AWS

• Experience designing evaluation harnesses for non-deterministic systems: golden datasets, LLM-as-judge, regression testing for agent behavior

• Experience with real-time or event-driven data (Kinesis, Kafka, or similar)

• Hands-on experience with Terraform for infrastructure as code

• Background in security technology or enterprise software

• Experience with observability tools and performance optimization

• Startup experience and/or early-stage SaaS experience

ADDITIONAL INFO :

• Salary range for this position: $160,000 - $200,000 per year

• Eligible to participate in HiveWatch Equity Incentive Plan

*Final offer will be at the company's sole discretion and determined by multiple factors, including years and depth of relevant experience and expertise, location, and other business considerations.

Benefits & Culture:

At HiveWatch, we're passionate about taking care of our people — and it shows in the benefits we offer. Our team enjoys:

• Comprehensive health coverage: medical, dental, vision, and life insurance

• Cutting-edge work in an emerging field with huge growth potential

• Competitive compensation packages designed to reward top talent

• A modern, newly renovated HQ right on Main Street in El Segundo, CA

• 401(k) with a 4% company match to help you invest in your future

• Flexible paid time off so you can recharge when you need it

• A family-friendly, compassionate culture that values balance and belonging

We encourage you to challenge the status quo, share your perspective, and leave fear at the (access-controlled) door.

Our EEO Statement:

HiveWatch is an equal opportunity employer and we are committed to cultivating a work environment that supports, inspires, and respects all individuals. We execute our hiring practices so that they are merit-based and we do not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity/expression, marital status, age, disability, medical condition, genetic information, national origin, ancestry, military or veteran status, or other protected characteristic.