Staff Engineer

HighLevel; Town of Texas (WI); Hybrid; USD 180,000 - 240,000; Full time

Job summary

HighLevel is seeking a Staff Engineer to architect the future of trust, controls, and safety across a real-time platform handling millions of events and multi-million-dollar transactions daily. You will own the technical direction, mentor engineers, and partner with EMs, PMs, finance, risk, and support to improve platform integrity.

You will redesign data models, optimize Elasticsearch-driven search at scale, and define clear contracts between storage, cache, and APIs.

Qualifications

• 8+ years of experience building high-scale backend systems where p99 regressions are unacceptable.
• Deep expertise in concurrency, event-driven architectures, and consistency primitives (CAS, atomic dry-runs, idempotency).
• Experience with metering, ledgers, payments, or other financially impactful systems.

Responsibilities

• Redesign and optimize data models, queries, and indexing across MongoDB, Firestore, and Elasticsearch.
• Own Elasticsearch reliability, ingestion, indexing, shard strategy, and query performance for billions of documents.
• Define data flow and integration boundaries between storage, cache, and APIs with clear contracts and fault isolation.
• Build visibility into data paths via metrics, traces, and dashboards.
• Enforce testing standards across backend modules and drive architectural reviews.

Skills

Distributed Systems Mastery

Concurrency

Event-driven Architecture

Performance Optimization

Tools

MongoDB

Firestore

Elasticsearch

Go

Node.js

Job description

We are looking for a Staff Engineer to architect the future of trust, controls, and safety. You will lead the technical direction of a real-time platform that secures critical workflows across HighLevel, including signups, SaaS checkouts, user management, and global communication usage. This system processes 160M+ events and $4M+ in transactions daily, serving as the essential foundation for platform integrity.

As a Staff Engineer, you'll set the technical direction, own the hardest problems on the platform, mentor engineers across squads, and partner closely with EMs, PMs, finance, risk, support, and the channel teams. Getting this right means customers stop getting bills they didn't intend, and abuse stops being a cost the honest majority carries.

Responsibilities:

• Database architecture: Redesign and optimize data models, queries, and indexing strategies across MongoDB, Firestore, and Elasticsearch.
• Search at scale: Own Elasticsearch reliability, ingestion, indexing, shard strategy, and query performance for billions of documents.
• Reliability and performance: Eliminate bottlenecks, improve replication health, and enforce predictable query and index latency.
• System design: Define data flow and integration boundaries between storage, cache, and APIs with clear contracts and fault isolation.
• Observability: Build visibility into every data path: metrics, traces, slow query logs, replication lag, and cluster health dashboards.
• Testing and quality: Make testing non-negotiable; enforce unit, integration, and load testing standards across all backend modules.
• Engineering culture: Drive RFCs, ADRs, and design reviews that push clarity and precision. Codify patterns that make good engineering repeatable.
• Hands-on leadership: Write code, design systems, and debug production issues. Lead by technical example, not by delegation.
• Mentorship and influence: Level up engineers around you with reviews that teach, feedback that sticks, and systems that outlive individuals.

Requirements:

• Heavyweight Engineering Pedigree: 8+ years of experience building high-scale, mission-critical backend systems where p99 regressions are unacceptable.
• Distributed Systems Mastery: Deep expertise in concurrency, event-driven architectures, and consistency primitives (CAS, atomic dry-runs, idempotency).
• Financial Rigour: Experience with systems where code correctness has direct financial impact: metering, ledgers, or payment gateways.
• Operational Instinct: A track record of technical leadership through influence, driving architecture for multi-team platforms from design to production.
• Experience with systems where correctness has financial consequences: metering, ledgers, billing, reservations, or comparable transactional domains.
• Strong experience with relational and NoSQL data models (especially with complex transactional data), plus working knowledge of streaming and analytical stores for time-windowed aggregation.
• Experience designing policy- or rules-driven systems that change through configuration rather than deploy versioning, safe rollout, and rollback of behavioral change.
• Sound judgement on the enforcement trade-off: how to set thresholds and actions that stop real harm without punishing legitimate customers, and how to measure whether you got it right.
• Familiarity with monitoring, SLOs, and root-cause analysis in production environments.
• Familiarity with hallucination mitigation strategies (e. g., code constraints, test scaffolding, embeddings, and context injection).
• Track record of technical leadership without formal authority, driving cross-team architecture to a decision, writing designs others build from, and lifting the engineering standard around you.
• Excellent communication and cross-functional collaboration skills, including with Finance, Risk, and Support stakeholders.
• A pragmatic, hands-on engineer who balances long-term vision with iterative execution.

Nice to Have:

• Experience with Elasticsearch cluster management, multi-tenant indexing, or lifecycle policies.
• Familiarity with ClickHouse or other OLAP systems for analytics.
• Background in event-driven systems, data pipelines, or message brokers.
• Contributions to open-source or public technical writing on database performance or systems design.

Preferred Experience:

• Abuse and Fraud Expertise: Experience building rule engines, velocity checks, or adversarial defense systems at a massive scale.
• Modern Data Stack: Hands-on with high-throughput tools like Go, Redis (Lua), ClickHouse, and Flink for real-time aggregation.
• Hyper-Growth Scaling: Experience navigating the transition from early architecture to a load-bearing platform at scale.
• Exposure to stream processing at scale (Flink, Kafka Streams, Beam) and columnar analytical stores (ClickHouse, BigQuery) for real-time aggregation.
• Hands-on experience with Node.js and Go in production services, particularly on latency-sensitive or high-throughput paths.
• Experience working with AI tools, billing, and metered systems.
• Contributions to open-source tools, internal platforms, or engineering blogs.