Senior Go Engineer - Data Ingestion & AI-Driven Security

Visa Hunt; United States; Hybrid; USD 150,000 - 210,000; Full time

Benefits offered by this job

Health coverage for you and family

Flexible locations and schedules

Generous vacation days

Donation matching

Volunteer time

Parental leave

Job summary

Elastic, the Search AI Company, is seeking a seasoned backend engineer to own the data ingestion surface for security telemetry. You will build host‑level collectors, integrations, and APIs against cloud vendors, ensuring scalable and reliable data pipelines.

You’ll collaborate across product and security research to prioritize high‑impact sources, apply AI automation to accelerate workflows, and mentor engineers while shaping the ingestion framework at scale.

Qualifications

• 5+ years of software engineering experience building production backend/data systems.
• Proficiency in Go (or robust systems-language experience).
• Experience integrating with third‑party/cloud APIs (auth, pagination, rate limits).
• Deep understanding of data pipelines / ETL / log & event processing.

Responsibilities

• Design, build, and maintain Beats data shippers, Filebeat inputs and modules, plus host/endpoint collectors such as Auditbeat, Winlogbeat, Packetbeat, and Osquerybeat in Go.
• Develop and maintain security integration packages for third‑party sources (identity/IdP, EDR/XDR, email security, firewalls, cloud security, threat intel, vulnerability management).
• Build robust Elastic managed/API‑based collection against external vendor and cloud‑provider APIs (AWS, Azure, GCP), handling auth (OAuth2, tokens), pagination, rate limits, and resilience.
• Improve reliability, performance, and observability of collection at scale; own quality for your areas and drive down failing‑test and reliability noise.
• Review community and partner contributions, mentor engineers, and set technical direction across the ingestion surface.
• Collaborate with product, security research, and field teams to prioritize high‑impact sources and unblock customer onboarding.
• Apply LLMs and AI automation to ease and accelerate the work e.g. scaffolding new integrations/modules, drafting and normalizing ECS mappings and ingest pipelines, generating parsers and tests from sample data, summarizing vendor API docs, and triaging failing‑test and reliability noise.
• Build and improve internal AI‑assisted tooling and automation (scripts, agents, CI steps) that reduce toil across a large catalog, and establish good practices for using AI safely and effectively (review, validation, guarding against hallucinated mappings).

Skills

Go

Backend development

Data pipelines

ETL processes

API integration

LLMs / AI tools

Open-source collaboration

Tools

Beats

Auditbeat

Winlogbeat

Packetbeat

Osquerybeat

Job description

Elastic, the Search AI Company, is seeking a seasoned backend engineer to own the data ingestion surface for security telemetry. You will build host‑level collectors, integrations, and APIs against cloud vendors, ensuring scalable and reliable data pipelines.

You’ll collaborate across product and security research to prioritize high‑impact sources, apply AI automation to accelerate workflows, and mentor engineers while shaping the ingestion framework at scale.