Senior Software Engineer: AI-Driven Observability

OneTrust, LLC; Atlanta (GA); Hybrid; USD 116,000 - 175,000; Full time

Benefits offered by this job

Healthcare coverage

Flexible PTO

Equity RSUs

Annual bonus opportunities

Retirement account support

Parental leave 14+ weeks

Career development opportunities

Certification exam fees

Job summary

OneTrust is seeking a Senior Software Engineer to join the Development team. You will partner with Engineering, Operations and Product to design, deliver, and maintain a highly available, high‑performing application platform.

You will build observability and incident response tooling, reduce toil, define SLIs/SLOs, tune alerts, and collaborate across functions to identify gaps and resolve reliability risks. Experience with CI/CD, Kubernetes, and cloud environments is required.

Qualifications

• Bachelor's degree in CS or related field.
• 4+ years of application development with Java.
• Experience with the Spring ecosystem.
• Cloud experience (Azure/AWS/GCP).
• Knowledge of performance tuning and optimization.
• Centralized logging, metrics, dashboards, and alerting.
• Databases (SQL/NoSQL) knowledge.
• Hands-on observability tooling (Datadog/Prometheus/Grafana).
• CI/CD pipelines and IaC (Terraform, Helm, Jenkins, GitLab).
• AI/LLM tooling and incident response experience.
• Vector DBs, embeddings or RAG for ops intelligence.
• Kubernetes and microservices in production.

Responsibilities

• Design, deliver, and maintain a highly available platform.
• Build observability and incident response tooling.
• Reduce toil by automation and tuning alerts.
• Define and monitor SLIs/SLOs aligned with UX.
• Collaborate with cross-functional teams to resolve reliability risks.
• Embed with product teams to review architectures for reliability.
• Develop scripts for operational automation (Python/ Bash/ Java).

Skills

Java

Spring

Cloud platforms

Observability

CI/CD

Kubernetes

Distributed systems

Logging/monitoring

JVM tuning

AI/LLM tooling

Incident response

Education

Bachelor's degree in computer science or related field

Tools

Datadog

Prometheus

Grafana

Terraform

Helm

Jenkins

GitLab

PagerDuty

Job description

The speed of AI adoption is exposing the next governance gap

Strength in Trust

OneTrust’s mission is to enable innovation through the responsible use of data and AI. We believe that ensuring data is trusted shouldn’t slow teams down—it should accelerate what’s possible. This led us to develop the first technology platform for responsible data use in 2016. Today, with AI representing the latest and most impactful expansion of data yet, OneTrust is once again redefining what responsible innovation looks like. OneTrust, the AI‑Ready Governance Platform™, unifies regulatory intelligence, automation, and connected governance workflows so businesses can continue to move at the speed of AI while ensuring good governance to prevent data misuse at scale. Trusted by thousands of organizations worldwide, OneTrust is shaping the future where trusted data becomes a transformative force for business and society.

The Challenge

We're looking for a Senior Software Engineer that will report to the Development Manager / R&D Head. In this role, you will part of the R&D Team that works on mission-critical applications.

Your Mission

Engage and partner with various Engineering, Operations, and Product teams to design, deliver, andmaintaina highly availableand performant application platform.

Build and implement application observability and platform monitoring tools to continuously improve the customer experience

Eliminatetoil by automating processes, tuning alerts, and improving code where it is most needed

Frequently evaluatenew ideasand trends toidentifypotentially useful tools and techniques

Collaborate with different functional groups toidentifygaps, prioritize, and resolve issues

Defining, implementing, and maintaining SLIs and SLOs aligned with customer experience.

Design and instrument SLIs such as latency, error rates, and availability across critical services

Manage and enforce error budgets to balance system reliability with productfeaturevelocity.

Improvingalertquality by reducing noise and focusing on actionable, high-signal alerts

Embed with product teams to review architectures and catch reliability risks early

Share your knowledge and experience with the Engineering organization

Share your findings with technical leadership and senior management

Build scripts in python/bash/java or ruby for operational automation and incident response

Your Experience Includes

Bachelor's degree in computer science, Engineering, or related technical or business field

4+ yrs. of application development experience with Java or other equivalent language

Experience with Spring environment

Experience in cloud-based infrastructure (Azure, AWS, GCP, etc.)

Experience with the factors that affect software application performance at different levels. These factors include database performance, network performance, CPU utilization, JVM tuning, memory analysis, thread management, and query performance.

A knowledge of the importance of centralizing logging,metricsdashboards, and alerting. Able to articulate about some of the tools used for these tasks

A good awareness of databases (ideally SQL/NoSQL)

Hands-on experience with observability tools (Datadog, Prometheus, Grafana, etc.)

Knowledge with CI/CD pipelines and infrastructure-as-code (Terraform, Helm,jenkins,gitlab)

Build andoperateAI-assisted incident response systems (root cause analysis, log summarization, anomaly triage)

Develop or integrate LLM-based tools to reduce MTTR and improvealertquality

Apply machine learning techniques for anomaly detection, capacity prediction, or failure pattern analysis

Experience deploying AI systems in production (not just experimentation)

Knowledge with vector databases, embeddings, or RAG architectures for operational intelligence

Well-developed insight of prompt engineering and evaluation of LLM outputs in the reliability workflow

Kubernetes and container orchestration (EKS/AKS/GKE)

Experience with distributed systems at scale

Familiarity with service meshes and microservices architectures

Nice to Have

Experience with chaos engineering tools (Gremlin, Chaos Monkey)

Background in product-facing services with high traffic scale

Understand how to use incident management platforms. This includes using tools like PagerDuty for alerts. It also includes working with DataDog for monitoring.

For California, Colorado, Connecticut, Nevada, New York, Rhode Island, and Washington-based candidates: the annual base pay range for this role is listed below. Within this range, individual pay is determined by several factors, including location, job-related skills, work experience, and relevant education and/or training. This role may also be eligible for discretionary bonuses, equity, and/or commissions, as well as benefits.

Salary Range

$116,475 — $174,712.50 USD

Where we Work

We are embracing an office-first culture, encouraging three days a week in office for most roles, with meaningful opportunities to collaborate and celebrate in person.Each role may have specific requirements or flexibility depending on the scope of the position, so we encourage you to verify this with your recruiter during your first interview.

Where we Work

We are embracing an office-first culture, encouraging three days a week in office for most roles, with meaningful opportunities to collaborate and celebrate in person.Each role may have specific requirements or flexibility depending on the scope of the position, so we encourage you to verify this with your recruiter during your first interview.

Benefits

As an employee at OneTrust , you will be part of the OneTeam . That means you’ll receive support physically, mentally, and emotionally so that you can do your best work both in and out of the office. This includes comprehensive healthcare coverage, flexible PTO, equity RSUs, annual performance bonus opportunities, retirement account support, 14+ weeks of paid parental leave, career development opportunities, company-paid privacy certification exam fees, and much more. Specific benefits differ by country. For more information, talk to your recruiter or visit onetrust.com/careers.

Resources

Check out the following to learn more about OneTrust and its people:

• OneTrust Careers on YouTube
• @LifeatOneTrust on Instagram

Your Data

You have the right to have your personal data updated or removed. You also have the right to have a copy of the information OneTrust holds about you. Further details about these rights are available on the website in our Privacy Overview .You can change your mind at any time and have your personal data removed from our database. In order to do this you must contact us and let us know you wish to be removed. The request should be made on the Data Subject Request Form . Recruitment fraud warning: OneTrust is aware of scams involving false offers of employment with our company.The fraudulent jobs, interviews and job offers use fake websites, email addresses, group chat and text messages. Be aware that we never ask candidates for personal information, IDs or bank information during the interview process. We do not interview prospective candidates via instant message or group chat, and do not require candidates to purchase products or services, or process payments on our behalf as a condition of any employment offer. Please note that any legitimate interview availability requests will come directly from a OneTrust recruiter with an "@onetrust.com" email address. You may also receive legitimate emails from "@us.greenhouse-mail.io". Recruiters will only reach out to candidates who have applied for a role through our ATS (Greenhouse) or prospects via LinkedIn InMail. Job offers will come from a recruiter and may have a "@docusign.net" email address. For more information or if you have been targeted please reach out to [e-mail] .

Our Commitment to You

When you join OneTrust you are stepping onto a launching pad — the countdown has begun. The destination? A career without boundaries working alongside a diverse and inclusive crew who is passionate about doing meaningful work. As a pioneer, your voice and expertise will help chart the direction of an entirely new category. Our commitment to putting people first starts with you. Your growth is part of the mission. Our goal is to give you the power to embark on the next phase of your uniquely, unique career. OneTrust provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by local laws.