Staff Security Detection Engineer

Databricks Inc.; United States; Remote; USD 100,000 - 125,000; Full time

Job summary

Join a forward-thinking company as a Security Detection Engineer, where you'll safeguard products and infrastructure against cyber threats. This role offers the opportunity to leverage cutting-edge machine learning techniques and collaborate with cross-functional teams to enhance threat detection capabilities. You'll design scalable intrusion detection solutions and contribute to shaping the long-term success of the organization's security infrastructure. If you have a passion for cybersecurity and a desire to make a significant impact, this is the perfect opportunity for you to thrive in a dynamic environment.

Qualifications

• 10+ years in security detection engineering or advanced degree + 7 years.
• Expertise in cloud environments like AWS, Azure, or GCP.

Responsibilities

• Design and implement advanced detection strategies for new log sources.
• Collaborate with teams to build efficient log ingestion pipelines.

Skills

Cybersecurity

Machine Learning

Log Analysis

Software Development

Communication Skills

Cloud Security

Network Security

Data Analysis

Mentoring

Education

Advanced Degree in a relevant field

Bachelor's Degree in a relevant field

Tools

Python

Git/GitHub

CI/CD Automation

Terraform

Pyspark

SQL

Job description

RDQ126R166; This role can be based anywhere in the United States.

As part of the Detection & Response team at Databricks, you will play a critical role in safeguarding our products, cloud infrastructure, endpoints, and employees from modern cyber threats. Our team combines deep expertise in machine learning, log analysis, cybersecurity, and software development to create a robust and scalable detection platform. Embracing the "Detection-as-Code" model, we build detections on our own Databricks platform, ensuring that our security measures are deeply integrated with our technology stack.

In this role, you will work with cutting-edge machine learning techniques to design and implement scalable intrusion detection solutions at the enterprise level. You'll partner with our data engineering team to optimize log ingestion pipelines, fuse diverse log sources, and develop anomaly-based and ML-driven detection strategies. You will be instrumental in enhancing the organization's threat detection capabilities by utilizing novel data sources, exploring new attack vectors, and refining our detection models.

You will be an individual contributor on the Security Detection team at Databricks, reporting to the Sr Manager of Detection Engineering.

The impact you will have:

• Design and implement advanced detection strategies by deeply understanding and analyzing new or unknown log sources, schemas, and raw data.
• Collaborate with cross-functional teams, including data engineering, to build efficient log ingestion pipelines and support large-scale data analytics.
• Engineer and deploy detection solutions on Databricks using Spark, Python, and other cutting-edge technologies with a strong emphasis on clean code, rigorous testing, and comprehensive documentation.
• Develop Rule-based and/or ML-based intrusion detection models and integrate them with Databricks' platform, ensuring high accuracy and minimal false positives.
• Partner with Incident Response teams to perform threat hunting and to provide detailed logging, alerts, and playbooks, empowering proactive threat detection and response.
• Influence the development of long-term technology strategies and roadmaps for detection engineering, ensuring alignment with broader business and security goals.
• Represent Databricks at security and engineering conferences, presenting novel detection approaches and thought leadership within the security community.

What we look for:

• 10+ years of relevant experience or advanced degree + 7 years of experience, with a focus on security detection engineering.
• 6+ years of software engineering experience, with 4+ years specifically in security-related engineering, particularly in detection engineering.
• Expertise in securing and operating at least one major cloud environment (AWS, Azure, GCP).
• Strong technical proficiency in key areas such as network security, cloud security, application/log analysis, and endpoint security.
• Proven experience in Python, Git/GitHub, and CI/CD automation (terraform knowledge is a plus).
• Familiarity with distributed computing environments (e.g., Pyspark), SQL, data analysis tools, and machine learning.
• A strong passion for continuous learning and staying updated on evolving attack techniques and defense strategies.
• Excellent communication skills, with the ability to collaborate effectively across teams and present complex ideas clearly to stakeholders at all levels.
• A leadership mindset with the ability to mentor peers, drive strategic initiatives, and influence the organization’s security direction.

In this role, you will be expected to work autonomously and take ownership of large projects. Your work will directly contribute to shaping the long-term success of Databricks' security infrastructure, and you'll be a key driver in the continued evolution of our detection systems.