Senior Software Engineer / Tech Lead - Message Security Detection

United States Digital Space LLC; Bengaluru; Hybrid; INR 4,000,000 - 6,000,000; Full time

Job summary

Abnormal AI is seeking a Tech Lead (SWE-3) to own the architectural blueprint for our distributed data pipelines (Spark/Airflow) and cloud-native infrastructure (AWS/EKS). You will mentor engineers, drive architectural excellence, and lead AI productivity initiatives to accelerate development across the team.

You will collaborate with Data Scientists, Security Researchers, and TPMs to translate complex needs into scalable solutions and ensure high-throughput, low-latency performance in our

Qualifications

• 8+ years of professional backend experience using Python or Go.
• Expert in building distributed data pipelines with Spark and Airflow.
• Extensive cloud experience on AWS (EKS) / GCP / Azure with container orchestration.
• Strong leadership, mentoring, and technical strategy contributions.
• Strong system design capabilities for high-throughput, low-latency systems.
• Experience with performance debugging and bottleneck resolution.
• Proficient in PostgreSQL or similar relational databases.
• BS in Computer Science, Applied Sciences, or related field.

Responsibilities

• Architect and scale high-throughput detection systems and backend services with low latency.
• Lead data pipelines using Spark and Airflow to supply ML data.
• Define technical roadmaps with Engineering Managers and mentor engineers.
• Drive AI productivity initiatives using Claude, OpenAI, and GitHub Copilot.
• Collaborate with TPMs, Product Managers, Data Scientists, and Security Researchers.
• Maintain cloud-native infra on AWS/EKS for cost, security and performance.

Skills

Backend development (Python/Golang)

Data pipelines (Spark/Airflow)

Cloud & container orchestration (AWS/E

Education

BS in Computer Science or related engineering field

Tools

Databricks

Snowflake

React/TypeScript

Job description

The Mission

We are building the high-throughput, low-latency infrastructure that powers our real-time threat detection capabilities. This isn't just about maintaining stable backend systems; it’s about architecting a cutting-edge data platform at massive scale to turn raw data into high-fidelity, actionable intelligence for our ML models and security researchers.

As a Tech Lead (SWE-3) on this team, you will be our technical anchor and force multiplier. You will own the architectural blueprint for our distributed data pipelines (Spark/Airflow) and cloud-native infrastructure (AWS/EKS), bridging the gap between complex system design and strategic execution. Beyond traditional engineering, you will lead by example spearheading our AI productivity initiatives to supercharge how our entire engineering team builds, tests, and ships software. If you are passionate about deep technical ownership, mentoring the next generation of engineers, and defining the future of our data ecosystem, this is your mission.

What You Will Do

• Architect & Scale: Design and maintain high-throughput detection systems and backend services, ensuring low latency and high reliability.
• Data Orchestration: Lead the development of robust data pipelines usingSpark and Airflow to supply detection teams with high-fidelity data for ML model training and evaluation.
• Technical Leadership: Partner with Engineering Managers to define technical roadmaps, mentor junior/mid-level engineers, and drive best practices in code quality and system design.
• AI-Driven Innovation: Lead the team in adopting AI productivity initiatives, leveraging tools like Claude, OpenAI, and GitHub Copilot to accelerate the development lifecycle and automate internal workflows.
• Cross-functional Collaboration: Work closely with TPMs, Product Managers, Data Scientists, and Security Researchers to translate complex business needs into scalable technical solutions.
• Operational Excellence: Manage and optimize cloud-native infrastructure on AWS/EKS, ensuring systems are cost-effective, secure, and performant.

Must Haves

• 8+ years of professional experience: Proven track record in production-level backend development with Python or Golang.
• Data Engineering Expertise: Expert-level proficiency in building and optimizing distributed data pipelines using Spark and Airflow (or equivalent orchestration tools).
• Cloud & Infrastructure: Extensive experience managing and scaling cloud-native applications on AWS (highly preferred), GCP, or Azure, with a proven track record of hands‑on container orchestration using EKS.
• Leadership & Mentorship: Experience leading technical projects, mentoring engineers, and contributing to the long-term technical strategy of a team.
• System Design: Strong ability to design complex integrations and handle significant throughput/latency challenges.
• Problem Solving: A methodical approach to performance debugging, benchmarking, and resolving bottlenecks in large-scale systems.
• Database Proficiency: Strong experience with Postgres or similar relational databases at scale.
• Education: BS in Computer Science, Applied Sciences, or a related engineering field.

Nice to Have

• Frontend Literacy: Familiarity with React and TypeScript to assist with internal tool visualizations.
• Data Ecosystems: Experience with Databricks, Snowflake, or similar data lakehouse architectures.
• Cybersecurity Domain: Background in threat detection, network security, or fraud prevention.
• Advanced Degrees: MS in Computer Science or Electrical Engineering.

#LI-LR1

**A note on AI in our process:**Abnormal AI uses AI-assisted tools to help our recruiting team prepare for candidate interviews. These tools analyze resume content and role requirements to suggest interview questions and areas for the interviewer to explore. They do not make hiring decisions or screen candidates automatically. Every decision about a candidacy is made by a person. Further, if your application is successful and Abnormal AI makes a conditional offer of employment, we will carry out pre-employment checks which must be successfully completed to progress to a final offer. All processes and pre-employment checks are in line with prevailing legislation and Abnormal AI's policies relevant to our security and privacy standards.

Abnormal AI is an equal opportunity employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability, protected veteran status or other characteristics protected by law. For our EEO policy statement please *click here*. If you would like more information on your EEO rights under the law, please*click here*.