Senior Backend Engineer (Python, Java, GO)

spiderSilk; Dubai; On-site; AED 350,000 - 650,000; Full time

Benefits offered by this job

Competitive compensation

Job summary

spiderSilk is seeking a Senior Software Engineer (Backend) to lead the design and delivery of mission-critical systems for its cybersecurity platform in the UAE. You will architect scalable, secure backend infrastructure, integrate ML pipelines, and develop agentic AI solutions at scale.

You’ll mentor junior engineers, drive architectural decisions, and uphold secure-by-design practices while delivering high-availability services for real-world threat intelligence workloads.

Qualifications

• Strong backend experience in Python/Go/Java across production systems.
• Experience building distributed, microservices-based architectures.
• Proven ability to design agentic AI or ML-powered backend features.

Responsibilities

• Architect and maintain scalable microservices for threat intelligence platform.
• Own design/delivery of standalone agentic AI solutions from architecture to production.
• Build and optimise ML pipelines, vector search, and real-time data streaming systems.
• Lead architectural discussions and mentor engineers on code quality and security.
• Champion secure-by-design practices across all stack layers.

Skills

Python

Go

Java

Docker

Kubernetes

Distributed systems

Event-driven

Vector databases

Concurrency

Tools

Kafka

RabbitMQ

Dagster

Airflow

S3

Job description

Who We Are

Back in 2019, spiderSilk was born with a bold idea: build regional, sovereign cybersecurity IP that could stand tall on the global stage. Our mission is to shake up the way organizations protect their ever-changing digital worlds - with continuous, intelligent, and autonomous security that doesn't miss a beat.

We're a global mix of curious minds, problem-solvers, and passionate builders, all united by one goal: making the internet a safer place for everyone. Around here, we thrive on vision, energy, and a strong sense of ownership.

If this feels like your kind of crew, you'll probably fit right in.

About the Role

As a Senior Software Engineer (Backend), you'll take a leadership role in designing and building the mission-critical systems that power spiderSilk's cybersecurity platform. You'll architect scalable, secure, and intelligent backend infrastructure - integrating ML pipelines, designing agentic AI solutions, and ensuring the platform can handle the demands of real-world threat intelligence at scale.

You won't wait to be told what to do. You'll jump into architectural discussions, challenge assumptions, and drive decisions. You'll mentor junior engineers and set the bar for engineering quality across the team.

What You'll Do

• Architect and maintain distributed, microservices-based systems that operate reliably at scale
• Own the design and delivery of standalone agentic AI solutions - from architecture through to production deployment
• Build and optimise ML pipelines, agent workflows, and vector-based similarity search systems
• Design and operate event-driven architectures using Kafka and message queues for real-time data streaming
• Engineer and optimise complex database schemas across relational, graph, search, and analytical data stores
• Lead data pipeline development for ingesting, processing, and surfacing threat intelligence data
• Proactively contribute to architectural decisions - bringing opinions, trade-off analysis, and technical leadership without waiting to be asked
• Mentor junior and mid-level engineers through code reviews, design discussions, and pair programming
• Champion secure-by-design engineering practices across every layer of the stack

Requirements

Languages & Stack

• Python, Go, and Java - strong proficiency in all three with production-grade backend experience across each
• Deep understanding of concurrency primitives - threads, goroutines, thread pools, locks, and race conditions; knowing when more parallelism makes things slower, not faster
• Solid working knowledge of Docker and Kubernetes for containerisation and orchestration
• Experience with Redis for caching, rate limiting, and distributed coordination

Distributed Systems & Architecture

• Deep experience designing and operating distributed systems and microservices architectures
• Proven track record building event-driven systems (Kafka, RabbitMQ, or equivalent)
• Expertise in data pipelines and stream processing at scale
• Strong system design instincts - able to reason independently about trade-offs across reliability, performance, and security, and step into architectural decisions without prompting
• Hands-on experience with wide-column stores (Cassandra or equivalent) for high-write, high-scale workloads
• Working knowledge of object storage (S3 or compatible) and workflow orchestration tools (Dagster, Airflow, or similar) for managing complex data pipelines

AI / Agentic Systems

• Hands-on experience building standalone agentic AI solutions in production - not just integrating APIs, but designing the agent architecture end-to-end
• Deep understanding of vector databases and similarity search (e.g. Pinecone, Weaviate, pgvector, Qdrant) and how to apply them effectively in AI-powered pipelines
• Proven work on agent optimisation and performance tuning - latency reduction, retrieval quality, context management, tool-call efficiency
• Familiarity with agentic frameworks (LangChain, LlamaIndex, AutoGen, or similar) and the ability to go beyond them when needed
• Practical experience with ML pipeline design, model serving, and feature engineering within production AI systems
• Strong prompt engineering skills and up-to-date fluency with the evolving AI tooling ecosystem

• Expert-level skills in database schema design and optimisation across multiple paradigms
• MySQL & PostgreSQL - advanced query optimisation, indexing strategies, performance tuning, and schema design for high-traffic systems
• Elasticsearch - indexing, search relevance tuning, and cluster operations
• Graph databases - production experience with Neo4j or equivalent (Amazon Neptune, ArangoDB); ability to model complex relationship data for threat intelligence or entity resolution
• Working knowledge of analytical / OLAP databases (e.g. ClickHouse, BigQuery, Redshift, DuckDB) and when to apply them
• Ability to select the right database technology for the problem and design schemas that scale

Data Collection & Web Technologies

• Experience collecting data from the web at scale - HTTP clients, headless browsers (Playwright or similar), handling sessions, rate limits, and sources that change without notice
• Ability to parse large, malformed, or untrusted files and archives safely and efficiently at volume
• Broad understanding of web servers, frameworks, CDNs and proxies - how they reveal themselves through headers, cookies, page content, and certificates
• Familiarity with software fingerprinting and vulnerability mapping - understanding how product versions map to known vulnerabilities (CPE, CVE)

Security

• Working knowledge of cybersecurity principles and common security tooling
• Demonstrated ability to apply secure coding practices and security-by-design thinking
• Understanding of threat modeling, vulnerability patterns, and secure API design
• Solid understanding of network security fundamentals - TCP/IP stack, DNS, TLS/SSL, HTTP/S, firewalls, and network traffic analysis
• Familiarity with network attack patterns - DDoS, MITM, port scanning, packet sniffing, and lateral movement techniques
• Experience working with or building systems that consume network telemetry data (e.g. NetFlow, PCAP, Zeek/Bro logs, SNMP)
• Understanding of zero-trust architecture principles and how to apply them in distributed, internet-facing systems
• Familiarity with security tooling such as Snort, Suricata, Wireshark, Nmap, or similar network inspection and monitoring tools

DevOps & Operational Awareness

• Basic working knowledge of CI/CD pipelines - setting up, maintaining, and debugging pipelines (GitHub Actions, GitLab CI, Jenkins, or equivalent)
• Comfortable with deployment workflows - deploying services to VMs and Kubernetes clusters, managing rollouts, and handling rollbacks
• Hands-on experience debugging live services on Kubernetes - inspecting pods, reading pod logs, exec-ing into containers, and triaging crashloops or resource issues
• Ability to fetch and analyse logs from VMs - SSH access, journald/systemd logs, log rotation, and using tools like grep, awk, tail, and jq to debug production issues
• Familiarity with observability tooling - working with log aggregation (ELK, Loki) and basic metrics/alerting (Prometheus, Grafana) to diagnose issues in production
• Understanding of Kubernetes autoscaling in depth - resource requests/limits, HPA, and event-driven autoscaling (KEDA); able to diagnose OOM kills, CPU throttling, and unclean shutdowns

Ownership & Leadership

• 5-8 years of backend engineering experience with a proven track record of delivering at scale
• Self-driven and proactive - comfortable jumping into architectural decisions, raising concerns early, and owning outcomes without hand‑holding
• Proven ability to lead projects and mentor engineers
• Strong written and verbal communication across technical and non-technical stakeholders

Nice to Have

• Background in cybersecurity tooling, threat intelligence platforms, or security research
• Exposure to cloud-native infrastructure (AWS, GCP, or Azure)
• Experience with time-series databases (InfluxDB, TimescaleDB) relevant to security telemetry
• Contributions to open-source AI or security tooling
• Background in offensive security, penetration testing, or vulnerability research
• Experience with threat intelligence work - tracking threat actors, leaked credentials, or underground forums
• High-volume pattern matching or text processing - large regex sets or parsers working on untrusted, multilingual input at scale
• Applied ML experience - clustering and visualising embeddings (e.g. UMAP) or other dimensionality reduction techniques for security data
• Leading a project or initiative that spans multiple teams or services