Druid DBA

Ztek Consulting; Tampa (FL); On-site; USD 150,000 - 210,000; Full time

Job summary

Ztek Consulting, Inc. seeks a Senior Apache Druid DBA to lead production-grade Druid clusters.

You will optimize ingestion pipelines, ensure sub-second OLAP queries, and drive reliability across petabyte-scale datasets in a distributed environment. Ideal candidates have deep distributed systems expertise, advanced database administration skills, JVM tuning, and real-time event processing, collaborating with networks, infra, and analytics teams to ensure robust data delivery and zero-downtime

Qualifications

• 12+ years of experience in mobile network applications, with at least 3 years in architecture and Kubernetes.
• Bachelor’s or master’s degree in electrical engineering, telecommunications, computer science, or a related field.

Responsibilities

• Architect, deploy, configure, and administer multi-node Apache Druid clusters across all roles.
• Execute zero-downtime rolling upgrades, patch management, and capacity sizing for storage pools.
• Perform deep JVM tuning, heap allocation, direct memory buffers, and GC optimization.
• Maintain cluster metadata stores (MySQL/PostgreSQL) and optimize deep storage interfaces (AWS S3, HDFS, Azure ADLS Gen2).

Skills

Apache Druid

SQL/PLSQL

Streaming systems

Distributed storage

Linux/JVM tuning

DevOps automation

Education

Bachelor’s or Master’s degree

Tools

MySQL

PostgreSQL

Oracle

Kafka

Flink/Spark

HDFS/AWS S3

Job description

Total Experience: 10 to 15+ years of experience in database administration, big data engineering, or systems infrastructure.

• Apache Druid Expertise: Minimum of 5+ years of dedicated, hands‑on production experience managing, troubleshooting, and tuning multi‑tier Apache Druid clusters.
• Database & Metadata Knowledge: Expert proficiency in SQL, PL/SQL, query optimization, and administering relational databases (MySQL, Oracle, and PostgreSQL) serving as Druid metadata stores.
• Streaming & Messaging: Hands‑on experience integrating Druid with real‑time streaming platforms (Apache Kafka, Amazon Kinesis, Pulsar).
• Distributed Systems & Storage: Solid knowledge of HDFS, AWS S3, or Blob Storage for deep storage management.
• OS & JVM Mastery: Deep understanding of Linux OS kernel settings, networking, JVM direct memory allocation, and GC optimization.
• Automation & DevOps: Proficiency with Linux/Bash scripting, Python, Ansible playbooks, Terraform, and CI/CD tools (Jenkins, GitLab).

We are seeking a Senior Apache Druid Database Administrator (DBA) with extensive hands‑on experience in managing, optimizing, and scaling production‑grade Apache Druid clusters. In this role, you will be responsible for maintaining high availability, optimizing sub‑second OLAP query performance, managing streaming and batch data ingestion pipelines, and ensuring seamless platform operations across petabyte‑scale datasets.

The ideal candidate will possess a deep background in distributed systems, advanced database administration, JVM tuning, and real‑time event processing. You will work closely with network architects, infrastructure teams, and analytics engineers to ensure robust data reliability, capacity planning, and zero‑downtime operations.

Key Responsibilities

1. Cluster Architecture & Administration

• Architect, deploy, configure, and administer multi‑node, multi‑tiered Apache Druid clusters across Coordinator, Overlord, Broker, Router, Historical, and MiddleManager/Indexer node roles.
• Execute zero‑downtime rolling upgrades, patch management, and capacity sizing for multi‑tier Historical storage pools and worker parallelism.
• Perform deep JVM tuning across all Druid service roles, including heap allocation, direct memory buffers, G1GC garbage collection parameters, and off‑heap memory settings.
• Maintain cluster metadata stores (MySQL / PostgreSQL) and optimize deep storage interfaces (AWS S3, HDFS, or Azure ADLS Gen2).
• Manage and troubleshoot real‑time streaming ingestion specs via Apache Kafka Supervisor services and Apache Flink/Spark integration.
• Design and execute batch ingestion specifications using Hadoop‑based or native SQL index tasks for historical network datasets (e.g., CDR, IPDR, and telemetry logs).
• Fine‑tune ingestion parameters, including taskCount, replicas, segmentGranularity, queryGranularity, and maxRowsPerSegment, to ensure sub‑5‑second end‑to‑end ingestion latency.
• Implement dead‑letter queue (DLQ) handling, schema evolution management, and schema‑validation enforcement.

3. Segment Lifecycle & Performance Tuning

• Design automatic compaction strategies, defining target segment size thresholds and skip offsets to eliminate segment fragmentation.
• Define data retention rules, load/drop policies, and tier assignment rules between hot and cold historical tiers to optimize storage costs against performance SLAs.
• Analyze slow query profiles via Broker access logs to tune dimension/metric cardinality, bitmap indexing, time‑based partitioning, and rollup strategies.
• Apply approximate algorithms (e.g., HyperLogLog, Theta Sketches, Quantiles) where appropriate to balance query execution speed with precision.

4. Operations, Observability & Security

• Build operational dashboards and alerts using Prometheus, Grafana, and Splunk/CloudWatch to track key cluster metrics (ingestion lag, segment count, cache hit ratios, GC pauses, memory pressure).
• Enforce security protocols, including Kerberos authentication, TLS/SSL encryption in transit and at rest, Role‑Based Access Control (RBAC), and Apache Ranger integration.
• Lead incident response, root cause analysis (RCA), and disaster recovery (DR) drills, maintaining runbooks for automated recovery and zero‑loss operations.

Education

Bachelor’s or master's degree in electrical engineering, telecommunications, computer science, or a related field.

Experience

12+ years of experience in mobile network applications, with at least 3 years in architecture and Kubernetes

Generic Managerial Skills:

• Crisis Management Staying calm during a massive network outage to direct the IT NOC team effectively.
• Stakeholder Mgmt. Explaining to non‑technical C‑level why a Druid migration is necessary