Druid Dba

Tata Consultancy Services; Tampa (FL); On-site; USD 110,000 - 125,000; Full time

Job summary

Tata Consultancy Services Limited in Tampa, FL seeks a Senior Apache Druid DBA with 10+ years in database/Big Data roles to manage production Druid clusters, ensure sub-second OLAP queries, and handle streaming ingestion pipelines. You will work with infrastructure, analytics engineers, and networks to ensure robust data reliability and zero-downtime operations.

The ideal candidate has deep distributed systems experience, JVM tuning, and real-time processing expertise.

Qualifications

• 10–15+ years in DB Admin, Big Data, or systems infra.
• 5+ years production Apache Druid experience across multi-tier clusters.
• SQL/PLSQL, and administering relational DBs serving as Druid metadata stores.
• Real-time streaming with Kafka, Kinesis, or Pulsar integration.
• Knowledge of HDFS, AWS S3, or Blob Storage for deep storage.
• Linux OS, JVM tuning, GC optimization, and memory management.
• Automation with Bash, Python, Ansible, Terraform; CI/CD tools.

Responsibilities

• Architect, deploy, configure, and administer multi-node Druid clusters.
• Perform rolling upgrades, patching, and capacity sizing for storage pools.
• Tune JVM across services and maintain metadata stores and deep storage.
• Design and optimize real-time streaming and batch ingestion pipelines.

Skills

Total Experience

Apache Druid

SQL/PLSQL

Streaming & Messaging

Distributed Systems

OS & JVM Mastery

Automation & DevOps

Education

Bachelor's or Master's in Electrical Engineering, Telecom, CS

Tools

MySQL

Oracle

PostgreSQL

Apache Kafka

Amazon Kinesis

Pulsar

HDFS

AWS S3

Blob Storage

Linux

Job description

Must Have Technical/Functional Skills

• Total Experience: 10 to 15+ years of experience in Database Administration, Big Data Engineering, or Systems Infrastructure.
• Apache Druid Expertise: Minimum of 5+ years of dedicated, hands‑on production experience managing, troubleshooting, and tuning multi‑tier Apache Druid clusters.
• Database & Metadata Knowledge: Expert proficiency in SQL, PL/SQL, query optimization, and administering relational databases (MySQL, Oracle, PostgreSQL) serving as Druid metadata stores.
• Streaming & Messaging: Hands‑on experience integrating Druid with real‑time streaming platforms (Apache Kafka, Amazon Kinesis, Pulsar).
• Distributed Systems & Storage: Solid knowledge of HDFS, AWS S3, or Blob Storage for Deep Storage management.
• OS & JVM Mastery: Deep understanding of Linux OS kernel settings, networking, JVM direct memory allocation, and GC optimization.
• Automation & DevOps: Proficiency with Linux/Bash scripting, Python, Ansible playbooks, Terraform, and CI/CD tools (Jenkins, GitLab).

Roles & Responsibilities

We are seeking a Senior Apache Druid Database Administrator (DBA) with extensive hands‑on experience in managing, optimizing, and scaling production‑grade Apache Druid clusters. In this role, you will be responsible for maintaining high availability, optimizing sub‑second OLAP query performance, managing streaming and batch data ingestion pipelines, and ensuring seamless platform operations across petabyte‑scale datasets.

The ideal candidate will possess a deep background in distributed systems, advanced database administration, JVM tuning, and real‑time event processing. You will work closely with network architects, infrastructure teams, and analytics engineers to ensure robust data reliability, capacity planning, and zero‑downtime operations.

Key Responsibilities

1. Cluster Architecture & Administration
   • Architect, deploy, configure, and administer multi‑node, multi‑tiered Apache Druid clusters across Coordinator, Overlord, Broker, Router, Historical, and MiddleManager/Indexer node roles.
   • Execute zero‑downtime rolling upgrades, patch management, and capacity sizing for multi‑tier Historical storage pools and worker parallelism.
   • Perform deep JVM tuning across all Druid service roles, including heap allocation, direct memory buffers, G1GC garbage collection parameters, and off‑heap memory settings.
   • Maintain cluster metadata stores (MySQL / PostgreSQL) and optimize deep storage interfaces (AWS S3, HDFS, or Azure ADLS Gen2).
2. Ingestion & Pipeline Engineering
   • Manage and troubleshoot real‑time streaming ingestion specs via Apache Kafka Supervisor services and Apache Flink/Spark integration.
   • Design and execute batch ingestion specifications using Hadoop‑based or native SQL index tasks for historical network datasets (e.g., CDR, IPDR, and telemetry logs).
   • Fine‑tune ingestion parameters including taskCount, replicas, segmentGranularity, queryGranularity, and maxRowsPerSegment to ensure sub‑5 second end‑to‑end ingestion latency.
   • Implement dead‑letter queue (DLQ) handling, schema evolution management, and schema‑validation enforcement.
3. Segment Lifecycle & Performance Tuning
   • Design automatic compaction strategies, defining target segment size thresholds and skip offsets to eliminate segment fragmentation.
   • Define data retention rules, load/drop policies, and tier assignment rules between Hot and Cold Historical tiers to optimize storage costs against performance SLAs.
   • Analyze slow query profiles via Broker access logs to tune dimension/metric cardinality, bitmap indexing, time‑based partitioning, and rollup strategies.
   • Apply approximate algorithms (e.g., HyperLogLog, Theta Sketches, Quantiles) where appropriate to balance query execution speed with precision.
4. Operations, Observability & Security
   • Build operational dashboards and alerts using Prometheus, Grafana, and Splunk/CloudWatch to track key cluster metrics (ingestion lag, segment count, cache hit ratios, GC pauses, memory pressure).
   • Enforce security protocols including Kerberos authentication, TLS/SSL encryption in transit and at rest, Role‑Based Access Control (RBAC), and Apache Ranger integration.
   • Lead incident response, root cause analysis (RCA), and disaster recovery (DR) drills, maintaining runbooks for automated recovery and zero‑loss operations.

Education

Education: Bachelor's or Master's degree in Electrical Engineering, Telecommunications, Computer Science, or related field.

Experience

Experience: 12+ years of experience in mobile network applications , with at least 3 years in an architectural and Kubernetes

Generic Managerial Skills

• Crisis Management Staying calm during a massive network outage to direct the IT NOC team effectively.
• Stakeholder Mgmt. Explaining to non-technical C-level why a Druid Migration is necessary

Salary Range

Salary Range: $110,000-$125,000 a year

Location

Tampa, FL

Function

TECHNOLOGY

Role

Senior Engineer

ID

438036

Skills

Database Administration

Additional Salary Range

$110,000 - $125,000