Big Data Administrator & Development Engineer

SMA Solar India; Bengaluru; On-site; INR 1,400,000 - 2,300,000; Full time

Benefits offered by this job

Equal opportunity employer

Job summary

SMA Solar India Pvt. Ltd. in Bengaluru is seeking a Big Data Administrator & Development Engineer to ensure stable on‑premise Hadoop, Kafka, and Druid clusters while driving automation and performance improvements.

This role blends hands‑on administration with development tasks, including ETL design with NiFi/Python/Spark, RBAC and security hardening, and collaboration with cross‑functional teams on architecture and upgrades.

Qualifications

• Minimum 8 years of hands‑on experience in administering on‑premise Hadoop, Kafka, and Druid clusters.
• Proven expertise with HDFS, Hive, Spark, Hue, and NiFi in enterprise data environments.
• Strong SQL proficiency, including query tuning and performance optimization.
• Solid understanding of Linux system administration and cluster maintenance best practices.
• Experience with automation and scripting tools such as Ansible, Python, or Bash.

Responsibilities

• Manage and Optimize Big Data Clusters: ensure reliable, secure, and high‑performance operation of on‑premise Hadoop, Kafka, and Druid clusters.
• Implement and Maintain Monitoring Solutions: deploy and maintain monitoring and alerting tools such as Cloudera Manager and Splunk to track system availability and performance.
• Administer Access and Security Controls: manage RBAC, encryption, and compliance policies to safeguard data integrity.
• Develop and Automate Data Workflows: design ETL processes and data transfers using SQL, NiFi, Python, or Spark.
• Support Architectural and Technical Evolution: collaborate on architecture design, system upgrades, and technology evaluations to improve scalability.

Skills

Big Data administration

Data platforms

System monitoring

Automation scripting

Education

Bachelor's or Master’s degree in Computer Science or related field

Tools

Hadoop

Kafka

Druid

HDFS

Hive

Spark

Hue

NiFi

Cloudera Manager

Splunk

Ansible

Python

Bash

Job description

Are you passionate about data platforms, high-performance systems, and large-scale data processing? Join our innovative team as a Big Data Administrator & Development Engineer, where you’ll be responsible for ensuring the stability, security, and efficiency of on-premise Big Data environments. You will manage and enhance clusters based on Hadoop, Kafka, and Druid, while also contributing to automation, system optimization, and strategic architectural decisions.

This role combines deep technical administration with development responsibilities, offering the opportunity to work on cutting‑edge data infrastructure that powers critical business insights and decision‑making.

Key Responsibilities

• Manage and Optimize Big Data Clusters: ensure the reliable, secure, and high‑performance operation of on‑premise Hadoop, Kafka, and Druid clusters, proactively monitoring performance and system health.
• Implement and Maintain Monitoring Solutions: deploy and maintain monitoring and alerting tools such as Cloudera Manager and Splunk to track system availability, detect anomalies, and ensure proactive issue resolution.
• Administer Access and Security Controls: manage role‑based access control (RBAC), encryption, and compliance policies to safeguard data integrity and meet security standards.
• Develop and Automate Data Workflows: design and implement ETL processes, data transfers, and analysis pipelines using SQL, NiFi, Python, or Spark to support evolving business requirements.
• Support Architectural and Technical Evolution: collaborate with cross‑functional teams on architecture design, system upgrades, and technology evaluations to improve scalability, reliability, and performance.

Required Qualifications

• Minimum 8 years of hands‑on experience in administering on‑premise Hadoop, Kafka, and Druid clusters.
• Proven expertise with HDFS, Hive, Spark, Hue, and NiFi in enterprise data environments.
• Strong SQL proficiency, including query tuning and performance optimization.
• Solid understanding of Linux system administration and cluster maintenance best practices.
• Experience with automation and scripting tools such as Ansible, Python, or Bash.

Preferred Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• Experience developing ETL pipelines and integrating diverse data sources.
• Familiarity with cloud‑based Big Data platforms (Azure preferred).
• Knowledge of disaster recovery, high‑availability architectures, and runbook creation.
• Excellent English communication skills and ability to collaborate effectively in agile, cross‑functional teams.

SMA is committed to equal opportunity and diversity – regardless of gender, age, origin, religion, disability, or sexual orientation.

Your Recruiting Contact

Anita Virnave | SMA Solar India Pvt. Ltd.