Staff Software Engineer

Epsilon; Bengaluru; On-site; INR 1,800,000 - 2,500,000; Full time

Job summary

Epsilon is looking for an experienced professional with over 12 years in Scala and extensive knowledge of Apache Spark for big data processing. The role requires proficiency in Python and a strong understanding of the Hadoop ecosystem.

Candidates should have hands-on experience with Databricks, database systems, and the ELK stack. Strong analytical skills and the ability to mentor junior staff are essential. The position is based in Bengaluru, Karnataka, India.

Skills

Scala

Apache Spark

Python

Hadoop ecosystem

SQL

Data Warehouse

GIT

Databricks

ELK stack

Data visualization tools

Docker

Kubernetes

Job description

Requirements:

• Strong experience (12+ years) in the Scala programming language and extensive experience with Apache Spark for big data processing for developing and supporting both on-prem and cloud operations, especially on AWS and as needed with GCP cloud.
• Using Python to develop infrastructure modules. Hence, hands-on experience with Python.
• In-depth understanding of the Hadoop ecosystem, including HDFS, YARN, and MapReduce.
• Solid grasp of database systems and SQL for writing efficient SQLs(RDBMS/Warehouse) to handle TBS of data.
• Experience in building, scheduling and maintaining DAG workflows.
• Familiarity with data warehousing concepts and technologies.
• End-to-end ownership with definition, development, and documentation of software's objectives, business requirements, deliverables, and specifications in collaboration with partners.
• Experience in working on GIT (or equivalent source control).
• Understand Unit and integration test frameworks.
• Must have the ability to collaborate with stakeholders/teams to understand requirements and develop a working solution.
• Ability to work within tight deadlines and effectively prioritise and complete tasks in a high-pressure environment.
• Must be able to mentor junior staff.
• Hands‑on with Databricks for unified data analytics, including Databricks Notebooks, Delta Lake, and Catalogues.
• Proficiency in using the ELK (Elasticsearch, Logstash, Kibana) stack for real-time search, log analysis, and visualisation.
• Strong background in analytics, including the ability to derive actionable insights from large datasets and support data-driven decision‑making.
• Experience with data visualisation tools like Tableau, Power BI, or Grafana.
• Familiarity with Docker for containerization and Kubernetes for orchestration.