Software Engineer

Alegeus; Bengaluru; On-site; INR 3,500,000 - 5,500,000; Full time

Job summary

Alegeus is seeking an Expert Software Engineer to design, build, and scale a next-generation Data Platform and Data-Driven APIs. You will work on distributed data processing (Apache Spark) and backend platform engineering (Java) to enable reliable, real-time data access from Bengaluru and beyond.

The role emphasizes end-to-end ownership, data contracts, and API-driven datasets for product and partner consumption, with an opportunity to shape Lakehouse, data mesh, and cloud-native architectures.

Qualifications

• 8+ years of software engineering experience with data platforms or distributed systems.
• Hands-on expertise in Spark, Scala, PySpark, Java and REST/microservices.
• Experience with cloud platforms and data governance concepts.

Responsibilities

• Design scalable data pipelines using Apache Spark (batch and streaming).
• Build and maintain data platform layers: ingestion, transformation, and serving.
• Optimize Spark jobs for performance, cost, and reliability.

Skills

8+ years experience

Apache Spark

Java

Python

Scala

REST / microservices

Data modeling

Cloud platforms

Kafka / Event Hubs

Kubernetes

CI/CD for data/microservices

Tools

Apache Spark

Kafka

Airflow

Dagster

Kubernetes

Delta Lake / Iceberg / Hudi

REST API design

Azure / AWS / GCP

Job description

We are looking for an Expert Software Engineer to design, build, and scale our next-generation Data Platform and Data-Driven APIs. This role combines distributed data processing (Apache Spark) with platform and microservices engineering (Java) to enable reliable, scalable, and real-time data access.

You will operate at the intersection of data engineering and backend platform engineering-building systems that not only process large volumes of data but also expose that data through robust, well-designed APIs and services.

This role goes beyond implementing requirements. We expect engineers to understand business context, challenge assumptions, and take end-to-end ownership of delivering meaningful outcomes.

Key responsibilities

• Design and develop scalable data pipelines using Apache Spark (batch and streaming)
• Build and maintain data platform layers: ingestion, transformation, and serving
• Optimize Spark jobs for performance, cost, and reliability (partitioning, skew handling, memory tuning)
• Implement data quality, observability, and lineage frameworks
• Contribute to data architecture decisions (Lakehouse, data mesh, storage formats, partition strategies)
• Define and enforce data contracts and schema evolution practices
• Design and build data-driven platform APIs using Java (preferred)
• Develop microservices that expose curated datasets for product and partner consumption
• Implement RESTful APIs and event-driven services for real-time and near real-time data access
• Ensure low-latency, high-availability data serving layers
• Integrate with upstream/downstream systems, including legacy APIs where required
• Build and deploy solutions on Azure (preferred) / AWS / GCP
• Leverage cloud-native services for data storage, compute, and messaging
• Work with event streaming systems (Kafka/Event Hubs) for real-time pipelines
• Support containerized deployments and orchestration (Kubernetes) where applicable

Quality, Observability & Engineering Excellence

• Champion unit tests across both data and service layers
• Build automated validation frameworks for data pipelines
• Implement end-to-end observability (metrics, logging, tracing) across pipelines and APIs
• Drive CI/CD practices for both data and application code
• Conduct code reviews and enforce engineering best practices

Product Mindset & Ownership

• Engage deeply with product and business stakeholders to understand why, not just what
• Translate business problems into scalable data and platform solutions
• Take end-to-end ownership from design through production and support
• Proactively identify performance bottlenecks, data issues, and system gaps

Required qualifications (Hard requirements)

• 8+ years of software engineering experience with strong focus on data platforms and/or distributed systems
• Hands-on expertise in Apache Spark or Scala or PySpark
• Strong programming skills in Java (preferred) / Scala / Python
• Experience developing backend services or APIs (REST/microservices)
• Deep understanding of:
• Distributed systems (partitioning, shuffle, fault tolerance)
• Data modeling and schema evolution
• Experience with cloud platforms (Azure/AWS/GCP)
• Familiarity with workflow orchestration tools (Airflow, Dagster, etc.)
• Strong system design and performance optimization skills

Preferred qualifications

• Experience with Spark Structured Streaming
• Exposure to Lakehouse architectures (Delta Lake, Iceberg, Hudi)
• Experience with event-driven architectures (Kafka, Event Hubs)
• Knowledge of data governance, catalog, and lineage tools
• Experience with CI/CD for data and microservices
• Familiarity with Kubernetes and containerized workloads

What success looks like

A successful engineer in this role will:

• Deliver high-quality, production-grade data pipelines and APIs that power real business outcomes
• Build systems that are scalable, observable, and resilient underload
• Take ownership end-to-end, ensuring data flows reliably from source to consumer
• Balance data correctness, performance, and cost efficiency
• Contribute to evolving a modern data platform integrated with product-facing services