Data engineer with Scala + spark + SQL

HMG AMERICA LLC; Bangalore Rural; On-site; INR 1,800,000 - 3,200,000; Full time

Job summary

HMG AMERICA LLC in Bangalore is seeking an experienced Data Engineer to design, build, and scale enterprise-grade data platforms using Scala, Spark, SQL, and cloud services (GCP or AWS/Azure).

You will develop scalable pipelines, implement real-time and batch processing with Spark and Kafka, and optimize ETL while collaborating with data scientists and analysts. Immediate joiners preferred.

Qualifications

• 3+ years of experience in Data Engineering mandatory.
• Strong expertise in Spark, Scala, SQL, and cloud technologies.
• Experience with real-time and batch data processing.
• Experience with GCP or AWS/Azure cloud platforms.
• Strong SQL development and performance tuning.

Responsibilities

• Design, develop, and maintain scalable, fault-tolerant data pipelines and streaming architectures.
• Build and optimize real-time and batch data processing applications using Apache Spark, Scala, and Apache Kafka.
• Develop cloud-based data engineering solutions on GCP or other cloud platforms (AWS/Azure).
• Write complex SQL queries and optimize data processing workflows.
• Design large-scale data architectures with focus on performance, scalability, reliability, and security.
• Collaborate with Data Scientists, Business Analysts, and cross-functional teams to deliver data-driven solutions.
• Optimize existing ETL pipelines and improve system efficiency.
• Drive modern data engineering practices including CI/CD, automated testing, and data quality frameworks.

Skills

Apache Spark

Scala

Apache Kafka

SQL

GCP / AWS / Azure

Tools

Docker

Kubernetes

Apache Airflow

Job description

Job Title: Data Engineer+ Scala + Spark + SQL

Experience: 3–10 Years

Employment Type: Full-Time

Job Summary

Notice Period: Immediate Joiners or candidates serving a short notice period will be preferred.

We are looking for an experienced Data Engineer with strong expertise in Scala, Apache Spark, SQL, and GCP (or any cloud platform) to build and scale enterprise-grade data platforms and pipelines.

Key Responsibilities

• Design, develop, and maintain scalable, fault-tolerant data pipelines and streaming architectures.
• Build and optimize real-time and batch data processing applications using Apache Spark, Scala, and Apache Kafka.
• Develop high-performance, cloud-based data engineering solutions on GCP or other cloud platforms (AWS/Azure).
• Write complex SQL queries and optimize data processing workflows.
• Design large-scale data architectures with a focus on performance, scalability, reliability, and security.
• Collaborate with Data Scientists, Business Analysts, and cross-functional teams to deliver data-driven solutions.
• Optimize existing ETL pipelines and improve system efficiency.
• Drive modern data engineering practices including CI/CD, automated testing, and data quality frameworks.

Mandatory Skills

• Strong hands-on experience with Apache Spark (Batch & Streaming)
• Strong programming expertise in Scala
• Experience with Apache Kafka (Streaming, Producers/Consumers, Topic Management)
• Strong SQL development and query optimization skills
• Hands-on experience with Google Cloud Platform (GCP) or any major cloud platform (AWS/Azure)

Good to Have

• Experience with Data Lakes and Lakehouse Architectures (Delta Lake, Apache Iceberg, etc.)
• Knowledge of Docker and Kubernetes
• Experience with workflow orchestration tools such as Apache Airflow
• Understanding of DevOps practices and CI/CD pipelines
• Exposure to monitoring and logging tools for distributed systems

Preferred Candidate Profile

• 3 + years of experience in Data Engineering mandatory
• Strong expertise in Scala, Spark, SQL, and Cloud technologies