Data Engineer

Moolya Software Testing

Bengaluru

On-site

INR 1,200,000 - 2,400,000

Full time

5 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Moolya Software Testing in Bengaluru, Karnataka, seeks an experienced data engineer to design, build, and maintain scalable ETL/ELT pipelines using Spark, PySpark, Spark SQL. You will optimize SQL queries, orchestrate workflows with Airflow or Dagster, and manage Databricks workloads.

Responsibilities include designing data models, lakehouse architectures, and governance practices, plus ownership of production workloads, troubleshooting, and collaboration with analysts and stakeholders to

Qualifications

  • Strong experience with Apache Spark, PySpark, and Spark SQL.
  • Proficiency in Python, Java, or Scala.
  • Strong SQL skills, including query optimization and performance tuning.
  • Proven experience building and operating production-grade ETL/ELT pipelines.
  • Experience with Airflow, Dagster, or similar orchestration tools.
  • Hands-on experience with Databricks or a comparable Spark-based platform.
  • Solid understanding of data modeling, data warehousing, and lakehouse concepts.
  • Experience with Parquet, Delta Lake, and/or Apache Iceberg.
  • Familiarity with data quality, metadata, lineage, and governance practices.
  • Experience owning production workloads, troubleshooting issues, and performing root-cause analysis.
  • Strong analytical, problem-solving, and communication skills.
  • Ability to work independently while collaborating effectively across teams.

Responsibilities

  • Design, build, and maintain scalable ETL/ELT pipelines using Apache Spark, PySpark, and Spark SQL.
  • Develop and optimize complex SQL queries, with a focus on performance tuning and cost efficiency.
  • Orchestrate and schedule workflows using Airflow, Dagster, or similar platforms.
  • Build and manage data workloads on Databricks or comparable Spark-based platforms.
  • Design data models, warehouse layers, and lakehouse structures that support analytics and reporting.
  • Work with Parquet, Delta Lake, and/or Apache Iceberg table formats.
  • Implement data quality checks, metadata management, lineage tracking, and governance practices.
  • Own production workloads: monitor jobs, troubleshoot failures, and perform root-cause analysis to prevent recurrence.
  • Collaborate with analysts, engineers, and business stakeholders to understand requirements and deliver trusted data.
  • Document pipelines, data models, and operating procedures.

Skills

Apache Spark
PySpark
Spark SQL
Python
Java
Scala
SQL
Airflow
Dagster
Databricks
Data modeling
Data warehousing
Lakehouse concepts

Tools

Airflow
Dagster
Databricks

Job description

Role & responsibilities
  • Design, build, and maintain scalable ETL/ELT pipelines using Apache Spark, PySpark, and Spark SQL.
  • Develop and optimize complex SQL queries, with a focus on performance tuning and cost efficiency.
  • Orchestrate and schedule workflows using Airflow, Dagster, or similar platforms.
  • Build and manage data workloads on Databricks or comparable Spark-based platforms.
  • Design data models, warehouse layers, and lakehouse structures that support analytics and reporting.
  • Work with Parquet, Delta Lake, and/or Apache Iceberg table formats.
  • Implement data quality checks, metadata management, lineage tracking, and governance practices.
  • Own production workloads: monitor jobs, troubleshoot failures, and perform root-cause analysis to prevent recurrence.
  • Collaborate with analysts, engineers, and business stakeholders to understand requirements and deliver trusted data.
  • Document pipelines, data models, and operating procedures.

Preferred candidate profile
  • Strong experience with Apache Spark, PySpark, and Spark SQL.
  • Proficiency in Python, Java, or Scala.
  • Strong SQL skills, including query optimization and performance tuning.
  • Proven experience building and operating production-grade ETL/ELT pipelines.
  • Experience with Airflow, Dagster, or similar orchestration tools.
  • Hands-on experience with Databricks or a comparable Spark-based platform.
  • Solid understanding of data modeling, data warehousing, and lakehouse concepts.
  • Experience with Parquet, Delta Lake, and/or Apache Iceberg.
  • Familiarity with data quality, metadata, lineage, and governance practices.
  • Experience owning production workloads, troubleshooting issues, and performing root-cause analysis.
  • Strong analytical, problem-solving, and communication skills.
  • Ability to work independently while collaborating effectively across teams.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

Quadrant Technologies • Hyderabad

On-site
INR 1,800,000 - 3,200,000
Databricks Data Engineer
Databricks Data Engineer

Infosys • Pune District, Bengaluru

On-site
INR 900,000 - 1,400,000
Data Engineer
Data Engineer

Bdreamz Global Solutions • Chennai

Hybrid
INR 1,200,000 - 2,200,000
Associate Data Engineer
Associate Data Engineer

Amgen Inc. • Hyderabad

On-site
INR 1,800,000 - 2,600,000
Data Engineer - Databricks
Data Engineer - Databricks

PwC • Hyderabad, Pune District, Bengaluru

On-site
INR 1,200,000 - 2,400,000
Data Engineer ( AWS & Databricks)
Data Engineer ( AWS & Databricks)

Tekskills Inc. • Dadri

On-site
INR 1,400,000 - 2,500,000
Senior Data Engineer
Senior Data Engineer

Amgen Technology • Hyderabad

Hybrid
INR 2,500,000 - 4,000,000
Data Engineer ( AWS & Databricks)
Data Engineer ( AWS & Databricks)

Tekskills Inc. • Kolkata District

On-site
INR 1,500,000 - 2,300,000
Data Engineer ( AWS & Databricks)
Data Engineer ( AWS & Databricks)

Tekskills Inc. • Bengaluru

On-site
INR 1,500,000 - 2,100,000
Data Engineer ( Azure & Databricks )
Data Engineer ( Azure & Databricks )

Lufthansa Technik AG • Bengaluru

Hybrid
INR 1,800,000 - 3,200,000