Senior Data Engineer with Databricks

Indra Philippines, Inc.; Pasig; On-site; PHP 2,000,000 - 3,200,000; Full time

Job summary

Indra Philippines, Inc. is seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines for batch and real-time processing in a cloud environment.

You will optimize Databricks workloads, implement ETL/ELT processes, and collaborate with data architects, scientists, and stakeholders to ensure data quality and performance.

Qualifications

• 8–10 years of experience in Data Engineering or related field.
• Hands-on Databricks experience in enterprise environments.
• Strong experience with Apache Spark / PySpark.
• Advanced SQL and solid Python skills.
• Experience with ETL/ELT, data warehousing, and pipelines.
• Experience with large-scale datasets and cloud services.
• Knowledge of data quality, data modeling, and performance optimization.
• Experience with source control and software development best practices.
• Strong analytical, problem-solving, and troubleshooting skills.

Responsibilities

• Design, develop, and maintain scalable data pipelines for batch and/or real-time data processing.
• Develop and optimize data engineering solutions using Databricks and related cloud data technologies.
• Build and maintain data pipelines, ETL/ELT processes, and data transformation workflows.
• Work with large and complex datasets to ensure data quality, consistency, availability, and performance.
• Develop and optimize SQL and Python code for data processing and transformation.
• Implement data engineering solutions using Apache Spark / PySpark.
• Collaborate with Data Architects, Data Scientists, Analysts, Application Developers, and business stakeholders to understand data requirements.
• Troubleshoot and resolve data pipeline, integration, and performance issues.
• Apply best practices for data governance, security, monitoring, and data quality.
• Participate in technical design, code reviews, testing, deployment, and production support.

Skills

Databricks

Apache Spark / PySpark

SQL

Python

ETL/ELT

Data warehousing

Data modeling

Data quality

Troubleshooting

Tools

Databricks

Apache Spark / PySpark

Job description

Key responsibilities

• Design, develop, and maintain scalable and reliable data pipelines for batch and/or real-time data processing.

• Develop and optimize data engineering solutions using Databricks and related cloud data technologies.

• Build and maintain data pipelines, ETL/ELT processes, and data transformation workflows.

• Work with large and complex datasets to ensure data quality, consistency, availability, and performance.

• Develop and optimize SQL and Python code for data processing and transformation.

• Implement data engineering solutions using Apache Spark / PySpark.

• Collaborate with Data Architects, Data Scientists, Analysts, Application Developers, and business stakeholders to understand data requirements.

• Troubleshoot and resolve data pipeline, integration, and performance issues.

• Apply best practices for data governance, security, monitoring, and data quality.

• Participate in technical design, code reviews, testing, deployment, and production support.

About you

• 8–10 years of experience in Data Engineering, Data Development, or a related field.

• Strong hands-on experience with Databricks in an enterprise data environment.

• Strong experience with Apache Spark / PySpark.

• Advanced proficiency in SQL and solid experience with Python.

• Strong understanding of ETL/ELT processes, data warehousing, and data pipeline development.

• Experience working with large-scale datasets and distributed data processing.

• Experience with cloud-based data platforms and services.

• Strong understanding of data quality, data modeling, and performance optimization.

• Experience with source control and software development best practices.

• Strong analytical, problem-solving, and troubleshooting skills.