Senior Data Engineer (Remote)

iScale Solutions; Makati; On-site; PHP 1,200,000 - 1,800,000; Full time

Job summary

iScale Solutions in Makati, Philippines, is seeking an experienced data engineer to own end-to-end data pipelines, enabling reliable ingestion and processing of large-scale datasets across platforms.

You will design scalable architectures, implement robust QA, and leverage AWS services, Hive, Spark, and Airflow while collaborating with analysts, product, and data science teams to translate requirements into dependable data solutions.

Qualifications

• 5+ years of professional experience in data engineering, with ownership of production pipelines and large-scale data processing.
• Strong SQL expertise with complex joins, window functions, transformations, query tuning, and troubleshooting.
• Experience with Hive and the Hadoop ecosystem, including partitioning and distributed processing concepts.
• Proficiency in Python for data processing, automation, and maintainable production code.
• Hands-on experience with S3, Athena, EC2, and EMR on AWS.
• Strong experience with Airflow or other ETL/workflow tools, Airflow preferred.
• Experience implementing data quality checks, monitoring, error handling, and operations support.
• Ability to translate business requirements into technical designs and deliver solutions with minimal supervision.
• Strong documentation and communication skills for stakeholders.
• Experience with Jira and Confluence for task tracking and delivery coordination.
• Hands-on experience creating AI agents or AI-assisted automations.

Responsibilities

• Own data pipeline development: Design, develop, and maintain ETL/ELT pipelines for large-scale datasets across platforms.
• Lead technical design: Define processing logic, data models, integration approaches, and workflow dependencies.
• Develop and optimize data processing: Write SQL, Hive, and Python; use Spark where appropriate.
• Build and manage AWS workflows: Use S3, Athena, EC2, EMR; monitor and optimize resources.
• Manage workflow orchestration: Build automated workflows with Airflow, including scheduling, dependencies, retries, alerts.
• Ensure production reliability: Monitor pipelines, investigate failures, perform root cause analysis, implement improvements.
• Maintain data quality and governance: Validation checks, lineage, access controls across data lifecycle.
• Lead and mentor a small team: Guide 2-3 engineers, break requirements into tasks, review code and designs.
• Collaborate with stakeholders: Work with analytics, product, engineering, and data science to clarify requirements.
• Maintain technical documentation: Create/update solution designs, data flow diagrams, dictionaries, runbooks in Confluence.
• Apply AI to engineering workflows: Identify AI opportunities, build simple AI agents, validate outputs.
• Support agile delivery: Use Jira, contribute to sprint planning and technical reviews.

Skills

SQL expertise

Hive & Hadoop

Python

AWS

Airflow

Data quality

Documentation

Jira & Confluence

AI for engineering

Team mentoring

Tools

PySpark

Snowflake

OpenAI Codex

Snowflake Cortex

Job description

• Own data pipeline development: Design, develop, and maintain reliable ETL/ELT pipelines for ingesting, transforming, and delivering large-scale datasets across multiple platforms and sources.
• Lead technical design: Define processing logic, data models, integration approaches, and workflow dependencies, considering scalability, maintainability, security, and cost.
• Develop and optimize data processing: Write efficient SQL, Hive queries, and Python code. Apply distributed processing frameworks such as Spark where appropriate.
• Build and manage AWS workflows: Develop and support data solutions using Amazon S3, Athena, EC2, and EMR, including monitoring and resource optimization.
• Manage workflow orchestration: Build and maintain automated workflows using Apache Airflow or comparable orchestration tools, including scheduling, dependencies, retries, alerting, and recovery.
• Ensure production reliability: Monitor pipeline performance and data freshness, investigate failures, perform root cause analysis, and implement improvements that prevent recurring issues.
• Maintain data quality and governance: Establish validation checks, reconciliation processes, lineage documentation, and appropriate access controls throughout the data lifecycle.
• Lead and mentor a small team: Guide 2-3 engineers, break requirements into actionable tasks, review code and designs, support troubleshooting, and track delivery commitments.
• Collaborate with stakeholders: Work closely with analytics, product, engineering, and data science teams to clarify requirements, agree on priorities, communicate progress, and manage dependencies.
• Maintain technical documentation: Create and update solution designs, data flow diagrams, data dictionaries, operating procedures, and support runbooks using Confluence or similar platforms.
• Apply AI to engineering workflows: Identify practical opportunities to use AI for development, documentation, troubleshooting, and automation. Build simple AI agents and validate their outputs before operational use.
• Support agile delivery: Manage work through Jira, contribute to sprint planning and technical reviews, and coordinate across regions and work schedules as required.

Requirements

• 5+ years of professional experience in data engineering, with demonstrated ownership of production data pipelines and large-scale data processing.
• Strong SQL expertise, including complex joins, window functions, transformations, query tuning, and troubleshooting.
• Strong working knowledge of Hive and the Hadoop ecosystem, including partitioning, storage formats, and distributed data processing concepts.
• Proficiency in Python for data processing, automation, integration, and maintainable production code.
• Hands-on experience with Amazon S3, Athena, EC2, and EMR.
• Strong experience with Apache Airflow or another ETL/workflow orchestration tool, with Airflow preferred.
• Experience implementing data quality checks, monitoring, error handling, and operational support processes.
• Ability to translate business requirements into technical designs and deliver solutions with minimal supervision.
• Strong documentation and communication skills, including the ability to explain technical issues clearly to business and technical stakeholders.
• Working experience with Jira and Confluence for task tracking, delivery coordination, and documentation.
• Practical understanding of AI capabilities and limitations, along with hands-on experience creating simple AI agents or AI-assisted automations.
• Experience with version control, code reviews, and collaborative development practices.

Preferred Qualifications

• Hands-on experience with Apache Spark using PySpark or Scala.
• Experience with Snowflake, including data transformation, query optimization, and integration with cloud data pipelines.
• Practical experience using OpenAI Codex for engineering tasks and Snowflake Cortex for AI-enabled data workflows.
• Previous experience mentoring engineers, leading technical delivery, or managing a small team. Formal people management experience is an added advantage.
• Experience supporting marketing technology, audience data, campaign reporting, or