Senior Databricks Data Engineer (Data Migration Project)

ERNI

Manila

Hybrid

PHP 1,800,000 - 3,200,000

Full time

2 days ago

Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Benefits offered by this job

Private HMO and insurance from day one

5-year jubilee trip to European ERNI

13th-month pay

Free training and certifications

Office snacks and coffee

Hybrid work setup

Career development opportunities

Global exposure

Job summary

ERNI invites a Senior Data Engineer to join our growing APAC team in Mandaluyong, Manila, with a hybrid work setup. You will design and optimize data pipelines, enable analytics-ready datasets, and partner with stakeholders to translate requirements into scalable solutions.

You will work on Azure and Databricks-based architectures, implement governance and data quality controls, and help advance our modern data platform for global clients across finance, MedTech, and more.

Qualifications

8+ years of professional data engineering experience with enterprise-scale data pipelines and analytics-ready datasets.
Strong Azure and Azure Databricks expertise with Databricks SQL, PySpark, Delta Lake.
Experience with cloud-native data engineering patterns, ingestion, transformation, orchestration, and monitoring.
Experience building pipelines from multiple source systems processing structured and semi/unstructured data.
Data modeling, data marts, curated layers, and consumer datasets for reporting and analytics.
Ability to clarify requirements with stakeholders and translate them into technical specs.
Experience with data governance, lineage, security, and compliance in cloud environments.
Experience with job orchestration and scheduling tools (Azure Data Factory, Databricks Workflows, Airflow).
Strong troubleshooting, root-cause analysis, and production incident handling.
Excellent communication and ability to work independently in cross-functional teams.

Responsibilities

Design, develop, and optimize automated data pipelines, ETL/ELT workflows, data models, and curated data layers.
Build scalable ingestion, transformation, validation, and publishing processes for enterprise data consumption.
Collaborate with stakeholders to clarify requirements and translate business rules into technical delivery plans.
Develop reusable data engineering patterns and components to improve delivery speed and reliability.
Ensure data quality, lineage, security, and compliance with governance standards.
Monitor pipeline health, troubleshoot incidents, and optimize performance and cost.

Skills

Azure

Azure Databricks

Delta Lake

PySpark

Spark SQL

SQL

ETL/ELT

Data pipeline design

Data modeling

Data quality controls

Production support

Performance tuning

Stakeholder requirements

Tools

Git

CI/CD

Databricks Workflows

Airflow

Job description

Founded in 1994 and headquartered in Switzerland, ERNI is a leading Software Development company with over 800 employees worldwide. Specializing in IT and software engineering, we drive innovation in process and technology. Our first service center in Asia Pacific, located in Metro Manila (Mandaluyong), supports clients across Europe, APAC, the Philippines, and the USA. As we continue to grow, we're looking for passionate and motivated individuals to join our team.

Why ERNI is the Perfect Place for You:

International Exposure: Work with global clients on cutting-edge projects.
Inclusive Culture: Thrive in a collaborative and diverse work environment.
Career Development: Enjoy continuous learning and professional growth opportunities.

Perks And Benefits

Career Stability: Enjoy a stable career path with ample project opportunities.
Immediate Coverage: Private HMO and insurance benefits from day one.
Jubilee Celebration: A 5-year milestone includes a complimentary trip to any European ERNI sites.
Comprehensive Benefits: Government-mandated benefits including 13th-month pay.
Skill Enhancement: Access free training and certifications.
Baby Basket: To welcome your newborn to the ERNI family.
Fruit Basket: Boost of vitamins during hospitalization.
Office Perks: Enjoy free snacks and coffee.

Growth And Opportunities

Free Training: Advance your skills through technical and non-technical training.
Challenging Projects: Engage in complex software projects across MedTech, Industry, Finance, and Transportation.
Supportive Environment: Benefit from a team dedicated to guiding and supporting your success.
Recognition and Advancement: Receive acknowledgment for your efforts and opportunities for promotion.
Open Communication: Experience transparency and value your input in our culture.

Flexibility

Hybrid Work Setup: Balance remote and in-person work for better work-life integration.

Events

Connect and Celebrate: Participate in a variety of events including leisure, summer, family, social, and year-end gatherings.

What Are Our Wishes

Minimum 8 years of professional data engineering experience, including hands-on design and delivery of enterprise-scale data pipelines, data platforms, ETL/ELT processes, and analytics-ready datasets.
Senior-level hands-on experience with Azure and Azure Databricks is required. Strong experience developing data engineering solutions using Databricks, PySpark, Spark SQL, Delta Lake, and SQL.
Strong understanding of cloud-native data engineering patterns, including scalable ingestion, transformation, orchestration, monitoring, performance tuning, and production support.
Experience building and maintaining data pipelines that process large volumes of structured, semi structured, and unstructured data from multiple source systems.
Strong experience with data modeling, data marts, curated data layers, and business-facing datasets for reporting, analytics, and operational consumption.
Strong ability to clarify requirements directly with stakeholders and translate business rules, data definitions, and process needs into technical implementation details.
Experience with source-to-target mapping, data profiling, data validation, reconciliation, and data quality management.
Experience with job orchestration and scheduling tools such as Azure Data Factory, Databricks Workflows, Airflow, or similar technologies.
Strong understanding of data governance, access control, metadata, lineage, security, and compliance considerations in a cloud data environment.
Experience with performance tuning and optimization of Spark jobs, Databricks clusters, SQL queries, and Delta tables.
Strong troubleshooting, problem-solving, and root-cause analysis skills in production data environments.
Strong communication skills, with the ability to explain technical concepts, risks, dependencies, and delivery options to business and technology stakeholders.
Ability to work independently with minimal supervision while collaborating effectively in cross-functional delivery teams.

Preferred / Nice-to-Have Qualifications

Experience supporting near-real-time, frequent refresh, or low-latency data/application refresh patterns.
Experience with Spark Structured Streaming, Auto Loader, Kafka, Event Hubs, change data capture, incremental ingestion, or streaming tables is preferred but not mandatory.
Experience designing lakehouse-style data layers such as raw, cleansed, conformed, curated, and consumption-ready layers.
Experience with CI/CD, DevOps, automated testing, version control, deployment pipelines, and environment management for data engineering solutions.
Experience with Azure services such as Azure Data Lake Storage, Azure Data Factory, Azure Key Vault, Azure Monitor, Azure Purview / Microsoft Purview, Azure Synapse, or related services.
Experience working in regulated, data-sensitive, or enterprise environments with strong governance, audit, and security requirements.
Experience supporting business intelligence, analytics, machine learning, or operational reporting use cases.

Technical Skill Expectations Required:

Azure, Azure Databricks, Delta Lake, PySpark, Spark SQL, SQL, ETL/ELT development, data pipeline design, data modeling, data quality controls, production support, performance tuning, and stakeholder requirements clarification. Strongly preferred: Azure Data Lake Storage, Azure Data Factory, Databricks Workflows, Git, CI/CD, DevOps practices, orchestration frameworks, metadata management, monitoring, and cost/performance optimization.

Nice to have

Structured Streaming, Auto Loader, Kafka, Event Hubs, CDC, incremental processing, near-real-time refresh patterns, and low-latency data pipelines.

How can you contribute to the team?

The IT Senior Data Engineer will design, build, optimize, and support scalable data engineering solutions on a modern cloud-native data platform. The role will focus on developing reliable, secure, and high performing data pipelines, curated data layers, and analytics-ready data assets that support business operations, reporting, advanced analytics, and new strategic initiatives. This role requires a senior-level data engineer who can work independently across technical and business teams, clarify requirements directly with stakeholders, translate business needs into technical delivery plans, and ensure that data products are designed for quality, performance, reusability, and long-term maintainability.

Key Responsibilities

Design, develop, and optimize automated data pipelines, ETL/ELT workflows, data models, and curated data layers using Azure, Azure Databricks, Delta Lake, PySpark, SQL, and related cloud-native data engineering tools.
Build scalable ingestion, transformation, validation, and publishing processes to support enterprise data consumption across reporting, analytics, application, and operational use cases.
Partner directly with business stakeholders, product owners, data consumers, analytics teams, and subject matter experts to clarify business requirements, data definitions, transformation rules, data quality expectations, refresh requirements, and acceptance criteria.
Translate business requirements into technical specifications, source-to-target mappings, data models, pipeline designs, delivery estimates, and implementation plans.
Facilitate requirement-discovery discussions, playback sessions, technical walkthroughs, and solution reviews with both technical and non-technical stakeholders.
Design and implement reliable data marts, analytical datasets, curated data products, and customized data extracts aligned with business priorities and consumption patterns.
Develop reusable data engineering frameworks, patterns, and components that improve delivery speed, operational reliability, monitoring, and maintainability.
Ensure data pipelines and data products are designed with appropriate controls for data quality, lineage, auditability, security, privacy, access management, and compliance.
Implement validation checks, reconciliation processes, exception handling, logging, and monitoring to ensure data accuracy, completeness, and timeliness.
Monitor pipeline health, performance, cost, and reliability; troubleshoot production incidents; perform root cause analysis; and implement preventive and corrective actions.
Optimize Databricks jobs, Spark workloads, SQL queries, Delta tables, partitioning strategies, and orchestration patterns for performance, scalability, and cost efficiency.
Work closely with data architects, platform engineers, product owners, data analysts, and data scientists to ensure that data infrastructure and data products are reliable, discoverable, well-governed, and fit for business use.
Support batch and scheduled data processing requirements, with experience in near-real-time or frequent application/data refresh patterns considered a strong advantage.
Contribute to data architecture decisions involving lakehouse design, data layering, ingestion patterns, transformation standards, orchestration, metadata management, and operational support.