Data Engineer

JobCubby; Hinoba-an; On-site; PHP 1,200,000 - 2,400,000; Full time

Job summary

JobCubby is seeking a Senior Product Software Data Engineer to design, develop, and troubleshoot data-driven software. You will collaborate with cross-functional teams to translate business needs into scalable data solutions and enterprise-grade pipelines.

You will build real-time and batch data pipelines with PySpark, Spark, Azure Databricks, and related Azure services; mentor junior engineers; ensure data quality, governance, and documentation; partner with data scientists to operationalize ML

Qualifications

• Experience building data pipelines for real-time and batch processing.
• 2+ years building data pipelines for structured and unstructured data.
• 3+ years total experience in data engineering.
• Strong knowledge of PySpark, Hadoop, Kafka, and related big data tools.
• Experience with SQL Server data warehousing concepts.
• Experience with Azure Data Factory, Azure Data Lake, Azure SQL DB, and Azure Synapse.
• Good to have Azure ML, Python, and Scala.
• Strong communication and collaboration skills.

Responsibilities

• Collaborate with cross-functional teams to translate requirements into scalable data solutions.
• Design and develop optimized, reliable, and secure enterprise-level data pipelines for real-time and batch processing.
• Analyze existing business processes, identify gaps, and recommend data-driven improvements.
• Build modern real-time data solutions using Scala, Apache Spark, and Azure Databricks.
• Partner with data scientists and business stakeholders to operationalize machine learning models.
• Champion data governance, metadata management, and operational excellence across data platforms.
• Ensure data quality and integrity by developing validation frameworks and monitoring mechanisms.
• Adhere to coding standards, participate in peer code reviews, and contribute to technical documentation.
• Mentor junior engineers and contribute to knowledge sharing across the team.

Skills

Software Engineering

Software Development

Data Pipelines

PySpark

Azure

SQL / Relational DB

Scala

Machine Learning concepts

Communication

Mentoring

Education

Bachelor’s degree in Computer Science or related field

Tools

Azure Data Factory

Azure Databricks

Azure Data Lake

Azure Synapse Analytics

Power BI

Git

Job description

About the Role:

Take your software engineering career to the next level as a Senior Product Software Data Engineer. You will play a key role in designing, developing, and troubleshooting software programs. Your technical skills and innovative mindset will contribute to the continuous improvement of our products, making a significant impact on user experience and satisfaction.

Responsibilities:

• Collaborate with cross-functional teams to review use cases and translate business requirements into scalable data solutions.
• Design and develop highly optimized, reliable, and secure enterprise-level data pipelines for real-time and batch processing.
• Analyze existing business processes, identify gaps, and recommend data-driven improvements.
• Build modern real-time data solutions using Scala, Apache Spark, and Azure Databricks.
• Partner with data scientists and business stakeholders to operationalize machine learning models.
• Champion data governance, metadata management, and operational excellence across data platforms.
• Ensure data quality and integrity by developing validation frameworks and monitoring mechanisms.
• Adhere to coding standards, participate in peer code reviews, and contribute to technical documentation.
• Mentor junior engineers and contribute to knowledge sharing across the team.

Skills:

• Software Engineering: The ability to design, develop, and maintain software systems and applications by applying principles and techniques of computer science, engineering, and mathematical analysis. This includes the capacity to understand user requirements, create and test the software, and resolve any software-related issues.
• Software Development: The ability to design, write, test, and implement software programs, applications, and systems. This includes understanding various programming languages, software architecture, and software testing methods. It also involves problem-solving capabilities to fix software issues and improve functionality.
• Problem Solving: The ability to understand a complex situation or issue and devise a solution by defining the problem, identifying potential strategies, and ultimately choosing and implementing the most effective course of action.
• Analysis: The ability to examine complex situations or problems, break them down into smaller parts, and understand how these parts work together.
• Testing: The skill of evaluating a system or process, often in software or product development. It involves the ability to identify problems, measure effectiveness, and ensure quality or functionality.
• Agile: The ability to swiftly and effectively respond to changes, with an emphasis on continuous improvement and flexibility. In the context of project management, it denotes a methodology that promotes adaptive planning and encourages rapid and flexible responses to changes.
• Source Code Repository: The ability to effectively use a source code repository, a file archive and web hosting facility where a large amount of source code, for software or for web pages, is kept, either publicly or privately. This skill involves the ability to manage and track changes to code, identify and fix issues, merge code from different branches, and collaborate with other developers.
• Relational Database: The ability to design, implement, and manipulate a relational database, a type of database that stores and organizes data in a structured way and where data is logically interrelated. This skill often requires proficiency in SQL, database management systems, and understanding of database design principles.
• Documentation: The ability to create, manage, organize, and maintain important information and documents in various physical and digital formats. This skill may include preparing reports, managing files, storing data, and keeping records organized and updated for easy retrieval and understanding.
• 3+ years of overall experience in data engineering or related fields.
• 2+ years of experience building data pipelines for structured and unstructured data.
• 1+ years of experience with big data technologies including PySpark, Hadoop, Kafka, EventHub, and Stream Analytics.
• 1+ years of experience in relational and dimensional data modeling using SQL Server and data warehousing concepts.
• Must have 2+ years of experience with T-SQL, stored procedures, and performance tuning.
• 2+ years of experience with Azure Data Factory, Azure Data Lake, Azure SQL DB, and Azure Synapse Analytics.
• Good to have 2+ years of experience with Azure ML, Python, and Scala.
• Good to have 2+ years of experience with Power BI and Azure Analysis Services.
• Experience working with business stakeholders for requirements gathering and use case analysis.
• Strong communication, collaboration, and creative problem-solving skills.

Preferred Qualifications

• Bachelor’s degree in computer science, Engineering, or equivalent practical experience.
• Experience with Agile/Scrum methodologies and DevOps practices.
• Familiarity with CI/CD pipelines and version control systems (e.g., Git).
• Azure Data Engineer Certification or equivalent is a strong advantage.

Our Interview Practices

To maintain a fair and genuine hiring process, we kindly ask that all candidates participate in interviews without the assistance of AI tools or external prompts. Our interview process is designed to assess your individual skills, experiences, and communication style. We value authenticity and want to ensure we’re getting to know you—not a digital assistant. To help maintain this integrity, we ask to remove virtual backgrounds and include in-person interviews in our hiring process. Please note that use of AI-generated responses or third-party support during interviews will be grounds for disqualification from the recruitment process. Applicants may be required to appear onsite at a Wolters Kluwer office as part of the recruitment process.