Data Engineer (Hybrid, BGC)

Seven Seven Global Services, Inc.; Taguig; Hybrid; PHP 600,000 - 900,000; Full time

Job summary

Seven Seven Global Services, Inc. is seeking a Data Engineer to design and implement data warehousing solutions in a fast-paced environment in Taguig. You will design data warehouses, build data pipelines, and ensure data quality and security across platforms.

You will work within an Agile Tribe, gather data requirements, optimize pipelines, and provide incident support for production systems, delivering clear data documentation and reliable data deliveries.

Qualifications

• Graduate of IT related courses
• 4-6 years IT project or data delivery experience
• Proficient in ETL for data warehousing
• Proficient in SQL, Python, R, Java, Scala
• Proficient in Agile delivery methodologies (Scrum/Kanban)
• Proficient in data modeling and data integrations
• Experience with Big Data, Hadoop, EMR, Hive
• Experience with Alibaba Cloud Dataworks, Maxcompute, OSS, Hologres
• Experience with Kafka, Confluent, real-time data streaming
• Proficient with AWS Cloud services (S3, Glue, Athena, Data Sync, Lake Formation, Airflow, IAM, etc.)
• Proficient with Terraform, GitLab

Responsibilities

• Data warehousing design following client standards
• Design and develop data pipelines ingesting data into warehouse
• Support and optimize pipelines to meet SLAs
• Ensure adherence to data engineering standards
• Manage data storage, compute, and security
• Attend agile ceremonies and gather data requirements
• Maintain data quality with validation rules
• Provide L2 incident support and data investigations

Skills

SQL

Python

Data modeling

Data integration

ETL

Data warehousing

Agile

Big Data

Hadoop

Kafka

Cloud computing

Terraform

GitLab

Education

BS Computer Science

Information Technology

Tools

AWS Cloud

Alibaba Cloud Dataworks

Maxcompute

OSS

Hologres

Kafka

Confluent

Airflow

Terraform

GitLab

Job description

Overall Duties and Responsibilities

Does (The tasks / responsibilities that the role performs to address requirements in Key Result Areas).

1. Data Warehousing Design

• Design data warehouse and operational data marts following the Client standards.

• Prepare data model, data dictionary, and mapping documents

2. Data Pipeline design, development, testing, and support

• Design and develop data pipelines that will ingest the data from the respective source systems into the data warehouse

• Support and optimize existing data pipelines to ensure that the downstream SLAs are met

• Ensures the data engineering standards and practices are followed like code version history, readability, functionality, extendibility, test runs, deployments, and post deployment support

• Manage data storage, computing unit requirements, and data security within the data warehouse

3. Comply with Client's agile ways of working

• Responsible in attending agile ceremonies as part of the Agile Tribe set-up and delivering the different demands of the Tribe related to data ingestions to the data warehouse platform

• Gather data requirements coming from Tribe demands

4. Maintain high data quality in the data warehouse

• Ensures data quality within the data warehouse tables by implementing data validation rules and data quality checks within the data pipeline

5. Incident Management Support

• Supports L2 data operations team during production incidents by troubleshooting data pipelines and understanding data behaviors

• Supports stakeholders of the data warehouse platform by solving data related issues and conducting data investigations

• Displays (The knowledge, skills and behaviors indicating how tasks / responsibilities will be performed).

• Good understanding of data warehousing concepts and data modeling

• Proficient in SQL and optimizing queries

• Good communication and problem solving skills

• Delivers (The specific outputs / tangible results produced by the role; resources responsible for)

• Data Delivery Documentations – Data Dictionary, Data Mapping Documents, Data Models

• Optimized data pipelines

• Root cause analysis for production related issues and incidents

Qualifications (Minimum requirements in terms of experience and background, skills and competencies for sourcing purposes)

• Graduate of any IT related courses like BS Computer Science, Information Technology or the likes

• 4-6 years of experience in IT project delivery or data development delivery particularly in Data Warehousing or Business Intelligence

• Proficient in ETL development related to Data Warehousing, Business Intelligence, or other Data Delivery projects.

• Proficient in optimizing data pipelines in order to meet SLA requirements and save cost

• Proficient in SQL, Python, R, Java, Scala

• Proficient in Agile delivery methodologies like Scrum or Kanban

• Proficient in data modeling, data integrations, data quality, and data architecture is a plus

• Competent in Big Data, Hadoop, EMR, Hive, and cloud environments is a plus

• Competent in using Alibaba Cloud products particularly Dataworks, Maxcompute, OSS, Hologres Competent in Kafka, Confluent, Real-time processing, Data Streaming is a plus

• Proficient in using AWS Cloud products particularly S3, Glue, Athena, Data Sync, Lake Formation, Airflow, Secrets Manager, IAM, KMS, Cloudwatch

• Competent in using Terraform, Gitlab