Principal Data Engineer (GCP) - Arbisoft

Taraki; Lahore; On-site; PKR 4,000,000 - 8,000,000; Full time

Job summary

Arbisoft in Lahore is seeking a Principal Data Engineer (GCP) to design, build, and maintain scalable data platforms. You will lead data pipeline design, data modeling in BigQuery, and orchestration with Airflow, while enforcing data governance and observability.

The role focuses on developing robust data solutions for analytics, reporting, and machine learning initiatives, with strong ownership and proactive leadership expectations.

Qualifications

• 5+ years of data engineering experience.
• Expertise in building scalable data pipelines on GCP.
• Experience with data warehousing, orchestration, CI/CD automation.
• Proficient in BigQuery data models and data governance.
• Strong SQL and Python development skills.

Responsibilities

• Design, build, and maintain scalable data pipelines and ingestion frameworks on GCP.
• Architect and implement data models, data marts, and curated datasets in BigQuery.
• Develop and maintain Apache Airflow DAGs for complex workflows.
• Build CI/CD pipelines and promote IaC practices.
• Collaborate with analytics teams to deliver trusted datasets for analytics, reporting, and ML initiatives.
• Provide reliability and on-call support for production data platforms.
• Mentor junior engineers and drive engineering best practices.

Skills

GCP

BigQuery

Apache Airflow

GCS

Dataflow/Beam

Dataform

Python

SQL

Cloud Monitoring

Cloud Logging

GitLab CI/CD

GitLab

Tools

GitLab CI/CD

Job description

Our client Arbisoft is hiring a Principal Data Engineer (GCP) in Lahore.

Experience Required

5+ years

We are seeking a highly skilled Principal Data Engineer to design, build, and maintain scalable, reliable, and secure data platforms on Google Cloud Platform (GCP). The ideal candidate will have deep expertise in modern data engineering practices, cloud-native architectures, data warehousing, orchestration, CI/CD automation, and large-scale data processing. This role will work closely with Analytics Engineers, Data Scientists, Software Engineers, and Business Stakeholders to deliver robust data solutions that power analytics, reporting, and machine learning initiatives. The most important criteria for success in this role is to be proactive rather than reactive and taking ownership.

Job Responsibilities

Data Platform Engineering

• Design, develop, and maintain scalable data pipelines and ingestion frameworks.
• Build and optimize ELT/ETL workflows using cloud-native technologies.
• Design and implement data models, data marts, and curated datasets in BigQuery.
• Ensure data quality, lineage, governance, and observability across the data
• Proficient in writing the queries.
• Optimize data processing performance, cost, and reliability.

Cloud Data Architecture

• Architect data solutions leveraging GCP services including BigQuery, Cloud Storage, Dataflow, and Cloud Composer (Airflow).
• Implement data lake and data warehouse solutions following best practices.
• Design scalable batch and streaming data processing architectures.
• Develop reusable frameworks and standards for data engineering teams.

Workflow Orchestration

• Develop and maintain Apache Airflow DAGs for complex data workflows.
• Implement monitoring, alerting, retry mechanisms, and SLA tracking.
• Troubleshoot and optimize workflow execution and dependencies.

CI/CD & DevOps

• Build and maintain CI/CD pipelines using GitLab CI/CD.
• Promote Infrastructure-as-Code practices.
• Implement automated code quality checks and deployment controls.

Data Transformation & Analytics Engineering

• Develop and maintain Dataform projects for data transformations.
• Implement testing, documentation, and data quality checks.
• Collaborate with analytics teams to create trusted datasets.

Reliability & Operations

• Monitor production systems and resolve data incidents. (Production support)
• Participate in on-call support and operational excellence initiatives.
• Perform root cause analysis and implement preventive measures.
• Improve platform observability through logging, monitoring, and alerting.

Technical Leadership

• Mentor junior and mid-level data engineers.
• Conduct code reviews and establish engineering best practices.
• Drive architectural discussions and technical decision-making

Technical Skills Required

• Cloud: GCP
• Data Warehouse: BigQuery
• Orchestration: Apache Airflow (Cloud Composer)
• Storage: GCS
• Processing: Dataflow / Apache Beam
• Transformation: Dataform
• CI/CD: GitLab CI/CD
• Languages: Python, SQL
• Monitoring: Cloud Monitoring, Cloud Logging
• Source Control: GitLab

Good To Have

• Event-driven architectures.
• Infrastructure as Code: Terraform.
• CDC (Change Data Capture) pipelines.
• Streaming platforms such as Kafka.