Lead Data Engineer

NorthBay Solutions LLC; Lahore; On-site; PKR 2,000,000 - 3,000,000; Full time

Job summary

NorthBay Solutions LLC is seeking a Lead Data Engineer to modernize a legacy Oracle platform into a scalable data warehouse on Amazon Redshift. The role requires 10+ years of experience in Data Engineering, expertise in AWS Data Engineering, Data Architecture, and strong proficiency in Amazon Redshift. Responsibilities include code conversion, performance tuning, and mentoring junior engineers. The position is hybrid, with opportunities in Lahore, Karachi, or Islamabad.

Qualifications

• 10+ years of experience in Data Engineering, Data Warehousing, and Database Development.
• Strong hands-on experience with Amazon Redshift and Oracle PLSQL.
• Proven experience in enterprise-scale data warehouse architecture.

Responsibilities

• Perform deep code analysis of Oracle PL/SQL objects to ascertain their translation capability.
• Create design artifacts such as object inventory, migration strategy, and Redshift rewrite patterns.
• Architect and enhance analytics platforms built on Amazon Redshift.

Skills

AWS Data Engineering

Data Architecture

Data Warehousing

Procedural SQL Development

Amazon Redshift

Airflow Orchestration

Python

Tools

Redshift

Oracle PL/SQL

Airflow (MWAA)

AWS S3

Amazon Lambda

Job description

Lead Data Engineer – AWS Redshift, Oracle & Airflow

Employment Type

Full-Time | Permanent

Location

Pakistan (Lahore / Karachi / Islamabad) – Hybrid

Role Overview

We are seeking a Lead Data Engineer to drive the end-to-end modernization of a legacy Oracle platform into a scalable data warehouse built on Amazon Redshift. This role involves leading, accessing Oracle procedural workloads, extracting business logic/rules, object dependencies and redesign/convert into Amazon Redshift-native, set-based, parallelized data processing with production-grade orchestration, validation, and operational controls integrated with workflow orchestration using Apache Airflow (MWAA).

Performance enhancement is one of the primary objectives by leveraging Redshift’s parallel processing capabilities. The Lead Data Engineer will be responsible for end-to-end delivery, including solution architecture, code conversion/refactoring to deliver enterprise-grade solution adhering to best practices/standards in performance optimization, scalability, and orchestration.

Experience

10+ years of experience in Data Engineering, Data Warehousing, and Database Development.

Primary Skills

AWS Data Engineering, Data Architecture, Data Warehousing, Procedural SQL Development, Amazon Redshift, Airflow Orchestration, Python

• Lead end-to-end modernization of database programming objects and business logic from Oracle into Amazon Redshift and MWAA-based orchestration
• Strong Amazon Redshift expertise, including schema design, MPP concepts, sort / distribution strategy, datashares, serverless or provisioned deployment models, and query tuning
• Converting procedural database logic into set-based and orchestration-driven workflows

Technologies

Must Have: Redshift, MPP, Oracle PL/SQL, PL/pgSQL, DAG, Airflow (MWAA)

Nice To Have: AWS (S3, Lambda), SQL, ETL/ELT, Data Modeling, Query Optimization, Performance Tuning, Linux/Unix, Git, CI/CD

Key Responsibilities

• Perform deep code analysis of Oracle PL/SQL objects to determine what can be translated, what must be refactored, and what must be re-architected
• Identify patterns that are incompatible or inefficient in Redshift, including nested loops, row-by-row processing, procedural orchestration, excessive temp-table chaining, exception-driven logic, and transaction-dependent flows
• Understand, transform, convert and optimize complex Oracle PL/SQL procedures, functions, and scripts & storage objects to AWS Redshift to deliver the most optimal performance for Redshift MPP architecture
• Create design artifacts such as object inventory, dependency maps, pseudo-code, migration strategy, and Redshift rewrite patterns
• Implement production-ready SQL and Airflow DAGs with clear operational behavior, logging, alerting, and restartability
• Tune execution to meet SLA targets through parallelism, query optimization, workload isolation, and efficient data movement
• Review code quality, enforce engineering standards, and mentor junior or mid-level engineers contributing to the migration
• Architect and enhance analytics platforms built on Amazon Redshift
• Define and enforce best practices for data governance, security, and data quality
• Collaborate with cross-functional stakeholders to design and deliver robust data solutions
• Provide technical leadership by mentoring engineers, conducting code reviews, and guiding best practices

Key Requirements

• Strong hands-on experience with Amazon Redshift, Oracle PLSQL, PL/pgSQL, Amazon MWAA (Apache Airflow), Aurora PostgreSQL, AWS DMS, S3, Lambda, Python, CloudWatch, GitHub
• Proven experience in enterprise-scale data warehouse architecture
• Expertise in performance tuning and large-scale data processing
• Prior experience in technical leadership or mentoring roles
• Deep hands-on experience with Oracle and/or PostgreSQL database programming, especially stored procedures, functions, cursors, transaction logic, and performance troubleshooting