Senior Big Data Platform Engineer - Spark, Python, Airflow

Virtej Technologies; Karachi Division; On-site; PKR 2,500,000 - 4,500,000; Full time

Job summary

Virtej Technologies seeks a highly motivated Big Data Platform Engineer to design, develop, and maintain scalable data platforms and pipelines. The role emphasizes Python, Spark (PySpark), SQL, and Linux-based environments with DevOps practices.

Responsibilities include building robust data pipelines, ETL/ELT processing, and optimizing SQL and data models. You will work with cross-functional teams to ensure platform reliability and performance across enterprise-scale analytics.

Qualifications

• 6 to 12 years (or 8 to 12+ years) in Data Engineering or Big Data Platform Development.
• Experience with enterprise-scale data platforms.
• Experience in distributed computing environments.
• Strong Python/Spark/SQL proficiency.
• Linux and DevOps practices exposure.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Python and Apache Spark.
• Implement efficient ETL/ELT processes for large-scale datasets.
• Develop and optimize complex SQL queries, data models, and transformations.
• Ensure data quality, integrity, and reliability across the platform.

Skills

Python

Apache Spark (PySpark)

SQL

Apache Airflow

Kubernetes AKS/SKE

MinIO / S3 storage

ETL/ELT Processing

Performance Tuning

Git Version Control

Agile/Scrum

Shell Scripting (Bash/KSH)

CI/CD concepts

Cloud platforms (Azure/AWS)

Tools

Docker

AWS

Azure

GKE/EKS

Linux

Job description

Virtej Technologies seeks a highly motivated Big Data Platform Engineer to design, develop, and maintain scalable data platforms and pipelines. The role emphasizes Python, Spark (PySpark), SQL, and Linux-based environments with DevOps practices.

Responsibilities include building robust data pipelines, ETL/ELT processing, and optimizing SQL and data models. You will work with cross-functional teams to ensure platform reliability and performance across enterprise-scale analytics.