Big Data Architect

Systems Limited; Punjab; On-site; PKR 33,296,000 - 49,945,000; Full time

Job summary

Systems Limited seeks an experienced Big Data expert to lead the design, development, and maintenance of the enterprise Big Data platform. You will collaborate with data engineers and BI analysts across multiple layers, delivering high-quality products with a focus on robust design and collaboration.

You will guide a team of Data Engineers and BI Developers, own streaming pipelines using Spark Streaming and Kafka, and ensure production-readiness, security, and compliance while mentoring growth

Qualifications

• 14 years of experience in enterprise data architecture, integration, and analytics.

Responsibilities

• Facilitate and lead roadmap and enhancements for the enterprise Big Data platform.
• Lead Data Engineers, BI Analysts, and BI engineers to architect and deliver solutions for business information needs.
• Ensure design, code, and procedures are production-ready with security and compliance standards.
• Participate in daily project and production delivery status meetings and provide technical support.
• Mentor and train future big data developers and architects.
• Design and implement scalable, near real-time data ingestion pipelines for low-latency processing.
• Architect and optimize data streaming solutions for high-throughput, distributed systems with data consistency and fault tolerance.

Skills

Leadership

Team management

Collaborative problem solving

Mentoring

Education

Bachelor's degree in CS or related field

Master's degree preferred

Tools

Apache Spark Streaming

Apache Kafka

Azure

ETL tools

NoSQL/SQL data modeling

Job description

We seek an experienced Big Data expert to lead our Big Data platform's design, development, and maintenance. Collaborating with data engineers and Big Data Business Analysts across various platform layers, you'll demonstrate a commitment to collaborative problem-solving, sophisticated design, and delivering high-quality products.

Responsibilities:

• Facilitate and lead the roadmap and enhancements for the enterprise Big Data platform.
• Lead Data Engineers, Business Analysts, and BI engineers to architect and deliver the solutions that fulfill the Business information needs.
• Ensure the design, code, and procedural aspects of the solution are production-ready, in terms of operational, security, and compliance standards.
• Participate in day-to-day project and production delivery status meetings, and provide technical support for faster resolution of issues.
• Yearning for continuous learning and training/mentoring future big data developers and architects.
• Design and implement scalable, near real-time data ingestion pipelinesto enable low-latency data processing and analytics.
• Architect and optimize data streaming solutions to handle high-throughput, distributed systems while ensuring data consistency, reliability, and fault tolerance in near real-time workflows.

Requirements:

• 14 years of experience in enterprise data architecture, integration, and analytics, with a proven track record of design and implementation of big data and analytics solutions in recent years.
• Must have experience with architecting streaming pipelines.
• Experience in leading a team of Data Engineers and BI Developers.
• Bachelor's degree in computer science or relevant field from an accredited college or university; Master's degree from an accredited college or university preferred.
• Strong technical experience in the following:
• Big data stack and solutions, including cloud technologies.
• Must have: Apache Spark Streaming, Apache Kafka
• Data lake architecture, including data model design and implementation (incl. relational, dimensional, and No-SQL).
• Enterprise Application and Data Integration design and implementation (e.g. ETL, messaging, replication, and APIs).
• Azure Solutions Architect Certification
• Awareness of Data management best practices (e.g. data quality, metadata management, data lineage).
• Awareness of Machine Learning techniques is nice to have
• Willingness to travel to UK (25% travelling might be required)