Lead, Big Data

Atomic Recruitment SEA; Selangor; On-site; MYR 120,000 - 180,000; Full time

Job summary

Atomic Recruitment SEA seeks an experienced Assistant Manager – Big Data & Data Solutions to oversee large-scale data environments and deliver analytics solutions, with a focus on MarTech and customer analytics. You will design scalable data pipelines, deploy predictive models, ensure data quality, and mentor junior engineers in an Agile setting.

The role emphasizes bridging business strategy with technical execution, leading cross-functional collaboration, and advancing data modernization

Qualifications

• Big Data Technologies: Spark (PySpark), Hadoop, Kafka.
• Programming & Infrastructure: Python, SQL, Terraform or Docker.
• Cloud Platforms: Huawei Cloud (DLI), Databricks, Amazon EMR.
• Data Warehousing & Orchestration: Data warehouse architectures; Airflow, Snowflake, BigQuery, Redshift, Huawei DWS.
• Data Modeling & ML: Supervised/unsupervised modeling concepts.
• Databases: MSSQL, PostgreSQL, NoSQL.
• BI & MarTech: Power BI, Tableau, QlikSense; CDP, MarTech tools in marketing.

Responsibilities

• Design, build, and maintain enterprise big data systems.
• Develop, optimize, and maintain data processing using Hadoop, Spark, Kafka.
• Apply predictive modeling, clustering, regression, and NLP for data products.
• Collaborate with Marketing, Engineering, Product and Strategy teams.
• Lead, mentor and manage junior engineers and analysts.
• Ensure governance, security and regulatory compliance.
• Stay updated on trends in big data, AI/ML, and MarTech.

Job description

Join the centralized technology powerhouse for a leading Malaysian conglomerate. Established in early 2000s, this centre of excellence focuses on driving digital transformation, process optimization, and large-scale IT innovation across multiple business units

About the Role

We are seeking an experienced and technical Assistant Manager – Big Data & Data Solutions to manage large-scale data environments, lead technical executions, and deliver high-impact analytics solutions—particularly supporting marketing technology (MarTech) and customer analytics. In this role, you will bridge the gap between business strategy and big data engineering by designing robust data pipelines, deploying predictive models, and ensuring data quality. You will also manage and mentor junior team members to ensure the successful delivery of key data initiatives in an Agile environment.

Responsibilities

• Architecture & System Design: Design, build, and maintain enterprise big data systems aligned with stakeholder requirements and industry best practices.
• Data Engineering & Pipelines: Develop, optimize, and maintain data processing solutions using Hadoop, Spark, and Kafka to guarantee data accuracy, consistency, and integrity.
• Advanced Analytics & Machine Learning: Apply predictive modeling, clustering, regression, and NLP techniques to build customer-centric data products and uncover actionable business insights.
• Cross-Functional Collaboration: Partner with Marketing, Engineering, Product, and Strategy teams to translate complex business requirements into clear technical specifications and data architectures.
• Team Leadership & Delivery: Lead, mentor, and manage junior engineers and analysts to deliver technical tasks and projects effectively.
• Governance & Security: Ensure full compliance with data privacy, security regulations and governance standards throughout the development lifecycle.
• Technology Modernization: Stay up to date with emerging trends in big data, AI/ML, and MarTech to introduce innovative methodologies to the organization.

Qualifications

• Big Data Technologies: Hands‑on experience with big data ecosystems, specifically Spark (PySpark is a strong plus), Hadoop, or Kafka.
• Programming & Infrastructure: Strong proficiency in Python and SQL, alongside working knowledge of Terraform or Docker.
• Cloud Platforms: Experience with big data analytics platforms such as Huawei Cloud (DLI), Databricks, or Amazon EMR.
• Data Warehousing & Orchestration: Familiarity with data warehousing architectures and orchestrators like Airflow, Snowflake, BigQuery, Redshift, or Huawei DWS.
• Data Modeling & ML: Solid grasp of data modeling concepts (both supervised and unsupervised training/learning).
• Databases: Proven experience working with relational and non‑relational databases, including MSSQL (preferred), PostgreSQL, and NoSQL solutions.
• Business Intelligence & MarTech: Experience with visualization tools like Power BI (preferred), Tableau, or QlikSense. Prior experience with CDP (Customer Data Platforms), MarTech tools, or the marketing industry is highly advantageous.
• Leadership & Agile: Proven ability to manage and upskill junior team members with experience working in Agile/Scrum development environments.