Senior Data Engineer - AI Infrastructure

Placements24; Queenstown; On-site; NZD 140,000 - 190,000; Full time

Benefits offered by this job

Competitive salary

Health, dental, and vision insurance

Paid time off and holidays

Professional development and training

Collaborative and innovative work env.

Retirement savings plan

Job summary

Placements24 is seeking a Senior Data Engineer in Queenstown to design and maintain scalable data pipelines, data warehouses, and data lakes powering AI initiatives. You will work closely with data scientists and AI engineers to ensure reliable data delivery for model training and deployment.

The role emphasizes robust ETL/ELT processes, data governance, security, and cost-efficient infrastructure, with opportunities for professional growth within a collaborative team.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
• 5+ years of professional experience in data engineering.
• Proven experience building and managing data pipelines using Apache Spark, Kafka, Airflow.
• Strong proficiency in SQL and NoSQL databases.
• Hands-on experience with cloud platforms such as AWS, Azure, or GCP, including their data services.
• Experience with data warehousing concepts and technologies (e.g., Redshift, Snowflake, BigQuery).
• Familiarity with big data technologies and distributed systems.
• Understanding of AI/ML concepts and data requirements is highly desirable.
• Excellent programming skills in Python or Scala.
• Strong analytical and problem-solving skills.

Responsibilities

• Design, build, and maintain scalable data pipelines for ingesting, transforming, and processing large volumes of data.
• Develop and manage data warehouses, data lakes, and other data storage solutions.
• Implement ETL/ELT processes to ensure data quality, consistency, and availability.
• Optimize data infrastructure for performance, cost-efficiency, and scalability to support AI/ML workloads.
• Collaborate with Data Scientists and AI Engineers to understand their data requirements and provide necessary data solutions.
• Implement data governance, security, and compliance best practices.
• Automate data infrastructure operations and monitoring.
• Troubleshoot and resolve data-related issues promptly.
• Stay current with emerging technologies in data engineering and cloud platforms.

Skills

Apache Spark

Kafka

Airflow

SQL

NoSQL

Python

Scala

AWS

Azure

GCP

Data warehousing

Data governance

ETL/ELT

Education

Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field

Tools

Redshift

Snowflake

BigQuery

Spark

Kafka

Airflow

Python

Job description

About the Role

Our client, a leader in data-driven innovation, is seeking a highly experienced Senior Data Engineer to build and maintain the robust infrastructure powering their AI initiatives in Queenstown, Eastern Cape . This role is critical for ensuring the seamless flow of data required for machine learning model training and deployment. You will be responsible for designing, implementing, and optimizing scalable data pipelines, data warehouses, and data lakes. The ideal candidate will possess a strong background in data engineering principles, cloud platforms, and experience working closely with data scientists and AI engineers. This is an excellent opportunity to play a key role in scaling our AI capabilities and working with a talented team dedicated to leveraging data for business success. Join us and contribute to building the foundation for advanced AI solutions.

Key Responsibilities

• Design, build, and maintain scalable and reliable data pipelines for ingesting, transforming, and processing large volumes of data.
• Develop and manage data warehouses, data lakes, and other data storage solutions.
• Implement ETL/ELT processes to ensure data quality, consistency, and availability.
• Optimize data infrastructure for performance, cost-efficiency, and scalability to support AI/ML workloads.
• Collaborate with Data Scientists and AI Engineers to understand their data requirements and provide necessary data solutions.
• Implement data governance, security, and compliance best practices.
• Automate data infrastructure operations and monitoring.
• Troubleshoot and resolve data-related issues promptly.
• Stay current with emerging technologies in data engineering and cloud platforms.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
• 5+ years of professional experience in data engineering.
• Proven experience building and managing data pipelines using tools like Apache Spark, Kafka, Airflow.
• Strong proficiency in SQL and NoSQL databases.
• Hands-on experience with cloud platforms such as AWS, Azure, or GCP, including their data services.
• Experience with data warehousing concepts and technologies (e.g., Redshift, Snowflake, BigQuery).
• Familiarity with big data technologies and distributed systems.
• Understanding of AI/ML concepts and data requirements is highly desirable.
• Excellent programming skills in Python or Scala.
• Strong analytical and problem-solving skills.

Benefits

• Competitive salary and performance bonuses.
• Comprehensive health, dental, and vision insurance.
• Generous paid time off and holidays.
• Opportunities for professional development and advanced training.
• A collaborative and innovative work environment in Queenstown, Eastern Cape .
• Retirement savings plan with company match.
• Support for continuous learning and skill enhancement.