Senior Cloud Data Engineer - Remote

Hire Resolve; Cape Town; Remote; ZAR 60 000 - 100 000; Full time

Job summary

An innovative firm is seeking a Senior Cloud Engineer to join their dynamic team. This exciting role involves architecting and optimizing cloud-agnostic data solutions using cutting-edge technologies like Azure, Databricks, and Spark. You will lead projects, mentor junior engineers, and tackle complex data engineering challenges. If you have a passion for cloud solutions and a knack for collaboration, this position is perfect for you. Join a forward-thinking company where your contributions will drive impactful data solutions and enhance engineering capabilities.

Qualifications

• 6+ years of experience in data engineering and cloud solutions.
• Strong expertise in Azure and cloud-agnostic data tools.

Responsibilities

• Architect and optimize scalable cloud data solutions using Azure.
• Lead projects and mentor junior engineers in data engineering.

Skills

Databricks

Spark

Lakehouse architectures

Delta Lake tables

T-SQL

Python

Azure

DevOps practices

Education

Computer Science Degree

Data Engineering Certification

Tools

Azure Synapse

Azure Data Factory

Azure DevOps

Git

Job description

Hire Resolve's client is looking for a Senior Cloud Engineer (Remote) to join their dynamic and growing Engineering team.

The ideal candidate will have strong expertise in Databricks, Spark, Lakehouse architectures, Delta Lake tables, and other modern data engineering technologies. They should be proficient in T-SQL and Python, with advanced coding abilities, and possess a deep understanding of cloud-based and cloud-agnostic data architectures. This is a senior role that requires the ability to lead projects, mentor junior team members, and work independently on complex data engineering challenges.

Requirements:

1. Data Engineering & Cloud Solutions
2. Architect, build, and optimize scalable and cloud-agnostic data solutions using Azure, Databricks, Spark, Lakehouse, and Delta Lake tables.
3. Develop, implement, and maintain big data pipelines for ingesting, processing, and storing large volumes of structured and unstructured data.
4. Manage and optimize data lake and data warehouse architectures for performance, cost, and scalability.

1. Cloud & DevOps
2. Work within Azure environments (Azure Synapse, Data Factory, ADLS, etc.) to develop and maintain cloud-based data solutions.
3. Implement best DevOps practices for CI/CD pipelines, infrastructure-as-code, and automation.
4. Utilize Azure DevOps and Git for managing code repositories, version control, and continuous integration/deployment.
5. Ensure high levels of security, compliance, and data governance across data engineering processes.

1. Big Data Processing & Development
2. Utilize Spark, Databricks, and distributed computing to process and analyze large datasets efficiently.
3. Write advanced Python and T-SQL scripts for data transformations, ETL/ELT processes, and real-time data processing.
4. Optimize performance for data pipelines and SQL queries for efficiency and cost-effectiveness.
5. Experience with Graph databases is a big advantage.

1. Collaboration & Leadership
2. Work closely with data scientists, analysts, and business stakeholders to understand data requirements and develop solutions that meet business objectives.
3. Lead initiatives to enhance data engineering capabilities, introduce new technologies, and drive best practices.
4. Mentor junior engineers, conduct code reviews, and contribute to building a culture of technical excellence.
5. Communicate effectively with technical and non-technical stakeholders, translating complex data concepts into actionable insights.

Minimum Requirements:

• 6+ years of extensive experience.
• Education / Degrees: Computer science degree or comparable data engineering certification in one of the cloud platforms.

Additional Skills:

• Extensive experience in cloud-based data engineering, with expertise in Azure (Azure Synapse, Azure Data Factory, ADLS, etc.).
• Strong expertise in cloud-agnostic data tools such as Databricks, Spark, Delta Lake, and Lakehouse architectures.
• Advanced proficiency in T-SQL and Python for developing complex data pipelines, transformations, and optimizations.
• Hands-on experience in big data processing, ETL/ELT, and data pipeline orchestration.
• Solid understanding of data modeling, warehouse design, and data lakehouse architecture.
• Experience with Azure DevOps (or similar tools), CI/CD, Git, and infrastructure-as-code.
• Excellent communication skills and ability to work independently on high-impact projects.

Interested applicants should submit their CV to Gaby Turner at [e-mail] or forward directly to the IT Department at [e-mail].