Data Engineer

Job Purpose

We are seeking a versatile and experienced Data Engineer to join our team. The ideal candidate will have a strong background in data engineering and machine learning, with the ability to design, build, and maintain data pipelines, as well as develop and deploy machine learning models. This role requires a deep understanding of data lakehouses, such as Databricks and Azure Fabric, and the ability to work collaboratively with cross-functional teams to deliver data-driven solutions.

Key Activities, Responsibility & Accountability

Key Responsibilities:

1. Data Engineering:
1. Design, build, and maintain scalable data pipelines and ETL processes.
2. Work with large datasets and data lakehouses, including Databricks and Azure Fabric.
3. Ensure data quality, integrity, and security across all data pipelines.
4. Optimize data storage and retrieval processes for performance and efficiency.
2. Machine Learning Engineering:
1. Develop, implement, and optimize machine learning models and algorithms.
2. Perform data analysis and model tuning to improve accuracy and performance.
3. Deploy machine learning models into production environments.
4. Monitor and maintain the performance of deployed models.
3. Collaboration:
1. Collaborate with cross-functional teams to understand business requirements and deliver data-driven solutions.
2. Work closely with data scientists, analysts, and other stakeholders to ensure seamless integration of data and models.
4. Continuous Learning:
1. Stay up-to-date with the latest advancements in data engineering and machine learning.
2. Participate in training and development opportunities to enhance skills and knowledge.

Qualifications

Bachelor's or Master's degree in Data Science, Computer Science, Engineering, or a related field.

Experience

1. 5-10 years of experience in data engineering and machine learning.
2. Proficiency in programming languages such as Python, R, or SQL.
3. Experience with data lakehouses, particularly Databricks and Azure Fabric.
4. Strong analytical and problem-solving skills.
5. Excellent communication and teamwork abilities.
6. Ability to work independently and manage multiple projects simultaneously.