Job Description
PURPOSE OF THIS ROLE
The purpose of the Senior Data Engineer role is to provide strategic leadership in data engineering, overseeing the development and maintenance of data infrastructure, and ensuring the availability, reliability, and performance of data solutions. This position plays a critical role in supporting data-driven decision-making across the organization.
Key responsibilities include:
- Providing strategic direction and technical leadership in data engineering.
- Leading the design, development, and deployment of ETL processes and data pipelines.
- Integrating data from various sources and transforming it for analytics.
- Maintaining data quality and implementing data governance best practices.
- Managing and mentoring a team of data engineers for professional growth.
- Collaborating with stakeholders to understand and fulfil data requirements.
- Documenting data processes, architectures, and best practices.
Challenges in this role may include:
- Ensuring data security and compliance in a rapidly evolving regulatory environment.
- Optimizing data infrastructure for performance and scalability.
- Resolving data-related issues efficiently.
- Managing a dynamic team in a fast-paced data environment.
FOUNDATIONAL/LEADERSHIP COMPETENCIES
- Strong leadership and team management skills.
- Excellent problem-solving and communication skills.
FUNCTIONAL COMPETENCIES
- Proficiency in programming languages such as Java, or Scala, SQL, Python.
- Extensive experience with ETL tools, data integration, and data transformation.
- In-depth knowledge of data storage technologies (relational databases, NoSQL, distributed file systems).
- Expertise in data modelling, database design and data warehousing.
- Familiarity with data governance, security, and compliance standards.
- Experience with big data technologies (e.g., Hadoop, Spark, Hive, Azure, Databricks) is a plus.
- Experience in container orchestration framework like Kubernetes is a plus.
- Experience in Infrastructure as Code and DevOps, MLOps and DataOps is a plus.
Job Requirements
- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
- Relevant certifications in data engineering and Cloud computing are advantageous.