About the Role
Our client, a prominent data analytics firm, is seeking an experienced Lead Data Engineer to manage and enhance their data warehousing infrastructure. This is an on-site role based in our Soweto, Gauteng office, crucial for fostering close collaboration with data scientists, analysts, and business stakeholders. You will be responsible for designing, building, and maintaining robust, scalable data pipelines and data warehouses that support critical business intelligence and reporting functions. We are dedicated to creating a high-performance data environment and promoting a culture of data-driven decision-making. Join us to shape the foundation of our data ecosystem and drive impactful insights.
Key Responsibilities
- Lead the design, development, implementation, and maintenance of enterprise data warehouses and data marts.
- Develop and optimize ETL/ELT processes to ingest, transform, and load data from various sources.
- Ensure data quality, integrity, and consistency across all data platforms.
- Collaborate with data scientists and business analysts to understand data requirements and provide solutions.
- Design and implement data models that support efficient querying and reporting.
- Monitor data warehouse performance and implement optimizations.
- Manage and maintain database systems and data integration tools.
- Mentor junior data engineers and provide technical guidance.
- Stay current with data warehousing technologies, best practices, and industry trends.
- Develop and enforce data governance policies and procedures.
Requirements
- Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or a related quantitative field.
- Minimum of 6 years of experience in data engineering, with a strong focus on data warehousing.
- Proven experience designing and implementing relational and dimensional data models.
- Expertise in SQL and experience with various database technologies (e.g., PostgreSQL, SQL Server, Oracle).
- Proficiency with ETL tools (e.g., Informatica, Talend, SSIS) or scripting languages for data manipulation (e.g., Python).
- Experience with big data technologies (e.g., Spark, Hadoop) is a plus.
- Solid understanding of data warehousing concepts, principles, and best practices.
- Strong analytical, problem-solving, and communication skills.
- Experience leading small teams or mentoring junior engineers is preferred.
Benefits
- Competitive salary and performance-based bonuses.
- Comprehensive health insurance and retirement benefits.
- Opportunities for professional development and training.
- Stable and secure work environment in our Soweto office.
- Be part of a team building the backbone of data analytics.
This role is ideal for someone who sits between Data Engineering and Analytics; someone with a solid understanding of data engineering principles, but who is equally comfortable exploring datasets, investigating data quality, performing analysis and translating business questions into meaningful data outputs.
What You'll Be Doing
- Perform exploratory data analysis (EDA) and validate datasets.
- Use Python extensively for data analysis, investigation and problem-solving.
- Work with real-world and imperfect datasets to identify patterns, issues and insights.
- Support analytics, reporting and insight-driven initiatives.
- Translate client and business questions into clear data outputs and findings.
- Apply data engineering best practices to ensure data is reliable and fit for analytical use.
- Investigate data and communicate findings clearly to both technical and business stakeholders.
- Engage with stakeholders to understand analytical requirements and provide data-driven solutions.
What We're Looking For
- 2+ years' experience in a Data Engineering, Analytics Engineering or similar data-focused role.
- A solid foundation in data engineering concepts.
- Strong hands-on experience working with data for analysis and insight generation.
- Strong Python skills, particularly pandas and NumPy.
- Experience performing data exploration, investigation and validation.
- Comfortable working with complex, imperfect or inconsistent datasets.
- Strong problem-solving abilities and an analytical mindset.
- The ability to interpret data within a business and client context.
- Strong communication skills, with the ability to explain data findings clearly.
We are currently building our IT talent database and are looking to connect with experienced Data Engineers for current and future opportunities. We are particularly interested in professionals with experience in data ingestion, ETL/ELT, SQL, data modelling, orchestration and cloud data platforms. Register your profile with us so your data engineering experience can be considered for suitable opportunities as they become available. If you enjoy building scalable data solutions and improving data quality, we'd love to hear from you.
- Develop and maintain reliable data pipelines for batch and/or streaming data ingestion.
- Build ETL/ELT processes and transform data for analytics, reporting and downstream applications.
- Design and optimise data models, database queries and data integration workflows.
- Monitor pipeline quality and performance, troubleshoot issues and document data processes.
Education
- A degree, diploma or equivalent relevant qualification in Computer Science, Information Systems, Data Engineering or a related field is advantageous.
- Relevant cloud, database or data engineering certifications will be advantageous.
- Training in analytics platforms or data architecture will be beneficial.
- Demonstrable hands-on data engineering experience is valued.
Experience
- Typically 4+ years of experience in data engineering, ETL development or a closely related role.
- Strong SQL skills and experience developing data pipelines or ETL/ELT processes.
- Experience with relational databases, data modelling and data quality controls.
- Cloud data platforms, Python, orchestration tools or streaming technologies will be highly advantageous.
Skills
- SQL, data modelling and database performance awareness.
- ETL/ELT, data ingestion, transformation and pipeline orchestration.
- Python or another scripting language.
- Cloud data services, data quality, monitoring and documentation.
Location: Gauteng