Showing 62 Hybrid Bi Data Lakes Etl Developer jobs in Johannesburg
About the Role
Our client, a prominent data analytics firm, is seeking an experienced Lead Data Engineer to manage and enhance their data warehousing infrastructure. This is an on-site role based in our Soweto, Gauteng office, crucial for fostering close collaboration with data scientists, analysts, and business stakeholders. You will be responsible for designing, building, and maintaining robust, scalable data pipelines and data warehouses that support critical business intelligence and reporting functions. We are dedicated to creating a high-performance data environment and promoting a culture of data-driven decision-making. Join us to shape the foundation of our data ecosystem and drive impactful insights.
Key Responsibilities
- Lead the design, development, implementation, and maintenance of enterprise data warehouses and data marts.
- Develop and optimize ETL/ELT processes to ingest, transform, and load data from various sources.
- Ensure data quality, integrity, and consistency across all data platforms.
- Collaborate with data scientists and business analysts to understand data requirements and provide solutions.
- Design and implement data models that support efficient querying and reporting.
- Monitor data warehouse performance and implement optimizations.
- Manage and maintain database systems and data integration tools.
- Mentor junior data engineers and provide technical guidance.
- Stay current with data warehousing technologies, best practices, and industry trends.
- Develop and enforce data governance policies and procedures.
Requirements
- Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or a related quantitative field.
- Minimum of 6 years of experience in data engineering, with a strong focus on data warehousing.
- Proven experience designing and implementing relational and dimensional data models.
- Expertise in SQL and experience with various database technologies (e.g., PostgreSQL, SQL Server, Oracle).
- Proficiency with ETL tools (e.g., Informatica, Talend, SSIS) or scripting languages for data manipulation (e.g., Python).
- Experience with big data technologies (e.g., Spark, Hadoop) is a plus.
- Solid understanding of data warehousing concepts, principles, and best practices.
- Strong analytical, problem-solving, and communication skills.
- Experience leading small teams or mentoring junior engineers is preferred.
Benefits
- Competitive salary and performance-based bonuses.
- Comprehensive health insurance and retirement benefits.
- Opportunities for professional development and training.
- Stable and secure work environment in our Soweto office.
- Be part of a team building the backbone of data analytics.
What You'll Be Doing
- Perform exploratory data analysis (EDA) and validate datasets.
- Use Python extensively for data analysis, investigation, and problem solving.
- Work with real-world and imperfect datasets to identify patterns, issues, and insights.
- Support analytics, reporting and insight-driven initiatives.
- Translate client and business questions into clear data outputs and findings.
- Apply data engineering best practices to ensure data is reliable and fit for analytical use.
- Investigate data and communicate findings clearly to both technical and business stakeholders.
- Engage with stakeholders to understand analytical requirements and provide data-driven solutions.
What We're Looking For
- 2+ years of experience in a Data Engineering, Analytics Engineering or similar data-focused role.
- A solid foundation in data engineering concepts.
- Strong hands-on experience working with data for analysis and insight generation.
- Strong Python skills, particularly pandas and NumPy.
- Experience performing data exploration, investigation and validation.
- Comfortable working with complex, imperfect or inconsistent datasets.
- Strong problem-solving abilities and an analytical mindset.
- The ability to interpret data within a business and client context.
- Strong communication skills, with the ability to explain data findings clearly.
Location
Soweto, Gauteng
Key Responsibilities
- Develop and maintain reliable data pipelines for batch and/or streaming data ingestion.
- Build ETL/ELT processes and transform data for analytics, reporting and downstream applications.
- Design and optimise data models, database queries and data integration workflows.
- Monitor pipeline quality and performance, troubleshoot issues and document data processes.
Job Experience & Skills Required / Ideal Candidate Profile
Education
- A degree, diploma or equivalent relevant qualification in Computer Science, Information Systems, Data Engineering or a related field is advantageous.
- Relevant cloud, database or data engineering certifications will be advantageous.
- Training in analytics platforms or data architecture will be beneficial.
- Demonstrable hands-on data engineering experience is valued.
Experience
- Typically 4+ years of experience in data engineering, ETL development or a closely related role.
- Strong SQL skills and experience developing data pipelines or ETL/ELT processes.
- Experience with relational databases, data modelling and data quality controls.
- Cloud data platforms, Python, orchestration tools or streaming technologies will be highly advantageous.
Skills
- SQL, data modelling and database performance awareness.
- ETL/ELT, data ingestion, transformation and pipeline orchestration.
- Python or another scripting language.
- Cloud data services, data quality, monitoring and documentation.
Requirements
- Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
- 5+ years of experience in data engineering, with a focus on building large-scale data solutions.
- Strong proficiency in SQL and programming languages such as Python or Scala.
- Experience with big data technologies (e.g., Spark, Hadoop).
- Proven experience with cloud data services on AWS, Azure, or GCP.
- Expertise in designing and implementing ETL/ELT processes and data warehousing concepts.
- Familiarity with data modeling techniques and database design.
- Understanding of data governance and data quality principles.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills, effective in a hybrid work environment.
Benefits
- Competitive salary and potential for performance bonuses.
- Hybrid work arrangement for a flexible work-life balance.
- Opportunities for professional development and acquiring new skills.
- Work on challenging and impactful data projects.
- Collaborative environment with experienced data professionals.
- Exposure to modern big data technologies and cloud solutions.
- Comprehensive health and retirement benefits.
About the Role
Our client is a dynamic data analytics firm, seeking a senior Data Engineer in Randburg, Gauteng, ZA. This hybrid role blends in-office collaboration with remote flexibility. Responsibilities include designing, building, and optimizing scalable data pipelines and ensuring that the data foundation supports AI and machine learning initiatives. The ideal candidate brings deep expertise in data warehousing, ETL, big data technologies, and cloud platforms. This role is pivotal for building the backbone of data analytics and AI performance.