Sql Data Engineer

Csg Recruit; Pretoria; On-site; ZAR 300 000 - 700 000; Full time

Job summary

An established industry player is seeking a skilled data engineer with a strong background in data management and automation. This role involves developing robust data solutions using advanced SQL, SSIS, and Python, while ensuring compliance with industry regulations. The successful candidate will work in an Agile environment, collaborating with stakeholders to deliver high-quality data pipelines and governance frameworks. If you are passionate about data and eager to contribute to a dynamic team culture, this opportunity is perfect for you.

Qualifications

• 5+ years in warehouse, data lake, or data vault development.
• Advanced proficiency in SQL and SSIS, including performance tuning.
• Experience with Python libraries for data manipulation.

Responsibilities

• Document business requirements and deliver innovative data solutions.
• Develop and maintain ETL/ELT data pipelines for large-scale data.
• Implement data governance frameworks and compliance protocols.

Skills

Problem-solving skills

Python automation

SQL Transact

Data governance

AWS services

Agile methodologies

DevOps practices

Education

Relevant Tertiary Qualification (e.g., BSc IT)

Tools

SSIS

SQL Server

GitLab

Power BI

Visual Studio

Job description

Relevant Tertiary Qualification: (e.g., BSc IT or similar)

Experience:

1. At least 5 years working experience in warehouse, data lake, or data vault development using SSIS and SQL Transact (not just fluency in SQL for 10 years).
2. 5 years+ of Python automation.
3. Strong problem-solving skills with a focus on delivering robust data solutions.
4. Ability to translate a documented specification into a robust implemented solution.
5. Strong understanding of regulatory compliance, data governance, and financial data structures.
6. Advanced proficiency in SQL Transact, SQL Server, SSIS, and SSRS, including performance tuning and query optimization.
7. Strong knowledge of AWS services related to data management.
8. Proficiency in GitLab for version control, CI/CD automation, and deployment pipelines.
9. Experience with Python libraries such as Pandas, NumPy, PySpark, and SQLAlchemy for data manipulation and integration.
10. Knowledge of Visual Studio, MDS, and Power BI.
11. Database administration (DBA) experience, including index management, query optimization, and performance tuning.

Methodologies:

1. Working experience in an Agile environment (Scrum, Kanban) with sprint planning and backlog grooming.
2. Proficiency in implementing DevOps practices for automated deployments, monitoring, and alerting.

Duties & Responsibilities:

1. Elicit and document business requirements directly from data clients.
2. Deliver innovative solutions aligned with business needs and strategies.
3. Develop and maintain ETL/ELT data pipelines using SSIS and AWS Glue for large-scale data ingestion and transformation.
4. Implement database solutions with partitioning, indexing, and stored procedures to improve query execution times.
5. Automate deployment pipelines using GitLab CI/CD to streamline development and operational workflows.
6. Implement data governance frameworks, access controls, and compliance protocols to meet financial industry regulations.
7. Build and maintain effective relationships with stakeholders.
8. Deliver consistent, high-quality service while managing expectations.
9. Promote a feedback-driven and service-oriented culture.

Self-Management & Teamwork:

1. Contribute to a positive and energizing workplace culture.
2. Align individual performance with team and organizational goals.
3. Encourage and embrace innovation, agility, and collaboration.
4. Take ownership of personal development through effective performance planning and feedback.