Senior SQL Data Engineer

UrbanSoft SA PTY; Johannesburg; On-site; ZAR 200,000 - 300,000; Full time

Job summary

An innovative firm is looking for a SQL Specialist to design and manage data processes using advanced SQL and big data technologies on AWS. In this role, you'll optimize SQL queries, ensure data integrity, and collaborate with cross-functional teams to align data systems with business goals. You will also provide insights through data analysis and reporting, troubleshoot data-related issues, and maintain documentation for best practices. If you are passionate about data and thrive in a collaborative environment, this position offers an exciting opportunity to make a significant impact on data-driven decision-making.

Qualifications

• Advanced knowledge of SQL and experience with big data technologies.
• Ability to manage data import/export and ensure data integrity.

Responsibilities

• Design and manage data processes using AWS cloud services.
• Optimize SQL queries and improve database performance.

Skills

SQL

Data Management

Database Performance Tuning

Data Analysis

Collaboration

Tools

AWS Athena

AWS Redshift

AWS Lambda

AWS Cloud Formation Stack

AWS EMR

Job description

Key Responsibilities

Design data across the full range from fast/clean ETL streams through to automation. Ability to use big data technologies on AWS cloud services: Athena, Redshift, Lambda, Cloud Formation Stack, EMR. Ability to work with large data sources. Advanced knowledge of SQL (Oracle, Redshift, PostgreSQL).

Writing and Optimizing SQL Queries:

• Writing complex SQL queries to extract, manipulate, and analyze data from relational databases.
• Optimizing SQL queries for performance, ensuring efficient execution on large datasets.

Data Management:

• Managing data import and export processes between databases or systems.
• Ensuring data consistency, accuracy, and integrity across the database.

Database Performance Tuning:

• Analyzing and improving database performance using indexing, query optimization, and resource management.
• Identifying and resolving database bottlenecks and performance issues.

Data Analysis and Reporting:

• Generating reports based on business requirements and analyzing data to provide insights for decision-making.
• Do proof of concepts and refine logic for transformation of new data sources.
• Creating stored procedures, triggers, and views to streamline reporting and analysis.
• Wrangle financial data to provide reconciliations between different data sources and financial systems.
• Write test cases to validate data transformations.

Collaboration with Other Teams:

• Working with developers, system administrators, and other stakeholders to ensure data systems are aligned with business goals.
• Providing SQL-related support for application development, ensuring smooth data interactions between applications and databases.

Troubleshooting and Support:

• Identifying and resolving data-related issues, including data corruption, missing data, inconsistent data, connectivity problems, or query performance issues.
• Providing ongoing support for production databases, addressing user queries and performance-related concerns.

Documentation:

• Documenting SQL scripts, database schemas, and queries for future reference or use by other team members.
• Maintaining clear documentation of processes and standards for database management and optimization.

Compliance and Best Practices:

• Ensuring code is compliant with industry standards, regulations, and best practices.
• Staying updated on new SQL technologies, database tools, and best practices to ensure continual improvement.
• These responsibilities can vary based on the size of the organization and the specific role of the SQL Specialist, but these are the common core duties associated with the position.