Senior Data Engineer - ETL, Python & API Pipelines

TechXSA; Selangor; On-site; MYR 90,000 - 180,000; Full time

Job summary

TechxSA Solution is a leading data and engineering services provider in the oil & gas sector. The role focuses on translating data requirements into robust ETL/ELT pipelines, data models, and API endpoints to support client systems and governance.

Candidates should have 3+ years in data engineering, strong Python/SQL skills, experience with Talend/SSIS/Azure Data Factory, and familiarity with BI tools. Collaboration with SDEs and architecture teams is essential for project success.

Qualifications

• Minimum 3 years’ experience in Data Engineering / Data Programming, preferably within the Oil & Gas industry.
• Strong programming and data engineering skills, particularly Python and SQL.
• Hands‑on experience in ETL/ELT pipeline development, data transformation, data validation, and automation.
• Experience with ETL tools such as Talend, SSIS, Azure Data Factory, or equivalent.
• Experience in data modelling, ERD development, source-to-target mapping, and business rule implementation.
• Knowledge of data quality, data governance, validation rules, and error handling.
• Experience developing and integrating APIs / data-serving endpoints.
• Familiar with data integration and enterprise data platforms, preferably within Oil & Gas / upstream / operational environments.
• Experience troubleshooting data mapping, transformation, pipeline, API, and integration issues.
• Familiar with Power BI, Tableau, Spotfire, or similar BI / analytics tools is an added advantage.
• Good understanding of technical documentation, including data mapping, ETL design, transformation logic, pipeline documentation, and API specifications.

Responsibilities

• Translate approved data requirements, taxonomy, source mapping, and business rules into technical implementation.
• Design and develop technical data models / ERDs aligned with UDP requirements and architecture.
• Develop, configure, and test ETL/ELT pipelines for data extraction, transformation, validation, and loading.
• Apply Python, SQL, ETL/ELT, data transformation, automation, and data validation skills.
• Implement data quality checks, validation rules, error handling, and data governance within ingestion pipelines.
• Develop and test data-serving APIs / endpoints and support integration with consuming systems.
• Coordinate with architecture and support teams for data model, pipeline, API, and integration validation.
• Work closely with Senior Data Engineers (SDEs) to resolve mapping, transformation, business-rule, and technical issues.
• Lead technical defect investigation and resolution, and support preparation of builds for UAT.
• Maintain technical documentation including ERD, data mapping, ETL design, transformation logic, pipeline documentation, and API specifications for R&M handover.

Skills

Python

SQL

ETL/ELT

Data modelling

ERD design

Data quality

Data governance

APIs

Data integration

Power BI

Tableau

Spotfire

Tools

Talend

SSIS

Azure Data Factory

Job description

TechxSA Solution is a leading data and engineering services provider in the oil & gas sector. The role focuses on translating data requirements into robust ETL/ELT pipelines, data models, and API endpoints to support client systems and governance.

Candidates should have 3+ years in data engineering, strong Python/SQL skills, experience with Talend/SSIS/Azure Data Factory, and familiarity with BI tools. Collaboration with SDEs and architecture teams is essential for project success.