A complete application in a minute
Tailored resume and cover letter, ready to send.
U3 INFOTECH PTE. LTD. is seeking a data engineer to design and maintain scalable ETL/ELT pipelines using Databricks, PySpark, SQL and Delta Lake.
The role focuses on integrating data from databases, APIs and files, with streaming where needed, and building curated datasets for reporting, analytics and AI applications. You will implement data quality checks, monitor pipelines, troubleshoot issues and optimise performance, while applying CI/CD and technical documentation practices to support
Design, develop and maintain scalable ETL/ELT data pipelines using Databricks, Python, PySpark, SQL and Delta Lake.
Integrate data from multiple sources, including databases, APIs and files, with streaming ingestion where required.
Develop and maintain curated datasets and data models to support business reporting, analytics and AI applications.
Implement data quality checks, data validation, reconciliation and incremental data processing.
Monitor data pipelines, investigate failures, troubleshoot issues and ensure data accuracy and availability.
Optimise data processing performance, pipeline execution and compute utilisation.
Implement version control, automated testing, CI/CD and technical documentation.
Identify and automate repetitive data engineering and support activities.
Explore practical AI