Data Lead

AVATAR MODERN TECHNO SERVICES PTE. LTD.; Singapore; On-site; SGD 180,000 - 240,000; Full time

Job summary

Avatar Modern Tech Services Pte. Ltd. in Singapore is seeking a Data Lead – Data Engineering to drive diagnostics, optimization, and stabilization of enterprise data platforms supporting large-scale ETL and batch processing.

You will lead design and optimization of ADF pipelines, Databricks workflows, SSIS packages, and SQL-based transformations, while improving monitoring, error handling, and production reliability.

Qualifications

• Proven experience leading data engineering initiatives.
• Strong knowledge of ETL/ELT design and batch processing.
• Experience with cloud-based data platforms (Azure) and on-premise tools.

Responsibilities

• Drive diagnostics, optimization, and stabilization of enterprise data platforms.
• Review and optimize ADF pipelines, SSIS packages, and Databricks workflows for performance and scalability.
• Analyze batch scheduling, dependencies, triggers, and SLA adherence.
• Investigate and resolve pipeline failures and runtime issues to ensure fast recovery.

Skills

Data pipeline design

SQL development

Performance tuning

Troubleshooting

Monitoring

Cross-team collaboration

Tools

Azure Data Factory

Databricks

SSIS

SQL Server

Control-M

Autosys

Azure Monitor

Log Analytics

Job description

We are looking for a Data Lead – Data Engineering to drive technical diagnostics, optimization, and stabilization of enterprise data platforms supporting large-scale ETL and batch processing systems.

Key Responsibilities

• Strong expertise in: Azure Data Factory (ADF),Databricks (PySpark / Spark SQL preferred),SSIS and SQL Server
• Hands-on experience with: Batch orchestration tools (Control-M / Autosys),ETL/ELT pipeline design and optimization
• Strong experience in: SQL development and performance tuning, Troubleshooting and debugging complex data pipelines
• Familiarity with: Monitoring and alerting tools (Azure Monitor, Log Analytics)
• Exposure to large-scale data processing and batch system
• Review and optimize ADF pipelines, SSIS packages, and Databricks workflows for performance, scalability, and
• Analyze batch scheduling and orchestration frameworks (Control-M / Autosys), including dependencies, triggers, and SLA adherence
• Investigate and resolve pipeline failures, job delays, and runtime issues, ensuring faster recovery and minimal business
• Evaluate source-to-target integration flows, SQL logic, and transformation layers for efficiency and correctness
• Define and improve error handling, retry mechanisms, restart ability, and reprocessing strategies across ETL and batch
• Optimize runtime performance of data pipelines, including query tuning and batch execution improvements
• Establish and enhance monitoring and alerting frameworks using tools like Azure Monitor, Control-M dashboards, and custom logging solutions
• Identify automation opportunities to reduce manual intervention and improve operational efficiency
• Ensure data quality, consistency, and reliability across ETL processes and reporting outputs
• Support modernization initiatives, including migration from legacy ETL (SSIS) to cloud-based platforms like Databricks
• Collaborate with data engineers, architects, business, and support teams to drive improvements and ensure stable production systems