ETL Lead

Delan Associates, Inc; Greenville (SC); On-site; USD 110,000 - 160,000; Full time

Job summary

Delan Associates, Inc. seeks an ETL Lead in Insurance domain to own end-to-end data operations. You will supervise production support, monitor ETL loads across SQL workloads, and guide onshore/offshore developers to maintain critical datasets.

Responsibilities include troubleshooting pipeline issues, implementing permanent fixes, refining runbooks, and driving modernization with Delta Lake, Fabric Lakehouse, and Medallion-style architectures. Onsite in Greenville, SC.

Qualifications

• 12+ years of ETL/Data Warehousing experience.
• Strong SQL, SSIS, and ETL design & development skills.
• Experience leading onshore/offshore teams and reviews.
• Proven ability to troubleshoot complex data pipelines and root-cause analysis.

Responsibilities

• Own end-to-end operational support for the data stack and ensure SLA adherence.
• Monitor daily ETL loads across SQL jobs and data hubs; troubleshoot failures and performance issues.
• Provide direction to AMS and ETL developers; mentor juniors and review code.
• Refactor or retire outdated ETL processes and build scalable ingestion pipelines.
• Modernize workloads using Delta Lake, Medallion Architecture, Fabric Lakehouse; explore Azure Data Factory/Databricks/Synapse pipelines.

Skills

Production Support

Operations & Reliability

Mentoring & Coaching

Technical Leadership

Code Review

Cross-functional Collaboration

ETL Best Practices

Tools

SQL Server

T-SQL

SQL Server Integration Services (SSIS)

Microsoft PowerShell

Guidewire Data Hub

Info Center

Oracle PL-SQL

Oracle Database

SAP BODS

Delta Lake

Databricks

Fabric Lakehouse

Job description

Job Tittle: ETL Lead | Insurance Domain

Job Location: Green Ville , SC (Onsite)

Duration : 6+ Months contract

Visa: USC and GC

Years of Exp: 12+

Key Responsibilities

• Production Support, Operations & Reliability


• The ETL Team Lead owns end-to-end operational support for Canal’s existing data stack:


• Ensure success of day-to-day production Operations.


• Monitor daily ETL loads across SQL jobs, DHIC (GW Data Hub and Info Center) and legacy SSIS packages.


• Work with AMS team where necessary to troubleshoot pipeline failures, performance issues, schema mismatches, permissions issues, and cloud resource failures.


• Work with AMS team where necessary to perform root-cause analysis and to implement permanent fixes.


• Ensure SLA adherence and on-time delivery of critical reporting datasets for scheduled ETL jobs .


• Provide direction for both AMS and ETL Developers for Legacy & Current ETL maintenance.


• Refactor or retire outdated or redundant ETL processes.


• Maintain and improve existing pipelines that utilize the following technologies:


• Microsoft SQL Server Database programming


• T-SQL Scripting


• SQL Server Integration Services


• Microsoft PowerShell


• Guidewire Data Hub and Info Center


• Oracle Database programming


• Oracle PL-SQL Scripting


• SAP BODS (SAP BusinessObjects Data Services)


• PostgreSQL Scripting


• Operational Excellence


• Work with AMS team to assist with the creation and/or enhancement of operational runbooks, SOPs, monitoring dashboards, and incident response workflows.


• Partner with other IT operational segments, business SMEs, and AMS team to minimize downtime and to ensure that business SLAs are met.


• Improve upon existing, as well as implement new proactive for daily processing.


• Business Continuity


• Work with AMS team to ensure development support coverage for critical data pipelines (rotation-based).


• Support month-end and quarter-end financial reporting cycles.


• Coordinate production releases and validate deployments.


• The ETL Tech Lead will become the steady-state technical owner of the entire data operations layer during the Canal modernization journey.


• Technical Leadership & Collaboration


• Serve as technical lead guiding onshore/offshore developers.


• Review code, enforce best practices, and mentor junior engineers.


• Partner with Scrum Masters, Project Mangers, Enterprise Architecture, QA Automation, Change Management, and AMS support teams.


• Data Ingestion, ETL/ELT Development & Optimization


• Develop reusable ingestion patterns for Guidewire DataHub and InfoCenter, HubSpot, Telematics, and other facets of the business.


• Work with Canal Architects to modernize existing ETL workloads using Delta Lake, Medallion Architecture, and Fabric Lakehouse.


• Build scalable data ingestion pipelines using potential upcoming technology (Azure Data Factory, MS Fabric , Databricks, Synapse Pipelines, etc.).


• Work to bring in Internal and External integrations data into the platform.


• Real-Time, Streaming & Event-Driven Engineering


• Proven experience designing and implementing real-time data pipelines using Event Hub, Fabric Real-Time Analytics, Databricks Structured Streaming, and KQL-based event processing.


• Develop and enable real-time operational insights and automation capabilities—including Telematics alerting, FNOL automation, and fraud/VNOS/VNOP detection—through event-driven architectures and streaming analytics.


• Modern Azure Data Stack Leadership


• Lead the strategy, design, and engineering of Canal’s modern Azure data ecosystem using next-generation tools and Medallion Architecture, including:


• Implementing Medallion Architecture (Bronze/Silver/Gold) across Fabric Lakehouse, Warehouse, Eventhouse, and KQL Database.


• Leveraging Delta tables with schema enforcement, ACID compliance, and versioning.


• Building ingestion and transformation pipelines using Dataflows Gen2, Fabric Pipelines, and Notebook-based ETL.


• Enabling real-time analytics using RTA, EventStreams, and Event Hub.


• Driving notebook-based transformations and advanced engineering using Databricks and Synapse runtimes.


• Data Modeling, Curation & Governance


• Develop curated, analytics-ready datasets to support Power BI, operational reporting, and advanced analytics use cases.


• Assist Canal architect with implementation of Data Governance tools.


• Establish robust data quality, validation, alerting, and observability frameworks to maintain accuracy, consistency, and trust in enterprise data.


• AI/ML Data Enablement (Optional)


• Prepare ML-ready datasets for pricing, risk, fraud detection, underwriting, claims leakage, and predictive insights