Data Engineer 2 - Guidewire & AI Pipelines

Jobtailor; Edison (NJ); On-site; USD 120,000 - 150,000; Full time

Job summary

Jobtailor is seeking a senior data engineer in New Jersey to design and optimize data solutions for Guidewire domains, including Policy Center and Billing Center. You will build robust ELT pipelines to ingest, transform, and load data into a Delta Lakehouse platform and mentor colleagues in data engineering best practices.

You will also design data models, dashboards, and governance standards while supporting Generative AI data tasks with structured and unstructured data processing and embedding

Qualifications

• Bachelor’s degree in a computer science, data science, analytics, or related field.
• 5+ years in IT or related field.
• 3+ years in data engineering and analytics with enterprise data models.
• 2+ years with Guidewire data structures and Guidewire ecosystem.
• 2+ years with Azure Databricks and Azure Data Factory.
• 2+ years in Power BI or equivalent tools.
• Expertise in Spark, Delta Lakehouse, and data warehousing.
• Proficiency in Azure services (SQL DB, Data Lake Gen2, Event Grid, Key Vault).
• Strong CI/CD and Agile experience.
• Knowledge of data governance, security, and metadata management.

Responsibilities

• Analyze requirements and design data solutions for Guidewire domains.
• Develop and optimize ELT pipelines for Delta Lakehouse.
• Build and maintain data models and schemas for quality data.
• Create dashboards and reports delivering actionable insights.
• Monitor pipelines and troubleshoot data issues promptly.
• Collaborate with analysts and business users to deliver end-to-end data solutions.
• Implement governance, security, and compliance standards.
• Conduct root-cause analysis and continuous improvements in pipelines.
• Prepare testing procedures to ensure pipeline reliability.
• Stay current with new technologies and suggest improvements.
• Mentor team members on data engineering best practices.
• Develop data pipelines for structured and unstructured data for Generative AI tasks.
• Prepare large datasets for training Generative AI models.
• Collab with AI teams to support model training and inference.
• Enrich data with metadata to improve model accuracy.
• Optimize storage and retrieval for fast AI-ready access.
• Monitor GenAI data workflows and troubleshoot issues.

Skills

ETL pipelines

Data modeling

SQL databases

Azure Databricks

Azure Data Factory

Apache Spark

Delta Lakehouse

Power BI

Data governance

CI/CD & Agile

Generative AI knowledge

Data security & privacy

Mentoring

Education

Bachelor’s degree in relevant field

Tools

Guidewire Policy Center

Oracle

Job description

Jobtailor is seeking a senior data engineer in New Jersey to design and optimize data solutions for Guidewire domains, including Policy Center and Billing Center. You will build robust ELT pipelines to ingest, transform, and load data into a Delta Lakehouse platform and mentor colleagues in data engineering best practices.

You will also design data models, dashboards, and governance standards while supporting Generative AI data tasks with structured and unstructured data processing and embedding