Hybrid DataOps Lead Engineer: Cloud & Pipelines

Selective Insurance; Millburn (NJ); On-site; USD 130,000 - 176,000; Full time

Job summary

Selective Insurance is seeking an energetic Data Engineering Team Lead to guide data, analytics, and DataOps initiatives within the Information Management group in Millburn, NJ. You will lead a high-performing team delivering key data components and establishing best practices across the organization.

The role partners with Data Architecture, Analytics, AI/ML, and Platform teams to operationalize data products end-to-end, ensuring reliability, change management, and consistent data consumption

Qualifications

• 8+ years of experience in data engineering, analytics engineering, or platform engineering.
• Strong experience with cloud data platforms (Azure, AWS, or GCP).
• Experience with Databricks, or modern lakehouse architecture.
• Proficient with Data pipelines and orchestration (e.g., Airflow, ADF, Dagster, dbt)
• Proficient with CI/CD tools (Azure DevOps, GitHub Actions, GitLab, etc.)
• Proficient in Data modeling, PySpark and SQL
• Proficient in modern data architecture
• Experience with DataOps or other automation frameworks.

Responsibilities

• Lead a high performing team in the delivery of key components and business deliverables.
• Lead the design and implementation of DataOps frameworks, standards, and best practices across the organization.
• Partner with Data Architecture, Analytics, AI/ML, and Platform teams to operationalize data products endtoend.
• Establish operational expectations for data products/data contracts, including reliability targets, change management, and consistency of consumption for analytics and AI use cases.
• Drive adoption of automation, orchestration, and observability capabilities for data pipelines.
• Ensure both on-time successful delivery, and the technical and architectural quality of the solutions.
• Collaborate with individuals from all areas of the business and IT organizations to develop accurate business requirements and design architectural solutions.
• Resolve and escalatie technical and project delivery issues as needed.
• Drive process improvements within the team and across the department to improve efficiency and quality.
• Work directly with the architecture group to coordinate new design patterns and technology.
• Assist in prioritizing new development projects, enhancement projects for existing systems, and system maintenance requests.
• Act as a key contributor in planning and estimation of projects
• Support development of talent within the organization through coaching and mentoring of team members.
• Ensure design, code, and test plan reviews are conducted as appropriate.

Skills

Data engineering

Analytics engineering

Platform engineering

Cloud data platforms

Databricks / lakehouse

Data pipelines / orchestration

CI/CD tools

PySpark / SQL

Data modeling

DataOps

Education

College degree in Computer Science or MIS

Tools

Azure

AWS

GCP

Airflow

Dagster

dbt

Azure DevOps

GitHub Actions

Job description

Selective Insurance is seeking an energetic Data Engineering Team Lead to guide data, analytics, and DataOps initiatives within the Information Management group in Millburn, NJ. You will lead a high-performing team delivering key data components and establishing best practices across the organization.

The role partners with Data Architecture, Analytics, AI/ML, and Platform teams to operationalize data products end-to-end, ensuring reliability, change management, and consistent data consumption