Lead Data Engineer - Cloud & Big Data Solutions

American Family Insurance Agency Staff; Hopkins (MN); On-site; USD 131,000 - 220,000; Full time

Job summary

American Family Insurance is seeking a Senior Data Engineer to lead an engineering team responsible for building applications within our big data platform and integrating them with enterprise architecture.

This role emphasizes data quality, scalable ETL pipelines, and collaboration with application teams to deliver robust data-driven solutions in a security-conscious environment. The position has approval to transfer a valid H1 visa and relocation support is available.

Qualifications

• Demonstrated experience providing customer-driven solutions, support or service.
• Must have GCP experience for this positon.
• In-depth knowledge of SQL or NoSQL and experience using a variety of data stores (e.g. RDBMS, analytic database, scalable document stores)

Responsibilities

• Leads an engineering team to meet project deadlines and priorities.
• Supervises assigned data engineering team members & activities.
• Ensures the quality, completeness, security, privacy, and integrity of data throughout the data lifecycle.
• Documents critical workflows and operational support aspects of team's responsibilities
• Develops deep understanding of data sources, granularity, availability, and limitations.
• Provides proactive technical oversight and advice to application architecture and development teams fostering re-use, design for scale, stability, and operational efficiency of data/analytical solutions.
• Creates maintainable, scalable code to load and manipulate data in the data warehouse.
• Facilitates communication upward and across project teams and business stakeholders.

Skills

GCP experience

SQL/NoSQL

Python programming

ETL workflows

Data modeling

Big data (Hadoop/Spark)

Machine learning toolkits (SparkML/H2O

Tools

Docker

CloudFormation

Terraform

Jenkins

Git

RESTful services

Data warehouses

Job description

American Family Insurance is seeking a Senior Data Engineer to lead an engineering team responsible for building applications within our big data platform and integrating them with enterprise architecture.

This role emphasizes data quality, scalable ETL pipelines, and collaboration with application teams to deliver robust data-driven solutions in a security-conscious environment. The position has approval to transfer a valid H1 visa and relocation support is available.