Lead Data Engineer

Purple Hires Inc; Madison (WI); Hybrid; USD 83,000 - 124,000; Part time

Job summary

Purple Hires Inc. in Madison, WI is seeking a Lead Data Engineer for a 3-month hybrid contract. The role requires 2–3 days in the office and proximity within 40–45 miles.

You will lead an engineering team, oversee data pipelines, and ensure data quality and security across the lifecycle. The ideal candidate has strong GCP experience, deep Python ETL expertise, and hands-on work with SQL/NoSQL, Spark/Hadoop, and data modeling.

Qualifications

• Experience delivering customer-focused data solutions.
• Proficient with GCP for data engineering.
• Strong knowledge of SQL/NoSQL databases and data stores.
• Extensive Python programming for ETL workflows and data pipelines.
• Able to write reusable, well-architected code.

Responsibilities

• Leads an engineering team to meet project deadlines and priorities.
• Supervises data engineering team members and activities.
• Ensures data quality, security, integrity across the data lifecycle.
• Documents critical workflows and operational support for team responsibilities.
• Develops understanding of data sources, granularity, availability, limitations.
• Provides technical oversight to architecture and development teams for scalable data solutions.
• Creates maintainable, scalable code to load and manipulate data in the warehouse.
• Facilitates communication with project teams and stakeholders.]
• Specialized knowledge of batch processing and big data tools (Hadoop/Spark).

Skills

GCP experience

Python ETL

SQL/NoSQL

Big data

Spark/Hadoop

CI/CD

Docker

Data modeling

Open source ML

Tools

Terraform

CloudFormation

Jenkins

Git

Docker

Job description

Position: Lead Data Engineer

Location: Madison WI 53783 - Hybrid - 2-3 Days in Office candidate needs to be within 40 to 45 miles of

3 months contract

Job Description

Primary Accountabilities

• Leads an engineering team to meet project deadlines and priorities.
• Supervises assigned data engineering team members & activities.
• Ensures the quality, completeness, security, privacy, and integrity of data throughout the data lifecycle.
• Documents critical workflows and operational support aspects of team's responsibilities
• Develops deep understanding of data sources, granularity, availability, and limitations.
• Provides proactive technical oversight and advice to application architecture and development teams fostering re-use, design for scale, stability, and operational efficiency of data/analytical solutions.
• Creates maintainable, scalable code to load and manipulate data in the data warehouse.
• Facilitates communication upward and across project teams and business stakeholders.

Specialized Knowledge & Skills Requirements

• Demonstrated experience providing customer-driven solutions, support or service.
• Must have GCP experience for this positon.
• In-depth knowledge of SQL or NoSQL and experience using a variety of data stores (e.g. RDBMS, analytic database, scalable document stores)
• Extensive hands-on Python programming experience, with an emphasis towards building ETL workflows and data-driven solutions.
• Able to employ design patterns and generalize code to address common use cases.
• Capable of authoring robust, high quality, reusable code and contributing to the division's inventory of libraries.
• Expertise in big data batch computing tools (e.g. Hadoop or Spark), with demonstrated experience developing distributed data processing solutions.
• Knowledge of open source machine learning toolkits, such as sklearn, SparkML, or H2O.
• Solid data understanding and business acumen in the data rich industries like insurance or financial
• Applied knowledge of data modeling principles (e.g. dimensional modeling and star schemas).
• Strong understanding of database internals, such as indexes, binary logging, and transactions.
• Experience using tools for infrastructure-as-code (e.g. Docker, CloudFormation, Terraform, etc.)
• Experience with software engineering tools and workflows (i.e. Jenkins, CI/CD, git).
• Practical experience authoring and consuming web services.