Senior Data Engineer

Jobtailor

Menomonee Falls (WI)

On-site

USD 120,000 - 180,000

Full time

2 days ago

Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

Jobtailor in Wisconsin is seeking a Senior Data Engineer to design, build, and scale cloud-based data ecosystems on GCP. You will develop robust ETL pipelines, optimize data models, and ensure data quality across batch and streaming workloads.

Collaborate with data scientists, product teams, and engineers to implement GenAI-enabled data improvements, enforce PCI/PII compliance, and drive cost-efficient performance. Proven TDD, CI/CD, and XP practices preferred.

Qualifications

4+ years of experience in software development.
Understanding of design patterns, event-driven architecture, databases, schemas, and testing strategies.
In-depth knowledge of CI/CD and TDD.
Degree in MIS/CS or related field preferred.

Responsibilities

Lead development of robust, observable, and measurable applications using XP practices and a user-centric approach.
Participate in the full application lifecycle with designers, product managers, and engineers.
Use data, experiments, and best practices to achieve business outcomes.
Facilitate group discussions and team ceremonies.
Establish and lead product engineering and software standards.
Ideate products from user problems into testable solutions.
Research current technology trends and practices.
Lead technical initiatives across the team and department.
Develop, automate, and maintain batch and streaming ETL pipelines.
Build and manage cloud-based data ecosystems on GCP.
Design and optimize data models for data lakes and warehouses.
Implement real-time ingestion and streaming data processing.
Optimize data performance, scalability, and cost on GCP.
Ensure PCI and PII data compliance.
Integrate GenAI tools for data quality and analytics enhancement.
Collaborate with stakeholders, data scientists, and full-stack engineers to deliver reusable data products.
Perform additional assigned tasks.

Skills

GCP Cloud Services

ETL Pipeline Development

Data Modeling (SQL, NoSQL)

CI/CD

Event-Driven Architecture

Performance Tuning

Apache Kafka

Data Mesh Principles

Streaming Data Processing

Apache Airflow

Apache Spark

Python

Scala

Education

Bachelor's Degree in MIS, Computer Science, or Related Field

Tools

Apache Airflow

Apache Spark

Python

Scala

BigQuery

MongoDB

Snowflake

Pub/Sub

Dataproc

Bigtable

Job description

Lead development of robust, observable, and measurable applications using XP practices and a user-centric approach
Participate in the full application lifecycle with designers, product managers, and engineers
Use critical thinking, experimentation, data, and industry best practices to achieve business outcomes
Facilitate group discussions and team ceremonies
Establish and lead product engineering and software standards
Ideate products from user problem spaces through ranked, testable solutions
Research current technology trends and practices
Lead technical initiatives across the team and department
Develop, automate, and maintain batch and streaming ETL pipelines
Build and manage cloud-based data ecosystems on GCP
Design and optimize data models for data lakes and warehouses
Implement real-time ingestion and streaming data processing
Optimize data performance, scalability, and cost efficiency across GCP
Ensure PCI and PII data compliance
Integrate GenAI tools for data quality and analytics enhancement
Collaborate with stakeholders, data scientists, and full-stack engineers to deliver trusted, documented, reusable data products
Perform additional assigned tasks

Requirements

4+ years of experience in software development
Understanding of application design patterns, event-driven architecture, databases, schemas, and testing strategies
In-depth knowledge and experience with continuous integration, continuous deployment, and test-driven development
Bachelor's Degree or equivalent in MIS, Computer Science, or a related field (preferred)
Experience with large-scale application troubleshooting and performance tuning (preferred)Exposure to major cloud platforms: GCP, AWS, or Azure (preferred)
Familiarity and experience with XP (Extreme Programming) (preferred)
Experience with Apache Airflow, Apache Spark, Python, and Scala for ETL pipelines
Experience with GCP services including BigQuery, Bigtable, Dataproc, Pub/Sub, Cloud Storage, IAM, and VPC
Experience designing SQL and NoSQL data models using BigQuery, MongoDB, and Snowflake
Experience writing complex SQL queries for data transformation, aggregation, and analytics optimization
Experience applying TDD to Airflow workflows, Spark jobs, and transformation logic
Knowledge of data mesh and data-as-a-product principles
Experience with Kafka Connect and streaming technologies such as Spark Streaming or Apache Flink
Knowledge of GCP data performance, scalability, and cost optimization
Knowledge of PCI, PII, GDPR, PCI DSS, SOX, and CCPA compliance
Experience integrating GenAI tools such as OpenAI, Gemini, and Anthropic LLMs
Ability to collaborate with stakeholders, data scientists, and full-stack engineers

Core Competencies

Demonstrates expertise in developing and optimizing cloud-based data ecosystems on GCP, utilizing ETL pipelines and real-time data processing. Proficient in applying XP practices, data compliance standards, and collaboration with cross-functional teams to deliver high-quality data products.

Highest-signal resume keywords