Senior Data Engineer - Hybrid, Spark & Python

Planet Group; Irving (TX); Híbrido; USD 96.000 - 110.000; Jornada completa

Descripción de la vacante

Planet Group is partnering with a Chicago-area firm to hire a Data Engineer who will design, build, and maintain scalable ETL pipelines powering BI, analytics, and downstream apps. The role emphasizes Python, Spark, and SQL skills, data quality, and production-grade solutions.

You will collaborate with technical and business stakeholders, translate requirements into efficient workflows, and help improve pipeline performance in a hybrid Chicago environment with on-site three days a week.

Formación

• 5+ years of hands-on experience developing and maintaining ETL pipelines in production environments.
• Strong Python programming skills for data processing, transformation, and automation.
• Practical experience using Apache Spark to process large datasets.
• Advanced SQL skills, including complex queries and data transformations.
• Solid understanding of data engineering principles, including scalability, reliability, data quality, and performance optimization.
• Experience troubleshooting production data workflows and resolving technical issues.
• Strong communication, problem-solving, and collaboration skills.

Responsabilidades

• Design, build, and maintain production ETL pipelines that extract, transform, and integrate large datasets.
• Develop data processing workflows using Python and Apache Spark.
• Write and optimize complex SQL queries to support data transformation, validation, and analysis.
• Partner with cross-functional teams to understand source data, define requirements, and deliver usable datasets.
• Implement data quality checks to improve accuracy and consistency across pipelines.
• Investigate pipeline failures, troubleshoot processing issues, and resolve performance bottlenecks.
• Improve the scalability, reliability, and maintainability of existing data solutions.
• Follow engineering best practices for version control, code documentation, and collaborative development.
• Participate in an agile team environment and communicate progress, issues, and technical decisions clearly.

Conocimientos

Python

Apache Spark

SQL

ETL pipelines

Data quality

Troubleshooting

Communication

Collaboration

Herramientas

Azure Data Factory

Databricks

Power BI

Pulumi

Descripción del empleo

Planet Group is partnering with a Chicago-area firm to hire a Data Engineer who will design, build, and maintain scalable ETL pipelines powering BI, analytics, and downstream apps. The role emphasizes Python, Spark, and SQL skills, data quality, and production-grade solutions.

You will collaborate with technical and business stakeholders, translate requirements into efficient workflows, and help improve pipeline performance in a hybrid Chicago environment with on-site three days a week.