Senior PySpark Data Engineer | ETL & Data Quality

Tech Mirrors; Irving (TX); Sur place; USD 96 000 - 152 000; Plein temps

Résumé du poste

Tech Mirrors in Irving, TX is seeking a PySpark Consultant to design and implement large-scale data pipelines using PySpark, Hadoop, Hive and Kafka. You will translate source PySpark code, ensure data quality and performance, and collaborate with data ingestion and DevOps teams.

The ideal candidate will have 4+ years of big data development experience, strong Python and SQL skills, and proven ability to lead data projects while maintaining data security and governance standards.

Qualifications

• 4+ years in big data development with Spark/Hadoop.
• Proficient in Python, PySpark and SQL.
• Experience with data quality and governance.
• Certification in big data or cloud tech preferred.

Responsabilités

• Design and implement PySpark data pipelines.
• Coordinate with ingestion and DevOps teams.
• Ensure data quality and security compliance.
• Troubleshoot PySpark apps and workflows.

Connaissances

Python

PySpark

SQL

Data Pipelines

DevOps

Communication

Leadership

Analytics

Troubleshooting

Outils

Hadoop

Hive

Kafka

Description du poste

Tech Mirrors in Irving, TX is seeking a PySpark Consultant to design and implement large-scale data pipelines using PySpark, Hadoop, Hive and Kafka. You will translate source PySpark code, ensure data quality and performance, and collaborate with data ingestion and DevOps teams.

The ideal candidate will have 4+ years of big data development experience, strong Python and SQL skills, and proven ability to lead data projects while maintaining data security and governance standards.