Senior PySpark Data Engineer | ETL & Data Quality

Tech Mirrors; Irving (TX); On-site; USD 96,000 - 152,000; Full time

Job summary

Tech Mirrors in Irving, TX is seeking a PySpark Consultant to design and implement large-scale data pipelines using PySpark, Hadoop, Hive and Kafka. You will translate source PySpark code, ensure data quality and performance, and collaborate with data ingestion and DevOps teams.

The ideal candidate will have 4+ years of big data development experience, strong Python and SQL skills, and proven ability to lead data projects while maintaining data security and governance standards.

Qualifications

• 4+ years in big data development with Spark/Hadoop.
• Proficient in Python, PySpark and SQL.
• Experience with data quality and governance.
• Certification in big data or cloud tech preferred.

Responsibilities

• Design and implement PySpark data pipelines.
• Coordinate with ingestion and DevOps teams.
• Ensure data quality and security compliance.
• Troubleshoot PySpark apps and workflows.

Skills

Python

PySpark

SQL

Data Pipelines

DevOps

Communication

Leadership

Analytics

Troubleshooting

Tools

Hadoop

Hive

Kafka

Job description

Tech Mirrors in Irving, TX is seeking a PySpark Consultant to design and implement large-scale data pipelines using PySpark, Hadoop, Hive and Kafka. You will translate source PySpark code, ensure data quality and performance, and collaborate with data ingestion and DevOps teams.

The ideal candidate will have 4+ years of big data development experience, strong Python and SQL skills, and proven ability to lead data projects while maintaining data security and governance standards.