Senior PySpark Data Engineer: ETL & Scalable Pipelines

Covetus; Irving (TX); On-site; USD 100,000 - 130,000; Full time

Job summary

Covetus, located in Irving, Texas, is seeking a Data Engineer who will be responsible for designing, developing, and maintaining data solutions in a Big Data environment using predominantly PySpark/Python. The role involves creating data pipelines, ensuring data quality, and implementing ETL processes across systems.

The ideal candidate should have over 8 years of experience in Hadoop and PySpark, with substantial exposure to AWS and Snowflake. Strong collaboration skills and problem-solving abilities are essential for success in this role.

Qualifications

• 8+ years of professional experience in Hadoop, PySpark/Python development.
• Proven expertise in PySpark and experience in handling huge volumes of data.
• 3+ years of working experience in AWS, Databricks/Snowflake, Airflow.

Responsibilities

• Design, develop, and maintain robust, scalable high-performance Data Pipelines using PySpark.
• Create data pipelines, ensuring data quality, and implement ETL processes to migrate and deploy data across systems.
• Migrate Ab Initio ETL Applications into PySpark based data pipelines.

Skills

Hadoop

PySpark/Python

AWS

Databricks/Snowflake

CI/CD

Git

Debugging

Communication

Job description

Covetus, located in Irving, Texas, is seeking a Data Engineer who will be responsible for designing, developing, and maintaining data solutions in a Big Data environment using predominantly PySpark/Python. The role involves creating data pipelines, ensuring data quality, and implementing ETL processes across systems.

The ideal candidate should have over 8 years of experience in Hadoop and PySpark, with substantial exposure to AWS and Snowflake. Strong collaboration skills and problem-solving abilities are essential for success in this role.