Pyspark developer (Data engineer)

MTK Technologies; Irving (TX); On-site; USD 90,000 - 120,000; Full time

Job summary

A technology company in Texas is seeking a Senior PySpark Developer / Data Engineer to design and maintain data solutions in a Big Data environment. Key responsibilities include creating data pipelines and implementing ETL processes. Candidates should have strong experience in PySpark and Python, excellent collaboration skills, and a commitment to data quality. This role offers an opportunity to innovate and leverage new advancements in data engineering.

Qualifications

• Experience with PySpark/Python in a Big Data environment.
• Strong ability to design and implement ETL processes and data pipelines.
• Familiarity with data quality concepts.

Responsibilities

• Design, develop, and maintain data solutions for data processing using PySpark.
• Create data pipelines, ensure data quality, and implement ETL processes.
• Collaborate with cross-functional teams to resolve data-related issues.

Skills

PySpark

Python

ETL processes

Data quality

Cross-team collaboration

Job description

Role

Senior PySpark Developer / Data Engineer

Job Description

As a Data Engineer, you will be responsible for designing, developing, and maintaining data solutions for data generation, collection, and processing in a Big Data environment using predominantly PySpark/Python. Your typical day will involve creating data pipelines, ensuring data quality, and implementing ETL processes to migrate and deploy data across systems using PySpark.

Responsibilities

• Design, develop, and maintain data solutions for data generation, collection, and processing using PySpark.
• Create data pipelines, ensure data quality, and implement ETL processes to migrate and deploy data across systems.
• Collaborate with cross-functional teams to identify and resolve data-related issues.
• Stay updated with the latest advancements in data engineering and integrate innovative approaches for sustained competitive advantage.

Qualifications

• Experience with PySpark/Python in a Big Data environment.
• Strong ability to design and implement ETL processes and data pipelines.
• Familiarity with data quality concepts and cross-team collaboration.
• Willingness to stay updated with industry advancements and apply innovative approaches.

Additional Information

For context, the listing contains related employer information and mentions of other roles, which are not part of the current job duties. The description focuses on the Senior PySpark Developer / Data Engineer responsibilities and requirements.