PySpark Data Engineer, Data Platform Specialist

IDC Technologies; Santa Clara (CA); On-site; USD 100,000 - 130,000; Full time

Job summary

A technology company in California is seeking experienced Python engineers with a strong background in data engineering. Candidates must be proficient in Python and have knowledge of data structures and algorithms, as well as experience with data platforms like Apache Presto and AWS Athena. The role involves writing data collection scripts and implementing storage technologies. Familiarity with tools such as Docker and Kubernetes is a plus, and candidates will interact with the engineering team during the interview process.

Qualifications

• Strong knowledge of data engineering concepts and practices.
• Excellent problem-solving skills in algorithm development.
• Experience with modern data platforms and workflow tools.

Responsibilities

• Write data collection scripts and aggregation algorithms.
• Implement storage technologies for business intelligence operations.
• Collaborate with existing engineering team during interviews.

Skills

Python

Data Engineering

Data Structures

Algorithms

Tools

Docker

Kubernetes

Apache Airflow

Apache Presto

AWS Athena

Job description

A technology company in California is seeking experienced Python engineers with a strong background in data engineering. Candidates must be proficient in Python and have knowledge of data structures and algorithms, as well as experience with data platforms like Apache Presto and AWS Athena. The role involves writing data collection scripts and implementing storage technologies. Familiarity with tools such as Docker and Kubernetes is a plus, and candidates will interact with the engineering team during the interview process.