PySpark Database Developer: JSON Data Pipelines

Supersourcing; Oregon (WI); Hybrid; USD 90,000 - 120,000; Full time

Job summary

A leading technology company is seeking a talented Database Developer to design and develop innovative data processing solutions using PySpark. You will play a crucial role in building a custom query builder and collaborating with teams to create efficient, scalable solutions for JSON data processing. Candidates should have strong Python programming skills and extensive experience with Apache Spark. Join a dynamic team and contribute to cutting-edge data solutions.

Qualifications

• Strong proficiency in Python programming.
• Extensive experience with PySpark and Apache Spark.
• Solid understanding of JSON data structures and SQL.

Responsibilities

• Design and develop a custom query builder for JSON data processing.
• Collaborate with cross-functional teams to meet business requirements.
• Develop and maintain data pipelines and workflows.

Skills

JSON data processing

PySpark

Python programming

SQL

Data processing best practices

Tools

Apache Spark

Job description

About the job Database Developer - Pyspark

We are seeking a talented Database Developer with expertise in JSON data processing and PySpark to join our team. The ideal candidate will play a crucial role in designing and developing a custom query builder for efficient JSON data processing using PySpark.

This is a fantastic opportunity to work with cutting-edge technologies and contribute to the development of innovative data processing solutions. As a Database Developer, you will collaborate with cross-functional teams, including data scientists and analysts, to understand business requirements and translate them into efficient and scalable solutions. You will be responsible for designing and implementing data models and database schemas for optimal storage and retrieval of JSON data.

Additionally, you will develop and maintain data pipelines and workflows, ensuring the accuracy and reliability of processed data through validation and quality checks. The successful candidate will have a strong proficiency in Python programming language and extensive experience with PySpark and Apache Spark for data processing.

A solid understanding of JSON data structures, SQL, and database systems is also required. Experience with distributed computing frameworks and knowledge of data processing best practices will be highly beneficial.

If you are passionate about building custom query builders, leveraging PySpark for JSON data processing, and thrive in a collaborative and innovative environment, we would love to hear from you.

Join our dynamic team and contribute to the development of cutting-edge data processing solutions.