Mid Data Engineer

RemoDevs; United States; Remote; USD 110,000 - 150,000; Full time

Job summary

RemoDevs is seeking a Mid Data Engineer to develop and maintain data pipelines using PySpark, with hands-on Databricks experience. The role focuses on large-scale data processing, optimization and reliable data delivery in a fast-paced project environment.

You will work on building ETL/ELT data pipelines, transforming data, and writing efficient SQL while collaborating with a team of engineers and stakeholders across data platforms.

Qualifications

• Hands-on PySpark experience required.
• Commercial data engineering experience, mid-level.
• Strong SQL and data transformation skills.
• Experience with ETL/ELT pipelines.
• Familiarity with cloud environments (Azure/AWS/GCP).

Responsibilities

• Develop and maintain data pipelines using PySpark and Apache Spark.
• Build and optimize data transformation processes for large datasets.
• Integrate data from multiple sources and ensure reliable data processing.
• Write and optimize SQL queries.
• Work with Databricks to run data processing workloads.
• Troubleshoot pipelines and improve performance.
• Implement data quality checks and reliable data delivery.
• Collaborate with engineers and stakeholders on data tasks.

Skills

PySpark

SQL

Python

ETL/ELT

Data pipelines

Distributed data processing

Tools

Databricks

Delta Lake

Azure

AWS

GCP

Job description

We are looking for a Mid Data Engineer with strong PySpark skills and basic hands-on experience with Databricks to join a data engineering project for an initial period of 3–6 months.The role focuses on developing, maintaining and optimizing data pipelines, working with large datasets and supporting modern data processing solutions.

What you'll do

• Develop and maintain data pipelines using PySpark and Apache Spark.
• Build and optimize data transformation processes for large datasets.
• Integrate data from different sources and ensure reliable data processing.
• Write, maintain and optimize SQL queries.
• Work with Databricks to develop and execute data processing workloads.
• Troubleshoot pipeline issues and improve processing performance.
• Implement data quality checks and support reliable data delivery.
• Collaborate with other engineers and technical stakeholders on data engineering tasks.

Your experience

• Strong, hands-on experience with PySpark — this is the key requirement.
• Commercial experience in Data Engineering, ideally at a mid-level.
• Very good knowledge of SQL and data transformation techniques.
• Practical understanding of Apache Spark and distributed data processing.
• Basic hands-on experience with Databricks — you don't need to be an expert.
• Experience building and maintaining ETL/ELT data pipelines.
• Familiarity with cloud environments (Azure, AWS or GCP).
• Ability to work independently on assigned tasks and solve technical problems.

Nice to have

• Experience with Delta Lake.
• Familiarity with workflow orchestration tools.
• Knowledge of data quality and monitoring practices.
• Experience optimizing Spark jobs and working with large-scale datasets.
• Understanding of lakehouse architecture.

Tech stack

• PySpark, Apache Spark, SQL, Databricks, PythonAdditional technologies: Delta Lake, Azure / AWS / GCP.

Project details

Position: Mid Data EngineerProject duration: 3–6 monthsMain focus: PySpark development and data pipeline engineeringDatabricks: Basic practical experience required

Why join?

You'll have the opportunity to work on a hands-on data engineering project, focusing on real-world PySpark development, data pipeline optimization and modern data processing technologies.This is a great opportunity for an experienced PySpark developer who wants to further develop their skills while gaining additional exposure to Databricks.