Senior Data Engineer - Remote, PySpark & Databricks

OpenKrill; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Remote work

Competitive USD pay

Paid time off

Autonomy

Work with American companies

Job summary

TrueLogic is seeking a skilled Data Engineer to join our client’s team. You will design, build, and optimize ETL pipelines, with hands-on work in PySpark and Databricks, tackling large-scale data and performance challenges.

Ideal candidates have 5–7 years of data engineering experience, strong Python skills, and a proven ability to communicate across technical and business stakeholders while improving data quality and governance.

Qualifications

• 5–7 years of hands-on data engineering experience building production-grade solutions.
• Strong proficiency in PySpark and extensive hands-on experience with Databricks.
• Advanced knowledge of Databricks AutoLoader, Python, best practices, performance tuning, and compute cost optimization.
• Data Architecture: End‑to‑end ETL processes, complex data transformations, and pipeline optimization.
• Core Competencies: Excellent problem-solving and ability to communicate technical concepts clearly across technical and business teams.

Responsibilities

• Design, build, and maintain scalable ETL pipelines for large-scale data processing.
• Work extensively with PySpark and Databricks to execute complex data transformations and workflows.
• Fine-tune data pipelines to maximize performance, scalability, and cloud cost-efficiency.
• Collaborate with engineering and business teams to translate requirements into high-quality data solutions.
• Debug, resolve, and prevent complex data processing and pipeline bottlenecks.
• Enforce data quality, governance, and system performance standards.

Skills

Problem solving

Communication

Tools

PySpark

Databricks

Python

Databricks AutoLoader

Job description

TrueLogic is seeking a skilled Data Engineer to join our client’s team. You will design, build, and optimize ETL pipelines, with hands-on work in PySpark and Databricks, tackling large-scale data and performance challenges.

Ideal candidates have 5–7 years of data engineering experience, strong Python skills, and a proven ability to communicate across technical and business stakeholders while improving data quality and governance.