Remote Senior Data Engineer — PySpark & Databricks Expert

Truelogic; Colombia; Remote; COP 382,226,000 - 509,635,000; Full time

Benefits offered by this job

100% Remote Work

Highly Competitive USD Pay

Paid Time Off

Autonomy in Work Style

Work with Top American Companies

Job summary

Truelogic is seeking a skilled Data Engineer with 5–7 years of hands-on experience to join a nearshore client’s team. You will design and optimize ETL pipelines using PySpark and Databricks, transforming data and ensuring scalable, cost-efficient workflows across cloud environments.

You will collaborate with data scientists, engineers, and business stakeholders to translate requirements into robust data solutions, uphold data quality, and drive performance improvements for large-scale processing.

Qualifications

• 5–7 years of hands-on data engineering experience building production-grade solutions.
• Core technologies include PySpark and extensive Databricks experience.
• Advanced knowledge of Databricks AutoLoader, Python, performance tuning, and compute cost optimization.
• Deep understanding of end-to-end ETL processes and complex data transformations.
• Strong ability to communicate technical concepts clearly across teams.

Responsibilities

• Design, build, and maintain scalable ETL pipelines for large-scale data processing.
• Work with PySpark and Databricks to execute complex data transformations and workflows.
• Fine-tune data pipelines to maximize performance, scalability, and cloud cost-efficiency.
• Coordinate with engineering and business teams to translate requirements into data solutions.
• Debug, resolve, and prevent data processing bottlenecks and maintain data quality.

Skills

ETL architecture

Data transformations

Performance optimization

Cross-functional communication

Problem-solving

Tools

PySpark

Databricks

Python

Databricks AutoLoader

Job description

Truelogic is seeking a skilled Data Engineer with 5–7 years of hands-on experience to join a nearshore client’s team. You will design and optimize ETL pipelines using PySpark and Databricks, transforming data and ensuring scalable, cost-efficient workflows across cloud environments.

You will collaborate with data scientists, engineers, and business stakeholders to translate requirements into robust data solutions, uphold data quality, and drive performance improvements for large-scale processing.