Senior Databricks Engineer: Data Pipelines & Insights Lead

Infinite Computer Solutions; Colombia; Presencial; COP 90.000.000 - 130.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Health insurance

Descripción de la vacante

Infinite Computer Solutions is seeking a Senior Data Engineer to define data requirements, build and optimize pipelines, and collaborate with data science and BI teams in Colombia. You will mentor junior engineers and drive data quality and analytics initiatives.

The role focuses on scalable data infrastructure, cloud platforms (AWS/GCP), and strong SQL/Python skills to enable insights and machine learning workflows.

Formación

• 7+ years of IT experience in data engineering/ETL roles.
• 5+ years in data engineering/ETL handling large datasets.
• 3+ years with Big Data tools (Hadoop, Spark, Kafka, Hive, S3) or cloud platforms (AWS/GCP).
• Databricks UI, Delta Lake, Python/Spark SQL proficiency.
• High-velocity streaming with Kafka and Spark Streaming.

Responsabilidades

• Define data requirements, wrangle large structured and unstructured data.
• Ingest, analyze, validate, normalize and clean data for BI and ML use.
• Develop data policies and retention models; anonymize data as needed.
• Implement data quality procedures and support analytics teams.
• Design and support scalable data infrastructure and pipelines.
• Collaborate with data science and BI teams on models and reports.
• Mentor junior data engineers and lead data engineering projects.
• Present insights and strategic recommendations to leadership.

Conocimientos

SQL

Python

PySpark

Spark SQL

Databricks

Kafka

ETL Development

Data Pipelines

Cloud AWS

Hadoop

GIT

Delta Lake

Spark Streaming

SQL Optimization

Tibco

IBM DataStage

Educación

Bachelor’s Degree

Herramientas

Databricks

Hadoop

Spark

Kafka

IBM DataStage

Delta Lake

Teradata

BigQuery

AWS

GCP

S3

Kinesis

Glue

TIBCO EMS

IBM MQ

Descripción del empleo

Infinite Computer Solutions is seeking a Senior Data Engineer to define data requirements, build and optimize pipelines, and collaborate with data science and BI teams in Colombia. You will mentor junior engineers and drive data quality and analytics initiatives.

The role focuses on scalable data infrastructure, cloud platforms (AWS/GCP), and strong SQL/Python skills to enable insights and machine learning workflows.