Senior Data Engineer: Spark, EMR & AI Workflows (Remote)

AgileEngine, LLC.; Pereira; Presencial; COP 120.000.000 - 180.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

Flexibility: remote work

Descripción de la vacante

AgileEngine is seeking a Senior Software Engineer to design scalable data processing systems using Spark and EMR. You will build batch and distributed pipelines and collaborate with AI and product teams to operationalize model-enabled use cases.

You should have 4+ years of software or data engineering experience, strong knowledge of Spark and AWS EMR, and proficiency in Java or Python. This role offers fully remote work with flexible hours and the chance to work with global brands.

Formación

• 4+ years of software or data engineering experience.
• Strong experience with Spark and distributed data processing.
• Experience with Amazon EMR or similar cloud-based data processing platforms.
• Proficiency in Java or Python.
• Exposure to AI or machine learning workflows, model integration, or data preparation for intelligent systems.
• Strong understanding of scalable data architecture and performance optimization.
• Strong debugging and collaboration skills.
• Able to bridge software engineering and data engineering responsibilities.

Responsabilidades

• Design and develop scalable data processing solutions using Spark and EMR.
• Build and maintain batch and distributed data pipelines.
• Collaborate with engineering, AI, and product teams to operationalize data-driven and model-enabled use cases.
• Develop software components for data transformation, feature preparation, and AI workflow integration.
• Optimize pipeline performance, cost efficiency, and production reliability.
• Troubleshoot data and application issues across development and production environments.
• Contribute to architecture discussions, documentation, and engineering standards.
• Ensure solutions align with data quality, governance, and security expectations.

Conocimientos

4+ years of software/data engineering

Spark and distributed data processing

AI workflows / model integration

Data architecture & performance tuning

Debugging

Cross-functional collaboration

Execution-focused

Educación

Bachelor’s degree in a relevant field

Herramientas

Java

Python

Spark

Amazon EMR

Kafka

Airflow

AWS observability tooling

Descripción del empleo

AgileEngine is seeking a Senior Software Engineer to design scalable data processing systems using Spark and EMR. You will build batch and distributed pipelines and collaborate with AI and product teams to operationalize model-enabled use cases.

You should have 4+ years of software or data engineering experience, strong knowledge of Spark and AWS EMR, and proficiency in Java or Python. This role offers fully remote work with flexible hours and the chance to work with global brands.