Senior Data Engineer - Spark, EMR & AI Pipelines (Remote)

AgileEngine; Colombia; Presencial; COP 372.694.000 - 559.041.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

Flexibility: 100% remote with flexible

Meaningful, modern projects

Collaborative culture

Well-being & support

Descripción de la vacante

AgileEngine is seeking a Senior Software Engineer to design scalable data processing systems using Spark and EMR. You will build batch and distributed pipelines and work with AI and product teams to operationalize model-enabled use cases, ensuring data quality and performance.

Ideal candidates have 4+ years in software or data engineering, experience with Spark/EMR, and proficiency in Java or Python, plus exposure to ML workflows.

Formación

• 4+ years of software or data engineering experience.
• Strong experience with Spark and distributed data processing.
• Experience with Amazon EMR or similar cloud platforms.
• Proficiency in Java, Python, or related programming languages.
• Exposure to AI or ML workflows and data preparation for intelligent systems.
• Strong understanding of scalable data architecture and performance optimization.
• Strong debugging and collaboration skills.
• Upper-intermediate English.

Responsabilidades

• Design and develop scalable data processing solutions using Spark and EMR.
• Build and maintain batch and distributed data pipelines.
• Collaborate with engineering, AI, and product teams on model-enabled use cases.
• Develop software components for data transformation and AI workflow integration.
• Optimize pipeline performance, cost efficiency, and reliability.
• Troubleshoot data and application issues across development and production environments.
• Contribute to architecture discussions, documentation, and engineering standards.
• Ensure solutions align with data quality, governance, and security expectations.

Conocimientos

4+ years exp

Spark & distributed processing

AWS EMR

Java / Python

ML workflows exposure

Data architecture & optimization

Debugging & collaboration

English: upper-intermediate

Herramientas

Spark

EMR

Java

Python

AWS

Descripción del empleo

AgileEngine is seeking a Senior Software Engineer to design scalable data processing systems using Spark and EMR. You will build batch and distributed pipelines and work with AI and product teams to operationalize model-enabled use cases, ensuring data quality and performance.

Ideal candidates have 4+ years in software or data engineering, experience with Spark/EMR, and proficiency in Java or Python, plus exposure to ML workflows.