Senior Software Engineer - Spark Pipelines (Remote)

AgileEngine; Perímetro Urbano Barranquilla; Presencial; COP 80.000.000 - 120.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth opportunities

Competitive pay

Remote work / flexible hours

Impactful projects

Collaborative culture

Well-being programs

Descripción de la vacante

AgileEngine is seeking a Senior Software Engineer to design scalable data processing systems using Spark and EMR. You will build batch and distributed pipelines, collaborating with AI and product teams to operationalize data-driven use cases and prepare data for intelligent workflows.

Seeking 4+ years of experience with Spark, EMR, and cloud platforms; strong Java or Python skills and an eye for architecture.

Formación

• 4+ years of software or data engineering experience.
• Strong experience with Spark and distributed data processing.
• Experience with Amazon EMR or similar cloud platforms.
• Proficiency in Java, Python, or related languages.
• Exposure to AI or ML workflows and data preparation.
• Strong architecture sense and optimization skills.
• Upper-intermediate English, strong collaboration abilities.

Responsabilidades

• Design and develop scalable data processing solutions with Spark and EMR.
• Build and maintain batch and distributed data pipelines.
• Collaborate with engineering, AI, and product teams on data-enabled use cases.
• Develop components for data transformation and AI workflow integration.
• Optimize pipeline performance, cost, and reliability.
• Troubleshoot data and application issues across environments.
• Contribute to architecture decisions and engineering standards.

Conocimientos

Spark

Distributed data processing

Java

Python

English communication

Debugging

Collaboration

Data architecture

Herramientas

Amazon EMR

Kafka

Airflow

AWS services

Data processing frameworks

Descripción del empleo

AgileEngine is seeking a Senior Software Engineer to design scalable data processing systems using Spark and EMR. You will build batch and distributed pipelines, collaborating with AI and product teams to operationalize data-driven use cases and prepare data for intelligent workflows.

Seeking 4+ years of experience with Spark, EMR, and cloud platforms; strong Java or Python skills and an eye for architecture.