Senior Data Engineer: Spark, EMR & AI Workflows (Remote)

AgileEngine, LLC.; Metropolitana; Presencial; COP 283.733.000 - 378.310.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

Flexibility — 100% remote

Meaningful, modern projects

Collaborative culture

Well-being & support

Descripción de la vacante

AgileEngine is seeking a Senior Software Engineer to design scalable data processing systems using Spark and EMR. You will build batch and distributed pipelines, collaborating with AI and product teams to operationalize model-enabled use cases and prepare data for intelligent workflows.

The role emphasizes a strong architecture sense, data quality, governance, and production reliability in data-intensive environments.

Formación

• 4+ years of software or data engineering experience.
• Strong experience with Spark and distributed data processing.
• Experience with Amazon EMR or similar cloud platforms.
• Proficiency in Java, Python, or related languages.
• Exposure to AI or machine learning workflows and data preparation.

Responsabilidades

• Design scalable data processing solutions using Spark and EMR.
• Build and maintain batch and distributed data pipelines.
• Collaborate with engineering, AI, and product teams to operationalize data-driven use cases.
• Develop components for data transformation and AI workflow integration.
• Optimize pipeline performance, cost efficiency, and reliability.
• Troubleshoot data and application issues across environments.
• Contribute to architecture discussions, documentation, and standards.
• Ensure data quality, governance, and security alignment.

Conocimientos

4+ years experience

Spark & distributed processing

Java or Python

AI/ML workflow exposure

Data architecture

Debugging & collaboration

Execution focus

Herramientas

Amazon EMR

Kafka

Airflow

AWS services

Descripción del empleo

AgileEngine is seeking a Senior Software Engineer to design scalable data processing systems using Spark and EMR. You will build batch and distributed pipelines, collaborating with AI and product teams to operationalize model-enabled use cases and prepare data for intelligent workflows.

The role emphasizes a strong architecture sense, data quality, governance, and production reliability in data-intensive environments.