#130529 - Software/Data Engineer - Spark, AWS EMR & AI

Upwork; Bogotá ciudad; Presencial; COP 279.520.000 - 372.694.000; Jornada completa

Descripción de la vacante

Upwork is seeking an experienced Software/Data Engineer to design and deliver scalable data processing systems and AI-enabled workflows. This contract role sits at the intersection of software engineering and data engineering, with a strong focus on Spark, cloud-based distributed processing, production reliability, and data preparation for analytics and machine learning use cases.

You will design and develop scalable data processing solutions, build batch and distributed pipelines, and

Formación

• 4+ years of software engineering or data engineering experience.
• Strong experience with Spark and distributed data processing.
• Experience with Amazon EMR or similar cloud-based data processing platforms.
• Proficiency in Java, Python, or a related programming language.
• Exposure to AI or machine learning workflows, model integration, or data preparation for intelligent systems.
• Strong understanding of scalable data architecture and performance optimization.
• Strong debugging and collaboration skills.
• Comfortable delivering in evolving, data-intensive environments.
• Ability to bridge software engineering and data engineering responsibilities.
• Strong execution focus with practical architecture judgment.

Responsabilidades

• Design and develop scalable data processing solutions using Spark and Amazon EMR or comparable cloud-based data processing platforms.
• Build and maintain batch and distributed data pipelines.
• Develop software components for data transformation, feature preparation, and AI or machine learning workflow integration.
• Collaborate with engineering, AI, and product teams to operationalize data-driven and model-enabled use cases.
• Optimize data pipeline performance, cost efficiency, scalability, and production reliability.
• Troubleshoot data and application issues across development and production environments.
• Contribute to architecture discussions, technical documentation, and engineering standards.
• Ensure solutions align with data quality, governance, and security expectations.

Conocimientos

Spark

Distributed processing

Amazon EMR

Java

Python

AI workflows

Data architecture

Debugging

Collaboration

Data security

Herramientas

Airflow

Kafka

AWS

Descripción del empleo

We are seeking an experienced Software/Data Engineer to design and deliver scalable data processing systems and AI-enabled workflows. This contract role sits at the intersection of software engineering and data engineering, with a strong focus on Spark, cloud-based distributed processing, production reliability, and data preparation for analytics and machine learning use cases

Key Responsibilities

• Design and develop scalable data processing solutions using Spark and Amazon EMR or comparable cloud-based data processing platforms.
• Build and maintain batch and distributed data pipelines.
• Develop software components for data transformation, feature preparation, and AI or machine learning workflow integration.
• Collaborate with engineering, AI, and product teams to operationalize data-driven and model-enabled use cases.
• Optimize data pipeline performance, cost efficiency, scalability, and production reliability.
• Troubleshoot data and application issues across development and production environments.
• Contribute to architecture discussions, technical documentation, and engineering standards.
• Ensure solutions align with data quality, governance, and security expectations.

Must-Have Skills

• 4+ years of software engineering or data engineering experience.
• Strong experience with Spark and distributed data processing.
• Experience with Amazon EMR or similar cloud-based data processing platforms.
• Proficiency in Java, Python, or a related programming language.
• Exposure to AI or machine learning workflows, model integration, or data preparation for intelligent systems.
• Strong understanding of scalable data architecture and performance optimization.
• Strong debugging and collaboration skills.
• Comfortable delivering in evolving, data-intensive environments.
• Ability to bridge software engineering and data engineering responsibilities.
• Strong execution focus with practical architecture judgment.

Nice-to-Have Skills

• Experience with Kafka, Airflow, data lakes, or data warehouse ecosystems.
• Familiarity with MLOps, feature stores, or AI platform integration.
• Experience with AWS-native services and observability tooling.
• Enterprise experience strongly preferred.

Required Tools & Platforms

• Apache Spark.
• Amazon EMR or a comparable cloud-based distributed data processing platform.
• Java, Python, or a related programming language.

Location, Time & Engagement

• Remote contract role.
• Candidates must be located in LATAM, excluding Mexico.
• U.S. Central Time coverage is required.
• Full-time allocation of approximately 40 hours per week.
• Current contract end date is March 31, 2027.