Big Data Engineer — PySpark, Kafka, Onsite Colombia

Tata Consultancy Services; Colombia; Presencial; COP 149.902.563 - 299.805.126; Jornada completa

Descripción de la vacante

Tata Consultancy Services is seeking a Big Data Engineer to join the technology team in Colombia. This position entails designing, developing, and optimizing large-scale data processing solutions using modern Big Data technologies like PySpark and Scala. Key responsibilities include implementing real-time data ingestion pipelines, collaborating with global clients, and ensuring data quality across platforms. Candidates should have 2 to 15 years of experience in Big Data engineering and proficiency in English and Spanish.

Formación

• 2 to 15 years of experience as a Big Data Engineer.
• Expertise in Big Data engineering and distributed data solutions.
• Proven hands-on experience with PySpark and Scala.
• Strong experience working with Kafka and other Big Data technologies.
• Solid understanding of distributed systems, ETL processes, and data engineering best practices.
• Experience with relational and/or NoSQL databases.
• Advanced English proficiency (spoken and written).
• Bilingual proficiency in English and Spanish.
• Strong communication skills and ability to work with global stakeholders.

Responsabilidades

• Develop batch and streaming solutions using PySpark and Scala.
• Implement and manage real-time data ingestion pipelines using Kafka.
• Optimize data workflows for performance, scalability, and reliability.
• Collaborate with cross-functional teams to translate business requirements into technical solutions.
• Ensure data quality, integrity, and consistency across data platforms.
• Participate in code reviews, troubleshooting, and continuous improvement initiatives.
• Actively contribute in Agile environments (Scrum / Kanban).

Conocimientos

Big Data engineering

PySpark

Scala

Kafka

Distributed systems

ETL processes

Relational databases

NoSQL databases

Advanced English proficiency

Bilingual in English and Spanish

Communication skills

Descripción del empleo

Tata Consultancy Services is seeking a Big Data Engineer to join the technology team in Colombia. This position entails designing, developing, and optimizing large-scale data processing solutions using modern Big Data technologies like PySpark and Scala. Key responsibilities include implementing real-time data ingestion pipelines, collaborating with global clients, and ensuring data quality across platforms. Candidates should have 2 to 15 years of experience in Big Data engineering and proficiency in English and Spanish.