Data Engineer Senior

Grupo Giga; A distancia; MXN 1,457,000 - 2,005,000; Jornada completa

Descripción de la vacante

A dynamic technology company in Veracruz is seeking a Senior Data Engineer to work with modern technologies for data ingestion, storage, and transformation. You'll develop software, utilize tools like Kafka and Databricks, and support data governance. The role promises opportunities for professional growth in a community-oriented environment. Ideal candidates will demonstrate strong skills in SQL and Python, alongside distributed data processing experiences. Remote work is available.

Servicios

Opportunities for professional development

Flexible work hours

15 business days PTO

Referral Program

Friendly working culture

Responsabilidades

• Software development focused on data ingestion, storage, transformation, and orchestration.
• Utilize Autoloader and Kafka for data ingestion.
• Perform data transformation using PySpark and Structured Streaming.

Conocimientos

SQL Server knowledge

Python knowledge

GitHub knowledge

Distributed data processing with Spark

Structured Streaming experience

Databricks Live Tables

Kafka / Azure EventHub experience

Proficiency in Python

Strong knowledge of SQL

English proficiency

Descripción del empleo

Summary

We are looking for a new GIGAnte to join our team as a Data Engineer Senior.

You will have the opportunity to work with modern technologies for everything related to data ingestion, storage, transformation, and orchestration.

What You'll Do

• Software development
• Utilize Autoloader and Kafka for data ingestion
• Work with ADLS for data storage
• Perform data transformation using PySpark and Structured Streaming
• Manage workflow orchestration with Databricks Workflows
• Handle data import / export using Kafka in Azure EventHub
• Implement data governance using Unity Catalog with RBAC and data quality tools like Great Expectations

We'll Expect You To Have

• SQL Server knowledge
• Python knowledge
• GitHub knowledge
• Solid understanding of distributed data processing with Spark
• Experience with Structured Streaming, Autoloader, UDFs, pandas
• Experience with Databricks Live Tables
• Experience with Databricks Asset Bundles
• Experience with Kafka / Azure EventHub
• Proficiency in Python
• Strong knowledge of SQL
• English

What we offer

• ?? An exciting growth journey and a never-ending ambition to develop as a company
• ?? A friendly, supportive and humble working culture where the team always comes first
• ?? Opportunities for professional development and personal growth
• ?? Passionate, high-performing and genuine colleagues
• ???? We care about the important things : results and well-being
• ?? Referral Program – Refer a colleague program : Enjoy an amazing reward if you recommend a friend or colleague to GIGA IT and they get hired
• ??? 15 business days PTO and flexibility program
• ?? Local holidays
• ?? Remote work – You can work from anywhere!
• ?????? Notebook
• ?? Work hours : Monday to Friday, from 8 a.m. to 5 p.m.
• ?? To be agreed

About GIGA IT

We believe that our differential value lies in our human capital, building a long-lasting relationship beyond mere deliverables.

You will be part of a Latin American company with expert teams that can enrich your knowledge.

Thanks to our flexible way of working, we leverage the talent of our collaborators located in different regions of the world.

Core Values

Trustworthy : We build trusted and rightful relations towards our external and internal clients

Adaptable : We are flexible to adapt ourselves to complex environments.

Passionate : We love what we do.

Committed : We commit to act with accountability and efficiency at every step of the way.

??Do you feel this is an exciting challenge for you?

Then, don't hesitate to apply.

We're eager to meet you and grow together??