Senior Data Engineer

Insight Global

México

A distancia

MXN 640.148 - 819.390

Jornada completa

14 días+

Recibe más respuestas de empleadores

Envía un currículum específico para el puesto de trabajo en cuestión de minutos.

Descripción de la vacante

A leading staffing agency is seeking a Lead Data Engineer for a fully remote contractor position focusing on AI-driven forecasting initiatives. The role involves designing data transformation jobs using Databricks and managing data workflows with Azure. Candidates should possess expertise in Databricks, Apache Spark, and Python, along with a solid understanding of data lake architecture. This position offers competitive hourly rates ranging from $25 to $32.

Formación

Expertise in databricks, Apache Spark, and Python.
Experience with cloud platforms (Azure preferred).
Solid understanding of data lake architecture and data modeling techniques.

Responsabilidades

Design and develop data transformation jobs using PySpark and Databricks.
Build and manage data orchestration workflows.
Collaborate with data analysts to ensure functional logic is accurately implemented.

Conocimientos

Databricks

Apache Spark

Python

Azure

Data Lake Architecture

Data Warehousing

Data Modeling

Herramientas

Azure Data Factory

Synapse Analytics

Teradata

Descripción del empleo

Get AI-powered advice on this job and more exclusive features.

This position is fully remote, a contractor position and salary range is 25-32/hr.

Required Skills & Experience

• Expertise in Databricks, Apache Spark, and Python.

• Experience with cloud platforms (Azure preferred; AWS or GCP acceptable).

• Solid understanding of data lake architecture, data warehousing, and data modeling techniques.

Nice to Have Skills & Experience

• Proven track record in data engineering roles within agile environments.

• Ability to manage technical deliverables and ensure data quality across the pipeline.

• Familiarity with AI/ML workflows and data preparation for model training.

Job Description

A client is looking for hands-on Lead Data Engineer to support AI-driven forecasting initiatives within their finance domain. This role involves working with modern cloud technologies, orchestrating data workflows, and collaborating with data science teams to ensure high-quality data delivery. They will be supporting our platform, Mosaic which delivers a systemized capability for the complete planning & forecasting process by each customer, product, and business unit across the sector, as well as reporting analytics for actuals. They will use Databricks to gather, cleanse, transform and analyze large datasets in preparation for data scientists to build data models with. They will then use Synapse Analytics to integrates big data and data warehousing to query structured and unstructured data as well as Teradata to handle enterprise-level data warehousing with SQL-based transformations. Involved in stakeholder management, guiding them with data driven insights.

Key Responsibilities:

• Design and develop data transformation jobs using PySpark and Databricks.

• Build and manage data orchestration workflows using Azure Data Factory or equivalent tools.

• Integrate data from multiple sources and prepare it for AI model consumption.

• Collaborate with data analysts and data scientists to ensure functional logic is accurately implemented.

• Maintain and optimize data lakes, warehouses, and cloud-based data platforms.

Seniority level