Remote Azure Data Engineer - Real-Time Pipelines

Turtle Trax S.A.; United States; À distance; USD 90 000 - 140 000; Plein temps

Avantages offerts par ce poste

Remote work

Résumé du poste

Turtle Trax S.A. is seeking a Data Engineer to design, build, and maintain scalable, cloud-native data pipelines and infrastructure that power analytics, reporting, BI, and real-time processing.

You will work with stakeholders to deliver high-quality data solutions using modern cloud, big data, and API-driven technologies. The role involves building ETL/ELT pipelines, data models, and lakehouse architectures, integrating RESTful APIs, and ensuring data quality, security, and compliance across

Qualifications

• Strong proficiency in SQL and relational databases such as Oracle, SQL Server, and MySQL.
• Strong programming skills in Python, PySpark, PL/SQL, Java, or Scala.
• Hands-on experience with Azure Cloud technologies: Azure Data Factory (ADF), Azure Databricks, ADLS, Synapse Analytics, Event Hubs, Functions, API Management, DevOps.
• Experience with Databricks Lakehouse architecture, Delta Lake, and Delta Tables.
• Expertise in API development, RESTful services, and microservices architecture.
• Experience processing high-volume and high-frequency data with low-latency requirements.
• Strong knowledge of real-time data ingestion and streaming technologies such as Kafka, Azure Event Hubs, or Kinesis.
• Experience with Spark, Hadoop, and distributed data processing frameworks.
• Hands-on experience with OpenShift, Kubernetes, Docker, and containerized deployments.
• Experience with workflow orchestration tools such as Apache Airflow and Azure Data Factory.
• Understanding of data governance, data security, and compliance best practices.

Responsabilités

• Design, develop, and maintain scalable ETL/ELT pipelines for batch and real-time data processing.
• Build and optimize data models, Delta Tables, and Lakehouse architectures to support analytics and reporting.
• Develop and integrate RESTful APIs and data services to facilitate seamless data exchange across enterprise systems.
• Implement real-time and high-frequency data ingestion frameworks using streaming technologies and event-driven architectures.
• Design and manage cloud-native data solutions leveraging Azure services including Data Factory, Databricks, ADLS, Event Hubs, and Synapse Analytics.
• Develop and optimize Databricks Spark applications for large-scale data transformation and processing.
• Ensure data quality, governance, security, and compliance across data platforms.
• Collaborate with data scientists, analysts, application teams, and business stakeholders to deliver scalable data solutions.
• Troubleshoot, monitor, and optimize pipeline performance and data platform reliability.
• Support DataOps and CI/CD practices for data pipeline deployment and automation.

Connaissances

Python

SQL

PySpark

Azure Cloud

REST APIs

Outils

Azure Data Factory

Azure Databricks

Delta Lake

Kafka

OpenShift

Kubernetes

Airflow

Description du poste

Turtle Trax S.A. is seeking a Data Engineer to design, build, and maintain scalable, cloud-native data pipelines and infrastructure that power analytics, reporting, BI, and real-time processing.

You will work with stakeholders to deliver high-quality data solutions using modern cloud, big data, and API-driven technologies. The role involves building ETL/ELT pipelines, data models, and lakehouse architectures, integrating RESTful APIs, and ensuring data quality, security, and compliance across