Data Engineer

MACHINE LEARNING TECHNOLOGIES LLC; Plano (TX); On-site; USD 118,000 - 136,000; Full time

Job summary

Machine Learning Technologies LLC in Plano, TX is seeking a Data Engineer to design, develop, and optimize data pipelines using Databricks and Azure Data Factory. You will build ETL/ELT processes, write SQL queries, and work with PySpark to handle large-scale data.

The role focuses on data quality, reliability, and collaboration with analytics teams to deliver robust data assets for reporting and analytics in a cloud-based environment.

Qualifications

• ETL/ELT concepts and data integration expertise.
• SQL proficiency for data extraction and transformation.
• Basic Python or PySpark for large-scale data processing.

Responsibilities

• Develop and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
• Work with Databricks for data processing and transformation.
• Write and optimize SQL queries for data extraction and transformation.
• Utilize PySpark or Python for large-scale data processing tasks.
• Integrate data from multiple sources into data warehouses or data lakes.
• Monitor, troubleshoot, and optimize data pipelines.
• Collaborate with teams to ensure data quality and reliability.

Tools

Databricks

Azure Data Factory (ADF)

Job description

Job Overview

Job ID: J55066

Job Title: Data Engineer

Location: Plano, TX

Duration: 12 Months + Extension

Domain Exposure: Utilities/ Energy

Client: To Be Discussed Later

Employment Type: W-2, 1099, C2C

Work Authorization: US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other

Bill Rate

$86/hour (C2C)

Job Description

We are looking for a Data Engineer with experience in Databricks and Azure Data Factory (ADF) to develop, maintain, and optimize data pipelines and ETL processes.

Responsibilities

• Develop and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
• Work with Databricks for data processing and transformation.
• Write and optimize SQL queries for data extraction and transformation.
• Work with PySpark or Python for large-scale data processing.
• Integrate data from multiple sources into data warehouses or data lakes.
• Monitor, troubleshoot, and optimize data pipelines.
• Collaborate with teams to ensure data quality and reliability.

Required Skills

• Experience with Databricks and Azure Data Factory (ADF).
• Strong understanding of ETL/ELT concepts.
• Good knowledge of SQL and data transformations.
• Basic experience with Python or PySpark.
• Understanding of Azure cloud services and data lake concepts.

Equal Opportunity Employer

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.