Data Engineer

Machine Learning Technologies, LLC; Austin (TX); In loco; USD 120.000 - 160.000; Tempo pieno

Descrizione del lavoro

Machine Learning Technologies, LLC is seeking Data Engineers with hands-on GCP data engineering expertise to design, build, and maintain production data pipelines. The project involves migrating from Azure to GCP and optimizing data processing at scale.

You will work with BigQuery, Dataflow, Cloud Storage, and ETL/ELT processes, using Python and SQL, PySpark, and Airflow/Cloud Composer to ensure data quality and performance across the enterprise data platform.

Competenze

• 6+ years of experience in data engineering, data integration, and data pipeline development.
• Strong hands-on GCP data engineering experience is mandatory.
• Experience with BigQuery, Dataflow, Cloud Storage, and relevant GCP data services.
• Strong expertise in Python, SQL, ETL/ELT development, and data transformations.
• Experience building scalable batch and/or streaming data pipelines.
• Experience with PySpark/Spark and data processing frameworks.
• Strong understanding of data warehousing, data modeling, data quality, and performance optimization.
• Experience with data pipeline orchestration and scheduling tools such as Airflow/Cloud Composer.
• Strong troubleshooting, problem-solving, and communication skills.

Conoscenze

GCP data engineering

BigQuery

Dataflow

Cloud Storage

Python

SQL

ETL/ELT

Data transformations

Airflow

PySpark

Data warehousing

Data modeling

Data quality

Strumenti

Airflow/Cloud Composer

Terraform

CI/CD

Descrizione del lavoro

We are seeking Data Engineerswith strong hands-on expertise in Google Cloud Platform (GCP), data pipeline development, ETL/ELT, and large-scale data processing.
The client is transitioning its enterprise data environment from Azure to GCP. Candidates must have solid, practical GCP data engineering experience, including building, optimizing, and supporting production data pipelines.

Required Skills:

• 6+ years of experience in data engineering, data integration, and data pipeline development.
• Strong hands-on GCP data engineering experience is mandatory.
• Experience with BigQuery, Dataflow, Cloud Storage, and relevant GCP data services.
• Strong expertise in Python, SQL, ETL/ELT development, and data transformations.
• Experience building scalable batch and/or streaming data pipelines.
• Experience with PySpark/Sparkand data processing frameworks.
• Strong understanding of data warehousing, data modeling, data quality, and performance optimization.
• Experience with data pipeline orchestration and scheduling tools such as Airflow/Cloud Composer.
• Strong troubleshooting, problem-solving, and communication skills.

Preferred Skills

• Experience migrating data platforms and pipelines from Azure to GCP.
• Knowledge of Azure Data Factory, Databricks, Azure Data Lake, or Synapse Analytics.
• Experience with Terraform, CI/CD, Pub/Sub, and automated data deployments.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.