Data Engineer

Quantum Technologies. LLC; Austin (TX); On-site; USD 120,000 - 170,000; Full time

Job summary

Quantum Technologies. LLC in Austin, TX seeks data engineers with strong hands-on GCP expertise to design, build, optimize, and support large-scale production data pipelines.

The role focuses on migrating / transitioning from Azure to GCP and delivering reliable data processing solutions. Candidates should demonstrate deep experience with BigQuery, Dataflow, Python, SQL, ETL/ELT, and orchestration tools like Airflow or Cloud Composer.

Qualifications

• 6+ years of experience in data engineering, data integration, and data pipeline development.
• Strong hands-on GCP data engineering experience is mandatory.
• Experience with BigQuery, Dataflow, Cloud Storage, and relevant GCP data services.
• Strong expertise in Python, SQL, ETL/ELT development, and data transformations.
• Experience building scalable batch and/or streaming data pipelines.
• Experience with PySpark/Spark and data processing frameworks.
• Strong understanding of data warehousing, data modeling, data quality, and performance optimization.
• Experience with data pipeline orchestration and scheduling tools such as Airflow/Cloud Composer.
• Strong troubleshooting, problem-solving, and communication skills.

Skills

GCP

BigQuery

Dataflow

Python

SQL

ETL/ELT

Airflow/Cloud Composer

PySpark

Spark

Data warehousing

Data modeling

Data quality

Performance optimization

Azure to GCP migration

Tools

BigQuery

Dataflow

Cloud Storage

Airflow

Cloud Composer

Terraform

CI/CD

Pub/Sub

Job description

We are seeking Data Engineerswith strong hands-on expertise in Google Cloud Platform (GCP), data pipeline development, ETL/ELT, and large-scale data processing.
The client is transitioning its enterprise data environment from Azure to GCP. Candidates must have solid, practical GCP data engineering experience, including building, optimizing, and supporting production data pipelines.

Required Skills:

• 6+ years of experience in data engineering, data integration, and data pipeline development.
• Strong hands-on GCP data engineering experience is mandatory.
• Experience with BigQuery, Dataflow, Cloud Storage, and relevant GCP data services.
• Strong expertise in Python, SQL, ETL/ELT development, and data transformations.
• Experience building scalable batch and/or streaming data pipelines.
• Experience with PySpark/Sparkand data processing frameworks.
• Strong understanding of data warehousing, data modeling, data quality, and performance optimization.
• Experience with data pipeline orchestration and scheduling tools such as Airflow/Cloud Composer.
• Strong troubleshooting, problem-solving, and communication skills.

Preferred Skills

• Experience migrating data platforms and pipelines from Azure to GCP.
• Knowledge of Azure Data Factory, Databricks, Azure Data Lake, or Synapse Analytics.
• Experience with Terraform, CI/CD, Pub/Sub, and automated data deployments.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.