Data Engineer

Valiant Technologies LLC; Austin (TX); On-site; USD 120,000 - 170,000; Full time

Job summary

Valiant Technologies LLC in Austin, TX seeks Data Engineers with hands-on GCP data engineering expertise to design and optimize large-scale data pipelines and support the Azure-to-GCP migration.

Applicants should have 6+ years in data engineering, strong Python/SQL skills, experience with BigQuery, Dataflow, Spark, and Airflow/Cloud Composer for orchestration.

Excellent communication and collaboration skills are essential as you will work closely with analytics and product teams.

Qualifications

• 6+ years of experience in data engineering and data pipeline development.
• Solid hands-on GCP data engineering experience.
• Experience with BigQuery, Dataflow, Cloud Storage and related GCP services.
• Strong Python and SQL skills for ETL/ELT development and data transformations.
• Experience building scalable batch and streaming data pipelines.
• Experience with PySpark/Spark and data processing frameworks.
• Understanding of data warehousing, data modeling, data quality, and performance optimization.
• Experience with Airflow/Cloud Composer for orchestration.

Skills

GCP

BigQuery

Dataflow

Cloud Storage

Python

SQL

ETL/ELT

PySpark

Airflow

Data warehousing

Tools

Terraform

CI/CD

Pub/Sub

Cloud Composer

Job description

We are seeking Data Engineerswith strong hands-on expertise in Google Cloud Platform (GCP), data pipeline development, ETL/ELT, and large-scale data processing.
The client is transitioning its enterprise data environment from Azure to GCP. Candidates must have solid, practical GCP data engineering experience, including building, optimizing, and supporting production data pipelines.

Required Skills:

• 6+ years of experience in data engineering, data integration, and data pipeline development.
• Strong hands-on GCP data engineering experience is mandatory.
• Experience with BigQuery, Dataflow, Cloud Storage, and relevant GCP data services.
• Strong expertise in Python, SQL, ETL/ELT development, and data transformations.
• Experience building scalable batch and/or streaming data pipelines.
• Experience with PySpark/Sparkand data processing frameworks.
• Strong understanding of data warehousing, data modeling, data quality, and performance optimization.
• Experience with data pipeline orchestration and scheduling tools such as Airflow/Cloud Composer.
• Strong troubleshooting, problem-solving, and communication skills.

Preferred Skills

• Experience migrating data platforms and pipelines from Azure to GCP.
• Knowledge of Azure Data Factory, Databricks, Azure Data Lake, or Synapse Analytics.
• Experience with Terraform, CI/CD, Pub/Sub, and automated data deployments.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.