BigQuery Engineer

Openkyber, LLC; Atlanta (GA); Hybrid; USD 66,000 - 103,000; Part time

Job summary

OpenKyber is seeking a Data Engineer to design, develop, and maintain enterprise-scale data pipelines and platforms within a cloud-based data ecosystem. You will collaborate with architects, engineers, analysts, and stakeholders to deliver scalable data solutions for analytics, reporting, and operations.

The ideal candidate will have hands-on experience with GCP, BigQuery, Spark, and a strong background in ETL/ELT design, Python/Java/Scala, and data modeling.

Qualifications

• Hands-on experience with Google Cloud Platform (GCP) and cloud-native data services.
• Advanced experience with BigQuery, including SQL development, query optimization, and large-scale data processing.
• Hands-on experience with Apache Spark and distributed computing frameworks.
• Experience with GCP Dataproc, Google Cloud Storage (GCS), and cloud-based data processing environments.
• Experience working with Apache Hadoop, Apache Hive, and other big data technologies.
• Proven experience building and supporting production-grade data engineering solutions.
• Strong expertise in ETL/ELT design, development, and optimization.
• Strong SQL skills and experience processing large, complex datasets.
• Proficiency in Python and/or Java/Scala.
• Understanding of data modeling, partitioning strategies, distributed processing concepts, and common data storage formats.
• Experience with DevOps practices, CI/CD pipelines, Git version control, automated deployments, and production support.
• Strong analytical, troubleshooting, and problem-solving skills.
• Experience developing reporting or analytics solutions using Looker and/or Power BI.

Responsibilities

• Design, develop, test, and maintain scalable data pipelines and ETL/ELT workflows for large-volume datasets.
• Build and optimize distributed data processing solutions using Apache Spark and GCP Dataproc.
• Develop and maintain analytical datasets and data workloads within BigQuery.
• Create reliable batch and streaming data processing solutions using technologies such as Spark, Hadoop, and Kafka.
• Develop data ingestion and transformation pipelines across source systems, data lakes, and analytics platforms.
• Optimize Spark and BigQuery workloads for performance, scalability, reliability, and cost efficiency.
• Troubleshoot production data pipelines, perform root-cause analysis, and implement long-term solutions.
• Implement data quality controls, monitoring, alerting, and operational support processes.
• Apply software engineering and DevOps best practices, including source control, automated testing, CI/CD, and deployment automation.
• Collaborate with cross-functional teams to translate business requirements into production-ready data solutions.
• Participate in code reviews and contribute to engineering standards, reusable frameworks, and best practices.

Skills

GCP Data Engineering

BigQuery

Apache Spark

Dataproc

Python

SQL

Looker/Power BI

ETL/ELT

Job description

Title : Data Engineer

Location : Brooklyn Park, MN | Hybrid

Job Type : Contract (6 months)

Compensation : $48.00 - $75.00 per hour (W2)

Industry: Retail

About the Role

Our client, a leading organization in the retail and consumer services industry, is seeking a Data Engineer to join a high-performing data and analytics team. This role is focused on building scalable, cloud-based data solutions that enable data-driven decision-making across a large, complex enterprise environment. The ideal candidate will have strong expertise in data engineering, distributed processing technologies, and Google Cloud Platform (GCP), with a passion for developing reliable, high-quality data products.

Job Description

As a Data Engineer, you will design, develop, and maintain enterprise-scale data pipelines and platforms that support analytics, reporting, and operational data needs. You will work closely with architects, engineers, analysts, and business stakeholders to deliver efficient and scalable data solutions.

Key Responsibilities

• Design, develop, test, and maintain scalable data pipelines and ETL/ELT workflows for large-volume datasets.
• Build and optimize distributed data processing solutions using Apache Spark and GCP Dataproc.
• Develop and maintain analytical datasets and data workloads within BigQuery.
• Create reliable batch and streaming data processing solutions using technologies such as Spark, Hadoop, and Kafka.
• Develop data ingestion and transformation pipelines across source systems, data lakes, and analytics platforms.
• Optimize Spark and BigQuery workloads for performance, scalability, reliability, and cost efficiency.
• Troubleshoot production data pipelines, perform root-cause analysis, and implement long-term solutions.
• Implement data quality controls, monitoring, alerting, and operational support processes.
• Apply software engineering and DevOps best practices, including source control, automated testing, CI/CD, and deployment automation.
• Collaborate with cross-functional teams to translate business requirements into production-ready data solutions.
• Participate in code reviews and contribute to engineering standards, reusable frameworks, and best practices.

Qualifications

Required Qualifications

• Hands-on experience with Google Cloud Platform (GCP) and cloud-native data services.
• Advanced experience with BigQuery, including SQL development, query optimization, and large-scale data processing.
• Hands-on experience with Apache Spark and distributed computing frameworks.
• Experience with GCP Dataproc, Google Cloud Storage (GCS), and cloud-based data processing environments.
• Experience working with Apache Hadoop, Apache Hive, and other big data technologies.
• Proven experience building and supporting production-grade data engineering solutions.
• Strong expertise in ETL/ELT design, development, and optimization.
• Strong SQL skills and experience processing large, complex datasets.
• Proficiency in Python and/or Java/Scala.
• Understanding of data modeling, partitioning strategies, distributed processing concepts, and common data storage formats.
• Experience with DevOps practices, CI/CD pipelines, Git version control, automated deployments, and production support.
• Strong analytical, troubleshooting, and problem-solving skills.
• Experience developing reporting or analytics solutions using Looker and/or Power BI.

Preferred Qualifications

• Experience with LookML development.
• Experience with BigLake, Apache Iceberg, Parquet, and modern data lake architectures.
• Knowledge of Terraform or other Infrastructure-as-Code (IaC) tools.
• Experience designing and implementing Kafka-based streaming data pipelines.
• Experience migrating data platforms from on-premises Hadoop/Hive environments to Google Cloud Platform.
• Experience supporting large-scale data modernization, cloud transformation, or enterprise data platform initiatives.
• Familiarity with data quality frameworks, governance practices, data lineage, and metadata management.
• Experience contributing to reusable data engineering frameworks and platform capabilities.

Dahl Consulting is proud to offer a comprehensive benefits package to eligible employees that will allow you to choose the best coverage to meet your family's needs. For details, please review the DAHL Benefits Summary: https://www.dahlconsulting.com/benefits-w2fta/.

Equal Opportunity Statement As an equal opportunity employer, OpenKyber welcomes candidates of all backgrounds and experiences to apply. If this position sounds like the right opportunity for you, we encourage you to take the next step and connect with us. We look forward to meeting you!