Senior Data Engineer

Vivid Resourcing; Charlotte (NC); On-site; USD 110,000 - 150,000; Full time

Job summary

Vivid Resourcing seeks aSenior Data Engineer to scale a leading marketing measurement platform in Charlotte. You will deepen data collection, processing, and client-facing data delivery, while scripting in Python and SQL to automate pipelines.

You will work with Databricks, BigQuery, Snowflake, and NoSQL DBs, plus Kubernetes, Docker, Terraform and Helm to deploy robust data infrastructure. Strong collaboration across teams and a bias for quality will be essential.

Qualifications

• Bachelor's degree in CS or equivalent.
• 3+ years of advanced SQL and Python, bash, etc.
• 3+ years Linux experience.
• Experience with Kubernetes, Docker, Terraform, Helm.
• 3+ years experience with SQL/NoSQL databases and data warehouses.
• Cloud providers (GCP, AWS, Azure).
• Experience with GitHub, CI/CD or similar tools.
• Understanding of BI and data warehousing concepts.
• Strong problem-solving and ability to handle multiple tasks.
• Excellent interpersonal and cross-functional collaboration.
• Process automation understanding.

Responsibilities

• Gain in-depth knowledge of how data is collected, processed, and externalized to clients within our architecture.
• Script in Python and SQL extensively.
• Create reports and data visualizations with Splunk/Grafana.
• Work with Databricks, BigQuery, Snowflake, OLTP, MongoDB, etc.
• Collaborate with teams to solve operational issues and implement quality improvements.

Skills

Advanced SQL

Python

Linux

Kubernetes

Docker

Terraform

CI/CD

GitHub

BI concepts

Data warehousing

Analytical thinking

Cross-functional collaboration

Education

Bachelor's degree in CS

Tools

Databricks

BigQuery

Snowflake

MongoDB

Job description

I'm partnered with a leading independent provider of marketing measurement software, data and analytics that authenticates the quality and effectiveness of digital media for the world's largest brands and media platforms look for a Senior Data Engineer to scale it's division in Charlotte.

Responsibilities

• You will gain in-depth knowledge of how data is collected, processed, and externalized to clients within our architecture
• You will script in Python and SQL extensively
• You will work with data analysis tools such as Splunk/Grafana to create reports and data visualization
• You will work with Databricks, BigQuery, Snowflake, OLTP, MongoDB, etc
• You will work with Kubernetes, Docker, Terraform, Helm charts, etc
• You will be thrilled at the prospect of building strong relationships with different teams in the company, solving operational issues, and implementing quality improvements

Requirements

• Bachelor's degree in CS or equivalent experience. Degree in a technical field preferred
• 3+ years of experience writing Advanced SQL and scripting languages Python, bash, etc
• 3+ years of Linux experience
• Experience working with Kubernetes, Docker, Terraform, Helm charts, and fundamentals of DevOps
• 3+ years of experience working with SQL/NoSQL Databases and data warehouses such as Databricks, BigQuery, Snowflake, MongoDB, etc.
• Knowledge of Cloud computing fundamentals and experience working with public cloud providers such as GCP, AWS, Azure, etc.
• Experience working with Github, CI/CD, GitLab or other automation/delivery tools
• Good understanding of BI and Data Warehousing concepts (ETL, OLAP vs. OLTP, Slowly Changing Dimensions)
• Demonstrated ability to adapt quickly, learn new skill sets, and be able to understand operational challenges
• Strong analytical, problem-solving, negotiation, and organizational skills with a clear focus under pressure
• Must be proactive with a proven ability to execute multiple tasks simultaneously
• Excellent interpersonal skills, including relationship building with diverse, global, cross-functional team
• Good understanding of process automation