Data Engineer - Cloud ETL/ML Pipelines (Azure & Snowflake)

Coca Cola Southwest Beverages

Inwood (TX)

On-site

USD 110,000 - 170,000

Full time

3 days ago

Be an early applicant

Application generator

Turn this role into an interview — a resume and cover letter built around what this employer wants.

Get past ATS filters

Job summary

Coca-Cola Southwest Beverages (CCSWB) is seeking a Data Engineer to design, develop, and maintain scalable data pipelines and architectures in Azure and Snowflake. You will optimize data workflows, ensure data quality, and support analytics, reporting, and AI/ML initiatives.

The role sits in the Advanced Analytics team, focusing on data integrity, governance, and cost-efficient storage while collaborating with data scientists, BI, and business stakeholders. 30% travel expected.

Qualifications

Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related field.
Advanced degree is a plus.
Strong experience with cloud platforms, especially Microsoft Azure services (Azure Data Factory, Databricks, Azure SQL).
Familiarity with Snowflake for data warehousing, including schema design and performance tuning.
Expertise in SQL and programming languages like Python, Scala, or Java.
Knowledge of ETL/ELT, data modeling, and best practices for data pipelines.
Familiarity with CI/CD, Infrastructure-as-Code (Terraform, ARM templates), and DevOps practices.
Understanding of data governance, security principles, and compliance standards.
Experience with Apache Spark, Airflow, and API development is a plus.
Strong SQL skills for database design, querying, and data manipulation.
Knowledge of scripting languages (Bash) for automation and data pipeline orchestration.
Understanding of data serialization formats like JSON, Avro, Parquet, and XML.
Familiarity with relational and NoSQL databases (SQL Server, PostgreSQL, MongoDB, Cassandra).
Core Data Engineering skills - SQL, Python/PySpark, ETL/ELT, data modeling, and distributed processing.
Current platform experience - Azure, Azure Data Factory, and Databricks.
Production engineering - pipeline troubleshooting, performance optimization, data quality, and monitoring.
DevOps/engineering practices - Git, CI/CD, testing, and deployment.
Broader/future skills - cloud architecture, Infrastructure as Code, APIs, and other data technologies.
30% travel projected

Responsibilities

Design, Develop, and Maintain scalable data pipelines to enable Advanced Analytics' initiatives and digital products.
Build and optimize ETL/ELT processes using Azure Data Factory, Databricks, and Snowflake.
Develop batch and real-time data pipelines to support reporting and AI/ML applications.
Implement data transformation and cleansing processes to ensure high data quality.
Automate data workflows to enhance efficiency and reduce manual interventions.
Monitor pipeline performance and troubleshoot issues to minimize downtime.
Oversee and Ensure data quality, integrity, and governance across Advanced Analytics' data ecosystem.
Implement data validation and anomaly detection techniques within pipelines.
Work with business users and analysts to understand data quality issues and implement solutions.
Maintain metadata and data lineage documentation for transparency and traceability.
Collaborate with cross-functional teams to ensure data consistency and reliability.
Monitor, Evaluate, and Optimize data storage and processing for performance and cost efficiency.
Design and implement efficient data models to support the Advanced Analytics team's needs.
Leverage partitioning, indexing, and clustering techniques for optimized query performance.
Monitor and manage cloud-based storage and compute costs to ensure cost-effectiveness.
Implement caching and performance tuning strategies for large-scale data processing.
Analyze workload patterns and recommend infrastructure improvements.
Collaborate with data scientists, analytics translators, and business stakeholders to deliver data solutions.
Gather requirements and translate business needs into scalable data engineering solutions.
Provide support to data scientists for feature engineering and model deployment.
Partner with business intelligence teams to improve data accessibility for reporting tools.
Develop reusable data assets and APIs for Analytics.
Conduct training and knowledge-sharing sessions to promote data-driven culture.
Maintain and Enhance security, reliability, and automation of data infrastructure.
Execute access control policies and role-based permissions in Azure according to definitions.
Automate deployment and monitoring processes using CI/CD pipelines and Infrastructure-as-Code (IaC).
Set up robust logging and alerting mechanisms to proactively detect issues.
Ensure compliance with internal and external data security regulations.
Continuously evaluate and implement new tools and best practices for data engineering.
End-to-end ownership - design, development, deployment, monitoring, production support, and enhancements.
Production engineering - troubleshooting, root-cause analysis, reliability, and performance.
Engineering practices - Git, CI/CD, code reviews, testing, and documentation.
Future readiness - ability to support current solutions while adapting to evolving products, technologies, and architecture.