Data Platform Engineer

Milestone Technologies, Inc.; New York (NY); Hybrid; USD 157,597,000 - 186,252,000; Full time

Job summary

Milestone Technologies, Inc. is seeking a Data Platform Engineer to join an innovative Enterprise data team. You will work with data engineers, architects, domain experts and analysts to build data platform solutions at scale in a hybrid NYC setting.

You will deploy modern data platforms (AWS, Azure, Snowflake) and manage CI/CD pipelines and IaC, while ensuring data quality and meeting SLAs. 5+ years in data warehousing, Snowflake, ETL, and Python are preferred.

Qualifications

• Bachelor's or Master's degree in computer science, engineering or related field.
• 5+ years of overall experience and 5+ years of experience with very large-scale data warehouse projects.
• Strong experience in SQL, dimensional modeling, supporting data warehouse, scaling, optimizing and performance tuning ETL pipelines.
• 3+ years of experience with Python and manipulation of various data formats for extraction and transformation.
• 3+ years of experience with Snowflake, Airflow, ETL/Integration tools administration.
• Hands-on experience with Data Warehouse technologies (Snowflake, Redshift) and Big Data technologies (Hadoop, Hive, Spark, Kafka, Kinesis).
• Experience evaluating new tools and enriching existing architecture.
• You are a self-starter, hands-on technical expert who communicates deliverables, timelines and tradeoffs.

Responsibilities

• Work closely with data engineers, data architects, domain experts and business analysts to understand business requirements and help to build best in class data platform solutions and architecture for scale.
• Build and automate modern data platforms (AWS, AZURE, Snowflake) with native technologies, deploy applications, and provision of infrastructure.
• Perform Administration, Maintenance and provide support to production systems (Snowflake, Fivetran, DBT, Power BI, Airflow)
• Build and maintain CI/CD pipelines and infrastructure as code (IaC) practice across the platform for safe and reproducible deployments.
• Design, build and implement automation framework for scale that deliver data with measurable quality under the SLA
• Drive technical conversations with stakeholders to understand the platform's needs and come up with a provisioning strategy.
• Work effectively using scrum with multiple team members to deliver value effectively.
• Reduce technical debt over time with root cause identification, issue resolution and promote best practices across the platform

Skills

SQL

Dimensional modeling

ETL pipelines

Python

Snowflake

Airflow

Big Data

Kinesis

Kafka

Education

Bachelor's or Master's in CS/Engineering

Tools

Snowflake

Redshift

Hadoop

Hive

Spark

Kafka

Kinesis

Airflow

Fivetran

DBT

Power BI

Job description

This is hybrid 2 - 3 days a week onsite in NYC.

W2 only as this may transition to FTE.

Seeking a Data Platform Engineer to be part of an innovative Enterprise data team that enables Marketing, Product, Sales, and Finance to explore data and take actions that differentiate us from our competition. You will work closely with data engineers, data architects, domain experts and business analysts to understand business requirements and help to build best in class data platform solutions and architecture for scale.

Responsibilities:

• You will work closely with data engineers, data architects, domain experts and business analysts to understand business requirements and help to build best in class data platform solutions and architecture for scale.
• Build and automate modern data platforms (AWS, AZURE, Snowflake) with native technologies, deploy applications, and provision of infrastructure.
• Perform Administration, Maintenance and provide support to production systems (Snowflake, Fivetran, DBT, Power BI, Airflow)
• Build and maintain CI/CD pipelines and infrastructure as code (IaC) practice across the platform for safe and reproducible deployments.
• Design, build and implement automation framework for scale that deliver data with measurable quality under the SLA
• Drive technical conversations with stakeholders to understand the platform's needs and come up with a provisioning strategy.
• Work effectively using scrum with multiple team members to deliver value effectively.
• Reduce technical debt over time with root cause identification, issue resolution and promote best practices across the platform

Experience and Qualifications

• Bachelor's or master's degree in computer science, Engineering or related field
• 5+ years of overall experience and 5+ years of experience with very large-scale data warehouse projects.
• Strong experience in SQL, knowledge of dimensional modeling, supporting data warehouse, scaling, optimizing and performance tuning ETL pipelines
• 3+ years of experience with Python and manipulation of various data formats for extraction and transformation
• 3+ years of experience with Snowflake, Airflow, ETL/Integration tools administration.
• Hands-on experience with Data Warehouse technologies (Snowflake, Redshift) and Big Data technologies (Hadoop, Hive, Spark, Kafka, Kinesis)
• Experience evaluating new tools and enriching existing architecture
• You are a self-starter, hands-on technical expert who continuously gather and synthesize high-impact needs from business partners, design and implement the appropriate technical platform solution, and effectively communicate about deliverables, timelines and tradeoffs

The estimated pay range for this position is USD $55.00/hr - USD $65.00/hr. Exact compensation and offers of employment are dependent on job-related knowledge, skills, experience, licenses or certifications, and location. We also offer comprehensive benefits. The Talent Acquisition Partner can share more details about compensation or benefits for the role during the interview process.