Senior Data Engineer: Scalable Pipelines & Data Platforms

Metropolis Technologies; United States; On-site; USD 140,000 - 190,000; Full time

Job summary

Metropolis Technologies in the United States seeks a Senior Data Engineer to design and build scalable data pipelines, data models, and analytics foundations powering multiple products in parking, mobility and real estate.

You will collaborate with engineers, product managers, and data scientists to translate data needs into end-to-end solutions, own data architecture for large-scale projects, mentor teammates, and uphold data quality, security, and governance in a fast-paced, inclusive

Qualifications

• Bachelor's degree in computer science, computer engineering, or a related technical field.
• 5+ years of experience in data engineering, database engineering, business intelligence, data warehousing, and ETL tools, with large datasets in the cloud.
• Proficiency in SQL, ETL/ELT, and data modeling, with extensive Snowflake and dbt experience.
• 5+ years of experience with Python, and building scalable Big Data solutions and ETL ecosystems.
• Hands-on experience with RDBMS such as MySQL, MS SQL Server, and PostgreSQL.
• Familiarity with integration tools like Airflow and Automic, and CI/CD pipelines.
• Ability to deliver high standards of code quality, reliability, and performance.
• Experience with Spark and PySpark for large-scale data processing.
• Strong proficiency in Terraform is required.

Responsibilities

• Collaborate with engineers, product managers, and data scientists to understand data needs and develop end-to-end data pipelines, solutions, and foundational data sets.
• Design and own the data architecture for large-scale projects, making informed design and operational trade-offs.
• Build, launch, and optimize sophisticated data pipelines, data models, and visualizations supporting multiple products/domains.
• Define and manage SLAs for all data sets within ownership areas.
• Implement data security models, ensure privacy safeguards, and evolve data governance processes.
• Solve complex data integration challenges using ETL/ELT patterns and techniques for structured/unstructured data.
• Maintain and enhance production processes by optimizing code with advanced algorithms.
• Optimize pipelines, dashboards, frameworks, and systems for data artifact development.
• Mentor team members and foster skill growth and collaboration.
• Ensure availability during evening hours.

Skills

SQL expertise

Python programming

Data modeling

ETL/ELT patterns

Mentoring

Team collaboration

Evening hours availability

Education

Bachelor's degree in CS or related field

Tools

Snowflake

dbt

Airflow

Terraform

Spark

PySpark

MySQL

PostgreSQL

MS SQL Server

AWS

CI/CD

Job description

Metropolis Technologies in the United States seeks a Senior Data Engineer to design and build scalable data pipelines, data models, and analytics foundations powering multiple products in parking, mobility and real estate.

You will collaborate with engineers, product managers, and data scientists to translate data needs into end-to-end solutions, own data architecture for large-scale projects, mentor teammates, and uphold data quality, security, and governance in a fast-paced, inclusive