Data Engineer - Hybrid, AI-Powered Data Pipelines

New York Post; New York (NY); Hybrid; USD 120,000 - 180,000; Full time

Job summary

The New York Post in New York, NY is seeking a Data Engineer to join our technology team. You will build and maintain data pipelines, data warehouses, and analytics services that power newsroom and product initiatives at scale.

You will collaborate with senior engineers, product managers, and data scientists, using Python, SQL, PySpark, and cloud services across AWS and Google Cloud. This is a hands-on role in a hybrid work environment.

Qualifications

• MS or BS in computer science, or related field.
• 2+ years of professional software engineering experience
• Proficiency in Python, SQL, and PySpark
• Working knowledge of AWS and/or GCP services
• Experience building and deploying data pipelines (Airflow, Dataflow, Glue, or similar tools)
• Knowledge of data warehouse technologies (BigQuery, Snowflake, Redshift, etc.)
• Familiarity with tools: AI tooling, IDEs, Git, JIRA, and agile methodologies
• Familiarity with web standards and protocols
• Comfort with code reviews, unit tests, QA and UAT processes, and end-to-end testing frameworks
• Strong problem-solving skills, attention to detail, eagerness to learn, and ability to manage multiple priorities

Responsibilities

• Design, build, and operate cloud-based data pipelines supporting needs of our websites, apps, and information services.
• Work daily with compute, query, storage, and big data services (Vertex AI, Lambdas, DynamoDB, Cloud Functions, S3, Kubernetes, Glue, BigQuery).
• Develop and maintain data ETL pipelines for real-time and batch processing of large-scale content and event data.
• Write clean, testable, and maintainable code (Python preferred).
• Collaborate with product, editorial, and data science teams to deliver infrastructure and services meeting business needs.
• Monitor, optimize, and improve the reliability, security, and cost efficiency of cloud-based systems.
• Write and present ideas clearly; document functional requirements for technical and non-technical audiences.
• Participate in code reviews, architecture discussions, and agile sprint ceremonies.

Skills

Problem solving

Attention to detail

Organized

Eagerness to learn

Collaboration

Education

MS or BS in Computer Science

Tools

Python

SQL

PySpark

Airflow

Dataflow

Glue

BigQuery

Snowflake

Redshift

AWS/GCP

Git

JIRA

Docker

Kubernetes

CloudWatch

Datadog

Splunk

REST/GraphQL APIs

Job description

The New York Post in New York, NY is seeking a Data Engineer to join our technology team. You will build and maintain data pipelines, data warehouses, and analytics services that power newsroom and product initiatives at scale.

You will collaborate with senior engineers, product managers, and data scientists, using Python, SQL, PySpark, and cloud services across AWS and Google Cloud. This is a hands-on role in a hybrid work environment.