Data Engineer

Storyful; New York (NY); Hybrid; USD 100,000 - 115,000; Full time

Job summary

The New York Post is seeking a skilled Data Engineer to design, build, and operate cloud-based data pipelines for our digital platforms. You will work with Python, SQL, and PySpark on AWS/GCP, developing ETL processes for real-time and batch content streams.

In a hybrid work environment, you will collaborate with product, editorial, and data science teams to deliver reliable, scalable infrastructure while participating in code reviews and agile ceremonies.

Qualifications

• BS or MS in computer science or related field.
• 2+ years of professional software engineering experience.
• Proficiency in Python, SQL, and PySpark.
• Working knowledge of AWS and/or GCP services.
• Experience building and deploying data pipelines (Airflow, Dataflow, Glue or similar).
• Knowledge of data warehouse technologies (BigQuery, Snowflake, Redshift, etc.).
• Familiarity with AI tooling, CI/CD, and containerization (Docker, Kubernetes).
• Testing, QA/UAT processes, and end-to-end testing familiarity.

Responsibilities

• Design, build, and operate cloud-based data pipelines for consumer websites, native apps, and information services.
• Work daily with compute, query, storage, and big data services on AWS/GCP.
• Develop and maintain ETL pipelines for real-time and batch processing of large-scale content and event data.
• Write clean, testable Python code and collaborate with product, editorial, and data science teams.
• Monitor, optimize, and improve reliability, security, and cost efficiency of cloud systems.
• Participate in code reviews, architecture discussions, and agile sprint ceremonies.

Skills

Python

SQL

PySpark

AWS

GCP

Airflow

Dataflow

Glue

BigQuery

Kubernetes

CI/CD

REST/GraphQL

Docker

Education

MS or BS in Computer Science or related field

Tools

AWS/GCP services

Airflow

Dataflow

Glue

BigQuery

Snowflake/Redshift

Git/JIRA

Job description

Job Description : The New York Post is one of America’s most iconic and recognizable news brands — delivering bold headlines, unforgettable scoops, and sharp commentary since 1801. With a legacy built on fearless journalism and a distinctly New York attitude, The Post has evolved far beyond its print roots to become a modern media powerhouse. Today, the New York Post Digital Network is one of the most influential voices in the industry, reaching over 80 million unique users each month across our expansive, ever-growing portfolio. Anchored by NYPost.com, PageSix.com, and Decider.com, our digital brands deliver must-read breaking news, sports, entertainment, pop culture, and lifestyle coverage with the same wit, edge, and energy that define The Post. With innovation at our core, we’ve built a true multi-platform ecosystem — spanning web, mobile apps, video, audio, social, print, TV, and commerce. From viral headlines and exclusive stories to original series, podcasts, and live events, we’re engaging audiences where they are — and always on the pulse of what’s next.

Key Responsibilities:

• Design, build, and operate cloud-based data pipelines supporting specialized needs of our consumer websites, native mobile apps, and information services.
• Work daily with compute services, query services, storage services, and big data services with technologies like Vertex AI, Lambdas, DynamoDB, Cloud Functions, S3, Kubernetes, Glue, and BigQuery inside AWS and GCP.
• Develop and maintain data ETL pipelines for real-time and batch processing of large-scale content and event data.
• Write clean, testable, and maintainable code (Python preferred).
• Collaborate with product, editorial, and data science teams to deliver infrastructure and services that meet business needs.
• Monitor, optimize, and improve the reliability, security, and cost efficiency of cloud-based systems.
• Write and present ideas clearly.
• Document functional requirements for both technical and non-technical audiences.
• Participate in code reviews, architecture discussions, and agile sprint ceremonies.

Qualifications:

• Must-Haves MS or BS in computer science, or related field.
• 2+ years of professional software engineering experience
• Proficiency in Python, SQL, and PySpark
• Working knowledge of common AWS and/or GCP services
• Experience building and deploying data pipelines (e.g., Airflow, Dataflow, Glue, or similar tools)
• Knowledge of data warehouse technologies (BigQuery, Snowflake, Redshift, etc.)
• Firm grasp of the tools of the trade including AI tooling, IDEs, Git, JIRA, and agile methodologies.
• Familiarity with web standards and protocols.
• Comfort with code reviews, writing unit tests, QA and UAT processes, and exposure to end-to-end testing frameworks.
• Strong problem-solving skills, attention to detail, thoroughness, and an eagerness to learn.
• Proactive approach to problem-solving.
• Highly organized with the ability to manage multiple priorities in a fast-paced environment.

Great-to-Have Experience:

• Experience Familiarity with content-management systems, media, news, publishing, video technology, ad technology, or other high-traffic consumer-facing industries.
• Exposure to personalization and recommendation systems.
• Comfortable with the CLI, Linux administration, shell scripting, ssh, crontabs, etc.
• Hands-on experience with generative-AI models, LLM prompting, and/or machine learning Experience with monitoring, logging, dashboard, and alarming tools (e.g., CloudWatch, Datadog, Splunk, etc.)
• Familiarity with REST and GraphQL APIs, authentication/authorization, and API best practices.
• Scaling techniques in cloud environments.
• Troubleshooting and debugging experience with internet services.
• Understanding of CI/CD workflows and containerization (Docker, Kubernetes a plus)

Note: The New York Post follows a hybrid work model. This position would be expected to be in the office a minimum of 3 days per week (subject to change).

Equal Opportunity Employer All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, national origin, protected veteran status, disability status or any other protected characteristic.

EEO/Disabled/Vets Reasonable Accommodation we are committed to providing reasonable accommodation for qualified individuals with disabilities in our job application and/or interview process. If you need assistance or accommodation in completing your application or participating in an interview due to a disability, email us at [e-mail]. Please put "Reasonable Accommodation" in the subject line and provide a brief description of the type of assistance you need. This inbox will not be monitored for application status updates. Please refer to the privacy notice at the bottom of this page for submitting any data access, deletion, or other data subject rights requests, where permitted under your local laws and regulations.

Base Pay Range: $100,000 - $115,000 We’re committed to offering competitive and flexible compensation to attract top talent. This pay range reflects our good faith estimate for the role and may vary based on a candidate’s experience, skills, location, and other relevant factors. For bonus-eligible roles, targets are determined based on multiple considerations, including market benchmarks and individual contributions. For benefits-eligible roles, we offer a comprehensive and competitive benefits package covering health, retirement, wellbeing, and more, along with optional benefits to meet the diverse needs of our employees.

New York Post Media Group is home to the oldest continuously-published daily newspaper in the United States, The New York Post, founded by Alexander Hamilton in 1801. Its portfolio also houses some of the nation's premier digital destinations for news, sports and entertainment, including The California Post and fabled Page Six gossip column, a world leader in breaking celebrity news that has evolved into its own iconic and powerful brand. The Post Digital Network is composed of the flagship NYPost.com, TheCaliforniaPost.com, PageSix.com, including Page Six Style, and Decider.com, covering streaming television and movies. The New York Post Media Group is owned by News Corp (Nasdaq: NWS, NWSA; ASX: NWS, NWSLV).

If you are a current employee in Workday, please navigate to the Jobs Hub and select Browse Jobs to apply internally.