Data Engineer — AI/ML Powered Data Pipelines

Photon; United States; On-site; USD 96,600 - 161,000; Full time

Benefits offered by this job

Medical benefits

Vision benefits

Dental benefits

401k

Paid time off

Paid holidays

Job summary

Photon seeks a data engineering professional to design and operate scalable data pipelines, leveraging DataProc, Dataflow, and Composer on Google Cloud. You will build streaming and batch processes, ensure data quality and governance, and enable AI/ML initiatives with insightful visualizations in Looker Studio.

You will collaborate with US-based cross-functional teams to deliver robust data solutions. The role emphasizes strong SQL/Python/Java skills, experience with Hadoop/Spark/Kafka, and a

Qualifications

• Bachelor’s degree in Computer Science, Data Engineering, or a related field; a Master’s degree is a plus.
• 5+ years in data engineering, big data processing, or a related field.
• Strong skills in SQL, Python, or Java, with experience in big data technologies like Hadoop, Spark, or Kafka.
• Proficient in stream and batch processing frameworks and data visualization tools.
• Skilled in implementing data governance practices.

Responsibilities

• Contribute to Mail Intelligence Mission: Support the development of AI/ML capabilities that enhance user personalization and insights.
• Innovate and Personalize: Assist in building scalable solutions that identify user interests and habits, contributing to personalized user experiences.
• Data Pipeline and Insights Development: Design and implement robust data pipelines using frameworks like DataProc, Dataflow, and Composer. Streamline data processing and orchestration, utilize Data Lake and Pub/Sub for storage and streaming, and create insightful visualizations with Looker Studio to support AI/ML initiatives.
• Collaboration: Work closely with cross-functional teams in the USA to integrate data solutions that enhance user experiences.
• Data Governance: Implement robust data governance practices to maintain data quality and compliance across the organization.
• Optimization: Continuously improve data workflows for efficiency and scalability.
• Technical Strategy: Collaborate with tech leads to align data strategies with business goals.
• Feedback and Recovery: Implement mechanisms to handle data processing failures gracefully.
• Tradeoff Management: Balance cost, quality, and speed in data solutions to optimize performance.

Skills

SQL

Python

Java

Data engineering

Data governance

Problem solving

Collaboration

Education

Bachelor's degree in Computer Science or related field

Master's degree a plus

Tools

Hadoop

Spark

Kafka

DataProc

Dataflow

Composer

Looker Studio

Pub/Sub

Data Lake

Job description

Photon seeks a data engineering professional to design and operate scalable data pipelines, leveraging DataProc, Dataflow, and Composer on Google Cloud. You will build streaming and batch processes, ensure data quality and governance, and enable AI/ML initiatives with insightful visualizations in Looker Studio.

You will collaborate with US-based cross-functional teams to deliver robust data solutions. The role emphasizes strong SQL/Python/Java skills, experience with Hadoop/Spark/Kafka, and a