Data Engineer: AI-Driven Pipelines in Hybrid Role

Commission Junction LLC; Agoura Hills (CA); Hybrid; USD 89,000 - 136,000; Full time

Benefits offered by this job

401K matching

Wellness programs

Medical coverage

Dental coverage

Vision coverage

Flexible time off

Paid holidays

Hybrid work arrangements

Employee referral program

Recognition awards

Parental bonding leave

Discount program

Global SOS program

Employee resource groups

Job summary

Commission Junction LLC in the United States is seeking a Data Engineer to design, build, and maintain scalable data platforms, pipelines, and infrastructure that support analytics, AI/ML, reporting, and enterprise data products.

This hybrid role requires 3 days a week in a CJ office location and collaborates with Data Science, Analytics, Product, and Engineering teams to deliver trusted data assets. Strong SQL, Python, PySpark and cloud experience are expected.

Qualifications

• 3–5 years of experience in Data Engineering, Analytics Engineering, or related disciplines.
• Experience with cloud-based data platforms such as Databricks, AWS, Snowflake, or Google Cloud.
• Familiarity with integrating AI and automation tools to enable faster, higher-quality code.
• Strong proficiency in SQL and Python.
• Experience with Spark and distributed data processing (PySpark preferred).
• Experience with data pipelines (ETL/ELT, event-driven, or workflow-based).
• Understanding of Data Lake, Lakehouse, and Medallion architecture concepts.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Python, PySpark, SQL, and Spark-based technologies.
• Advocate for AI-native engineering workflows to improve productivity, code quality, and efficiency.
• Build and support data ingestion frameworks leveraging APIs, SFTP, cloud storage, databases, and third-party integrations.
• Develop and maintain data products within a modern Data Lakehouse architecture following Medallion design principles.
• Create and optimize data workflows, orchestration, and monitoring to ensure pipeline reliability and performance.
• Manage cloud-based data infrastructure across Databricks, AWS, Snowflake, and Google Cloud.
• Build operational tooling and dashboards to track pipeline health, resource utilization, and uptime.
• Collaborate with Data Science, Analytics, Product, and Engineering teams to deliver trusted data assets.
• Maintain source control, CI/CD processes, and deployment best practices across data platforms.

Skills

SQL

Python

PySpark

Spark

ETL/ELT pipelines

Data Lakehouse

Medallion architecture

Cloud platforms

Tools

Databricks

AWS

Snowflake

Google Cloud

Job description

Commission Junction LLC in the United States is seeking a Data Engineer to design, build, and maintain scalable data platforms, pipelines, and infrastructure that support analytics, AI/ML, reporting, and enterprise data products.

This hybrid role requires 3 days a week in a CJ office location and collaborates with Data Science, Analytics, Product, and Engineering teams to deliver trusted data assets. Strong SQL, Python, PySpark and cloud experience are expected.