Data Engineer

Encore Talent Solutions; Brea (CA); On-site; USD 120,000 - 160,000; Full time

Job summary

Encore Talent Solutions is seeking a Data Engineer to help build and scale its modern cloud data platform in California. The role focuses on designing and developing data pipelines, integrating diverse data sources, and delivering reliable, scalable analytics-ready datasets.

You will collaborate with Data Scientists and business stakeholders to transform raw data into actionable insights, leveraging Azure Databricks, AWS, and Python to support reporting and machine learning initiatives.

Qualifications

• Bachelor's degree in Computer Science, Information Systems, Data Engineering, Software Engineering, or a related field.
• 3+ years of experience in Data Engineering or a similar role.
• Hands-on experience with Azure Databricks.
• Experience working within AWS.
• Strong programming skills in Python.
• Advanced SQL development skills.
• Experience building scalable ETL/ELT pipelines.
• Experience with Git and source control.
• Strong understanding of relational databases and data warehousing concepts.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks and PySpark.
• Build robust data integration solutions for structured and unstructured data sources.
• Design and optimize data models to support reporting, analytics, and machine learning.
• Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and scalability.
• Implement data quality checks and governance best practices.
• Design and maintain cloud-based data storage solutions using Amazon S3.
• Configure secure AWS environments for data ingestion from internal and third-party data providers.
• Build automated processes for ingesting files into S3 and orchestrating downstream processing.
• Work with AWS services including: Amazon S3, AWS Glue, Lambda, IAM, CloudWatch.
• Support future cloud-native initiatives utilizing AWS container technologies such as ECS, EKS, or Kubernetes.
• Develop internal applications, automation tools, APIs, and utilities using Python.
• Build reusable services that streamline business processes and improve operational efficiency.
• Collaborate with cross-functional teams to deliver scalable, data-driven solutions.
• Partner with Sales, Marketing, Finance, Supply Chain, and Operations teams to understand business requirements.
• Support Data Scientists by preparing high-quality datasets for advanced analytics and AI initiatives.
• Work closely with IT and business stakeholders to implement enterprise data solutions.

Skills

Python

SQL

ETL/ELT pipelines

Data modeling

Cloud architectures

Collaboration & communication

Education

Bachelor's degree in CS/IS/Data Eng or related

Tools

Azure Databricks

AWS

PySpark

Amazon S3

Lambda

Glue

ECS/EKS/Kubernetes

Git

Job description

Sorry we are not considering any candidates that require sponsorship or transfer of any kind as the client needs to be able to convert this person without any kind of transfer or sponsorship of any visa. Local candidates only.

Data Engineer to help build and scale its modern cloud data platform. This role will be responsible for designing and developing data pipelines, integrating internal and external data sources, and building scalable cloud solutions using Azure Databricks, Amazon Web Services (AWS), and Python.

The ideal candidate is passionate about cloud technologies and data engineering, enjoys building applications and automation tools, and has experience creating reliable, high-performance data solutions that support analytics and business intelligence. This position will work closely with Data Scientists, Business Analysts, and business stakeholders to transform raw data into trusted, actionable insights.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks and PySpark.
• Build robust data integration solutions for structured and unstructured data sources.
• Design and optimize data models to support reporting, analytics, and machine learning.
• Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and scalability.
• Implement data quality checks and governance best practices.
• Design and maintain cloud-based data storage solutions using Amazon S3.
• Configure secure AWS environments for data ingestion from internal and third-party data providers.
• Build automated processes for ingesting files into S3 and orchestrating downstream processing.
• Work with AWS services including:
• Amazon S3
• AWS Glue
• Lambda
• IAM
• CloudWatch
• Support future cloud-native initiatives utilizing AWS container technologies such as ECS, EKS, or Kubernetes.

Application Development

• Develop internal applications, automation tools, APIs, and utilities using Python.
• Build reusable services that streamline business processes and improve operational efficiency.
• Collaborate with cross-functional teams to deliver scalable, data-driven solutions.
• Partner with Sales, Marketing, Finance, Supply Chain, and Operations teams to understand business requirements.
• Support Data Scientists by preparing high-quality datasets for advanced analytics and AI initiatives.
• Work closely with IT and business stakeholders to implement enterprise data solutions.

Required Qualifications

• Bachelor's degree in Computer Science, Information Systems, Data Engineering, Software Engineering, or a related field.
• 3+ years of experience in Data Engineering or a similar role.
• Hands-on experience with Azure Databricks.
• Experience working within Amazon Web Services (AWS).
• Strong programming skills in Python.
• Advanced SQL development skills.
• Experience building scalable ETL/ELT pipelines.
• Experience with Git and source control.
• Strong understanding of relational databases and data warehousing concepts.

Preferred Qualifications

• Experience working with syndicated market data providers such as NielsenIQ, Circana (IRI), SPINS, or retailer POS data.
• Experience with:
• PySpark
• Spark SQL
• Snowflake
• Amazon Athena
• Amazon Redshift
• Experience with Docker, Kubernetes, or containerized applications in AWS.
• Familiarity with CI/CD pipelines and DevOps practices.