Data Engineer

Skillerszone LLC; Los Angeles (CA); On-site; USD 90,000 - 120,000; Full time

Job summary

A leading technology company is seeking a talented Data Engineer to build robust data infrastructure. The role involves designing efficient data pipelines, integrating data from various sources, and ensuring high data quality. Ideal candidates have solid experience in data engineering, proficiency in SQL and NoSQL databases, and familiarity with cloud solutions like AWS or Azure. Join us to drive innovation through data!

Qualifications

• Proven experience as a Data Engineer or similar role.
• Strong knowledge of data engineering principles and architecture.
• Experience with ETL tools and data integration.
• Familiarity with cloud storage solutions.

Responsibilities

• Design and maintain data pipelines for large data sets.
• Integrate data from various sources for seamless flow.
• Develop and manage data warehouses and lakes.
• Monitor and ensure data quality and integrity.

Skills

Data engineering principles

Data modeling

SQL

NoSQL

ETL processes

Apache Spark

Hadoop

Cloud platforms

Python

Java

Scala

Redshift

BigQuery

Snowflake

Education

Bachelor's degree in Computer Science

Advanced degrees or certifications

Job description

Overview

About Optiboostmedia is a leading provider of affiliate marketing and recruitment software solutions. We help businesses optimize their marketing and recruitment processes through innovative, user-friendly technology. We are seeking a talented Data Engineer to join our team and build the infrastructure to support the collection, storage, and processing of large data sets, ensuring that we have reliable access to the data we need to drive decisions and innovation.

Key Responsibilities

• Data Pipeline Development: Design, build, and maintain scalable and efficient data pipelines that handle large volumes of structured and unstructured data.
• Data Integration: Integrate data from various internal and external sources, ensuring seamless data flow between systems, databases, and storage solutions.
• Data Warehousing: Develop and manage data warehouses and data lakes, ensuring that data is properly organized, stored, and optimized for analysis.
• ETL Processes: Implement and optimize ETL (Extract, Transform, Load) processes to ensure data is transformed and stored correctly for downstream analysis.
• Database Management: Manage and optimize databases (e.g., SQL, NoSQL) to ensure high performance, availability, and scalability.
• Data Quality & Monitoring: Monitor data pipelines to identify and resolve issues, ensuring high data quality, integrity, and reliability.
• Collaboration with Data Teams: Work closely with data scientists, analysts, and business teams to understand data requirements, provide data access, and ensure alignment with business goals.
• Automation & Optimization: Implement automation strategies to reduce manual intervention and optimize data processing workflows.
• Performance Tuning: Optimize and fine-tune data processes and queries to enhance performance and reduce latency.
• Documentation & Best Practices: Document data pipeline processes, workflows, and best practices, ensuring clarity for internal teams and future development.

Qualifications

• Proven experience as a Data Engineer, Data Analyst, or in a similar data-related role.
• Strong knowledge of data engineering principles, data modeling, and data architecture.
• Proficiency with databases (e.g., SQL, NoSQL) and data processing frameworks (e.g., Apache Spark, Hadoop).
• Experience with ETL tools and processes, including data integration and transformation.
• Familiarity with cloud platforms and data storage solutions (e.g., AWS, Azure, Google Cloud).
• Knowledge of programming languages such as Python, Java, or Scala for data processing and scripting.
• Experience with data warehousing technologies (e.g., Redshift, BigQuery, Snowflake) and tools for building data pipelines.
• Strong analytical and problem-solving skills with a focus on optimizing data processes.
• Ability to work in a fast-paced environment and collaborate with cross-functional teams.
• Bachelors degree in Computer Science, Engineering, or a related field; advanced degrees or certifications are a plus.