Data Engineer (Won)

Inclusion Cloud

United States

Remote

USD 120,000 - 160,000

Full time

9 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Inclusion Cloud is seeking a data-focused engineer to design, build, and optimize large-scale data pipelines in a hybrid cloud setting. You will assemble diverse data sets, implement scalable processing, and deliver timely insights.

You will work with cross-functional teams, apply Agile practices, and ensure data security across the pipeline while leveraging Hadoop, Spark, Kafka, and Airflow. This role offers opportunities to impact product data quality and decision-making.

Qualifications

  • Experience building and optimizing big data pipelines architectures and data sets.
  • Knowledge of message queuing, stream processing and scalable big data stores.
  • Processes for data transformation, metadata, dependency and workload management.
  • Proven history of extracting value from data sets.
  • Familiarity with Agile methodologies.
  • Experience with cross-functional teams in dynamic environments.
  • Experience with Hadoop, Spark, Kafka, and Airflow.
  • Experience with public cloud: Google Cloud, AWS, Azure.

Responsibilities

  • Create and maintain optimal data pipeline architectures in a hybrid cloud environment.
  • Assemble large and complex data sets meeting functional and non-functional business needs.
  • Identify and implement internal process improvements to automate data workflows and improve delivery.
  • Build and maintain the infrastructure to transfer and hold data from diverse sources.
  • Create analytics tools that provide actionable insights and KPI metrics.
  • Ensure data security throughout the pipeline in a hybrid cloud environment.
  • Collaborate with stakeholders on data-related issues and needs.

Skills

Big data pipelines
Streaming data
Data transformation
Metadata management
Agile methodologies
Root cause analysis

Tools

Hadoop
Spark
Kafka
Airflow
SQL Server
PostgreSQL
Cassandra

Job description

About You:

You are passionate about data big data. You are familiar with techniques for streaming and moving data between applications. You have developed applications that efficiently and effectively move and transform data as needed.. You want to drive the accuracy, timeliness and completeness of data for all products. You want to improve processes and outcomes for data for client consumption.

Primary Accountabilities:
  • Technical (100%)
  • Create and maintain optimal data pipeline architectures in a hybrid cloud environment
  • Assemble large and complex data sets that meet both functional and non-functional business needs
  • Identify, design and implement internal process improvements such as automating manual data processes, optimizing data delivery and scalability
  • Build and maintain the infrastructure that is required to transfer and hold data from a wide variety of data sources
  • Create analytics tools that utilize the data pipeline to provide actionable insights, operational efficiencies and other key performance metrics/indicators (KPM/KPI)
  • Keep data secure throughout the pipeline in our hybrid cloud environment
  • Create data tools for analytics and data science team members that will assist them in building and optimizing product portfolio
  • Work with stakeholders on data-related technical issues and their data needs
Required Qualifications:
  • Experience building and optimizing big data pipelines architectures and data sets.
  • Working knowledge of message queuing, stream processing and highly scalable big data data stores
  • You have built processes that support data transformation, data structures, metadata, dependency and workload management
  • A successful history of manipulation, processing and extracting value from data sets
  • Familiarity with Agile methodologies and development processes.
  • You have performed root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
  • Experience supporting and working with cross-functional teams in a dynamic environment
  • Experience with big data tools such as Hadoop, Spark, Kafka, etc.
  • Experience with public cloud technologies in Google, AWS and Azure
  • Experience with relational and NoSQL databases such as SQL Server, Postgres, Cassandra, etc.
  • Experience with data pipeline and workflow management tools such as Airflow
  • Experience with object oriented/object function scripting languages such as Python, Java, Scala
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

UNAVAILABLE • McLean (VA)

On-site
USD 110,000 - 160,000
Data Engineer
Data Engineer

Vein Standard Metals • United States

Remote
USD 110,000 - 160,000
Data Engineer
Data Engineer

TechDigital Group • Burbank (CA)

On-site
USD 120,000 - 150,000
Data Platform Engineer
Data Platform Engineer

Beecruit Pte Ltd • United States

Remote
USD 120,000 - 170,000
Data Engineer
Data Engineer

EdgeQ Technologies Private Limited • United States

On-site
USD 120,000 - 180,000
Data Engineer
Data Engineer

Aptdata Solutions Inc. • Farmington Hills (MI)

On-site
USD 90,000 - 115,000
Data Engineer
Data Engineer

TechDigital Group • Santa Clara (CA)

On-site
USD 90,000 - 150,000
Data Engineer
Data Engineer

VITAL SAÚDE &SEGURANÇA DO TRABALHO • United States

Hybrid
USD 90,000 - 120,000
Competitive salary
Performance incentives
Professional development
+1
Data Engineer
Data Engineer

Compunnel Inc. • Columbus (OH)

On-site
USD 85,000 - 115,000
Senior Data Engineer - Scalable Pipelines & Cloud Analytics
Senior Data Engineer - Scalable Pipelines & Cloud Analytics

Initvalue • Fremont (CA)

Hybrid
USD 130,000 - 190,000