Data Engineer

Openkyber; United States; On-site; USD 110,000 - 160,000; Full time

Job summary

Openkyber is seeking a senior Data Engineer to design and implement data pipelines, collaborate with the data science team, and ensure robust data storage and analysis across platforms.

The role emphasizes building scalable data models, warehouses, and efficient SQL/Scala-based queries while leveraging AWS services like S3, Glue, and EMR. Strong experience with Snowflake and Databricks is preferred.

Qualifications

• 5+ years of experience in data modeling, warehousing, and analysis.

Responsibilities

• Design and implement data pipelines using Databricks, Snowflake and Spark.
• Work with large datasets and perform data modeling, warehousing, and analysis.
• Utilize AWS technologies like S3, AWS Glue, and EMR for data processing and storage.
• Develop and optimize queries in SQL, SparkSQL or Scala for data analysis.
• Handle structured and unstructured datasets in relational databases like Teradata.
• Collaborate with the data science team to identify trends and anomalies in data.
• Implement and maintain IAM roles and permissions for data access.

Skills

Data modeling

Data warehousing

Data analysis

Python

KornShell

Education

Bachelor's or Master's degree in Data Science, Statistics, Computer Science, or Mathematics

Tools

Snowflake

Databricks

Spark

AWS Glue

Amazon EMR

S3

Job description

Data Engineer Introduction:

The Data Engineer will be responsible for designing and implementing data solutions to support the organization's data needs. They will work closely with the data science team to ensure data is properly collected, stored, and analyzed for business insights.

Responsibilities:

• Design and implement data pipelines using technologies such as Databricks, Snowflake, and Spark
• Work with large datasets and perform data modeling, warehousing, and analysis
• Utilize AWS technologies like S3, AWS Glue, and EMR for data processing and storage
• Develop and optimize queries in SQL, SparkSQL, or Scala for data analysis
• Handle both structured and unstructured datasets in relational databases like Teradata
• Collaborate with the data science team to identify trends and anomalies in the data
• Implement and maintain IAM roles and permissions for data access

Requirements:

• Required Skills: 5+ years of experience in data modeling, warehousing, and analysis
• Experience with AWS technologies like S3, AWS Glue, and EMR
• Proficiency in at least one query language (e.g., SQL, SparkSQL, Scala)
• Experience working with relational databases and handling structured and unstructured datasets
• Bachelor's or Master's degree in Data Science, Statistics, Computer Science, or Mathematics
• Experience with scripting languages such as Python or KornShell

Preferred Skills:

• Proficiency in Python
• Experience with NoSQL databases or data stores
• Experience with Snowflake and Databricks

Good to Have:

• Proficiency in data visualization tools like Tableau or Power BI
• Understanding of financial industry metrics and business processes
• AWS Cloud Practitioner or Developer certifications