Senior Data Engineer: AWS Lakehouse, Streaming & Analytics

Sahi Softtech, Inc; United States; On-site; USD 130,000 - 180,000; Full time

Job summary

Sahi Softtech, Inc. is seeking a Staff Data Engineer to design and implement scalable data pipelines and data platforms. The role involves building streaming and batch ingestion, data lakes on AWS, and analytics using Spark, Snowflake, and BI tools.

You will participate in architecture discussions and in agile development processes to deliver enterprise-grade solutions. The candidate should have extensive AWS experience, strong SQL/NoSQL skills, and a background in data warehousing and

Qualifications

• 8 years of experience in Enterprise Information Solution Architecture, Design, and development required.
• 8 years of hands-on experience with AWS services ( S3, Kinesis, Lambda, Athena, Glue, EMR) required.
• 8 years of experience with integration architectures such as ETL, SOA, Microservices or other integration technologies.
• 5 years of Web development experience with Python and Node.js.
• 3 years of experience building cloud data warehouse using Snowflake and/or Redshift.
• 3 years of NoSQL database DynamoDB, Marklogic, MangoDB.
• 3 years of Business Intelligence applications like Qlik Sense, Tableau, or PowerBI.
• 3 years of analytics tools like SAS, R, Python, and other advanced statistical software.
• Experience with Machine Learning and Artificial Intelligence.
• Experience with DevOps and Terraform.
• Ability to multi-task effectively.
• Ability to work collaboratively as part of an Agile Team.
• Knowledge of multiple programming environments, debugging tools, and test tools required.
• Domain knowledge of NoSQL or relational database required.
• Experience developing enterprise grade production software required.
• Understanding of database architecture and performance implications required.
• Excellent written and verbal communication skills, sense of ownership, urgency and drive.

Responsibilities

• Designs and develops code and data pipelines to ingest from relational databases (Oracle, SQL Server, DB2, Aurora), file shares, and web services.
• Streaming ingestion with Kinesis Streams, Kinesis Firehose, Kinesis Analytics and Kafka(MSK)
• Build Data Lake on AWS S3 with optimal performance considerations by partitioning and compressing data.
• Data Engineering and Analytics using AWS Glue, Informatica, EMR, Spark, Athena, Python, PowerBI and Sagemaker.
• Data modeling and building Data Warehouse using Snowflake.
• Participates in requirements definition, system architecture design, and data architecture design.
• Participates in all aspects of the software life cycle using Agile development methodologies.

Tools

AWS S3

Kinesis

Lambda

Athena

Glue

EMR

Snowflake

Redshift

DynamoDB

MarkLogic

MongoDB

PowerBI

Tableau

Qlik Sense

SAS

R

Python

Node.js

Terraform

Kafka

Snowflake

Job description

Sahi Softtech, Inc. is seeking a Staff Data Engineer to design and implement scalable data pipelines and data platforms. The role involves building streaming and batch ingestion, data lakes on AWS, and analytics using Spark, Snowflake, and BI tools.

You will participate in architecture discussions and in agile development processes to deliver enterprise-grade solutions. The candidate should have extensive AWS experience, strong SQL/NoSQL skills, and a background in data warehousing and