Senior Data Engineer - Informatica & PySpark Expert

ThoughtStorm Inc.

Cleveland (OH)

On-site

USD 110,000 - 170,000

Full time

8 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

ThoughtStorm Inc. is seeking an experienced data engineer to drive data pipelines, analytics, and ETL architectures. You will design and optimize large-scale data processes, working across teams to deliver reliable data products.

You will leverage Informatica PowerCenter/IDQ, PySpark, and Hadoop ecosystem tools, collaborating with stakeholders and applying agile practices. Experience with AWS data services and ML concepts is advantageous.

Qualifications

8–10 years in data engineering and data analysis.
Strong hands-on Informatica PowerCenter/IDQ for ETL design, development, and optimization.
Advanced PySpark development for large-scale processing, transformation, and analytics.
Solid Hadoop ecosystem knowledge (HDFS, Hive, Sqoop, MapReduce).
Proficiency in Python and Kafka for streaming and batch pipelines.
Well-versed in data modeling, ETL workflows, data quality checks, and loading.
Experience with Agile delivery and Jira for project coordination.
Client-facing roles with clear communication and leadership across SDLC.
Exposure to AWS data components and analytics; familiarity with ML concepts.
Experience with Erwin as a data modeling tool.