Senior Data Engineer - AI & Analytics Pipelines

Mindex; City of Rochester (NY); On-site; USD 90,000 - 140,000; Full time

Benefits offered by this job

Health insurance

Paid holidays

Flexible time off

401k retirement plan with companymatch

Dental insurance

Vision insurance

Job summary

Mindex, a software development and cloud services company with 30+ years of success, seeks a Data Engineer to design, develop, and optimize data platforms for analytics, reporting, ML and AI initiatives.

You’ll work with cross-functional teams to influence architecture decisions and translate complex concepts for clients and stakeholders. Strong Python/SQL and big data experience are essential.

Qualifications

• BS degree in Computer Science, Data Science, Engineering, or equivalent software/services experience required.
• 4+ years working with SQL, Snowflake, Databricks, Spark, and related big data technologies; 4+ years using Python, SQL, PySpark, R, or similar languages.
• 4+ years building and optimizing data pipelines, architectures, and datasets.
• 2+ years supporting large-scale data processing and storage using Azure Data Factory, Databricks, Spark, Cosmos DB.
• 2+ years addressing privacy, compliance, and security aspects of data storage/processing; Agile environments.
• 2+ years with software development and CI/CD tools for automated infrastructure code and MLOps.

Responsibilities

• Assemble large, complex data sets from diverse sources through ETL/ELT processes.
• Provide operational support and troubleshooting for existing data processes.
• Collaborate with architects, data owners, data scientists, and stakeholders to groom data platform backlog and plan features.
• Deliver automated, efficient processes to ensure high throughput and performance of the data/analytics platform.
• Ensure data extraction, transformation, and loading meet security and compliance requirements.
• Engage with data source platform leads to align with AI/ML and Data Office standards.
• Create data tools for data scientists to build and optimize models.

Skills

SQL

Python

PySpark

R

ETL

Data visualization

Education

BS in CS/Data Science/Engineering

Tools

Snowflake

Databricks

Spark

Hadoop

Azure Data Factory

Azure Data Lake

Cosmos DB

Azure ML

Salesforce FSC

Salesforce Data Cloud

CI/CD tools (Ansible Tower, Terraform, Azure DevOps, GitHub Actions)

AWS

Job description

Mindex, a software development and cloud services company with 30+ years of success, seeks a Data Engineer to design, develop, and optimize data platforms for analytics, reporting, ML and AI initiatives.

You’ll work with cross-functional teams to influence architecture decisions and translate complex concepts for clients and stakeholders. Strong Python/SQL and big data experience are essential.