Databricks Data Engineer

Data Based Development Systems; Farmington (CT); On-site; USD 120,000 - 180,000; Full time

Job summary

Talantir in Farmington, CT is seeking a software engineer/developer or ETL/data integration/big data developer to support data ingestion, transformation, and distribution for end consumers. You will work with BI and stakeholders to design the data warehouse architecture and implement scalable data solutions across real-time and batch workflows.

The role requires experience with Python, SQL, PLSQL, VB, Oracle/SQL Server/MySQL, Synapse, Databricks, AWS and Azure cloud services, and data

Qualifications

• Proficiency in Python, SQL, PLSQL, and VB for data processing.
• Experience with Oracle, SQL Server, MySQL databases.
• Knowledge of Synapse, Databricks, and cloud platforms (AWS, Azure).
• Experience with HVR data replication and data warehousing concepts.

Responsibilities

• Design and implement ETL/ELT pipelines for real-time and batch workloads.
• Collaborate with BI and data stakeholders to surface accessible data assets.
• Develop data models, database objects, views, and stored procedures.
• Maintain documentation and troubleshoot end-to-end data issues.

Skills

Python

SQL

PLSQL

VB

Tools

Oracle

SQL Server

MySQL

Synapse

Databricks

AWS

Azure

HVR

Job description

Must be able to be onsite in Farmington, CT 2 days a week for collaboration

The Opportunity: We are seeking a software engineer/developer or ETL/data integration/big data developer with experience in projects emphasizing data processing and storage. This person will be responsible for supporting the data ingestion, transformation, and distribution to end consumers. Candidate will perform requirements analysis, design/develop process flow, unit and integration tests, and create/update process documentation.

• Work with the Business Intelligence team and operational stakeholders to design and implement both the data presentation layer available to the user community, as well as the underlying technical architecture of the data warehousing environment.
• Develop scalable and reliable data solutions to move data across systems from multiple sources in real time as well as batch modes.
• Design and develop database objects, tables, stored procedures, views, etc.
• Independently analyze, solve, and correct issues in real time, providing problem resolution end-to-end
• Design and develop ETL Processes that will transform a variety of raw data, flat files, xl spreadsheets into SQL Databases
• Understands the concept of Data marts and Data lakes and experience with migrating legacy systems to data marts/lake
• Uses additional cloud technologies (e.g., understands concept of Cloud services like Azure SQL server)
• Maintain comprehensive project documentation
• Aptitude to learn new technologies and the ability to perform continuous research, analysis, and process improvement.
• Strong interpersonal and communication skills to be able to work in a team environment to include customer and contractor technical, end users, and management team members.
• Manage multiple projects, responsibilities and competing priorities.

Requirements Experience Needed

• Programming languages, frameworks, and file formats such as: Python, SQL, PLSQL, and VB
• Database platforms such as: Oracle, SQL Server, MySQL
• Big data concepts and technologies such as Synapse & Databricks
• AWS and Azure cloud computing
• HVR data replication