Data Engineer Databricks / Python / Database

Artifintech; Jersey City (NJ); On-site; USD 55,000 - 96,000; Full time

Job summary

ARTIFICIAL INTELLIGENCE TECHNOLOGIES LLC in Jersey City, NJ is seeking an experienced Data Engineer to design, build, and optimize scalable data pipelines using Databricks, Python, and Spark. You will work across structured and unstructured data sources, ensuring data quality and governance while supporting analytics and ML workloads.

The role requires strong SQL, ETL/ELT expertise, and experience with cloud data platforms (AWS/Azure/GCP).

Qualifications

• Experience designing and maintaining ETL/ELT pipelines on Databricks.
• Strong Python and Spark programming skills.
• Extensive SQL, query tuning, and schema design expertise.
• Experience with large-scale datasets and NoSQL/SQL databases.
• Ability to implement data quality, validation, and monitoring frameworks.
• Experience with cloud platforms (AWS/Azure/GCP) and data integration tools.

Responsibilities

• Design, build, and maintain ETL/ELT workflows on Databricks.
• Develop scalable data pipelines using Python and Spark.
• Optimize data transformation logic for performance and cost efficiency.
• Work with structured and unstructured data from various sources.
• Develop and manage data models, schemas, and tables across databases (SQL/NoSQL).
• Implement data quality, validation, and monitoring frameworks.
• Collaborate with Data Scientists, Analysts, and Cloud Engineers to support analytics and ML workloads.
• Ensure data security, governance, and compliance in all solutions.
• Troubleshoot pipeline failures and continuously improve pipeline reliability.
• Integrate data from APIs, streaming sources, and cloud storage.

Skills

SQL

Query tuning

Schema design

Stored procedures

Cloud platforms

ETL/ELT tools

Data processing at scale

Tools

ADF

Glue

Dataflow

Big data frameworks

Job description

Location:

Jersey city, NJ

Duration:

12Months+ Extension

Hourly Rate:

Depending on Experience (DOE)

Work Authorization:

Overview: We are seeking an experienced Data Engineer with strong expertise in Databricks, Python, and Database engineering. The ideal candidate will design, build, and optimize scalable data pipelines, ensure data quality, and support advanced analytics and business intelligence initiatives across the organization.

Key Responsibilities:

• Design, build, and maintain ETL/ELT workflows on Databricks.
• Develop scalable data pipelines using Python and Spark.
• Optimize data transformation logic for performance and cost efficiency.
• Work with structured and unstructured data from various sources.
• Develop and manage data models, schemas, and tables across databases (SQL/NoSQL).
• Implement data quality, validation, and monitoring frameworks.
• Collaborate with Data Scientists, Analysts, and Cloud Engineers to support analytics and ML workloads.
• Ensure data security, governance, and compliance in all solutions.
• Troubleshoot pipeline failures and continuously improve pipeline reliability.
• Integrate data from APIs, streaming sources, and cloud storage.

Required Skills: Core Technical Skills:-

• Database Engineering – Strong SQL, query tuning, schema design, stored procedures
• Cloud Platforms (one or more): AWS / Azure / GCP
• ETL/ELT tools: ADF, Glue, Dataflow, or equivalent
• Experience working with large-scale datasets

ARTIFICIAL INTELLIGENCE TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. ARTIFICIAL INTELLIGENCE TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will ARTIFICIAL INTELLIGENCE TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract