Cloud Data Engineer with Databricks

Diaconia; United States; Remote; USD 110,000 - 150,000; Full time

Job summary

Diaconia is hiring a full-time Cloud Data Engineer with Databricks experience to design and maintain data infrastructure. This role involves developing data pipelines, ensuring data quality, and supporting compliance in the financial sector. Candidates should have a relevant degree, certifications, and at least 3 years of experience.

Qualifications

• U.S. Citizenship required.
• Databricks Certified Data Engineer Professional certification needed.
• At least 3 years of relevant experience.

Responsibilities

• Develop and maintain data pipelines and analytics platforms.
• Design and implement data transformation processes.
• Collaborate with cross-functional teams.

Skills

Data lake architecture

Databricks

Python

SQL

ETL pipelines

Data governance

Troubleshooting

Education

Bachelor’s or higher degree in computer science, data engineering, or related field

Tools

Apache Airflow

GitHub

AWS

Job description

Diaconia is seeking a talented Cloud Data Engineer with Databricks experience to join our team!

We are committed to supporting our colleagues, clients, and community. Our team values diverse thinking, collaboration, and making a difference. We are currently hiring a full-time Cloud Data Engineer with Databricks.

Job Summary: The Cloud Data Engineer will design, build, and maintain data infrastructure and ETL pipelines using Databricks and cloud technologies. This role supports data-driven decision-making and regulatory compliance.

Key Responsibilities:

1. Develop and maintain data pipelines and analytics platforms; integrate data sources.
2. Prototype solutions for emerging business use cases.
3. Design and implement data transformation and migration processes, especially from on-premises to AWS.
4. Ensure data quality, security, and compliance, maintaining documentation.
5. Collaborate with cross-functional teams to meet data requirements.
6. Troubleshoot data issues and stay updated on industry trends.

Minimum Qualifications:

• U.S. Citizenship (per Federal requirements).
• Bachelor’s or higher degree in computer science, data engineering, or related field.
• Databricks Certified Data Engineer Professional certification.
• At least 3 years of experience with:
• Data lake, warehouse, and lakehouse architectures in cloud environments.
• Databricks, Python, SQL, and cloud services like AWS (S3, EC2, RDS, Lambda, etc.).
• Building scalable ETL pipelines and data models.
• Data pipeline automation tools such as Apache Airflow or AWS Step Functions.
• Data governance, troubleshooting, and code management (GitHub).

Preferred Skills:

• Experience with financial/regulatory data, Agile/DevSecOps, DevOps, machine learning, and AI.

Clearance: Must obtain and maintain a Public Trust clearance and be a verifiable US Citizen.

This role is vital for ensuring data accuracy, integrity, and compliance, supporting the agency’s mission to protect consumers in the financial sector.

Note: Applicant will be subject to security investigation. Diaconia is an Equal Opportunity Employer. Salary: $110,000 - $150,000 depending on experience.