Databricks Data Engineer

Square Peg Technologies; Washington (District of Columbia); Sur place; USD 110 000 - 140 000; Plein temps

Résumé du poste

Square Peg Technologies is seeking a Data Engineer to design, implement, and maintain data pipelines for diverse datasets and customer solutions. The role leverages Databricks and cloud data management to enable AI/ML initiatives.

The candidate will participate in extensive training and must bring proficiency in Python/SQL/Java, API development, and data modeling, with experience in cloud platforms like AWS and Snowflake.

Qualifications

• Proficient in Python, SQL or Java and API development.
• Experience with big data modeling and data pipelines.
• Familiar with cloud data platforms including Databricks and AWS.
• Knowledge of SQL databases and data mapping.
• Experience with Spark/Hadoop for large-scale processing.

Responsabilités

• Design, implement, and maintain data pipelines for customers.
• Develop automated pipelines for diverse data types and datasets.
• Leverage Databricks and related cloud platforms to solve data problems.
• Participate in training programs to stay current with AI/ML technologies.

Connaissances

Python

SQL

Java

API development

Data modeling

Outils

Databricks

Splunk

Cloudera

Snowflake

AWS

Description du poste

Job Description

Clearance Required:Position Requires a Top Secret (TS/SCI) Clearance with a Polygraph.Location:Northern Virginia

The Mission

Square Peg is seeking a Data Engineer to be responsible for designing, implementing, and maintaining data pipelines for our customers. The Data Engineer will develop automated pipelines for a variety of data types and datasets. Using Databricks, which was named to both the Magic Quadrant for Cloud Database Management Systems and the Magic Quadrant for Data Science and Machine Learning Platforms, you'll using best in class technology to help our customer solve some of their most challenging data problems.

Don't know Databricks, no problem! Sign up for our extensive training program to be a part of shaping the future of AI/MLBasic Qualifications:

• Proficient with one or more programming languages (Python,SQL, Java)
• Prepare big data, implement data models and develop database to support the business solutions
• Understanding of SQL database structures and mapping data structures between different SQL databases
• Knowledge of API development techniques
• Demonstrated experience in data engineering, data processing and modeling techniques using cloud-based data management, data science, and ML platforms such as Databricks, Splunk, Cloudera, and Snowflake
• Demonstrated experience with cloud based database services such as Databricks and AWS

Preferred Qualifications

• Experience with usingApache Spark and Hadoop to conduct large-scale data processing
• Extensive experience in Extract-Transform-Load (ETL) processes
• Demonstrated extensive experience with cloud services (AWS strongly preferred)
• Familiarity with open source data engineering tools and processes

Square Peg Technologies Company Information

Square Peg Technologies is a boutique technology consulting firm that specializes in data solutions using the latest technologies in analytics, data science, data engineering and data management. Our mission is to develop and integrate cutting edge technology solutions that inspire our customers to push the fields of science and technology forward. Here at Square Peg, you will not get lost in a crowd. We value each person’s contribution, celebrate wins together and invest in the personal and professional development of our employees. Along with a strong positive culture that allows employees to thrive and grow, we also offer top notch benefits to include a large amount of vacation, annual bonuses, 401k matching, paid training, and 100% health care coverage.

At Square Peg Tech, we aim to be at the frontier of technology and innovation. Our ideal candidate must be forward leaning and eager to engage with customers, academia and industry leaders on the future applications of AI/ML, as well as best practices in advanced analytics. With a firm footprint in theIntelligenceCommunity, Square Peg Tech is looking to expand its capabilities into the Department of Defense, NASA and other prominent government agencies.