Software Engineer Intern (AI Engineer)

Grayhat Developers Private Limited; Islamabad; On-site; PKR 446,000 - 670,000; Full time

Benefits offered by this job

Paid internship

Mentorship from engineers

Portfolio-building experience

Job summary

Grayhat Developers Private Limited is seeking a curious intern to join our AI infrastructure team. You will work directly on our self-hosted LLM stack, addressing real production challenges in model performance and usability.

You will build tooling, explore quantization and other optimization methods, and collaborate with the broader engineering team to shape how we deploy AI internally. This is an onsite role with hands-on learning and mentorship.

Qualifications

• Solid Python skills and comfort with Linux/server environments.
• Solid understanding of how transformer-based LLMs work, not just prompting.
• Exposure to LLM serving frameworks like Ollama, vLLM, llama.cpp, or TGI.
• Genuine interest in model optimization and quantization techniques.

Responsibilities

• Dig into bottlenecks in our self-hosted LLM stack around latency and hardware utilization.
• Build internal tools and interfaces for non-technical users to interact with the AI stack.
• Experiment with model optimization techniques like quantization, pruning, and batching within hardware constraints.
• Identify gaps in tooling and create lightweight integrations or wrappers to close them.
• Maintain clear engineering notes so work is reproducible for the next engineer.

Skills

Python

Linux/server

Transformer-based LLMs

Builder mindset

Education

CS/AI/Data Science student or fresh graduate

Tools

Ollama

vLLM

llama.cpp

TGI

Job description

Looking to work on practical AI systems that go beyond tutorials and actually reach users? We're looking for curious builders excited about AI, rapid iteration, and solving real product problems.

The stepping stone for 80% of engineers at Grayhat.

The Role

We're building an AI infrastructure from the ground up, and this internship is a founding piece of that. You'll work directly on our self-hosted LLM infrastructure, solving real problems around model performance, usability, and adoption. If you're excited about the nuts and bolts of running LLMs in production (not just prompting them), this is your role.

What You'll Do

• Dig into bottlenecks in our self-hosted LLM stack around latency, throughput, and hardware utilization.
• Build internal tools and interfaces that make our AI stack easier for non-technical team members to actually use.
• Experiment with model optimization techniques like quantization, pruning, and batching strategies, all within our hardware constraints.
• Identify gaps in how our tools are being used and build lightweight integrations or wrappers to close them.
• Keep clear engineering notes so your work is reproducible and the next person can pick up where you left off.
• Talk to the broader engineering team, understand their AI needs, and build toward them.

What We're Looking For

• Solid Python skills and comfort working in a Linux/server environment.
• A real understanding of how transformer-based LLMs work, not just how to prompt them.
• Some exposure to LLM serving frameworks like Ollama, vLLM, llama.cpp, or TGI.
• Genuine interest in model optimization. You know what quantization means and you're not afraid to get into it.
• A builder's mindset. You don't wait for perfect tooling, you make do and move forward.
• Second year, third year, final year student (CS, AI, Data Science, or related) or fresh graduate.
• We prefer onsite people for this role.

Nice to Have

• Experience with on-prem or edge AI deployments, not just cloud APIs.
• Familiarity with FastAPI or similar frameworks for wrapping model endpoints.
• Prior work with ONNX, llama.cpp, or hardware-specific inference optimization.
• Some understanding of RAG pipelines or agent frameworks like LangChain or LangGraph.

What You'll Get

• A monthly stipend. This is a paid internship.
• Hands-on experience running and optimizing production LLM infrastructure. This is not a "call the API and move on" role.
• The chance to directly shape how an entire studio uses AI tooling.
• Mentorship from engineers who've built across the full product stack.
• Work that genuinely stands out in a portfolio. Self-hosted AI infrastructure experience is rare at this level.
• A potential full-time offer if you stand out.