FitMyLLM
HuggingFace/Dense

HuggingFaceSmolLM2 360M

Compact SmolLM for mobile and embedded AI applications.

chatTool Use
0.36B
Parameters
8K
Context length
6
Benchmarks
6
Quantizations
150K
HF downloads
Architecture
Dense
Released
2024-11-21
Layers
32
KV Heads
5
Head Dim
64
Family
smollm

Quantization Options

Context length:
QuantBitsVRAM @ 8KQuality
Q4_K_M4.89
0.9 GB
0.7 + 0.2 KV
good
Q5_K_S5.57
0.9 GB
0.7 + 0.2 KV
good
Q5_K_M5.7
0.9 GB
0.7 + 0.2 KV
good
Q6_K6.56
0.9 GB
0.8 + 0.2 KV
excellent
Q8_08.5
1.0 GB
0.9 + 0.2 KV
lossless
FP1616
1.4 GB
1.2 + 0.2 KV
lossless

Select your GPU above to see speed estimates and compatibility for each quantization.

Deploying for a team or in production? Size GPUs, cost & scaling in Enterprise →
▸ READY TO RUN THIS?RENT BY THE HOUR

RENT A GPU AND RUN SMOLLM2 360M NOW

Rent on Vast.ai →Or RunPod →

Spin up an A100 / H100 / 4090 in ~60s. Pay by the second. Cancel anytime.

Community Ratings

Loading ratings...

Benchmarks (6)

IFEval38.4
BBH4.2
MUSR2.8
MATH1.5
MMLU-PRO1.3
GPQA0.7

Run this model

▸Easiest way to get started·Beginners
DOCS ↗
curl -fsSL https://ollama.com/install.sh | sh
$ollama run smollm:360m-base-v0.2-q4_K_M

Downloads and runs automatically. Add --verbose for speed stats.

▸ SETUP GUIDE
>_

Auto-setup with fitmyllm CLI

Detects your GPU, recommends the best model, downloads it, and starts chatting — zero config. Benchmarks your speed and contributes anonymous data to improve predictions.

pip install fitmyllmthen run fitmyllmLearn more
Auto-detect GPULive tok/s in chatSpeed benchmarks9 inference engines
HuggingFaceOllama LibraryGGUF DownloadsBuild Hardware

GPUs that can run this model

At Q4_K_M quantization. Sorted by minimum VRAM.

AMD FireGL V7350
1 GB VRAM • 41.6 GB/s
AMD
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA Quadro FX 5500
1 GB VRAM • 32.3 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA Quadro FX 5500 SDI
1 GB VRAM • 32.3 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
AMD Stream Processor
1 GB VRAM • 41.5 GB/s
AMD
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
AMD FireGL V8600
1 GB VRAM • 111.1 GB/s
AMD
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce 9600M GS
1 GB VRAM • 25.6 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce 9650M GT
1 GB VRAM • 25.6 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce 9800M GTS
1 GB VRAM • 51.2 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce 9800M GTX
1 GB VRAM • 51.2 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce GTX 280
1 GB VRAM • 141.7 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce GTX 285
1 GB VRAM • 159 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA Quadro FX 3700M
1 GB VRAM • 51.2 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA Quadro FX 3800M
1 GB VRAM • 64 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA Quadro FX 4700 X2
1 GB VRAM • 51.2 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
AMD FireStream 9250
1 GB VRAM • 63.5 GB/s
AMD
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
AMD FirePro V8700
1 GB VRAM • 108.8 GB/s
AMD
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
AMD Radeon HD 4570
1 GB VRAM • 8 GB/s
AMD
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
AMD Radeon HD 4710
1 GB VRAM • 25.6 GB/s
AMD
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
AMD Radeon HD 4730 OEM
1 GB VRAM • 25.3 GB/s
AMD
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
AMD Radeon HD 4870 X2
1 GB VRAM • 115.2 GB/s
AMD
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce 210 OEM
1 GB VRAM • 6.4 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce 210 Rev. 2
1 GB VRAM • 6.4 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce G110M
1 GB VRAM • 11.2 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce GT 120M
1 GB VRAM • 16 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce GT 130M
1 GB VRAM • 25.6 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce GT 140 OEM
1 GB VRAM • 57.6 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce GT 220
1 GB VRAM • 22.4 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce GT 220M
1 GB VRAM • 16 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai
NVIDIA GeForce GT 230M
1 GB VRAM • 25.3 GB/s
NVIDIA
AmazonNeweggeBayPCPartPickerRent on RunPodRent on Vast.ai

Find the best GPU for SmolLM2 360M

Build Hardware for SmolLM2 360M

Model Card

View on HuggingFace

Compact SmolLM for mobile and embedded AI applications.

▸ SPEC SHEET

SmolLM2 360M — 0.36B Dense.

▸ SPECIFICATIONS
PARAMETERS
0.36B
ARCHITECTURE
Dense Transformer
CONTEXT LENGTH
8K tokens
CAPABILITIES
chat
RELEASE DATE
2024-11-21
PROVIDER
HuggingFace
FAMILY
smollm
▸ VRAM REQUIREMENTS
QUANTBPWVRAMQUALITY
Q4_K_M4.890.7 GB94%
Q5_K_S5.570.7 GB96%
Q5_K_M5.70.7 GB96%
Q6_K6.560.8 GB97%
Q8_08.50.9 GB100%
FP16161.2 GB100%
§ 01BENCHMARK SCORES
MMLU-PRO1.3
MATH1.5
IFEval38.4
BBH4.2
GPQA0.7
MUSR2.8
§ 02RUN COMMAND

Run SmolLM2 360M locally with Ollama — needs 0.7 GB VRAM at Q4_K_M:

$ollama run smollm:360m
§ 03COMPATIBLE GPUs
30 @ Q4_K_M
AMD FireGL V7350
1 GB · 41.6 GB/s
NVIDIA Quadro FX 5500
1 GB · 32.3 GB/s
NVIDIA Quadro FX 5500 SDI
1 GB · 32.3 GB/s
AMD Stream Processor
1 GB · 41.5 GB/s
AMD FireGL V8600
1 GB · 111.1 GB/s
NVIDIA GeForce 9600 GT Mac Edition
1 GB · 17 GB/s
NVIDIA GeForce 9600M GS
1 GB · 25.6 GB/s
NVIDIA GeForce 9650M GT
1 GB · 25.6 GB/s
NVIDIA GeForce 9800M GTS
1 GB · 51.2 GB/s
NVIDIA GeForce 9800M GTX
1 GB · 51.2 GB/s
NVIDIA GeForce GTX 280
1 GB · 141.7 GB/s
NVIDIA GeForce GTX 285
1 GB · 159 GB/s
NVIDIA Quadro FX 3700M
1 GB · 51.2 GB/s
NVIDIA Quadro FX 3800M
1 GB · 64 GB/s
NVIDIA Quadro FX 4700 X2
1 GB · 51.2 GB/s