Skip to main content

Hugging Face

warning

🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.

Cortex.cpp supports all GGUF and ONNX models from the Hugging Face Hub, along with its built-in models. For TensorRT-LLM models, only built-in models in the Cortex Model Repos are supported.

info

To pull a supported model from HuggingFace, use the format ORG_ID/MODEL_ID.

GGUF​

HF GGUF To view all available GGUF models on HuggingFace, select the GGUF tag in the Libraries section.


# Pull the Codestral-22B-v0.1-GGUF model from the bartowski organization
cortex pull bartowski/Codestral-22B-v0.1-GGUF
# Pull the gemma-7b model from the google organization
cortex pull google/gemma-7b

ONNX​

HF ONNX To view all available ONNX models on HuggingFace, select the ONNX tag in the Libraries section.


# Pull the XLM-Roberta-Large-Vit-B-16Plus model from the immich-app organization
cortex pull immich-app/XLM-Roberta-Large-Vit-B-16Plus
# Pull the mt0-base model from the bigscience organization
cortex pull bigscience/mt0-base

TensorRT-LLM​

We are still working to support all available TensorRT-LLM models on HuggingFace. For now, Cortex.cpp only supports built-in TensorRT-LLM models, which can be downloaded from the Cortex Model Repos.