Skip to main content

Cortex

warning

🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.

Cortex Cover Image

Cortex.cpp lets you run AI easily on your computer.

Cortex.cpp is a C++ command-line interface (CLI) designed as an alternative to Ollama. By default, it runs on the llama.cpp engine but also supports other engines, including ONNX and TensorRT-LLM, making it a multi-engine platform.

Supported Accelerators​

  • Nvidia CUDA
  • Apple Metal
  • Qualcomm AI Engine

Supported Inference Backends​

If GPU hardware is available, Cortex is GPU accelerated by default.

info

Real-world Use: Cortex.cpp powers Jan, our on-device ChatGPT-alternative.

Cortex.cpp has been battle-tested across 1 million+ downloads and handles a variety of hardware configurations.

Supported Models​

Cortex.cpp supports the following list of Built-in Models:

Model IDVariant (Branch)Model sizeCLI command
codestral22b-gguf22Bcortex run codestral:22b-gguf
command-r35b-gguf35Bcortex run command-r:35b-gguf
gemma7b-gguf7Bcortex run gemma:7b-gguf
llama3gguf8Bcortex run llama3:gguf
llama3.1gguf8Bcortex run llama3.1:gguf
mistral7b-gguf7Bcortex run mistral:7b-gguf
mixtral7x8b-gguf46.7Bcortex run mixtral:7x8b-gguf
openhermes-2.57b-gguf7Bcortex run openhermes-2.5:7b-gguf
phi3medium-gguf14B - 4k ctx lencortex run phi3:medium-gguf
phi3mini-gguf3.82B - 4k ctx lencortex run phi3:mini-gguf
qwen27b-gguf7Bcortex run qwen2:7b-gguf
tinyllama1b-gguf1.1Bcortex run tinyllama:1b-gguf
info

Cortex.cpp supports pulling GGUF and ONNX models from the Hugging Face Hub. Read how to Pull models from Hugging Face

Cortex.cpp Versions​

Cortex.cpp offers three different versions of the app, each serving a unique purpose:

  • Stable: The official release version of Cortex.cpp, designed for general use with proven stability.
  • Beta: This version includes upcoming features still in testing, allowing users to try new functionality before the next official release.
  • Nightly: Automatically built every night, this version includes the latest updates and changes from the engineering team but may be unstable.
info

Each of these versions has a different CLI prefix command.