Ollama 是一个基于 Go 语言开发的可以本地运行大模型的开源框架。

官网：ollama.com/

GitHub 地址：github.com/ollama/olla…

在 Ollama 官网根据操作系统类型选择对应的安装包，这里选择 macOS 下载安装。 安装完在终端输入 ollama，可以看到 ollama 支持的命令。

bash 代码解读 复制代码Usage: ollama [flags] ollama [command] Available Commands: serve Start ollama create Create a model from a Modelfile show Show information for a model run Run a model pull Pull a model from a registry push Push a model to a registry list List models cp Copy a model rm Remove a model help Help about any command Flags: -h, --help help for ollama -v, --version Show version information Use "ollama [command] --help" for more information about a command. 

查看 ollama 版本

bash 代码解读 复制代码ollama -v ollama version is 0.1.31 

查看已下载模型

bash 代码解读 复制代码ollama list NAME ID SIZE MODIFIED gemma:2b b50d6c999e59 1.7 GB 3 hours ago 

我本地已经有一个大模型，接下来我们看一下怎么下载大模型。

安装完后默认提示安装 llama2 大模型，下面是 Ollama 支持的部分模型

Llama 3 是 Meta 2024年4月19日 开源的大语言模型，共80亿和700亿参数两个版本，Ollama均已支持。

这里选择安装 gemma 2b，打开终端，执行下面命令：

shell 代码解读 复制代码ollama run gemma:2b bash 代码解读 复制代码pulling manifest pulling c1864a5eb193... 100% ▕██████████████████████████████████████████████████████████▏ 1.7 GB pulling 097a36493f71... 100% ▕██████████████████████████████████████████████████████████▏ 8.4 KB pulling bec39c... 100% ▕██████████████████████████████████████████████████████████▏ 136 B pulling 22a838ceb7fb... 100% ▕██████████████████████████████████████████████████████████▏ 84 B pulling b89a90... 100% ▕██████████████████████████████████████████████████████████▏ 483 B verifying sha256 digest writing manifest removing any unused layers success 

经过一段时间等待，显示模型下载完成。

上表仅是 Ollama 支持的部分模型，更多模型可以在 ollama.com/library 查看，中文模型比如阿里的通义千问。

下载完成后，可以直接在终端进行对话，比如提问“介绍一下React”

bash 代码解读 复制代码>>> 介绍一下React 

输出内容如下：

markdown 代码解读 复制代码>>> /? Available Commands: /set Set session variables /show Show model information /load 
  
    
    
      Load a session or model /save 
     
       Save your current session /bye Exit /?, /help Help for a command /? shortcuts Help for keyboard shortcuts Use """ to begin a multi-line message. 
      
    

bash 代码解读 复制代码>>> /show Available Commands: /show info Show details for this model /show license Show model license /show modelfile Show Modelfile for this model /show parameters Show parameters for this model /show system Show system message /show template Show prompt template 

bash 代码解读 复制代码>>> /show info Model details: Family gemma Parameter Size 3B Quantization Level Q4_0 

除了在终端直接对话外，ollama 还可以以 API 的方式调用，比如执行 ollama show --help 可以看到本地访问地址为：http://localhost:11434

bash 代码解读 复制代码ollama show --help Show information for a model Usage: ollama show MODEL [flags] Flags: -h, --help help for show --license Show license of a model --modelfile Show Modelfile of a model --parameters Show parameters of a model --system Show system message of a model --template Show template of a model Environment Variables: OLLAMA_HOST The host:port or base URL of the Ollama server (e.g. http://localhost:11434) 

下面介绍主要介绍两个 api ：generate 和 chat。

• 流式返回

json

代码解读 复制代码curl http://localhost:11434/api/generate -d ‘{ “model”: “gemma:2b”, “prompt”:“介绍一下React，20字以内” }’ json

代码解读 复制代码{“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:32.Z”,“response”:“React”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:32.Z”,“response”:” 是”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:32.Z”,“response”:“一个”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:32.Z”,“response”:“用于”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:32.Z”,“response”:“构建”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:32.Z”,“response”:“用户”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:32.Z”,“response”:“界面”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:32.Z”,“response”:“（”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:32.Z”,“response”:“UI”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:33.080361Z”,“response”:“）”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:33.Z”,“response”:“的”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:33.Z”,“response”:” JavaScript”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:33.Z”,“response”:” 库”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:33.Z”,“response”:“。”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:33.Z”,“response”:“它”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:33.Z”,“response”:“允许”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:33.Z”,“response”:“开发者”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:33.Z”,“response”:“轻松”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:34.007706Z”,“response”:“构建”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:34.09201Z”,“response”:“可”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:34.Z”,“response”:“重”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:34.Z”,“response”:“用的”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:34.Z”,“response”:” UI”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:34.Z”,“response”:“，”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:34.Z”,“response”:“并”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:34.Z”,“response”:“与”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:34.Z”,“response”:“各种”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:34.Z”,“response”:” JavaScript”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:35.008527Z”,“response”:” “,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:35.088936Z”,“response”:“框架”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:35.Z”,“response”:“一起”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:35.Z”,“response”:“使用”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:35.34085Z”,“response”:“。”,“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T10:12:35.Z”,“response”:“”,“done”:true,“context”:[106,1645,108,25661,18071,22469,,,,,,107,108,106,2516,108,22469,23437,5121,40163,81964,16464,57881,,5639,,,22978,,,,64032,,64727,81964,,,37694,13566,,,,34384,22978,,90141,19600,7060,,107,108],“total_duration”:,“load_duration”:,“prompt_eval_duration”:,“eval_count”:34,“eval_duration”:}

• 非流式返回

通过设置 “stream”: false 参数可以设置一次性返回。

“bash curl http://localhost:11434/api/generate -d ‘{ “model”: “gemma:2b”, “prompt”:“介绍一下React，20字以内”, “stream”: false }’

json

代码解读 复制代码 “`json { “model”: “gemma:2b”, “created_at”: “2024-04-19T08:53:14.Z”, “response”: “React 是一个用于构建用户界面的大型 JavaScript 库，允许您轻松创建动态的网站和应用程序。”, “done”: true, “context”: [106, 1645, 108, 25661, 18071, 22469, , , , , , 107, 108, 106, 2516, 108, 22469, 23437, 5121, 40163, 81964, 16464, , 26546, 66240, 22978, , , 64032, , 64727, 22957, 80376, , 37188, , 79826, , 107, 108], “total_duration”: , “load_duration”: , “prompt_eval_duration”: , “eval_count”: 23, “eval_duration”: }

• 流式返回

bash

代码解读 复制代码curl http://localhost:11434/api/chat -d ‘{ “model”: “gemma:2b”, “messages”: [

{ "role": "user", "content": "介绍一下React，20字以内" } 

] }’

可以看到终端输出结果：

bash

代码解读 复制代码{“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:54.86791Z”,“message”:{“role”:“assistant”,“content”:“React”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:54.Z”,“message”:{“role”:“assistant”,“content”:“是”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:55.034272Z”,“message”:{“role”:“assistant”,“content”:“用于”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:55.Z”,“message”:{“role”:“assistant”,“content”:“构建”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:55.Z”,“message”:{“role”:“assistant”,“content”:“用户”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:55.Z”,“message”:{“role”:“assistant”,“content”:“界面”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:55.37054Z”,“message”:{“role”:“assistant”,“content”:” React”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:55.45099Z”,“message”:{“role”:“assistant”,“content”:“.”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:55.Z”,“message”:{“role”:“assistant”,“content”:“js”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:55.Z”,“message”:{“role”:“assistant”,“content”:“框架”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:55.Z”,“message”:{“role”:“assistant”,“content”:“，”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:55.Z”,“message”:{“role”:“assistant”,“content”:“允许”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:55.Z”,“message”:{“role”:“assistant”,“content”:“开发者”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:55.Z”,“message”:{“role”:“assistant”,“content”:“轻松”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:56.012203Z”,“message”:{“role”:“assistant”,“content”:“地”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:56.098045Z”,“message”:{“role”:“assistant”,“content”:“创建”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:56.Z”,“message”:{“role”:“assistant”,“content”:“动态”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:56.Z”,“message”:{“role”:“assistant”,“content”:“网页”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:56.Z”,“message”:{“role”:“assistant”,“content”:“。”},“done”:false} {“model”:“gemma:2b”,“created_at”:“2024-04-19T08:45:56.Z”,“message”:{“role”:“assistant”,“content”:“”},“done”:true,“total_duration”:,“load_duration”:,“prompt_eval_count”:11,“prompt_eval_duration”:,“eval_count”:20,“eval_duration”:}

默认流式返回，同样可以通过 “stream”: false 参数一次性返回。

generate 和 chat 的区别在于，generate 是一次性生成的数据。chat 可以附加历史记录，多轮对话。

除了上面终端和 API 调用的方式，目前还有许多开源的 Web UI，可以本地搭建一个可视化的页面来实现对话，比如：

• open-webui

github.com/open-webui/…

• lollms-webui

github.com/ParisNeo/lo…