Compare commits

...

1 Commits

Author SHA1 Message Date
Matt Williams
e2389b63aa add examples of streaming in python and node
Signed-off-by: Matt Williams <m@technovangelist.com>
2023-09-14 07:12:09 -07:00
2 changed files with 39 additions and 0 deletions

View File

@ -20,6 +20,10 @@ Model names follow a `model:tag` format. Some examples are `orca-mini:3b-q4_1` a
All durations are returned in nanoseconds.
### Streams
Many API responses are streams of JSON objects showing the current status. For examples of working with streams in various languages, see [streaming.md](./streaming.md)
## Generate a completion
```

35
docs/streaming.md Normal file
View File

@ -0,0 +1,35 @@
# Streaming responses in the Ollama Client API
## JavaScript / TypeScript / Deno
```javascript
const pull = async () => {
const request = await fetch("http://localhost:11434/api/pull", {
method: "POST",
body: JSON.stringify({ name: "llama2:7b-q5_0" }),
});
const reader = await request.body?.pipeThrough(new TextDecoderStream());
if (!reader) throw new Error("No reader");
for await (const chunk of reader) {
const out = JSON.parse(chunk);
if (out.status.startsWith("downloading")) {
console.log(`${out.status} - ${(out.completed / out.total) * 100}%`);
}
}
}
pull();
```
## Python
```python
import requests
import json
response = requests.post("http://localhost:11434/api/pull", json={"name": "llama2:7b-q5_0"}, stream=True)
for data in response.iter_lines():
out = json.loads(data)
if "completed" in out:
print(out["completed"] / out["total"] * 100)
```