jmorganca
763d7b601c
sync
2024-09-03 21:15:13 -04:00
jmorganca
4d0e6c55b0
remove perl docs
2024-09-03 21:15:13 -04:00
jmorganca
3375b82c56
remove build scripts
2024-09-03 21:15:13 -04:00
jmorganca
a632a04426
fix output
2024-09-03 21:15:13 -04:00
jmorganca
110f37ffb0
arch build
2024-09-03 21:15:13 -04:00
jmorganca
f2f03ff7f2
add temporary makefile
2024-09-03 21:15:13 -04:00
jmorganca
9966a055e5
fix cgo flags for darwin amd64
2024-09-03 21:15:13 -04:00
jmorganca
43efc893d7
basic progress
2024-09-03 21:15:13 -04:00
jmorganca
20afaae020
add more runner params
2024-09-03 21:15:13 -04:00
jmorganca
b2ef3bf490
embeddings
2024-09-03 21:15:12 -04:00
jmorganca
ce15ed6d69
remove dependency on llm
2024-09-03 21:15:12 -04:00
jmorganca
c0b94376b2
grammar
2024-09-03 21:15:12 -04:00
jmorganca
72be8e27c4
sampling
2024-09-03 21:15:12 -04:00
jmorganca
d12db0568e
better example module, add port
2024-09-03 21:15:12 -04:00
jmorganca
ec17359a68
wip
2024-09-03 21:15:12 -04:00
jmorganca
fbc8572859
add llava to runner
2024-09-03 21:15:12 -04:00
jmorganca
28bedcd807
wip
2024-09-03 21:15:12 -04:00
jmorganca
b22d78720e
cuda linux
2024-09-03 21:15:12 -04:00
jmorganca
9547aa53ff
disable log file
2024-09-03 21:15:12 -04:00
jmorganca
a8f91d3cc1
add llava
2024-09-03 21:15:12 -04:00
jmorganca
e86db9381a
avx2 should only add avx2
2024-09-03 21:15:12 -04:00
jmorganca
9fe48978a8
move runner package down
2024-09-03 21:15:12 -04:00
jmorganca
01ccbc07fe
replace static build in llm
2024-09-03 21:15:12 -04:00
jmorganca
0110994d06
Initial llama Go module
2024-09-03 21:15:12 -04:00
jmorganca
2ef3a217d1
add sync of llama.cpp
2024-09-03 21:15:12 -04:00
Michael Yang
fccf8d179f
partial decode ggml bin for more info
2023-08-10 09:23:10 -07:00
Bruce MacDonald
984c9c628c
fix embeddings invalid values
2023-08-09 16:50:53 -04:00
Bruce MacDonald
09d8bf6730
fix build errors
2023-08-09 10:45:57 -04:00
Bruce MacDonald
7a5f3616fd
embed text document in modelfile
2023-08-09 10:26:19 -04:00
Michael Yang
f2074ed4c0
Merge pull request #306 from jmorganca/default-keep-system
...
automatically set num_keep if num_keep < 0
2023-08-08 09:25:34 -07:00
Bruce MacDonald
a6f6d18f83
embed text document in modelfile
2023-08-08 11:27:17 -04:00
Jeffrey Morgan
5eb712f962
trim whitespace before checking stop conditions
...
Fixes #295
2023-08-08 00:29:19 -04:00
Michael Yang
4dc5b117dd
automatically set num_keep if num_keep < 0
...
num_keep defines how many tokens to keep in the context when truncating
inputs. if left to its default value of -1, the server will calculate
num_keep to be the left of the system instructions
2023-08-07 16:19:12 -07:00
Michael Yang
b9f4d67554
configurable rope frequency parameters
2023-08-03 22:11:58 -07:00
Michael Yang
c5bcf32823
update llama.cpp
2023-08-03 11:50:24 -07:00
Michael Yang
74a5f7e698
no gpu for 70B model
2023-08-01 17:12:50 -07:00
Michael Yang
319f078dd9
remove -Werror
...
there are compile warnings on Linux which -Werror elevates to errors,
preventing compile
2023-07-31 21:45:56 -07:00
Jeffrey Morgan
7da249fcc1
only build metal for darwin,arm target
2023-07-31 21:35:23 -04:00
Bruce MacDonald
184ad8f057
allow specifying stop conditions in modelfile
2023-07-28 11:02:04 -04:00
Michael Yang
3549676678
embed ggml-metal.metal
2023-07-27 17:23:29 -07:00
Michael Yang
fadf75f99d
add stop conditions
2023-07-27 17:00:47 -07:00
Michael Yang
ad3a7d0e2c
add NumGQA
2023-07-27 14:05:11 -07:00
Michael Yang
cca61181cb
sample metrics
2023-07-27 09:31:44 -07:00
Michael Yang
c490416189
lock on llm.lock(); decrease batch size
2023-07-27 09:31:44 -07:00
Michael Yang
f62a882760
add session expiration
2023-07-27 09:31:44 -07:00
Michael Yang
3003fc03fc
update predict code
2023-07-27 09:31:44 -07:00
Michael Yang
35af37a2cb
session id
2023-07-27 09:31:44 -07:00
Michael Yang
726bc647b2
enable k quants
2023-07-25 08:39:58 -07:00
Michael Yang
cb55fa9270
enable accelerate
2023-07-24 17:14:45 -07:00
Michael Yang
b71c67b6ba
allocate a large enough tokens slice
2023-07-21 23:05:15 -07:00
