runner: Initialize numPredict

numPredict is used to enforce a limit on the number of tokens to
generate. Is it passed in from Ollama but it is never stored to
be checked.
This commit is contained in:
Jesse Gross 2024-08-13 10:38:03 -07:00 committed by jmorganca
parent ebdf781397
commit 0c2f95f3de

View File

@ -91,6 +91,7 @@ func (s *Server) NewSequence(prompt string, numPredict int, stop []string, param
return &Sequence{
tokens: tokens,
n_prompt_tokens: len(tokens),
numPredict: numPredict,
responses: make(chan string, 1),
embedding: make(chan []float32, 1),
samplingCtx: sc,