3 Commits

Author	SHA1	Message	Date
Bruce MacDonald	a897e833b8	do not cache prompt (#2018) ... - prompt cache causes inferance to hang after some time	2024-01-16 13:48:05 -05:00
Daniel Hiltgen	2ecb247276	Fix intel mac build ... Make sure we're building an x86 ext_server lib when cross-compiling	2024-01-13 14:46:34 -08:00
Daniel Hiltgen	39928a42e8	Always dynamically load the llm server library ... This switches darwin to dynamic loading, and refactors the code now that no static linking of the library is used on any platform	2024-01-11 08:42:47 -08:00