File tree Expand file tree Collapse file tree 1 file changed +6
-6
lines changed Expand file tree Collapse file tree 1 file changed +6
-6
lines changed Original file line number Diff line number Diff line change @@ -24,13 +24,13 @@ def __init__(
24
24
"""Load a llama.cpp model from `model_path`.
25
25
26
26
Args:
27
- model_path: Path to the model directory .
28
- n_ctx: Number of tokens to keep in memory .
27
+ model_path: Path to the model.
28
+ n_ctx: Maximum context size .
29
29
n_parts: Number of parts to split the model into. If -1, the number of parts is automatically determined.
30
- seed: Random seed.
31
- f16_kv: Use half-precision for key/value matrices .
32
- logits_all: Return logits for all tokens, not just the vocabulary .
33
- vocab_only: Only use tokens in the vocabulary.
30
+ seed: Random seed. 0 for random.
31
+ f16_kv: Use half-precision for key/value cache .
32
+ logits_all: Return logits for all tokens, not just the last token .
33
+ vocab_only: Only load the vocabulary no weights .
34
34
n_threads: Number of threads to use. If None, the number of threads is automatically determined.
35
35
36
36
Raises:
You can’t perform that action at this time.
0 commit comments