Skip to content

Fix ctypes definitions of llama_kv_cache_view_update and llama_kv_cache_view_free. #1028

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Dec 18, 2023
Merged

Fix ctypes definitions of llama_kv_cache_view_update and llama_kv_cache_view_free. #1028

merged 1 commit into from
Dec 18, 2023

Conversation

e-c-d
Copy link
Contributor

@e-c-d e-c-d commented Dec 18, 2023

Using the functions llama_kv_cache_view_update and llama_kv_cache_view_free causes a crash. The root cause is a discrepancy between the ctypes definition and the actual definition of the C functions. For example,

# LLAMA_API void llama_kv_cache_view_free(struct llama_kv_cache_view * view);         # <--- This is a pointer to struct
def llama_kv_cache_view_free(view: llama_kv_cache_view):
    """Free a KV cache view. (use only for debugging purposes)"""
    return _lib.llama_kv_cache_view_free(view)


_lib.llama_kv_cache_view_free.argtypes = [llama_kv_cache_view]         # <--- This is the struct itself

@abetlen abetlen merged commit 2993936 into abetlen:main Dec 18, 2023
@abetlen
Copy link
Owner

abetlen commented Dec 18, 2023

@e-c-d thank you!

@e-c-d e-c-d deleted the bugfix/kv-cache-view-pointer-types branch December 19, 2023 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@e-c-d @abetlen