Repository navigation
gguf : add tensor shape accessor - #24405
Conversation
github-actions
Bot
added
testing
|
Is there a concrete software project for which this functionality would be needed or are you just adding it for general convenience? |
|
Yes, a concrete project: OpenASR, a local speech-to-text engine built on ggml. It validates GGUF model/adapter packs before running them: adapter packs are bound to one exact base model, so every tensor's shape and type is checked up front through the gguf API, without creating a ggml context. The public API exposes name/type/size/offset but not the shape, so right now we carry exactly these two accessors as a patch on our fork: QuintinShaw/openasr-ggml@91473a9. Call site: https://github.com/QuintinShaw/openasr/blob/fd05fffbecaec842f9ee9207f28b95d976c6bdd6/crates/openasr-core/src/ggml_runtime/gguf_tensor_index.rs#L308-L335. Upstreaming would let us drop the patch. |
| GGML_API uint32_t gguf_get_tensor_n_dims(const struct gguf_context * ctx, int64_t tensor_id); // trailing dims of size 1 are not counted (see ggml_n_dims) | ||
| GGML_API int64_t gguf_get_tensor_dim (const struct gguf_context * ctx, int64_t tensor_id, int dim); // returns ne[dim], which is 1 for dim >= n_dims; requires dim < GGML_MAX_DIMS |
There was a problem hiding this comment.
In terms of the API I think it would be preferable to return gguf_tensor_info::t::ne as const int64_t *.
There was a problem hiding this comment.
Done in 6633b95 — replaced gguf_get_tensor_dim with gguf_get_tensor_ne returning const int64_t *, and re-aligned the declaration block. Kept gguf_get_tensor_n_dims since GGUF stores the rank explicitly and it matches ggml_n_dims semantics — happy to drop it if you'd prefer just the ne accessor.
| GGML_API int64_t gguf_find_tensor (const struct gguf_context * ctx, const char * name); // returns -1 if the tensor is not found | ||
| GGML_API size_t gguf_get_tensor_offset(const struct gguf_context * ctx, int64_t tensor_id); | ||
| GGML_API const char * gguf_get_tensor_name (const struct gguf_context * ctx, int64_t tensor_id); | ||
| GGML_API uint32_t gguf_get_tensor_n_dims(const struct gguf_context * ctx, int64_t tensor_id); // trailing dims of size 1 are not counted (see ggml_n_dims) |
There was a problem hiding this comment.
I meant that the GGUF API should not have an explicit function to return the number of dimensions. This is something that the user code can easily determine itself from gguf_get_tensor_ne so I think it's preferable to keep the API a bit simpler.
There was a problem hiding this comment.
Makes sense — dropped gguf_get_tensor_n_dims in 35626b3, the PR now only adds gguf_get_tensor_ne.
QuintinShaw
changed the title
* gguf : add tensor shape accessors * gguf : return tensor shape as const int64_t * * gguf : remove n_dims accessor, keep only gguf_get_tensor_ne
* gguf : add tensor shape accessors * gguf : return tensor shape as const int64_t * * gguf : remove n_dims accessor, keep only gguf_get_tensor_ne
* gguf : add tensor shape accessors * gguf : return tensor shape as const int64_t * * gguf : remove n_dims accessor, keep only gguf_get_tensor_ne
* gguf : add tensor shape accessors * gguf : return tensor shape as const int64_t * * gguf : remove n_dims accessor, keep only gguf_get_tensor_ne
* gguf : add tensor shape accessors * gguf : return tensor shape as const int64_t * * gguf : remove n_dims accessor, keep only gguf_get_tensor_ne
* gguf : add tensor shape accessors * gguf : return tensor shape as const int64_t * * gguf : remove n_dims accessor, keep only gguf_get_tensor_ne
4 participants
Overview
This adds a small GGUF accessor for tensor shapes:
gguf_get_tensor_ne()— returns the shape asconst int64_t *(an array ofGGML_MAX_DIMSelements)The GGUF API already exposes tensor name, type, offset and byte size. This makes the tensor rank and dimensions available through the public API as well, so tools can inspect GGUF metadata without parsing tensor headers themselves or relying on internal structures.
I also extended
tests/test-gguf.cppto check this accessor against the existing handcrafted tensor metadata cases.Tested:
cmake --build build --target test-gguf -j 8 ctest --test-dir build -R '^test-gguf$' --output-on-failureRequirements