tests: Source-level separation between llama.cpp and ggml

[ckastner]

Overview

The tests don't use anything from llama.cpp, only from ggml, so they can be moved there. This enables a source-level separation of llama.cpp and ggml.

Additional information

Tests are taken over by ggml PR1551. This needs a separate PR there because the ggml/tests directory where they go isn't part of llama.cpp.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: NO

[ckastner]

I found one more instance on the cmake level, but that should be it.

[ggerganov]

The test-opt.cpp and test-backend-ops.cpp also don't depend on llama.cpp. However, we have to keep them here because most of the ggml development happens in the llama.cpp repo.

[ckastner]

Understood.

Would you be open to conditionalizing the tests in cmake on LLAMA_USE_SYSTEM_GGML similar to the change for test-export-graph-ops, or should we just do this downstream only? All the same to us, I'm just asking in case this is also useful upstream.

[ggerganov]

Would you be open to conditionalizing the tests in cmake on LLAMA_USE_SYSTEM_GGML similar to the change for test-export-graph-ops, or should we just do this downstream only?

Yes, that would be acceptable.

I think the ideal version is to move the ggml tests from /tests to ggml/tests and keep only the llama.cpp tests in the existing /tests folder. But this a bit of extra work - can do it next.

[ckastner]

Yes, that would be acceptable.

Thanks, I force-pushed a change just for that.

I think the ideal version is to move the ggml tests from /tests to ggml/tests and keep only the llama.cpp tests in the existing /tests folder. But this a bit of extra work - can do it next.

Agreed. I considered proposing this, but then guessed keeping ggml/tests out of the sync was a design decision that you'd like to keep.

[ggerganov]

Should the llama_build(test-export-graph-ops.cpp) line be inside the if?

[ckastner]

I thought it's useful to test-backend-ops which is always built, so I conditionalized only the use of the embedded-ggml headers.

But that is because I initially misunderstood test-backend-ops as being llama-specific, whereas you clarified above it's not.

That made me wonder why test-backend-ops didn't need that target_include_directories that test-export-graph-ops needs. So I rechecked, and it's because the former includes <ggml.h> not "ggml.h".

So maybe the CMake change is the wrong change, and the backend approach should be mirrored in test-export-graph-ops with the following change - what do you think?

-#include "ggml.h"
+#include <ggml.h>
 #include "gguf-model-data.h"
-#include "gguf.h"
-#include "ggml-backend.h"
+#include <gguf.h>
+#include <ggml-backend.h>

test-gguf and test-alloc refer to non-public ggml headers, so they continue to need the guard.

[ggerganov]

Agreed. I considered proposing this, but then guessed keeping ggml/tests out of the sync was a design decision that you'd like to keep.

No, I think the reason was that it was simpler to have all tests in one place at the beginning. Now it makes sense to differentiate between ggml tests and llama.cpp tests.

[ckastner]

Closing, superseded by #25616