Spaces:
Running
Running
Vulkan Improvements (llama/5835)
Browse files* Improve dequant shaders, add fast q4_0 dequant
* Optimize dmmv non-kquants for GCN
Remove unnecessary SPIR-V shader duplication
* Fix q4_0 dequant dispatch sizes
Fix backend free bug
* Optimize dequant shaders for q4_1, q5_0, q5_1 and q8_0
* Add unary and binary op shader templates
* Fix Vulkan check results
* Enable non-contiguous support for simple ops
* Add argsort
Basic q4_0 mmq shader and unit test
* Speed up q4_0 dequant code, enable mmq for q4_0
* Rework matmul pipeline selection
* Add soft_max alibi support
* Add q4_1, q5_0, q5_1 and q8_0 dequant mat mat mul shaders
* Add environment variable GGML_VK_FORCE_MAX_ALLOCATION_SIZE to limit max buffer size
Rename GGML_VULKAN_DISABLE_F16 to GGML_VK_DISABLE_F16 for consistency
- ggml-vulkan.cpp +0 -0
- ggml-vulkan.h +1 -0
ggml-vulkan.cpp
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
ggml-vulkan.h
CHANGED
|
@@ -10,6 +10,7 @@ extern "C" {
|
|
| 10 |
#define GGML_VK_NAME "Vulkan"
|
| 11 |
#define GGML_VK_MAX_DEVICES 16
|
| 12 |
|
|
|
|
| 13 |
GGML_API void ggml_vk_init_cpu_assist(void);
|
| 14 |
|
| 15 |
GGML_API void ggml_vk_preallocate_buffers_graph_cpu_assist(struct ggml_tensor * node);
|
|
|
|
| 10 |
#define GGML_VK_NAME "Vulkan"
|
| 11 |
#define GGML_VK_MAX_DEVICES 16
|
| 12 |
|
| 13 |
+
GGML_API void ggml_vk_instance_init(void);
|
| 14 |
GGML_API void ggml_vk_init_cpu_assist(void);
|
| 15 |
|
| 16 |
GGML_API void ggml_vk_preallocate_buffers_graph_cpu_assist(struct ggml_tensor * node);
|