KBaba7/llama.cpp

1# Pull requests (for contributors)2 3- Test your changes:4    - Execute [the full CI locally on your machine](ci/README.md) before publishing5    - Verify that the perplexity and the performance are not affected negatively by your changes (use `llama-perplexity` and `llama-bench`)6    - If you modified the `ggml` source, run the `test-backend-ops` tool to check whether different backend implementations of the `ggml` operators produce consistent results (this requires access to at least two different `ggml` backends)7    - If you modified a `ggml` operator or added a new one, add the corresponding test cases to `test-backend-ops`8- Consider allowing write access to your branch for faster reviews, as reviewers can push commits directly9- If your PR becomes stale, don't hesitate to ping the maintainers in the comments10 11# Pull requests (for collaborators)12 13- Squash-merge PRs14- Use the following format for the squashed commit title: `<module> : <commit title> (#<issue_number>)`. For example: `utils : fix typo in utils.py (#1234)`15- Optionally pick a `<module>` from here: https://github.com/ggerganov/llama.cpp/wiki/Modules16- Consider adding yourself to [CODEOWNERS](CODEOWNERS)17 18# Coding guidelines19 20- Avoid adding third-party dependencies, extra files, extra headers, etc.21- Always consider cross-compatibility with other operating systems and architectures22- Avoid fancy-looking modern STL constructs, use basic `for` loops, avoid templates, keep it simple23- Vertical alignment makes things more readable and easier to batch edit24- Clean-up any trailing whitespaces, use 4 spaces for indentation, brackets on the same line, `void * ptr`, `int & a`25- Use sized integer types such as `int32_t` in the public API, e.g. `size_t` may also be appropriate for allocation sizes or byte offsets26- Declare structs with `struct foo {}` instead of `typedef struct foo {} foo`27    - In C++ code omit optional `struct` and `enum` keyword whenever they are not necessary28    ```cpp29    // OK30    llama_context * ctx;31    const llama_rope_type rope_type;32 33    // not OK34    struct llama_context * ctx;35    const enum llama_rope_type rope_type;36    ```37 38    _(NOTE: this guideline is yet to be applied to the `llama.cpp` codebase. New code should follow this guideline.)_39 40- Try to follow the existing patterns in the code (indentation, spaces, etc.). In case of doubt use `clang-format` to format the added code41- For anything not covered in the current guidelines, refer to the [C++ Core Guidelines](https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines)42- Tensors store data in row-major order. We refer to dimension 0 as columns, 1 as rows, 2 as matrices43- Matrix multiplication is unconventional: [`C = ggml_mul_mat(ctx, A, B)`](https://github.com/ggerganov/llama.cpp/blob/880e352277fc017df4d5794f0c21c44e1eae2b84/ggml.h#L1058-L1064) means $C^T = A B^T \Leftrightarrow C = B A^T.$44 45![matmul](media/matmul.png)46 47# Naming guidelines48 49- Use `snake_case` for function, variable and type names50- Naming usually optimizes for longest common prefix (see https://github.com/ggerganov/ggml/pull/302#discussion_r1243240963)51 52    ```cpp53    // not OK54    int small_number;55    int big_number;56 57    // OK58    int number_small;59    int number_big;60    ```61 62- Enum values are always in upper case and prefixed with the enum name63 64    ```cpp65    enum llama_vocab_type {66        LLAMA_VOCAB_TYPE_NONE = 0,67        LLAMA_VOCAB_TYPE_SPM  = 1,68        LLAMA_VOCAB_TYPE_BPE  = 2,69        LLAMA_VOCAB_TYPE_WPM  = 3,70        LLAMA_VOCAB_TYPE_UGM  = 4,71        LLAMA_VOCAB_TYPE_RWKV = 5,72    };73    ```74 75- The general naming pattern is `<class>_<method>`, with `<method>` being `<action>_<noun>`76 77    ```cpp78    llama_model_init();           // class: "llama_model",         method: "init"79    llama_sampler_chain_remove(); // class: "llama_sampler_chain", method: "remove"80    llama_sampler_get_seed();     // class: "llama_sampler",       method: "get_seed"81    llama_set_embeddings();       // class: "llama_context",       method: "set_embeddings"82    llama_n_threads();            // class: "llama_context",       method: "n_threads"83    llama_adapter_lora_free();    // class: "llama_adapter_lora",  method: "free"84    ```85 86    - The `get` `<action>` can be omitted87    - The `<noun>` can be omitted if not necessary88    - The `_context` suffix of the `<class>` is optional. Use it to disambiguate symbols when needed89    - Use `init`/`free` for constructor/destructor `<action>`90 91- Use the `_t` suffix when a type is supposed to be opaque to the user - it's not relevant to them if it is a struct or anything else92 93    ```cpp94    typedef struct llama_context * llama_context_t;95 96    enum llama_pooling_type llama_pooling_type(const llama_context_t ctx);97    ```98 99    _(NOTE: this guideline is yet to be applied to the `llama.cpp` codebase. New code should follow this guideline)_100 101- C/C++ filenames are all lowercase with dashes. Headers use the `.h` extension. Source files use the `.c` or `.cpp` extension102- Python filenames are all lowercase with underscores103 104- _(TODO: abbreviations usage)_105 106# Preprocessor directives107 108- _(TODO: add guidelines with examples and apply them to the codebase)_109 110    ```cpp111    #ifdef FOO112    #endif // FOO113    ```114 115# Documentation116 117- Documentation is a community effort118- When you need to look into the source code to figure out how to use an API consider adding a short summary to the header file for future reference119- When you notice incorrect or outdated documentation, please update it120 121# Resources122 123The Github issues, PRs and discussions contain a lot of information that can be useful to get familiar with the codebase. For convenience, some of the more important information is referenced from Github projects:124 125https://github.com/ggerganov/llama.cpp/projects126