DEFAULT_NUM_CTX additions
adding further changes for DEFAULT_NUM_CTX, including docs
This commit is contained in:
@@ -1,4 +1,7 @@
|
||||
# Contributing to Bolt.new Fork
|
||||
## DEFAULT_NUM_CTX
|
||||
|
||||
The `DEFAULT_NUM_CTX` environment variable can be used to limit the maximum number of context values used by the qwen2.5-coder model. For example, to limit the context to 24576 values (which uses 32GB of VRAM), set `DEFAULT_NUM_CTX=24576` in your `.env.local` file.
|
||||
|
||||
First off, thank you for considering contributing to Bolt.new! This fork aims to expand the capabilities of the original project by integrating multiple LLM providers and enhancing functionality. Every contribution helps make Bolt.new a better tool for developers worldwide.
|
||||
|
||||
@@ -80,6 +83,19 @@ ANTHROPIC_API_KEY=XXX
|
||||
```bash
|
||||
VITE_LOG_LEVEL=debug
|
||||
```
|
||||
|
||||
- Optionally set context size:
|
||||
```bash
|
||||
DEFAULT_NUM_CTX=32768
|
||||
```
|
||||
|
||||
Some Example Context Values for the qwen2.5-coder:32b models are.
|
||||
|
||||
* DEFAULT_NUM_CTX=32768 - Consumes 36GB of VRAM
|
||||
* DEFAULT_NUM_CTX=24576 - Consumes 32GB of VRAM
|
||||
* DEFAULT_NUM_CTX=12288 - Consumes 26GB of VRAM
|
||||
* DEFAULT_NUM_CTX=6144 - Consumes 24GB of VRAM
|
||||
|
||||
**Important**: Never commit your `.env.local` file to version control. It's already included in .gitignore.
|
||||
|
||||
### 🚀 Running the Development Server
|
||||
|
||||
Reference in New Issue
Block a user