大语言模型对话

admin · 2024 年12 月 16 日 08:43

介绍

目前市面上任何与对话,大语言模型相关的功能,其实本质上就是提示词,就是拼字符串,哪怕是各种结构化输出,也是依赖提示词约束实现

schema目前可以约束格式,但是无法约束内容

对话配置

默认对话(全局使用)

    "shenghuabi.chat": {
        "model": "qwen2.5:7b",
        "temperature": 0.1,
        "topP": 0.8,
        "maxTokens": 8192,
        "baseURL": "http://127.0.0.1:11434/v1",
        "vendor": "openai",
        "vendorOptions": {
            "hunyuan": {
                "extraOptions": {
                    "model": "hunyuan-lite"
                }
            }
        }
    },

支持多个厂商(vendor),大部分都是Openai兼容

有的厂商(hunyuan)可能同时存在openai兼容接口和自研接口两种,这里只提供Openai兼容访问

[
                "openai",
                "360",
                "azure",
                "moonshot",
                "baichuan",
                "minimax",
                "mistralai",
                "groq",
                "lingyiwanwu",
                "stepfun",
                "deepseek",
                "together.ai",
                "volcengine",
                "novita",
                "siliconflow",
                "tongyi",
                "zhipu",
                "spark",
                "hunyuan",
                "qianfan", // 手动适配
                "gemini", // 手动适配
                "claude" // 手动适配
]

举例

使用硅基流动的api


    "shenghuabi.chat": {
        "model": "deepseek-ai/DeepSeek-R1",
        "vendor": "siliconflow",
        "apiKey": "sk-xxxxxxxxxxx"
    },

deepseek

"shenghuabi.chat": {
        "model": "deepseek-chat",
        "temperature": 0.1,
        "topP": 0.8,
        "maxTokens": 8192,
        "baseURL": "https://api.deepseek.com/v1",
        "vendor": "openai",
        "apiKey": "sk-xxxxxx",
     
    },

千问

"shenghuabi.chat": {
        "model": "qwen-turbo",
        "temperature": 0.1,
        "topP": 0.8,
        "maxTokens": 8192,
        "baseURL": "https://dashscope.aliyuncs.com/compatible-mode/v1",
        "apiKey": "sk-xxxxxxxx",
        "vendor": "openai",
    },

将key写到配置中方便切换

    "shenghuabi.chat": {
        "model": "deepseek-ai/DeepSeek-R1",
        "vendor": "siliconflow",
        "vendorOptions": {
            "siliconflow": {
                "extraOptions": {
                    "apiKey": "sk-xxxxxxxxxxx"
                }
            }
        }
    },

配置切换

和默认对话字段一致,多了一个name字段用于展示切换的名字

   "shenghuabi.chatModelList": [
        {
            "name": "默认",
            "model": "qwen2.5:7b",
            "temperature": 0.1,
            "topP": 0.8,
            "maxTokens": 8192,
            "baseURL": "http://127.0.0.1:11434/v1",
            "vendor": "openai",
            "vendorOptions": {}
        },
        {
            "name": "3b模型",
            "model": "qwen2.5:3b",
            "temperature": 0.1,
            "topP": 0.8,
            "maxTokens": 8192,
            "baseURL": "http://127.0.0.1:11434/v1",
            "vendor": "openai",
            "vendorOptions": {}
        }
    ],

在文本编辑器内和侧边栏,脑图卡片内切换

对话种类

直接对话

与其他聊天工具一样,输入即可

传入图片对话,需要使用支持的大语言模型,如minicpm-v:8b

{
            "name": "图像",
            "model": "minicpm-v:8b",
            "temperature": 0.1,
            "topP": 0.8,
            "maxTokens": 4096,
            "baseURL": "http://127.0.0.1:11434/v1",
            "vendor": "openai"
        }