如何使用StreamingTextResponse将消息元数据传递给useChat?

How to pass message metadata with StreamingTextResponse to useChat?

提问人:Aram Boyajyan 提问时间:11/15/2023 更新时间:11/15/2023 访问量:20

问:

我正在尝试传递消息元数据并在客户端使用 .StreamingTextResponseuseChat

这是我正在使用的示例代码:

import {NextRequest, NextResponse} from "next/server";
import {experimental_StreamData, Message as VercelChatMessage, StreamingTextResponse} from "ai";
import { ChatOpenAI } from "langchain/chat_models/openai";
import {PromptTemplate} from "langchain/prompts";
import {BytesOutputParser} from "langchain/schema/output_parser";

export async function POST(req: NextRequest) {
  try {
    const body = await req.json();
    const messages: VercelChatMessage[] = body.messages ?? [];
    const currentMessageContent = messages[messages.length - 1].content;

    const model = new ChatOpenAI({
      temperature: 0,
      streaming: true,
    });
    const prompt = PromptTemplate.fromTemplate(`
      You are a cheerful bot that should respond to all questions in an upbeat manner.
      
      User input: {input}
      
      AI response:
    `);
    const outputParser = new BytesOutputParser();
    const chain = prompt
      .pipe(model)
      .pipe(outputParser);
    const stream = await chain.stream({input: currentMessageContent});

    return new StreamingTextResponse(stream);
  }
  catch (e: any) {
    return NextResponse.json({ error: e.message }, { status: 500 });
  }
}

我尝试使用数据对象并将其作为第三个参数传递给:experimental_StreamData()StreamingTextResponse()

const data = new experimental_StreamData();
data.append({text: new Date().toISOString()});
data.close();

return new StreamingTextResponse(stream, {}, data);

但是,这只会字符串化数据并将其附加到响应消息之前。

我应该在这里更改什么以将数据对象与响应分开传递?

谢谢!

下一个.js vercel openai-api langchain vercel-ai

评论


答: 暂无答案