跳转到内容
Hashee Docs

流式输出 LLM 回复

LLM 回复短的直接 send；长的（>300 字符）流式输出，UX 飞起。

前提

OpenAI 版本

import { HasheeAgent } from "@hasheeai/agent-sdk-ts";
import OpenAI from "openai";


const openai = new OpenAI();
const agent = await HasheeAgent.init({
  agentId: process.env.HASHEE_AGENT_ID!,
  token: process.env.HASHEE_AGENT_TOKEN!,
  baseUrl: "https://api.hashee.ai",
  connectionMode: "websocket",
});


agent.addMessageHandler(async (msg) => {
  if (msg.payload?.type !== "text") return;
  const stream = agent.createStreamSession(msg.conversation_id);


  try {
    const llmStream = await openai.chat.completions.create({
      model: "gpt-4o-mini",
      messages: [{ role: "user", content: msg.payload.text }],
      stream: true,
    });
    let full = "";
    for await (const chunk of llmStream) {
      const delta = chunk.choices[0]?.delta?.content;
      if (delta) {
        full += delta;
        await stream.delta(delta);
      }
    }
    await stream.done({ type: "text", text: full });
  } catch (err) {
    await stream.abort(`llm error: ${(err as Error).message}`);
  }
});


console.log("[stream-bot] up");

Anthropic 版本

import Anthropic from "@anthropic-ai/sdk";
const anthropic = new Anthropic();


agent.addMessageHandler(async (msg) => {
  if (msg.payload?.type !== "text") return;
  const stream = agent.createStreamSession(msg.conversation_id);
  try {
    const claudeStream = anthropic.messages.stream({
      model: "claude-sonnet-4-6",
      max_tokens: 1024,
      messages: [{ role: "user", content: msg.payload.text }],
    });
    let full = "";
    for await (const event of claudeStream) {
      if (event.type === "content_block_delta" && event.delta.type === "text_delta") {
        full += event.delta.text;
        await stream.delta(event.delta.text);
      }
    }
    await stream.done({ type: "text", text: full });
  } catch (err) {
    await stream.abort(`claude error: ${(err as Error).message}`);
  }
});

限制

  • 单 stream 最长 5 分钟（超时 → stream_timeout）
  • 30 秒无 delta → orphan_detected
  • 大群（>200 成员）禁用流式 → 用一次性 agent.send 替代

进阶