Build a Telegram Bot That Streams Live AI Replies

A Telegram bot can show an AI answer while the model is still generating it. The trick is to consume the model’s token stream, collect the text, and periodically edit one Telegram message. This walkthrough uses Node.js, Telegram’s Bot API, and Fluxio to turn an ordinary request-and-response bot into a live chat experience.

§1 Create the Telegram bot and Node.js project

Open Telegram, find the official BotFather account, and send /newbot. Choose a display name and username, then save the bot token. Treat this token as a password: anyone holding it can operate your bot.

Install a current Node.js LTS release. In a new project directory, run npm init -y, followed by npm install grammy openai dotenv. grammY handles Telegram updates, while the OpenAI SDK consumes the AI response stream.

Create a .env file containing TELEGRAM_BOT_TOKEN, FLUXIO_API_KEY, and AI_MODEL. Add .env to .gitignore before committing anything. Set AI_MODEL to an exact model identifier from Fluxio’s catalog; Claude Haiku 4.5 is one example to look for.

§2 Connect the OpenAI SDK to Fluxio

Fluxio fits this project because it provides a streaming-first, pay-per-token API relay through an OpenAI-compatible interface. One key gives your application access to multiple model providers, so you can change the selected model without rebuilding the Telegram message flow.

Create an index.mjs file and load dotenv/config. Import Bot from grammy and OpenAI from openai. Initialize the Telegram bot with TELEGRAM_BOT_TOKEN, then initialize the AI client with FLUXIO_API_KEY and baseURL: 'https://api.fluxioapi.com/v1'. OpenAI-style clients need the /v1 suffix.

Keep the key on your server. Telegram users should send messages to your bot without receiving or supplying your API credentials.

§3 Open a token stream for each text message

Register a grammY handler with bot.on('message:text', async (ctx) => { ... }). Inside it, send a placeholder using ctx.reply('Thinking…') and retain the returned message_id. You will edit that message as the answer grows.

Call client.chat.completions.create with model set to AI_MODEL, messages containing the user’s text, and stream set to true. For the first version, send only the current question. Add conversation history after the basic streaming loop works.

Consume the result with for await (const chunk of stream). Read chunk.choices[0]?.delta?.content and append any text to an answer buffer. The SDK handles the underlying server-sent events, so you do not need to parse SSE frames yourself.

§4 Turn token chunks into live Telegram edits

Telegram does not accept an SSE stream as a message body. Your application translates incoming chunks into successive editMessageText requests. Keep an answer buffer, the last text successfully displayed, and a timestamp for the last edit.

• – Start with roughly one edit per second per active chat, then adjust based on observed Telegram limits.
• – Await each edit so requests for the same answer remain ordered.
• – Skip an edit when the visible text has not changed.
• – Flush the remaining text when the model stream finishes.

Call ctx.api.editMessageText with the chat ID, saved message ID, and accumulated answer. Use plain text initially: partial Markdown can contain unfinished formatting that Telegram rejects. Per-chat pacing helps, but multiple simultaneous chats also need shared rate control.

§5 Handle long answers and interrupted streams

Telegram text messages have a 4,096-character limit. Split long output conservatively below that limit, keeping Unicode characters intact. Finalize each full segment and create another message for subsequent text, tracking the new message ID.

Wrap generation and editing in error handling. If Telegram returns a rate-limit response, respect retry_after before retrying. If the AI stream fails, preserve the partial answer and append a brief interruption notice when possible. Log diagnostic details on the server.

Allow only one active generation per chat initially, and cap input size and output tokens to control costs. Test short replies, long replies, overlapping requests, and a forced connection failure before running the bot continuously with bot.start().

Sign up for Fluxio to power your bot’s live replies.