Tool · software testing / Telegram bots / LLM applications
TG-Probe
2025 · Tool · Python 3.11+, Telethon
An AI tool for automated testing of Telegram bots, especially LLM-based ones: scenarios, exploration, attacks, regression.
Acting as a regular user, TG-Probe runs YAML scenarios against any Telegram bot, and in exploration mode the AI walks through the bot on its own and builds a state graph. A separate adversarial mode tests LLM bots for prompt injection and jailbreaks, and a regression mode compares answers against a baseline. Reports in JSON / HTML / JUnit, examples for GitHub Actions and GitLab CI.
The task
Telegram bots, especially LLM bots, are hard to test automatically: the answers are non-deterministic and the interface is a live chat.
What’s inside
- Scenario mode: YAML scenarios with assertions (contains text, buttons, response time)
- Exploration mode: the AI explores the bot autonomously and builds a state graph
- Adversarial mode: prompt injection, jailbreaks and other vectors
- Regression mode: comparison against a baseline
- Dual-LLM: an orchestrator (Claude Sonnet 4) + an executor (Gemini 2.5 Flash) via OpenRouter
- CLI (init / auth / run / explore / attack / baseline / validate), reports in JSON, HTML, JUnit and the console
Machine translation — being proofread.