A chat screen that waits for the whole model response before showing anything feels broken in 2026. Users expect tokens to appear as they are generated. This tutorial builds a streaming AI chat screen in Flutter: a server-sent events (SSE) client, incremental rendering with a markdown widget, cancellation, and the error and latency handling that turns a demo into something you can ship.
Architecture: stream from your backend, not the model vendor
Do not call an LLM provider directly from the app with an API key in the binary. Keys get extracted, and you lose the ability to change vendors, add rate limits, or log usage. The app talks to your backend over SSE; the backend talks to the model (Gemini through Firebase AI Logic, the Claude API with streaming, or anything else) and forwards text deltas.
The wire format we use is deliberately minimal — one JSON object per SSE event:
event: delta
data: {"text": "Flutter draws "}
event: delta
data: {"text": "every pixel itself"}
event: done
data: {"usage": {"input_tokens": 412, "output_tokens": 88}}
event: error
data: {"message": "rate_limited", "retryable": true}
Dependencies
dependencies:
flutter:
sdk: flutter
http: ^1.2.0
flutter_markdown_plus: ^1.0.0 # or gpt_markdown; both render incremental markdown well
package:http supports streamed responses, which is all SSE needs. (The original flutter_markdown package is discontinued; flutter_markdown_plus is a maintained community fork, and gpt_markdown is designed for LLM output specifically.)
The SSE client
// lib/chat/sse_client.dart
import 'dart:async';
import 'dart:convert';
import 'package:http/http.dart' as http;
sealed class ChatEvent {}
class TextDelta extends ChatEvent { final String text; TextDelta(this.text); }
class Done extends ChatEvent { final Map<String, dynamic> usage; Done(this.usage); }
class ChatError extends ChatEvent {
final String message; final bool retryable;
ChatError(this.message, {required this.retryable});
}
class SseChatClient {
SseChatClient(this._baseUrl, {http.Client? client}) : _client = client ?? http.Client();
final Uri _baseUrl;
final http.Client _client;
/// Streams events for one user message. Cancel by cancelling the subscription.
Stream<ChatEvent> send({required String conversationId, required String message}) {
final controller = StreamController<ChatEvent>();
http.StreamedResponse? response;
Future<void> run() async {
final request = http.Request('POST', _baseUrl.resolve('/chat/stream'))
..headers['content-type'] = 'application/json'
..headers['accept'] = 'text/event-stream'
..body = jsonEncode({'conversation_id': conversationId, 'message': message});
try {
response = await _client.send(request).timeout(const Duration(seconds: 15));
if (response!.statusCode != 200) {
controller.add(ChatError('http_${response!.statusCode}', retryable: response!.statusCode >= 500));
return;
}
String? eventName;
final lines = response!.stream.transform(utf8.decoder).transform(const LineSplitter());
await for (final line in lines) {
if (controller.isClosed) break;
if (line.startsWith('event:')) {
eventName = line.substring(6).trim();
} else if (line.startsWith('data:')) {
final data = jsonDecode(line.substring(5).trim()) as Map<String, dynamic>;
switch (eventName) {
case 'delta': controller.add(TextDelta(data['text'] as String));
case 'done': controller.add(Done((data['usage'] as Map?)?.cast() ?? {}));
case 'error': controller.add(ChatError(data['message'] as String, retryable: data['retryable'] == true));
}
} else if (line.isEmpty) {
eventName = null; // blank line ends an SSE event
}
}
} on TimeoutException {
controller.add(ChatError('timeout', retryable: true));
} catch (e) {
if (!controller.isClosed) controller.add(ChatError(e.toString(), retryable: true));
} finally {
await controller.close();
}
}
controller.onListen = run;
controller.onCancel = () => _client.close(); // aborts the HTTP request
return controller.stream;
}
}
Two details matter. Cancelling the stream subscription closes the HTTP client, which aborts the in-flight request — that is how the Stop button works. And the timeout covers only time-to-first-byte; once tokens are flowing, a slow model should not trip it.
State: one message being streamed at a time
// lib/chat/chat_controller.dart
import 'dart:async';
import 'package:flutter/foundation.dart';
import 'sse_client.dart';
class ChatMessage {
ChatMessage({required this.role, required this.text, this.streaming = false, this.error});
final String role; // 'user' | 'assistant'
String text;
bool streaming;
String? error;
}
class ChatController extends ChangeNotifier {
ChatController(this._client, this.conversationId);
final SseChatClient _client;
final String conversationId;
final messages = <ChatMessage>[];
StreamSubscription<ChatEvent>? _sub;
DateTime? _startedAt;
Duration? lastTimeToFirstToken;
bool get isStreaming => _sub != null;
void send(String text) {
if (isStreaming || text.trim().isEmpty) return;
messages.add(ChatMessage(role: 'user', text: text));
final reply = ChatMessage(role: 'assistant', text: '', streaming: true);
messages.add(reply);
_startedAt = DateTime.now();
notifyListeners();
_sub = _client.send(conversationId: conversationId, message: text).listen((event) {
switch (event) {
case TextDelta(:final text):
if (reply.text.isEmpty) lastTimeToFirstToken = DateTime.now().difference(_startedAt!);
reply.text += text;
case Done():
reply.streaming = false;
case ChatError(:final message, :final retryable):
reply.error = retryable ? 'Something went wrong. Tap to retry.' : message;
reply.streaming = false;
}
notifyListeners();
}, onDone: () {
reply.streaming = false;
_sub = null;
notifyListeners();
});
}
void stop() {
_sub?.cancel();
_sub = null;
final last = messages.lastOrNull;
if (last != null && last.streaming) last.streaming = false;
notifyListeners();
}
void retryLast() {
if (messages.length < 2) return;
final assistant = messages.removeLast();
final user = messages.removeLast();
if (assistant.role == 'assistant' && user.role == 'user') send(user.text);
}
@override
void dispose() { _sub?.cancel(); super.dispose(); }
}
Using ChangeNotifier keeps the example dependency-free; in a real app this slots into Riverpod or Bloc unchanged.
Rendering tokens incrementally
Calling notifyListeners() on every delta can mean dozens of rebuilds per second. Two things keep that cheap: rebuild only the message being streamed, and throttle repaints to the frame rate.
// lib/chat/chat_screen.dart
import 'package:flutter/material.dart';
import 'package:flutter_markdown_plus/flutter_markdown_plus.dart';
import 'chat_controller.dart';
class ChatScreen extends StatefulWidget {
const ChatScreen({super.key, required this.controller});
final ChatController controller;
@override
State<ChatScreen> createState() => _ChatScreenState();
}
class _ChatScreenState extends State<ChatScreen> {
final _input = TextEditingController();
final _scroll = ScrollController();
@override
Widget build(BuildContext context) {
final c = widget.controller;
return Scaffold(
appBar: AppBar(title: const Text('Assistant')),
body: Column(children: [
Expanded(
child: ListenableBuilder(
listenable: c,
builder: (_, __) {
WidgetsBinding.instance.addPostFrameCallback((_) {
if (_scroll.hasClients) _scroll.jumpTo(_scroll.position.maxScrollExtent);
});
return ListView.builder(
controller: _scroll,
padding: const EdgeInsets.all(16),
itemCount: c.messages.length,
itemBuilder: (_, i) => _MessageBubble(message: c.messages[i], onRetry: c.retryLast),
);
},
),
),
SafeArea(
child: Row(children: [
Expanded(
child: TextField(
controller: _input,
decoration: const InputDecoration(hintText: 'Ask something'),
onSubmitted: (_) => _submit(),
),
),
ListenableBuilder(
listenable: c,
builder: (_, __) => c.isStreaming
? IconButton(icon: const Icon(Icons.stop_circle), onPressed: c.stop)
: IconButton(icon: const Icon(Icons.send), onPressed: _submit),
),
]),
),
]),
);
}
void _submit() { widget.controller.send(_input.text); _input.clear(); }
}
class _MessageBubble extends StatelessWidget {
const _MessageBubble({required this.message, required this.onRetry});
final ChatMessage message;
final VoidCallback onRetry;
@override
Widget build(BuildContext context) {
final isUser = message.role == 'user';
return Align(
alignment: isUser ? Alignment.centerRight : Alignment.centerLeft,
child: Container(
margin: const EdgeInsets.symmetric(vertical: 4),
padding: const EdgeInsets.all(12),
constraints: const BoxConstraints(maxWidth: 320),
decoration: BoxDecoration(
color: isUser ? Theme.of(context).colorScheme.primaryContainer : Theme.of(context).colorScheme.surfaceContainerHighest,
borderRadius: BorderRadius.circular(12),
),
child: message.error != null
? InkWell(onTap: onRetry, child: Text(message.error!, style: TextStyle(color: Theme.of(context).colorScheme.error)))
: message.text.isEmpty && message.streaming
? const SizedBox(width: 24, height: 24, child: CircularProgressIndicator(strokeWidth: 2))
: MarkdownBody(data: message.text, selectable: true),
),
);
}
}
Markdown is rendered from the partial text on every delta. Markdown parsers cope well with incomplete input — an unclosed code fence simply renders as a code block that keeps growing — so there is no need to buffer until a block is complete. If profiling shows the parse cost is noticeable on very long replies, coalesce deltas with a 16 ms timer before notifying.
Latency and error UX that users accept
- Show something within 300 ms. The spinner inside the empty assistant bubble covers time-to-first-token. Record
lastTimeToFirstTokenand send it to analytics; it is the number that tells you whether your backend or the model is slow. - Stop must be instant. Cancelling the subscription aborts the request; the partial text stays on screen as the final answer. Do not discard it.
- Retry, not refresh. A retryable error shows inline in the bubble and re-sends the same user message. Non-retryable errors (content policy, quota) show the server's message.
- Offline. Check connectivity before sending and queue nothing: chat is synchronous from the user's point of view, and a silently queued message that fires an hour later is worse than a clear "you are offline" state.
What is not in this tutorial
Tool calling and agent loops (where the model asks your backend to run a function mid-stream), on-device fallbacks, and cost controls are covered in our post on shipping AI features in Flutter apps. If you are building one of these screens for production and want a senior review of the architecture, contact us.