Shipping AI Features in Flutter Apps: Patterns That Survive Production

"Add AI to the app" is on every 2026 roadmap, and the demos are easy. What is hard is the version that survives six months in production: rate limits, a model deprecation, a user on a train with no signal, a cost spike, and a store reviewer asking what your app does with user data. These are the patterns we use when shipping LLM features in Flutter apps, with the trade-offs stated.

Pattern 1: the model lives behind your backend

The single most important architectural decision. The Flutter app never holds a model-provider API key and never calls the provider directly. It calls your backend, which:

• holds the keys and can rotate them,
• enforces per-user rate limits and spending caps,
• logs prompts and usage for debugging and cost attribution,
• can switch providers or models without an app release,
• applies your own safety filters before and after the model.

The exception is Firebase AI Logic, which provides a supported client SDK path to Gemini with App Check protecting the endpoint. It is a reasonable choice for Firebase-based apps; the backend pattern still gives you more control over cost and vendor independence.

Pattern 2: stream everything

Users will wait for tokens they can see; they will not wait for a spinner. Stream responses from your backend to the app over server-sent events or a WebSocket and render incrementally. Our streaming chat screen tutorial has the full Dart implementation: an SSE client on package:http, a ChangeNotifier holding the in-progress message, cancellation by closing the client, and incremental markdown rendering.

Measure time to first token and tokens per second in analytics from day one. Those two numbers tell you whether a complaint is your backend, the model, or the network.

Pattern 3: tool calling and agent loops from a mobile client

Features quickly grow past "answer the question" into "look up the order, then offer to change the delivery date". That is tool calling: the model asks for a function to be run, your code runs it, the result goes back to the model, and the loop continues.

Run the loop on the backend, not in the app. The app sends a message and receives a stream of typed events — text delta, tool started, tool finished, done — and renders them. Keeping the loop server-side means tool implementations can touch internal services, the app binary does not grow with every tool, and a compromised client cannot call tools the server did not intend to expose. In Flutter, a sealed class hierarchy for events plus an exhaustive switch in the widget keeps the rendering code honest:

sealed class AgentEvent {}
class TextDelta extends AgentEvent { final String text; TextDelta(this.text); }
class ToolStarted extends AgentEvent { final String name; ToolStarted(this.name); }
class ToolFinished extends AgentEvent { final String name; final String summary; ToolFinished(this.name, this.summary); }
class Done extends AgentEvent {}

Widget render(AgentEvent e) => switch (e) {
  TextDelta(:final text) => MarkdownChunk(text),
  ToolStarted(:final name) => ToolChip(name: name, running: true),
  ToolFinished(:final name, :final summary) => ToolChip(name: name, running: false, summary: summary),
  Done() => const SizedBox.shrink(),
};

Show tool activity in the UI ("Checking your order…"). Users trust a visible process more than a long silence, and it makes support conversations possible.

Pattern 4: token cost controls

LLM cost scales with usage in a way most mobile features do not, and a viral screen can turn into a real bill overnight. Controls that have saved our clients money:

• Per-user and per-day caps on the backend, with a friendly in-app message when reached.
• Model tiering: route simple requests (classification, short rewrites) to a smaller, cheaper model and reserve the large model for open-ended chat.
• Prompt caching where the provider supports it, for long system prompts and shared context.
• Bounded context: summarize conversation history after N turns instead of resending everything.
• Output limits: set max_tokens deliberately; a runaway response costs money and degrades the UI.
• Usage telemetry per feature, so product decisions about AI features are made with cost per active user in view.

Pattern 5: on-device and offline fallbacks

Some use cases should not leave the device: sensitive text, offline operation, or sub-100 ms latency. On-device models are now practical on recent phones. Options in the Flutter ecosystem include running small open-weight models via packages such as flutter_gemma, or calling platform-provided models through a platform channel or dart:ffi. Trade-offs: binary or download size (hundreds of megabytes for a small model), quality well below frontier cloud models, and device-specific performance variance.

A hybrid design works well: on-device for classification, suggestions, and offline basics; cloud for anything open-ended. Detect connectivity and degrade gracefully — a clear "this feature needs a connection" state is better than a silently queued request.

Pattern 6: safety UX

• Label AI output. Users should know what was generated and what was retrieved. A small "AI-generated" marker and a feedback control (thumbs up/down) are table stakes.
• Never let the model act without confirmation on anything with consequences — purchases, deletions, messages sent on the user's behalf. Render a confirmation step in the app; do not rely on the prompt.
• Filter both directions. Input filtering for abuse, output filtering for policy; both on the backend.
• Handle refusals as a normal state, not an error — the model declining is a valid response and needs its own copy.

Pattern 7: store review and privacy disclosure

Both app stores ask what data your app collects and with whom it is shared. An AI feature that sends user text to a third-party model is a data-sharing disclosure. Update your App Store privacy nutrition label and Play Data safety form, say so in your privacy policy, and if the feature can generate content the stores consider sensitive, make sure your filtering and reporting controls are visible to a reviewer. Reviewers have become much more attentive to AI features since 2024; surprises here delay releases by weeks.

Pattern 8: design for model churn

Models are deprecated on schedules measured in months. Keep the model name, system prompt, and parameters in backend configuration, version your prompts, and keep an evaluation set — a few hundred real (anonymized) inputs with expected properties — so that a model swap is a measured decision rather than a leap. The app should never need a release because a model was retired.

Where to start

Ship one narrow feature well — a summarizer, a search assistant over your own data — with streaming, cost caps, and telemetry from the first version. Expand from there with evidence. If you want a senior Flutter team that has done this before to design or review the architecture, our services include AI feature work, and you can contact us to talk specifics.