Knit

What Knit is, and what it isn't

Knit is an on-device LLM gallery for Android. Pick a model, pick an agent, chat. Your conversations, documents, and Gmail tokens never leave the phone in plaintext — inference happens locally on the CPU (with KleidiAI ARM optimisations) or the GPU (via Google AI Edge LiteRT-LM, Mali / Adreno).

Three defaults, thirteen starters, infinite custom

Three agents ship as defaults and can't be deleted. Thirteen more come pre-seeded as deletable starters. Tap + New agent in Home to write your own — system prompt, optional knowledge file, optional tool access — and it shows up in the picker.

Always present

Pre-seeded, removable

34 models. Two backends. One picker.

LiteRT-LM (.task / .litertlm files) targets the GPU on Mali / Adreno phones with CPU fallback via TFLite XNNPACK. llama.cpp (.gguf Q4_K_M files) runs on CPU with KleidiAI matmul kernels. The dispatcher picks based on the model's format field — you just pick the model. The in-app picker groups the catalog by engine (GPU vs CPU), exactly like the two tables below, with models you've already downloaded floated to the top.

LiteRT-LM (Google AI Edge)

llama.cpp (GGUF Q4_K_M)

A voice assistant that never phones home

Three offline voice capabilities built on sherpa-onnx (Apache-2.0), wired as Knit's own backends — the same way llama.cpp and LiteRT-LM are. Audio is captured, transcribed, and synthesised entirely on the phone. Models download once from Settings → Voice.

Reach past the phone — on your terms

Everything above runs on-device. But sometimes you want an agent to touch a real service — Slack, GitHub, Notion, your inbox at large. Knit speaks the Model Context Protocol as a remote client: link an MCP server and its tools appear to your agents, alongside the native ones. Off by default — Knit stays fully on-device until you flip the switch, and the UI is blunt that calls to these tools leave the phone.

Architecture

Knit is layered. The Compose UI talks to ViewModels (Hilt + StateFlow). ViewModels talk to a single InferenceBackend interface implemented by a dispatcher that routes to llama.cpp or LiteRT-LM. Tools and RAG sit alongside, accessed through the agent loop.

The tool-calling loop

A tool-using agent emits a single JSON object (optionally wrapped in a <tool_call> tag):

{"tool":"gmail.list_unread","args":{"max":10}}

The chat layer parses it, dispatches via RealToolRegistry, then appends the result back into the conversation. The loop continues until the model emits plain text (final answer) or hits MAX_TOOL_ITERATIONS = 25.

The reliability framework

Small on-device models fumble JSON, arithmetic, and multi-step plans. Rather than hand-tune every agent's prompt, Knit wraps generation in a reliability layer (ReliabilityRuntime) that makes agents robust by construction. Four layers:

Who Knit is for

Built for people who want AI without giving it their data. These are real workflows where the on-device + agent combination unlocks something cloud chatbots can't do (or shouldn't be trusted to).

What ships today, what's planned, what won't ship

Explicitly out of scope

• Cloud chat fallback — Knit's value is the on-device promise. We don't add an OpenAI/Claude proxy.
• Model training / fine-tuning — phones don't have the memory or thermal headroom.
• Banking actions or trades — agents will surface info; they will never authorise money movement.
• Always-on wake word — battery and privacy disaster.
• WhatsApp / Telegram triage via Accessibility Service — Play Store will reject it.

Get running in 3 minutes

Install the APK

1. Download the latest Knit-debug.apk from the GitHub Releases page.
2. Allow install from unknown sources in your Android settings (one-time toggle).
3. Open the app. On first launch you'll see the model picker.
4. Pick a model — start with SmolLM 135M (159 MB) or Qwen2.5 0.5B Instruct (LiteRT) (521 MB) for fastest first impression.
5. Wait for the download (Wi-Fi-only is off by default; toggle it on in Settings if you're on cell data).
6. Tap an agent — Quick Chat is the simplest to start with.

Enable voice (optional, one-time)

1. Settings → Voice (on-device).
2. Download STT model (~101 MB) for the mic 🎙, and/or Download TTS voice (~40 MB) for spoken replies. Use Wi-Fi.
3. Tap Test voice to hear it, flip Speak agent replies on, and use 🎧 in any chat for hands-free mode (downloads the ~2 MB VAD model on first run).

Enable Email Triage (one-time, ~5 min)

Knit doesn't ship a developer OAuth Client ID. You make your own in Google Cloud — quota and tokens stay yours.

1. Go to console.cloud.google.com → APIs & Services → Credentials. Create credentials → OAuth client ID → Application type Android (or Desktop). Google accepts the app's reversed-client-ID redirect automatically for these types — there's no redirect URI to paste.
2. In APIs & Services → Library, enable the Gmail API.
3. In OAuth consent screen, add the three Gmail scopes (gmail.readonly, gmail.modify, gmail.send) and add your Gmail address as a Test user.
4. Copy the full Client ID (the whole …apps.googleusercontent.com string). Open Knit → Settings → Gmail (Email Triage) → paste it → Save → Connect Gmail.

Build from source

If you'd rather build the APK yourself, the toolchain is pinned:

git clone https://github.com/folkadonis/knit.git
cd knit
./gradlew assembleDebug
# → app/build/outputs/apk/debug/app-debug.apk

Requirements: JDK 17, Android SDK 34, NDK 27.x (for llama.cpp native build). Gradle 8.10.2, AGP 8.8.2, Kotlin 2.2.0, KSP 2.3.6, Hilt 2.58, Room 2.7.2 — all pulled by the wrapper / catalog.

Where your data goes (and doesn't)

The developer

Tech stack

Licenses

App code: Apache 2.0. Models each carry their own license — Apache 2.0 (Qwen, SmolLM, TinyLlama), MIT (Phi-4, DeepSeek-R1 distill), Gemma terms (Gemma 2/3), Llama 3 Community (Llama 3.2), CC-BY-NC-4.0 (xLAM-2 — non-commercial only). LiteRT-LM: Apache 2.0 (Google). llama.cpp: MIT (Georgi Gerganov). sherpa-onnx: Apache 2.0 (k2-fsa). AppAuth: Apache 2.0.