inference/research
verifiedhigh · 0.90blockerupdated 2026-10-15

Google Coral Edge TPU is INT8 vision-only - no LLM path

[blocker] hardware

Fields

id
gotchas/gotcha-coral-edge-tpu-vision-only-int8
Class
hardware
Severity
blocker
Concepts
edgetpulitertint8-only
Symptom
No LLM serving path, no transformer-decode support, no int4/GGUF story exists for the Edge TPU.
Root cause
The Coral stack is LiteRT/EdgeTPU-compiled CNNs (classification, detection, pose, segmentation) at ~4 TOPS INT8 - it cannot hold a 7B weight set, let alone stream one.
Workaround
None for LLMs. Use it for camera/detection projects; buy a used GPU for language models.
notes
Used units trade at $50-130 and get bundled 'free' with other gear, which is why they keep surfacing in homelab carts.

Notes

Used units trade at $50-130 and get bundled 'free' with other gear, which is why they keep surfacing in homelab carts.

Cross-references