TGI metrics reference (queue size, queue duration, batch size and failure counters)

Full metric list with types. Operationally relevant: tgi_queue_size (Gauge, current queue size), tgi_request_queue_duration (Histogram, time spent in the queue per request), tgi_request_success (Counter), tgi_request_count (Counter), tgi_request_duration (Histogram, end-to-end), tgi_batch_current_size (Gauge), tgi_batch_current_max_tokens (Gauge, maximum tokens for the current batch), tgi_batch_inference_success (Counter), tgi_request_input_length and tgi_request_max_new_tokens (Histograms). Header states these 'can be used to monitor the performance of TGI, autoscale deployment and to help identify bottlenecks'. Note there is no per-tenant label on any of them, so TGI metrics cannot answer fairness or per-key questions.