FuriosaAI Hot Chips 2024 blog: RNGD GPT-J about 12 queries per second

Fields

id

sources/furiosa-blog-hot-chips-2024-gptj-12qps

URL

https://furiosa.ai/blog/furiosaai-rngd-hot-chips-2024

Publisher

FuriosaAI

Kind

blog

Published

2024-08-26

Accessed

2026-10-16

notes

Vendor blog dated 2024-08-26. Prose: a single RNGD is generating about 12 queries per second on GPT-J 6B, expected to rise as the software stack is refined. The comparison table (GPT-J 6B MLPerf at 99 percent accuracy, queries per second and power versus L40S, Gaudi 2, TPU v5e) is an image and is captioned as unverified by MLPerf, so those comparison numbers were not transcribed. TDP on this page is 180 W.

Notes

Vendor blog dated 2024-08-26. Prose: a single RNGD is generating about 12 queries per second on GPT-J 6B, expected to rise as the software stack is refined. The comparison table (GPT-J 6B MLPerf at 99 percent accuracy, queries per second and power versus L40S, Gaudi 2, TPU v5e) is an image and is captioned as unverified by MLPerf, so those comparison numbers were not transcribed. TDP on this page is 180 W.

Cross-references

This record references no other records.