xai-org/grok-1 model card and run.py

Fields

id

sources/model-fr-hf-config-grok-1

URL

https://huggingface.co/xai-org/grok-1

Publisher

xAI

Kind

repo

Published

2024-03-11

Accessed

2026-10-03

notes

Model card states 314B parameters, MoE of 8 experts with 2 used per token, 64 layers, 48 query / 8 key-value attention heads, embedding size 6144, SentencePiece vocab 131072, RoPE, 8192 max sequence length, Apache 2.0. Weights ship as int8 custom-format ckpt shards, not safetensors, so the repo has no config.json.

Notes

Model card states 314B parameters, MoE of 8 experts with 2 used per token, 64 layers, 48 query / 8 key-value attention heads, embedding size 6144, SentencePiece vocab 131072, RoPE, 8192 max sequence length, Apache 2.0. Weights ship as int8 custom-format ckpt shards, not safetensors, so the repo has no config.json.

Cross-references

This record references no other records.