NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 model card and config.json

Primary source for the Dory 1.0 4B record. HF API createdAt 2026-10-16T18:25:29Z; card Release Date line is October 16, 2026. Card: 4B total, looped (latent-recurrent) normalized-GPT with interleaved SWA and full attention, context up to 256K, OpenMDW-1.1, English, text generation. Card diagram caption says Input and Output blocks have 8 layers each and the Recurrent block has 20; config.json (fetched 2026-10-17) stores those as attention+mlp pairs, so n_input_layers 16, n_recurrent_layers 40, n_output_layers 16, num_hidden_layers 72, n_recurrent_loops default 2. Other config fields: hidden_size 2560, intermediate_size 7168, num_attention_heads 16, num_key_value_heads 4, head_dim 256, vocab_size 131072, max_position_embeddings 262144, sliding_window 1024, hidden_act silu, model_type dory, architectures DoryForCausalLM. layer_types is 36 attention + 36 mlp: 27 sliding_attention and 9 full_attention (full at indices 6, 14, 22, 30, 38, 46, 54, 62, 70). safetensors total 3,975,229,440 BF16 parameters (single model.safetensors, ~7.97 GB). Benchmark table on the card is a vendor claim measured by NVIDIA under NeMo Gym / NeMo Evaluator; the card itself marks AA v4.3.2 numbers as internal estimates and the Qwen3.6-35B-A3B SWE-bench score as reported rather than remeasured. Companion MXFP8, NVFP4, and Q5_K_M GGUF checkpoints are quantizations of this model, not separate architectures. DGX Spark table reports MXFP8/NVFP4 speedups versus BF16, not absolute tok/s. vLLM support is pending (vllm-project/vllm#60149); llama.cpp support is pending (ggml-org/llama.cpp#30306).