BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

The bidirectional-encoder architecture behind every embedding and cross-encoder reranking workload in this repo. Two properties matter for flop accounting and neither appears in generative-LLM records: attention is FULL (non-causal), so attention flops are 2x the causal figure at the same length, and there is no KV cache, so the model is run as one forward pass over the whole sequence with every weight read once for L rows. Base config (768 hidden, 12 layers, 12 heads x 64 head_dim, 3072 intermediate) derives to 113.2M parameters against a released 110M.