1Covenant/Covenant-72B: Largest model so far to be trained on decentralized permissionless GPU nodes

To reduce communication overhead, Covenant AI used their introduced method SparseLoco, built on top of DiLoCo that reduces synchronization frequency and uses a local AdamW optimizer, it also adds aggressive top-K sparsification to solve the bandwidth bottleneck.