Covenant-72B: How 70 Strangers Trained a 72B AI Model Over the Internet

Short answer: Covenant-72B is an open-weight language model with 72 billion parameters, pre-trained from scratch by independent GPU operators who coordinated over the public internet through Bittensor’s Subnet 3, then called Templar. Training began on September 12, 2025 and was announced complete on March 10, 2026. The team’s arXiv preprint reports that it beat Meta’s LLaMA-2-70B on two of seven standard tests, MMLU and ARC-Easy, and trailed it on the other five. Its builder, Covenant AI, left Bittensor on April 9, 2026, and as of October 21, 2026, Subnet 3 runs a different training competition called Teutonic.

Key takeaways

  • At least 70 unique peers contributed over the run, but only 16.9 on average in any one round (the cap was 20), and each was expected to bring at least eight NVIDIA B200 GPUs (arXiv 2603.08163).
  • Anyone could join. The two earlier internet-scale runs in the paper’s comparison, INTELLECT-1 and Psyche Consilience, used whitelisted participants; Covenant-72B let a validator score whoever showed up.
  • Compression made it work: peers sent updates more than 146 times smaller than dense gradients and spent about 70 seconds communicating per 20-minute round, for about 94.5 percent compute utilization.
  • It is a training-method milestone, not a frontier model. Its benchmark peer is a 2023 model, and Epoch AI estimated in December 2025 that the largest decentralized runs used about 1,000 times less compute than frontier models.
  • The base and chat weights are on Hugging Face under the Apache 2.0 license. Models with more parameters have since been started on Subnet 3; as of October 21, 2026, none had a finished model or technical report.
Covenant-72B Details
Model 72 billion parameters; a dense, LLaMA-style transformer with 80 layers and grouped-query attention; Gemma 3 tokenizer
Context length 2,048 tokens for the base model; Covenant-72B-Chat was fine-tuned up to 8,192
Training data About 1.1 trillion tokens: about 1.09 trillion of DCLM web text, then a 14.2-billion-token annealing phase on higher-quality data
Training window Launched September 12, 2025; completion announced March 10, 2026
Where Bittensor Subnet 3 (Templar), run by Covenant AI
Participants At least 70 unique peers over the run; 16.9 contributing peers per round on average, with a cap of 20
Hardware per peer At least eight NVIDIA B200 GPUs expected; some peers brought more
Method The SparseLoCo optimizer (30 local steps per round, then a heavily compressed update) and the Gauntlet validator, which scores each peer’s update
Headline result Beat LLaMA-2-70B on MMLU (67.1 vs 65.6) and ARC-Easy (80.9 vs 79.6); trailed it on five other tests
License Apache 2.0, for both Covenant-72B and Covenant-72B-Chat
Download size About 291 GB for the base weights (32-bit) and about 145 GB for the chat weights (16-bit), as listed on Hugging Face on October 21, 2026
Paper Lidin et al., arXiv 2603.08163, first posted March 9, 2026 and revised March 10; a preprint, not a peer-reviewed journal article
Covenant-72B at a glance. Sources: the Covenant-72B paper (version 2), the Covenant-72B model card on Hugging Face, and Templar’s posts of October 4, 2025 and March 10, 2026, all read October 21, 2026.

What is Covenant-72B?

Covenant-72B is an open-weight large language model whose pre-training ran on GPU servers operated by many unrelated parties, linked over the internet and coordinated by a blockchain network rather than inside one company’s data center. Its authors describe it as “the largest collaborative decentralized pre-training run to date”, measured by compute and model size among globally distributed runs.

The team behind it, Covenant AI, operated Templar, the project on Bittensor’s Subnet 3. Bittensor is a network that pays participants in tokens for machine-learning work that other participants score; our explainer on what Bittensor is and how TAO works covers the basics. The paper has nine authors, led by Joel Lidin, with Covenant AI founder Samuel (Sam) Dare among them and two co-authors from Mila, the Montreal AI institute.

Templar announced on X on March 10, 2026 that the run was complete, calling it the largest decentralized LLM pre-training run in history. The base model, Covenant-72B, and a fine-tuned chat version, Covenant-72B-Chat, are public on Hugging Face under the 1Covenant account; other accounts host copies and converted versions.

Who trained Covenant-72B, and on what hardware?

Independent operators running data-center GPUs trained it. By the paper’s count, at least 70 unique peers contributed updates over the run, but no more than 20 were counted in any one round, and the average was 16.9. Each peer was expected to run at least eight NVIDIA B200 GPUs, and some brought more.

So “70 strangers” holds as a headcount over six months, with two caveats:

  • The 70 is a floor. The authors counted blockchain IDs, which can change hands, and call the figure a lower bound on the number of distinct participants.
  • The strangers were not hobbyists. A machine with eight B200s is server hardware, not a gaming PC. The Hugging Face model card gives the minimum as “8×B200 or equivalent”.

What made the peers strangers is that nobody vetted them. On Subnet 3, anyone who registered could submit work, and the network paid them according to a validator’s scores, in Subnet 3’s own token under Bittensor’s Dynamic TAO system. The two earlier internet-scale pre-training runs the paper compares against, Prime Intellect’s INTELLECT-1 and Nous Research’s Psyche Consilience, admitted only whitelisted participants.

How do you train a 72B model over the internet?

By talking rarely and compressing hard. GPUs in a data center share their results after every training step over very fast links; Covenant-72B’s peers trained alone for 30 steps, then shared a heavily compressed summary of what they had learned. Each round, as the paper describes it:

  1. Train locally. Each peer copies the current shared model and runs 30 optimizer steps on its assigned slice of the data, using only its own GPUs.
  2. Compress. The peer measures how far its copy moved (a “pseudo-gradient”), keeps the 64 largest values in each block of 4,096, stores each in 2 bits, and carries what it dropped into later rounds, so that information can still be sent. The result is more than 146 times smaller than sending full gradients.
  3. Upload. The peer puts its update in its own Cloudflare R2 storage bucket, where the rest of the network can read it. Peers never need direct connections to one another.
  4. Score. A validator running Gauntlet, the team’s incentive system, checks each submission, mainly by testing how much it lowers the model’s error on small batches of data, and picks up to 20 for the round. Rewards follow those scores.
  5. Merge. Every peer downloads the selected updates and applies the same combined step, so all of them start the next round from the same model.

The timing is what makes this practical. Each round gave peers a fixed 20-minute compute window, and communication took about 70 seconds on average, under the paper’s assumption that each peer’s bandwidth was capped at 500 megabits per second down and 110 up. That works out to about 94.5 percent compute utilization. For comparison, the paper notes that INTELLECT-1, a 10-billion-parameter run from 2024, spent about 8.3 minutes per round synchronizing, for about 82 percent.

Peers could also come and go. The paper reports that the number of contributing peers stayed close to the cap of 20 for most of the run, partly because the rewards were tuned to draw in replacements quickly when someone left.

How good is Covenant-72B?

About as good as Meta’s LLaMA-2-70B from 2023, and not better overall. In the paper’s Table 1, Covenant-72B beat LLaMA-2-70B on MMLU and ARC-Easy and trailed it on the other five tests.

Zero-shot test Covenant-72B LLaMA-2-70B (Meta) LLM360 K2 INTELLECT-1 Psyche Consilience
MMLU 67.1 65.6 65.5 32.7 24.2
ARC-Easy 80.9 79.6 76.0 71.8 55.8
ARC-Challenge 56.8 57.4 53.8 44.8 31.1
PIQA 81.6 82.6 82.5 77.4 76.1
OpenBookQA 44.0 49.4 48.0 43.8 35.2
HellaSwag 80.6 84.3 82.9 70.3 63.7
WinoGrande 75.9 80.4 76.4 63.3 57.0
Parameters 72B 70B 65B 10B 40B
Training tokens 1.1T 2T 1.4T 1T 1.2T (checkpoint tested)
Where trained Internet, open to anyone Data center Data center Internet, whitelisted Internet, whitelisted
Zero-shot accuracy in percent, from Table 1 of the Covenant-72B preprint (arXiv 2603.08163v2), which evaluated every model’s public checkpoint with the same harness. The higher score in each Covenant-72B vs LLaMA-2-70B pair is in bold.

Three things help read the table:

  • LLaMA-2-70B saw nearly twice the data. It was trained on 2 trillion tokens against Covenant-72B’s 1.1 trillion. The largest gaps are on HellaSwag, OpenBookQA and WinoGrande, which the authors attribute to differences in data and training recipe rather than to the internet setup. They call that a hypothesis and say the comparisons are not fully controlled.
  • Use the table, not the summary. The paper’s text says Covenant-72B was “exceeding or on par with LLaMA-2-70B” on ARC-Challenge, MMLU and ARC-Easy, but its own table gives ARC-Challenge as 56.8 against 57.4. This page follows the table.
  • It is a big step for decentralized training. The earlier internet runs in the same table scored 32.7 (INTELLECT-1) and 24.2 (a Psyche Consilience checkpoint) on MMLU.

The chat version did better against its Meta counterpart. After 14.8 billion tokens of supervised fine-tuning, Covenant-72B-Chat beat LLaMA-2-70B-Chat on 8 of the 13 tests in the paper’s Table 2, including five-shot MMLU (67.4 vs 63.1), GSM8K math word problems (63.9 vs 52.2), IFEval instruction-following (64.7 vs 40.7) and MATH (26.3 vs 10.7); its IFEval and MATH scores were the highest of the chat models compared. It trailed LLaMA-2-70B-Chat on the other five (ARC-Challenge, HellaSwag, WinoGrande, BBH and MuSR), and K2-Chat, another chat model trained in a data center, scored higher than it on 9 of the 13, including MMLU (67.9) and GSM8K (79.0).

Is Covenant-72B a frontier model?

No. Its benchmark peer is a 2023 model, and the open-weight leaders of 2026 are far stronger; our guide to open-source LLMs in 2026 covers them.

The gap is mostly compute. In an analysis published December 29, 2025, while Covenant-72B was still training, Epoch AI found that:

  • the largest decentralized models had been trained with about 1,000 times less compute than frontier models;
  • decentralized training compute had grown about 20 times a year since 2020, against about 5 times a year for frontier training;
  • Templar’s network delivered an effective 9 × 1017 operations per second, about 300 times less than a frontier AI data center (Epoch AI).

A rough estimate of our own, using the common rule of 6 × parameters × training tokens, puts Covenant-72B at about 4.8 × 1023 operations. The same rule gives LLaMA-2-70B about 8.4 × 1023, and Epoch puts INTELLECT-1 at 6 × 1022. That makes Covenant-72B roughly eight times INTELLECT-1 and a little over half of LLaMA-2-70B: a large step for decentralized training, still far below the frontier.

TAO, the series’ book on Bittensor, gives the same verdict in its chapter on the run:

Covenant-72B was not the best model in its weight class.

Excerpted from TAO: The Bittensor Story and the Rise of Decentralized Intelligence by Ryan Bethencourt, Chapter 13, “Covenant: The 72-Billion Proof”, licensed CC BY-SA 4.0.

How does Covenant-72B compare with other decentralized training runs?

Among the runs below, Covenant-72B is the largest completed pre-training run with published benchmark results, and it let anyone join. Models with more parameters have since been started on the same subnet, but by October 21, 2026 none had a finished model or technical report.

Run Who Model Who could take part Dates and status
INTELLECT-1 Prime Intellect 10B dense, 1T tokens Whitelisted (per the Covenant-72B paper): 30 compute providers, up to 14 nodes and 112 H100 GPUs at once Released November 29, 2024 after 42 days of training
INTELLECT-2 Prime Intellect 32B reasoning model; reinforcement learning on top of QwQ-32B, not pre-training Permissionless compute contributors, per its report Report posted May 12, 2025
Gauntlet 1.2B run Templar AI, on Bittensor 1.2B Permissionless; an earlier run of the same Gauntlet incentive system Paper posted May 27, 2025
Psyche Consilience Nous Research, coordinated on Solana 40B dense, 20T-token target Whitelisted (per the Covenant-72B paper); Psyche started as a permissioned testnet Announced May 14, 2025; last public checkpoint August 14, 2025; on September 29, 2025, Nous said Psyche would move on to training new models; we found no final model or report as of October 21, 2026
Covenant-72B Covenant AI (Bittensor Subnet 3) 72B dense, about 1.1T tokens Permissionless; at least 70 peers, 16.9 per round on average September 12, 2025 to March 10, 2026; complete, weights released
Teutonic-LXXX Teutonic (Bittensor Subnet 3) 80B mixture-of-experts (7.6B active), from random weights Registered miners compete to replace the best checkpoint Started May 7, 2026; the repository moved to smaller models from May 21; we found no published result
Teutonic-I Teutonic (Bittensor Subnet 3) 10B, from random weights 73 distinct wallets submitted checkpoints June 2 to August 10, 2026; technical report August 2026
Teutonic-II Teutonic (Bittensor Subnet 3) 110B mixture-of-experts (about 7.3B active), from random weights Registered miners compete to replace the best checkpoint Starting checkpoint created August 21, 2026; still the active run on October 21, 2026; live benchmark scores on its dashboard, but no finished model or report
Decentralized training runs over the internet, as of October 21, 2026. Sources: each project’s paper, blog or model card (linked), the Covenant-72B paper’s Table 1 and baselines, and the Teutonic code repository’s configuration history and dashboard for the two Teutonic mixture-of-experts runs.

Two cautions when reading it:

  • Parameter counts are not like for like. A mixture-of-experts model such as Teutonic-II has 110 billion parameters in total but uses about 7.3 billion for each token, so it needs far less compute per token than a dense 72-billion-parameter model.
  • “Largest” depends on the yardstick. News coverage called Teutonic’s 80B run the largest decentralized training effort yet when it started, but a start is not a result. Covenant-72B’s own claim is scoped to collaborative, globally distributed pre-training, measured by compute and model size. 0G’s DiLoCoX 107B has more parameters, but its developer calls that 4,000-step run a proof of concept, trained over a simulated 1 Gbps link, and published no benchmark scores for it (0G Labs, March 24, 2026).

Other Bittensor subnets train models in different ways, such as Macrocosmos’s IOTA on Subnet 9; our guide to Bittensor subnets lists the training subnets as of October 2026.

Why did Covenant AI leave Bittensor, and what came next?

Covenant AI left Bittensor on April 9, 2026, a month after the run was announced complete, with its founder saying the network’s governance conflicted with its stated commitment to decentralization; co-founder Jacob Steeves disputed some of his claims. Subnet 3 was then rebuilt as Teutonic, a competition in which miners compete to improve a single model. In order:

  • March 10, 2026: Templar announced the completed run on X. In the following weeks it drew attention from public figures including Nvidia’s Jensen Huang and investor Chamath Palihapitiya, The Block reported on April 2, 2026.
  • April 9, 2026 (US Eastern time): Covenant AI said it was leaving Bittensor and that its training research would continue outside it. Founder Sam Dare said Bittensor’s “triumvirate”, the three people who manage the multisig for network upgrades, was presented as distributed governance: “It is not. It is decentralization theatre.” He said Steeves kept effective control, and he described steps that he said Steeves had taken against Covenant, including suspending emissions to its subnets, as “punitive actions taken by one man who never relinquished control of a network he tells the world he no longer runs” (The Block, April 9, 2026). tao.media, which described Covenant AI as the team behind Templar, Basilica and Grail, reported the exit the same day.
  • Within about two hours: TAO fell 15 percent after the announcement, from $338 to a low of $285, The Block reported.
  • April 9 and 10, 2026: Steeves first responded only indirectly, The Block reported, posting that the departure would “birth the first subnets on Bittensor that run headless and as true commodities.” By the next morning he had denied the emissions claim on X, writing “I do not have the ability to suspend emissions,” and had answered Dare’s other specific claims one by one. The Block reported that he did not address Dare’s broader claims about Bittensor’s governance and the multisig (The Block, April 10, 2026).
  • April 12, 2026: the code repository for Teutonic, Subnet 3’s new training system, was created on GitHub.
  • May 7, 2026: Teutonic switched to an 80-billion-parameter mixture-of-experts model starting from random weights, according to the repository’s history. Steeves announced the run publicly on May 11 (tao.media, May 12, 2026). From May 21, the repository’s active competition moved to smaller models seeded with existing Qwen3 weights, and we found no published result for the 80B run.
  • June 2 to August 10, 2026: Teutonic-I, a 10-billion-parameter model trained from random weights, with checkpoints from 73 distinct wallets. Its August report lists Steeves first among four authors. It reports a higher 11-benchmark average for its best checkpoint than for Covenant-72B, using scores gathered from other sources, and the authors say they did not rerun those benchmarks and that the comparison does not show the competition method caused the difference.
  • August 21, 2026: the repository set up Teutonic-II, starting from a 110-billion-parameter checkpoint that, its model card says, “is initialized with random weights and has not yet undergone pretraining”. It was still the active run on October 21, 2026. Teutonic’s dashboard posts benchmark scores, measured by the project itself, for each new best checkpoint, but no finished model or technical report had been released.

This page reports what each side said about the split and takes no side in it.

What did Covenant-72B prove, and what didn’t it?

It proved that a permissionless network can pre-train a 72-billion-parameter model to roughly 2023 open-model quality. It did not prove that decentralized training can match frontier labs, or that it is cheaper.

What it showed

  • Open entry works at this scale. A validator that scores every update kept the run on course even though nobody screened the participants.
  • Ordinary bandwidth was not the bottleneck. With rare synchronization and more than 146-fold compression, peers spent about 94.5 percent of their time computing.
  • Results can be checked. The weights, the paper and the evaluation method are public, so anyone can rerun the benchmarks.

What it did not show

  • Frontier quality. By Epoch AI’s estimates, the compute gap to frontier training is about a thousandfold.
  • Lower cost. The paper publishes no cost figure; each peer supplied at least eight B200 GPUs, and the network rewarded peers with newly issued tokens, not customer revenue.
  • Wide participation in any one round. About 17 peers contributed per round, one team wrote the training and scoring code, and when that team left, the subnet moved to new software.
  • A controlled comparison. The authors say their benchmark comparisons differ in data, tokenizers, recipes and token budgets.

For the broader argument about open networks and centralized labs, see our explainer on Bittensor vs OpenAI.

Frequently asked questions

Is Covenant-72B open source?

Its weights are open. Covenant-72B and Covenant-72B-Chat are published on Hugging Face under the Apache 2.0 license, which allows commercial use, and the training method is described in an arXiv preprint. The model card names DCLM-baseline, a public web-text collection, as its pre-training dataset.

Did Covenant-72B beat LLaMA-2-70B?

On two of seven tests. In the paper’s Table 1, Covenant-72B scored 67.1 on MMLU against 65.6 for Meta’s LLaMA-2-70B and 80.9 against 79.6 on ARC-Easy, and it trailed on ARC-Challenge, PIQA, OpenBookQA, HellaSwag and WinoGrande. LLaMA-2-70B was trained on 2 trillion tokens, nearly twice Covenant-72B’s 1.1 trillion. The chat version did better, beating LLaMA-2-70B-Chat on 8 of 13 tests in the paper’s Table 2.

Can I run Covenant-72B myself?

Not on a typical home computer. The official weights are on Hugging Face as 1Covenant/Covenant-72B and 1Covenant/Covenant-72B-Chat, and the chat model’s weights alone take about 145 GB in 16-bit precision, far more memory than a gaming graphics card has. The base model reads up to 2,048 tokens of context and the chat model up to 8,192. As of October 21, 2026, Hugging Face listed no hosted inference provider for either model.

How many people trained Covenant-72B?

At least 70 unique peers contributed over the six-month run, according to the paper, which calls the count a lower bound because it tracks blockchain IDs. No more than 20 peers were counted in any round, and the average was 16.9. Each peer was expected to run at least eight NVIDIA B200 GPUs.

Could Covenant-72B be trained on home computers?

No. The internet part was the connection between peers, not their hardware: each peer was expected to bring at least eight NVIDIA B200 GPUs, which is data-center equipment. What the run showed is that those machines did not need to sit in one building: the paper’s timing figures assume each peer’s connection is capped at 500 megabits per second down and 110 up.

Who built Covenant-72B, and what is Templar?

Covenant AI, the company founded by Sam Dare, built it. Templar was Covenant AI’s project on Bittensor’s Subnet 3, where its Gauntlet validator scored the peers who trained the model and the network paid them according to those scores. Two of the paper’s nine authors are from Mila, the Montreal AI institute. Covenant AI left Bittensor on April 9, 2026.

What is Teutonic on Bittensor?

Teutonic is the training competition that has run on Subnet 3 since Covenant AI left. Miners submit model checkpoints, and a new one replaces the current best only if it lowers the model’s error by a set margin. It produced Teutonic-I, a 10-billion-parameter model trained from June 2 to August 10, 2026, and was training a 110-billion-parameter mixture-of-experts model on October 21, 2026.

Is Covenant-72B the largest decentralized training run?

Among completed pre-training runs over the internet with published results, it was the largest we found as of October 21, 2026, and its authors claim the record by compute and model size. Models with more parameters have been started since, including Teutonic’s 80-billion and 110-billion-parameter mixture-of-experts models on the same subnet, but neither had a finished model or technical report by that date.

Sources and further reading

  1. Lidin, Sarfi, Miahi, Anthony, Chauhan, Pappas, Thérien, Belilovsky and Dare, “Covenant-72B: Pre-Training a 72B LLM with Trustless Peers Over-the-Internet,” arXiv 2603.08163, posted March 9, 2026 (version 2, March 10, 2026): sections 2 to 5, Tables 1 and 2. arxiv.org; HTML version
  2. Covenant-72B and Covenant-72B-Chat model cards (license, architecture, dataset, minimum hardware), read October 21, 2026. Covenant-72B; Covenant-72B-Chat
  3. Templar, “Checkpoint One,” October 4, 2025 (launch on September 12, 2025). templarresearch.substack.com
  4. Templar (@tplr_ai) on X, completion announcement, March 10, 2026. x.com
  5. Jaime Sevilla, “How far can decentralized training over the internet scale?”, Epoch AI, December 29, 2025. epoch.ai
  6. Jaghouar et al., “INTELLECT-1 Technical Report,” arXiv 2412.01152, December 2, 2024; Prime Intellect, “INTELLECT-1 Release,” November 29, 2024. arxiv.org; primeintellect.ai
  7. Prime Intellect Team et al., “INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning,” arXiv 2505.07291, May 12, 2025. arxiv.org
  8. Lidin et al., “Incentivizing Permissionless Distributed Learning of LLMs” (Gauntlet), arXiv 2505.21684, May 27, 2025. arxiv.org
  9. Nous Research, “Democratizing AI: The Psyche Network Architecture,” May 14, 2025; “The Next Phase of Psyche,” September 29, 2025; and the Consilience 40B model card. nousresearch.com; nousresearch.com; huggingface.co
  10. The Block, “‘It is decentralization theatre’: Covenant AI exits Bittensor, TAO drops 15%,” April 9, 2026, 10:42 PM EDT. theblock.co
  11. The Block, “Bittensor co-founder denies suspending subnet emissions after Covenant AI exit,” April 10, 2026, 9:02 AM EDT. theblock.co
  12. tao.media, “Covenant Bails on Bittensor, Citing Centralization Concerns,” April 9, 2026. tao.media
  13. The Block, “Bittensor breakout fuels AI token rally as distributed training gains credibility,” April 2, 2026. theblock.co
  14. tao.media, “Teutonic Subnet Begins Training 80B AI Model on Bittensor,” May 12, 2026. tao.media
  15. Teutonic code repository: README, chain configuration and its commit history (May 7, 2026 switch to the 80B model; Teutonic-II initialized August 21, 2026), read October 21, 2026; Teutonic-II 110B Genesis model card. github.com; huggingface.co
  16. Steeves, Warchoł, Korzeniewski and Bombała, “Teutonic-I 10B: Competition Parallel Decentralized Training,” August 2026. teutonic.ai
  17. Teutonic dashboard and its daily benchmark results for the current Teutonic-II checkpoints, read October 21, 2026. teutonic.ai
  18. 0G Labs Research Team, “DiLoCoX: training a 107B model across decentralized nodes,” March 24, 2026. 0g.ai
  19. Ryan Bethencourt, TAO: The Bittensor Story and the Rise of Decentralized Intelligence (2026), Chapter 13. CC BY-SA 4.0. greyarealabs.co/the-tao-story

More on Grey Area Labs

About the author

Ryan Bethencourt is an author, biotech investor and founder who writes about decentralized systems and applied frontier technologies. He is the author of The Decentralized Revolution Series: ZERO (Zcash and zero-knowledge privacy), TAO (Bittensor and decentralized AI), LIGHT (Solana), PUNKS (NFTs and crypto art), DOGE (memecoins) and HYPE (decentralized finance), and of AGENT (AI agents and money), Book One of The Frontier Series. He co-founded IndieBio, the world’s largest seed-stage biotech accelerator, co-founded and led Wild Earth as CEO, and has backed 180+ early-stage biotech, food-tech and longevity companies. He studied molecular genetics at Warwick and bioscience enterprise at Cambridge and MIT. He writes Accelerated on Substack and publishes at Grey Area Labs.

Disclosure: Ryan holds TAO. Nothing here is financial advice.

More about Ryan Bethencourt · All books by Ryan Bethencourt

Updated October 21, 2026: rewritten around the paper’s Table 1 (Covenant-72B beat LLaMA-2-70B on two of seven tests and trailed on five), the real participation figures (at least 70 peers over the run, 16.9 per round, eight or more B200 GPUs each), how the training worked, a dated table of other decentralized training runs, and what happened after Covenant AI left Bittensor on April 9, 2026; unsourced claims about cost and hardware were removed.

1 thought on “Covenant-72B: How 70 Strangers Trained a 72B AI Model Over the Internet”

  1. Pingback: Bittensor vs OpenAI: The Case for Decentralized AI - Grey Area Labs

Leave a Comment

Scroll to Top