Openτensor Foundaτion @opentensor

Here's a draft of the tech report on the model training method I've been experimenting with, "Parallax". chutes.ai/parallax.pdf TL;DR: MoE models' params are mostly routed experts, and you can massively reduce VRAM and FLOPS per participant by splitting up those experts. You can also offload the expert training to commodity hardware further saving compute/VRAM per island. The crazy cool thing about these sketches is, you can actually onboard workers nearly instantly (sync time with ternary weights is a few MB), and they never need to download or stream the raw datasets (sketches contain all the work they need to do and are tiny). You'd probably want the first couple layers of the model in your own infra if you had sensitive data because otherwise you could do gradient inversion attacks to reconstitute the raw text, but beyond the first couple layers and not knowing which layer/expert you're training I think it's infeasible so privacy is pretty baked in. Decoupled DiLoCo/RDA-diloco style backbone sync, surrogates for non-owned routed experts with low rank updates to sync those, tiered sync cadences for various components, "sketches" to offload expert work, etc. 20b tested two different ways, plenty of small model iterations, and 176b params just to prove out feasibility. There are hundreds of additional experiments and loads of data we could also highlight as well, but the guts are there. Variations: - freeze routed expert weights instead of using surrogates, eliminates adam state, backward pass, etc., though you'd need different sync methods vs. low rank updates to the surrogates (the surrogates are already tiny, rank-8 updates to those even smaller) - hierarchical parallax, i.e. each node itself becomes an rda diloco style multi-learner which then syncs with the outer/global islands (the point here is to enable GPUs without NVLink/etc. and reduce GPU<=>GPU comms to maximize MFU on less-capable/commodity GPUs) - pipeline parallelize each island itself such that each island can decompose the backbone etc.

This is why we're building Exploit. Because the future of AI is being decided right now. Not by you. Not by us. By three companies with closed doors. Bittensor was built to change that. Exploit is where we choose how 👇

Today we’re releasing Quasar-Preview! Our first public proof that the Quasar architecture works at real scale. [ 18B MoE - 2B active / 5M context ] Built with Loop Transformer + Quasar attention Trained on Bittensor through decentralized infrastructure 👇

This week, @const_reborn speaks with @sebyrubino #SN46 @zipcodenetwork building accurate property appraisal models on Bittensor. SN46 Miners submit models ~30 days before evaluation; evaluation uses properties listed and sold within the last 30 days - ensuring models are tested on data they couldn't have seen Novelty Search E076 >> Live via Bittensor Discord

NOVA Conviction Update + Incentive Alignment We didn't choose drug discovery because it was an easy or fast problem to solve. We chose it because it's one of the hardest and most meaningful problems AI can tackle. Our ambition and track record reflects our commitment to long-term value creation. Since launching, NOVA has shipped 3 incentive mechanisms, established 3 relevant partnerships with other biotech companies outside of the Bittensor ecosystem, provided open-source access to optimized and fine-tuned SOTA models, run ongoing screening campaigns across multiple therapeutic modalities, and much more. That work has been funded by token sales and it will continue on multi-year timelines regardless of short-term price action. As the conviction mechanism rolls out across Bittensor, we want to be transparent about how we're approaching it by locking company tokens and aligning founder incentives with subnet growth. Company Wallet — 50% Locked Even before conviction, we've always balanced self-financing with token health. We locked 50% of our company tokens formalizing that commitment (and we’re one of the top subnets in % and TAO equivalent). From here, the owner's cut will be automatically locked so our conviction will only grow over time. The liquid 50% funds operations through token sales and OTC, and gives us flexibility to navigate price volatility. We will continue to minimize price impact and optimize for token health. Founder Incentive Alignment We're allocating 20% of what we earn split across the four founders as a direct incentive. Each founder has real skin in the game, individually motivated to grow NOVA. Based on the current wallet distribution, each founder's individual position will remain below the top 10 holders after tax. This distribution naturally decentralizes stake across the team. Our upside is tied directly to yours. Built for the Long Term We came to Bittensor to build infrastructure that accelerates drug discovery. The problems we're solving have the potential to reshape how medicine is made with multi-year timelines and billions in implications. Long term thinking is baked into everything we do. Long-term thinking is baked into everything we do. We remain focused on: • building valuable discovery systems • advancing therapeutic development • expanding partnerships • improving incentive mechanisms • and creating lasting value for the network Thank you to everyone helping us build. This is only the beginning.

Cloudflare reported that agent traffic surpassed human traffic for the first time. That's exactly why we're making Cacheon docs agent-ready. Every page now has a clean markdown source, an LLM-friendly text endpoint, and one-click handoff to ChatGPT, Claude, Cursor, and more.

Cacheon @cacheon_ai

3 days ago

Cacheon docs are now agent-ready. On every doc page: • Open in Claude, Cursor, or ChatGPT • or Copy Markdown For grounding: • cacheon.ai/llms-full.txt • Per-page markdown URLs As more development workflows move from browsers to agents, documentation needs to be optimized

1 3 21 7K 2

This is the current state of Bittensor-based, end-user-facing products on Apple Silicon: Right now, @IOTA_SN9 and @actualinc are the only working examples. @IOTA_SN9: Train LLMs and help @MacrocosmosAI turn years of research around decentralized training into a real business, while earning competitive incentives. @actualinc: Run local models and cluster any kind of compute. It is Architecture-agnostic and you can pool inference speed and VRAM across any kind of compute and say goodbye to API credits and subscriptions by gettingyour work done locally. You can hire other people’s clusters or enable mining and get your own cluster hired.

Proof of Talk: done ✓ Bittensor showed up loud. Now comes Exploit - our conference. Our room. Our rules. 1,000 builders and breakers in Montréal. Sept 28–29. Phase 1 tickets close in a month! Don't sleep on it 👇

bittensor:native is the printing press of intelligence. 21 million tokens mined with useful ai work. the world will tap into Bittensor’s decentralized compute & intelligence like a faucet intelligence will be commoditized the way information was after gutenberg this is one of the greatest asymmetric opportunities of all time

My agents now have a shared brain that I can optimize??? Yea… that’ll do it.

Ditto @heydittoai

5 days ago

We've been obsessed with memory since day one. Memory is the crux of high performance inference because your agents are only as good as what they remember. The issue with memory is that it must be meticulously organized. This is why we created Atlas…

3 8 47 12K 4

Bittensor Ecosystem Highlights of the Week #64 // SUBNET UPDATES & ACHIEVEMENTS ➤ @MacrocosmosAI SN9 They launched the first stage of Project Orion and unveiled their early pretraining run of Orion 100B, which reaches over 65% of data center training efficiency. (bit.ly/3RNR04M) @const_reborn successfully orchestrated a decentralized training run in 5 minutes for 5 TAO on @IOTA_SN9. (bit.ly/4fuTleC) ➤ @TargonCompute SN4 They announced Targon Tower Pro, which allows anyone to buy high performance computing hardware. (bit.ly/4e6zDDM) ➤ @webuildscore SN44 They announced that TaoWeave invested in @manakoai and partnered with them on North American commercialization. (x.com/webuildscore/s…) ➤ @LeadpoetAI SN71 They shared that Leadpoet has begun paid pilots with a large publicly listed enterprise software company and an AI compute infrastructure platform. (x.com/LeadpoetAI/sta…) ➤ @zipcodenetwork SN46 Their new website is now live. (x.com/zipcodenetwork…) ➤ @chutes_ai SN64 The Chutes platform currently earns around $280K for every trillion tokens that move through it. (x.com/chutes_ai/stat…) Chutes is now a provider on @AntSeedAI. (x.com/chutes_ai/stat…) ➤ @chutes_ai SN64 x @desearch_ai SN22 Chutes Search is now powered by Desearch AI. (x.com/chutes_ai/stat…) ➤ @QuasarModels SN24 SN3 is now training Quasar models, starting with a 10B model built on the new Quasar architecture. (x.com/QuasarModels/s…) ➤ @b1m_ai SN105 They showed they could transfer 10 GB from R2 to Internet to R2 in 21 seconds on a distributed network. (x.com/b1m_ai/status/…) ➤ @oroagents SN15 They announced the next stage of their subnet: an automated post training pipeline that should help them build the best product for AI shopping. (x.com/oroagents/stat…) ➤ @green_compute_ SN110 In their first month on Bittensor, they generated over £82,000, so they are doing $8,200 in token buybacks. (x.com/green_compute_…) ➤ @qBitTensorLabs SN63 Just 14 hours after launch, their miners were able to produce a generalized solution for breaking RSA 340. (x.com/qBitTensorLabs…) ➤ @inference_labs SN2 They shared Sertn’s Model Art, which helps visualize what needs to be done to improve a model. (x.com/inference_labs…) ➤ @trishoolai SN23 They announced that they have been accepted into the Google for Startups Web3 Program. (x.com/trishoolai/sta…) ➤ @vocence_bt SN78 They shipped new features, including a Noise Remover. (x.com/vocence_bt/sta…) ➤ @almanac_market SN41 They shared an update about where they’re headed, and said they will soon introduce a second incentive mechanism within their subnet. (x.com/almanac_market…) // NEW SUBNET ➤ @EndureNet SN30 Endure is a new “decentralized risk intelligence network” on Bittensor. (x.com/EndureNet/stat…) // BITTENSOR ECOSYSTEM ➤ @proofoftalk This week, Proof of Talk took place in Paris, with a full section of the event dedicated to Bittensor and its subnets. (x.com/taomedia_/stat…) ➤ @TAO_dot_com The Tao(.)com wallet is now available on Android. (x.com/TAO_dot_com/st…) ➤ @mentatminds They released their new Sum of Mining Subnets (SoMS) index, composed of productive mining subnets that don’t burn 100% of their emissions. (x.com/mentatminds/st…) ➤ @taostats You can now check out the full table of which subnets have conviction on Taostats. (x.com/taostats/statu…) // PODCASTS & ARTICLES ➤ @TAO_dot_com Episode 13 with @Carrot_____1 and @KeithSingery (x.com/TAO_dot_com/st…) ➤ @MacrocosmosAI interview with their AI Research Lead, Dr. Alan Aboudib. (x.com/MacrocosmosAI/…) $TAO

Macrocosmos @MacrocosmosAI

5 days ago

Our AI Research Lead, Dr. Alan Aboudib, has been instrumental in building @IOTA_SN9 and launching Orion-100B, the largest distributed LLM pretraining run conducted over the open internet across globally distributed infrastructure. We spoke with Alan about his background,

5 17 109 13K 10

Micaela @micaelabazo, CEO of NOVA (SN68) at PoT: "We're part of the most important investment category, potentially the most rewarding, because our success will not just be measured in dollars, euros, or $TAO, but also HUMAN LIFE." 📽️ Credit: @proofoftalk @SubnetSummerTAO

Proof of Talk, Louvre - pro snaps - Vol 1

keyboard_arrow_left Previous keyboard_arrow_right Next

Bittensor is going to save crypto; spiritually, reputationally, and economically.

At 'Proof of Talk' in Paris this past week, @const_reborn asked Macrocosmos to fire up a decentralized AI model pre-training run for him. They made him pay 5 TAO. Within minutes, the IOTA network of computers around the world began training. Liquid, scalable, fault-tolerant, on-demand training for AI models. No Colossus required. Computer owners are earning crypto for AI training -- AirBnB'ing their hardware. This was a magical little 'Hello, World' moment.

Macrocosmos @MacrocosmosAI

a week ago

Today, we are launching the first stage of Project Orion. Our early pre-training run of Orion-100B achieves upward of 65% of data-center training efficiency on hardware costing a fraction of the price. Orion-100B is the first proof point for a simple idea: that underutilized

23 79 407 489K 61

@ProofofTalk made one thing clear: for Bittensor to reach the next level, enterprise adoption has to accelerate. At Leadpoet, we’re helping drive that future forward. We’re excited to share that Leadpoet has begun paid pilots with a large, publicly listed enterprise software company and an AI compute infrastructure platform, bringing our sales enablement product into sophisticated sales workflows. We’re more convinced than ever that frontier subnet products are heading to the Fortune 500. If you’re an enterprise, or selling into enterprises, let’s bring Bittensor to market together.

proof of stage see you next year @proofoftalk

Our AI Research Lead, Dr. Alan Aboudib, has been instrumental in building @IOTA_SN9 and launching Orion-100B, the largest distributed LLM pretraining run conducted over the open internet across globally distributed infrastructure. We spoke with Alan about his background, education, experiences, and perspectives on AI. Alan was a lead architect behind ResBM, our SOTA compression technique designed specifically for IOTA. He has pushed our work to the frontier of distributed AI development, helping us achieve feats that were only imaginable several years ago.

Vision AI is not solved. We are solving it in the open. On Bittensor. Accurate enough to trust. Fast enough to act. Efficient enough to deploy. Up to 400x more efficient than a frontier model, equal or better accuracy, small enough to run next to the camera. @tm0klc's full @proofoftalk keynote below 👇

Score - Subnet 44 @webuildscore

5 days ago

x.com/i/article/2062…

3 26 121 15K 14

Thanks @GSchvey for your thoughtful summary of our Orion-100B work. The reason we are excited about these results is: - IOTA allows us to train very significant model sizes without requiring expensive hardware - model parallelism allows us to extract useful work from commodity GPUs at a time where compute scarcity is a limiting factor for other labs - we do not appear to incur a large overhead due to our approach (65% as fast as a data center, and growing) - our system can extract value from very short bursts of cheap compute I also want to stress that we are committed to providing the evidence required to substantiate our claims. We see distributed training as an economically disruptive force, and this is the core of the thesis we want to prove. Watch this space. Plenty more to share soon.

Greg Schvey @GSchvey

a week ago

I've been digging into the @MacrocosmosAI @IOTA_SN9 100B parameter pretraining run to understand what it is (or isn't) and potential implications. My honest assessment below in case its helpful for others:

5 23 162 94K 67