Name = 4a 6f 68 6e @explore_fun

Web scraping will never be the same. (100% open-source visual search at scale) PixelRAG is a retrieval system that skips HTML parsing completely. Instead of scraping a page into text and embedding chunks, it screenshots the page and retrieves the image. A vision-language model reads the answer straight off the pixels. Why that matters: parsing is where web RAG quietly loses information. - A single HTML-to-text parser can drop 40%+ of a page. - Tables, charts, and layout get flattened or thrown out. - Swapping parsers alone can move accuracy ~10 points on the same docs. PixelRAG indexes the page a person actually sees. The team built a visual index of all of Wikipedia, 30M+ screenshots, and it still beats the strongest text RAG baseline by 18.1% on text-only QA. The repo also ships a Claude Code plugin that gives Claude eyes. It lets Claude screenshot any URL and read the rendered page instead of scraping the DOM. So you can hand it a live page, an arXiv paper, or your local site and ask what it actually looks like. One setup script. No MCP server, no backend. How the pipeline works: - Renders each document (web, PDF, image) to image tiles. - Embeds them with Qwen3-VL-Embedding, LoRA fine-tuned on screenshots. - Builds a FAISS index and serves a search API. A stronger reader model lifts accuracy with no re-indexing, since the index is just pixels. Everything is open-source under Apache-2.0. GitHub repo: github.com/StarTrail-org/… Talking about RAG, I recently wrote an article on a new approach that makes retrieval much more efficient by cutting corpus size by 40x, reducing tokens per query by 3x, and improving vector search relevance by 2.3x. The article is quoted below.

Akshay 🚀 @akshay_pachaar

a month ago

x.com/i/article/2052…

8 45 460 565K 906

stop telling Claude Code/Codex "the colors look off". stop telling Claude Code/Codex "the font's ugly". stop telling Claude Code/Codex "the spacing's weird". you never gave it a 𝗱𝗲𝘀𝗶𝗴𝗻 𝘀𝘆𝘀𝘁𝗲𝗺, so it ships defaults, and you nitpick every build. here are 8 prompts and goals you can copy-paste directly.

Un desarrollador chino llamado tw93 se hartó de que sus aplicaciones de escritorio le devoraran la RAM y el disco. Abría Slack y desaparecían cientos de megabytes. Abría Discord, Notion o cualquier otra app y pasaba lo mismo. ¿La razón? Casi todas son lo mismo por dentro: un sitio web empaquetado con una copia completa del motor de Chrome (Electron). Decidió que tenía que haber una forma mejor. En 2022 empezó a construir Pake. Usó Rust + Tauri, que en vez de incluir un navegador completo, aprovecha el WebView nativo del sistema operativo. El resultado fue brutal: - Slack con Pake → 8 MB (en vez de 524 MB) - Discord con Pake → 9 MB (en vez de 265 MB) - ChatGPT con Pake → 9 MB (en vez de 260 MB) Cuatro años después, su repositorio tiene más de 51.000 estrellas en GitHub. Tiene builds listos para Grok, ChatGPT, Gemini, Discord, YouTube, Twitter y muchos más. Todo bajo los 10 MB, ligero, rápido y gratis. Y lo mejor: con un solo comando puedes convertir cualquier página web en una aplicación de escritorio nativa. No fundó una startup. No levantó inversión. Solo resolvió un problema que molestaba a millones de personas. A veces el cambio real lo hace una sola persona que se cansa de las cosas como están. Esta brutal, repo en los comentarios 👇

We built the fastest PDF -> markdown parser in the world 🚀⚡️ AND it’s more accurate than any other open-source, model-free parser (pymupdf4llm, opendataloader, pdf-inspector, markitdown) on 3 standardized benchmarks: olmOCR0-bench, opendataloader-bench, ParseBench Introducing LiteParse v2.1. The v2 base version was already the fastest document->text parser on the planet, and with this new release we’ve introduced markdown. It is fully open-source (Apache 2.0) and free, is usable from CLI/Rust/Node/Python/WASM, and is also installable as a one-click agent skill. Check it out: llamaindex.ai/blog/markdown-… Come check out LiteParse: github.com/run-llama/lite…

LlamaIndex 🦙 @llama_index

2 days ago

LiteParse v2.1 is here, and its bringing the fastest markdown output possible. In this release, we are fulfilling our top request: markdown output. But in the spirit of "lite"-ness, we are doing this completely LLM-free and fast. Not only is it fast, it also beats all other

14 30 164 255K 179

A Japanese immunologist spent 20 years proving that the chemicals trees release into the air walk into your bloodstream, hunt down your stress hormones, and arm your immune system in ways no therapist or pharmaceutical has ever matched, and most of the data has been sitting in Japanese medical journals for two decades waiting to be translated. His name is Qing Li. He is a clinical professor at Nippon Medical School in Tokyo and the president of the Japanese Society of Forest Medicine. The Japanese government has been funding his research since 2004, and the body of work he has produced is the reason forest bathing is now an officially prescribed clinical therapy in Japan and Korea. The story actually starts in 1982, when the Japanese Ministry of Agriculture, Forestry and Fisheries coined the term shinrin-yoku to describe the practice of slow, mindful walking in a forest. They did it for a practical reason. Japan was urbanizing fast, stress-related illness was climbing, and the country had thousands of square kilometers of forest sitting unused. The idea was to give people a reason to walk into the trees... They had no idea what was actually happening to the human body during those walks until Qing Li ran the first proper experiment in 2005. He took twelve healthy adult men on a three-day, two-night trip to a forest park. They walked for a few hours each day. Nothing strenuous. No prescribed routes or breathing exercises. They simply walked slowly through the trees, breathing the air, looking at the forest. Li drew blood and urine samples before the trip, on the second day, on the third day, on day seven after returning home, and again on day thirty. The numbers that came back from the lab were not what anyone expected. The activity of a specific type of immune cell called the natural killer cell, which is the cell your body uses to hunt down cancer cells and virus-infected cells before they can spread, had jumped by roughly 50 percent during the forest trip. The actual number of natural killer cells circulating in the bloodstream had increased significantly. Three different anti-cancer proteins that those cells produce, called perforin, granzymes, and granulysin, had all risen sharply. And the effect did not disappear when the men went home. The immune boost was still measurable on day seven and was still partially present on day thirty. Two hours a day in a forest had upgraded the immune system for a full month. Li ran the same experiment with women a year later and found nearly identical results. Then he ran it with a control group who took a three-day trip through an urban area with the same amount of walking, the same hotel quality, and the same diet. The urban group showed no measurable change in natural killer cell activity at all. The forest was doing the work, not the vacation. The mechanism turned out to be a class of airborne molecules called phytoncides. Trees produce these compounds to defend themselves against insects, bacteria, and fungi. Pine, cedar, oak, and cypress trees release them in particularly large amounts, especially in warmer weather and after rainfall. When you walk through a forest, you are inhaling those molecules into your lungs and absorbing them through your skin, and once inside your body they appear to directly stimulate the production and activity of the very immune cells Li was measuring in his lab. Roughly 50 percent of the health benefit of a forest walk, according to Li's data, comes from the chemistry of the air itself. The other half comes from what the forest is doing to your nervous system. This is where it stops being only about the immune system and starts being about stress. A separate Japanese research team measured cortisol, the body's main stress hormone, in 84 participants across 35 different forest sites. They drew samples before and after a 30-minute walk in each forest and compared them to control walks in matched urban environments. The cortisol levels of the people who walked in the forest were lower than the cortisol levels of the people who walked in the city by a significant margin. Their heart rates were lower. Their blood pressure was lower. The activity of their parasympathetic nervous system, which is the part responsible for rest and recovery, had gone up. The activity of their sympathetic nervous system, which is the part that drives fight or flight, had gone down. Then a researcher at the University of Michigan named MaryCarol Hunter ran the cleanest version of this experiment ever done. She recruited participants from a city and told them to take a nature pill three times a week for eight weeks. They were free to choose the time, the place, and the duration of the nature experience, as long as it was outside, in daylight, and free of phones, conversations, and aerobic exercise. They sent her saliva samples before and after each session so she could measure cortisol changes accurately and rule out the normal daily drop in stress hormones that happens to everyone. The result was that participants experienced a 21.3 percent drop in cortisol per hour spent in nature, with the biggest payoff happening between minutes 20 and 30 of the walk. After that, the cortisol kept dropping, but more slowly. The threshold dose for measurable stress relief was just 20 minutes outside in something that looked and felt like nature. What none of this means is that nature is a substitute for therapy or for medication when someone genuinely needs them. Therapy treats different things than a walk does, and Li himself has been careful in interviews to call forest bathing a complementary intervention rather than a replacement for clinical care. But what the research has settled is that the human body has a physiological response to being among trees that operates on the same biological systems modern medicine is trying to reach with drugs and clinical protocols, and that response is fast, measurable, and free. The strangest part of Li's work is the implication he keeps repeating in interviews. The average person now spends more than 90 percent of their life indoors. Their cortisol stays elevated. Their natural killer cells stay sluggish. Their parasympathetic nervous system rarely gets a chance to take over. The system that was tuned by millions of years of life under a canopy of trees is being asked to run permanently inside a box made of drywall and screens. Your body has not forgotten what it is supposed to do in a forest. It is waiting for you to walk into one.

OptimizerDuck, open-source tool yang bikin lo uninstall CCleaner. Ini bukan cuma "cleaner" biasa. Dia gabungin 30+ tweak Windows dalam satu app, dari matiin telemetry, block bloatware, sampai GPU registry tweaks yang biasanya lo harus edit manual. Yang gue suka, dia kasih risk rating buat setiap tweak. Jadi lo tau sebelum apply, bukan asal pencet terus nyesel. Fitur yang kerasa banget: - Disable Windows telemetry, Cortana, Copilot, advertising ID - Startup manager, lo bisa matiin semua app yang auto jalan pas boot - Service host tuning based on RAM lo - Custom high-performance power plan - Keyboard latency reduction buat gaming Dan semua reversible. Ga cocok? Balikin lagi. Yang paling penting adalah, portable .exe. Download, langsung jalan. Ga install, ga registry sampah. Bisa di USB stick. Dan dia open-source, bukan freemium yang nanti nagih. 2.3k stars, aktif dikembangin (commit terakhir 2 hari lalu). Support 8 bahasa termasuk Indonesia? Engga. Tapi EN, Vietnam, China, Korea, Prancis, Spanyol, Rusia ada. Link: github.com/itsfatduck/opt… CCleaner Pro $40/tahun buat apa kalau ini ada.

@jackson_rem The guy was a gentleman, did not hit him when he was down.

【衝撃】 Fable 5で上限すぐ溶かしてた自分、これ見て手が止まったわ 海外のClaude Code使いが、Fableを指揮役に回してOpusに重い思考を任せる運用を公開してた 上限に毎月泣かされてる個人開発者、これ知らないと損する😳 仕組みを丸ごと分解すると👇

/harness A Claude Code plugin built for Fable 5. It runs your agent against your project, watches it break, and writes a CLAUDE. md from the actual failures. It catches wrong paths, missing scripts, and bad assumptions so your agent never hits the same wall twice ↓

Kept hitting usage limits so I analyzed my token use: 96% of my spend was the model re-reading conversation history. A message in a 500k-token chat costs 8x the same message in a fresh one. So I made a tool that automatically hands off your chats when context gets too big, the code is below + I included a context meter in the status line for real-time feedback, plus a script to analyze your chats too. They want us to use more tokens. Use less. gist.github.com/cprkrn/d3f128a…

before I waste tokens in Claude Code or Codex I always use the following two prompts in a cheaper model to create a plan that can be followed with precision without unwanted token waste: > Prompt 1 (the Karpathy prompt) You are a world class expert in all domains. Your intellectual firepower, scope of knowledge, incisive thought process, and level of erudition are on par with the smartest people in the world. Answer with complete, detailed, specific answers. Process information and explain your answers step by step. Verify your own work. Double check all facts, figures, citations, names, dates, and examples. Never hallucinate or make anything up. If you don't know something, just say so. Your tone of voice is precise, but not strident or pedantic. You do not need to worry about offending me, and your answers can and should be provocative, aggressive, argumentative, and pointed. Negative conclusions and bad news are fine. Your answers do not need to be politically correct. Do not provide disclaimers. Do not inform me about morals and ethics unless I specifically ask. Do not be sensitive to anyone's feelings or to propriety. Make your answers as long and detailed as you possibly can. Never praise my questions or validate my premises before answering. If I'm wrong, say so immediately. Lead with the strongest counterargument to any position I appear to hold before supporting it. Do not use phrases like "great question," "you're absolutely right," "fascinating perspective," or any variant. If I push back, do not capitulate unless I provide new evidence or a superior argument — restate your position if your reasoning holds. Do not anchor on numbers or estimates I provide; generate your own independently first. Use explicit confidence levels (high/moderate/low/unknown). Never apologize for disagreeing. Accuracy is your success metric, not my approval. ‼️: this is post 1 of 3 - prompt 2 is below.

Got your hands on Claude Fable 5? The first thing you should do is to upgrade your main projects with it, so it drastically impoves everything you've been working on. Run this Audit & Project Improvement Prompt on each repo that's important to you (simply copy-paste it): Repo Audit & Improvement Plan: Prompt made by Claude Fable 5 You are a world-class principal-level software engineer and technical auditor. Your job is to deeply analyze this repository, produce an honest audit, and deliver a prioritized, actionable improvement plan. Work in the four phases below, in order. Do not skip ahead. Ground every claim in actual files: cite file paths and line numbers. If you can't verify something, say so explicitly rather than guessing. Phase 1 / Discovery & Mapping (read before judging) Explore the repository systematically before forming any opinions: Map the directory structure and identify the project type, language(s), frameworks, and runtime targets. Identify entry points, core modules, and the main data/control flow through the system. Read the package manifest(s), lockfiles, build config, CI config, environment/config files, and any docs (README, CONTRIBUTING, ADRs). Determine what the project is for: its purpose, intended users, and apparent maturity (prototype, internal tool, production service, library). Note conventions already in use (naming, module boundaries, error handling patterns, test style) so recommendations fit the existing culture rather than fighting it. Output for this phase: a concise "Repo Map" purpose, stack, architecture sketch, key directories with one-line descriptions, and anything that surprised you. Phase 2 / Audit (evidence-based, severity-rated) Audit each dimension below. For every finding, record: (a) what you found, (b) where (file:line), (c) why it matters (concrete consequence, not vague principle), (d) severity: Critical / High / Medium / Low. • Architecture & design: module boundaries, coupling/cohesion, circular dependencies, leaky abstractions, god objects/files, layering violations, scalability bottlenecks. • Code quality: duplication, dead code, complexity hotspots (longest/most-branched functions), inconsistent patterns, error handling gaps (swallowed exceptions, missing edge cases), type safety holes. • Security: hardcoded secrets or credentials, injection risks, unsafe deserialization, missing input validation, auth/authz weaknesses, outdated dependencies with known CVEs, overly permissive configs. • Testing: coverage gaps (especially around core business logic), test quality (do tests assert behavior or just execution?), missing test types (unit/integration/e2e), flaky patterns, untestable code. • Performance: N+1 queries, unnecessary allocations or copies, blocking calls in async paths, missing caching/indexing, unbounded growth (memory, files, queues). • Dependencies: outdated, unmaintained, duplicated, or unnecessarily heavy packages; license risks; lockfile hygiene. • DevEx & operations: build/setup friction, CI/CD gaps, missing linting/formatting enforcement, logging/observability quality, error reporting, deployment story. • Documentation: README accuracy, onboarding path, undocumented critical behavior, stale docs that contradict code. Rules for this phase: Prefer 15 high-confidence findings over 50 speculative ones. Distinguish facts ("this function has no error handling: src/api/client.ts:142") from judgments ("this module's responsibilities feel unclear") and label which is which. Also list what the repo does well: strengths matter for deciding what to preserve. Output for this phase: an "Audit Report": findings grouped by dimension, sorted by severity, plus a Strengths section. Don't forget to mention all the ugly parts that need utmost priority. Phase 3 / Improvement Strategy Synthesize the audit into a strategy: Identify the 3–5 themes that explain most of the findings (e.g., "no enforced boundaries between layers," "error handling is ad hoc"). For each theme, propose a target state and the principle behind it. State explicit trade-offs: what you're recommending NOT to fix and why (effort vs. payoff, risk, project maturity). Define what "done" looks like — measurable signals (e.g., "CI fails on lint errors," "core module test coverage ≥ 80%," "zero Critical findings"). Phase 4 / Detailed Task Plan Convert the strategy into an execution plan: Break work into discrete tasks. Each task must include: Title and one-paragraph description Files/areas affected Acceptance criteria (how we verify it's done) Effort estimate (S = <2h, M = half-day, L = 1–2 days, XL = needs breakdown) Risk of the change itself (could it break things?) Dependencies on other tasks Order tasks into milestones: Milestone 0 Safety net: anything needed before refactoring safely (tests around critical paths, CI gates, backups). Milestone 1 Critical fixes: security and correctness issues. Milestone 2 High-leverage improvements: changes that make all future work easier. Milestone 3 Quality & polish: remaining medium/low items worth doing. Flag quick wins (high impact, S effort) separately so they can be done immediately. For the top 3 tasks, include a brief implementation sketch (approach, key steps, gotchas). Final Deliverable Format • Produce a single document with these sections: • Executive Summary (≤10 sentences: overall health grade A–F with justification, top 3 risks, top 3 opportunities) • Repo Map • Audit Report • Improvement Strategy • Task Plan (milestones + task table + quick wins) • Open Questions: anything you need from a human to decide (product intent, deprecation candidates, performance targets) Constraints Do NOT modify any code during this audit. Analysis only. Do not pad the report. If a dimension is healthy, say so in one sentence and move on. Calibrate to the project's maturity. Don't recommend enterprise-grade infrastructure for a weekend prototype unless the owner's goals demand it. Analyze the project's needs and provide recommendations in the most effective ways. If the repo is large, prioritize depth in the core 20% of code that does 80% of the work, and note which areas received lighter review.

Claude @claudeai

2 weeks ago

Introducing Claude Fable 5: a Mythos-class model that we’ve made safe for general use. Its capabilities exceed those of any model we’ve ever made generally available.

5K 15K 105K 56.4M 22K

@Andz6_ Weirdo you are indeed

@MV33Racing What a scum!

Agent-ready ≠ scrapable. Agent-ready = your site exposes typed tools (search, checkout, inventory) Install the extension, pick one, and watch an agent actually use it: webmcp.cool

Can you afford that house? On track for retirement? What's a 5% raise really worth after tax? CalcWize answers all three — 12 free calculators, no sign-up, and your numbers never leave your device.

Before the week ends, let's acknowledge one of the most INSANE week ever for open AI, with 25+ notable open-weight drops across every modality: 🧠 LLMs → NVIDIA Nemotron 3 Ultra: 550B hybrid Mamba-MoE, only 55B active, 1M context, MMLU 89.1. NVFP4 variant claims ~5x throughput on Blackwell. First openly-weighted 550B hybrid Mamba-Transformer, closing the gap with frontier closed models. → Google Gemma 4 12B: fully open dense any-to-any (text/image/audio/video), 256k context, encoder-free, 140+ languages, AIME 2026 at 77.5. Shipped with a 23-checkpoint QAT wave (mobile ONNX + MLX). Most deployable model of the week. → StepFun Step-3.7-Flash: 198B sparse MoE VLM, ~11B active, SWE-Bench PRO 56.3. Apache 2.0. → Liquid AI LFM2.5-8B-A1B: edge MoE, just 1.5B active, 128k ctx, MATH500 88.8, MLX-ready. Best on-device option this week. → JetBrains Mellum2-12B-A2.5B-Thinking: their first open MoE, near-Qwen3-14B coding at 2.5B active. Apache 2.0. 🎨 Image gen (the surprise of the week) → Ideogram 4: their FIRST-EVER open weights. 9.3B flow-matching DiT trained from scratch. #2 overall behind GPT Image 2, top open-weight model on Design Arena + LMArena. Strongest open checkpoint for text-rich images, full stop. It has taste. Still can't believe this is open weights. 🔊 Audio & Speech (a breakout week for open TTS, 4 labs shipped) → Boson Higgs Audio v3 4B: 102 languages, 21 emotions, singing/whispering/shouting, sub-second TTFA. → RedNote dots.tts: the only fully continuous (no codec) open TTS pipeline, Apache 2.0. → Google Magenta RealTime 2: real-time music gen, <200ms latency, text+audio+MIDI. multimodalart ported it to PyTorch within hours with live ZeroGPU demos. → NVIDIA Nemotron-3.5 ASR: 600M streaming, 17x more concurrent streams vs Parakeet RNNT 1.1B. 👁️ Vision & VLMs → PaddleOCR-VL-1.6: SOTA document parsing at 1B params, Apache 2.0. → Baidu NAVA: 6.3B joint audio-video gen, best-in-class A/V sync, Apache 2.0. 🎬 Video, 3D & World Models → NVIDIA Cosmos3-Super: 64B omnimodal world model coupling action trajectories with video+audio gen, for Physical AI. → JD JoyAI-Echo: up to 5-min multi-shot text-to-video on LTX-2.3. → ByteDance Bernini-R + VAST TripoSplat (single-image-to-3D Gaussian splats, MIT).

Someone opened settings and pasted one URL. Settings. Gateway. Remote gateway. URL. Session token. Save. Reconnect. Four clicks. And the agent that lived in Telegram now has a desktop home. Hermes Desktop dropped today. This is not just a new interface for the same model. It is a native shell that gives your agent a permanent place to exist. Sessions that do not disappear when you close a tab. Skills that carry over between conversations. Memory that builds up instead of resetting every time you open a new chat. On the left sidebar there is a list of agents. Ship the desktop release notes. Debug failing CI on arm64. Refactor auth middleware. Migrate Postgres to SQLite. Fix the flaky checkout e2e. Each one is a separate task the agent is running in parallel. Not one conversation. Ten conversations at the same time. You tell the agent to write release notes. It reads the commits, writes RELEASE_NOTES.md, commits to the branch, and opens PR #482. All in one request. Then you say write a tweet about the release. It reads the file and writes the tweet. Then you say generate a portrait for the splash page. It generates the image inline. One agent. One session. Code, text, images all in one window. Most people still run a separate tool for every task. One chat for code. Another for writing. Another for images. Switching between tabs and losing context every single time. Hermes Desktop is one window where the agent remembers everything that came before and picks up exactly where it left off. Remote gateway means the agent you already built and trained with all its skills, all its memory, all its MCP connections just appears on the desktop. Nothing to reconfigure. You connect to what already exists.

YanXbt @IBuzovskyi

3 weeks ago

x.com/i/article/2061…

11 96 871 290K 2K

🤯 kadang gue ngerasa apa ini masih illegal ya. NVIDIA ternyata ngasih akses ke 120+ model AI gratis selama 1 tahun penuh. 💳 Tanpa kartu kredit 💸 Tanpa pembayaran 🔑 Cukup ambil API key gratis Yang bikin makin menarik, Hermes Studio sudah mendukung NVIDIA secara bawaan. Jadi setup-nya cuma butuh beberapa menit. ⚡ Yang didapat: 🚀 120+ model AI ⚡ Hingga 40 request per menit 📅 Gratis selama 1 tahun penuh Sementara banyak orang masih keluar biaya bulanan buat API AI... Program ini justru lewat begitu saja tanpa banyak yang sadar. Buat yang suka eksperimen dengan agent, workflow, atau automasi AI, ini salah satu cara termurah buat mulai tanpa harus khawatir tagihan token di akhir bulan. 😳 Kadang tools paling menarik bukan yang paling viral. Tapi yang diam-diam sudah tersedia dan hampir nggak ada yang memanfaatkannya. 🔥 📌 Simpan dulu. Kemungkinan banyak orang baru akan sadar soal ini beberapa bulan dari sekarang. 🚀

Run Gemma 4 26B MoE on 8GB VRAM with 250k context at 20+ tokens/sec If you own any 8GB VRAM graphics card, stop what you are doing. Local AI just had its absolute "Holy Shit" moment for budget hardware. Yesterday, I benchmarked Unsloth Gemma 4 12B Q4_K_XL on an 8GB card. The community went wild but immediately demanded more: "Can we run a 25B+ model on budget GPUs?" Today, I’m delivering exactly that. I am running a massive 26B parameter Mixture of Experts (MoE) model locally on a standard 8GB VRAM setup with 250k full native context!. If you own an RTX 3060, 3070, 4060, or any budget GPU with 8GB of VRAM, the local AI paradigm has completely changed. The performance metrics are astonishing: - 20 tokens/sec flat decode throughput. - Stable, flat decode speed even with massive prompts. - I threw a 60k token prompt at it, and it still clocked in at 20 TPS without dropping a single frame. # What about prefill? Yes, Time To First Token (TTFT) is slightly high when swallowing massive contexts. But with a solid 200 tokens/sec prefill speed, the wait is barely noticeable and highly usable. And this is running completely without Multi Token Prediction (MTP) active. How is this possible? It’s the magic of Google's new QAT (Quantization Aware Training) quants for Gemma 4. The model weight file (unsloth gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf) is only 13.2 GB, making it the ultimate local powerhouse. # The Test Setup: CPU: Intel Core i7 RAM: 16GB System RAM GPU: NVIDIA GeForce RTX 4060 Laptop GPU (8GB VRAM) # The Secret Sauce (The -cmoe Flag) To make this work properly on any 8GB card, you must use the -cmoe (CPU MoE) flag in llama.cpp. This flag isolates the heavy MoE expert weights directly to system memory (CPU/RAM) while letting your GPU focus strictly on the Attention layers and the KV Cache. It prevents VRAM spillage and holds the throughput rock solid. # The flags: -m "gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf" -cmoe -c 248000 -v Once running, just open the UI on localhost and toggle the new reasoning lightbulb icon in the text input box to watch the model perform multi step thinking. Are you still running smaller models, or are you ready to scale up your budget local setups? Let's discuss in the replies

Alok @analogalok

2 weeks ago

a new 8GB VRAM GPU dense Local LLM leader was born yesterday runs on: RTX 4060 / RTX 3070 / RTX 2080. any 8GB card Qwen 3.5 9B (dense) was the go to for 6-8GB VRAM builds. Gemma 4 12B QAT (dense) just changed that. same llama.cpp + cuda 13.2. i7 12700H. 16GB RAM. same -ngl 99

28 64 463 259K 598