GPU MODE has powered much of the public GPU kernel work online, with a permissive license from day one and generous credit from researchers, NVIDIA, AMD, and others.
Today we’re moving our datasets to the Researcher Reciprocity License.
June 9th Researcher Reciprocity License
"if you train on it, you let us generate - reverse terms of use void"
Status quo
1. We teach frontier devs with ICLR/NeurIPS papers, OSS Github contributions
2. They use it to make frontier models
3. Then ban us from exploring our ideas
Since the Humanity's Last Hackathon from @huggingface didn’t happen, I set up my own mini version using Kernelbot and Popcorn from @gpu_mode.
> The goal was to test how well LLMs can generate code for difficult tasks, like writing faster kernels for Apple’s MPS with @PyTorch.
> My strategy was to let the LLM submit a kernel, get feedback from the benchmark, and then iterate based on the learnings.
> The hardest part was not the code generation itself, but coordinating all the systems. Kernelbot, Popcorn, submissions, feedback, orchestration...
> The benchmark eats almost all my RAM, so parallelizing too many submissions is hard. My machine starts crashing if I push it too much.
Overall, I need more time to tune the prompts, experiment with better feedback loops, and maybe try some RL-style iteration. There are still lots of techniques worth exploring here.
In the video:
Left: task orchestrator
Right: live dashboard tracking submissions, code, and lessons learned
Codex is so good at writing kernels that it felt appropriate to do a Codex only kernel competition. Metal is great because you'll be able to tangibly feel the perf improvements in your local models
Humanity's Last Hackathon is NOW OPEN for registration.
This is not a normal hackathon. You will be judged on the context, not the code!
Use Codex @OpenAIDevs to build and optimize models for local inference (kernels on Max metal). Submit through @GPU_MODE.
Climb the
Humanity's Last Hackathon is NOW OPEN for registration.
This is not a normal hackathon. You will be judged on the context, not the code!
Use Codex @OpenAIDevs to build and optimize models for local inference (kernels on Max metal). Submit through @GPU_MODE.
Climb the leaderboard. Top performers qualify for the final battle.
Launches May 4th. Registration is live now.
AMD's @AnushElangovan explains why he thinks his company's open source ethos combined with agentic AI superpowers their leverage as a company:
Because AMD publishes a lot of technical details about its hardware, when engineers use AI tools, the models already “understand” AMD’s
We helped host a kernel competition for @tri_dao's course at Princeton's COS 484: Natural Language Processing
If you're a university or educator that's interested in live programming problems for your students please reach out!
My colleagues Jack Carlisle and Jay Shah gave a fantastic lecture for @GPU_MODE yesterday on our categorical foundations for CuTe layout algebra! They were joined by Cris Cecka, the inventor of CuTe, and @marksaroufim as moderators. Bravi tutti!
youtu.be/MVh_guNbWMA?si…
our "@pleiasfr and friends" team got second place at the @GPU_MODE hackathon in Paris last week! 🥈
we had a lot of fun optimizing our training throughput, so trying out 8 bit training, muon, RoPE/NoPE, conv architectures, ... Basically nanogpt speedrunning on B300s.
We're back on schedule Tuesday April 14 at 9am PST we'll have a talk from Andrei Panferov and Erik Schultheis on their improved recipe for nvfp4 pretraining. They'll cover both math and kernels youtube.com/watch?v=HB4up2…
Trtllmgen kernels are now open. Fastest prefill and decode kernels for our target workloads. We wrote these to win InferenceX, MLPerf, other benchmarks. Powering some of today’s top served models. Dive in, learn, use them, or level up your own. Enjoy.
github.com/flashinfer-ai/…
195 Followers 1K Followingalways building • exploring healthtech, erp systems for smb, indic speech processing, agritech • day @yourtokenio B2B for D2C, ex @Medlr_in • n&w s2
37K Followers 1K FollowingCEO and co-founder of Core Automation
former VP of RL @ OpenAI : reasoning models, o3, o1, GPT4, ChatGPT, Codex, RL for robots
cautious AI optimist
93K Followers 145 FollowingBuilding beautiful things like Mojo🔥 and MAX @Modular, lifting the world of production AI/ML software into a new phase of innovation. We’re hiring! 🚀🧠
306K Followers 1K FollowingBuilding new things @thinkymachines. Also dabble in robotics at NYU. Cofounded @PyTorch. AI is delicious when it is accessible and open-source.
45K Followers 905 FollowingCreator of bitsandbytes. Professor @CarnegieMellon and Research Scientist @allen_ai . I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.