
Key points
- The build uses a 40-core 3rd-gen Xeon Scalable CPU with discontinued Optane PMem 200 modules.
- Inference speed reaches 0.85 tokens per second with 1-bit quantization, slower than previous builds.
- Active DIMM cooling is mandatory due to high thermal output from the memory modules.
Tom's Hardware reported that a Reddit homelab user successfully ran Moonshot AI’s 2.8-trillion-parameter Kimi K3 model on CPU hardware alone, avoiding the need for dedicated graphics accelerators.reported The system relies on second-generation Optane PMem 200 memory sticks and a 40-core 3rd-gen Xeon Scalable processor to handle the inference workload entirely within the central processing unit.reportedA user proved it is possible to run massive AI models on standard computer processors by using specialized, older memory technology.The inference workload bypasses GPU acceleration entirely, relying on Optane PMem 200 bandwidth and Xeon CPU throughput to process the 2.8T-parameter model.
Evidence: reported
Homelabber uses second-gen Optane memory sticks to run a 2.8-trillion-parameter AI model on CPU aloneaifleet-tomshardware
Evidence: reported
Homelabber uses second-gen Optane memory sticks to run a 2.8-trillion-parameter AI model on CPU aloneaifleet-tomshardware
The evidence
The builder achieved a throughput of 0.85 tokens per second when utilising a 1-bit quantization of the model, according to the published report This performance metric represents a significant reduction compared to earlier experiments, such as a first-generation Optane build that delivered roughly four tokens per second.reported That previous iteration benefited from an RTX 3060 handling part of the model, a larger 2-bit quantization, and the smaller 1-trillion-parameter K2.5 model.reported
Evidence: reported
Homelabber uses second-gen Optane memory sticks to run a 2.8-trillion-parameter AI model on CPU aloneaifleet-tomshardware
Evidence: reported
Homelabber uses second-gen Optane memory sticks to run a 2.8-trillion-parameter AI model on CPU aloneaifleet-tomshardware
The decision
Tom's Hardware noted that Moonshot AI recommends data centre inference engines for Kimi K3, indicating the model is not designed for home deployment.reported Even the smallest community quantization of the model requires at least 610GB of RAM, a capacity that is difficult to reach with standard DRAM due to skyrocketing prices.reported Using secondhand Optane memory is presented as one of the few viable methods to achieve this capacity at home for a fraction of the cost.
Evidence: reported
Homelabber uses second-gen Optane memory sticks to run a 2.8-trillion-parameter AI model on CPU aloneaifleet-tomshardware
Evidence: reported
Homelabber uses second-gen Optane memory sticks to run a 2.8-trillion-parameter AI model on CPU aloneaifleet-tomshardware
The takeaway
The project runs hot, requiring dual 12W blower fans aimed at the memory modules to keep temperatures under control, with the builder advising that active DIMM cooling is mandatory.reported Performance decreases further with higher precision quantization; using a 3-bit quantization dropped the speed to 0.75 tokens per second.reported
Evidence: reported
Homelabber uses second-gen Optane memory sticks to run a 2.8-trillion-parameter AI model on CPU aloneaifleet-tomshardware
Evidence: reported
Homelabber uses second-gen Optane memory sticks to run a 2.8-trillion-parameter AI model on CPU aloneaifleet-tomshardware
Building on this
Developers can explore Intel’s Optane Persistent Memory 200 Series, which supports both Memory Mode and App Direct Mode on two-socket Xeon platforms.reported Memory Mode uses DRAM as a cache for the Optane acting as system memory, a method used in the earlier first-generation build. App Direct Mode exposes Optane as a separate, persistent region that software can map directly, which the creator of the first build, u/APFrisco, stated could offer more control over where model tensors reside. The second-generation modules deliver up to 32% more memory bandwidth than the first generation, operating at a higher effective MT/s rate.reported
Evidence: reported
Homelabber uses second-gen Optane memory sticks to run a 2.8-trillion-parameter AI model on CPU aloneaifleet-tomshardware
Evidence: reported
Homelabber uses second-gen Optane memory sticks to run a 2.8-trillion-parameter AI model on CPU aloneaifleet-tomshardware
Why it matters
- Enterprise IT buyers can assess the viability of persistent memory for large model inference where GPU clusters are unavailable or too costly.
- The cost differential between secondhand Optane and equivalent DRAM capacity highlights an alternative procurement strategy for high-memory workloads.
- Thermal management requirements for Optane PMem 200 in high-load inference scenarios must be factored into chassis and cooling designs.
- Performance trade-offs between quantization levels and throughput provide data for optimizing batch processing jobs that tolerate high latency.
Topics: Kimi K3, Optane PMem, CPU inference, quantization, homelab
Updated
Then / now / next
A previous homelab build used first-gen Optane and an RTX 3060 to run the smaller K2.5 model at roughly four tokens per second.
A CPU-only build using second-gen Optane PMem 200 runs the larger Kimi K3 model at 0.85 tokens per second with 1-bit quantization.
Developers may monitor community discussions for further optimizations in tensor placement using App Direct Mode on persistent memory.
Story timeline
- Then A previous homelab build used first-gen Optane and an RTX 3060 to run the smaller K2.5 model at roughly four tokens per second.
- 08 Nov 2026 Homelabber uses second-gen Optane memory sticks to run a 2.8-trillion-parameter AI model on CPU alone
- Now A CPU-only build using second-gen Optane PMem 200 runs the larger Kimi K3 model at 0.85 tokens per second with 1-bit quantization.
- Next Developers may monitor community discussions for further optimizations in tensor placement using App Direct Mode on persistent memory.
The strongest doubt
Counterpoint from the sources
Moonshot AI recommends data centre inference engines for Kimi K3, suggesting the model is not intended for home or low-resource environments.
For readers
Developers
- App Direct Mode allows direct mapping of persistent memory regions for tensor placement control.
- 1-bit quantization enables fitting large models into available memory but reduces inference speed.
Business
- Secondhand Optane memory offers a lower-cost alternative to equivalent DRAM capacity for high-memory needs.
- CPU-only inference avoids GPU procurement costs but introduces significant latency for large models.
Researchers
- Performance comparison shows second-gen Optane delivers up to 32% more bandwidth than first-gen.
- Thermal management is critical for sustained high-load inference on persistent memory modules.
Ask the article
Local keyword search of this article; no generated answers.Did this article give you the evidence you needed?
Your choice stays on this device. No public tally.Terms in this story
Optane PMem 200
Intel's second-generation persistent memory modules launched in 2020 for 3rd-gen Xeon CPUs.
App Direct Mode
A mode where Optane is exposed as a separate persistent region for direct software mapping.
Quantization
A technique to reduce model size and memory usage, such as 1-bit or 3-bit quantization.
Tokens per second
A metric measuring the speed of text generation or processing in AI models.
Questions readers ask
What hardware was used to run the Kimi K3 model?
The build used a 40-core 3rd-gen Xeon Scalable CPU and second-gen Optane PMem 200 memory sticks, without any GPU involvement.
How fast is the inference speed with 1-bit quantization?
The system achieved 0.85 tokens per second with 1-bit quantization, according to the report by Tom's Hardware.
Is active cooling required for this setup?
Yes, the builder advised treating active DIMM cooling as mandatory, using dual 12W blower fans to control temperatures.
Reading receipt
- The build uses a 40-core 3rd-gen Xeon Scalable CPU with discontinued Optane PMem 200 modules.
- Inference speed reaches 0.85 tokens per second with 1-bit quantization, slower than previous builds.
- Active DIMM cooling is mandatory due to high thermal output from the memory modules.