Samsung detailed LPDDR5X-PIM at Hot Chips 2026 on August twenty fifth and twenty sixth, the first processing in memory design for low power DRAM, and the numbers explain why anyone would bother. Sixteen compute blocks sit inside the DRAM banks with their own multiply accumulate trees, giving 614 gigabytes per second of internal bandwidth against 76.8 on the conventional side of the same part. On silicon, running Llama 3.1 8B on Samsung's own edge accelerator, the part finished a 320 token run in five point four seconds instead of twelve point three, and output climbed from twenty seven tokens per second to eighty one point three.
The short answer
Samsung presented LPDDR5X-PIM at Hot Chips 2026 on August 25 and 26. Sixteen processing in memory blocks sit inside the DRAM banks with parallel multiply accumulate trees and an ALU handling both floating point and integer work. Fifteen MAC precision combinations are selectable through the configuration register, a first for low power PIM. On silicon, in Samsung's own edge AI accelerator, a 320 token Llama 3.1 8B run finished in 5.4 seconds instead of 12.3. The package is JEDEC standard with 561 balls and 16 GB across four dies per rank.
Every language model on every device spends most of its time doing the same unglamorous thing: moving weights from memory into a processor so it can multiply them by a vector, then throwing them away and fetching the next batch. The arithmetic is trivial. The moving is the whole cost. Samsung's answer is to stop moving.
The idea, and why the bandwidth number looks odd
A DRAM chip has far more bandwidth inside it than it can ever expose on its pins. The banks are wide and parallel, the external interface is narrow by comparison, and everything the outside world sees has been funnelled through that bottleneck. That gap has existed for decades and mostly did not matter, because most workloads are not bandwidth starved.
Inference is. So Samsung put the compute on the inside of the bottleneck.
LPDDR5X-PIM places sixteen PIM blocks within the DRAM banks. Each has parallel multiply accumulate trees and an ALU that supports floating point and integer operations. Those units see 614 gigabytes per second at the x64 9600 megabits per second operating point. The conventional side of the same part offers 76.8 gigabytes per second.
The eight times figure is the difference between those two, and it is worth reading precisely. The external bus did not get faster. The compute simply moved to where the bandwidth already was.
What Samsung measured
The benchmark is the part that makes this more than an architecture talk, because Samsung built the same system twice.
Running on its own edge AI accelerator SoC, Samsung compared a conventional LPDDR5X build against an LPDDR5X-PIM build on Llama 3.1 8B with a 320 token context, using eight bit signed integer activations, four bit signed integer weights and thirty two bit signed integer output.
The PIM build finished in 5.4 seconds against 12.3, a 2.28 times gain on total runtime. Output throughput climbed from 27.0 tokens per second to 81.3, a 3.01 times gain. The throughput number is larger than the runtime number because prefill does not benefit as much as decode does, which is exactly what you would predict from the mechanism.
Peak arithmetic is 2.4 TOPS at four bit signed integer weights, or roughly 1.2 teraflops per package at FP8. Fifteen precision combinations are selectable through MAC precision fields in the configuration register, which Samsung says makes this the first low power PIM product with multi precision support.
Where it fits, and where it does not
This is low power DRAM. It is not competing with the HBM stacks that sit next to data centre accelerators, and nothing here changes the economics of a rack.
The target is stated as server, mobile and client, and the mobile and client half is the interesting one. Three times the token throughput on an 8 billion parameter model is the difference between an on device assistant that people abandon after a week and one they keep. It also moves work off the network, which removes both a latency floor and a recurring cost.
The obvious limit is the same one that applies to every specialised unit: it accelerates one operation. Matrix vector multiplication against weights held in memory is most of decode, so the win is real, but nothing else in the system gets faster and the prefill phase benefits far less. A 2.28 times overall runtime gain from a 3.01 times decode gain is that arithmetic showing up honestly.
The precision choice deserves a flag too. Four bit weights are normal at the edge now, but the published figures come from the most aggressive quantisation the part supports. Anyone planning around these numbers should measure at the precision they actually intend to ship.
The standardisation question
Silicon validated is not the same as available, and this is where the story is genuinely open.
The specification describes a JEDEC standard 561 ball package with 16 gigabytes across four dies per rank, which is the encouraging part: a standard package means a device maker can adopt it without redesigning a board around a proprietary part. Samsung also said LPDDR6-PIM is in development and heading toward JEDEC standardisation.
That last point decides everything. If PIM lands in a JEDEC standard, it becomes a capability the whole industry can build against and toolchains will follow. If it stays a Samsung specific extension, it will show up in Samsung's own devices and go no further, which has been the fate of most memory side compute proposals over the past thirty years.
Hot Chips 2026 has been unusually memory heavy in general, with Nvidia detailing its Groq 3 LPX decode accelerator at the same event. The common thread across both is that the interesting work has moved from adding arithmetic to arranging not to move data.
What to take from it
If you build for edge devices, this is the clearest signal yet that on device inference gets substantially better in the next hardware generation, and that planning for a hosted fallback everywhere may be over engineering by 2028.
If you buy memory, note that this arrives while Samsung is raising foundry prices by up to 15 percent and memory costs are pushing AI server prices up around 15 percent for early 2027. Compute inside DRAM is, among other things, a way to sell more expensive DRAM.
And if you just want the principle: the bottleneck was never the multiplication. It was the distance.
Sources and further reading
- Samsung LPDDR5X-PIM at Hot Chips 2026, ServeTheHome
- Hot Chips 2026: Samsung makes LPDDR5X smart with logic unit in memory, Tom's Hardware
- New Samsung LPDDR5X DRAM adopts HBM's PIM tech for faster mobile AI, Sammy Fans, August 25, 2026
- Samsung Electronics unveils blueprint for world first computing low power DRAM, The Herald Business
Frequently asked questions
What does processing in memory actually mean here?
It means arithmetic units built into the DRAM die itself, next to the banks, rather than in a separate processor that has to pull the data across a bus. Samsung places sixteen PIM blocks inside the banks, each with parallel multiply accumulate trees and an ALU that handles both floating point and integer work. The point is not that the DRAM becomes a good general purpose processor, because it does not. The point is that for one specific operation, multiplying a vector by a large weight matrix, the data no longer has to leave the chip. That operation happens to be almost all of what a language model does during decode, which is why this is being pitched at inference rather than at anything else.
Where does the 8x bandwidth figure come from?
From the difference between what the PIM units see and what the external interface can deliver. The PIM side reads at 614 gigabytes per second at the x64 9600 megabits per second operating point. The conventional side of the same part offers 76.8 gigabytes per second, which is the number the outside world is limited to. So the eight times is not a claim that the memory got faster on its external bus, it is a statement that the internal banks always had more bandwidth than the pins could carry, and PIM is a way to use it. That framing matters because it explains why the technique helps memory bound work specifically and does nothing for anything else.
What precisions does it support?
Fifteen combinations, selectable through MAC precision fields in the configuration register, which makes this the first low power PIM product with multi precision support. Peak throughput is 2.4 TOPS with four bit signed integer weights, and roughly 1.2 teraflops per package at FP8. The benchmark Samsung published used eight bit signed integer activations with four bit signed integer weights and thirty two bit signed integer output. That is an aggressive quantisation choice and it is a fair one for edge inference, where four bit weights are already normal, but it does mean the headline figures come from the most favourable precision the part supports.
Is this shipping or is it a paper design?
It is silicon validated but not a product you can order. Samsung ran the comparison on its own edge AI accelerator SoC, building the same system twice, once with conventional LPDDR5X and once with LPDDR5X-PIM, which is the right way to do it. The specification describes a JEDEC standard 561 ball package holding 16 gigabytes across four dies per rank, targeting server, mobile and client. Samsung also said LPDDR6-PIM is in development and heading toward JEDEC standardisation, which is the part that actually decides whether this becomes an industry capability or stays a Samsung demo.
Does this change anything for people renting cloud inference?
Not directly, because this is low power DRAM aimed at edge and client devices rather than the HBM stacks that data centre accelerators use. The interesting consequence is on the other side: work that currently goes to a data centre because a phone or a laptop cannot serve a model fast enough may stop needing to. Three times the token throughput on an 8 billion parameter model at the edge is the difference between an on device assistant that feels broken and one that feels usable. The same idea has been applied to HBM before, so the technique is not new, but making it work inside a low power part is what puts it in devices.