New FPGA Accelerator LUT-LLM Outperforms Top GPUs in Speed and Efficiency

Researchers have created LUT-LLM, a groundbreaking FPGA accelerator for large language models. It is the first system of its kind capable of running models with over one billion parameters using memory-based operations. The design promises faster performance and lower energy use than top GPUs. LUT-LLM replaces traditional arithmetic with fast table lookups. This approach improves speed and cuts power consumption. The system uses activation-weight co-quantization and two-dimensional lookup tables to store pre-computed results, reducing repeated calculations.

The accelerator features a spatial-temporal hybrid design. This balances dataflow and sequential processing to boost throughput and shrink buffer needs. Vector quantization further trims memory use, enabling efficient table construction.

Tested on a V80 FPGA, LUT-LLM delivered a 1.66x speedup over the AMD MI210 GPU. It also achieved a 4.1x energy efficiency gain. Against the NVIDIA A100, it reduced latency by 1.66x and improved energy efficiency by 1.72x. Bandwidth-aware parallel centroid search and prefix-sum-based 2D lookups enhance its performance.

Scalability tests showed even greater gains. When extended to the Qwen 3 32B model, energy efficiency improved by another 2.16x. LUT-LLM outperforms leading GPUs in speed and energy use. Its memory-based design and hybrid processing offer a new path for efficient large language model acceleration. The system demonstrates clear advantages in both performance and scalability.