ETRI breaks the “memory wall” in large-scale AI training

South Korean researchers have successfully developed a core technology that can fundamentally resolve “memory shortages,” a chronic bottleneck in large-scale artificial intelligence (AI) training. This technology is a next-generation memory expansion technology based on Ethernet, which is expected to drive infrastructural innovation across the entire AI and big data industries in the future.

Electronics and Telecommunications Research Institute (ETRI) announced that it has developed “OmniXtend,” a new memory technology that overcomes GPU memory capacity limits and data movement overheads, which are regarded as the biggest problems in large-scale AI training.

Recently, as demand for large-scale AI models and high-performance computing (HPC) has surged, the volume of data to be processed is growing exponentially. However, no matter how much GPU performance improves, the problem of the “Memory Wall”—where computational efficiency drops sharply due to memory capacity—has remained an unresolved challenge.

OmniXtend, developed by ETRI, addresses this by leveraging standard Ethernet as a memory interconnect fabric, enabling memory sharing across servers and accelerators, effectively treating distributed resources as a single, massive “Memory Pool”.

In other words, memory resources that were traditionally tightly coupled and locally constrained are now disaggregated and exposed over the network, allowing dynamic and scalable allocation of memory capacity for AI workloads.

As AI models continue to scale in size and the amount of memory they require keeps increasing, a “scalable architecture that shares and utilizes memory in the form of a pool” is gaining attention as a key technology for next-generation AI infrastructure.

ETRI’s OmniXtend demonstrated this scalable shared-memory architecture over Ethernet, achieving performance, scalability, and cost-efficiency of hyperscale AI training at the same time.

First, by minimizing data movement latency, it improved AI training speed. In addition, memory capacity can be expanded without replacing servers, leading to reduced data center deployment and operational costs.

In particular, conventional architectures based on high-speed serial interfaces such as PCIe had limitations in inter-device connectivity distance and system scalability. In contrast, OmniXtend leverages conventional Ethernet switches to aggregate multiple physically distributed devices into a unified memory pool, making it well-suited for highly scalable, large-scale AI system environments.

ETRI researchers developed key enabling technologies including ▲a Field-Programmable Gate Array (FPGA)-based memory expansion node and ▲an Ethernet-based memory transfer engine, and verified the stable operation of the system.

In an actual demonstration, they successfully showed multiple devices in an Ethernet environment forming a shared memory pool and accessing each other’s memory in real time.

Furthermore, through a computational workload test using a large language model (LLM), they confirmed that the OmniXtend architecture contributes to performance improvement even in actual AI training environments. Experimental results showed that in environments with insufficient memory capacity, LLM inference performance degraded significantly, whereas when memory was expanded using Ethernet, performance recovered by more than twofold. This demonstrates that it can maintain processing performance at a level similar to that of a conventional environment with sufficient memory.

ETRI drew significant attention by unveiling the technology at “RISC-V Summit Europe 2025,” held in Paris, France, in May 2025, and “RISC-V Summit North America 2025,” held in Santa Clara, the United States.

In addition, ETRI is leading the Interconnect Working Group under the CHIPS Alliance of the Linux Foundation, contributing to the establishment and global dissemination of open-source standards for AI networking and memory expansion.

ETRI plans to promote commercialization of this technology in the future by transferring this technology, particularly to data center hardware and software companies. This technology is expected to be applied to AI training and inference servers, memory expansion devices, and network switches, thereby generating tangible industrial impact in the next-generation AI infrastructure market.

Furthermore, ETRI plans to pursue follow-up research to expand this technology into a large-capacity memory interconnect for high-reliability embedded systems such as automotives and ships, and to advance a shared-memory architecture across heterogeneous accelerators such as NPUs, GPUs, and CPUs.

Kim Kang Ho, Assistant Vice President of the Future Computing Research Division at ETRI, said, “We plan to significantly expand research on memory interconnect technologies centered on neural processing units (NPUs) and accelerators through new project initiatives,” adding, “We will continue to advance the technology and strengthen international collaboration to ensure its adoption in next-generation systems of global AI and semiconductor companies.”

###

This research was conducted as part of the “Research on Memory-Centric Next-Generation Computing System Architecture” project under the SW Computing Industry Original Technology Development Program, supported by the Ministry of Science and ICT and the Institute of Information and Communications Technology Planning and Evaluation (IITP).

About Electronics and Telecommunications Research Institute (ETRI)

ETRI is a non-profit government-funded research institute. Since its foundation in 1976, ETRI, a global ICT research institute, has been making its immense effort to provide Korea a remarkable growth in the field of ICT industry. ETRI delivers Korea as one of the top ICT nations in the World, by unceasingly developing world’s first and best technologies.