NTT has demonstrated the potential for IOWN APN in two tests this week. It successfully connected two data centres in the US and two data centres in the UK over 4km and 89km, respectively. In both cases, it sustained connectivity of less than 1 millisecond of network delay. This compares to a typical connection speed of over 2 milliseconds.
The company announced the success of the tests at its Upgrade 2024 conference in San Francisco. According to Vito Mabrucco, Chief Marketing Officer, NTT, “We’re connecting data centres at a level of latency and speed, where the data centres don’t know that they’re physically in different places. Having GPU in one place and data in another, to the system, it looks like one computer even though it’s 80 kilometres apart.”
Why did NTT do these tests?
At first glance, these tests can be seen as proving a faster fibre interconnect between data centres. These first tests are to show that you can get a sub-millisecond speed of data transfer over a wider distance than before. Even with 30 years of fibre interconnect, there are still limits to the speed with which we can transfer data. Those limits are caused by the components we use in our networking stack.
To overcome those limits, NTT has deployed technologies it has been working on for several years. At the core of this is IOWN, which stands for Innovative Optical and Wireless Network. It is a photonics-based infrastructure that NTT believes will replace many of the silicon-based components we currently rely on.
As part of this test, NTT used an all-photonics-network (APN) between the pairs of data centres. It removed much of the latency that would otherwise have slowed down the transmission speed. Among the use cases that NTT sees this technology appealing to are:
- Real-time AI analytics
- Financial services.
- Industrial IoT
- Predictive maintenance
- Smart surveillance systems
- Smart grid
- Energy management
- Natural disaster detection and response
NTT is already proving the technology through a joint project with the Japanese financial services organisation MUFG. NTT Data and MUFG are building the Financial Services APN infrastructure. It uses an APN that allows for near-instant migration of workloads in case of disasters or in case of failures. For banks and financial services, this is a major resilience and availability type solution.
Enabling a new model for disaggregated computing over distance
Interestingly, this is about more than faster backup. Other use cases are opening up, such as disaggregated computing.
The idea of disaggregated computing is nothing new. It has been around for more than a decade. The idea is that you can have computing and storage in different places, and they talk to each other without moving data to the compute. At present, however, it is hard to make it work at speed due to latency and other issues.
Mabrucco sees the potential for IOWN APN networks to resolve that problem. In a podcast with Enterprise Times to be published shortly, he talks about a project that NTT is working on with NVIDIA and Red Hat around training large language models (LLMs).
GPUs are housed in a custom data centre designed for their power requirements and the need to dissipate the heat they generate. The other holds the data. The goal is to have the GPUs build the LLMs without moving the data. Using today’s technology requires significant amounts of caching, replication and a high-speed WAN.
What the three companies are testing is the goal of leaving the data where it is and using the ultra-low latency of an APN to allow the data to be consumed by the GPUs. Mabrucco says that doing this with existing technology will take two to three weeks under the current test. The goal is to reduce that to under one week. It will also require much less power and generate far less heat.
Addressing datacentre issues
There are three traditional problems with the data centre – heat, power and location. Mabrucco says that as photonics under the IOWN banner develops, all of these can be addressed as part of the disaggregated compute model.
He sees power-hungry data centres built next to the power-generating facilities, especially where those are renewables. These could house the CPUs and GPUs that consume the most power. They would talk to smaller data centres housing the data, which could be distributed in other locations.
However, there are many steps that need to happen first. NTT must prove the distance over which disaggregated computing is viable, hence the NVIDIA and Red Hat project. It also needs to move into the data centre to deal with how silicon generates heat and change that. One technology for doing that is a photonics electronics convergence device (PEC). Mabrucco talks about it in the podcast.
In brief, it is a technology that replaces silicon-based systems. NTT is working to miniaturise it through a new company called NTT Innovative Devices. The eventual goal is to have them small enough to be embedded into a chip, and NTT is talking with Qualcomm and NVIDIA about this.
If photonics are deployed inside computers and at the chip level, Mabrucco sees much of the heat and power demand data centres create disappearing.
Enterprise Times: What does this mean?
This test of NTT’s IOWN APN delivering sub-millisecond speeds over distances up to 89 km is likely to be just the start. The company is likely to push the distances to see how far it can go while maintaining that low latency, low jitter network.
It will be interesting to see how many different cable environments it will test. Testing brand-new fibre is less insightful than using multiple generations of existing fibre. For this technology to have the maximum impact, it has to work with both new and old fibre.
Another key moment will be when NTT Data and MUFG deliver their first white paper on the Financial Services APN infrastructure. If it exceeds existing technology, expect financial services companies to ask their data centre providers when it will become available.
For the disaggregated model, the NTT, NVIDIA, and Red Hat numbers will be interesting. Hit that sub one week or even lower, and suddenly, a raft of new business models open up.
While this announcement is about two data centre tests, it is just the start. Expect to hear more about this and other IOWN use cases at the IOWN General Forum in Vancouver at the end of April.
