AI is an Energy Consuming Monster
Few years ago, a hyperscale data center consumed about 50-100 Megawatts (MW) of power. While tech giants today, like Microsoft and Google are planning gigawatt-scale clusters. The Stargate project proposed by US government is forecasting a power requirement of up to 10 Gigawatts (GW).
What does 10GW look like?
- Annual power consumption of Singapore: approximately 58 TWh annually in 2024 (roughly 6.6GW average).
- Annual power consumption of New York City: approximately 50 TWh annually in 2024 (roughly 5.7GW average).
Why do AI data centers need to consume this much energy? A traditional server rack consumed 10kW, while a modern NVIDIA GB200 NVL72 rack consumes 120kW, which has been increasing by more than 10 times.
The "Data Movement Tax"
Communication becomes very critical in AI data center, as it allows shared memory and parallelism, which are the key for trillion parameter model. Jensen Huang, the founder and CEO of NVIDIA said in GTC 2025, the AI server rack is transporting huge amount of data, which is more than the peak capacity of global internet traffic (~900Tbps). This results in very huge power consumption just for moving the data. And this is exactly the "Data Movement Tax" or “Interconnect Wall”.
Just considering only GPU to GPU interconnection in AI data center, a GB300 NVL72 system has 72 GPUs connected through sophisticated scale-up technology, providing roughly 130 TB/s (~1 Pbps) data movement in single rack. Let’s see what power consumptions under different interconnect scenarios are:
*An estimation including the Serializer-Deserializer circuit and the module power consumption
Internal communication within the rack consumes 5-10% of computation power, other data movement behavior includes memory access, on-board PCIe, switch tray, etc., is severely blocking the further scaling of AI data center.
Why Traditional Optics Power
Despite copper cable are bulky and possibly blocking cooling airflow, pluggable optics are not one of the options for scale-up, due to excessive power consumption and cost.
Currently, pluggable optics consume around 20 pJ/bit, mainly due to the existence of digital signal processing chips (DSP). In a standard pluggable transceiver (e.g., 800G OSFP), the electrical signal must travel a long distance from the GPU, through the PCB, to the front panel. By the time the signal arrives, it is distorted and noisy. To fix this, optical modules use a DSP to clean up and reconstruct the signal. But the DSP alone accounts for nearly 70% of the module's power consumption.
Co-packaged optics (CPO), is a new technology by placing the optical engine directly adjacent to the GPU (millimeters away), the electrical path is shortened effectively to zero, so the system can work without using DSP. Industry giants are actively developing their CPO technologies, such as NVIDIA’s Quantum-X switch and Broadcom’s Tomahawk Ethernet Swtich.
RVi's Next Generation CPO
RVi’s XmartLink™ technology is a next-generation CPO solution that delivers superior power efficiency and per-module bandwidth. While traditional optical engines rely on external lasers coupled with silicon modulators, XmartLink™ utilizes high-yield, reliable Micro-VCSELs. By integrating the light source directly, we significantly minimize optical insertion loss, allowing XmartLink™ to achieve a lower power profile than current CPO solutions on the market.