Modern supercomputers are behind the scene powerhouses for scientific and human advancement applications such as weather prediction, Numerical simulations, bio-informatics, and material research. CDAC was entrusted with design and development of indigenous core technologies for supercomputing and has built several generations of PARAM supercomputing systems. One of the core components of a PARAM supercomputer is PARAMNet interconnect network. Under the NSM project, CDAC has developed three generations of latest PARAMNet interconnects called as Trinetra series of networks. Trinetra design and development effort spans multiple domains related with state of art hardware system design. It has multiple hardware and software components, such as NCC (Network Controller Chip: VLSI communication co-processor design), NIC (Network Interface Card), and LWP (Lightweight protocol) software. Together, these components realize a high bandwidth, low latency, scalable network fabric supporting industry standard programming interfaces. 

Trinetra development has been done in three phases.

1. Trinetra-POC: This was proof of concept phase, with focus being getting know-how about switchless networks, 3D Torus topology, and onload/hybrid architecture of Network Controller Chip. The hardware used was based 40Gbps physical link layer for IO and PCI-e Gen3 2x interface for host interface. The PoC hardware was used for extensive validation of key architectural concepts and hardware/software co-verification.

2. Trinetra-A is based on100Gbps physical link layer, PCI-e  Gen3 X8 host interface, and uses 3D Torus as network topology. All hardware, software and firmware components of the network have been developed successfully. Trinetra-A is currently deployed in PARAM Rudra 1PF system at CDAC Pune.

3. Trinetra-B is the latest hardware developed with performance and topology enhancements, leveraging experience gained during PoC and TrA development phases. TrB is based around PCI-e Gen3, x16 host interface, and 200Gbps physical link layer. Ten links @ 200Gbps allow for Dragonfly/ Supercluster topology. TrB planned to be deployed in PARAM 20PF system.

Trinetra development effort is expected to continue under forthcoming NSM2.0 with primary aim of Indigenous development of Exascale systems.

1. HPC and Enterprise applications

1. High Performance: High bandwidth and low latency, supported by innovative hardware capabilities and a lightweight software stack.
2. Scalability and Power Efficiency: Designed to ensure real performance, scalability, and power efficiency, while facilitating experimentation and application optimization.
3. Topology Flexibility: Supports 3D Torus (Trinetra-A) and Dragonfly/Supercluster (Trinetra-B) topologies for large-scale scalability eliminating need for dedicated switches.
4. Multi-Processor Support: Compatible with multiple processor and platform architectures.
5. Industry Standards Compliance: Supports standard HPC programming interfaces like MPI and legacy socket interfaces.

Trinetra-A:

• 6*100Gbps full duplex interfaces
• PCI-e GEN3x8 host interface
• 3D Torus topology
• NCC-I Co-processor

Trinetra-B:

• 10*200Gbps full duplex interfaces
• PCI-e GEN3x16 host interface
• Dragonfly/ Supercluster topology
• NCC-II Co-processor

System Software

• Includes device drivers and kernel/user components, with OFED support for seamless integration with existing systems.

Application Support:

• Designed to support both HPC and enterprise/socket applications

Name: Atul Bodas

Email: atulb@cdac.in