Loongson 3A5000/3B5000 Processor Reference Manual - Multicore Processor Architecture, Register Descriptions and System Software Programming Guide

The copyright of this document belongs to Loongson Technology Corporation Limited. Without written permission, no company or individualmay disclose, reproduce or otherwise distribute any part of this document to third parties. Otherwise, they will be held legally responsible.

This document provides only periodic information, and the contents contained may be updated at any time without notice, depending on the actual situation of the product. Loongson Technology Corporation Limited is not responsible for any direct or indirect damage aused by the improper use of the document.

Building No.2, Loongson Industrial Park,
Zhongguancun Environmental Protection Park, Haidian District, Beijing

Tel: 010-62546668

Fax: 010-62600826

This manual introduces the Loongson 3A5000/3B5000 multicore processor architecture and register descriptions. It provides detailed descriptions of the chip system architecture, functions and configurations of the main modules, register lists and bit fields.

These documents were translated by Yanteng Si and Feiyang Chen.

This is the translation of https://github.com/loongson/LoongArch-Documentation/releases/latest/download/Loongson-3A5000-usermanual-v1.03-CN.pdf.

Due to the limited knowledge of the translators, there are some inevitable errors and omissions existing in this document, please feel free to correct.

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

Since the release of the project, we have gotten several errata and content changes donated. Here are all the people who have contributed to LoongArch Documentation as an open source project. Thank you everyone for helping make this a better book for everyone.

The contributors are listed in alphabetical order.

Loongson processors mainly include three series. Loongson Series 1 processor adopts 32-bit processor cores and integrates various peripheral interfaces to form application-specific monolithic solutions, which are mainly applied to IOT terminals, instrumentation devices, data acquisition and other fields. Loongson Series 2 processor adopts 32-bit/64-bit processor cores and integrates various peripheral interfaces to form a high-performance low-power SoC chip for network devices, industrial terminals, intelligent manufacturing, etc. Loongson Series 3 processors integrate multiple 64-bit processor cores and necessary storage and IO interfaces on-chip, targeting high-end embedded computers, desktops, servers and other applications.

The Loongson 3 multi-core series processors are designed based on a scalable multi-core interconnect architecture, which integrates multiple high-performance processor cores and a large amount of Level 2 Cache on a single chip, and interconnects multiple chips through high-speed I/O interfaces to form a larger scale system.

The scalable interconnect architecture adopted by Loongson 3 is shown in the figure below. Each node consists of 8 × 8 cross-switches, with each cross-switch connecting four processor cores and four shared caches, and interconnecting with other nodes in four directions: East (E), South (N), West (W), and North (N).

The node structure of the Loongson 3 is shown in the figure below. Each node has two levels of AXI cross-switches connecting the processor, the shared Cache, the memory controller, and the I/O controller. The first level AXI cross-switch (called X1 Switch) connects the processor and the shared Cache, and the second level cross-switch (called X2 Switch) connects the shared Cache and the memory controller.

In each node, up to 8 × 8 X1 cross switches are connected to four processor cores (P0, P1, P2, P3 in the figure) through four Master ports. The four interleave shared Cache blocks (S0, S1, S2, S3 in the figure) are universally addressed through the four slave ports. Other nodes or I/O nodes in the East, South, West and North directions are connected via four pairs of Master/Slave ports (EM/ES, SM/SS, WM/WS, NM/NS in the figure).

The X2 cross-switch connects four shared Caches via four Master ports, at least one Slave port to a memory controller, and at least one slave port to a configuration module (Xconf) of the cross-switch that is used to configure the address windows of X1 and X2 of this node. Additional memory controllers, I/O ports, can be connected as needed.

The Loongson 3A5000/3B5000 is a quad-core Loongson processor with a stable operating frequency of 2.0-2.5GHz.

The main technical features are as follows:

The architecture of the Loongson 3A5000/3B5000 is designed to increase the shared Cache capacity based on the 3A4000 and supports 16-way interconnect.

The Loongson 3B5000 supports consistent interconnects on the HT0 interface compared to the 3A5000, with special filtering based on server scenario requirements. There is no difference in the other parts from the hardware and software perspective, and they are collectively referred to as 3A5000.

The overall architecture of the Loongson 3A5000 chip is based on multi-level interconnects and is shown in the figure below.

The first level of interconnect uses a 5 × 5 frequency division switch to connect four LA464 cores (as masters), four shared Cache modules (as slaves), and one I/O port to I/O-RING (The I/O port uses one Master and one Slave).

The second level interconnect uses a 5 × 3 cross-switch to connect four shared Cache modules (as masters), two DDR3/4 memory controllers, and one I/O port to the I/O-RING.

The I/O-RING contains 8 ports and the connections include 4 HT controllers, MISC module, SE module and two level cross switches. The two HT controllers (lo/hi) share the 16-bit HT bus, which is used as two 8-bit HT buses, or lo can occupy the 16-bit HT bus exclusively. A DMA controller is integrated into the HT controller, which is responsible for the DMA control of the I/O and the maintenance of inter-chip consistency.

All of these interconnect structures use read/write separated data channels with a 128-bit data channel width operating at the same frequency as the processor core to provide high-speed on-chip data transport. In addition, a one-level cross-switch connects the four processor cores to the SCache with a 256-bit read data channel to increase the read bandwidth of the on-chip processor cores accessing the SCache.

Depending on the structure of the constituent systems, the Loongson 3A5000 consists of two main operating modes. Single-chip mode: The system contains only 1 chip of Loongson 3A5000, which is a symmetric multiprocessor system (SMP). Multi-chip interconnect mode: The system contains 2, 4, or 16 chips of the Loongson 3A5000 interconnected through HT ports to form a non-uniform access multiprocessor system (CC-NUMA).

Main control pins include DO_TEST, ICCC_EN, NODE_ID[2:0], CLKSEL[9:0], and CHIP_CONFIG[5:0].

The Loongson 3A5000 processor can be directly connected with 2/4/8/16 3A5000 chips to build a CC-NUMA system, the processor number of each chip is determined by the pin NODEID, and the address space of each chip is distributed as follows:

When the number of system nodes is less than 16 nodes, the nodemask field of the route setting register (0x1fe00400) should be set to ensure that a response can be obtained even if there is no physical node address when a guessed access occurs (2-way: 0x1; 4-way: 0x3; 8-way: 0x7; 16-way: 0xF).

The Loongson 3A5000 uses a single node 4-core configuration, so the corresponding addresses of the DDR memory controller and HT bus integrated in the Loongson 3A5000 chip are contained in a 44-bit address space from 0x0 (inclusive) to 0x1000_0000_0000 (exclusive). Within each node, the 44-bit address space is further divided among all devices connected within the node, and requests are routed to the four shared Cache modules only when the access type is cached. Depending on the chip and system architecture configuration, if there is no slave device connected on a port, the corresponding address space is reserved address space and access is not allowed.

The address space corresponding to each slave device side of the Loongson 3A5000 chip internal interconnect is as follows:

Unlike the directional port mapping relationship, the Loongson 3A5000 can determine the cross-addressing method of the shared Cache based on the access behavior of the actual application. The address space corresponding to the four shared Cache modules in the node is determined based on one or two select bits of the address bits, and can be dynamically configured and modified by software. A configuration register named SCID_SEL is set to determine the address selection bits, as shown in the following table. By default the [7:6] address hash is used for distribution, i.e., the two bits of address [7:6] determine the corresponding shared Cache number. This register is addressed as 0x1fe00400 and can also be accessed using the configuration register instruction (IOCSR).

The default distribution of the internal 44-bit physical addresses for each node of the Loongson 3A5000 processor is shown in the table below:

The routing of the loongson 3A5000 is mainly implemented through the system’s two-level cross switch with IO-RING. The software can configure the routing of the requests received by each Master port. Each Master port has 8 address windows, and the target routing of 8 address windows can be completed. Each address window consists of three 64-bit registers, BASE, MASK and MMAP, with BASE aligned by K bytes; MASK adopts a format similar to network mask with high bit of 1; the lower four bits of MMAP indicate the number of the corresponding target Slave port; MMAP[4] indicates allow fetch instrustions; MMAP[5] indicates allow block read; MMAP[6] indicates allow interleaved access enable; MMAP[7] indicates window enable.

Window hit formula: (IN_ADDR & MASK) == BASE

Since Loongson 3 uses fixed routing by default, the configuration window is closed at power-up and requires system software to enable it for use.

When the SCache/memory interleaved access configuration is enabled, the slave number is only valid when it is 0 or 4. 0 indicates routing to SCACHE and SCID_SEL determines how interleaved access is performed across the 4 SCaches. 4 indicates routing to memory and interleave_bit determines how interleaved accesses are performed across the 2 MCs.

The address window translation registers are shown in the table below. The base address is 0x1FE0_0000, or accessed via the IOCSR instruction.

The secondary xbar mainly connects 2 memory controllers and IO-RING as slave devices, with 4 SCache (4, representing 4xxx, same as 5, 6, 7) and IO-RING (9) as master devices for window mapping, which can use these window configuration registers (4, 5, 6, 7, 9) for memory window configuration and address translation.

Each address window consists of three 64-bit registers, BASE, MASK, and MMAP, with BASE aligned in K bytes, MASK in a format similar to the network mask high bit 1, and MMAP containing the converted address, routing, and enable control bits, as shown in the following table:

Among them, the devices corresponding to the slave device number are shown in the following table:

The meaning of the window enable bits is shown in the table below:

Note that the window configuration cannot perform address translation for Cache consistency requests, otherwise the address at the SCache will not match the address at the processor-level Cache, resulting in a Cache consistency maintenance error.

Window hit formula: (IN_ADDR & MASK) == BASE

New address conversion formula: OUT_ADDR = (IN_ADDR & ~MASK) | {MMAP[63:10], 10’h0}

According to the default register configuration, the CPU’s address range of 0x00000000-0x0fffffff after the chip is booted (256M) mapped to the address interval 0x00000000-0x0fffffff of the DDR. 0x10000000-0x17ffffff are mapped to the PCI_MEM space of the bridge chip. 0x18000000-0x19ffffff are mapped to the PCI_IO space of the bridge chip. 0x1a000000-0x1affffff are mapped to the bridge chip’s PCI configuration space (Type0). 0x1b000000-0x1bffffff are mapped to the bridge chip’s PCI configuration space (Type1). 0x40000000-0x7fffffff are mapped to the bridge chip’s PCI_MEM space. Software can implement the new address space routing and translation by modifying the corresponding configuration registers.

In addition, when there is a read access to an illegal address due to CPU guessing execution, all 8 address windows are not hit and random data is returned to prevent the CPU from dying, etc.

The offset address is 0x0000.

This register identifies a number of software-related processor features for software to view before enabling a specific function. The offset address is 0x0008.

This register is used to identify the vendor name. The offset address is 0x0010.

This register is used to identify the chip name. The offset address is 0x0020.

The offset address is 0x0180.

The offset address is 0x0188.

The offset address is 0x0190.

The offset address is 0x0198.

The following sets of software multiplier setting registers are used to set the operating frequency of the chip master clock and the memory controller clock when CLKSEL is configured in software control mode (refer to the CLKSEL setting method in Descriptions of Pins).

Among other things, the MEM CLOCK configuration supports multiple modes. In 4x mode (mem div), MEM CLOCK should be 4x the memory controller clock; in 2x mode (mem div), MEM CLOCK should be 2x the memory controller clock; in 1x mode (mem div), MEM CLOCK should be the memory controller clock frequency .

The memory bus operates at 2 times the memory controller clock and the bus operates at 4 times the memory controller clock.

NODE CLOCK corresponds to the clock frequency of the processor core, on-chip network, and shared Cache.

Each clock configuration generally has three parameters, DIV_REFC, DIV_LOOPC, and DIV_OUT. The final clock frequency is (reference clock / DIV_REFC * DIV_LOOPC) / DIV_OUT.

In software control mode, the default corresponding clock frequency is the frequency of the external reference clock (100MHz or 25MHz), which needs to be set in software during the processor startup. The process of setting the individual clocks should be done as follows:

The following register is the configuration register of Main CLOCK, Main Clock is used to generate the maximum operating frequency of node clock, core clock, etc. The base address is 0x1fe00000 and the offset address is 0x1b0:

Note: PLL ouput = (clk_ref /div_refc * div_loopc) / div_out.

The result of clk_ref/div_refc for the PLL should be 25/50/100MHz, with 50MHz recommended. The VCO frequency (the part in parentheses in the above equation) must be in the range 4.8GHz-6.4GHz. This requirement also applies to memory PLLs.

In addition, the recommended setting for div_loopc is less than 255. The recommended setting for div_out is 1/2/4/6 and above 6, and 3/5 is not recommended.

The following register is the MEM CLOCK configuration register, the MEM CLOCK clock frequency should be configured to 1/2 of the final DDR bus clock frequency. The base address is 0x1fe00000, offset address is 0x1c0:

The following registers are used for dynamic frequency division of the processor core. Using this register to set the frequency of the processor core, the frequency conversion operation can be done within 100ns with no additional overhead. The base address is 0x1fe00000 and the offset address is 0x01d0.

Note: The clock frequency value after software dividing is equal to the original (dividing control value + 1)/8.

The following registers are used for software-controlled reset of the processor core. To reset, set resetn to 0, resetn_pre to 0, wait 500 microseconds, resetn_pre to 1, and reset n to 1 to complete the reset process. The base address of this register is 0x1fe00000 and the offset address is 0x01d8.

The following registers are used to control some of the routing settings within the chip. The base address is 0x1fe00000 and the offset address is 0x0400.

The following registers are used to control some of the functions enabled within the chip. The base address is 0x1fe00000 and the offset address is 0x0420.

The following registers are used to observe the chip internal temperature sensor values. In degrees Celsius. The base address is 0x1fe00000 and the offset address is 0x0428. This register is available only when CSR[0x0008][0] is valid.

The following registers are used to adjust the operating frequency of Sram inside the processor core. The offset address is 0x0430.

The following registers are used to observe the Fuse0 values visible to some software. The offset address is 0x0460.

The following registers are used to observe the Fuse1 values visible to some software. The offset address is 0x0470.