The picture on the preceding page is more complex than it would be for a CPU, because the GPU reserves certain areas of memory for specialized useduring rendering. Here, we summarize the roles of each type of GPU memory for doing GPGPU computations.
The first list covers the on-chip memory areas that are closest to the CUDA cores.They are part of every SM.
See Also
L2 cache definition – Glossarylevel 2 childcare course, childcare courses, Montessori course, dubai, sharjah, abudhabi UAEA Comprehensive Guide to CACHE Qualification- Register File - denotes the area of memory that feeds directly into the CUDA cores. Accordingly, it is organized into 32 banks,matching the 32 threads in a warp. Think of the registerfile as a big matrix of 4-byte elements, having many rows and 32 columns. A warp operates on full rows; within a given row, each thread(CUDA core) operates on a different column (bank).
- L1 Cache - refers to the usual on-chip storage location providing fast access to data that are recently read from, or written to, mainmemory (RAM). Additionally, L1 serves as the overflow region when the amount of active data exceeeds what an SM's register file can hold, a conditionwhich is termed "register spilling". In L1, the cache lines and spilled registers are organized into banks, just as in the register file.
- Shared Memory - is a memory area that physically resides in the same memory as the L1 cache, but differs from L1 in that all its data maybe accessed by any thread in a thread block. This allows threads to communicate and share data with each other. Variables that occupy it must bedeclared explicitly by an application. The application can also set the dividing line between L1 and shared memory.
- Constant Caches - are special caches pertaining to variables declared as read-only constants in global memory. Such variables can be read byany thread in a thread block. The main and best use of these caches is to broadcast a single constant value to all the threads in a warp.
The second list pertains to the more distant, larger memory areas that are shared by all the SMs.
- L2 Cache - is a further on-chip cache for retaining copies of the data that travel back and forth between the SMs and main memory. Like theL1, the L2 cache is intended to speed up subsequent reloads. But unlike the L1 cache(s), there is just one L2 that is shared by all the SMs. The L2cache is also situated in the path of data moving on or off the device via PCIe or NVLink.
- Global Memory - represents the bulk of the main memory of the device, equivalent to RAM in a CPU-based processor. For performance reasons,the Tesla V100 has special HBM2 high-bandwidth memory, while the Quadro RTX 5000 has fast GDDR6 graphics memory.
- Local Memory - corresponds to specially mapped regions of main memory that are assigned to each SM. Whenever "register spilling" overflowsthe L1 cache on a particular SM, the excess data are further offloaded to L2, then to "local memory". The performance penalty for reloading aspilled register becomes steeper for every memory level that must be traversed in order to retrieve it.
- Texture and Constant Memory - are regions of main memory that are treated as read-only by the device. When fetched to an SM, variables witha "texture" or "constant" declaration can be read by any thread in a thread block, much like shared memory. Texture memory is cached in L1, whileconstant memory is cached in the constant caches.
