Beyond GPU Memory Limits with Unified Memory On Pascal > 자유게시판

• 목록
• 답변
• 글쓰기
• 게시판 리스트 옵션
  • 수정
  • 삭제

Modern laptop architectures have a hierarchy of memories of various measurement and performance. GPU architectures are approaching a terabyte per second memory bandwidth that, coupled with high-throughput computational cores, creates a perfect device for Memory Wave knowledge-intensive duties. Nonetheless, all people knows that quick memory is expensive. Modern functions striving to resolve larger and bigger problems may be restricted by GPU memory capacity. For the reason that capacity of GPU memory is significantly lower than system memory, it creates a barrier for Memory Wave builders accustomed to programming just one memory house. With the legacy GPU programming model there isn't any straightforward option to "just run" your application when you’re oversubscribing GPU memory. Even if your dataset is simply barely larger than the available capability, you'll nonetheless must manage the active working set in GPU memory. Unified Memory is a way more intelligent Memory Wave Program management system that simplifies GPU improvement by offering a single memory house instantly accessible by all GPUs and CPUs within the system, with automated web page migration for data locality.

Migration of pages permits the accessing processor to profit from L2 caching and the decrease latency of native memory. Moreover, migrating pages to GPU memory ensures GPU kernels take advantage of the very excessive bandwidth of GPU memory (e.g. 720 GB/s on a Tesla P100). And web page migration is all completely invisible to the developer: the system robotically manages all data motion for you. Sounds great, proper? With the Pascal GPU structure Unified Memory is much more powerful, thanks to Pascal’s bigger virtual memory handle space and Page Migration Engine, enabling true virtual memory demand paging. It’s also value noting that manually managing memory motion is error-prone, which affects productivity and delays the day when you can lastly run your entire code on the GPU to see these nice speedups that others are bragging about. Developers can spend hours debugging their codes because of memory coherency issues. Unified memory brings enormous advantages for developer productiveness. In this post I'll present you ways Pascal can allow functions to run out-of-the-field with bigger memory footprints and obtain great baseline performance.

For a moment you may utterly forget about GPU memory limitations while developing your code. Unified Memory was introduced in 2014 with CUDA 6 and the Kepler architecture. This relatively new programming model allowed GPU functions to make use of a single pointer in both CPU capabilities and GPU kernels, which enormously simplified memory management. CUDA 8 and the Pascal architecture significantly improves Unified Memory performance by adding 49-bit digital addressing and on-demand page migration. The large 49-bit virtual addresses are ample to enable GPUs to entry the entire system memory plus the memory of all GPUs within the system. The Page Migration engine permits GPU threads to fault on non-resident memory accesses so the system can migrate pages from anywhere within the system to the GPUs memory on-demand for efficient processing. In different words, Unified Memory transparently enables out-of-core computations for any code that's utilizing Unified Memory for allocations (e.g. `cudaMallocManaged()`). It "just works" with none modifications to the application.

CUDA 8 also adds new methods to optimize knowledge locality by providing hints to the runtime so it continues to be possible to take full control over data migrations. Lately it’s laborious to find a high-efficiency workstation with only one GPU. Two-, four- and eight-GPU techniques are becoming frequent in workstations in addition to massive supercomputers. The NVIDIA DGX-1 is one instance of a excessive-performance built-in system for deep studying with 8 Tesla P100 GPUs. In case you thought it was difficult to manually handle data between one CPU and one GPU, now you might have eight GPU memory areas to juggle between. Unified Memory is essential for such systems and it enables more seamless code development on multi-GPU nodes. At any time when a particular GPU touches information managed by Unified Memory, this knowledge may migrate to local memory of the processor or the driver can establish a direct access over the accessible interconnect (PCIe or NVLINK).