codekingpro/portable-devtools

1/*2 * SPDX-FileCopyrightText: Copyright (c) 2021 NVIDIA CORPORATION & AFFILIATES. All rights reserved.3 *4 * NVIDIA SOFTWARE LICENSE5 *6 * This license is a legal agreement between you and NVIDIA Corporation ("NVIDIA") and governs your use of the NVIDIA/CUDA C++ Library software and materials provided hereunder (“SOFTWARE”).7 *8 * This license can be accepted only by an adult of legal age of majority in the country in which the SOFTWARE is used. If you are under the legal age of majority, you must ask your parent or legal guardian to consent to this license. By taking delivery of the SOFTWARE, you affirm that you have reached the legal age of majority, you accept the terms of this license, and you take legal and financial responsibility for the actions of your permitted users.9 *10 * You agree to use the SOFTWARE only for purposes that are permitted by (a) this license, and (b) any applicable law, regulation or generally accepted practices or guidelines in the relevant jurisdictions.11 *12 * 1. LICENSE. Subject to the terms of this license, NVIDIA grants you a non-exclusive limited license to: (a) install and use the SOFTWARE, and (b) distribute the SOFTWARE subject to the distribution requirements described in this license. NVIDIA reserves all rights, title and interest in and to the SOFTWARE not expressly granted to you under this license.13 *14 * 2. DISTRIBUTION REQUIREMENTS. These are the distribution requirements for you to exercise the distribution grant:15 * a.      The terms under which you distribute the SOFTWARE must be consistent with the terms of this license, including (without limitation) terms relating to the license grant and license restrictions and protection of NVIDIA’s intellectual property rights.16 * b.      You agree to notify NVIDIA in writing of any known or suspected distribution or use of the SOFTWARE not in compliance with the requirements of this license, and to enforce the terms of your agreements with respect to distributed SOFTWARE.17 *18 * 3. LIMITATIONS. Your license to use the SOFTWARE is restricted as follows:19 * a.      The SOFTWARE is licensed for you to develop applications only for use in systems with NVIDIA GPUs.20 * b.      You may not reverse engineer, decompile or disassemble, or remove copyright or other proprietary notices from any portion of the SOFTWARE or copies of the SOFTWARE.21 * c.      You may not modify or create derivative works of any portion of the SOFTWARE.22 * d.      You may not bypass, disable, or circumvent any technical measure, encryption, security, digital rights management or authentication mechanism in the SOFTWARE.23 * e.      You may not use the SOFTWARE in any manner that would cause it to become subject to an open source software license. As examples, licenses that require as a condition of use, modification, and/or distribution that the SOFTWARE be (i) disclosed or distributed in source code form; (ii) licensed for the purpose of making derivative works; or (iii) redistributable at no charge.24 * f.      Unless you have an agreement with NVIDIA for this purpose, you may not use the SOFTWARE with any system or application where the use or failure of the system or application can reasonably be expected to threaten or result in personal injury, death, or catastrophic loss. Examples include use in avionics, navigation, military, medical, life support or other life critical applications. NVIDIA does not design, test or manufacture the SOFTWARE for these critical uses and NVIDIA shall not be liable to you or any third party, in whole or in part, for any claims or damages arising from such uses.25 * g.      You agree to defend, indemnify and hold harmless NVIDIA and its affiliates, and their respective employees, contractors, agents, officers and directors, from and against any and all claims, damages, obligations, losses, liabilities, costs or debt, fines, restitutions and expenses (including but not limited to attorney’s fees and costs incident to establishing the right of indemnification) arising out of or related to use of the SOFTWARE outside of the scope of this Agreement, or not in compliance with its terms.26 *27 * 4. PRE-RELEASE. SOFTWARE versions identified as alpha, beta, preview, early access or otherwise as pre-release may not be fully functional, may contain errors or design flaws, and may have reduced or different security, privacy, availability, and reliability standards relative to commercial versions of NVIDIA software and materials. You may use a pre-release SOFTWARE version at your own risk, understanding that these versions are not intended for use in production or business-critical systems.28 *29 * 5. OWNERSHIP. The SOFTWARE and the related intellectual property rights therein are and will remain the sole and exclusive property of NVIDIA or its licensors. The SOFTWARE is copyrighted and protected by the laws of the United States and other countries, and international treaty provisions. NVIDIA may make changes to the SOFTWARE, at any time without notice, but is not obligated to support or update the SOFTWARE.30 *31 * 6. COMPONENTS UNDER OTHER LICENSES. The SOFTWARE may include NVIDIA or third-party components with separate legal notices or terms as may be described in proprietary notices accompanying the SOFTWARE. If and to the extent there is a conflict between the terms in this license and the license terms associated with a component, the license terms associated with the components control only to the extent necessary to resolve the conflict.32 *33 * 7. FEEDBACK. You may, but don’t have to, provide to NVIDIA any Feedback. “Feedback” means any suggestions, bug fixes, enhancements, modifications, feature requests or other feedback regarding the SOFTWARE. For any Feedback that you voluntarily provide, you hereby grant NVIDIA and its affiliates a perpetual, non-exclusive, worldwide, irrevocable license to use, reproduce, modify, license, sublicense (through multiple tiers of sublicensees), and distribute (through multiple tiers of distributors) the Feedback without the payment of any royalties or fees to you. NVIDIA will use Feedback at its choice.34 *35 * 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS" WITHOUT ANY EXPRESS OR IMPLIED WARRANTY OF ANY KIND INCLUDING, BUT NOT LIMITED TO, WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT, OR FITNESS FOR A PARTICULAR PURPOSE. NVIDIA DOES NOT WARRANT THAT THE SOFTWARE WILL MEET YOUR REQUIREMENTS OR THAT THE OPERATION THEREOF WILL BE UNINTERRUPTED OR ERROR-FREE, OR THAT ALL ERRORS WILL BE CORRECTED.36 *37 * 9. LIMITATIONS OF LIABILITY. TO THE MAXIMUM EXTENT PERMITTED BY LAW, NVIDIA AND ITS AFFILIATES SHALL NOT BE LIABLE FOR ANY SPECIAL, INCIDENTAL, PUNITIVE OR CONSEQUENTIAL DAMAGES, OR ANY LOST PROFITS, PROJECT DELAYS, LOSS OF USE, LOSS OF DATA OR LOSS OF GOODWILL, OR THE COSTS OF PROCURING SUBSTITUTE PRODUCTS, ARISING OUT OF OR IN CONNECTION WITH THIS LICENSE OR THE USE OR PERFORMANCE OF THE SOFTWARE, WHETHER SUCH LIABILITY ARISES FROM ANY CLAIM BASED UPON BREACH OF CONTRACT, BREACH OF WARRANTY, TORT (INCLUDING NEGLIGENCE), PRODUCT LIABILITY OR ANY OTHER CAUSE OF ACTION OR THEORY OF LIABILITY, EVEN IF NVIDIA HAS PREVIOUSLY BEEN ADVISED OF, OR COULD REASONABLY HAVE FORESEEN, THE POSSIBILITY OF SUCH DAMAGES. IN NO EVENT WILL NVIDIA’S AND ITS AFFILIATES TOTAL CUMULATIVE LIABILITY UNDER OR ARISING OUT OF THIS LICENSE EXCEED US$10.00. THE NATURE OF THE LIABILITY OR THE NUMBER OF CLAIMS OR SUITS SHALL NOT ENLARGE OR EXTEND THIS LIMIT.38 *39 * 10. TERMINATION. Your rights under this license will terminate automatically without notice from NVIDIA if you fail to comply with any term and condition of this license or if you commence or participate in any legal proceeding against NVIDIA with respect to the SOFTWARE. NVIDIA may terminate this license with advance written notice to you if NVIDIA decides to no longer provide the SOFTWARE in a country or, in NVIDIA’s sole discretion, the continued use of it is no longer commercially viable. Upon any termination of this license, you agree to promptly discontinue use of the SOFTWARE and destroy all copies in your possession or control. Your prior distributions in accordance with this license are not affected by the termination of this license. All provisions of this license will survive termination, except for the license granted to you.40 *41 * 11. APPLICABLE LAW. This license will be governed in all respects by the laws of the United States and of the State of Delaware as those laws are applied to contracts entered into and performed entirely within Delaware by Delaware residents, without regard to the conflicts of laws principles. The United Nations Convention on Contracts for the International Sale of Goods is specifically disclaimed. You agree to all terms of this Agreement in the English language. The state or federal courts residing in Santa Clara County, California shall have exclusive jurisdiction over any dispute or claim arising out of this license. Notwithstanding this, you agree that NVIDIA shall still be allowed to apply for injunctive remedies or an equivalent type of urgent legal relief in any jurisdiction.42 *43 * 12. NO ASSIGNMENT. This license and your rights and obligations thereunder may not be assigned by you by any means or operation of law without NVIDIA’s permission. Any attempted assignment not approved by NVIDIA in writing shall be void and of no effect.44 *45 * 13. EXPORT. The SOFTWARE is subject to United States export laws and regulations. You agree that you will not ship, transfer or export the SOFTWARE into any country, or use the SOFTWARE in any manner, prohibited by the United States Bureau of Industry and Security or economic sanctions regulations administered by the U.S. Department of Treasury’s Office of Foreign Assets Control (OFAC), or any applicable export laws, restrictions or regulations. These laws include restrictions on destinations, end users and end use. By accepting this license, you confirm that you are not a resident or citizen of any country currently embargoed by the U.S. and that you are not otherwise prohibited from receiving the SOFTWARE.46 *47 * 14. GOVERNMENT USE. The SOFTWARE has been developed entirely at private expense and is “commercial items” consisting of “commercial computer software” and “commercial computer software documentation” provided with RESTRICTED RIGHTS. Use, duplication or disclosure by the U.S. Government or a U.S. Government subcontractor is subject to the restrictions in this license pursuant to DFARS 227.7202-3(a) or as set forth in subparagraphs (b)(1) and (2) of the Commercial Computer Software - Restricted Rights clause at FAR 52.227-19, as applicable. Contractor/manufacturer is NVIDIA, 2788 San Tomas Expressway, Santa Clara, CA 95051.48 *49 * 15. ENTIRE AGREEMENT. This license is the final, complete and exclusive agreement between the parties relating to the subject matter of this license and supersedes all prior or contemporaneous understandings and agreements relating to this subject matter, whether oral or written. If any court of competent jurisdiction determines that any provision of this license is illegal, invalid or unenforceable, the remaining provisions will remain in full force and effect. This license may only be modified in a writing signed by an authorized representative of each party.50 *51 * (v. August 20, 2021)52 */53 54#ifndef _CUDA_ANNOTATED_PTR55#define _CUDA_ANNOTATED_PTR56 57#include <cuda/std/cstdint>58#include <cuda/barrier>59 60#include "std/detail/__access_property"61 62_LIBCUDACXX_BEGIN_NAMESPACE_CUDA63 64class access_property {65  private:66    std::uint64_t __descriptor = 0;67 68  public:69    struct shared {};70    struct global {};71    struct persisting {72      _LIBCUDACXX_HOST_DEVICE constexpr operator cudaAccessProperty() const noexcept {73        return cudaAccessProperty::cudaAccessPropertyPersisting;74      }75    };76    struct streaming {77      _LIBCUDACXX_HOST_DEVICE constexpr operator cudaAccessProperty() const noexcept {78        return cudaAccessProperty::cudaAccessPropertyStreaming;79      }80    };81    struct normal {82      _LIBCUDACXX_HOST_DEVICE constexpr operator cudaAccessProperty() const noexcept {83        return cudaAccessProperty::cudaAccessPropertyNormal;84      }85    };86 87    _LIBCUDACXX_HOST_DEVICE constexpr access_property(global) noexcept : __descriptor(__detail_ap::__sm_80::__interleave_normal()) {}88    _LIBCUDACXX_HOST_DEVICE constexpr access_property() noexcept : __descriptor(__detail_ap::__sm_80::__interleave_normal()) {}89    constexpr access_property(access_property const&) noexcept = default;90    access_property& operator=(const access_property& other) noexcept = default;91 92    _LIBCUDACXX_HOST_DEVICE constexpr access_property(normal, float __fraction) : __descriptor(__detail_ap::__interleave(normal{}, __fraction)) {}93    _LIBCUDACXX_HOST_DEVICE constexpr access_property(streaming, float __fraction) : __descriptor(__detail_ap::__interleave(streaming{}, __fraction)) {}94    _LIBCUDACXX_HOST_DEVICE constexpr access_property(persisting, float __fraction) : __descriptor(__detail_ap::__interleave(persisting{}, __fraction)) {}95    _LIBCUDACXX_HOST_DEVICE constexpr access_property(normal, float __fraction, streaming) : __descriptor(__detail_ap::__interleave(normal{}, __fraction, streaming{})) {}96    _LIBCUDACXX_HOST_DEVICE constexpr access_property(persisting, float __fraction, streaming) : __descriptor(__detail_ap::__interleave(persisting{}, __fraction, streaming{})) {}97 98    _LIBCUDACXX_HOST_DEVICE constexpr access_property(normal) noexcept : access_property(normal{}, 1.0) {}99    _LIBCUDACXX_HOST_DEVICE constexpr access_property(streaming) noexcept : access_property(streaming{}, 1.0) {}100    _LIBCUDACXX_HOST_DEVICE constexpr access_property(persisting) noexcept : access_property(persisting{}, 1.0) {}101 102    _LIBCUDACXX_HOST_DEVICE constexpr access_property(void* __ptr, std::size_t __hit_bytes, std::size_t __total_bytes, normal)103      : __descriptor(__detail_ap::__block(__ptr, __hit_bytes, __total_bytes, normal{})) {}104    _LIBCUDACXX_HOST_DEVICE constexpr access_property(void* __ptr, std::size_t __hit_bytes, std::size_t __total_bytes, streaming)105      : __descriptor(__detail_ap::__block(__ptr, __hit_bytes, __total_bytes, streaming{})) {}106    _LIBCUDACXX_HOST_DEVICE constexpr access_property(void* __ptr, std::size_t __hit_bytes, std::size_t __total_bytes, persisting)107      : __descriptor(__detail_ap::__block(__ptr, __hit_bytes, __total_bytes, persisting{})) {}108    _LIBCUDACXX_HOST_DEVICE constexpr access_property(void* __ptr, std::size_t __hit_bytes, std::size_t __total_bytes, normal, streaming)109      : __descriptor(__detail_ap::__block(__ptr, __hit_bytes, __total_bytes, normal{}, streaming{})) {}110    _LIBCUDACXX_HOST_DEVICE constexpr access_property(void* __ptr, std::size_t __hit_bytes, std::size_t __total_bytes, persisting, streaming)111      : __descriptor(__detail_ap::__block(__ptr, __hit_bytes, __total_bytes, persisting{}, streaming{})) {}112 113    _LIBCUDACXX_HOST_DEVICE constexpr explicit operator std::uint64_t() const noexcept { return __descriptor; }114};115 116_LIBCUDACXX_END_NAMESPACE_CUDA117 118#include "std/detail/__annotated_ptr"119 120_LIBCUDACXX_BEGIN_NAMESPACE_CUDA121 122template <class _Tp, class _Property>123_LIBCUDACXX_HOST_DEVICE124_Tp* associate_access_property(_Tp* __ptr, _Property __prop) {125  static_assert(126    std::is_same<_Property, access_property>::value ||127    std::is_same<_Property, access_property::persisting>::value ||128    std::is_same<_Property, access_property::streaming>::value ||129    std::is_same<_Property, access_property::normal>::value ||130    std::is_same<_Property, access_property::global>::value ||131    std::is_same<_Property, access_property::shared>::value132      , "property is not convertible to cuda::access_property");133  return __detail_ap::__associate(__ptr, __prop);134}135 136template <class _Shape>137_LIBCUDACXX_HOST_DEVICE138void apply_access_property(const volatile void* __ptr, const _Shape __shape, access_property::persisting __prop) noexcept {139  NV_IF_TARGET(NV_PROVIDES_SM_80,(140    if (!__isGlobal((void*)__ptr)) return;141 142    char* __p = reinterpret_cast<char*>(const_cast<void*>(__ptr));143    static constexpr std::size_t _LINE_SIZE = 128;144    std::size_t __nbytes = static_cast<std::size_t>(__shape);145    std::size_t __end = ((std::uintptr_t)(__p + __nbytes) % _LINE_SIZE) ? __nbytes + _LINE_SIZE : __nbytes;146    __end /= _LINE_SIZE;147 148    //Apply to all 128 bytes aligned cache lines inclusive of __p149    for (std::size_t __i = 0; __i < __end; __i += _LINE_SIZE) {150      asm volatile ("prefetch.global.L2::evict_last [%0];" ::"l"(__p + (__i * _LINE_SIZE)) :);151    }152  ))153}154 155template <class _Shape>156_LIBCUDACXX_HOST_DEVICE157void apply_access_property(const volatile void* __ptr, const _Shape __shape, access_property::normal __prop) noexcept {158  NV_IF_TARGET(NV_PROVIDES_SM_80,(159    if (!__isGlobal((void*)__ptr)) return;160 161    char* __p = reinterpret_cast<char*>(const_cast<void*>(__ptr));162    static constexpr std::size_t _LINE_SIZE = 128;163    std::size_t __nbytes = static_cast<std::size_t>(__shape);164    std::size_t __end = ((std::uintptr_t)(__p + __nbytes) % _LINE_SIZE) ? __nbytes + _LINE_SIZE : __nbytes;165    __end /= _LINE_SIZE;166 167    //Apply to all 128 bytes aligned cache lines inclusive of __p168    for (std::size_t __i = 0; __i < __end; __i += _LINE_SIZE) {169      asm volatile ("prefetch.global.L2::evict_normal [%0];" ::"l"(__p + (__i * _LINE_SIZE)) :);170    }171  ))172}173 174inline175_LIBCUDACXX_HOST_DEVICE176void discard_memory(volatile void* __ptr, std::size_t __nbytes) noexcept {177  NV_IF_TARGET(NV_PROVIDES_SM_80,(178    if (!__isGlobal((void*)__ptr)) return;179 180    char* __p = reinterpret_cast<char*>(const_cast<void*>(__ptr));181    static constexpr std::size_t _LINE_SIZE = 128;182    std::size_t __start = (reinterpret_cast<std::uintptr_t>(__p) % _LINE_SIZE) ? 1 : 0;183    std::size_t __end = (reinterpret_cast<std::uintptr_t>(__p + __nbytes) % _LINE_SIZE) ? __nbytes - _LINE_SIZE : __nbytes;184    __end /= _LINE_SIZE;185 186    //Trim the first block and last block if they're not 128 bytes aligned187    for (std::size_t __i = __start; __i < __end; __i += _LINE_SIZE) {188      asm volatile ("discard.global.L2 [%0], 128;" ::"l"(__p + (__i * _LINE_SIZE)) :);189    }190  ))191  (void)__ptr;192  (void)__nbytes;193}194 195template<class _Tp, class _Property>196class annotated_ptr: public __detail_ap::__annotated_ptr_base<_Property> {197  public:198    using value_type = _Tp;199    using size_type = std::size_t;200    using reference = value_type&;201    using pointer = value_type*;202    using const_pointer = value_type const*;203    using difference_type = std::ptrdiff_t;204 205  private:206    using __self = annotated_ptr<_Tp, _Property>;207 208    // Converting from a 64-bit to 32-bit shared pointer and maybe back just for storage might or might not be profitable.209    pointer __repr = (pointer)((size_type)nullptr);210 211    _LIBCUDACXX_HOST_DEVICE pointer __get(bool __skip_prop = false, difference_type __n = 0) const {212      NV_IF_TARGET(NV_IS_DEVICE,(213        if (!__skip_prop) {214          return static_cast<pointer>(this->__apply_prop(const_cast<void*>(static_cast<const volatile void*>(__repr + __n))));215        }216      ))217      return __repr + __n;218    }219    _LIBCUDACXX_HOST_DEVICE pointer __offset(difference_type __n, bool __skip_prop = false) const {220      return __get(__skip_prop, __n);221    }222 223  public:224    _LIBCUDACXX_HOST_DEVICE pointer operator->() const {225      return __get();226    }227 228    _LIBCUDACXX_HOST_DEVICE reference operator*() const {229      return *__get();230    }231 232    _LIBCUDACXX_HOST_DEVICE reference operator[](difference_type __n) const {233      return *__offset(__n);234    }235 236    _LIBCUDACXX_HOST_DEVICE constexpr difference_type operator-(annotated_ptr o) const {237      return __repr - o.__repr;238    }239 240    constexpr annotated_ptr() noexcept = default;241    constexpr annotated_ptr(annotated_ptr const&) noexcept = default;242    // No constexpr for c11 as the method can't be const243    _LIBCUDACXX_CONSTEXPR_AFTER_CXX11 annotated_ptr& operator=(annotated_ptr const& other) noexcept = default;244 245    _LIBCUDACXX_HOST_DEVICE explicit annotated_ptr(pointer __p)246      : __repr(__p)247    {248      NV_IF_TARGET(NV_IS_DEVICE,(249        _LIBCUDACXX_DEBUG_ASSERT((std::is_same<_Property, shared>::value && __isShared(__p) || __isGlobal(__p)), "");250      ))251    }252 253    template <typename _RuntimeProperty>254    _LIBCUDACXX_HOST_DEVICE annotated_ptr(pointer __p, _RuntimeProperty __prop)255      : __detail_ap::__annotated_ptr_base<_Property>(static_cast<std::uint64_t>(access_property(__prop))), __repr(__p)256    {257      static_assert(std::is_same<_Property, access_property>::value,258		    "This method requires annotated_ptr<T, cuda::access_property>");259      static_assert(std::is_same<_RuntimeProperty, access_property::global>::value ||260                    std::is_same<_RuntimeProperty, access_property::normal>::value ||261                    std::is_same<_RuntimeProperty, access_property::streaming>::value ||262                    std::is_same<_RuntimeProperty, access_property::persisting>::value ||263                    std::is_same<_RuntimeProperty, access_property>::value,264                    "This method requires RuntimeProperty=global|normal|streaming|persisting|access_property");265      NV_IF_TARGET(NV_IS_DEVICE,(266        _LIBCUDACXX_DEBUG_ASSERT((__isGlobal(__p) == true), "");267      ))268    }269 270    template<class _TTp, class _Prop>271    _LIBCUDACXX_HOST_DEVICE annotated_ptr(const annotated_ptr<_TTp,_Prop>& __other);272 273    _LIBCUDACXX_HOST_DEVICE constexpr explicit operator bool() const noexcept {274      return __repr != nullptr;275    }276 277    _LIBCUDACXX_HOST_DEVICE pointer get() const noexcept {278      constexpr bool __is_shared = std::is_same<_Property, access_property::shared>::value;279      return __is_shared ? __repr : &(*annotated_ptr<value_type, access_property::global>(__repr));280    }281 282    _LIBCUDACXX_HOST_DEVICE _Property __property() const noexcept {283      return this->__get_property();284    }285};286 287 288template<class _Tp, class _Property>289template<class _TTp, class _Prop>290_LIBCUDACXX_HOST_DEVICE annotated_ptr<_Tp, _Property>::annotated_ptr(const annotated_ptr<_TTp,_Prop>& __other)291  : __detail_ap::__annotated_ptr_base<_Property>(__other.__property()), __repr(__other.get())292{293  static_assert(std::is_assignable<pointer&, _TTp*>::value, "pointer must be assignable from other pointer");294  static_assert((std::is_same<_Property, access_property>::value && !std::is_same<_Prop, access_property::shared>::value) ||295		std::is_same<_Property, _Prop>::value, "Property must be either access_property or other property, and both properties must have same address space");296  // note: precondition "__other.__rep must be compatible with _Property" currently always holds297}298 299template<class _Dst, class _Src, class _SrcProperty, class _Shape, class _Sync>300_LIBCUDACXX_HOST_DEVICE301void memcpy_async(_Dst* __dst,302    annotated_ptr<_Src,_SrcProperty> __src,303    _Shape __shape, _Sync & __sync) {304  memcpy_async(__dst, &(*__src), __shape, __sync);305}306 307template<class _Dst, class _DstProperty, class _Src, class _SrcProperty,308  class _Shape, class _Sync>309_LIBCUDACXX_HOST_DEVICE310void memcpy_async(annotated_ptr<_Dst,_DstProperty> __dst,311    annotated_ptr<_Src,_SrcProperty> __src,312    _Shape __shape, _Sync & __sync){313  memcpy_async(&(*__dst), &(*__src), __shape, __sync);314}315 316template<class _Group, class _Dst, class _Src, class _SrcProperty,317  class _Shape, class _Sync>318_LIBCUDACXX_HOST_DEVICE319void memcpy_async(const _Group & __group,320    _Dst * __dst,321    annotated_ptr<_Src,_SrcProperty> __src,322    _Shape __shape, _Sync & __sync) {323  memcpy_async(__group, __dst, &(*__src), __shape, __sync);324}325 326template<class _Group, class _Dst, class _DstProperty, class _Src, class _SrcProperty,327  class _Shape, class _Sync>328_LIBCUDACXX_HOST_DEVICE329void memcpy_async(const _Group & __group,330    annotated_ptr<_Dst,_DstProperty> __dst,331    annotated_ptr<_Src,_SrcProperty> __src,332    _Shape __shape, _Sync & __sync) {333  memcpy_async(__group, &(*__dst), &(*__src), __shape, __sync);334}335 336_LIBCUDACXX_END_NAMESPACE_CUDA337 338#endif // _CUDA_ANNOTATED_PTR339