xdecoder/Instruct-X-Decoder
163
1U
2 E$�c� � @ s6 d Z ddlZddlZddlmZ G dd� dej�ZdS )z33Various positional encodings for the transformer.4� N)�nnc s6 e Zd ZdZd
� fdd� Zddd �Zdd d �Z� ZS )�PositionEmbeddingSinez�5 This is a more standard version of the position embedding, very similar to the one6 used by the Attention is all you need paper, generalized to work on images.7 �@ �' FNc sP t � �� || _|| _|| _|d k r4|dkr4td��|d krFdtj }|| _d S )NFz+normalize should be True if scale is passed� ) �super�__init__�
num_pos_feats� temperature� normalize�8ValueError�math�pi�scale)�selfr r9 r r �� __class__� �;/data/arXiv/demo/Demo/xdecoder/modules/position_encoding.pyr s 1011z PositionEmbeddingSine.__init__c C s� |d kr4t j|�d�|�d�|�d�f|jt jd�}| }|jd|jd�}|jd|jd�}| jr�d}||d d �dd �d d �f | | j }||d d �d d �dd �f | | j }t j | j12|j|jd �}| j d|d | j13 }|d d �d d �d d �d f | }|d d �d d �d d �d f | } t j |d d �d d �d d �dd d�f �
� |d d �d d �d d �dd d�f �� fd14d ��d�}t j | d d �d d �d d �dd d�f �
� | d d �d d �d d �dd d�f �� fd15d ��d�} t j| |fdd ��dddd�}16|17S ) Nr r � )�device�dtype� )r g�����ư>�����)r r � )�dim)�torch�zeros�sizer �bool�cumsumr r r �aranger r18 �stack�sin�cos�flatten�cat�permute) r �x�maskZnot_maskZy_embedZx_embed�epsZdim_tZpos_xZpos_y�posr r r �forward s6 ,(( J ��J �� z PositionEmbeddingSine.forwardr c sZ d| j j }d�| j�d�| j�d�| j�d�| j�g}|g� fdd�|D � }d�|�S ) NzPositional encoding znum_pos_feats: {}ztemperature: {}z
normalize: {}z scale: {}c s g | ]}d � | �qS )� r )�.0�line�� _repr_indentr r �19<listcomp>? s z2PositionEmbeddingSine.__repr__.<locals>.<listcomp>�20)r �__name__�formatr r21 r r �join)r r1 �head�body�linesr r0 r �__repr__6 s 22232425�z PositionEmbeddingSine.__repr__)r r FN)N)r )r4 �26__module__� __qualname__�__doc__r r, r: �
__classcell__r r r r r s 27r )r= r
r r �Moduler r r r r �<module> s