\;jWdZddlZddlZddlZddlmZddlmZddlZ gZ GddeZ GddeZ d Z d Zd Zd Zd ZdZdZej(aiadZdZe j2ddfdZe j6ddfdZy)z# Utilities of Auto SParsity (ASP). N)Enum) permutationsceZdZdZdZdZdZy)MaskAlgoz A collection of all mask generating algorithms. There currently are three algorithms, `MASK_1D`, `MASK_2D_GREEDY` and `MASK_2D_BEST` get_mask_1dget_mask_2d_greedyget_mask_2d_bestN)__name__ __module__ __qualname____doc__MASK_1DMASK_2D_GREEDY MASK_2D_BESTbG:\00. PROJECTS\API\Inventory\templateJSON\kerjaOCR\Lib\site-packages\paddle/incubate/asp/utils.pyrrsG)N%Lrrc(eZdZdZdZdZedZy) CheckMethodzz A collection of all sparsity checking approaches. There currently are two methods, `CHECK_1D` and `CHECK_2D` check_mask_1d check_mask_2dct|tsJd|tjk(rtjStj S)a Get sparsity checking method by mask generating algorithm. Args: mask_algo (MaskAlgo): The algorithm of mask generating. Returns: CheckMethod: The corresponded sparsity checking method. Examples: .. code-block:: python >>> import numpy as np >>> from paddle.incubate.asp import CheckMethod, MaskAlgo >>> print(CheckMethod.get_checking_method(MaskAlgo.MASK_1D)) CheckMethod.CHECK_1D >>> print(CheckMethod.get_checking_method(MaskAlgo.MASK_2D_GREEDY)) CheckMethod.CHECK_2D >>> print(CheckMethod.get_checking_method(MaskAlgo.MASK_2D_BEST)) CheckMethod.CHECK_2D z!mask_algo should be MaskAlgo type) isinstancerrrCHECK_1DCHECK_2D) mask_algos rget_checking_methodzCheckMethod.get_checking_method0sK* x  / . /  (( ('' ''' 'rN)r r r r rr staticmethodrrrrrr(s%HH((rrc|j}ttj|dj|jz S)a Return the density of the input tensor. Args: x (nparray): The input tensor. Returns: float, The density of :attr:`x`. Examples: .. code-block:: python >>> import paddle >>> import numpy as np >>> x = np.array([[0, 1, 3, 0], ... [1, 1, 0, 1]]) >>> out = paddle.incubate.asp.calculate_density(x) >>> print(out) 0.625 r)flattenfloatnpnonzerosize)x x_flatteneds rcalculate_densityr'Ns90))+K K(+00 1K4D4D DDrct|jdk(sJd|jd|z}|jd|zdkDrotj|jd|jd||z zf}||ddd|jdf<|j}|j d||fS|j d||jfS)a Reshape the input 2D matrix to shape (-1, m). If the second dimension of :attr:`mat` is not a multiples of :attr:`m`, then this function would pad the remainder with 0 before reshaping. .. math:: remainder = mat.shape[1] % m Args: mat (nparray): The input 2D matrix. m (int): The second dimension of reshaped matrix. Returns: tuple: A pair of the reshaped and padded matrix and the shape of padded matrix (non-reshaping). $The input mat should be a 2D matrix!rN)lenshaper"zerosreshape)matm remainder mat_paddedr.s r _reshape_1dr5js syy>Q F FF  ! q I yy|a!XXsyy|SYYq\Q]-KLM (+ 1n ! n$%  !!"a(%//{{2q!399,,rct|jdkr-t|jd|jd|\}}nt||\}}|D],}t j |dj ||z kDs,yy)a Check if every row of the input matrix :attr:`mat` is in 1D `n:m` sparse pattern. This function would pad the second dimension of :attr:`mat` by zero to be a multiples of :attr:`m` if necessary. 1D `n:m` sparse pattern: At least :attr:`n` zeros in every :math:`1 \times m` block. Args: mat (nparray): The input matrix. n (int): n of `n:m` sparse pattern. m (int): m of `n:m` sparse pattern. Returns: bool: True if every row of :attr:`mat` is in 1D n:m sparse pattern, else False. Examples: .. code-block:: python >>> import numpy as np >>> import paddle.incubate.asp as sparsity >>> x = np.array([[0, 1, 3, 0], ... [1, 0, 0, 1]]) >>> y = sparsity.check_mask_1d(x, 2, 4) >>> print(y) True >>> x = np.array([[0, 1, 5, 4], ... [1, 0, 0, 1]]) >>> y = sparsity.check_mask_1d(x, 2, 4) >>> print(y) False >>> # x would be padded to shape (2, 8) >>> x = np.array([[0, 1, 0, 4, 6], ... [1, 0, 0, 1, 7]]) >>> y = sparsity.check_mask_1d(x, 2, 4) >>> print(y) True r+rFT)r-r.r5r0r"r#r$)r1nr2 mat_flatternr.sub_mats rrrs|N 399~)#++a1*FJ e)#q1 e ::g q ! & &!a% 0  rct||\}}tj|}tj|}t|jdD]G}||}tj tj |} d||| d|jf<I|j|}|ddd|jdf|ddddf<|S)a Generate 1D `n:m` sparse pattern mask of the input matrix :attr:`mat` in row-directory. This function would pad the second dimension of :attr:`mat` by zero to be a multiples of :attr:`m` before mask generation. 1D `n:m` sparse pattern: At least :attr:`n` zeros in every :math:`1 \times m` block. Args: mat (nparray): The input matrix. n (int): n of `n:m` sparse pattern. m (int): m of `n:m` sparse pattern. Returns: nparray: The 1D `n:m` sparse mask of :attr:`mat`. Examples: .. code-block:: python >>> import numpy as np >>> import paddle.incubate.asp as sparsity >>> mat = np.array([[0, 1, 5, 4], ... [2, 7, 3, 6]]) >>> mask = sparsity.get_mask_1d(mat, 2, 4) >>> print(mask) [[0 0 1 1] [0 1 0 1]] >>> y = sparsity.check_mask_1d(mask, 2, 4) >>> print(y) True rNr+) r5r" ones_likeranger.argsortabsolutetolistr0) r1r7r2r8r. mask_flatternmaskir9min_order_indicess rrrs:&c1-L%LL.M << D <%%a( )q/JJr{{7';<;< a*2A.55778*"))%0Mq.CIIaL.01DAJ Krct|jdk(sJd|jd|z}|jd|z}|dk(r|jdn|jd||z z|dk(r|jdn|jd||z zf}tj|}||d|jdd|jdf<tj|j d||z}d}t d|jd|D]b}||z} t d|jd|D]>} | |z} tj||| | | fj d} | ||<|dz }@d||jfS)a3 Reshape the input 2D matrix to shape (-1, :math:`m \times m`). In each dimension of :attr:`mat`, if it is not a multiples of :attr:`m`, then this function would pad the remainder with 0 before reshaping. .. math:: remainder_0 = mat.shape[0] % m \\ remainder_1 = mat.shape[1] % m Args: mat (nparray): The input 2D matrix. m (int): The square root of second dimension of reshaped matrix. Returns: tuple: A pair of the reshaped and padded matrix and the shape of padded matrix (non-reshaping). r)r*rr+Nr,)r-r.r"r/emptyr0r<squeeze) r1r2 remainder_0 remainder_1 new_shaper4r8curr_idx row_startrow_end col_startcol_endr9s r _reshape_2drOs" syy>Q F FF ))A,"K))A,"K$q( ! ciila+o.N#q( ! ciila+o.NI)$J14J~1~~1~-.88I&..r1q59LH1j..q115 a-q*"2"21"5q9I!mGjj9W,i.??@HHLG&-L " MH :6 )) ))rc xt||\}}|D]}tjtj|j ||dkD}tj tj |d||z kDdk7srtj tj |d||z kDdk7syy)a Check if every :math:`m \times m` block of the input matrix :attr:`mat` is in 2D `n:m` sparse pattern. This function would pad each dimension of :attr:`mat` by zero to be a multiples of :attr:`m` if necessary. 2D `n:m` sparse pattern: At least :math:`n \times n` zeros in every :math:`m \times m` block under the constraint of at least :attr:`n` zeros for each row and column. Args: mat (nparray): The input matrix. n (int): n of `n:m` sparse pattern. m (int): m of `n:m` sparse pattern. Returns: bool: True if every :math:`m \times m` block of the input matrix :attr:`mat` is in 2D `n:m` sparse pattern, else False. Examples: .. code-block:: python >>> import numpy as np >>> import paddle.incubate.asp as sparsity >>> x = np.array([[0, 8, 9, 0], ... [9, 0, 0, 10], ... [5, 0, 0, 6], ... [0, 4, 6, 0]]) >>> y = sparsity.check_mask_2d(x, 2, 4) >>> print(y) True >>> x = np.array([[0, 8, 0, 9], ... [9, 0, 0, 10], ... [0, 5, 0, 6], ... [0, 4, 6, 0]]) >>> y = sparsity.check_mask_2d(x, 2, 4) >>> print(y) True >>> # x would be padded to shape (8, 8) >>> x = np.array([[0, 8, 0, 9], ... [9, 0, 7, 0], ... [0, 5, 0, 6], ... [3, 0, 6, 0], ... [1, 1, 0, 1]]) >>> y = sparsity.check_mask_2d(x, 2, 4) >>> print(y) True rr+axisFT)rOr"r>rFr0sum)r1r7r2r4r.r9sub_masks rrr s^$C+J;;rzz'//!Q*?@AAE FF266(+q1u5 6! ; FF266(+q1u5 6! ;  rct||\}}tj|jd||}t t |D] }tj tj||}tj||}tj|} | D cgc]} t| |z | |zf} } tj} tj} t t | dz ddD]K}| |}| |d|k(s | |d|k(rd||d|df<| |dxxdz cc<| |dxxdz cc<M tj|}d}t d|d|D]4}||z}t d|d|D]}||z}|||||||f<|dz }6|d|jdd|jdfScc} w)a Greedily generate 2D `n:m` sparse pattern mask of the input matrix :attr:`mat`. This function would pad each dimension of :attr:`mat` by zero to be a multiples of :attr:`m` before mask generation. 2D `n:m` sparse pattern: At least :math:`n \times n` zeros in every :math:`m \times m` block under the constraint of at least :attr:`n` zeros for each row and column. Greedily generating: For each :math:`m \times m` block, selecting values to keep in descent order. Args: mat (nparray): The input matrix. n (int): n of `n:m` sparse pattern. m (int): m of `n:m` sparse pattern. Returns: nparray: The 2D `n:m` sparse mask of :attr:`mat`. Examples: .. code-block:: python >>> import numpy as np >>> import paddle.incubate.asp as sparsity >>> mat = np.array([[9, 8, 3, 7], ... [9, 2, 1, 10], ... [5, 1, 3, 6], ... [2, 4, 6, 1]]) >>> mask = sparsity.get_mask_2d_greedy(mat, 2, 4) >>> print(mask) [[1. 1. 0. 0.] [1. 0. 0. 1.] [0. 0. 1. 1.] [0. 1. 1. 0.]] >>> y = sparsity.check_mask_2d(mask, 2, 4) >>> print(y) True r,r+rg?N)rOr" zeros_liker0r<r-r>rFr=int collectionsCounterrEr.)r1r7r2r4r. mask_paddedidxr9rTmin_order_1d_indicesr%min_order_2d_indices row_counter col_counterrB matrix_entryrArJrKrLrMrNs rrrFsF$C+J-- +33B1=KS_%++bjjC9:::k#./!zz'2)= )=ASQZQ )=  "))+ !))+ s/014b"=A/2LLO,1LO,19&, 88E?DH1eAh* a-q%(A.I!mG9DX9ND7"Ig$55 6 MH/+ #))A,#))A,. //3 s*Gc ~|d|}|tvr t|Stj|}d|d|tt t |j }||z}tjtt t ||}|jd|kjd|k(jdjd}tj|jd||f}||dd|ddtj|t|<tj|S)a Compute all vaild 2D `n:m` sparse patterns. 2D `n:m` sparse pattern: At least :math:`n \times n` zeros in every :math:`m \times m` block under the constraint of at least :attr:`n` zeros for each row and column. Args: n (int): n of `n:m` sparse pattern. m (int): m of `n:m` sparse pattern. Returns: dictionary: A dictionary with key: *m_n* (string) and value: all vaild 2D `n:m` sparse patterns. _r+NrQrr,)_valid_2d_patternsr"r/listsetrr?asarrayrSr#r0rEr._valid_2d_patterns_lockacquirerelease)r7r2 valid_keypatternsvalidvalid_patternss r_compute_valid_2d_patternsrns" #Qqc I&&!),,88A;! L):;<=h&::d3|Ha'@#ABClll"a' , ,! , 4 9 WYq  WR[  5;;q>1a"89$U1X.q'')(69%'')rc Jt||}t||\}}tj|j d||}tj tj ||j |jd||zjd}||dd|ddtj|}d} td|d|D]4} | |z} td|d|D]} | |z} || || | | | f<| dz } 6|d|jdd|jdfS)a Generate 2D `n:m` sparse pattern mask of the input matrix :attr:`mat` to form sparse matrix with maximun L1 norm .This function would pad each dimension of :attr:`mat` by zero to be a multiples of :attr:`m` before mask generation. 2D `n:m` sparse pattern: At least :math:`n \times n` zeros in every :math:`m \times m` block under the constraint of at least :attr:`n` zeros for each row and column. *Note*: L1 norm of sparse matrix from `Best` API is greater than or equal to the one from `Greedy`. Args: mat (nparray): The input matrix. n (int): n of `n:m` sparse pattern. m (int): m of `n:m` sparse pattern. Returns: nparray: The 1D `n:m` sparse mask of :attr:`mat`. Examples: .. code-block:: python >>> import numpy as np >>> import paddle.incubate.asp as sparsity >>> mat = np.array([[2, 8, 9, 9], ... [9, 1, 3, 9], ... [5, 6, 3, 9], ... [2, 4, 6, 9]]) >>> mask_greedy = sparsity.get_mask_2d_greedy(mat, 2, 4) >>> mask_best = sparsity.get_mask_2d_best(mat, 2, 4) >>> print("L1 norm of `greedy` sparse matrix", np.multiply(mat, mask_greedy).sum()) L1 norm of `greedy` sparse matrix 56.0 >>> print("L1 norm of `best` sparse matrix", np.multiply(mat, mask_best).sum()) L1 norm of `best` sparse matrix 61.0 r,rr+rQN) rnrOr"r;r0argmaxmatmulr.TrEr<)r1r7r2rkr8r.r@pmaxrArJrKrLrMrNs rr r s6D*!Q/H%c1-L%LL.66r1a@M 99 , 0 01BAE J L LM  D  Q(M! 88E?DH1eAh* a-q%(A.I!mG9Fx9PD7"Ig$55 6 MH/+ #))A,#))A,. //rr)c|j}|j}|jt}t |t sJdt |ttjt|jd}t|dk(r|jd|d}nt|dk(r|j|d|d}nt|dk(r|j|d|dz|d}nt|dk(r|jgdj|d|dz|dz|d}|||| }|j|d|d|d|dgjgdj|Std t||||| }|j|j|S) aH Create `n:m` sparse pattern mask of the input tensor via function given by :attr:`func_name`. Currently only support tensor with dimension less than or equal to 4. Args: tensor (nparray): The input tensor. func_name (MaskAlgo, optional): The function name to generate spase mask. Default is `MaskAlgo.MASK_1D`. All options please refer to `MaskAlgo`. n (int, optional): n of `n:m` sparse pattern. Default is 2. m (int, optional): m of `n:m` sparse pattern. Default is 4. Returns: nparray: The `n:m` sparse mask of :attr:`tensor` generated by :attr:`func_name`. Examples: .. code-block:: python >>> import numpy as np >>> import paddle.incubate.asp as sparsity >>> tensor = np.array([[2, 8, 9, 9], ... [9, 1, 3, 9], ... [5, 6, 3, 9], ... [2, 4, 6, 9]]) >>> mask_1d = sparsity.create_mask(tensor, func_name=sparsity.MaskAlgo.MASK_1D) >>> print(mask_1d) [[0 0 1 1] [1 0 0 1] [0 1 0 1] [0 0 1 1]] >>> mask_2d = sparsity.create_mask(tensor, func_name=sparsity.MaskAlgo.MASK_2D_BEST) >>> print(mask_2d) [[0 1 1 0] [1 0 0 1] [1 1 0 0] [0 0 1 1]] zLfunc_name argumet of create_mask is only accepted as type MaskAlgo. But got Nr+rr)rtrr+rvr)r7r2gThe dimension of input tensor is not supported in create_mask, Only dimension < 4 is supported but got )r.dtypeastyper!rrtypegetattrsysmodulesr valuer-r0 transpose ValueError) tensor func_namer7r2r.rztfuncrAs r create_maskrsF LLE LLE eA i *  ?# % * 3;;x()//4 @D 5zQ IIaq " Uq IIeAha ) Uq IIeAhq)58 4 Uq KK % - - !HuQx %( *E!H Aa  LL%(E!HeAhaA B Y| $ VE]  77:5zl D  Q! D <<  % %e ,,rc|j}|jt}t|tk(sJdt|t t jt|jd}t|dk(r|jd|d}nt|dk(r|j|d|d}nt|dk(r|j|d|dz|d}n\t|dk(r7|jgdj|d|dz|dz|dg}ntd t||||| S) a Check if input tensor is in `n:m` sparse pattern via function given by :attr:`func_name`. Currently only support tensor with dimension less than or equal to 4. Args: tensor (nparray): The input tensor. func_name (CheckMethod, optional): The function name to generate spase mask. Default is `CheckMethod.CHECK_1D`. All options please refer to `CheckMethod`. n (int, optional): n of `n:m` sparse pattern. Default is 2. m (int, optional): m of `n:m` sparse pattern. Default is 4. Returns: bool: True if tensor pass checking of function given by :attr:`func_name`, else False. Examples: .. code-block:: python >>> import numpy as np >>> import paddle.incubate.asp as sparsity >>> tensor = np.array([[2, 8, 9, 9], ... [9, 1, 3, 9], ... [5, 6, 3, 9], ... [2, 4, 6, 9]]) >>> mask_1d = sparsity.create_mask(tensor, func_name=sparsity.MaskAlgo.MASK_1D) >>> print(mask_1d) [[0 0 1 1] [1 0 0 1] [0 1 0 1] [0 0 1 1]] >>> y = sparsity.check_sparsity(mask_1d, func_name=sparsity.CheckMethod.CHECK_1D) >>> print(y) True >>> y = sparsity.check_sparsity(mask_1d, func_name=sparsity.CheckMethod.CHECK_2D) >>> print(y) True zRfunc_name argumet of check_sparsity is only accepted as type CheckMethod. But got Nr+rr)rvrtrwryrx)r.r{r!r|rr}r~rr rr-r0rr)rrr7r2r.rrs rcheck_sparsityr9sNF LLE eA  ?k )  ?# % ) 3;;x()//4 @D 5zQ IIaq " Uq IIeAha ) Uq IIeAhq)58 4 Uq KK % - - 1Xa 58 +U1X 6  77:5zl D  Q! r)r rXr~ threadingenumr itertoolsrnumpyr"__all__rrr'r5rrrOrrLockrgrcrnr rrrrrrrrs " &t&#($#(LE8-8/d'T(*V6rD0N))..*&R50p#+"2"2a1D-N&1%9%9Q!<r