\;j3ddlZddlmZddlmZmZmZddlmZddl m Z m Z m Z ddl mZgZGd d eZGd d eZGd deZy)N)_C_ops)core framework unique_name)check_variable_and_dtype)_current_expected_placein_dygraph_mode in_pir_mode) Initializerc,eZdZdZdfd ZddZxZS)XavierInitializera This class implements the Xavier weight initializer from the paper `Understanding the difficulty of training deep feedforward neural networks `_ by Xavier Glorot and Yoshua Bengio. This initializer is designed to keep the scale of the gradients approximately same in all the layers. In case of Uniform distribution, the range is [-x, x], where .. math:: x = \sqrt{\\frac{6.0}{fan\_in + fan\_out}} In case of Normal distribution, the mean is 0 and the standard deviation is .. math:: \sqrt{\\frac{2.0}{fan\_in + fan\_out}} Args: uniform (bool, optional): whether to use uniform ,if False use normal distribution. Default is True. fan_in (float, optional): fan_in for Xavier initialization. If None, it is inferred from the variable. Default is None. fan_out (float, optional): fan_out for Xavier initialization. If None, it is inferred from the variable. Default is None. seed (int, optional): Random seed. Default is 0. Note: It is recommended to set fan_in and fan_out to None for most cases. cj|J|Jt|||_||_||_||_yN)super__init___uniform_fan_in_fan_out_seed)selfuniformfan_infan_outseed __class__s eG:\00. PROJECTS\API\Inventory\templateJSON\kerjaOCR\Lib\site-packages\paddle/nn/initializer/xavier.pyrzXavierInitializer.__init__CsB"""     c 6 ddl}|j|}t|tj|j jfsJt||j j jst|dgdd|j|\}}|j|n |j}|j|n |j}|jdk(r|jj|_ |jt j j"j$k(s=|jt j j"j&k(r|j(st j j"j*}|j-t/j0dj3d|j4dg|j6|t j j"j8d } nw|jt j:j<t j:j>fvr)|j(st j:j@}|} n|j}|} tCrv|j(r\tEjFd tI||zz } tKjL| j6|| | |jtO} n\tEjFd tI||zz } tO} tKjP| j6d | |j|| } |jt j j"j$k(s=|jt j j"j&k(r>|j(s2tKjR| |j} | jU|y| jU|ytWr6|j(rbtEjFd tI||zz } |jXjM| j6|| | |jtO} nZtEjFd tI||zz } tKjP| j6d | |j|tO} |jt j:j<t j:j>fvr,|j(s tKjR| |jS| S|j(rXtEjFd tI||zz } |j[d id| i| j6|| | |jdd}n_tEjFd tI||zz } |j[dd| i| j6| jd | |jdd}|jt j j"j$k(s=|jt j j"j&k(r<|j(s0|j[dd| id|i| j|jd||_.|S)aXInitialize the input tensor with Xavier initialization. Args: var(Tensor): Tensor that needs to be initialized. block(Block, optional): The block in which initialization ops should be added. Used in static graph only, default None. Returns: The initialization op rNOut)uint16float16float32float64 xavier_init.tmpF)nameshapedtypetype persistableg@g@guniform_random)r*r+minmaxrT)r,inputsoutputsattrs stop_gradientgaussian_random)r*r+meanstdr)r,r2r3r4castX)in_dtype out_dtype)r,r1r2r3)/paddle _check_block isinstancerBlockpirr ParameterMetar _compute_fansrrrprogram random_seedr+VarDescVarTypeFP16BF16rFP32 create_varrgeneratejoinr)r* LOD_TENSORDataTypeFLOAT16BFLOAT16FLOAT32r mathsqrtfloatrrr gaussianr8_share_underline_tensor_tor _pir_ops append_opop)rvarblockr<f_inf_outrrr;out_varlimitr7placevar_tmprYs rforwardzXavierInitializer.forwardLs !!%(%)//6::3C3C!DEEE#vzz<<= $;  ((- e-4<<==0%dmm ::?22DJ 99 ,,11 1 II--22 24== ,,11I&& ))HHmSXXu=>ii\\))44!'G II$--//1G1GH HMM --IG IG  }} #fw.>(?"?@ ..MMFJJ+- iieFW,<&= =>/1 //MM3TZZEyyDLL00555 T\\11666t}} ++gsyy922372237 ]}} #fw.>(?"?@ //11MMFJJ+- iieFW,<&= => //MMJJ+-  dmm33T]]5K5KLL {{7CII66N}} #fw.>(?"?@__)"G,!(!* %v$ $ #'% iieFW,<&= =>__*"G,!(!( #" $ #'% yyDLL00555 T\\11666t}}>"CL'.}}399M  CFIr)TNNrr)__name__ __module__ __qualname____doc__rrb __classcell__rs@rrrs!FZrrc$eZdZdZdfd ZxZS) XavierNormala This class implements the Xavier weight initializer from the paper `Understanding the difficulty of training deep feedforward neural networks `_ by Xavier Glorot and Yoshua Bengio, using a normal distribution whose mean is :math:`0` and standard deviation is .. math:: \sqrt{\frac{2.0}{fan\_in + fan\_out}}. Args: fan_in (float, optional): fan_in for Xavier initialization, which is inferred from the Tensor. Default is None. fan_out (float, optional): fan_out for Xavier initialization, which is inferred from the Tensor. Default is None. name (str, optional): For details, please refer to :ref:`api_guide_Name`. Generally, no setting is required. Default: None. Returns: A parameter initialized by Xavier weight, using a normal distribution. Examples: .. code-block:: python >>> import paddle >>> paddle.seed(1) >>> data = paddle.ones(shape=[3, 1, 2], dtype='float32') >>> weight_attr = paddle.framework.ParamAttr( ... name="linear_weight", ... initializer=paddle.nn.initializer.XavierNormal()) >>> bias_attr = paddle.framework.ParamAttr( ... name="linear_bias", ... initializer=paddle.nn.initializer.XavierNormal()) >>> linear = paddle.nn.Linear(2, 2, weight_attr=weight_attr, bias_attr=bias_attr) >>> print(linear.weight) Parameter containing: Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[-0.21607460, 0.08382989], [ 0.29147008, -0.07049121]]) >>> print(linear.bias) Parameter containing: Tensor(shape=[2], dtype=float32, place=Place(cpu), stop_gradient=False, [1.06076419, 0.87684733]) >>> res = linear(data) >>> print(res) Tensor(shape=[3, 1, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[[1.13615966, 0.89018601]], [[1.13615966, 0.89018601]], [[1.13615966, 0.89018601]]]) c,t|d||dy)NFrrrrrrrrrrr)rs rrzXavierNormal.__init__s vwQOrNNNrcrdrerfrrgrhs@rrjrjs3jPPrrjc$eZdZdZdfd ZxZS) XavierUniforma+ This class implements the Xavier weight initializer from the paper `Understanding the difficulty of training deep feedforward neural networks `_ by Xavier Glorot and Yoshua Bengio. This initializer is designed to keep the scale of the gradients approximately same in all the layers. In case of Uniform distribution, the range is :math:`[-x,x]`, where .. math:: x = \sqrt{\frac{6.0}{fan\_in + fan\_out}}. Args: fan_in (float, optional): fan_in for Xavier initialization, which is inferred from the Tensor. Default is None. fan_out (float, optional): fan_out for Xavier initialization, which is inferred from the Tensor. Default is None. name (str, optional): For details, please refer to :ref:`api_guide_Name`. Generally, no setting is required. Default: None. Returns: A parameter initialized by Xavier weight, using a uniform distribution. Examples: .. code-block:: python >>> import paddle >>> paddle.seed(1) >>> data = paddle.ones(shape=[3, 1, 2], dtype='float32') >>> weight_attr = paddle.framework.ParamAttr( ... name="linear_weight", ... initializer=paddle.nn.initializer.XavierUniform()) >>> bias_attr = paddle.framework.ParamAttr( ... name="linear_bias", ... initializer=paddle.nn.initializer.XavierUniform()) >>> linear = paddle.nn.Linear(2, 2, weight_attr=weight_attr, bias_attr=bias_attr) >>> print(linear.weight) Parameter containing: Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[-1.18095720, 0.64892638], [ 0.43125069, -1.11156428]]) >>> print(linear.bias) Parameter containing: Tensor(shape=[2], dtype=float32, place=Place(cpu), stop_gradient=False, [-0.27524316, 1.13808715]) >>> res = linear(data) >>> print(res) Tensor(shape=[3, 1, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[[-1.02494967, 0.67544925]], [[-1.02494967, 0.67544925]], [[-1.02494967, 0.67544925]]]) c,t|d||dy)NTrrlrmrns rrzXavierUniform.__init__[s fgANrrorprhs@rrrrr#s5nOOrrr)rRr<rbaserrrbase.data_feederrbase.frameworkr r r initializerr __all__rrjrrrrrzsX 008 % G GT7P$7Pt9O%9Or