\;jPddlZddlZddlmZmZddlmZmZmZgZejjZ ejjZ ejZ ejZej ZdZdZdZdZd Zd ZGd d ZGd dZGddZGddZy)N)core framework) cpu_places cuda_places xpu_placescPtj}|j||SN)rPlace set_place)placeps ]G:\00. PROJECTS\API\Inventory\templateJSON\kerjaOCR\Lib\site-packages\paddle/base/compiler.py _place_objrs AKK Hc|r|ntj}|jjD]}|jdvsyy)N)sendrecvTF)rdefault_main_program global_blockopstype) main_programmainops r_is_pserver_moder$sB'eZdZdZd dZdZdZd dZdZdZ d Z y) CompiledPrograma :api_attr: Static Graph The CompiledProgram is used to transform a program or graph for various optimizations according to the configuration of build_strategy, for example, the operators' fusion in the computation graph, memory optimization during the execution of the computation graph, etc. For more information about build_strategy, please refer to :code:`paddle.static.BuildStrategy`. Args: program_or_graph (Graph|Program): This argument is the Program or Graph being executed. build_strategy(BuildStrategy): This argument is used to compile the program or graph with the specified options, such as operators' fusion in the computational graph and memory optimization during the execution of the computational graph. For more information about build_strategy, please refer to :code:`paddle.static.BuildStrategy`. The default is None. Returns: CompiledProgram Example: .. code-block:: python >>> import numpy >>> import paddle >>> import paddle.static as static >>> paddle.enable_static() >>> place = paddle.CPUPlace() >>> exe = static.Executor(place) >>> data = static.data(name='X', shape=[None, 1], dtype='float32') >>> hidden = static.nn.fc(x=data, size=10) >>> loss = paddle.mean(hidden) >>> paddle.optimizer.SGD(learning_rate=0.01).minimize(loss) >>> exe.run(static.default_startup_program()) >>> compiled_prog = static.CompiledProgram( ... static.default_main_program()) >>> x = numpy.random.random(size=(10, 1)).astype('float32') >>> loss_data, = exe.run(compiled_prog, ... feed={"X": x}, ... fetch_list=[loss.name]) Nct|tjr||_d|_nht|t j r7t|tj|j|_||_ntdt|zd|_ d|_ d|_ d|_d|_d|_d|_||_d|_y)Nz[The type of program_to_graph parameter is wrong, expected Graph or Program, but received %sF) isinstancerGraph_graph_programrProgramr/desc TypeErrorr_scope_place _executor _compiled _is_inference_share_vars_from_places_build_strategy_exec_strategy)selfprogram_or_graphbuild_strategys r__init__zCompiledProgram.__init__s & 3*DK DM ()*;*; < , -**%5%:%:;DK,DMm'()    " $ -"rc|jrJdtt|tt|tgsJd|_||_|S)zAdd inference optimize Args: config: instance of `NativeConfig` or `AnalysisConfig` to create predictor Returns: self z6Already compiled with inference, cannot be recompiled.T)rTanyrIInferNativeConfigInferAnalysisConfig _infer_config)rYconfigs r_with_inference_optimizez(CompiledProgram._with_inference_optimizese"" D C D "6#456#67    "# rctd)NzFSubclass of CompiledProgram should implement _with_distributed method.)NotImplementedErrorrYs r_with_distributedz!CompiledProgram._with_distributeds! T  rc  |jrl|rtjjd|jj t d|jjj |_n|Jdg|_t|ttfsJdjt||jt|_ t|j |j_|j$t'|_||j$_|j$j*dk(r|j$j(t,j.k(rt1|dz|j$_nV|j$j(t,j2k(rd|j$_nt1|dz|j$_d t5j6vrVt5j6d r?|j$j*dk7r&t9j:d d|j$_|j r|jj<dkDrf|j j>rP|j j>}|jj<t1|k(sJd ||j_ |j ro|j jB|j_"|j jF|j_$|j jJ|j_&|jjNrd |j_(|j |j jRr|j$j(t,j.k(sJd |jj<t1|zdkDsJd|jjTtjVjXk(sJdd|j_-g|_.|j^jaD]}|jcs|je%|jejgsD|jejt4jhjjjlk7s|jo}|j tq|j |s|j\js|jottutv|}tty|j\|_.|j\j{t5j|r tdt5j||j\d|j|j|j$|j|j^S)Nz*share_vars_from is set, scope is ignored. zSThe shared Program is not compiled and executed, so there is no variables to share.zOCurrently, The places type can only be list or tuple, but the input type is {}.rrFLAGS_use_cinnzwAt present, when CINN is turned on, each process can only contain one thread, so reset the number of threads to 1 here.z5The trainer numbers is not equal to endpoint numbers.Tz%DGC only used under CUDA environment.z.DGC is not avaliable for single card training.zADGC only can be used for AllReduce BuildStrategy.FzBCUDA Graph is not allowed to capture when running the first batch.)BrUsysstderrwriterR ValueError local_scopes _local_scopesrIlisttupleformatrrW BuildStrategyrrLis_distributionrXExecutionStrategy _use_device num_threads DeviceTypeCUDAlenXPUrglobalswarningswarn num_trainers_trainers_endpointstrainers_endpoints_nccl_comm_num nccl_comm_num_use_hierarchical_allreduceuse_hierarchical_allreduce$_hierarchical_allreduce_inter_nranks#hierarchical_allreduce_inter_nrankssync_batch_normenable_sequential_execution _enable_dgcreduce_strategyReduceStrategy AllReducefuse_all_reduce_ops_persistable_varsrKris_varrD persistableVarDescVarTypeRAWnamerEr)maprsetsortis_cuda_graph_capturing RuntimeErrorParallelExecutorrP)rYplaces use_devicescopetpsr#rs r_compile_data_parallelz&CompiledProgram._compile_data_parallels     !NO$$..6 *"&!6!6!@!@!M!M!OD $ (b ($!#D  T5M  \ c c L      '#0?D /? /N,    &"3"5D *4'    * *a /""..*//A36f+/##/$$00JNNB23##/25f+/##/   . /0##//14 MMU /0D   + MM$$11A5 11--33C''449 GF G7:D 3 ==151M1MD . 99  ; BB  D    / /?CD < == $)B)B##//:??B 76 7B$$11CK?!C @? @C$$44 //99: ?? ?: 8=D 4!#KK%%'D HHJ*HHJ**,HHJOO%)=)=)A)AAyy{MM-7 tL**11$))+>(c*f-. "&c$*@*@&A!B ##%  ' ' )T $$   " "  KK        KK  rc@tj|jSr )rcreate_paddle_predictorrarfs r_compile_inferencez"CompiledProgram._compile_inference@s++D,>,>??rc|jrF|r|j|k7r td|r&|jj |s td|Sd|_||_||_|j r|j |_|S|jg|_t|jtjrtj}nEt|jtjrtj}ntj }|j#||j|j|_|S)aCompile the program based on the configs. Args: scope: The variables (resources) that are associated with this compiled program. place: The location that the compiled program will be run on. Returns: self z,Cannot compile program with different scope.z,Cannot compile program with different place.T)rrr)rSrPrprQ_equalsrTrrRrVrIr CUDAPlacer{r|XPUPlacer~CPUr)rYrr rs r_compilezCompiledProgram._compileCs >>- !OPPT[[007 !OPPK    !446DN !KK=DL$++t~~6'__ DKK7'^^ '^^ !88%T[[9DN rc,|du}|r0|D]*}|j|jk(r%JdnTt|tjr t }n/t|tj r t }n t}|sJd|S)Nz7Place type not match. You may set wrong type of places.zNo places for execution.)_typerIrrrrrr)rYr place_list has_set_placers r _get_placeszCompiledProgram._get_placesis"$. GGI.MLM. %0(] E4==1'\ '\ 555zrr ) __name__ __module__ __qualname____doc__r\rcrgrrrrrrrGrGWs//b#2, A F@$LrrGcneZdZdZgZdZe d dZedZedZ edZ edZ y) IpuDynamicPatcherz1 Patcher for IPU dynamic2static support. cyr rrfs rr\zIpuDynamicPatcher.__init__s rNc ddlddlmddlm}ddlm}j}j}j}jj|fd}jr|jd jD]Z} j| j j#} | j%j#} | j'| \|r!|d dD cgc]} | | j c} n|D cgc]} | | j c} |D cgc]} | j c} j(rl|_j,} | j.j1D]6\}}|j1D]\}}| j2j5|j d |j6|j8|j:d }| j=|}||5| j2j?|j@jBjEd dddjG|j j#}|j%j#}|j'||| j.||<9|fd}|}|_Scc} wcc} wcc} w#1swYxYw)zD Convert the ConcreteProgram to IPUConcreteProgram. rNrk)backward)switch_to_static_graph) device_guardcj}j|jjj j }jj }|j||Sr )rgradients_with_optimizer _optimizerstaticExecutorCPUPlacedefault_startup_programrun)r+exestartup_programrconcrete_program ipu_strategypaddles rappend_backward_desczHIpuDynamicPatcher.convert_concrete_program..append_backward_descsf&33G  - -g|7N7N O--(():;C$mmCCEO GGO $Nrfloat16)dtyperT)rrrrshapebelong_to_optimizerg)value) initializercjrjjj}dh|_jjj j |d}jjjjj |tj j}|S)NcumsumF)use_fp16_guard)to_fp16_var_names)rr) enable_fp16ramp CustomOpListsunsupported_listcast_model_to_fp16rcast_parameters_to_fp16rIpuCompiledProgramcompile) amp_listrr+r feed_list fetch_listrrrs r func_compilez@IpuDynamicPatcher.convert_concrete_program..func_compiles''!==,,::<-5J)$*MM$5$5$H$H$11#(%I%!  !!99OO%$11&7: ) --)gi,  Nr)$rbaserbase.dygraph.baserbase.frameworkrinputsoutputsrr global_scoperto parametersrDr get_tensorr_share_data_with is_trainingrr _accumulatorsitemshelpercreate_global_variablerrr_get_device_for_paramset_variable_initializernnrConstantfind_var)rrclass_instancerrrrrrparam_or_bufferparam_or_buffer_tensor src_tensorelem optimizerkv param_namevar_tmprDdeviceparam_or_lr_tensor optim_tensorrrrrrrrs`` @@@@@rconvert_concrete_programz*IpuDynamicPatcher.convert_concrete_programs #>1!(("***:: **,      # #   I  . 0::O%*YY$$&jl #)..0;;=J " 3 3J ? ; /5abzNztT=MzNI/5Jvt9IvJI,34GDdiiG4  # #,@,B  )$//I!//5571+,779'J#**AA$\\$(%mm$\\%mm,0 BC'< `max_tracing_count`:{}. Too much cached programs will bring expensive overhead. The reason may be: (1) passing tensors with different shapes, (2) passing python objects instead of tensors.)rIrprrhash _recent_key_caches need_compiler _build_oncerrrr}ru) rYitemitem_idr_current_tracing_countrrrrrs r patch_getterz;IpuDynamicPatcher.patch_program_cache..patch_gettersTdH- D4j))*4jG&D dll*l.G.Gdll*!&&M<< (A(A!&&6'+&6&6t&<# !#4#M#M "2D4G4G$ %(($*=*=T*I) W%),DLL(9%(+CC!&&GHNHN13KH<<( (r __getitem__N) paddle.jit.dy2staticr$paddle.jit.dy2static.partial_programr'paddle.jit.dy2static.program_translatorrrr rr patcher_cacher))rr  old_getterrrrrrs` @@@@rpatch_program_cachez%IpuDynamicPatcher.patch_program_cachesU 7M  "-- ' )' )R$0  ''.. =* 5 rcddlm}|jdfd }||_tjj |dgy)Nr LRSchedulercR||jd|jiy)Nlr) set_optionslast_lr)rYepochrold_steps r patch_stepz8IpuDynamicPatcher.patch_lr_scheduler..patch_step:s$ T5 !  $ $dDLL%9 :rstepr )paddle.optimizer.lrrr&rrr))rrr%r$s` @rpatch_lr_schedulerz$IpuDynamicPatcher.patch_lr_scheduler2s=3## ;& ''.. VX/NOrcXtj|tj|yr )rrr()rs rregister_patchz IpuDynamicPatcher.register_patchAs--l;,,\:rcPtjD]\}}}t|||yr )rrsetattr)modulekeyattrs r release_patchzIpuDynamicPatcher.release_patchFs&!2!@!@ FC FC &"Arr ) rrrrrr\ staticmethodrrr(r*r0rrrrr{sM 7;h h T? ? B P P;;''rrceZdZdZdZdZdZdZdZ ddZ ddZ dd Z dd Z d Z d ZdZdZedZedZedZedZy ) IpuStrategya Help users precisely control the graph building in :code:`paddle.static.IpuCompiledProgram` . Returns: The IpuStrategy instance. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> paddle.enable_static() >>> ipu_strategy = static.IpuStrategy() c$tjrStj|_dddddd}|jj |d|_g|_d|_n tdddl m }|r|jyy) Nrr)on_chipuse_replicated_tensor_sharding)location_optimizer+accumulation_and_replication_reduction_type4mean_accumulation_and_replication_reduction_strategyFTz\Can not use IpuStrategy in non IPU compiled environment, please re-compile with WITH_IPU=ON.in_dynamic_mode) ris_compiled_with_ipur3 _ipu_strategyr!has_custom_opscustom_op_namesrrrr;r*)rYdefault_optionsr;s rr\zIpuStrategy.__init__`s  $ $ &!%!1!1!3D  !67'@AHI O    * *? ;"'D #%D $D n  +      ! rc.tj|y)a Register patchs function to support dynamic to static on IPU. This operation would break the dy2static functionality on CPU. Use `release_patch` to release the patch. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> ipu_strategy = static.IpuStrategy() >>> ipu_strategy.register_patch() N)rr*rfs rr*zIpuStrategy.register_patchxs" ((.rc,tjy)aa Release the registered IPU functions. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> ipu_strategy = static.IpuStrategy() >>> ipu_strategy.release_patch() N)rr0rfs rr0zIpuStrategy.release_patchs '')rcddlm}|r4||_|j|}|jj |yt d)a Set optimizer to ipu_strategy in dynamic mode. Args: optimizer (Optimizer): Optimizer to be used in training. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> linear = paddle.nn.Linear(10, 10) >>> optimizer = paddle.optimizer.SGD(learning_rate=0.01, ... parameters=linear.parameters()) >>> ipu_strategy = static.IpuStrategy() >>> ipu_strategy.set_optimizer(optimizer) rr:z,Only needs to set optimizer in dynamic mode.N)rr;rparse_optimizerr=r!r)rYrr;optimizer_attrss r set_optimizerzIpuStrategy.set_optimizersD. +  'DO"229=O    * *? ;MN NrcZfd}|g}ddi}|D]}|j||S)a Parse optimizer attributes for IPU dynamic to static support. Currently only support parse lr. Args: optimizer (Optimizer): Optimizer to be parsed. Returns: Dict. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> linear = paddle.nn.Linear(10, 10) >>> optimizer = paddle.optimizer.SGD(learning_rate=0.01, ... parameters=linear.parameters()) >>> ipu_strategy = static.IpuStrategy() >>> attrs = ipu_strategy.parse_optimizer(optimizer) cddlm}tjtrdjiStj|rdjiSy)Nrrr )r'rrI_learning_ratefloat)rrs rget_lrz+IpuStrategy.parse_optimizer..get_lrsQ 7)22E:i6677I44kBi66899Cr is_dynamicT)update)rYrrKattr_fnrEfns ` rrDzIpuStrategy.parse_optimizers;2 :('.B  " "24 (rcX|dk(r |r td||||d}|j|y)a] Set graph configuration to the IpuStrategy instance. Args: num_ipus (int, optional): Number of IPU devices. Default 1, which means only use 1 IPU. is_training (bool, optional): True is training graph, False is inference graph. Default True, which means is training mode. batch_size (int, optional): The batch-size in the graph. Used to make the graph batch-size fixed, if the batch-size in the graph is dynamic. Default 1, which means the batch-size would be set 1, if the batch-size is dynamice. enable_manual_shard (bool, optional): Enable graph sharding or not. Only if num_ipus > 1, enable_manual_shard is able to be set True. Default False, which means disabled. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> paddle.enable_static() >>> ipu_strategy = static.IpuStrategy() >>> ipu_strategy.set_graph_config(num_ipus=1, ... is_training=True, ... micro_batch_size=1, ... enable_manual_shard=False) rzAOnly if num_ipus > 1, enable_manual_shard is able to be set True.)num_ipusrmicro_batch_sizeenable_manual_shardN)rr!)rYrQrrRrSoptionss rset_graph_configzIpuStrategy.set_graph_configsDJ q=0S !& 0#6   !rct|jd}|s |r td||||d}|j|y)a Set pipelining configuration to the IpuStrategy instance. Used to optimize the throughput performance. Args: enable_pipelining (bool, optional): Enable data pipelining between subgraphs. Only if enable_manual_shard=True, enable_pipelining is able to be set True. Default False, which means disabled. batches_per_step (int, optional): Set the batches per run in data pipelining mode. Only if enable_pipelining=True, batches_per_step is able to be set > 1. Default 1, which means no data pipelining. enable_gradient_accumulation (bool, optional): Enable to accumulate gradients before updating the weights in training mode. Only if enable_pipelining=True, enable_gradient_accumulation is able to be set True. Default False, which means no gradient accumulation. accumulation_factor (int, optional): Specify the number of micro-batches to accumulate before applying the varUpdate. Default 1, which means disable the accumulation. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> paddle.enable_static() >>> ipu_strategy = static.IpuStrategy() >>> ipu_strategy.set_pipelining_config(enable_pipelining=False, ... batches_per_step=1, ... enable_gradient_accumulation=False, ... accumulation_factor=1) rSzKOnly if enable_manual_shard=True, enable_pipelining is able to be set True.)enable_pipeliningbatches_per_stepenable_gradient_accumulationaccumulation_factorN) get_optionrr!)rYrWrXrYrZrSrTs rset_pipelining_configz!IpuStrategy.set_pipelining_configsPN#oo.CD"'8] "3 0,H#6   !rc.d|i}|j|y)a Set half computation configuration to the IpuStrategy instance. Used to optimize the performance. Args: enable_fp16 (bool, optional): Enable FLOAT16 mode and transform FLOAT32 to FLOAT16. Default False, which means disable FLOAT16 mode. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> paddle.enable_static() >>> ipu_strategy = static.IpuStrategy() >>> ipu_strategy.set_precision_config(enable_fp16=False) rN)r!)rYrrTs rset_precision_configz IpuStrategy.set_precision_configIs 0 ;  !rNc||}||||d}|jd|i|jj||jsd|_yy)a Add a mapping to use popart custom ops running on the IPU. Args: paddle_op(str): the name of custom op in paddle. popart_op(str): the name of custom op in popart. domain(str): domain name of custom op in popart. version(int): version of custom op in popart. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> paddle.enable_static() >>> ipu_strategy = static.IpuStrategy() >>> ipu_strategy.add_custom_op('paddle_relu', 'popart_relu') N) paddle_op popart_opdomainversion custom_opT)r!r?r)r>)rYr`rarbrcrds r add_custom_opzIpuStrategy.add_custom_opesa>  !I""  +y12 ##I."""&D #rcv|jj|dh}|j|z rd|_yy)a# Set options from dict. Args: options(dict): dict of options. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> paddle.enable_static() >>> ipu_strategy = static.IpuStrategy() >>> options = {'num_ipus':1, 'enable_fp16': True} >>> ipu_strategy.set_options(options) r TN)r=r!keysr)rYrTrecompile_white_lists rr!zIpuStrategy.set_optionss<0 &&w/ $v <<>0 0 $D  1rc>|jj|dS)a Get option. Args: option(str): name of option. Returns: option value. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> paddle.enable_static() >>> ipu_strategy = static.IpuStrategy() >>> num_ipus = ipu_strategy.get_option('num_ipus') r)r=r[)rYoptions rr[zIpuStrategy.get_options .!!,,V4W==rc:|jj|y)a Enable PopART pattern to optimize the graph. Args: pattern(string): the name of the pattern. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> paddle.enable_static() >>> ipu_strategy = static.IpuStrategy() >>> ipu_strategy.enable_pattern("ViewSimplifyPattern") N)r=enable_patternrYpatterns rrlzIpuStrategy.enable_patterns. ))'2rc:|jj|y)a Disable PopART pattern. Args: pattern(string): the name of the pattern. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> paddle.enable_static() >>> ipu_strategy = static.IpuStrategy() >>> ipu_strategy.disable_pattern("ViewSimplifyPattern") N)r=disable_patternrms rrpzIpuStrategy.disable_patterns. **73rc$|jdS)zJ Get the number of IPU devices from IpuStrategy instance. rQr[rfs rrQzIpuStrategy.num_ipuss z**rc$|jdS)zU Get the boolean of training or inference from IpuStrategy instance. rrrrfs rrzIpuStrategy.is_training }--rc$|jdS)zX Get the boolean of enable pipelining or not from IpuStrategy instance. rWrrrfs rrWzIpuStrategy.enable_pipelinings 233rc$|jdS)zS Get the boolean of float16 mode or not from IpuStrategy instance. rrrrfs rrzIpuStrategy.enable_fp16rtr)rTrF)FrFr)F)Nz custom.opsr)rrrrr\r*r0rFrDrUr\r^rer!r[rlrppropertyrQrrWrrrrr3r3Ls&"0/&*$O@%R! /"f %* 2"h":GH*'X%<>23242++ .. 44 ..rr3ceZdZdZddZdZy)ral The IpuCompiledProgram is used to transform a program to a ipu-target program, such as forward graph extraction, computing graph transformation, useless scale Ops clean, etc. Args: program(Program, optional): This parameter represents the :code:`Program` to be executed. Default is None, which means the program will be set to the default program :code:`paddle.static.default_main_program()` . scope(Scope, optional): The scope used to run this program, you can switch it to different scope. Default is None, which means use the global scope :code:`paddle.static.global_scope()` . ipu_strategy(IpuStrategy, optional): This argument is used to build the program with the specified options, such as half computation, training or inference session, the number of IPUs, etc. Default is None, which means build the program based on the default `ipu_strategy`. Returns: IpuCompiledProgram Example: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> paddle.enable_static() >>> a = static.data(name='data', shape=[None, 1], dtype='int32') >>> b = a + 1 >>> main_prog = static.default_main_program() >>> ipu_strategy = static.IpuStrategy() >>> ipu_strategy.set_graph_config(num_ipus=1, is_training=True, micro_batch_size=1) >>> ipu_strategy.set_pipelining_config(enable_pipelining=False, batches_per_step=1, enable_gradient_accumulation=False, accumulation_factor=1) >>> ipu_strategy.set_precision_config(enable_fp16=False) >>> ipu_compiled_program = static.IpuCompiledProgram( ... main_prog, ... ipu_strategy=ipu_strategy) Nctjs td|tj}t |tj stdt|z||_ d|_ |||_ n#ddl }|jj|_ |||_nt!|_|j"rt%|j&|_nd|_tj*j-|_y)NzECan not use this function since PaddlePaddle is not compiled with IPUz:The type of program is wrong, expected Program, but got %sFrr)rr<rprrrIrMrOrrLrSrPrrrr=r3r>rr?_custom_op_names IpuBackend get_instance_backend)rYr+rrrs rr\zIpuCompiledProgram.__init__Bs((*W  ?446G'9#4#45Lw-       DK  --446DK  #!-D !,D   & &$' (D(D$ED !$&D !446 rc||jj|j|jj|jj|j j }g}t|jD]P\}}|jjd|jdk(s|jdk(s@|j|R|dddD]}|j|dD]%}|j|s|j|'|j jj!t#j$|j j|_|jj(r;ddg} | D]2} t#j*| } | j-|j&4gd } | D]I} t#j*| } | d k(r| j/d || j-|j&Kt#j*d } |j0r| j/d |j0| j-|j&gd} | D]V} t#j*| } | j/d || j/d|| j-|j&Xt#j*d} t#j2} | j5d| | j-|j&t6j8j;| }t=|j drs|j j>j@}|j j>|_|j j }|jB||j>_"|j }|D].}|jG|}|jjId0t=|ds|j |_%d|j_&|S)aK This interface is used to compile the input Program to a program to run the model on the ipu. Args: feed_list(list): This parameter represents the input Tensors of the model. fetch_list(list): This parameter represents the Tensors that need to be returned after the model. Returns: Program Example: .. code-block:: python >>> # doctest: +REQUIRES(env:IPU) >>> import paddle >>> import paddle.static as static >>> paddle.enable_static() >>> a = static.data(name='data', shape=[None, 1], dtype='int32') >>> b = a + 1 >>> main_prog = static.default_main_program() >>> ipu_strategy = static.IpuStrategy() >>> ipu_strategy.set_graph_config(num_ipus=1, is_training=True, micro_batch_size=1) >>> ipu_strategy.set_pipelining_config(enable_pipelining=False, batches_per_step=1, enable_gradient_accumulation=False, accumulation_factor=1) >>> ipu_strategy.set_precision_config(enable_fp16=False) >>> program = static.IpuCompiledProgram( ... main_prog, ... ipu_strategy=ipu_strategy).compile([a.name], [b.name]) Fr&fetchNr')r&roptimizer_extract_passoptimizer_state_align_pass)forward_graph_extract_passinfer_shape_passavg_shard_passdelete_scale_op_passrrpopart_canonicalization_pass custom_ops)ipu_inplace_passipu_graph_builder_passipu_runtime_replacer_passrgraph_to_program_passr+ lr_scheduler org_program)'r} set_scoperPset_ipu_strategyr=rLrr(rrN set_is_targetrr)r*has_var _remove_varflushrrJrKrget_passapplyrrz ProgramDesc set_not_ownedrrM_construct_from_deschasattrr _var_namer@lr_varrDset_need_check_feedrr)rYrrrneed_to_remove_op_indexr-rr.rDpasses pass_namea_pass convert_passrNr+ lr_var_nameprogram_global_block feed_namefeed_vars rrzIpuCompiledProgram.compilehsZJ  , &&t'9'9'G'GH}}113 "$|//0EAr GG ! !% (ww& BGGw$6'..q11 -TrT2E  # #E *3%C##C(((-%   "jj!3!34    ) )(,F$ y1 T[[)$   I]]9-F.. ; 2 LL %  =>  JJ|T%:%: ; T[[!   I]]9-F JJ{I . JJ|Z 0 LL %  }}%<= !""9d34;;'##88> 4==. 1--44>>K#'==#=#=G  ==557L*6*;*;K*HG ' '335"I+// :H MM - -e 4#w ."&--G */'r)NNN)rrrrr\rrrrrrs(T$7L}rr)rmrrirrrrr__all__rrxrv NativeConfigr_AnalysisConfigr`r{rrr$r/r<rErGrr3rrrrrs :: ));;%%33 %%)) __  1aaH N'N'bH.H.VNNr