\;j2PddlZddlZddlZddlZddlZddlmZddlZddl m Z ddl m Z m Z mZmZddlmZmZmZmZddlmZdd lmZm Z mZmZmZmZmZmZmZmZdd l m!Z"dd l#m$Z$m%Z%dd l&m'Z'gZ(ejRa*ejVZ,ejZZ.d Z/dZ0e'dZ1d.dZ2dZ3dZ4d/dZ5d/dZ6dZ7 d0dZ8 d1dZ9 d.dZ:dZ;dZdZ?dZ@dZAd ZBd!ZCd"ZDd3d#ZEd$ZFed%ZGGd&d'ZHGd(d)ZIGd*d+ZJGd,d-ZKy)4N) lru_cache)pir)OpResultProgramValuetranslate_to_pir)compilercore framework unique_name) convert_dtype) OperatorrVariable _apply_passconvert_np_dtype_to_dtype_default_main_program get_flags in_pir_modepaddle_type_to_proto_type set_flags)auto_checkpoint)FetchHandlerMonitorTrainerFactory)signature_safe_contextmanagerctS)a' :api_attr: Static Graph Get the global/default scope instance. There are a lot of APIs use :code:`global_scope` as its default value, e.g., :code:`Executor.run` Returns: Scope: The global/default scope instance. Examples: .. code-block:: python >>> import paddle >>> import numpy >>> paddle.static.global_scope().var("data").get_tensor().set(numpy.ones((2, 2)), paddle.CPUPlace()) >>> numpy.array(paddle.static.global_scope().find_var("data").get_tensor()) g_scope]G:\00. PROJECTS\API\Inventory\templateJSON\kerjaOCR\Lib\site-packages\paddle/base/executor.py global_scoper#9s & Nr!ct}|a|SNrscopeexs r" _switch_scoper)Os BG Ir!c#bKt|} dt|y#t|wxYww)a This function switches scope through python `with` statement. Scope records the mapping between variable names and variables ( :ref:`api_guide_Variable` ), similar to brackets in programming languages. If this function is not invoked, all variables and variable names are recorded in the default global scope. When users need to create variables with the same name, they need to switch scopes through this function if they do not want the mapping of variables with the same name to be overwritten. After switching through the `with` statement, all variables created in the `with` block will be assigned to a new scope. Parameters: scope: The new scope. Returns: None Examples: .. code-block:: python >>> import paddle >>> import numpy >>> paddle.enable_static() >>> new_scope = paddle.static.Scope() >>> with paddle.static.scope_guard(new_scope): ... paddle.static.global_scope().var("data").get_tensor().set(numpy.ones((2, 2)), paddle.CPUPlace()) >>> numpy.array(new_scope.find_var("data").get_tensor()) array([[1., 1.], [1., 1.]]) N)r)r&s r" scope_guardr+Vs+H u B b bs / / ,/ct|tjr|Dcgc]}t||c}St|tr|Dcgc]}t||c}St|tj sJ|j }t|dkDr td|jr,|rtj|Stj|Sycc}wcc}w)a# Convert a Tensor to a numpy.ndarray, its only support Tensor without LoD information. For higher dimensional sequence data, please use LoDTensor directly. Examples: .. code-block:: python >>> import paddle.base as base >>> import numpy >>> new_scope = base.Scope() >>> with base.scope_guard(new_scope): ... base.global_scope().var("data").get_tensor().set(numpy.ones((2, 2)), base.CPUPlace()) >>> tensor = new_scope.find_var("data").get_tensor() >>> base.executor.as_numpy(tensor) # or numpy.array(new_scope.find_var("data").get_tensor()) Args: tensor(Variable): a instance of Tensor copy(bool, optional): Whether to use deep copy. Returns: numpy.ndarray rzSome of your fetched tensors hold LoD information. They can not be completely cast to Python ndarray. Please set the parameter 'return_numpy' as 'False' to return LoDTensor itself directly.N) isinstancer LoDTensorArrayas_numpylist LoDTensorlodlen RuntimeError_is_initializednparrayasarray)tensorcopytr2s r"r/r/s0&$--.+126aD!622&$+126aD!622 fdnn -- - **,C 3x!| /   88F# #::f% %%32s C! C&ct|tjjs t |}t|tjjs t |}||k(S)a Returns True if the first dtype can be compatible the second one. Currently, we require the two dtype's have to be same. Args: dtype (np.dtype|VarType|str): The type of data: float32, int64, etc. Returns: True if the two types are same. )r-r VarDescVarTyper)firstseconds r"dtype_is_compatible_withrAsJ eT\\11 2*51 fdll22 3+F3 F?r!ct|}|t|k7ryt|D]+}||||dkr||||dkr||||k7s+yy)aw Returns True if the two dimensions are compatible. A dimension is compatible with the other if: 1. The length of the dimensions are same. 2. Each non-negative number of the two dimensions are same. 3. For negative number or 'None' in a dimension, it means unknown so it is compatible with any number. Args: first (list/tuple): integers representing shape. "None" or negative number means unknown. second (list/tuple): integers representing shape. "None" or negative number means unknown. Returns: True if the two dimensions are compatible. FrT)r3range)r?r@dim_lenis r"dimension_is_compatible_withrFsp(%jG#f+ 7^ 8 uQx!|  !9 q A  8vay   r!c|jjrPtj||j|}|9t d|j t |j|j|fzt|j|jst|jtjjrt|jn |j}t|jtjjrt|jn|j}t dj|j ||y)a Returns True if the variable doesn't require feed check or it is compatible with the shape and have same dtype as the fed value. A dimension is compatible with the other if: 1. The length of the dimensions are same. 2. Each non-negative number of the two dimensions are same. 3. For negative number or 'None' in a dimension, it means unknown so it is compatible with any number. Args: var (Variable): the Variable object feed (LoDTensor): the fed value, which must be a LoDTensor num_places: an integer value indicating the number of places. ParallelExecutor will divide data into devices (CPU/GPU) evenly. Returns: True if the shape and dtype of variable is compatible with the feed value Raises: ValueError: if the shape or dtype of the variable is not compatible with the feed value eThe fed Variable %r should have dimensions = %d, shape = %r, but received fed shape %r on each deviceBThe data type of fed Variable {!r} must be {!r}, but received {!r}T)descneed_check_feedr diff_tensor_shape ValueErrornamer3shaperA_dtypedtyper-r=r>rformat)varfeed num_places diff_shapevar_dtype_formatfeed_dtype_formats r"check_feed_shape_typerYs, xx!++D#((JG  !?88S^SYY CD  ( syyAcii)=)=>cii(YY dkkmT\\-A-ABdkkm,[[]  T[[HH.0A  r!ctj|||}|td|t|||fzt |j |st |tjjr t|n|}t |j tjjrt|j n|j }tdj|||y)a Returns True if the variable doesn't require feed check or it is compatible with the shape and have same dtype as the fed value. A dimension is compatible with the other if: 1. The length of the dimensions are same. 2. Each non-negative number of the two dimensions are same. 3. For negative number or 'None' in a dimension, it means unknown so it is compatible with any number. Args: feed (LoDTensor): the fed value, which must be a LoDTensor name (str): name of the variable target_shape (list): the shape that will be compared with feed dtype (core.VarDesc.VarType): the dtype that will be compared with feed num_places: an integer value indicating the number of places. ParallelExecutor will divide data into devices (CPU/GPU) evenly. Returns: True if the shape and dtype of variable is compatible with the feed value Raises: ValueError: if the shape or dtype of the variable is not compatible with the feed value rHrIT) r rLrMr3rArPr-r=r>rrR)rTrN target_shaperQrUrVrWrXs r"pir_check_feed_shape_typer\s0''lJGJ ;S& jA B  $DKKM5 9%!5!56 %  $++-)=)=> $++- (   P W W&(9   r!c^d}|jD]z}|jjdk(rZ|dz }|jjdd|k(sJ|jj dd}||vskt d|dn|dkDr|t |k7r t d|dkDS) aCheck whether the block already has feed operators. Return false if the block does not have any feed operators. If some feed operators have been prepended to the block, check that the info contained in these feed operators matches the feed_targets and feed_holder_name. Raise exception when any mismatch is found. Return true when the block has feed operators with matching info. Args: block: a block instance (typically global block of a program) feed_targets: a dictionary of {feed_target_name: feed_target_data} feed_holder_name: the name of the variable that holds the data of all feed targets. The type of this feed_holder variable is FEED_MINIBATCH, which is essentially vector. Returns: A boolean value that indicates whether a block has feed operators that match the info contained in feed_targets and feed_holder_name. rrTr XOutz'feed_targets' does not have variablez:Feed operators in program desc do not match 'feed_targets')opsrJtypeinputoutput Exceptionr3)block feed_targetsfeed_holder_name feed_countopfeed_target_names r"has_feed_operatorsrlGs*Jii 77<<>V # !OJ77==%a(,<< <<!ww~~e4Q7 |334D3EYO A~*L(99 H   >r!c,d}|jD]}|jj|k(s!|dz }|jjdd|k(sJ|jj dd}||Dcgc]}|jj c}vrt d|d|jjd}|||jj k(rJ|dkDr|t|k7r t d|dkDScc}w) aCheck whether the block already has fetch operators. Return false if the block does not have any fetch operators. If some fetch operators have been appended to the block, check that the info contained in these fetch operators matches the fetch_targets and fetch_holder_name. Raise exception when any mismatch is found. Return true when the block has fetch operators with matching info. Args: block: a block instance (typically global block of a program) fetch_targets: a dictionary of {fetch_target_name: fetch_target_data} fetch_holder_name: the name of the variable that holds the data of all fetch targets. The type of this fetch_holder variable is FETCH_LIST, which is essentially vector. fetch_op: the operator name of fetch Return: A boolean value that indicates whether a block has fetch operators that match the info contained in fetch_targets and fetch_holder_name. rr r_r^z'fetch_targets' does not have r`colzX % 1 K77>>%(+/@@ @@ " c 21 5  +8)+8C =) 45F4GyQ'',,u%C$ c(:(?(?(D(D(FF FFQ;#m*<< J   ?)s;!Dcggg}|jD]^}|j|k(s|dj|jd|dj|j d`g}t |D]E\}}t |tr||dvstd|d||dvs5|j|G|S)aCheck whether the block already has fetch operation. Return false if the block does not have any fetch operation. If some fetch operation have been appended to the block, check that the info contained in these fetch operation matches the fetch_targets. Raise exception when any mismatch is found. Return true when the block has fetch operation with matching info. Args: block: a block instance (typically global block of a program) fetch_targets: a list of fetch_target_data fetch_op: the operator name of fetch Return: A boolean value that indicates whether a block has fetch operators that match the info contained in fetch_targets. rr rNzFound fetch_target[z)] is type(str) and doesn't have fetch op.) rarNappendoperand_sourceattrs enumerater-strre) rfrprqrr fetch_inforjneed_fetch_inforE fetch_vars r"has_fetch_operationsrs*bJii 779 qM !2!21!5 6 qM F!3 4 O!-0 9 i % 1 -)!,UVjm +  " "9 -1 r!cp|j}|j}||jvr|j|}n6|j |t j jjd}||jvr|j|} n6|j |t j jjd} t|||skt|D]]\} } |j| r/|j| } |jdd|gid| gid| iFtjd| z_|rd } nd } t!|||| s\t|D]N\} }t#|t$t&fsJd | d t)||j+| d|gid| gid| iP|S) NTrNrb persistablerTr^r_rnrbinputsoutputsrzIThe variable %s is not found in program. It is not declared or is pruned.fetch_v2fetchWrong type for fetch_list[]: )clone global_blockvarsrS create_varr r=r>FEED_MINIBATCH FETCH_LISTrlr{has_var _prepend_opwarningswarnrvr-rr|rb append_op)programrT fetch_list feed_var_namefetch_var_name use_fetch_v2 tmp_programrfeed_varrrErNoutrrrSs r"_add_feed_fetch_opsrs--/K++-L )))##M2**%%44+  *** $$^4  ++%%00,  lD- @ GAt##D)"&&t,(((,"SEN !* ) _' j.(  +FAsh_ >+A3c$s)= >  " "cU| ,aj #  , r!c ddl}|j}d}t||||}|r|jj |5t |D]a\}}t |ttfsJd|dt||jj||t|z|} d| _ c dddyy#1swYyxYw)Nr pd_op.fetchrrT)paddlerrstatic program_guardr{r-rrrb_pir_opsrr|r) rrrrrrrr~rE fetch_inputrs r"_add_pir_fetch_opsrs'')LH*j.(O ]] ( ( 1"+O"<;!(E!2J/s#d;6G5HIJoo++#a&!8!#'#=2 1 1 1s A0B==Cc|dkr|St||zdk(sJtdt||Dcgc] }||||z }}|Dcgc]}tj|c}Scc}wcc}w)Nr r)r3rCr6r7)r9micro_batch_numrE chunk_tensorchunks r"_merge_tensorsrs! v; (A -- -q#f+77A q1&'7*6 6BHHUO 66  7s A'A,ctjrdnd}||d}ddd}t}|rd}t||||||r|rd}t|||||yyy)NTF)use_cudamem_opt_skip_varsboolz list[str]buffer_shared_inplace_passinplace_addto_op_pass)r is_compiled_with_cudarr) renable_inplace enable_addtoskip_var_namesrrz attr_typesempty_startup_program pass_names r"_apply_inplace_addto_passr$su113tH! GE$;GJ#I0  *Iuj +  *Iuj !|r!c t|tsJ| t}t|tjsJ|j t |}| Jd|zdz|j}|r t|d}|S)aR Fetch the value of the variable with the given name from the given scope. Args: name(str): name of the variable. Typically, only persistable variables can be found in the scope used for running the program. scope(core.Scope|None): scope object. It should be the scope where you pass to Executor.run() when running your program. If None, global_scope() will be used. Default None. return_numpy(bool): whether convert the tensor to numpy.ndarray. Default True. Returns: LodTensor|numpy.ndarray z Cannot find zm in scope. Perhaps you need to make the variable persistable by using var.persistable = True in your program.Tr:) r-r|r#r _Scopefind_var _to_name_str get_tensorr/)rNr' return_numpyrSr9s r" _fetch_varr9s" dC   } eT[[ )) ) ..d+ ,C ?! ? ^^ F&t, Mr!cd}t|tr|d}t|tr&|Dcgc] }|| }}dj|S||Scc}w)Nct|tr|jjSt|tr|St|tr t |St|t rt t |St|tr t |St|tr t |Stt |dz)Nz$ should be Variable, Operator or str) r-rrJrNr|ridrr TypeErrorrSs r"_to_strz_to_name_str.._to_str\s c8 $88==? " S !J S !s8O X &r#w<  X &s8O U #s8OCH'MMN Nr!r,)r-tupler0join)rSritemss r"rr[s\O$#u!f#t'* +stWT]s +xx{s| ,sAcddlm}ddlm}|dd}|j d}|j }t tjdd}||_ t|}t|D]>\}}|j} || _ || _|jj!| @t#j$|j'} | S) Nr)getenv_or_backup)fleet_executor_desc_pb2PADDLE_TRAINER_ENDPOINTSrPADDLE_TRAINER_IDr)distributed.backup_envrdistributed.fleet.protorsplitFleetExecutorDescintosgetenvcur_rankr3r{RankInforankip_port cluster_inforxr FleetExecutorSerializeToString) rrtrainer_endpoints_strtrainer_endpointsfleet_exe_descrnrankrendpoint rank_info fleet_exes r"_prepare_fleet_executorrws9A,-GL-33C8,>>@N2990!45H&N ! "E#$56h+446  $ ##**95 7 "">#C#C#EFI r!ct|tr2t|t|jzt ||zS|j j t|jzt ||zSr%)r- PirProgramr| raw_address_get_program_cache_keyrJcached_hash_str)rr'rTrs r")_get_strong_program_cache_key_for_new_exersy':& L%##%& '$T:6 7  LL ( ( *%##%& '$T:6 7 r!cd}t|tjr |jn|}||jdt t |zt||zS)Ncg}t|jjD]}|j|dj |S)N )r0rkeysrxr)rf block_strvar_names r"_get_varname_from_blockz>_get_strong_program_cache_key.._get_varname_from_blocks> UZZ__./H   X &0yy##r!r)r-r CompiledProgram_programblocksr|rr)rrTrr inner_programs r"_get_strong_program_cache_keyrsf$ gx77 8   4 4Q 78 bk   z 2 3r!c g}t|trt|j}nEt|ttfr/t |D]!\}}|t|jz }#tt t|}||zSr%)r-dictr0rrr{mapr)rTrfeed_var_namesrEeachfetch_var_namess r"_get_feed_fetch_var_namesrsrN$diik* D4- ( GAt d499;/ /N'3|Z89O O ++r!c,tt||Sr%)r|r)rTrs r"rrs (z: ;;r!cht|tjs|Jdt|tjj r t |n|}tj|r%tj|j|}n}t|ttfrOtj|}|jtjk(r td|j|}ntdt|dtj }|j#|||S)a Convert numpy.ndarray to Tensor, its only support Tensor without LoD information. For higher dimensional sequence data, please use LoDTensor directly. Examples: .. code-block:: python >>> import numpy as np >>> import paddle.base as base >>> place = base.CPUPlace() >>> exe = base.Executor(place) >>> data = np.array((100, 200, 300)) >>> np_outs = map(lambda x: base.executor._as_lodtensor(x, place), data) Args: data(numpy.ndarray|list|tuple|scalar): a instance of array, scalar, list or tuple data(core.Place): the place of created tensor dtype(core.VarDesc.VarType|str): the expected data type of created tensor Returns: LoDTensor z:The dtype should be given when feed data is not np.ndarrayz Faild to convert input data to a regular ndarray : * Usually this means the input data contains nested lists with different lengths. Please consider using 'base.create_lod_tensor' to convert it to a LoD-Tensor.zConvert data of type z to Tensor is not supported)r-r6ndarrayr r=r>risscalarr7astyper0rrQobject_rrbr1set)dataplacerQr9s r" _as_lodtensorr s2 dBJJ '   H G H %!5!56 %   ;;t 88D>((/D tUm ,88D>DzzRZZ'd ;;u%D'T |3NO  ^^ F JJtU Mr!c&t|tjxs$t|jtj}|rOt|tjr|n |j}|jrt j dtyy)Nz-Standalone executor is not used for inferenceFT)r-r r_graph _is_inferencerr UserWarning)rr compiledcompiled_programs r"_can_use_interpreter_corers}'8#;#;< 00AH'8#;#;<    ) ) MM?  r!c.tj|yr%)loggingwarning)msgs r" _warning_oncer s OOCr!c*eZdZddZdZedZy) FetchHandlerNc(|J||_||_yr%)var_dict period_secs)selfrrs r"__init__zFetchHandler.__init__s###  &r!c|D]L}t||tjus"tjj |d||ddNtjj dy)Nz[0]: r r)rbr6rsysstdoutwrite)rres_dictkeys r"handlerzFetchHandler.handlers^CHSM"bjj0   C5hsmA.>-?q!AB r!ctdy)Na class FetchHandlerExample(FetchHandler): def handler(self, res_dict): print(res_dict["auc"]) print("auc: {}, {}".format(res_dict["auc"], time.ctime())) auc = Variable() var_dict = {"auc": auc} handler = FetchHandlerExample(var_dict=var_dict) )printr r!r"helpzFetchHandler.helps   r!)N<)__name__ __module__ __qualname__rr' staticmethodr*r r!r"rrs '      r!rcBeZdZdZ ddZdej fdZdZy)_StandaloneExecutorctj|_|jj|||_||_|j |_yr%)r Place_place set_place_plan_scope_create_new_executor_new_exe)rr planr's r"rz_StandaloneExecutor.__init__,s?jjl  e$  113 r!c0|jj||j}|rAt|d}t dds$t ||j jS|S|j jdkDr td|S)a Args: feed_names(list): This parameter represents the input names of the model. fetch_list(list): This parameter represents the Tensors that need to be returned after the model runs. The default is None. return_numpy(bool): This parameter indicates whether convert the fetched Tensors (the Tensor specified in the fetch list) to numpy.ndarray. if it is False, the type of the return value is a list of :code:`LoDTensor`. The default is True. TrFLAGS_enable_pir_in_executorr z;`merge_tensor` does not support when return_numpy is False.) r9run _move_to_listr/rrr6rr4)r feed_namesrenable_job_schedule_profilertensorss r"r=z_StandaloneExecutor.run3s--## 4 -/  wT2G;<.>zz/I/I/KLLNzz))+a/"QNr!returnc<|jj|}|Sr%)r9 run_profile)rr? program_descs r"rDz_StandaloneExecutor.run_profilePs}}00< r!cptj|j|j|j}|Sr%)r StandaloneExecutorr4r6r7)rnew_exes r"r8z(_StandaloneExecutor._create_new_executorTs'))$++tzz4;;Or!N)TF) r,r-r.rr=r ProgramDescrDr8r r!r"r1r1+s*4KP:)9)9r!r1cDeZdZGddZdZdZdZdZdZdZ y ) _ExecutorCacheceZdZdZdZdZy)_ExecutorCache._CachedDatac||_||_||_||_||_||_||_t|jtjr|jjsFtj|jjj|j_ tt!|jj|j |j|j|_ytt!|j|j |j|j|_yr%)rrTrrrr r'r-r rrr IrGraphr to_programhashrr&rrrTrrrr r's r"rz#_ExecutorCache._CachedData.__init__[s#DLDI(DO!.D "0D DJDJ$,,(@(@A||,,-6,=,= ++- jlLL) = --    = djj$))T__r!clt|tjxr|j|jk(Sr%)r-rK _CachedDatar&)rothers r"__eq__z!_ExecutorCache._CachedData.__eq__s,5."<"<=*HH ) r!c|jSr%)r&rs r"__hash__z#_ExecutorCache._CachedData.__hash__s 88Or!N)r,r-r.rrVrYr r!r"rTrMZs( T   r!rTctd|j|_td|j|_y)N)maxsize)r_get_program_and_executor _get_cached_program_and_executor_get_pir_program_and_executor)_get_cached_program_and_executor_pir_moderXs r"rz_ExecutorCache.__init__sB1E !0D  * *1 -:N19M  . .: 6r!c8|jjyr%)r^ cache_clearrXs r"clearz_ExecutorCache.clears --99;r!c N|j|j|||||||Sr%)r^rTrRs r"get_program_and_executorz'_ExecutorCache.get_program_and_executors:44      r!c|j}t|tjr |jn|}|j }|j }|j}|j}|j}|j} t|tjs$t|jtjrt|tjr|n |j} | j} d} | "| jrd} d| _ tddi| j| || rd| _ t!j"| j} | j%}t'|dr|j(|_|}n'd} ddlm}m}|r|t1k(r||}t3|||||d}| | j4rdnd}| | j6rdnd}t9ddrd}d}|s|rt;||}t=|||||j?}|j@r4d |j@vr&dd l!m"}|j@d }|d }|||||}n`tGjHd }t9ddrd tK|jLi}nd |jLi}tGjN|g|}|jPrd |jPvrtS|jPd dkDrptUjV}|jPd D]}|jY||j[D]$}|j]|}|j_|&ta||| }||fS)NFT!FLAGS_new_executor_use_cuda_graph lr_schedulerr) prim2orig prim_enabled)rrTrrrrr<standalone_opt) apply_pass schedule_modedefault pass_list)1rr-r rrrTrrrr r'r_build_strategyallow_cuda_graph_capturer_compiler rOrPhasattrrhpaddle.incubate.autogradrirjrrrrrrrr _pipeline_opt1paddle.distributed.passes.pipeline_scheduler_passrlr Jobr rJPlan _pass_optr3r PassManageradd_pass job_types ir_programr=r1) r cached_datarrrTrrrr r'rbuild_strategyuse_cuda_graphir_graphconverted_programrirjrrr new_programrlrkrr: default_jobtype_to_programpmpjob_typer}rHs r"r]z(_ExecutorCache._get_program_and_executorst%%'8#;#;<      ++ #11 $33!!!! gx77 8J NNH44= gx'?'?@^^  .==N"N*";;!%:?7>EF  % %eU 3:>7 (()9)@)@AH ( 3 3 5 }n51>1K1K!.-M"N H~'-A-C"C #M%!!')  %)F)F  )n.I.I   3 4 * #N L \6tZHN %~ mmo  % % K$=$== )667GHN&7I[)^D((9-K78./ 0@0@A#$-k.>.>"?99k]O hhy) $g.yy+8%eT59r!N) r,r-r.rTrrcrer]rr_r r!r"rKrKYs-22h  < ,C$J , r!rKc.eZdZdZd4dZdZdZdZdZdZ d Z d Z d Z d Z d ZdZdZdZdZdefdZdZdZdZdZdZedZe d5dZedZ dZdZ d6dZ d Z!d!Z"d"Z#d#Z$d7d$Z%d%Z&d&Z' d8d'Z( d9d(Z) d:d)Z*d*dddgdfd+Z+ d;d,Z,d-Z-e dd1Z1 d?d2Z2 d>d3Z3y)@Executora :api_attr: Static Graph An Executor in Python, supports single/multiple-GPU running, and single/multiple-CPU running. Args: place(paddle.CPUPlace()|paddle.CUDAPlace(n)|str|None): This parameter represents which device the executor runs on. When this parameter is None, PaddlePaddle will set the default device according to its installation version. If Paddle is CPU version, the default device would be set to `CPUPlace()` . If Paddle is GPU version, the default device would be set to `CUDAPlace(0)` . Default is None. If ``place`` is string, it can be ``cpu``, and ``gpu:x``, where ``x`` is the index of the GPUs. Note: users only pass one Place or None to initialize Executor when using multiple-cards. Other APIs will override the cards. See `document for multiple-cards `_ Returns: Executor Examples: .. code-block:: python >>> import paddle >>> import numpy >>> import os >>> # Executor is only used in static graph mode >>> paddle.enable_static() >>> # Set place explicitly. >>> # use_cuda = True >>> # place = paddle.CUDAPlace(0) if use_cuda else paddle.CPUPlace() >>> # exe = paddle.static.Executor(place) >>> # If you don't set place, PaddlePaddle sets the default device. >>> exe = paddle.static.Executor() >>> train_program = paddle.static.Program() >>> startup_program = paddle.static.Program() >>> with paddle.static.program_guard(train_program, startup_program): ... data = paddle.static.data(name='X', shape=[None, 1], dtype='float32') ... hidden = paddle.static.nn.fc(data, 10) ... loss = paddle.mean(hidden) ... paddle.optimizer.SGD(learning_rate=0.01).minimize(loss) ... >>> # Run the startup program once and only once. >>> # Not need to optimize/compile the startup program. >>> exe.run(startup_program) >>> # Run the main program directly without compile. >>> x = numpy.random.random(size=(10, 1)).astype('float32') >>> loss_data, = exe.run(train_program, feed={"X": x}, fetch_list=[loss.name]) >>> # Or, compiled the program and run. See `CompiledProgram` >>> # for more details. >>> compiled_prog = paddle.static.CompiledProgram( ... train_program) >>> loss_data, = exe.run(compiled_prog, feed={"X": x}, fetch_list=[loss.name]) NcT|tj}||_ntj||_i|_i|_i|_i|_i|_i|_ i|_ tj}|j|jtj||_d|_i|_d|_t'j(d|_t-|_d|_d|_tj4j7|_d|_y)NF__auto_checkpoint_executor__)r _current_expected_place_r _get_paddle_placeprogram_caches ctx_cachestrainer_caches scope_cachesmicro_scope_cache var_cachespruned_program_cachesr r3r5r_default_executor_closedpruned_program_scope_caches_prepare_to_run_calledrgenerate_auto_checkpoint_namerK_executor_cache_fleet_executor_fleet_executor_with_standaloneop_proto_and_checker_makerkOpRoleAttrName op_role_keyr@)rr expected_placers r"rzExecutor.__init__s =&??AN'DJ"44U;DJ  !#%'" JJL DJJ!%q!1 +-(&+#%0%9%9 *& " ./#05,::JJL,1)r!c|j|jvxrTt|j|jttj j jzSr%)r attr_namesr all_attrsr rOpRoleOptimize)rrjs r"_is_optimizer_opzExecutor._is_optimizer_ops[2==0AS LLN4++ ,6 //66?? @6A Ar!cX|j|jjyr%)closerrcrXs r"__del__zExecutor.__del__s  ""$r!c:|jj|dSr%)rgetrprogram_cache_keys r"_get_scope_cachezExecutor._get_scope_caches  $$%6==r!c:|jj|dSr%)rrrs r"_get_ctx_cachezExecutor._get_ctx_caches""#4d;;r!c:|jj|dSr%)rrrs r"_get_trainer_cachezExecutor._get_trainer_cache""&&'8$??r!c:|jj|dSr%)rrrs r"_get_program_cachezExecutor._get_program_cacherr!c"||j|<yr%)rrrrs r"_add_program_cachezExecutor._add_program_caches18-.r!c:|jj|dSr%)rrrs r"_get_pruned_program_cachez"Executor._get_pruned_program_caches))--.?FFr!c"||j|<yr%)rrs r"_add_pruned_program_cachez"Executor._add_pruned_program_caches8?""#45r!c:|jj|dSr%)rrrs r"_get_pruned_program_scope_cachez(Executor._get_pruned_program_scope_caches//334EtLLr!c"||j|<yr%)rrs r"_add_pruned_program_scope_cachez(Executor._add_pruned_program_scope_caches>E(():;r!c"||j|<yr%)r)r ctx_cache_keyctxs r"_add_ctx_cachezExecutor._add_ctx_caches), &r!c"||j|<yr%)r)rtrainer_cache_keyrs r"_add_trainer_cachezExecutor._add_trainer_caches14-.r!c"||j|<yr%)r)rscope_cache_keyr's r"_add_scope_cachezExecutor._add_scope_caches-2/*r! micro_scopesc"||j|<yr%)r)rrrs r"_add_micro_scopes_cachez Executor._add_micro_scopes_caches4@01r!c:|jj|dSr%)rrrs r"_get_micro_scopes_cachez Executor._get_micro_scopes_caches%%))*;TBBr!c"td|dy)Nz"use_program_cache is force set to z! by FLAGS_FORCE_USE_PROGRAM_CACHE)r)ruse_program_caches r"_log_force_set_program_cachez%Executor._log_force_set_program_caches01B0CCd e r!c \|j}|jD] }|jjdk(r|jj dd}||}|j |} | j tjjjk7rGt|tjs!t||j| j }t| ||jj!d} d} t#| | rtj$|||| |g} d} |j&rd|j&vr}|j&dd} t)|j*r|j+dn|j*d}|| zdk(sJt-j.t-j0|| d} t3| D]N}| dkDr$t| ||j| j n| |}tj$|||| | z|zP yy) NrTr_rrnr<r rknum_micro_batches)rrarJrbrdrSrQr r=r>STRINGSr-r1r r rYrorset_feed_variablerucallablerOr6rr7rC)rrrTrr'rrjrkcur_feedrSru pir_flag_namemicro_cur_feednum_micro_batch batch_sizerE micro_feeds r" _feed_datazExecutor._feed_datas++- ""Bww||~'#%77>>%#8#;  01"&&'7899 4 4 < <<%h?#0$djj#))$*#x8ggll5) > ]+M:**x)93'/ZN&'O--,0E0EE*1*?*?,+-+/ (7%NN,Q/!)!2# *O;q@@@)+HHX.*#?3 /2* .q 14::syy"0!2 #..!&)/1A5 4g#r!cvt}|j}|jD]}|jdk(r|j d}|j |t |j d}|j d} ||} t| tjst| |j|} t| || |tj|| |dnt|jD]0} | |vs|j!| t#j$d| z2y)Nz pd_op.datarNrQrOrzFThe value %s is not found in program. It is not declared or is pruned.)r rrarNrzaddrr-r r1r r r\rr0rpoprr) rrrTr'feed_target_namesrrjrkvar_type var_shaper feed_names r"_pir_feed_datazExecutor._pir_feed_data8sE++- ""BwwyL(#%88:f#5 !%%&674RXXZ5HIHHJw/  01!(DNN;,Xtzz8LH). 8&&uh8H!L#$diik*I 11# \ +r!c|tt|Dcgc]}tj|||}}|Scc}wr%)rCr3r get_fetch_variable)rrrr'rEoutss r" _fetch_datazExecutor._fetch_dataVsH3z?+ +  # #E>1 =+    s9c xg}g}d}t|D]\}}t|tr|D] }||||(t|trat|dttfs3t dj |||t |dj|dD] }||||||||||fS)a4 Split optimize_ops from fetch_list, which provided to specify program prunning. Args: fetch_list(list): The original fetch_list. Possible types of fetch_list are: fetch_list = ['loss'] fetch_list = [[sgd, sgd], 'loss'] fetch_list = [([sgd, sgd], [(param, grad)]), 'loss'] Returns: optimize_ops(list): The optimize operators splited from fetch_list. fetch_list(list): The updated fetch_list which does not contain optimize operators. ct|tr-|jr|j|yt dt|t t fr|j|yt dt|)Nz0The operator in fetch_list is not an optimize_opzOThe item in fetch_list should be str, variable or optimize_op, but received %s.)r-r_is_optimize_oprxrrr|rb) _optimize_ops _fetch_listrs r" _get_targetsz@Executor._split_optimize_ops_in_fetch_list.._get_targetsosk$)'')!((.#JD8S/2""4(eJr!rzRequires fetch_list[{}][0] shall be one of (list, tuple) when type(fetch_list[{}]) is `tuple`, but received fetch_list[{}][0]'s type is `{}`.)r{r-r0rrrRrbr,)clsrrrrindexrrEs r"!_split_optimize_ops_in_fetch_listz*Executor._split_optimize_ops_in_fetch_list]s    %Z0KE4$%A  Q?D%(!$q'D%=9#hoo!5%d1g1G1G aA  Q?!]K>#1&M))r!ct|tj}|r/|jr |j}nt j dy|}g}t|t rt|j}nEt|ttfr/t|D]!\}} |t| jz }#|sD|jD]5} | jD]$} | js|j| &7||z} |j|| } |r4| |_t!j"| j$|_d|_|S| }|S)a Prune operators and variables which are not needed to generate :code:`fetch_list` and optimize operators. Prune operators and variables which are needed to generate variables to be feeded. Notes: This is a very low level API. Users should not use this API directly. Args: program(Program): the origin program feed(list|dict): feed dict or list. fetch_list(list|Variable): A list of variables need to be fetched optimize_ops(list[Operator]): A list of optimizer operators Returns: Program: A new, pruned program. z[The program holds no _program, maybe it is constructed by graph, which can't be pruned yet.NF)r-r rrrrrr0rrr{rrarrx_prune_with_inputr GraphrJr _compiled)rrrTr optimize_opsrorigin_programr?rErrfrjtargetspruned_programs r"_prune_programzExecutor._prune_programs3,gx'?'?@ !(!1!1 q$N dD !diik*J tUm ,$T?4d499;// +'..))B))+$++B/$/ |+'99*gN  .G !ZZ(;(;|jr|jj }n't j d|S|j }t|tr[t|jD]=}|j|r|j|t j d|z?|St|ttfrlt|D]^\}}t|jD]=}|j|r|j|t j d|z?`|S)ae Update the feed dict, remove the feed item which is pruned in program. Notes: This is a very low level API. Users should not use this API directly. Args: program(Program): the pruned program. feed(list|dict): feed dict or list. Returns: feed:(list|dict) updated feed. z@The program holds no _program, maybe it is constructed by graph.r)r-r rrrrrrr0rrrrr{)rrrTrrrrErs r" _update_feedzExecutor._update_feeds gx'?'?@ &//<<>  V "//1L dD !!$))+. #++I6HHY'MMc#$/" tUm ,$T?4!%diik!2I'// :+ g'("3+ r!c|js`d|_|jjD]!\}}|jj |~#|jj yy)a2 Close the executor. This interface is used for distributed training (PServers mode). This executor can not be used after calling the interface, because this interface releases resources associated with the current Trainer. Returns: None Examples: .. code-block:: python >>> import paddle >>> cpu = paddle.CPUPlace() >>> exe = paddle.static.Executor(cpu) >>> # execute training or testing >>> exe.close() TN)rritemsrrelease_trainerr)rktrainer_instances r"rzExecutor.closesc(||DL'+':':'@'@'B##&&667GH$(C  " " ( ( * r!c|jry|jjD]!\}}|jj |~#|jj y)z7 flush all trainer param to root_scope N)rrrrrrc)r_rs r"flushzExecutor.flush"sY << #'#6#6#<#<#> A  " " 2 23C D $? !!#r!Fc tjjdd} | | dv}|j|t r|j |||||||} | S|j |||||||||  } tj| S)a Run the specified :code:`Program` or :code:`CompiledProgram`. It should be noted that the executor will execute all the operators in :code:`Program` or :code:`CompiledProgram` without pruning some operators of the :code:`Program` or :code:`CompiledProgram` according to fetch_list. And you could specify the scope to store the :code:`Tensor` during the executor running if the scope is not set, the executor will use the global scope, i.e. :code:`paddle.static.global_scope()`. Args: program(Program|CompiledProgram): This parameter represents the :code:`Program` or :code:`CompiledProgram` to be executed. If this parameter is not provided, that parameter is None, the program will be set to :code:`paddle.static.default_main_program()`. The default is None. feed(list|dict): This parameter represents the input Tensors of the model. If it is single card training, the feed is dict type, and if it is multi-card training, the parameter feed can be dict or list of Tensors. If the parameter type is dict, the data in the feed will be split and sent to multiple devices (CPU/GPU), that is to say, the input data will be evenly sent to different devices, so you should make sure the number of samples of the current mini-batch must be greater than the number of places; if the parameter type is list, those data are copied directly to each device, so the length of this list should be equal to the number of places. The default is None. fetch_list(list): This parameter represents the Tensors that need to be returned after the model runs. The default is None. feed_var_name(str): This parameter represents the name of the input Tensor of the feed operator. The default is "feed". fetch_var_name(str): This parameter represents the name of the output Tensor of the fetch operator. The default is "fetch". scope(Scope): the scope used to run this program, you can switch it to different scope. default is :code:`paddle.static.global_scope()` return_numpy(bool): This parameter indicates whether convert the fetched Tensors (the Tensor specified in the fetch list) to numpy.ndarray. if it is False, the type of the return value is a list of :code:`LoDTensor`. The default is True. use_program_cache(bool): This parameter indicates whether the input :code:`Program` is cached. If the parameter is True, the model may run faster in the following cases: the input program is :code:`paddle.static.Program`, and the parameters(program, feed Tensor name and fetch_list Tensor) of this interface remains unchanged during running. The default is False. use_prune(bool): This parameter indicates whether the input :code:`Program` will be pruned. If the parameter is True, the program will be pruned accroding to the given feed and fetch_list, which means the operators and variables in program that generate :code:`feed` and are not needed to generate :code:`fetch_list` will be pruned. The default is False, which means the program will not pruned and all the operators and variables will be executed during running. Note that if the tuple returned from :code:`Optimizer.minimize()` is passed to :code:`fetch_list`, :code:`use_prune` will be overrided to True, and the program will be pruned. Returns: List: The fetched result list. Examples: .. code-block:: python :name: code-example-1 >>> import paddle >>> import numpy >>> # First create the Executor. >>> paddle.enable_static() >>> place = paddle.CPUPlace() # paddle.CUDAPlace(0) >>> exe = paddle.static.Executor(place) >>> data = paddle.static.data(name='X', shape=[None, 1], dtype='float32') >>> hidden = paddle.static.nn.fc(data, 10) >>> loss = paddle.mean(hidden) >>> adam = paddle.optimizer.Adam() >>> adam.minimize(loss) >>> i = paddle.zeros(shape=[1], dtype='int64') >>> array = paddle.tensor.array_write(x=loss, i=i) >>> # Run the startup program once and only once. >>> exe.run(paddle.static.default_startup_program()) >>> x = numpy.random.random(size=(10, 1)).astype('float32') >>> loss_val, array_val = exe.run(feed={'X': x}, ... fetch_list=[loss.name, array.name]) >>> print(array_val) >>> # doctest: +SKIP("Random output") [array(0.16870381, dtype=float32)] >>> # doctest: -SKIP .. code-block:: python :name: code-example-2 >>> # doctest: +REQUIRES(env:GPU) >>> import paddle >>> import numpy as np >>> # First create the Executor. >>> paddle.enable_static() >>> place = paddle.CUDAPlace(0) >>> exe = paddle.static.Executor(place) >>> data = paddle.static.data(name='X', shape=[None, 1], dtype='float32') >>> class_dim = 2 >>> prediction = paddle.static.nn.fc(data, class_dim) >>> loss = paddle.mean(prediction) >>> adam = paddle.optimizer.Adam() >>> adam.minimize(loss) >>> # Run the startup program once and only once. >>> exe.run(paddle.static.default_startup_program()) >>> build_strategy = paddle.static.BuildStrategy() >>> binary = paddle.static.CompiledProgram( ... paddle.static.default_main_program(), build_strategy=build_strategy) >>> batch_size = 6 >>> x = np.random.random(size=(batch_size, 1)).astype('float32') >>> prediction, = exe.run(binary, ... feed={'X': x}, ... fetch_list=[prediction.name]) >>> # If the user uses two GPU cards to run this python code, the printed result will be >>> # (6, class_dim). The first dimension value of the printed result is the batch_size. >>> print("The prediction shape: {}".format( ... np.array(prediction).shape)) The prediction shape: (6, 2) >>> print(prediction) >>> # doctest: +SKIP("Random output") [[-0.37789783 -0.19921964] [-0.3577645 -0.18863106] [-0.24274671 -0.12814042] [-0.24635398 -0.13003758] [-0.49232286 -0.25939852] [-0.44514108 -0.2345845 ]] >>> # doctest: -SKIP FLAGS_FORCE_USE_PROGRAM_CACHEN)r 1TTruetrue)rrTrrrr'r) rrTrrrr'rr use_prune) renvironrrr _run_pir_impl _run_implr update_autotune_status) rrrTrrrr'rrrforce_use_program_cacheress r"r=z Executor.run-s\#%**.. +T#  # . 7<!   - -.? @ =$$%+-)%C, ..%+-)"3#! C  ' ' ) r!c |jr td|du} | t}|j|}ddlm} t |tr|jr| s{d|jvr;|jt|_ |j||||j|Sd|jvr|jd}n|j|||St |tr|jr|jd} tj | } t#j$} | j'| t#j(| |_d|jvr|jd}t |tr?t-|j/j0dk(r| rd }t3j4|| t7}|}|}|j9|\}}|rd } | rt;|||}|j=|}|t |t>j@r|jCtEtG|}tIjH|}||_%|jCtEtG| $|jMtEtG|||jO||||}|jQ||n|}|jS||}|}tU||jVr@|i}n0t |tXtZfrt-|d k(sJd |d}t |t\st_d ta|z|jS||}i}t |t>j@s$t |jbt>j@rt |t>j@r|n |jb}|jd}|~|jfrrddg}|D]O}tijj|d}t |tDr|jm}|dk(rd nd}to|||<Qtq|Dcic]}|d c}|jrju||||||jV|\}}|jw||||ty|drNddl=m>}t |j~|sJd|j~}|} |j/j|j}!tj| gjt|!j}"t#j||j}#t|"t#j}$t#jrt3j4dnQt#jr!|#j|$|#jn|#j|$|jV|jtY|j||j}%tq||%St |t>j@}&| r|&r|jj/}'n|j/}'|'jD]}(|'jj|(j})|'j|(}*|)jdusN|)jat"jjjk(s|)jd us|*jd us|*jd us|*jdus|(|vstd|(ztj|||j||jV|jsJd|j|j|j|Scc}w)N"Attempted to use a closed Executorr)use_new_executor fleet_opt)rrTrwith_standalone_executorrstartup_program)rr heter_placeNow you are using default_main_program, but there are no operators in the program to be executed. Please ensure you create model correctly or you can pass the Program or the CompiledProgram manually.Tr Not compiled with data parallel9feed requires dict as its Parameter. But you passed in %sFLAGS_new_executor_serial_run!FLAGS_new_executor_sequential_runFrrh LRSchedulermust be LRSchedulerCaution!!! When capturing CUDA Graph, the learning rate scheduler would not take any effect! Please set the learning rate manually before each batch!zNeed feed data for variable %sz0Program must have _is_inference = True, but get )drr4r_check_fetch_list-paddle.distributed.auto_parallel.static.utilsrr-rrurr_run_using_fleet_executorr _run_pipeline_heter_pipeline_optr rr r3r5rrr3rrarrr#rrrr rrr|rr:_share_vars_fromrrrrrr r0rrrrbrrpsequential_runrrlowerrrrrerrspaddle.optimizer.lrr%rhr _var_namer6r7rrrQget_variable_tensorr CPUPlaceis_cuda_graph_capturingis_compiled_with_ipu _copy_fromr4r=rr@rrJrencoderr=r> LOD_TENSORrK stop_gradientis_databelong_to_optimizerrMacp_auto_checkpointrrr_run_inference _executor)+rrrTrrrr'rrruse_default_main_programrrr error_info_origin_fetch_list_origin_programr cache_keycached_pruned_programprogram_scope_cacher stored_flagrr schedule_flagflagvaluefrHr%rhlr_valuelr_varr r9 cpu_tensorretrrvarnamevardescvarobjs+ r"rzExecutor._run_impls@ <<CD D#*d? ?*,G++J7  w (%%$&g333''/+B+DD(55#)-1-Q-Q!- 6!G$9$99!//0AB)))&7* gw 'G,G,G"55mDK#55kBK A KK $%)]]1%5D " G$?$??!556GH w (G((*../14'C  j) = NE(!#'#I#I $  L I 51I%)$B$B9$M !$,gx'?'?@*.*N*NB/0+'#ii0G/BG,<<? 34  <<? 34g"&!4!4T:|"..y.I!6$$^T:D$G $Wdjj 9|D4-04yA~H'HH~AwdD)ODz#$$Wd3DK'8#;#;<  8 8A "'8+C+CD ! "2!A!A!-.2O2O7;%M!. " $ 6%eS1$)KKME,1VODE,0K D) !.  > 1q$w >?#33LL   GW OOGT=% @w/;!((+)() '33 '> --/44\5K5KLxx +22=3NO11%9O9OP*4A //1MMd..0%%j&--/B%%j$**=++TYY[!11C k "Jgx'?'?@ &//<<> &335 ',,&++44W^^5EF%**73'')U2 $,,*>*>*I*II//1T9,,4$.22e;t+$%E%OPP- T7+ +  ! ! V =g>S>S=T U V !""7#4#4d;;U?s _#c ddl}|jj} |jjj} |j r t d|du} | r| }|j|}t|| r?t|jjdk(r| rd} tj| | t}|i}n0t|tt frt|dk(sJd|d}t|t"st%dt'|z|j(j+||||||j,|\}} |j/|||t1|dr;ddlm}t|j6|sJd |j6}|}|j8}t;j<|gj?tA|jB}tjDt|jF}tI|tjJ}tjLrtjd nQtjNr!|jQ||jSn|jQ||j,| jUt|jW|}|S) Nrrrr r r!rhr$r&r'),rrrr rrr4r(r-r3rrarrr#r0rrrrbrrr rrsr0r%rhrMr6r7rrrQr2r1r r3r4r5r6r4r=r)rrrTrrrr'rrrrr@rArHr%rhrLrMr r9rNrOs r"rzExecutor._run_pir_implss **$$%zzCC <<CD D#*d? #*,G++J7  w (G((*../14'C  j) = NE <D tUm ,t9> D#D D>7D$%K:   //LL      JJ   GT51 7N + 7$$k %$ %#//L#~H^^F88XJ'..}V\\/JKD-- 6 6F'tT]]_=J++- `**,!!*fmmo>!!*djj9kk$tyy{+\: r!c$|j|Sr%)r=)rexerTs r"r>zExecutor._run_inference%swwt}r!c dd}|gS|r|gS||sJdjt|g}t|D]\}}|r|j| ||rAt fd|Dr|j t |W|j|itdj|t|j|S)NcBt|ttttfSr%)r-rr|rrrs r"z,Executor._check_fetch_list..)s: (C51$ r!c.t|ttfSr%)r-rr0rs r"rXz,Executor._check_fetch_list..,sJsUDM$Br!zCurrently , The fetch_list type only should be list or tuple, but the input type is {}. For more information please refer to the executor.run(...).c3.K|] }|ywr%r ).0v is_fetch_vars r" z-Executor._check_fetch_list..?s41|AszPRequire fetch_list[{}] 's type shall be one of (OpResult, str), but received {}.) rRrbr{rxallextendr0rr,)rr is_tuple_listrrErSr]s @r"r(zExecutor._check_fetch_list(s C  I  #< Z(  %%+VD,<%= (  +FAsC  3s#444JJtCy)JJsOfmm49--,  r!clttt|dzd5}|jt|ddd|jrLd|jvr=tdd5}|jt|jddddyyy#1swYbxYw#1swYyxYw)Nz_train_desc.prototxtw fleet_desczfleet_desc.prototxt)openr|rr$ _fleet_opt)rrtrainerfouts r"_dump_debug_infozExecutor._dump_debug_infoLs #bk"%;;S AT JJs7| $B   ,'2D2D"D+S1T 3w11,?@A21#E B A21sB+(B*B'*B3ct|jj}||kr|}td||fz|||ddzkrtd||z|fz||z|dd<|j |dd|z|S)Nz[Pipeline training: setting the pipeline num to %d is enough because there are only %d filesconcurrency_listrznPipeline training: setting the 1st element in concurrency_list to %d is enough because there are only %d files)r3dataset get_filelistr) set_thread)r pipeline_optrl pipeline_numfilelist_lengths r"_adjust_pipeline_resourcez"Executor._adjust_pipeline_resourceSsgoo::<= \ )*L m"O45  \L9K,LQ,OO O A"l2ODE   </ + ,Q / <(:;A>MNr!c g}g}d}d}|jjD]}|j|rnz|jds(|j ddk7r|j dn|}|||k7r"|j g|j ||dj ||}t |}d} d} | |kr| |kr|| |k(r| dz } | |kr || |k(r| |k(rn|| } | dz } | } d} | |kr&|| | k7r|| |k7rd} n| dz } | |kr || | k7r| |k(rn@| smt| | D](} || D]}|jd| | || <| dz } *| |krd}g}g}t|D]M} |||| k7r%|j g|j || |dj|| || }Ot}|jjD]I}|jj|}|jr/|j|jKt |}t}t}t|D cgc]} g}} t|D]} t}t}|| D]w}|j D].}|j#|D]}||vs|j|0|j$D])}|j'|D]}|j|+y| dk(rg|| <n| dk(r ||z|z|| <n||z|| <t)j*|}t)j*|}t |}g}g}t|D cgc]} g}} d}g}t|D]} |j ||j |t || zdz |t || z }| |dz kr#|| jt-|| dz|j/} || |k7r<| j1t-|| t-|| } |j | |j | |D cgc] } t-| }!} d}"d}#tt |D]} || }$|$|k7s|"r t3d d}"| }# |# t3d y||#||#||#|!|#||#gScc} wcc} wcc} w) Ncpu op_devicerrr TFz'only one region of program can be heterzwarning: non heter program)rrarhas_attrrorxr3rC _set_attrr`r rrSrrrN input_namesrc output_namesrdr:deepcopyr0rrr))%rrops_list type_listpretype_cpurjcur_attrlrE type_heterstartvalidjmerged_ops_listmerged_type_list data_varsrrS inputs_pre outputs_pre in_from_prerrrctmprd start_listend_list send_listsum program_listprog recv_listfound heter_indexr;s% r"split_program_by_devicez Executor.split_program_by_devicefs_ &&(,,B$$R({{;'ww{+r1GGK(! ;#/OOB'$$X. ##B'-  N  !ea%IaLH4Qa%IaLH4Av"1J FAEEa%IaLJ6Q<8+!EQ a%IaLJ6 Av5!_"1+BLLj9&) ! Q %+!e6qA{cYq\1&&r* '' ! 5 B  & &x{ 3A,C E %%',,A&&(,,Q/C?? chh'-  U e #(8,8ar8 ,qAUFeG%a(^^E!xxg-"JJsO /,!ooF!yy0 C( 1. )Av!# Aa"- "9V!C A!,v!5 Av.J--0K%(   !&q*AR * qA   c " OOC#oa&8"99A= > 3q)* *C1q5y! ##DQU);$<===?D"h.--Q($y|*<##D)##D)'22kT!Wk 2 s+,-A #AH}CD .   . /;'%+&+&[)  o-0+"3s) S* S/9S4c d} d} |j`|jjdddk(rd} |jjdddk(rd} |jjdd rd } | t}|g}|g}t|t|k(sJt |t j } | r|j|} | s|jr$tj|j} nn|jr$tj|j} n>tj|j} | j|j| j|| r| j n|jr.tj|j j} ng|jr.tj|j j} n-tj|j j} | j|j |dkr`| r(| j#t|jd nG|j$dkr t'd | j#|j$n| j#|| j)|| j+||||| fS) Nr worker_classrHeterCpuWorkerr rgHeterXpuTrainer use_ps_gpuFT worker_placeszSYou should set thread num first, either in Datasetor in Executor.train_from_dataset)rfrr#r3r-r rrrur_create_trainerr,_set_thread_barrier_is_distributed _set_program_set_heter_infor _set_thread thread_numr4 _set_debug_set_fetch_var_and_info)rrrlr'threaddebugrr} print_periodis_heterrrrOrgs r"_prepare_trainerzExecutor._prepare_trainers    )!!%%nb9=MM!!%%i48II!!%%lE:! = NE  J  J:#j/111gx'?'?@ ..w7C$$(*::)),,(*:://)*::7;M;MN++G,C,CD   )'',$$(*::OO11,,(*::OO77)*::OO..   1 Q;##C(:(:?(K$LM##q("8 ##G$6$67    '5!'' J Mg~r!c  |jddl} | tdg} |jjj D] } | j s| j| "| jjjd}|jd|jd|jdg|j| nz|j`|jd}|jd}|dk7rd|jvrddl} | td g} |jjj D] } | j s| j| "| jjjd }|jd|jd|jdg|j| n | td t!j"|}t%j&}|j)|t%j*||_n | td |j/g}|jr|jd }|D]O}t1|t2r |j4}n|}||jjvs?|j|Qt7|jd g|d d|jd <|jd j9d}|j:D]F}|j<dk(s|j?dt$j@jBjDHd}|jG|||||||| \}}|jI||jK|j|j|jM|||jNdur+|jPjNrtSjTd|jW|jPjX|jduxs,|jZduxr|jZj]dd}|durA|j,j_|j`|jc||jd}n|g}tg|d|}|ji|}|S|j,j_|j`|jc||jd}|jk||n|jm|jd| x|jod}tq|| }|js|j,ju||jw|durV|j,jy|n:|j,ju||dur|j,jy||j{|j}|r:|jdj}|j}t|Sy)Nr(dataset should be None for pipeline modeFileInstantDatasetr Nonepipeline_stager is_fl_modez.dataset should be None for heter pipeline modeInMemoryDatasetz)dataset is need and should be initializedsection_programrTrrrTrrrop_rolerrlr'rrrr}rrrgF0dataset should call set_use_ps_gpu in PsGpu moder)Crurr4rrvaluesr:rxbaseDatasetFactorycreate_datasetset_batch_sizern set_filelist set_use_varr,r rr r3r5rr_prepare_to_runr-rrNrrfrarbrxrrrr _set_infer_gen_trainer_descrir proto_descrr_dynamic_adjust_before_trainrrfrinit_for_datasetrJ_descrlrrr ResetDatasetget_worker_scoperrrun_from_datasetstopr_dynamic_adjust_after_train_finish_to_runrget_fetch_listr>r/)rrrlr'ris_inferrrr}r fetch_handlerrrrSstage_idrrreal_fetch_list real_programrr main_blockrjrgreused_trainerrrDscope0 fetch_monitorarrrAs r"_run_from_datasetzExecutor._run_from_dataset7 s  , ""#MNNI++-2299;;;$$S)<kk002AA$G  " "1 %   q !  & *    *  ( ( 4223CDH!55mDK1}w'B'BB!**L !#I&335::AAC;;%,,S1 D%kk88:II)G**1-&&q)((&2'' 2?&C$55kBK A KK $%)]]1%5D ""#NOO!  "001BCL' i2%.^^N%.N!\%>%>%@%E%EE#**95 (8K--.?@*$& 8G ! !"3 4!../@AGGJJ nn77g%LL!77>>GG %J..!!%/  w 8$!!#  (**2%%gw%G    &7+=+=+H+H OON O,,W-?-?-J-JK 44D@   d * <""&&|U;  U "&&77LL'--/5'// ! 5gtZPI#66yA '#'#9#9#J#JLL'--/5'//$ '' 3CD --goo>  $%66q9F/ FM    !  " " 3 34D E    &&&667GH  " " 3 34D E&&&667GH++- ..)88:C'')GG$ $r!c jJ|Jdtd} |j| } | r| | Sddlfd}|}fd}|\}}|jd<d|j |||||| \}}|j ||j |jdur+|jjrtjd|j|jj|j}|jjj |||j"}|||g} | r|j%| | | S) Nrrcg}jjjD] }|js|j |"j j jd}|jd|jd|jdg|j||j|S)Nrr r) rrrr:rxrrrrrnrrr)rrSrlrrs r" _get_datasetz4Executor._prepare_pipeline_ctx.._get_dataset sI++-2299;;;$$S)<kk002AA$G  " "1 %   q !  & *    *  # # %Nr!cjd}g}D]O}t|tr |j}n|}||j j vs?|j |Qt|g|dd}|jd}|jD]F}|jdk(s|jdtjjjH||fS)NrrTrrrr)rur-rrNrrrxrrfrarbrxr rrr)rrrrrrjrrs r"_get_real_program_fetch_listzDExecutor._prepare_pipeline_ctx.._get_real_program_fetch_list s"001BCL O' i2%.^^N%.N!\%>%>%@%E%EE#**95 (/$*$& L&++A.J nn77g%LL!77>>GG % 0 0r!rrFr)rurrrrrrrrrrrrrrrrJrlr)rrrlr'rrrrr}rrrrDrrrrrrg trainer_descrrs ` ` @r"_prepare_pipeline_ctxzExecutor._prepare_pipeline_ctx sy$$000J JJ1'4L !!), J . 1:)E(F% o3?/0 ..!!%/  w 8$!!#    &7+=+=+H+H OON O,,W-?-?-J-JK}} 11BB LL,w o'78     3 / r!rc d|vr|dnd}ttjdd}tjddjd} t | } d|vs d |vsJd d |vr!CMM"MN)0MM"5q9MM/26, *&h&P$y0#y'AA"/2;?1D\[\D&)3!"56!;WVW;)/)B&`cfd"'Ig +:I' (   #'2Y&>Ik "B " !!  LL      %  gFs G!c t|||}|j|} |j|} |j|} |jd} | t } |j || | Xg} d| vrR| drMtt| dD]!} | j| j#|j|| | E|jsJd|gn|}|}d|jvr|jd}t|||||} | jd}|jD]F}|jdk(s|j!d t"j$j&j(H|j+|| |jd} d | vr| d d}t-d |j/|j1}|j3||| }|j5|| d d }t-d|j/|j1}|j7|||}|jd}|jD]F}|jdk(s|j!d t"j$j&j(H|j5|g}d| vr@| dr;tt| dD]!}|j| j#|j9|| | | | ||r|j;| ||| ddlm}tA|dr|jB}tE||sJd|}|jGjH|jJ}tMjN|gjQtS|jT}t#jV| |jJ}|jY||jZ|j\j_|d| vreg}| D]\}g} | dD]<}!d} t#jV||!}|r ta|}|s,| j|>| sL|j| ^|S|r:| jc|ji}#|#jk}$ta|$Sy#|jc|!}"|"je}|r ta|}n tg|}YxYw)Nrinference_generationrz2program should have _pipeline_opt to start carrierrrrrrrzInserting feed ops for task)rrTrrvzInserting fetch ops for task)rrr)rr'rrrr$rhr&r)6rrrrrur#rrCrrx new_scoperrrfrarbrxr rrrrr)task_id get_program _add_feed_ops set_program_add_fetch_opsrrr0r%rsrhr-rrr1r6r7rrrQr2r r rr=r/rget_lod_tensor_arrayr0rr>)%rrrTrrrrrrDcached_program cached_scopemicro_cached_scopesrr  real_feedrrrj feed_task feed_program fetch_task fetch_programrrEr%rhrLrMr r9 result_listr'scope_result_listrPrSrrAs% r"r*z"Executor._run_using_fleet_executor s2'4L 00;,,Y7 "::9E))+6  '>L  ! !)\ :  &"$ &)345s9-@#ABCA'..|/E/E/GHD,,Y8KL  !%% DC D%"lI"L G$9$99&445FG 0$%+- N(--a0J nn77g%LL!77>>GG %  # #I~ >--k:I)#&g.q1 3Y5F5F5HI(446 #11(""/ 2  %%l3'w/3 4j6H6H6JK * 6 6 8 $ 3 3))#1!4! +003 $..Bww') % ;;BBKK )&&}5! &)345s9-@#ABCA$++L,B,B,DED  0 0&"#!4)A 1   OOND- N3 7N +"//LlK8 O:O O8#~H))+001G1GHF88XJ'..}V\\/JKD--l44F JJtTZZ (   + ) # K,$&!(5G!F 2!%!9!9%!I'%-f%5F)008 6"%&&'89)-*  ''7FFHC'')GG$ $'2#nnW5!$!9!9!;'%-f%5F%)&\Fs ,#R::rrlr{rrrr) rrrTrrrrrErNrs r"rzExecutor._add_feed_opsN smmo "//1 L-- -#'' 6H#.."\\))88 /H",mD$T?4''-&**40C ,,# #hZ0!&$aj -MMc+r!c|j}|j}||jvr|j|}n6|j |t j jjd}|rd}nd}t||||s\t|D]N\} } t| ttfsJd| dt| |j|d| gid|gid | i P|S) NTrrrrrr^r_rnr)rrrrSrr r=r>rrvr{r-rr|rbr) rrrrrrrrrrrErSs r"rzExecutor._add_fetch_opso smmo "//1 \.. .$((8I$//#\\))44 0I !HH# *nh $J/3!(CB/s#d3i[AB&&!#<"YK0 !* ' 0r!c|j}|j}t|j}t t |D]0}|j|j |k(s |j|2|Sr%)rrr3rareversedrCrb _remove_op)rr fetch_op_namerrop_numrus r"_remove_fetch_opszExecutor._remove_fetch_ops smmmo "//1 \%%&E&M*C$))]:'',+r!c |j||||||||| | | \}} } ddlm}t|dr|j}t ||sJd|}|j j|j}tj|gjt|j}tj||j}|j!||j"|j$j'| | s|j$j)| | r:|j+dj-}|j/}t1|Sy)Nrr$rhr&r)rr0r%rsrhr-rrr1r6r7rrrQr r2r r rrrrrr>r/)rrrlr'rrrrr}rrrrrr%rhrLrMr r9rrAs r"r+zExecutor._run_pipeline sD483M3M            4 0 0 4 7N +"//LlK8 O:O O8#~H))+001G1GHF88XJ'..}V\\/JKD--e\5K5KLF JJtTZZ ( //0@A  " " 2 23C D ..)88:C'')GG$ $r!c 6|j||||d||||| S)ak Infer from a pre-defined Dataset. Dataset is defined in paddle.base.dataset. Given a program, either a program or compiled program, infer_from_dataset will consume all data samples in dataset. Input scope can be given by users. By default, scope is global_scope(). The total number of thread run in training is `thread`. Thread number used in training will be minimum value of threadnum in Dataset and the value of thread in this interface. Debug can be set so that executor will display Run-Time for all operators and the throughputs of current infer task. The document of infer_from_dataset is almost the same as train_from_dataset, except that in distributed training, push gradients will be disabled in infer_from_dataset. infer_from_dataset() can be used for evaluation in multi-threadvery easily. Args: program(Program|CompiledProgram): the program that needs to be run, if not provided, then default_main_program (not compiled) will be used. dataset(paddle.base.Dataset): dataset created outside this function, a user should provide a well-defined dataset before calling this function. Please check the document of Dataset if needed. default is None scope(Scope): the scope used to run this program, you can switch it to different scope for each run. default is global_scope thread(int): number of thread a user wants to run in this function. Default is 0, which means using thread num of dataset debug(bool): whether a user wants to run infer_from_dataset, default is False fetch_list(Tensor List): fetch Tensor list, each Tensor will be printed during training, default is None fetch_info(String List): print information for each Tensor, default is None print_period(int): the number of mini-batches for each print, default is 100 fetch_handler(FetchHandler): a user define class for fetch output. Returns: None Examples: .. code-block:: python >>> import paddle >>> paddle.enable_static() >>> place = paddle.CPUPlace() # you can set place = paddle.CUDAPlace(0) to use gpu >>> exe = paddle.static.Executor(place) >>> x = paddle.static.data(name="x", shape=[None, 10, 10], dtype="int64") >>> y = paddle.static.data(name="y", shape=[None, 1], dtype="int64", lod_level=1) >>> dataset = paddle.base.DatasetFactory().create_dataset() >>> dataset.set_use_var([x, y]) >>> dataset.set_thread(1) >>> # you should set your own filelist, e.g. filelist = ["dataA.txt"] >>> filelist = [] >>> dataset.set_filelist(filelist) >>> exe.run(paddle.static.default_startup_program()) >>> exe.infer_from_dataset(program=paddle.static.default_main_program(), ... dataset=dataset) Tr rrrlr'rrrr}rrs r"infer_from_datasetzExecutor.infer_from_dataset s6D%%            r!c H|j|d|d||||\}}|jd|j|j|||jj |j |j|d} |jj| | S)Nr rFr) rrrrirrrJrr) rrr'rrr}rrrgrs r"start_heter_trainerzExecutor.start_heter_trainer# s..!!%/  w 5!!!# gw?11BB LL'--/5$  //0@A r!c 6|j||||d||||| S)a Train from a pre-defined Dataset. Dataset is defined in paddle.base.dataset. Given a program, either a program or compiled program, train_from_dataset will consume all data samples in dataset. Input scope can be given by users. By default, scope is global_scope(). The total number of thread run in training is `thread`. Thread number used in training will be minimum value of threadnum in Dataset and the value of thread in this interface. Debug can be set so that executor will display Run-Time for all operators and the throughputs of current training task. Note: train_from_dataset will destroy all resources created within executor for each run. Args: program(Program|CompiledProgram): the program that needs to be run, if not provided, then default_main_program (not compiled) will be used. dataset(paddle.base.Dataset): dataset created outside this function, a user should provide a well-defined dataset before calling this function. Please check the document of Dataset if needed. scope(Scope): the scope used to run this program, you can switch it to different scope for each run. default is global_scope thread(int): number of thread a user wants to run in this function. Default is 0, which means using thread num of dataset debug(bool): whether a user wants to run train_from_dataset fetch_list(Tensor List): fetch Tensor list, each variable will be printed during training fetch_info(String List): print information for each Tensor, its length should be equal to fetch_list print_period(int): the number of mini-batches for each print, default is 100 fetch_handler(FetchHandler): a user define class for fetch output. Returns: None Examples: .. code-block:: python >>> import paddle >>> paddle.enable_static() >>> place = paddle.CPUPlace() # you can set place = paddle.CUDAPlace(0) to use gpu >>> exe = paddle.static.Executor(place) >>> x = paddle.static.data(name="x", shape=[None, 10, 10], dtype="int64") >>> y = paddle.static.data(name="y", shape=[None, 1], dtype="int64", lod_level=1) >>> dataset = paddle.base.DatasetFactory().create_dataset() >>> dataset.set_use_var([x, y]) >>> dataset.set_thread(1) >>> # you should set your own filelist, e.g. filelist = ["dataA.txt"] >>> filelist = [] >>> dataset.set_filelist(filelist) >>> exe.run(paddle.static.default_startup_program()) >>> exe.train_from_dataset(program=paddle.static.default_main_program(), ... dataset=dataset) Frrs r"train_from_datasetzExecutor.train_from_datasetO s6B%%            r!r%)NNN) NNNrTrNTFF)NN)NNNrFNNd) NNNrFFNNrN) NNNrFFNNrNF)NNrTrNFTFr) NNNrFNNrN)NNFNNrN)4r,r-r.__doc__rrrrrrrrrrrrrrrr0rrrrrr classmethodrrrrr r=rrr>r(rirrrrrrrr*rrr r+rrrr r!r"rrbs=~!2FA %><@@9G@MF-53AtAC 6p<4*4*l?C;;z,,\ +6 $ qfer's`   54&   8@<  $**,%%)),''T+\$ F.b/d%R7>*\7D%RLQ>B'(7 *D8&  (,<5p,    8++\F F Ry  y  r!