\;jUZddlZddlZddlmZddlmZddlmZddl m Z m Z ddZ d dZ y) N) framework)stream)convert_object_to_tensorconvert_tensor_to_objectc4tj|||||S)ak Scatter a tensor to all participators. As shown below, one process is started with a GPU and the source of the scatter is GPU0. Through scatter operator, the data in GPU0 will be sent to all GPUs averagely. .. image:: https://githubraw.cdn.bcebos.com/PaddlePaddle/docs/develop/docs/api/paddle/distributed/img/scatter.png :width: 800 :alt: scatter :align: center Args: tensor (Tensor): The output Tensor. Its data type should be float16, float32, float64, int32, int64, int8, uint8, bool or bfloat16. tensor_list (list|tuple): A list/tuple of Tensors to scatter. Every element in the list must be a Tensor whose data type should be float16, float32, float64, int32, int64, int8, uint8, bool or bfloat16. Default value is None. src (int): The source rank id. Default value is 0. group (Group, optional): The group instance return by new_group or None for global default group. sync_op (bool, optional): Whether this op is a sync op. The default value is True. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> if dist.get_rank() == 0: ... data1 = paddle.to_tensor([7, 8, 9]) ... data2 = paddle.to_tensor([10, 11, 12]) ... dist.scatter(data1, src=1) >>> else: ... data1 = paddle.to_tensor([1, 2, 3]) ... data2 = paddle.to_tensor([4, 5, 6]) ... dist.scatter(data1, tensor_list=[data1, data2], src=1) >>> print(data1, data2) >>> # [1, 2, 3] [10, 11, 12] (2 GPUs, out for rank 0) >>> # [4, 5, 6] [4, 5, 6] (2 GPUs, out for rank 1) )rscatter)tensor tensor_listsrcgroupsync_ops qG:\00. PROJECTS\API\Inventory\templateJSON\kerjaOCR\Lib\site-packages\paddle/distributed/communication/scatter.pyr r sV >>&+sE7 CCcVtjsJdtj}g}g}||k(rC|D]2}t |\}} |j ||j | 4t |} ntjgd} tj| |t| j} g} |D]O} | j}tj|| g}tj |}| j |Qtj| gd}t#|||k(r| nd||tjgd}t#|||k(r|nd|||j%|j t'||jy)a Scatter picklable objects from the source to all others. Similiar to scatter(), but python object can be passed in. Args: out_object_list (list): The list of objects to store the scattered objects. in_object_list (list): The list of objects to scatter. Only objects on the src rank will be scattered. src (int): The source rank in global view. group (Group): The group instance return by new_group or None for global default group. Returns: None. Warning: This API only supports the dygraph mode. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> out_object_list = [] >>> if dist.get_rank() == 0: ... in_object_list = [{'foo': [1, 2, 3]}, {'foo': [4, 5, 6]}] >>> else: ... in_object_list = [{'bar': [1, 2, 3]}, {'bar': [4, 5, 6]}] >>> dist.scatter_object_list(out_object_list, in_object_list, src=1) >>> print(out_object_list) >>> # [{'bar': [1, 2, 3]}] (2 GPUs, out for rank 0) >>> # [{'bar': [4, 5, 6]}] (2 GPUs, out for rank 1) z6scatter_object_list doesn't support static graph mode.int64)dtypeuint8N)rin_dynamic_modedistget_rankrappendmaxpaddleemptyr broadcastintitemnumpynpresize to_tensorr clearr)out_object_listin_object_listr r rankin_obj_tensors in_obj_sizesobj obj_tensorobj_sizemax_obj_size_tensor max_obj_sizein_tensor_listr numpy_data in_tensor out_tensorout_tensor_sizes rscatter_object_listr3JsJ !!#@?@# ==?DNL s{!C#;C#@ J  ! !* -    )"",/$ll2W= (#.*//12LN \\^ YYzL>: $$Z0 i( ! |nGr9s- !3 +D^8<Gr