\;j2ddlZddlmZddlmZGddZGddeZGd d eZdd Z Gd d eZ GddeZ y)N)core)randpermceZdZdZddZdZy)Samplera8 An abstract class to encapsulate methods and behaviors of samplers. All sampler used by :code:`paddle.io.BatchSampler` should be a subclass of :code:`paddle.io.Sampler`, BatchSampler subclasses should implement following methods: :code:`__iter__`: return sample index iterably, which iterate over indices of dataset elements :code:`__len__`: the number of sample in :attr:`data_source` Args: data_source(Dataset, optional): this could be an instance of :code:`paddle.io.Dataset` other Python object which implemented :code:`__len__` for Sampler to get indices as the range of :attr:`dataset` length. Default None. Returns: Sampler: an iterable object for sample indices iterating Examples: .. code-block:: python >>> from paddle.io import Dataset, Sampler >>> class RandomDataset(Dataset): ... def __init__(self, num_samples): ... self.num_samples = num_samples ... ... def __getitem__(self, idx): ... image = np.random.random([784]).astype('float32') ... label = np.random.randint(0, 9, (1, )).astype('int64') ... return image, label ... ... def __len__(self): ... return self.num_samples ... >>> class MySampler(Sampler): ... def __init__(self, data_source): ... self.data_source = data_source ... ... def __iter__(self): ... return iter(range(len(self.data_source))) ... ... def __len__(self): ... return len(self.data_source) ... >>> sampler = MySampler(data_source=RandomDataset(100)) >>> for index in sampler: ... print(index) 0 1 2 ... 99 see `paddle.io.BatchSampler` see `paddle.io.DataLoader` Nc||_yN data_sourceselfr s eG:\00. PROJECTS\API\Inventory\templateJSON\kerjaOCR\Lib\site-packages\paddle/io/dataloader/sampler.py__init__zSampler.__init__W &ctr )NotImplementedErrorr s r__iter__zSampler.__iter__Zs!!rr )__name__ __module__ __qualname____doc__rrrrrrs?B'"rrc"eZdZdZdZdZdZy)SequenceSamplera Iterate samples sequentially, yield :code:`0, 1, 2, ..., len(data_source) -1` generally, Args: data_source(Dataset): dataset to sample, this could be an instance of :code:`paddle.io.Dataset` other Python object which implemented :code:`__len__`. Returns: Sampler: a Sampler yield sample index sequentially Examples: .. code-block:: python >>> from paddle.io import Dataset, SequenceSampler >>> class RandomDataset(Dataset): ... def __init__(self, num_samples): ... self.num_samples = num_samples ... ... def __getitem__(self, idx): ... image = np.random.random([784]).astype('float32') ... label = np.random.randint(0, 9, (1, )).astype('int64') ... return image, label ... ... def __len__(self): ... return self.num_samples ... >>> sampler = SequenceSampler(data_source=RandomDataset(100)) >>> for index in sampler: ... print(index) 0 1 2 ... 99 see `paddle.io.Sampler` c||_yr r r s rrzSequenceSampler.__init__rrcPttt|jSr )iterrangelenr rs rrzSequenceSampler.__iter__sE#d../011rc,t|jSr )r!r rs r__len__zSequenceSampler.__len__s4##$$rNrrrrrrr#rrrrras)V'2%rrc6eZdZdZ ddZedZdZdZy) RandomSamplera Iterate samples randomly, yield shuffled indices, if :attr:`replacement=False`, yield shuffled indices of the whole data souce, if :attr:`replacement=True`, :attr:`num_samples` can set to specify the sample number to draw. Args: data_source(Dataset): dataset to sample, this could be an instance of :ref:`api_paddle_io_Dataset` or :ref:`api_paddle_io_IterableDataset` or other Python object which implemented :code:`__len__` to get indices as the range of :code:`dataset` length. Default None. replacement(bool, optional): If False, sample the whole dataset, If True, set :attr:`num_samples` for how many samples to draw. Default False. num_samples(int, optional): set sample number to draw if :attr:`replacement` is True, then it will take samples according to the number you set. Default None, disabled. generator(Generator, optional): specify a generator to sample the :code:`data_source`. Default None, disabled. Returns: RandomSampler: a Sampler yield sample index randomly. Examples: .. code-block:: python >>> import numpy as np >>> from paddle.io import Dataset, RandomSampler >>> np.random.seed(2023) >>> class RandomDataset(Dataset): ... def __init__(self, num_samples): ... self.num_samples = num_samples ... ... def __getitem__(self, idx): ... image = np.random.random([784]).astype('float32') ... label = np.random.randint(0, 9, (1, )).astype('int64') ... return image, label ... ... def __len__(self): ... return self.num_samples ... >>> sampler = RandomSampler(data_source=RandomDataset(100)) >>> for index in sampler: ... print(index) 56 12 68 ... 87 NcT||_||_||_||_t |jt st d|j|j |s tdt |jtr|jdkrtd|jy)Nz:expect boolean value for replacement, but got replacement=z>num_samples should not be specified while replacement is Falserz>num_samples should be a positive integer, but got num_samples=) r replacement _num_samples generator isinstancebool TypeError ValueError num_samplesint)r r r(r/r*s rrzRandomSampler.__init__s'&'"$**D1#//02     (P $**C0D4D4D4I''+'7'7&8: 5Jrc\|jt|jS|jSr )r)r!r rs rr/zRandomSampler.num_sampless+    $t''( (   rc#6Kt|j}|jr5t|jD]} t |j}|y|jrWtjjtj||jdjD]}|ytjjtj||djD]}|y#t $rYywxYww)NT)replaceF) r!r r*r r/next StopIterationr(nprandomchoicearangetolist)r niindexs rrzRandomSampler.__iter__s   ! >>4++, 0E -YY--IIaL$"2"2D.&(E K  YY--IIaL!U.&(E K%s):DD B7D DDDDc|jSr )r/rs rr#zRandomSampler.__len__sr)FNN) rrrrrpropertyr/rr#rrrr&r&s2/dKO2!!  * rr&c*t|tjr|j}t|tt frt j|}t|t jsJdt|jdksJd|jd|jdf}t j|dk\sJdt j|t jk(rJdt j|t jk(rJdt j |dkDd }t j|d kDsJd |st j||k\sJd ||j!d z }g}t#|jd D]E}t j$j'|jd ||||}|j)|Gt j|S)Nz=weights should be paddle.Tensor, numpy.ndarray, list or tuplez$weights should be a 1-D or 2-D arraygz weights should be positive valuezweights shoule not be INFzweights shoule not be NaN)axisrz#weights should have positive valueszUweights positive value number should not less than num_samples when replacement=False)r+r LoDTensornumpylisttupler6arrayndarrayr!shapereshapeallanyinfnansumr r7r8append)weightsr/r( non_zerosretsr<rets r_weighted_samplerWs'4>>*--/'D%=)((7#  GFG  w}}  "J$JJ "oor7==#456G 66'S. !E#EE !vvg'(E*EE (vvg'(E*EE (w}1-I 66)a- G"GG vvi;./  ; /   ++G D 7==# $ii MM! k;   C % 88D>rc$eZdZdZddZdZdZy)WeightedRandomSampleraG Random sample with given weights (probabilities), sampe index will be in range [0, len(weights) - 1], if :attr:`replacement` is True, index can be sampled multiple times. Args: weights(numpy.ndarray|paddle.Tensor|list|tuple): sequence of weights, should be numpy array, paddle.Tensor, list or tuple num_samples(int): set sample number to draw from sampler. replacement(bool): Whether to draw sample with replacements, default True Returns: Sampler: a Sampler yield sample index randomly by given weights Examples: .. code-block:: python >>> import numpy as np >>> from paddle.io import WeightedRandomSampler >>> np.random.seed(2023) >>> sampler = WeightedRandomSampler( ... weights=[0.1, 0.3, 0.5, 0.7, 0.2], ... num_samples=5, ... replacement=True ... ) >>> for index in sampler: ... print(index) 2 4 3 1 1 ct|tr|dkr tdt|ts td||_||_||_y)Nrz(num_samples should be a positive integerz%replacement should be a boolean value)r+r0r.r,rSr/r()r rSr/r(s rrzWeightedRandomSampler.__init__FsL+s+{a/?GH H+t,DE E &&rct|j|j|j}t |j dj SNrB)rWrSr/r(rrLr:)r idxss rrzWeightedRandomSampler.__iter__OsB LL$**D,<,< DLL$++-..rctj|jj|jjdz}|j|zSr\)r6prodrSrKr/)r muls rr#zWeightedRandomSampler.__len__Us?ggdll(()T\\-?-?-CC#%%rNTr$rrrrYrY!s"H'/ &rrYc(eZdZdZdZdZdefdZy)SubsetRandomSamplera Randomly sample elements from a given list of indices, without replacement. Args: indices (sequence): a sequence of indices Examples: .. code-block:: python >>> import paddle >>> from paddle.io import SubsetRandomSampler >>> paddle.seed(2023) >>> sampler = SubsetRandomSampler(indices=[1, 3, 5, 7, 9]) >>> for index in sampler: ... print(index) 9 3 7 5 1 cDt|dk(r td||_y)NrzHThe length of `indices` in SubsetRandomSampler should be greater than 0.)r!r.indices)r res rrzSubsetRandomSampler.__init__us' w<1 Z  rc#tKtt|jD]}|j|ywr )rr!re)r r<s rrzSubsetRandomSampler.__iter__|s,#dll+,A,,q/ !-s68returnc,t|jSr )r!rers rr#zSubsetRandomSampler.__len__s4<<  rN)rrrrrrr0r#rrrrcrcZs4"!!rrcra) rFr6 frameworkrtensorrrrr&rWrYrcrrrrksXF"F"X3%g3%lg Gg T@6&G6&r'!''!r