\;j/ x d dl Z d dlZd dlZd dlmZ d dlZd dlmZ d dl m
Z
g Z G d de
Z G d de Z
y) N)Image)_check_exists_and_download)Datasetc x e Zd ZdZdZdZedz ZdZedz ZdZ edz Z
d Z ed
z Z d Z
dd
ZddZd Zd Zy )MNISTa4
Implementation of `MNIST `_ dataset.
Args:
image_path (str, optional): Path to image file, can be set None if
:attr:`download` is True. Default: None, default data path: ~/.cache/paddle/dataset/mnist.
label_path (str, optional): Path to label file, can be set None if
:attr:`download` is True. Default: None, default data path: ~/.cache/paddle/dataset/mnist.
mode (str, optional): Either train or test mode. Default 'train'.
transform (Callable, optional): Transform to perform on image, None for no transform. Default: None.
download (bool, optional): Download dataset automatically if
:attr:`image_path` :attr:`label_path` is not set. Default: True.
backend (str, optional): Specifies which type of image to be returned:
PIL.Image or numpy.ndarray. Should be one of {'pil', 'cv2'}.
If this option is not set, will get backend from :ref:`paddle.vision.get_image_backend `,
default backend is 'pil'. Default: None.
Returns:
:ref:`api_paddle_io_Dataset`. An instance of MNIST dataset.
Examples:
.. code-block:: python
>>> import itertools
>>> import paddle.vision.transforms as T
>>> from paddle.vision.datasets import MNIST
>>> mnist = MNIST()
>>> print(len(mnist))
60000
>>> for i in range(5): # only show first 5 images
... img, label = mnist[i]
... # do something with img and label
... print(type(img), img.size, label)
... # (28, 28) [5]
>>> transform = T.Compose(
... [
... T.ToTensor(),
... T.Normalize(
... mean=[127.5],
... std=[127.5],
... ),
... ]
... )
>>> mnist_test = MNIST(
... mode="test",
... transform=transform, # apply transform to every image
... backend="cv2", # use OpenCV as image transform backend
... )
>>> print(len(mnist_test))
10000
>>> for img, label in itertools.islice(iter(mnist_test), 5): # only show first 5 images
... # do something with img and label
... print(type(img), img.shape, label)
... # [1, 28, 28] [7]
mnistz$https://dataset.bj.bcebos.com/mnist/t10k-images-idx3-ubyte.gz 9fb629c4189551a2d022fa330f9573f3t10k-labels-idx1-ubyte.gz ec29112dd5afa0611ce80d1b7f02629ctrain-images-idx3-ubyte.gz f68b3c2dcbeaaa9fbdd348bbdeb94873train-labels-idx1-ubyte.gz d53e105ee54ea40749a09fcbcd1e9432Nc 0 |j dv s
J d| | t j j }|dvrt d| || _ |j | _ || _ | j a|sJ d |dk( r | j n | j }|dk( r | j n | j }t |||| j | | _ || _ | j u|sJ d | j dk( r | j n | j } | j dk( r | j" n | j$ }
t || |
| j | | _ || _ | j) t j* | _ y )N)traintestz*mode should be 'train' or 'test', but got )pilcv2z4Expected backend are one of ['pil', 'cv2'], but got z?image_path is not set and downloading automatically is disabledr z?label_path is not set and downloading automatically is disabled)lowerpaddlevisionget_image_backend
ValueErrorbackendmode
image_pathTRAIN_IMAGE_URLTEST_IMAGE_URLTRAIN_IMAGE_MD5TEST_IMAGE_MD5r NAME
label_pathTRAIN_LABEL_URLTEST_LABEL_URLTRAIN_LABEL_MD5TEST_LABEL_MD5 transform_parse_datasetget_default_dtypedtype) selfr r# r r( downloadr image_url image_md5 label_url label_md5s eG:\00. PROJECTS\API\Inventory\templateJSON\kerjaOCR\Lib\site-packages\paddle/vision/datasets/mnist.py__init__zMNIST.__init__h s zz|
? 8v > ?
? mm557G . ( FwiP
JJL $ ?? "
QP
Q )-$$T=P=P
)-$$T=P=P
9Iy$))X DO % ?? "
QP
Q 99' $$((
99' $$((
9Iy$))X DO #
--/
c z g | _ g | _ t j | j d 5 }|j }t j | j d 5 }|j }d}d}d}t j ||| \ } }
} } |t j | z
}d}
d}t j |||
\ }}|
t j | z
}
||k\ rn*dt | z dz }t j |||
}|
t j | z
}
||z
}dt || z | z z dz }t j ||| }t j ||| | z f j d }|t j | z
}t | D ]e }| j j ||d d f | j j t j || g j d g 1 d d d d d d y # 1 sw Y xY w# 1 sw Y y xY w) Nrbr z>IIIIz>II>Bfloat32int64)imageslabelsgzipGzipFiler readr# struct unpack_fromcalcsizestrnpreshapeastyperangeappendarray)r, buffer_size
image_fileimg_buf
label_filelab_buf
step_label
offset_imgmagic_byte_img magic_img image_numrowscols
offset_labmagic_byte_lab magic_lab label_num fmt_labelr<
fmt_images images_tempr; is r2 r) zMNIST._parse_dataset s
]]4??D
1Z oo'G t5$//+
")393E3E"GZ40 9dD foon ==
!&'-'9'9"GZ($ 9 foon ==
!Y. #c+&6 6 > %NN5)E <<5 %,,w// /||DJJ' g)>>>r4 c , t | j S )N)lenr< )r, s r2 __len__z
MNIST.__len__ s 4;; r4 )NNr NTN)d )__name__
__module__ __qualname____doc__r"
URL_PREFIXr r! r% r' r r r$ r& r3 r) rf ri r4 r2 r r s~ >@ D7J "==N7N "==N7N #??O8O #??O8O
>0@0d
? r4 r c P e Zd ZdZdZdZedz ZdZedz ZdZ edz Z
d Z ed
z Z d Z
y )
FashionMNISTa
Implementation of `Fashion-MNIST `_ dataset.
Args:
image_path (str, optional): Path to image file, can be set None if
:attr:`download` is True. Default: None, default data path: ~/.cache/paddle/dataset/fashion-mnist.
label_path (str, optional): Path to label file, can be set None if
:attr:`download` is True. Default: None, default data path: ~/.cache/paddle/dataset/fashion-mnist.
mode (str, optional): Either train or test mode. Default 'train'.
transform (Callable, optional): Transform to perform on image, None for no transform. Default: None.
download (bool, optional): Whether to download dataset automatically if
:attr:`image_path` :attr:`label_path` is not set. Default: True.
backend (str, optional): Specifies which type of image to be returned:
PIL.Image or numpy.ndarray. Should be one of {'pil', 'cv2'}.
If this option is not set, will get backend from :ref:`paddle.vision.get_image_backend `,
default backend is 'pil'. Default: None.
Returns:
:ref:`api_paddle_io_Dataset`. An instance of FashionMNIST dataset.
Examples:
.. code-block:: python
>>> import itertools
>>> import paddle.vision.transforms as T
>>> from paddle.vision.datasets import FashionMNIST
>>> fashion_mnist = FashionMNIST()
>>> print(len(fashion_mnist))
60000
>>> for i in range(5): # only show first 5 images
... img, label = fashion_mnist[i]
... # do something with img and label
... print(type(img), img.size, label)
... # (28, 28) [9]
>>> transform = T.Compose(
... [
... T.ToTensor(),
... T.Normalize(
... mean=[127.5],
... std=[127.5],
... ),
... ]
... )
>>> fashion_mnist_test = FashionMNIST(
... mode="test",
... transform=transform, # apply transform to every image
... backend="cv2", # use OpenCV as image transform backend
... )
>>> print(len(fashion_mnist_test))
10000
>>> for img, label in itertools.islice(iter(fashion_mnist_test), 5): # only show first 5 images
... # do something with img and label
... print(type(img), img.shape, label)
... # [1, 28, 28] [9]
z
fashion-mnistz,https://dataset.bj.bcebos.com/fashion_mnist/r bef4ecab320f06d8554ea6380940ec79r bb300cfdad3c16e7a12a480ee83cd310r
8d4fb7e6c68d591d4c3dfef9ec88bf0dr 25c81989df183df01b3e8a0aad5dffbeN)rk rl rm rn r" ro r r! r% r' r r r$ r& rp r4 r2 rr rr sT >@ D?J "==N7N "==N7N #??O8O #??O8Or4 rr )r= r@ numpyrD PILr r paddle.dataset.commonr paddle.ior __all__r rr rp r4 r2 r| s>
<
N G N bJ95 J9r4