\;j-<ddlZddlZddlmZddlZddlZddlZddlZddl Z ddl Z ddl m Z ddl Z e jeZdgZej"dZdZdZdd Zd Zd Z dd ZdZd eiZddZddZdZdZddZdZ dZ!y#GddZ YmxYw)N)tqdmc&eZdZddZdZdZdZy)rNc ||_d|_y)Nr)totaln)selfrs ^G:\00. PROJECTS\API\Inventory\templateJSON\kerjaOCR\Lib\site-packages\paddle/utils/download.py__init__z tqdm.__init__sDJDFcn|xj|z c_|j/tjj d|jddnGtjj dd|jzt |jz ddtjj y)N z.1fz bytesd%)rrsysstderrwritefloatflush)rrs r updatez tqdm.update#s~ FFaKFzz!   2dffS\!89   2cDFFlU4::5F&Fs%K1!MN JJ   r c|SN)rs r __enter__ztqdm.__enter__+sKr cBtjjdy)N )rrr)rexc_typeexc_valexc_tbs r __exit__z tqdm.__exit__.s JJ  T "r r)__name__ __module__ __qualname__r rrrrr r rrs    #r rget_weights_path_from_urlz~/.cache/paddle/hapi/weightscJ|jdxs|jdS)zR Whether path is URL. Args: path (string): URL string or not. zhttp://zhttps://) startswith)paths r is_urlr(=s! ??9 % D)DDr c(t|t|}|S)a:Get weights path from WEIGHT_HOME, if not exists, download it from url. Args: url (str): download url md5sum (str): md5 sum of download package Returns: str: a local path to save downloaded weights. Examples: .. code-block:: python >>> from paddle.utils.download import get_weights_path_from_url >>> resnet18_pretrained_weight_url = 'https://paddle-hapi.bj.bcebos.com/models/resnet18.pdparams' >>> local_weight_path = get_weights_path_from_url(resnet18_pretrained_weight_url) )get_path_from_url WEIGHTS_HOME)urlmd5sumr's r r#r#Fs( S, 7D Kr cbtj|d}|}tj||S)N)ospsplitjoin)r,root_dirfnamefpaths r _map_pathr6^s* IIcN2 E E 88He $$r c|jt}t}|D]=}|jdd}||vr|j||j|?tj d||S)N:rzunique_endpoints )sortsetr1addloggerinfo)trainer_endpointsipsunique_endpointsendpointips r _get_unique_endpointsrCes{ %Cu% ^^C  # 9   X& &  KK#$4#567 r getcddlm}t|s Jd|dt||}t |j dd}t j|r'|r%t||rtjd|nw|j|vrt||||}nTtjj|s5tj dtjj|s5|j|vr7|r5t#j$|st'j(|r t+|}|S) aDownload from given url to root_dir. if file or directory specified by url is exists under root_dir, return the path directly, otherwise download from url and decompress it, return the path. Args: url (str): download url root_dir (str): root dir for downloading, it should be WEIGHTS_HOME or DATASET_HOME md5sum (str): md5 sum of download package decompress (bool): decompress zip or tar file. Default is `True` method (str): which download method to use. Support `wget` and `get`. Default is `get`. Returns: str: a local path to save downloaded models & weights & datasets. r) ParallelEnvzdownloading from z not a urlNzFound )method)paddle.distributedrFr(r6rCr>r0exists _md5checkr<r=current_endpoint _downloadosr'timesleeptarfile is_tarfilezipfile is_zipfile _decompress) r,r3r- check_exist decompressrGrFfullpathr@s r r*r*ts(/ #;;+C5 ;;;h'H-[]-L-LQ-OP zz( (F0K fXJ'( = ) )-= = hvFHggnnX. 1 ggnnX.}%%)99    x (G,>,>x,H"8,H Or c >tj|} tjd|dd5}|jdk7rt d|d|jd|dz}|j jd }t|d 5}|r_tt|d zd z 5}|jd D]$}|j||jd& dddn+|jd D]}|s|j|dddtj|||cdddS#1swY2xYw#1swY6xYw#1swYyxYw#t $r1} t"j%d|d|dt'| Yd} ~ yd} ~ wwxYw)NGETT)timeoutfollow_redirectszDownloading from z failed with code !_tmpzcontent-lengthwbii)r) chunk_sizerH Downloading  from z failed with exception F)r0basenamehttpxstream status_code RuntimeErrorheadersrDopenrint iter_bytesrrshutilmove Exceptionr<r=str) r,fullnamer4req tmp_fullname total_sizefpbarchunkes r _get_downloadrys LL "E \\ 3t #%"'u,>'q* $f,L)9:JlD)QS_t%;$DE%(^^t^%DEGGEN KKN&EFE "%4!@ GGEN"A* KK h /-  FE*)  0  5'u,CCF8 L  skE"AEE &:D>! E E E4 E">E E  E EEE"E"" F+'FFcZ|tvs Jd|dtj|stj|tj |d}tj ||}d}tjd|d|tj|r t||stjd|d||tkr|d z }ntd |d t|||stjd tj|stt||s|S) z Download from url, save to path. url (str): download url path (str): download to given path md5sum (str): md5 sum of download package method (str): which download method to use. Support `wget` and `get`. Default is `get`. z make sure `z ` implementedr/rrbrcz md5check z and rHzDownload from z failed. Retry limit reached)_download_methodsr0rJrNmakedirsr1r2r<r=rKDOWNLOAD_RETRY_LIMITrhrOrP)r,r'r-rGr4rq retry_cnts r rMrMs & &K+fX](KK & ::d  D IIcN2 Exxe$HI KK,ugVC512zz(# (F(C izvh78 + + NI %EE !(h7 JJqM zz(# (F(C Or c ^|ytjd|dtj}t |d5t fddD]}|j | ddd|j}||k7r tjd|d|d|d y y#1swY?xYw) NTzFile z md5 checking...rbc&jdS)Ni)read)rusr z_md5check..s !&&,r r z md5 check failed, z (calc) != z(base)F)r<r=hashlibmd5rjiterr hexdigest)rqr-rrw calc_md5sumrus @r rKrKs ~ KK%z!123 ++-C h .4E JJu 5 --/Kf H:0 Zhf     s &B##B,ctjd|dtj|r t |}|St j |r t|}|Std|)z) Decompress for zip and tar file zDecompressing z...zUnsupport compress file type ) r<r=rQrR_uncompress_file_tarrSrT_uncompress_file_zip TypeError)r4uncompressed_paths r rUrUst KK.s+,% 07    E "07 7w?@@r c^tj|d5}|j}g}|D]#}|j|j dd%t j j|}t|r8|d}t j j||}|j|ndt|rt j j|djt jdjt jd}t j j||}|j|nt j j|djt jd}t j j||}t j j!|st j"||jt j j|||cdddS#1swYyxYw)Nr../rr/)rSZipFilenamelistappendreplacerNr'dirname_is_a_single_filer2 extractall_is_a_single_dirsplitextstripsepr1rJr|)filepathfiles file_list_tmp file_listfilefile_dirrootpathrs r rrs 3 '5(  !D   T\\%4 5"77??8, Y ' |H " Xx @    X & i (ww'' ! (:(:266(BCAFLLH!# Xx @    X &ww''1!4::266B2FH " Xx @ 77>>"34 -.   RWW\\(H= > 9 ( ' 's HH##H,ctj||5}|j}g}|D]6}|ddk7s Jd|d|j|j dd8t j j|}t|r8|d}t j j||}|j|ndt|rt j j|djt jdjt jd}t j j||}|j|nt j j|djt jd}t j j||}t j j!|st j"||jt j j|||cdddS#1swYyxYw)Nr/zuncompress file path z should not start with /rrr/)rQrjgetnamesrrrNr'rrr2rrrrrr1rJr|) rmoderrrrrrrs r rr6s h %(  !DQ3 F&tf,DE F   T\\%4 5 " 77??8, Y ' |H " Xx @    X & i (ww'' ! (:(:266(BCAFLLH!# Xx @    X &ww''1!4::266B2FH " Xx @ 77>>"34 -.   RWW\\(H= > ; & % %s HH66H?clt|dk(r&|djtjdkryy)NrHrTF)lenfindrNr)rs r rrWs. 9~y|0081< r cg}|D]\}d|vr!|jdtj}n$d|vr |jdtj}|j|^|dj tjd}t dt |D],}|||j tjdk7s,yy)Nr\rrHFT)rrNrrr1ranger)r new_file_list file_path file_nameis r rr]sM ) !))#rvv6I Y !))$7IY' a &&rvv.q1I 1c-( )  a(..rvv6q9 9* r r)NTTrD)NrD)zr:*)"rrNos.pathr'r0rmrrQrOrSrerlogging getLoggerr r<__all__ expanduserr+r}r(r#r6rCr*ryr{rMrKrUrrrrrr r rs    #.   8 $ & 's~~<= E0%  KP,^ FM*!H(*!@!B C ###s B B