403Webshell
Server IP : 54.37.205.81  /  Your IP : 216.73.216.76
Web Server : nginx/1.22.1
System : Linux vps-249481fa 6.1.0-50-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.176-1 (2026-07-02) x86_64
User : debian ( 1000)
PHP Version : 8.2.32
Disable Function : NONE
MySQL : OFF  |  cURL : ON  |  WGET : ON  |  Perl : ON  |  Python : OFF  |  Sudo : ON  |  Pkexec : OFF
Directory :  /usr/lib/python3/dist-packages/nltk/__pycache__/

Upload File :
current_dir [ Writeable ] document_root [ Writeable ]

 

Command :


[ Back ]     

Current File : /usr/lib/python3/dist-packages/nltk/__pycache__/downloader.cpython-311.pyc
�

 T�ck��8�dZ	ddlZddlZddlZddlZddlZddlZddlZddlZddl	Z	ddl
Z
ddlZddlm
Z
ddlmZ	dZddlmZmZmZmZmZmZmZmZmZddlmZddlmZdd	lm Z n#e!$rd
Ze"ZYnwxYwddl#m$Z$m%Z%ddl&m'Z'ddl(Z(Gd
�d��Z)Gd�d��Z*Gd�d��Z+Gd�de+��Z,Gd�de+��Z-Gd�de+��Z.Gd�de+��Z/Gd�de+��Z0Gd�de+��Z1Gd�d e+��Z2Gd!�d"e+��Z3Gd#�d$e+��Z4Gd%�d&e+��Z5Gd'�d(e+��Z6Gd)�d*e+��Z7Gd+�d,e+��Z8Gd-�d.��Z9Gd/�d0��Z:Gd1�d2��Z;d3�Z<d4�Z=d`d5�Z>d`d6�Z?d7�Z@dad9�ZAd:�ZBd;�ZCd<�ZDd=�ZEe9��ZFeFjGZGd>�ZHd?�ZId@�ZJeKdAk�rKddBlLmMZMeM��ZNeN�OdCdDdEdFdG�H��eN�OdIdJdKdLd
dM�N��eN�OdOdPdQdLd
dR�N��eN�OdSdTdUdLd
dV�N��eN�OdWdXdYejP�QdZ��d[�\��eN�R��\ZSZTe9eSjU�]��ZVeTrEeTD]@ZWeV�GeWeSjXeSjYeSjZeSj[�^��Z\e\d
kr
eSj[rdS�AdSeV�GeSjXeSjYeSjZeSj[�_��dSdS)baH	
The NLTK corpus and module downloader.  This module defines several
interfaces which can be used to download corpora, models, and other
data packages that can be used with NLTK.

Downloading Packages
====================
If called with no arguments, ``download()`` will display an interactive
interface which can be used to download and install new packages.
If Tkinter is available, then a graphical interface will be shown,
otherwise a simple text interface will be provided.

Individual packages can be downloaded by calling the ``download()``
function with a single argument, giving the package identifier for the
package that should be downloaded:

    >>> download('treebank') # doctest: +SKIP
    [nltk_data] Downloading package 'treebank'...
    [nltk_data]   Unzipping corpora/treebank.zip.

NLTK also provides a number of "package collections", consisting of
a group of related packages.  To download all packages in a
colleciton, simply call ``download()`` with the collection's
identifier:

    >>> download('all-corpora') # doctest: +SKIP
    [nltk_data] Downloading package 'abc'...
    [nltk_data]   Unzipping corpora/abc.zip.
    [nltk_data] Downloading package 'alpino'...
    [nltk_data]   Unzipping corpora/alpino.zip.
      ...
    [nltk_data] Downloading package 'words'...
    [nltk_data]   Unzipping corpora/words.zip.

Download Directory
==================
By default, packages are installed in either a system-wide directory
(if Python has sufficient access to write to it); or in the current
user's home directory.  However, the ``download_dir`` argument may be
used to specify a different installation target, if desired.

See ``Downloader.default_download_dir()`` for more a detailed
description of how the default download directory is chosen.

NLTK Download Server
====================
Before downloading any packages, the corpus and module downloader
contacts the NLTK download server, to retrieve an index file
describing the available packages.  By default, this index file is
loaded from ``https://raw.githubusercontent.com/nltk/nltk_data/gh-pages/index.xml``.
If necessary, it is possible to create a new ``Downloader`` object,
specifying a different URL for the package index file.

Usage::

    python nltk/downloader.py [-d DATADIR] [-q] [-f] [-k] PACKAGE_IDS

or::

    python -m nltk.downloader [-d DATADIR] [-q] [-f] [-k] PACKAGE_IDS
�N)�md5)�ElementTreeT)	�Button�Canvas�Entry�Frame�IntVar�Label�Menu�TclError�Tk)�	showerror)�Table)�ShowTextF)�	HTTPError�URLError)�urlopenc�R�eZdZdZ											d
d�Zed���Zd�Zd	�ZdS)�Packagea4
    A directory entry for a downloadable package.  These entries are
    extracted from the XML index file that is downloaded by
    ``Downloader``.  Each package consists of a single file; but if
    that file is a zip file, then it can be automatically decompressed
    when the package is installed.
    N��UnknownTc�D�||_	|p||_	||_	||_	t	|��|_	t	|��|_	||_	||_	|	|_		|
|_
	||_	||_	tj�|�d��d��d}tj�|||z��|_	t't	|
����|_	|j�|��dS)N�/����)�id�name�subdir�url�int�size�
unzipped_size�checksum�svn_revision�	copyright�contact�license�author�os�path�splitext�split�join�filename�bool�unzip�__dict__�update)�selfrrrrr!r"r#r$r%r&r'r(r0�kw�exts                �1/usr/lib/python3/dist-packages/nltk/downloader.py�__init__zPackage.__init__�s��"���3��J�B��	�-����	4����E���I�I��	�:� ��/�/���	�!��
�4�(���<�"���0����	,����3����%��g���s�y�y��~�~�b�1�2�2�1�5������V�R�#�X�6�6��
�	3��#�e�*�*�%�%��
�	�	
�
���R� � � � � �c���t|t��rtj|��}|jD]$}t|j|��|j|<�%tdi|j��S)N�)�
isinstance�strr�parse�attribr)�xml�keys  r6�fromxmlzPackage.fromxmlsf���c�3���	)��#�C�(�(�C��:�	3�	3�C�!�#�*�S�/�2�2�C�J�s�O�O��$�$���$�$�$r8c�"�|j|jkS�N�r�r3�others  r6�__lt__zPackage.__lt__����w���!�!r8c��d|jzS)Nz<Package %s>rD�r3s r6�__repr__zPackage.__repr__ s�����'�'r8)NrNNNNrrrrT�	�__name__�
__module__�__qualname__�__doc__r7�staticmethodrArGrKr:r8r6rr�s�����������
���������C!�C!�C!�C!�J�%�%��\�%�"�"�"�(�(�(�(�(r8rc�<�eZdZdZdd�Zed���Zd�Zd�ZdS)�
Collectionz�
    A directory entry for a collection of downloadable packages.
    These entries are extracted from the XML index file that is
    downloaded by ``Downloader``.
    Nc�~�||_	|p||_	||_	d|_	|j�|��dSrC)rr�children�packagesr1r2)r3rrUrr4s     r6r7zCollection.__init__+sR�����6��J�B��	�0� ��
�	,���
�	3�	
�
���R� � � � � r8c��t|t��rtj|��}|jD]$}t|j|��|j|<�%d�|�d��D��}t
dd|i|j��S)Nc�8�g|]}|�d����S)�ref��get)�.0�childs  r6�
<listcomp>z&Collection.fromxml.<locals>.<listcomp>Cs$��F�F�F��E�I�I�e�$�$�F�F�Fr8�itemrUr:)r;r<rr=r>�findallrS)r?r@rUs   r6rAzCollection.fromxml=s����c�3���	)��#�C�(�(�C��:�	3�	3�C�!�#�*�S�/�2�2�C�J�s�O�O�F�F�#�+�+�f�2E�2E�F�F�F���:�:�8�:�s�z�:�:�:r8c�"�|j|jkSrCrDrEs  r6rGzCollection.__lt__FrHr8c��d|jzS)Nz<Collection %s>rDrJs r6rKzCollection.__repr__Is�� �4�7�*�*r8rCrLr:r8r6rSrS$sk��������!�!�!�!�$�;�;��\�;�"�"�"�+�+�+�+�+r8rSc��eZdZdZdS)�DownloaderMessagezSA status message object, used by ``incr_download`` to
    communicate its progress.N)rMrNrOrPr:r8r6rdrdRs������!�!�!�!r8rdc��eZdZdZd�ZdS)�StartCollectionMessagez<Data server has started working on a collection of packages.c��||_dSrC��
collection�r3ris  r6r7zStartCollectionMessage.__init__Z�
��$����r8N�rMrNrOrPr7r:r8r6rfrfWs)������F�F�%�%�%�%�%r8rfc��eZdZdZd�ZdS)�FinishCollectionMessagez=Data server has finished working on a collection of packages.c��||_dSrCrhrjs  r6r7z FinishCollectionMessage.__init__arkr8Nrlr:r8r6rnrn^s)������G�G�%�%�%�%�%r8rnc��eZdZdZd�ZdS)�StartPackageMessagez-Data server has started working on a package.c��||_dSrC��package�r3rts  r6r7zStartPackageMessage.__init__h�
������r8Nrlr:r8r6rqrqe�)������7�7�����r8rqc��eZdZdZd�ZdS)�FinishPackageMessagez.Data server has finished working on a package.c��||_dSrCrsrus  r6r7zFinishPackageMessage.__init__orvr8Nrlr:r8r6ryryl�)������8�8�����r8ryc��eZdZdZd�ZdS)�StartDownloadMessagez.Data server has started downloading a package.c��||_dSrCrsrus  r6r7zStartDownloadMessage.__init__vrvr8Nrlr:r8r6r}r}sr{r8r}c��eZdZdZd�ZdS)�FinishDownloadMessagez/Data server has finished downloading a package.c��||_dSrCrsrus  r6r7zFinishDownloadMessage.__init__}rvr8Nrlr:r8r6r�r�z�)������9�9�����r8r�c��eZdZdZd�ZdS)�StartUnzipMessagez,Data server has started unzipping a package.c��||_dSrCrsrus  r6r7zStartUnzipMessage.__init__�rvr8Nrlr:r8r6r�r��s)������6�6�����r8r�c��eZdZdZd�ZdS)�FinishUnzipMessagez-Data server has finished unzipping a package.c��||_dSrCrsrus  r6r7zFinishUnzipMessage.__init__�rvr8Nrlr:r8r6r�r��rwr8r�c��eZdZdZd�ZdS)�UpToDateMessagez/The package download file is already up-to-datec��||_dSrCrsrus  r6r7zUpToDateMessage.__init__�rvr8Nrlr:r8r6r�r��r�r8r�c��eZdZdZd�ZdS)�StaleMessagez3The package download file is out-of-date or corruptc��||_dSrCrsrus  r6r7zStaleMessage.__init__�rvr8Nrlr:r8r6r�r��s)������=�=�����r8r�c��eZdZdZd�ZdS)�ErrorMessagez Data server encountered an errorc�x�||_t|t��rt|��|_dS||_dSrC)rtr;�	Exceptionr<�message)r3rtr�s   r6r7zErrorMessage.__init__�s7������g�y�)�)�	#��w�<�<�D�L�L�L�"�D�L�L�Lr8Nrlr:r8r6r�r��s)������*�*�#�#�#�#�#r8r�c��eZdZdZd�ZdS)�ProgressMessagez4Indicates how much progress the data server has madec��||_dSrC)�progress)r3r�s  r6r7zProgressMessage.__init__�s
�� ��
�
�
r8Nrlr:r8r6r�r��s)������>�>�!�!�!�!�!r8r�c��eZdZdZd�ZdS)�SelectDownloadDirMessagez:Indicates what download directory the data server is usingc��||_dSrC)�download_dir�r3r�s  r6r7z!SelectDownloadDirMessage.__init__�s��(����r8Nrlr:r8r6r�r��s)������D�D�)�)�)�)�)r8r�c�H�eZdZdZdZ	dZ	dZ	dZ	dZ	dZ		d(d	�Z
						d)d�Zd
�Zd�Z
d�Zd�Zd�Zd*d�Zd�Zd�Zd�Zdddddd
dejfd�Zd+d�Zd+d�Zd+d�Zd+d�Zd�Zd,d�Zd+d�Zd�Zd �Z d!�Z!d"�Z"d#�Z#e$e"e#��Z%d$�Z&d%�Z'd&�Z(e$e'e(��Z)d'�Z*dS)-�
Downloaderzy
    A class used to access the NLTK data server, which can be used to
    download corpora and other data packages.
    izChttps://raw.githubusercontent.com/nltk/nltk_data/gh-pages/index.xml�	installedz
not installedzout of date�partialNc���|p|j|_	i|_	i|_	||_	d|_	d|_	i|_	d|_	|j�|�	��|_dSdSrC)
�DEFAULT_URL�_url�_collections�	_packages�
_download_dir�_index�_index_timestamp�
_status_cache�_errors�default_download_dir)r3�server_index_urlr�s   r6r7zDownloader.__init__�s���$�8��(8��	�7����E����?�)���I����@� $���	L� ���	�
���R���%�!%�!:�!:�!<�!<�D����&�%r8TFc��d}|�|j}td|z��|rjtddt|j��zz��td|jz��tddt|j��zz��|dz
}dx}}	g}
|r|
�d��|r|
�d	��|
D�]�}td
|���z��|dz
}t
t||����t���D�](}|�	||��}
|
|j
kr|r�'|
|jkrd
}|
|jkrd
}	|j
d|jd|jd|j
di|
}tjd|jp|jzdd���dd�}td�||j�dd��|����|t|�d����z
}|r1|dkr+t+d��}|���dvrdSd}��*t�����d}|r|dz
}|	r|dz
}ttj|d zdd!�"����dS)#Nrz!Using default data directory (%s)�=�z Data server index for <%s>�FrV�collectionsz%s:r�r@T�*�-�P� �---------------------------�K�                           ��subsequent_indent�z  [{}] {} {}��.�
zHit Enter to continue: )�x�qz([*] marks installed packagesz+; [-] marks out-of-date or corrupt packagesz+; [P] marks partially installed collections�)�L)r��width)r��print�lenr��append�
capitalize�sorted�getattrr<�status�	INSTALLED�STALE�PARTIAL�
NOT_INSTALLED�textwrap�fillrr�format�ljustr,�input�lower)r3r��
show_packages�show_collections�header�more_prompt�skip_installed�lines�staler��
categories�category�infor��prefixr�
user_input�msgs                  r6�listzDownloader.list	s��������-�L��5��D�E�E�E��	��#��c�$�)�n�n�,�-�.�.�.��/�$�)�;�<�<�<��#��c�$�)�n�n�,�-�.�.�.��Q�J�E������
��	*����j�)�)�)��	-����m�,�,�,�"�	�	�H��%�(�-�-�/�/�/�0�0�0��Q�J�E��6�w�t�X�6�6�8�8�c�B�B�B�
�
�����T�<�8�8���T�^�+�+��+���T�Z�'�'� �E��T�\�)�)�"�G��N�C��J���L�#��&��	�
��� �}���	� 4�T�W�5�r�X�����#�#����n�+�+�F�D�G�M�M�"�c�4J�4J�D�Q�Q�R�R�R���T�Z�Z��-�-�.�.�.����5�2�:�:�!&�'@�!A�!A�J�!�'�'�)�)�Z�7�7������E���G�G�G�G�-���	A��@�@�C��	A��@�@�C�
�h�m�C�#�I��B�G�G�G�H�H�H�H�Hr8c�\�|���|j���SrC)�
_update_indexr��valuesrJs r6rVzDownloader.packagesEs(���������~�$�$�&�&�&r8c�p�|���d�|j���D��S)Nc�.�g|]\}}|jdk�|��S��corpora�r�r\r�pkgs   r6r^z&Downloader.corpora.<locals>.<listcomp>K�(��W�W�W�	��S�s�z�Y�?V�?V��?V�?V�?Vr8�r�r��itemsrJs r6r�zDownloader.corporaI�6��������W�W�T�^�%9�%9�%;�%;�W�W�W�Wr8c�p�|���d�|j���D��S)Nc�.�g|]\}}|jdk�|��Sr�r�r�s   r6r^z%Downloader.models.<locals>.<listcomp>Or�r8r�rJs r6�modelszDownloader.modelsMr�r8c�\�|���|j���SrC)r�r�r�rJs r6r�zDownloader.collectionsQs)��������� �'�'�)�)�)r8c�Z�t|t��r|�|��S|SrC)r;r<r�)r3�
info_or_ids  r6�_info_or_idzDownloader._info_or_idYs,���j�#�&�&�	��9�9�Z�(�(�(��r8c#�@K�|�|j}t|��V�t|ttf��r|�|||��Ed{V��dS	|�|��}n7#ttf$r#}tdd|�d|����V�Yd}~dSd}~wwxYwt|t��rFt|��V�|�|j
||��Ed{V��t|��V�dS|�|||��Ed{V��dS)NzError loading �: )r�r�r;r��tuple�_download_listr��OSError�
ValueErrorr�rSrf�
incr_downloadrUrn�_download_package)r3r�r��forcer��es      r6r�zDownloader.incr_downloadhs��������-�L�*�<�8�8�8�8�8��j�4��-�0�0�	��*�*�:�|�U�K�K�K�K�K�K�K�K�K��F�	��#�#�J�/�/�D�D����$�	�	�	��t�%G�j�%G�%G�A�%G�%G�H�H�H�H�H��F�F�F�F�F�����	����
�d�J�'�'�	I�(��.�.�.�.�.��)�)�$�-��u�M�M�M�M�M�M�M�M�M�)�$�/�/�/�/�/�/�/��-�-�d�L�%�H�H�H�H�H�H�H�H�H�H�Hs�A/�/B#�B�B#c�X�t|t��rdSt|j��S)Nr)r;rr�rV)r3r_s  r6�
_num_packageszDownloader._num_packages�s)���d�G�$�$�	&��1��t�}�%�%�%r8c#�p�K�tt|����D]Y}	��||��||<�"#ttf$r$}t|||��V�Yd}~dSd}~wwxYwt
�fd�|D����}d}t|��D]�\}}t|t��rd|z}	nt|j
��|z}	��|||��D]8}
t|
t��rt||
j
|	zz��V��4|
V��9|d|	zz
}��dS)Nc3�B�K�|]}��|��V��dSrC)r�r\r_r3s  �r6�	<genexpr>z,Downloader._download_list.<locals>.<genexpr>�s1�����F�F��4�-�-�d�3�3�F�F�F�F�F�Fr8rg�?�d)�ranger�r�r�r�r��sum�	enumerater;rrVr�r�r�)r3r�r�r�ir�num_packagesr�r_�deltar�s`          r6r�zDownloader._download_list�s�������s�5�z�z�"�"�	�	�A�
��+�+�E�!�H�5�5��a�����Z�(�
�
�
�"�5��8�Q�/�/�/�/�/������������
����
�F�F�F�F��F�F�F�F�F���� ��'�'�	$�	$�G�A�t��$��(�(�
:��l�*����D�M�*�*�\�9���)�)�$��e�D�D�
�
���c�?�3�3��)�(�S�\�E�5I�*I�J�J�J�J�J�J��I�I�I�I���e��#�H�H�	$�	$s�A�A7�A2�2A7c#��K�t|��V�td��V�|�||��}|s@||jkr5t	|��V�td��V�t|��V�dS|j�|jd��tj
�||j��}tj
�
|��r0||jkrt|��V�tj|��tj
�
|��stj|��tj
�
tj
�||j����s7tjtj
�||j����t'|��V�td��V�	t)|j��}t-|d��5}t/d|jdz��}t3j��D]a}	|�d��}
|�|
��|
sn2|	dzdkr(tt;ddd	|	|zzz����V��bddd��n#1swxYwY|���n=#t>$r0}tA|d
|j�d|j�d|����V�Yd}~dSd}~wwxYwtC|��V�td��V�|j�"d
��r�tj
�||j��}|j#sBtj
�
tj
�||j����rBtI|��V�tK||d���D]
}
||
_&|
V��tO|��V�t|��V�dS)Nrr	��wbr�@��Pr�zError downloading z from <�>:
  �.zipF)�verbose)(rqr�r�r�r�ryr��poprr)r*r-r.�existsr�r��remove�makedirsrr}rr�open�maxr!�	itertools�count�read�write�min�closer�r�r��endswithr0r��_unzip_iterrtr�)r3r�r�rr��filepath�infile�outfile�
num_blocks�block�sr�zipdirr�s              r6rzDownloader._download_package�sL����!�$�'�'�'�'�'��a� � � � � ����T�<�0�0���	��4�>�1�1�!�$�'�'�'�'�'�!�#�&�&�&�&�&�&�t�,�,�,�,�,��F�	
����t�w��-�-�-��7�<�<��d�m�<�<��
�7�>�>�(�#�#�	 ����#�#�"�4�(�(�(�(�(��I�h�����w�~�~�l�+�+�	&��K��%�%�%��w�~�~�b�g�l�l�<���E�E�F�F�	A��K�����\�4�;�?�?�@�@�@�#�4�(�(�(�(�(��a� � � � � �	��T�X�&�&�F��h��%�%�
V�� ��D�I��$;�<�<�
�&�_�.�.�V�V�E����I�.�.�A��M�M�!�$�$�$������q�y�A�~�~�-�c�"�a�"��
�@R�:S�6S�.T�.T�U�U�U�U�U��
V�
V�
V�
V�
V�
V�
V�
V�
V�
V�
V����
V�
V�
V�
V�
�L�L�N�N�N�N���	�	�	����>B�g�g�g�t�x�x�x�QR�QR�S���
�
�
�
�F�F�F�F�F�����	����$�D�)�)�)�)�)��b�!�!�!�!�!��=�!�!�&�)�)�
	/��W�\�\�,���<�<�F��z�
/�R�W�^�^�B�G�L�L����,I�,I�J�J�
/�'��-�-�-�-�-�&�x���G�G�G���C�"&�C�K��I�I�I�I�(��.�.�.�.�.�"�4�(�(�(�(�(�(�(s=� $J>�BJ�J>�J"�"J>�%J"�&J>�>
K8�%K3�3K8�[nltk_data] c	
�����tjt|����|�|�||_|���dSd��fd�	}	|�|||��D�]}
t
|
t��r�|	|
j��|rt|
j���|rdSd|_
|s_�d��t�����}|dvr)|�
|
jj|||�||��sdSn|dvrdS|�sRt
|
t ��r)|	d	|
jjz���d
z
�������t
|
t$��rP�����dd��|j
r|	d|
jjz����7|	d
|
jjz����Qt
|
t&��r|	d|
jj�d|�d������t
|
t(��r|	d|
jjzd�����t
|
t*��r|	d|
jjzd�����t
|
t.��r|
j}��dS)N)�fileTrc�Z���tj|�|z�|zdz�����dS)N�    )�initial_indentr�)r�r�)r,�prefix2r��print_tos  ��r6�showz!Downloader.download.<locals>.showsM������M��'-��'7�*0�7�*:�W�*D��������r8Fz(Error installing package. Retry? [n/y/e])�y�Y)r�E�Downloading collection %rz   | ���z$Downloaded collection %r with errorszDone downloading collection %szDownloading package z to z...z!Package %s is already up-to-date!�  z
Unzipping %s.�r)�	functoolsr�r�r��_interactive_downloadr�r;r�r�r�r�r��strip�downloadrtrrfrirnrqr�r�r.r�r�)
r3r�r��quietrr��
halt_on_error�raise_on_error�print_error_tor6r��choicer5s
     `      @r6rAzDownloader.download�s�����$�U��@�@�@�����'�%1��"��&�&�(�(�(��4�
�
�
�
�
�
�
��)�)�*�l�E�J�J�<
8�<
8���c�<�0�0�)��D���%�%�%�%�6�(���5�5�5�$�%�$�u�u�#'�D�L� �)� ��!K�L�L�L�!&��������!�Z�/�/�#'�=�=� #��� ,� %� %� &� -� .�$�$�	-�(-�u�u�	-�$�z�1�1�#(�5�5��!8�!�#�'=�>�>�8���8�3�>�;L�L�M�M�M��'�)�� ���(�(�(�(�#�C�)@�A�A�8� ���(�(�(�!'�������<�W� �D� F�"%�.�"3�!4�����
!�D�!A�C�N�DU�!U�V�V�V�V�$�C�)<�=�=�8����"�{�~�~�~�|�|�|�=�����$�C��9�9�
8���@�3�;�>�Q�SW�X�X�X�X�$�C�):�;�;�8���_�s�{�/C�C�T�J�J�J�J�$�C�)A�B�B�8�'*�'7����tr8c�@�|�||��|jkSrC)r�r��r3r�r�s   r6�is_stalezDownloader.is_staleHs���{�{�:�|�4�4��
�B�Br8c�@�|�||��|jkSrC)r�r�rHs   r6�is_installedzDownloader.is_installedKs���{�{�:�|�4�4���F�Fr8c�v�|�|j���dS|j�|d��dSrC)r��clearr�r3rs  r6�clear_status_cachezDownloader.clear_status_cacheNsA��
�:���$�$�&�&�&�&�&���"�"�2�t�,�,�,�,�,r8c�B��|��j}��|��}t|t��rc�fd�|jD��}�j|vr�jS�j|vr�jS�j|vr�j|vr�jS�j|vr�jS�jStj
�||j��}|�jkr��
||��S|j�jvr#��
||���j|j<�j|jS)z�
        Return a constant describing the status of the given package
        or collection.  Status can be one of ``INSTALLED``,
        ``NOT_INSTALLED``, ``STALE``, or ``PARTIAL``.
        Nc�D��g|]}��|j����Sr:)r�r)r\r�r3s  �r6r^z%Downloader.status.<locals>.<listcomp>`s'���G�G�G�#�$�+�+�c�f�-�-�G�G�Gr8)r�r�r;rSrVr�r�r�r�r)r*r-r.�_pkg_statusrr�)r3r�r�r��
pkg_statusr's`     r6r�zDownloader.statusTs<������-�L����
�+�+���d�J�'�'�	3�G�G�G�G���G�G�G�J��z�Z�'�'��z�!����+�+��|�#���:�-�-�$�2D�
�2R�2R��|�#��#�z�1�1��)�)��~�%��w�|�|�L�$�-�@�@�H��t�1�1�1��'�'��h�7�7�7��7�$�"4�4�4�26�2B�2B�4��2R�2R�D�&�t�w�/��)�$�'�2�2r8c��tj�|��s|jS	tj|��}n#t
$r
|jcYSwxYw|jt|j��kr|j	St|��|jkr|j	S|�d��r�|dd�}tj�|��s|j
Stj�|��s|j	Std�tj|��D����}||jkr|j	S|j
S)Nrr;c3�K�|]D\}}}|D];}tjtj�||����jV��<�EdSrC)r)�statr*r-�st_size)r\�d�_�files�fs     r6rz)Downloader._pkg_status.<locals>.<genexpr>�sp���� � ��A�q�%�� � ��������Q��*�*�+�+�3� � � � � � � r8)r)r*rr�rVr�rWr r!r��
md5_hexdigestr#r%r��isdirr�walkr")r3r�r'�filestat�unzipdirr"s      r6rRzDownloader._pkg_statusvsV���w�~�~�h�'�'�	&��%�%�	&��w�x�(�(�H�H���	&�	&�	&��%�%�%�%�	&������s�4�9�~�~�-�-��:����"�"�d�m�3�3��:�����V�$�$�
	"�����}�H��7�>�>�(�+�+�
&��~�%��7�=�=��*�*�
"��z�!�� � �#%�7�8�#4�#4� � � ���M�
�� 2�2�2��z�!��~�s�=�A�Ac���|���|���D]8}|�|��|jkr|�|||����9dS)zA
        Re-download any packages whose status is STALE.
        )rBr�N)rOrVr�r�rA)r3rBr�r�s    r6r2zDownloader.update�si��	
���!�!�!��=�=�?�?�	?�	?�C��{�{�3���4�:�-�-��
�
�c��v�
�>�>�>��	?�	?r8c���|j�(|�&tj��|jz
|jksdS|p|j|_t
j�tj	t|j���������|_tj��|_d�|j�d��D��}d�|D��|_
d�|j�d��D��}d�|D��|_|j���D]�}t!|j��D]m\}}||j
vr|j
||j|<�$||jvr|j||j|<�Ct%d�|����|j|=�n��|j���D]u}i}|g}|D]R}t)|t*��r|�|j���2t)|t.��r|||j<�R�S|���|_�v|j���dS)	z�A helper function that ensures that self._index is
        up-to-date.  If the index is older than self.INDEX_TIMEOUT,
        then download it again.Nc�B�g|]}t�|����Sr:)rrA�r\�ps  r6r^z,Downloader._update_index.<locals>.<listcomp>�s$��X�X�X�1�G�O�O�A�&�&�X�X�Xr8�packages/packagec��i|]
}|j|��Sr:rDrds  r6�
<dictcomp>z,Downloader._update_index.<locals>.<dictcomp>�s��4�4�4�a�!�$��4�4�4r8c�B�g|]}t�|����Sr:)rSrA�r\�cs  r6r^z,Downloader._update_index.<locals>.<listcomp>�s3��
�
�
�&'�J���q�!�!�
�
�
r8�collections/collectionc��i|]
}|j|��Sr:rDrjs  r6rhz,Downloader._update_index.<locals>.<dictcomp>�s��:�:�:��Q�T�1�:�:�:r8z.removing collection member with no package: {})r��timer��
INDEX_TIMEOUTr��nltk�	internals�ElementWrapperrr=r�getrootr`r�r�r�rrUr�r�r;rS�extendrrrVr�rM)	r3rrVr�rir
�child_id�queuer]s	         r6r�zDownloader._update_index�s���
�K�����y�{�{�T�2�2�T�5G�G�G��F��$�4�9��	��n�3�3���g�d�i�0�0�1�1�9�9�;�;�
�
���!%�	�����Y�X���0C�0C�DV�0W�0W�X�X�X��4�4�8�4�4�4���
�
�+/�;�+>�+>�?W�+X�+X�
�
�
��;�:�k�:�:�:����+�2�2�4�4�	/�	/�J�(��)<�=�=�
/�
/���8��t�~�-�-�-1�^�H�-E�J�'��*�*���!2�2�2�-1�->�x�-H�J�'��*�*��H�O�O�$������
#�+�A�.�.�
/��+�2�2�4�4�
	4�
	4�J��H��L�E��
�
���e�Z�0�0���L�L���0�0�0�0���w�/�/��).�H�U�X�&�&��"*�/�/�"3�"3�J���	
�� � �"�"�"�"�"r8c�8�|���|jS)z�
        Return the XML index describing the packages available from
        the data server.  If necessary, this index will be downloaded
        from the data server.
        )r�r�rJs r6�indexzDownloader.index�s��	
�������{�r8c��|���||jvr
|j|S||jvr
|j|Std|z���)zKReturn the ``Package`` or ``Collection`` record for the
        given item.�Package %r not found in index)r�r�r�r�rNs  r6r�zDownloader.info�s`��	
������
������>�"�%�%�
��"�"�"��$�R�(�(��8�2�=�>�>�>r8c�6�|���|j�d��D]}|�d��|kr|cS� |j�d��D]}|�d��|kr|cS� t	d|z���)z-Return the XML info record for the given itemrfrrlrz)r�r�r`r[r�)r3rrtris    r6�xmlinfozDownloader.xmlinfo�s����������{�*�*�+=�>�>�	�	�G��{�{�4� � �B�&�&�����'��+�-�-�.F�G�G�	"�	"�J��~�~�d�#�#�r�)�)�!�!�!�!�*��8�2�=�>�>�>r8c��|jS)z)The URL for the data server's index file.)r�rJs r6�_get_urlzDownloader._get_urls
���y�r8c�Z�|j}	|�|��dS#||_�xYw)z�
        Set a new URL for the data server. If we're unable to contact
        the given url, then the original url is kept.
        N)r�r�)r3r�original_urls   r6�_set_urlzDownloader._set_urls=��
�y��	����s�#�#�#�#�#��	�$�D�I����s� �
*c���dtjvrdStjjD]D}tj�|��r#tj�|��r|cS�Etj	dkr!dtjvrtjd}n4tj�
d��}|dkrtd���tj�|d��S)a�
        Return the directory to which packages will be downloaded by
        default.  This value can be overridden using the constructor,
        or on a case-by-case basis using the ``download_dir`` argument when
        calling ``download()``.

        On Windows, the default download directory is
        ``PYTHONHOME/lib/nltk``, where *PYTHONHOME* is the
        directory containing Python, e.g. ``C:\Python25``.

        On all other platforms, the default directory is the first of
        the following which exists or which can be created with write
        permission: ``/usr/share/nltk_data``, ``/usr/local/share/nltk_data``,
        ``/usr/lib/nltk_data``, ``/usr/local/lib/nltk_data``, ``~/nltk_data``.
        �APPENGINE_RUNTIMEN�win32�APPDATAz~/z+Could not find a default download directory�	nltk_data)
r)�environrp�datar*rrq�is_writable�sys�platform�
expanduserr�r-)r3�nltkdir�homedirs   r6r�zDownloader.default_download_dirs���"�"�*�,�,��F��y�~�	�	�G��w�~�~�g�&�&�
�4�>�+E�+E�g�+N�+N�
�������<�7�"�"�y�B�J�'>�'>��j��+�G�G��g�(�(��.�.�G��$��� �!N�O�O�O��w�|�|�G�[�1�1�1r8c��|jS)a
        The default directory to which packages will be downloaded.
        This defaults to the value returned by ``default_download_dir()``.
        To override this default on a case-by-case basis, use the
        ``download_dir`` argument when calling ``download()``.
        )r�rJs r6�_get_download_dirzDownloader._get_download_dir@s
���!�!r8c�F�||_|j���dSrC)r�r�rMr�s  r6�_set_download_dirzDownloader._set_download_dirIs%��)����� � �"�"�"�"�"r8c��trV	t|�����dS#t$r%t	|�����YdSwxYwt	|�����dSrC)�TKINTER�
DownloaderGUI�mainloopr�DownloaderShell�runrJs r6r?z Downloader._interactive_downloadTs����	(�
,��d�#�#�,�,�.�.�.�.�.���
,�
,�
,���%�%�)�)�+�+�+�+�+�+�
,����
�D�!�!�%�%�'�'�'�'�'s�!,�+A�A)NN)NTTTFF)NFrC)Fr.)+rMrNrOrPror�r�r�r�r�r7r�rVr�r�r�r�r�rr�rr��stderrrArIrKrOr�rRr2r�rxr�r|r~r��propertyrr�r�r�r�r?r:r8r6r�r��s����������M��X�K�!��I�$�#�M���E�"��G�E�=�=�=�=�N������:I�:I�:I�:I�x'�'�'�X�X�X�X�X�X�*�*�*����I�I�I�I�8&�&�&�$�$�$�2B)�B)�B)�L��������z�_�_�_�_�BC�C�C�C�G�G�G�G�-�-�-�-� 3� 3� 3� 3�D"�"�"�H?�?�?�?�<#�<#�<#�<#�|���?�?�?�	?�	?�	?����
�
�
��(�8�X�
&�
&�C�%2�%2�%2�N"�"�"�#�#�#�
�8�-�/@�A�A�L�	(�	(�	(�	(�	(r8r�c�>�eZdZd�Zd�Zd�Zd�Zd�Zd�Zd�Z	d�Z
d	S)
r�c��||_dSrC)�_ds)r3�
dataservers  r6r7zDownloaderShell.__init__as
������r8c���td��dtd�|D����z
t|��dz
zdz}td|�|��z��td��dS)NzK---------------------------------------------------------------------------�Dc3�4K�|]}t|��V��dSrC)r�)r\�os  r6rz;DownloaderShell._simple_interactive_menu.<locals>.<genexpr>fs(����0�0�1��A���0�0�0�0�0�0r8rr�r2)r�rr�r-)r3�options�spcs   r6�_simple_interactive_menuz(DownloaderShell._simple_interactive_menudsq��
�h�����C�0�0��0�0�0�0�0�0�c�'�l�l�Q�6F�G�#�M��
�f�s�x�x��(�(�(�)�)�)�
�h�����r8c��td��	|�dddddd��td	�����}|st���M|������d
}|���dd�}	|dkr6t��|j�|jjd
d���n�|dkr|�	��nj|dkr|�
��nO|dvrdS|dkr|�|��n-|dkr|���ntd|z��nR#t$r}td|z��Yd}~n1d}~wt$r!}td|jz��Yd}~nd}~wwxYwt�����)N�NLTK DownloaderTzd) Downloadzl) Listz
 u) Updatez	c) Configzh) Helpzq) QuitzDownloader> rr�lF)r�r��hrk)r�r�rX�uzCommand %r unrecognizedzError reading from server: %szError connecting to server: %s)r�r�r�r@r�r,r�r�r��_simple_interactive_help�_simple_interactive_config�_simple_interactive_download�_simple_interactive_updaterr�reason)r3r��command�argsrs     r6r�zDownloaderShell.runjs��
�� � � �%	��)�)�������

�
�
��~�.�.�4�4�6�6�J��
������ �&�&�(�(�.�.�0�0��3�G��#�#�%�%�a�b�b�)�D�
C��c�>�>��G�G�G��H�M�M�$�(�"7��SW�M�X�X�X�X���^�^��1�1�3�3�3�3���^�^��3�3�5�5�5�5��
�*�*��F���^�^��5�5�d�;�;�;�;���^�^��3�3�5�5�5�5��3�j�@�A�A�A����
;�
;�
;��5��9�:�:�:�:�:�:�:�:������
C�
C�
C��6���A�B�B�B�B�B�B�B�B�����
C����
�G�G�G�K%	s,�&A6E(�A	E(�(
F7�2F	�	
F7�F2�2F7c��|rQ|D]L}	|j�|d���� #ttf$r}t	|��Yd}~�Ed}~wwxYwdS	t	��t	d��td��}|���dkr)|j�|jjddd����n|���d	vrdS|rc|�	��D]L}	|j�|d���� #ttf$r}t	|��Yd}~�Ed}~wwxYwdS��)
Nr2�r�Tz*Download which package (l=list; x=cancel)?�  Identifier> r�F)r�r�r��r�r�r)
r�rAr�r�r�r�r�r�r�r,)r3r��argrr�rs      r6r�z,DownloaderShell._simple_interactive_download�s����	��
�
����H�%�%�c�&�%�9�9�9�9����,�����!�H�H�H�H�H�H�H�H���������
�
�
������B�C�C�C�"�#3�4�4�
��#�#�%�%��,�,��H�M�M���-�$�$(�'+�	"������%�%�'�'�>�9�9��F���(�.�.�0�0�%�%��%� �H�-�-�b��-�@�@�@�@�� '��4�%�%�%�!�!�H�H�H�H�H�H�H�H�����%�����E�)
s,�%�A�A
�
A�3D�D:�!D5�5D:c	�v�	g}dx}}tt|jd����t���D]K}|j�|��|jjkr!|�|j|jf���Lt��|�rtd��|D]^\}}tjd|zdd�	��d
d�}td�|�
dd
��|�����_t��td��}|���dkrT|D]O\}}	|j�|d����##t"t$f$r}	t|	��Yd}	~	�Hd}	~	wwxYwdS|���dvrdSntd��dS���)NTFrVr�z/Will update following packages (o=ok; x=cancel)r�r�r�r�r�z  [ ] {} {}r�r�r�r�r2r�r�zNothing to update.)r�r�r�r<r�r�r�rrr�r�r�r�r�r�r�rAr�r�)
r3�stale_packagesr�r�r��pid�pnamerr�rs
          r6r�z*DownloaderShell._simple_interactive_update�s���	��N�#�#�E�G��<�w�t�x��<�<�>�>�C�H�H�H�
@�
@���8�?�?�4�(�(�D�H�N�:�:�"�)�)�4�7�D�I�*>�?�?�?���G�G�G��
��G�H�H�H�"0�J�J�J�C��#�=� �E�*�B�(�����c�c��D��-�.�.�s�y�y��S�/A�/A�4�H�H�I�I�I�I�����"�#3�4�4�
��#�#�%�%��,�,�&4�%�%�
��U�%� �H�-�-�c�&�-�A�A�A�A�� '��4�%�%�%�!�!�H�H�H�H�H�H�H�H�����%�����E��%�%�'�'�>�9�9��F�:��*�+�+�+���;	s�E � F
�1F�F
c��t��td��td��td��td��dS)Nz	Commands:zH  d) Download a package or collection     u) Update out of date packagesz1  l) List packages & collections          h) Helpz1  c) View & Modify Configuration          q) Quit)r�rJs r6r�z(DownloaderShell._simple_interactive_help�sT��
����
�k����
�V�	
�	
�	
�	�A�B�B�B�
�A�B�B�B�B�Br8c���t��td��td|jjz��tdt|j�����z��tdt|j�����z��t��td��td|jjz��dS)NzData Server:z
  - URL: <%s>z$  - %d Package Collections Availablez$  - %d Individual Packages AvailablezLocal Machine:z  - Data directory: %s)r�r�rr�r�rVr�rJs r6�_show_configzDownloaderShell._show_config�s���
����
�n����
�o����,�-�-�-�
�4�s�4�8�;O�;O�;Q�;Q�7R�7R�R�S�S�S�
�4�s�4�8�;L�;L�;N�;N�7O�7O�O�P�P�P�
����
�����
�&���)>�>�?�?�?�?�?r8c�<�|���	t��|�dddd��td��������}|dkr|����n|dkrttd	�����}|d
vrtd��n�tj�|��r
||j	_
n�td|z��n�|d
kr�td�����}|d
vrtd��n\|�d��sd|z}	||j	_n4#t$r}td|�d|����Yd}~nd}~wwxYw|dkrdS���)NTzs) Show Configzu) Set Server URLzd) Set Data Dirzm) Main MenuzConfig> r,rXz  New Directory> )rr�r��X�Qz  Cancelled!z)Directory %r not found!  Create it first.r�z  New URL> )�http://zhttps://r�zError reading <r�m)r�r�r�r�r@r�r)r*r]r�r��
startswithrr�)r3r��
new_dl_dir�new_urlrs     r6r�z*DownloaderShell._simple_interactive_config�s���������	��G�G�G��)�)� �"5�7H�.�
�
�
��z�*�*�0�0�2�2�8�8�:�:�J��S� � ��!�!�#�#�#�#��s�"�"�"�#6�7�7�=�=�?�?�
��!9�9�9��.�)�)�)�)��W�]�]�:�.�.�T�,6�D�H�)�)��E�
�R�S�S�S�S��s�"�"��
�.�.�4�4�6�6���6�6�6��.�)�)�)�)�"�-�-�.E�F�F�6�"+�g�"5��F�'.������$�F�F�F��D��D�D��D�D�E�E�E�E�E�E�E�E�����F�����s�"�"���9	s�E(�(
F�2F�FN)rMrNrOr7r�r�r�r�r�r�r�r:r8r6r�r�`s�������������'�'�'�R���<���@C�C�C�@�@�@�����r8r�c�"�eZdZdZgd�Z	ddddd�Z	dddd	d	d
d�Z	dZ	gd
�Z	eD]Z	e	evsJ��	eD]Z	e	evsJ��	eD]Z	e	evsJ��	dZ
ejdej
dejdejdiZdZdZdZdZdZdCd�Zd�Zd�Zd�Zd�Zd�Zd�ZdDd �Zd!�Zd"�Zd#�Z d$�Z!d%�Z"d&�Z#d'�Z$d(Z%dZ&d)�Z'd*�Z(d+�Z)d	Z*d,�Z+d-�Z,d.�Z-d/�Z.d0�Z/d1�Z0d2�Z1d3�Z2d4�Z3d5�Z4d6�Z5e6j7d7��Z8d8�Z9d9�Z:dZ;d:�Z<d;�Z=d<�Z>d=�Z?d>�Z@Gd?�d@eAjB��ZCdAZDdB�ZEdS)Er�zU
    Graphical interface for downloading packages from the NLTK data
    server.
    )r�
Identifier�Name�Size�Status�
Unzipped Size�	Copyright�Contact�License�Author�Subdir�Checksumrr)rr�r�r�rr��-�
�)rr�r�r�r�r��)rr�r�r�r�)z#000z#ccc)z#afaz#080)z#ffaz#880)z#faaz#800)�#fffz#888)r�z#45c)�#aaaz#67a)z#f00r�zhelvetica -16 boldTc���||_||_tj��|_g|_g|_d|_i|_g|_	d|_
|�d��t��x}|_
|�d��|�d��|�|jd���|�d|j��|�d	|j��d|_i|_|���|���	|���nN#t2$r}t5d
|��Yd}~n/d}~wt6$r}t5d|j��Yd}~nd}~wwxYw|���|���|j� d��|j�d|j!��dS)
NFrzNLTK Downloader Started!z+50+50r�r)�
backgroundz<Control-q>z<Control-x>�Error reading from server�Error connecting to serverz	<Destroy>)"r��_use_threads�	threading�Lock�_download_lock�_download_msg_queue�_download_abort_queue�_downloading�_afterid�
_log_messages�_log_indent�_logr
�top�geometry�title�	configure�_BACKDROP_COLOR�bind�destroy�
_destroyed�_column_vars�
_init_widgets�
_init_menu�_fill_tablerrrr��
_show_info�_select_columns�_table�select�_destroy)r3r��use_threadsr�rs     r6r7zDownloaderGUI.__init___s
�����'���(�n�.�.���#%�� �%'��"�!�����
� �������	�	�,�-�-�-������d�h����X�����	�	�#�$�$�$��
�
��!5�a�!8�
�9�9�9�	������-�-�-�������-�-�-�������	
������������	>����������	6�	6�	6��1�1�5�5�5�5�5�5�5�5������	>�	>�	>��2�A�H�=�=�=�=�=�=�=�=�����	>����	
����������������1����	
�����d�m�4�4�4�4�4s$�3E�
F�E'�'
F�4F�Fc��|j�d�tj��d|jz|����dS)Nz{} {}{}z | )r�r�r�rn�ctimer��r3r�s  r6r�zDownloaderGUI._log�sJ����!�!����T�Z�\�\�5�4�3C�+C�S�I�I�	
�	
�	
�	
�	
r8c���t�jdddd���}|�ddd�	��|�dd
���|�dd
���t|d����dd�
��t|��}|�dd
d���t|��}|�ddd���t|��}|�ddd���t|d����dd�
��t|��}|�ddd���t|d����dd�
��t�jdd�jd
���}|�dd���d�jd<d�jd<gd��_i�_t�j��D]s\}}t||�j���}	|	�d|d
zdzdz���|	�d�j
��|	�j|���<�t�fd ��jD��}
t!|�j|
dd!�j�"���_�j�d�jd�#��t�j��D]A\}}�j�|�j��}�j�||�$���B�j�dd�%���j����j�d&�j���j�d'�j���j�d(�j���j�d)�j���j�d*�j���j�d+�j��|�d
d
���d,d-�jfd.d/�j fg}
i�_!t|
��D]�\}\}}	}t||	�0���d|d1���tE|d2d3d4d4�5��}||f�j!|<|�d(�j#��|�d|f�fd6�	��|�d
|d7������j�d�j#��tI|d8�jd�9���_%�j%�d�:��tI|d;�j&d�9���_'�j'�d<�:��t|d=�jd�jd
�>���_(tS|d?d!�j*d
d@d
�A���_+��,���j+�d<�:���j(�d�:��dS)BN�raisedr�r)�relief�border�padx�padyr�T�both)�sid�expandr�r)�weight)�height)�column�row�news)rr	�stickyr��r�)rrr��bottomr�)�sider�r�highlightthickness)�Collections�Corpora�ModelszAll Packages)�text�font�leftr�)rrz
<Button-1>c�F��g|]}�j�|d����S)r)�COLUMN_WEIGHTSr[)r\rr3s  �r6r^z/DownloaderGUI._init_widgets.<locals>.<listcomp>�s,���X�X�X��$�-�1�1�&�!�<�<�X�X�Xr8�)�column_weightsr�listbox_height�reprfunc)�
foreground�r�)rr�z<Double-Button-1>z<space>z<Return>z<Left>z<Right>z<Control-a>rz
Server Index:r�zDownload Directory:)rr�courier�groovez#007aff)rr��disabledforegroundrc�.����|��SrC��
_info_edit)rr@r3s  �r6�<lambda>z-DownloaderGUI._init_widgets.<locals>.<lambda>�s�������8L�8L�r8�ew�Download)rr�r�)r�Refresh�rightr)rrr����sunken)r�rr�r�r)-rr��pack�grid_rowconfigure�grid_columnconfigure�gridr��
_tab_names�_tabsrr
�	_TAB_FONTr��_select_tabr��COLUMNSr�_table_reprfuncr��columnconfig�_MARK_COLOR�
COLUMN_WIDTHSr[�DEFAULT_COLUMN_WIDTH�focus�bind_to_listboxes�	_download�_table_mark�	_prev_tab�	_next_tab�	_mark_allr�r��_infor�
_info_saver�_download_button�_refresh�_refresh_button�_progresslabelr�_PROGRESS_COLOR�_progressbar�_init_progressbar)r3�f1�tabframe�
tableframe�buttonframe�	infoframe�
progressframer
�tab�labelrrr�r�r@�callback�entrys`                r6r�zDownloaderGUI._init_widgets�sX���
�4�8�H�Q�Q�Q�
G�
G�
G��
���E�$�V��4�4�4�
���Q�q��)�)�)�
����!��,�,�,�
�b����� � ��q� �1�1�1���9�9���
�
�Q�A�f�
�5�5�5��2�Y�Y�
����q�a���7�7�7��B�i�i������q���8�8�8�
�b����� � ��q� �1�1�1��"�I�I�	����a�Q�v��6�6�6�
�b����� � ��q� �1�1�1���H�1�1��1E�a�1H�
�
�
�
�	����s��3�3�3������)*���%�&�O�N�N�����
����0�0�	,�	,�F�A�s��(��4�>�B�B�B�E��J�J�F�1�q�5�A�+��);�J�<�<�<��J�J�|�T�%5�6�6�6�&+�D�J�s�y�y�{�{�#�#�Y�X�X�X�4�<�X�X�X�����L�)� ���)�

�
�
���	
�� � ��t�/?��/B� �C�C�C�"�4�<�0�0�	5�	5�I�A�v��&�*�*�6�4�3L�M�M�E��K�$�$�Q�e�$�4�4�4�4������6��2�2�2����������%�%�&9�4�>�J�J�J������D�$4�5�5�5������T�^�4�4�4������4�>�2�2�2������D�N�3�3�3��������7�7�7�	�&�&�q��&�3�3�3��O�T�]�3�
�2�D�4J�K�
����
�+4�T�?�?�	5�	5�'�Q�&��e�X��)�%�(�(�(�-�-�Q�A�c�-�J�J�J�����#,�$����E� %�h�/�D�J�s�O��J�J�z�4�?�3�3�3��J�J�|�3�%L�%L�%L�%L�%L�M�M�M��J�J�a�Q�t�J�4�4�4�4�	
��
�
�l�D�O�4�4�4�!'��j�$�.��!
�!
�!
���	
��"�"��"�/�/�/�%��i���a� 
� 
� 
���	
��!�!�w�!�/�/�/�$����+�A�.��+�A�.�	
�
�
���#�����+�A�.���

�
�
���	
��� � � �����G��,�,�,��� � �f� �-�-�-�-�-r8c����t�j��}t|d���}|�dd�jd���|���|�dd�fd��	��|�d
d�fd��	��|���|�dd
�j�	��|���|�dd�jd���|�dd|���t|d���}�jj	dd�D]g}t�j��}|�jvsJ�|�j|<|�jvr|�
d��|�|d|�j����h|�dd|���t|d���}�jj	dd�D]!}|�d|z|f�fd�	����"|����jj	dd�D]!}|�d|z|f�fd�	����"|�dd|���t|d���}|�dd�j�	��|�dd�jd���|�dd|����j�d �j���j�|�!��dS)"Nr)�tearoffr'�Return)rQ�	underliner��acceleratorzChange Server Index�c�.����d��S)Nrr#rJs�r6r%z*DownloaderGUI._init_menu.<locals>.<lambda>s���D�O�O�E�2�2�r8)rQrWr�zChange Download Directoryc�.����d��S)Nr�r#rJs�r6r%z*DownloaderGUI._init_menu.<locals>.<lambda>s���D�O�O�N�;�;�r8zShow Logr�ExitrzCtrl-x�File)rQrW�menur)rQrW�variabler��Viewz
Sort by %sc�:���j�|d��S)N�	ascending�r��sort_by�rkr3s �r6r%z*DownloaderGUI._init_menu.<locals>.<lambda>8s���$�+�*=�*=�a��*M�*M�r8)rQr�zReverse sort by %sc�:���j�|d��S)N�
descendingrcres �r6r%z*DownloaderGUI._init_menu.<locals>.<lambda>?s���$�+�*=�*=�a��*N�*N�r8�Sort�About�Instructions�F1�Helpz<F1>)r^)rr��add_commandr<�
add_separator�	_show_logr��add_cascader��column_namesr	r��INITIAL_COLUMNS�set�add_checkbuttonr��about�helpr��config)r3�menubar�filemenu�viewmenur�var�sortmenu�helpmenus`       r6r�zDownloaderGUI._init_menus�����t�x�.�.�����+�+�+�������4�>�x�	�	
�	
�	
�	��� � � ����'��2�2�2�2�	�	
�	
�	
�
	���-��;�;�;�;�	�	
�	
�	
�
	��� � � ����:��D�N��S�S�S���� � � �����A�t�|��	�	
�	
�	
�	���&�A�H��E�E�E�
���+�+�+���k�.�q�r�r�2�	�	�F����"�"�C���!2�2�2�2�2�(+�D��f�%���-�-�-�����
�
�
��$�$���C��AU�
%�
�
�
�
�	���&�A�H��E�E�E�
���+�+�+���k�.�q�r�r�2�	�	�F�� � �"�V�+�"(�M�M�M�M�M�
!�
�
�
�
�	��� � � ��k�.�q�r�r�2�	�	�F�� � �*�V�3�"(�N�N�N�N�N�
!�
�
�
�
�	���&�A�H��E�E�E����+�+�+�����7�a����L�L�L���� �A�t�y�d�	�	
�	
�	
�	���&�A�H��E�E�E���
�
�f�d�i�(�(�(�����W��%�%�%�%�%r8c���|j���D]N\}}|���r|j�|���4|j�|���OdSrC)r�r�r[r��show_column�hide_column)r3rr{s   r6r�zDownloaderGUI._select_columnsMsp��!�.�4�4�6�6�	0�	0�M�V�S��w�w�y�y�
0���'�'��/�/�/�/���'�'��/�/�/�/�		0�	0r8c�4�|j���	|���nN#t$r}t	d|��Yd}~n/d}~wt
$r}t	d|j��Yd}~nd}~wwxYw|j�d��dS)Nr�r�r)	r�rOr�rrrr�r�r��r3rs  r6rDzDownloaderGUI._refreshTs�����#�#�%�%�%�	>����������	6�	6�	6��1�1�5�5�5�5�5�5�5�5������	>�	>�	>��2�A�H�=�=�=�=�=�=�=�=�����	>��������1�����s �0�
A;�A�
A;�A6�6A;c��|���|j|\}}d|d<d|d<|���dS)N�normal�stater+r�)rBrAr:)r3�info_keyrSrRs    r6r$zDownloaderGUI._info_edit^sF�������� �J�x�0����!��g��"��h��
���
�
�
�
�
r8Nc��|j}|j���D]R\}}|ddkr�|�|j|ur|jdkr|}�+d|d<d|d<||������S|���dS)Nr��disabledrVr r�)r�rAr��widget�keysymr[r:)r3rr:rSrRs     r6rBzDownloaderGUI._info_savees������#�z�0�0�2�2�	&�	&�O�E�8��W�~��+�+���}���U�!2�!2�q�x�8�7K�7K����!+��g��"*��h���������%�%�%�%�
���
�
�
�
�
r8c��|jj|�d��r>t|t��rd|zS|dkrd|dzzS|dkrd|dzzSd	|d
zzS|dvrt	|��Sd|zS)Nr�z  %siz	  %.1f KBg�@i@z	  %.1f MBg0Az	  %.1f GBg�A)rr)r�rqr%r;r<)r3r	�col�vals    r6r5zDownloaderGUI._table_reprfuncrs����;�#�C�(�1�1�&�9�9�	7��#�s�#�#�
7���|�#��w���"�c�I�o�6�6��w���"�c�I�o�6�6�"�c�I�o�6�6��'�>�>��s�8�8�O��C�<�r8c��||jjkrdS	||j_|���n4#t$r'}t	dt|����Yd}~nd}~wwxYw|���dS)NzError Setting Server Index)r�rr�r�rr<r�)r3rrs   r6r�zDownloaderGUI._set_url�s����$�(�,����F�	<��D�H�L����������	<�	<�	<��2�C��F�F�;�;�;�;�;�;�;�;�����	<�����������s� 5�
A&�A!�!A&c�2�|jj|krdS||j_	|���nN#t$r}t	d|��Yd}~n/d}~wt
$r}t	d|j��Yd}~nd}~wwxYw|���dS)Nr�r�)r�r�r�rrrr�r�)r3r�rs   r6r�zDownloaderGUI._set_download_dir�s����8� �L�0�0��F�!-����	>����������	6�	6�	6��1�1�5�5�5�5�5�5�5�5������	>�	>�	>��2�A�H�=�=�=�=�=�=�=�=�����	>�����������s �5�
B�A�
B�!A;�;Bc��td|jj��|j���D] \}}d|d<|�dd���!|jdd�d|jj��|jdd�d|jj��|j���D]
\}}d|d<�dS)	Nzshowing infor�r�r�endrr�r�)r�r�rrAr��delete�insertr�)r3rS�cbs   r6r�zDownloaderGUI._show_info�s���
�n�d�h�l�+�+�+���*�*�,�,�	#�	#�I�E�2�%�E�'�N��L�L��E�"�"�"�"��
�5��!��#�#�A�t�x�|�4�4�4��
�>�"�1�%�,�,�Q���0E�F�F�F���*�*�,�,�	(�	(�I�E�2�'�E�'�N�N�	(�	(r8c��t|j��D]�\}}|���|jkr�|dkr�|j|dz
���|_	|���cS#t
$r}t
d|��Yd}~��d}~wt$r}t
d|j��Yd}~��d}~wwxYw��dS)Nrrr�r�)	rr0r��_tabr�rrrr��r3rr
rPs    r6r>zDownloaderGUI._prev_tab�s�����0�0�	F�	F�F�A�s��y�y�{�{�d�i�'�'�A��E�E� �O�A��E�2�8�8�:�:��	�F��+�+�-�-�-�-�-�� �>�>�>��9�1�=�=�=�=�=�=�=�=������F�F�F��:�A�H�E�E�E�E�E�E�E�E�����F�����	F�	Fs$�%A;�;
C�B�
C�'C�Cc���t|j��D]�\}}|���|jkr�|t	|j��dz
kr�|j|dz���|_	|���cS#t$r}td|��Yd}~��d}~wt$r}td|j
��Yd}~��d}~wwxYw��dS)Nrr�r�)rr0r�r�r�r1r�rrrr�r�s    r6r?zDownloaderGUI._next_tab�s�����0�0�	F�	F�F�A�s��y�y�{�{�d�i�'�'�A��T�Z���1�1D�,E�,E� �O�A��E�2�8�8�:�:��	�F��+�+�-�-�-�-�-�� �>�>�>��9�1�=�=�=�=�=�=�=�=������F�F�F��:�A�H�E�E�E�E�E�E�E�E�����F�����	F�	Fs$�:B�
C�B/�/
C�<C�Cc��|jd���|_	|���dS#t$r}td|��Yd}~dSd}~wt$r }td|j��Yd}~dSd}~wwxYw)Nrr�r�)r�r�r�r�rrrr�)r3�eventrs   r6r3zDownloaderGUI._select_tab�s����L��(�.�.�0�0��	�	>�����������	6�	6�	6��1�1�5�5�5�5�5�5�5�5�5������	>�	>�	>��2�A�H�=�=�=�=�=�=�=�=�=�����	>���s!�<�
B	�A�
B	�)B�B	r�c����j���}�j����jdkr�j���}n��jdkr�j���}n\�jdkr�j���}n7�jdkr�j���}nJd�jz����fd�|D��}�j�	|���j
���D]k\}}|�jkr.|��j
d�j
d�	���>|��jd�jd�	���l�j�d
d��������j�|���jjd
�jjjg�jj����R��jjd�jjjg�jj����R�dS)Nzall packagesr�r�r�rzbad tab value %rc�:��g|]}��|����Sr:)�_package_to_columnsrs  �r6r^z-DownloaderGUI._fill_table.<locals>.<listcomp>�s'���A�A�A�4��(�(��.�.�A�A�Ar8r�rr�r�rb)�order�i,)r��selected_rowrMr�r�rVr�r�r�rtr1r�r��_FRONT_TAB_COLOR�_BACK_TAB_COLORrd�_color_tabler�r��after�
_scrollbarrs�_mlb�yview)r3r�r��rowsrPrQs`     r6r�zDownloaderGUI._fill_table�sN����{�/�/�1�1����������9��&�&��H�%�%�'�'�E�E�
�Y�)�
#�
#��H�$�$�&�&�E�E�
�Y�(�
"�
"��H�O�O�%�%�E�E�
�Y�-�
'�
'��H�(�(�*�*�E�E�4�(�4�9�4�4�4�4�A�A�A�A�5�A�A�A������4� � � ��*�*�*�,�,�
	�
	�J�C���d�i������#�4�Q�7�#�4�Q�7� �����
���#�3�A�6�#�3�A�6� �����
	
����L���<�<�<�����������<�(�(�(�	����s�D�K�2�6�R���9I�9O�9O�9Q�9Q�R�R�R�R�����s�D�K�2�6�R���9I�9O�9O�9Q�9Q�R�R�R�R�R�Rr8c���tt|j����D]5}|j�|j|df��}||j|df<�6|���dS)Nr�r�)r
r�r�r�r�r�)r3�row_numr�s   r6�_update_table_statusz"DownloaderGUI._update_table_status�sn���S���-�-�.�.�	4�	4�G��X�_�_�T�[��,�1F�%G�H�H�F�-3�D�K���)�*�*��������r8c�b���jr
�j|�S�fd�tt�j����D��}�j���}|s|��j|dfg}�j�|�jj��}d�_	��
||��dS)Nc�T��g|]$}�j|dfdk��j|df��%S�rrr��r��r\r	r3s  �r6r^z+DownloaderGUI._download.<locals>.<listcomp>��E���
�
�
���{�3��6�"�b�(�(�
�K��\�)�*�(�(�(r8r�r)r��_download_threadedr
r�r�r�r�r�r�r��_download_cb)r3r�marked�	selection�
download_iters`    r6r<zDownloaderGUI._download�s������	/�*�4�*�A�.�.�
�
�
�
��S���-�-�.�.�
�
�
��
�K�,�,�.�.�	��	<�)�/��k�)�\�"9�:�;�F���.�.�v�t�x�7L�M�M�
�������-��0�0�0�0�0r8c�n��	t|��}nP#t$rC�����j�d�jd��}|�jd<YdSwxYw�fd�}t|t��r��|j	���n�t|t��rM||j��|j���
|jj����d��dSt|t��r*|d|jjz���xjdz
c_�nJt|t$��r|d|jjz���nt|t&��r|d|jjz��n�t|t(��r|d	|jjz��n�t|t*��r|d
|jjz��n�t|t.��rH�xjdzc_|d|jjz����|jj��n4t|t2��r��|jj���j��j�j||��}|�jd<dS)Nr�rr�c�F��|�jd<��|��dS�Nr�rFr��r,r3s �r6r6z(DownloaderGUI._download_cb.<locals>.show�#���*+�D���'��I�I�a�L�L�L�L�Lr8zDownloading collection %srzDownloading package %s�Package %s is up-to-date!�Finished downloading %r.�Unzipping %s�#Finished downloading collection %r.)�next�
StopIterationr�r�r��_show_progressr�r;r�r�r�r�rt�_selectrrfrir�rqr�r�r�r.rn�_clear_markry�	_DL_DELAYr�)r3r��idsr��afteridr6s`     r6r�zDownloaderGUI._download_cbs����	��}�%�%�C�C���	�	�	��%�%�'�'�'��h�n�n�R��)<�a�@�@�G�,3�D�M�.�)��F�F�	����	�	�	�	�	��c�?�+�+�	-������-�-�-�-�
��\�
*�
*�	-��D�������{�&����S�[�^�,�,�,�����%�%�%��F�
��3�
4�
4�	-��D�,�s�~�/@�@�A�A�A�����!����
��0�
1�
1�	-��D�)�C�K�N�:�;�;�;�;�
��_�
-�
-�
	-��D�,�s�{�~�=�>�>�>�>���2�
3�
3�		-��D�+�c�k�n�<�=�=�=�=�
��.�
/�
/�	-��D��#�+�"6�6�7�7�7�7�
��4�
5�
5�	-�����!����D�6���9J�J�K�K�K����S�^�.�/�/�/�/�
��1�
2�
2�	-����S�[�^�,�,�,��(�.�.����1B�M�SV�W�W��(/��
�n�%�%�%s��A	A �A c��tt|j����D]2}|j|df|kr|j�|��dS�3dS)Nr�)r
r�r�r��r3rr	s   r6r�zDownloaderGUI._select8sf����T�[�)�)�*�*�	�	�C��{�3��,�-��3�3���"�"�3�'�'�'����4�	�	r8c�8�tt|j����D]w}|j|j|df\}}d\}}|j�|||||���|j�|d|jd|jd����xdS)Nr�)�black�white)r�selectforegroundr��selectbackgroundrrr�)r
r�r��
_ROW_COLOR�	rowconfig�
itemconfigurer7)r3r	�bg�sbg�fg�sfgs      r6r�zDownloaderGUI._color_table>s�����T�[�)�)�*�*�
	�
	�C��o�d�k�#�x�-�&@�A�G�B��(�G�B���K�!�!���!$��!$�
"�
�
�
�
�K�%�%��Q�4�#3�A�#6�4�CS�TU�CV�
&�
�
�
�
�
	�
	r8c��tt|j����D]!}|j|df|krd|j|df<�"dS)Nr�rr�r
r�r�r�s   r6r�zDownloaderGUI._clear_markOsU����T�[�)�)�*�*�	)�	)�C��{�3��,�-��3�3�&(���C��F�#��	)�	)r8c�f�tt|j����D]}d|j|df<�dS)Nr�rr�)r3rr	s   r6r@zDownloaderGUI._mark_allTs@����T�[�)�)�*�*�	&�	&�C�"%�D�K��Q����	&�	&r8c���|j���}|dkr0|j|ddkr
d|j|df<nd|j|df<|j�d���dS)Nrrr�r)r)r�r�r�)r3rr�s   r6r=zDownloaderGUI._table_markXsw���K�,�,�.�.�	���>�>��{�9�%�a�(�B�.�.�,.���I�q�L�)�)�,/���I�q�L�)�������#�#�#�#�#r8c�f�d�|j��}t|jd|��dS)Nr�zNLTK Downloader Log)r-r�rr�)r3rs  r6rozDownloaderGUI._show_logas1���y�y��+�,�,�����0�$�7�7�7�7�7r8c��g}t|j��D]�\}}|dkr|�d���!|dkr|�|j���B|dkr.|�|j�|�����v|����dd��}|�t||d������|S)z�
        Given a package, return a list of values describing that
        package, one for each column in ``self.COLUMNS``.
        rrr�r�r�rYzn/a)	rr4r�rr�r�r��replacer�)r3r�r	�column_index�column_name�attrs      r6r�z!DownloaderGUI._package_to_columnses���
��)2�4�<�)@�)@�		6�		6�%�L�+��q� � ��
�
�2�������,�,��
�
�3�6�"�"�"�"���(�(��
�
�4�8�?�?�3�/�/�0�0�0�0�"�(�(�*�*�2�2�3��<�<���
�
�7�3��e�4�4�5�5�5�5��
r8c�X�|jrdS|j���d|_dS)NT)r�r�r�r�s  r6r�zDownloaderGUI.destroy{s0���?�	��F������������r8c��|j�6|j���D]}|j�|���|jr|jr|���|j���dSrC)	r�r�r��after_cancelr�r��_abort_downloadr�rM)r3rr�s   r6r�zDownloaderGUI._destroy�s����8���=�/�/�1�1�
/�
/����%�%�g�.�.�.�.���	#��!2�	#�� � �"�"�"�
	
����!�!�!�!�!r8c�*�|jj|i|��dSrC)r�r�)r3r��kwargss   r6r�zDownloaderGUI.mainloop�s"������4�*�6�*�*�*�*�*r8an    This tool can be used to download a variety of corpora and models
    that can be used with NLTK.  Each corpus or model is distributed
    in a single zip file, known as a "package file."  You can
    download packages individually, or you can download pre-defined
    collections of packages.

    When you download a package, it will be saved to the "download
    directory."  A default download directory is chosen when you run

    the downloader; but you may also select a different download
    directory.  On Windows, the default download directory is


    "package."

    The NLTK downloader can be used to download a variety of corpora,
    models, and other data packages.

    Keyboard shortcuts::
      [return]	 Download
      [up]	 Select previous package
      [down]	 Select next package
      [left]	 Select previous tab
      [right]	 Select next tab
    c���	t|jd|j���dd���dS#t|jd|j���d���YdSxYw)NzHelp: NLTK Downloaderr��fixed)r�rr)rr��HELPr@r�s  r6rvzDownloaderGUI.help�s���		U����'��	���!�!���
�
�
�
�
�
��	U��T�X�6��	���8I�8I�QS�T�T�T�T�T�T�T���s	�04�1A(c��d}d}	ddlm}|||������dS#t$rt	|j||��YdSwxYw)Nz'NLTK Downloader
Written by Edward LoperzAbout: NLTK Downloaderr)�Message)r�r�)�tkinter.messageboxr�r6�ImportErrorrr�)r3r�ABOUT�TITLEr�s     r6ruzDownloaderGUI.about�s���?��(��	-�2�2�2�2�2�2��G�E��/�/�/�4�4�6�6�6�6�6���	-�	-�	-��T�X�u�e�,�,�,�,�,�,�	-���s�%-� A�Ac�D�|j}t|d��t|d��}}tdt|d��dz|jz��D]Z}|�||jzdzd||jz|z
dz
|dz|jddt|d	zd
z
��dzzz����[|�d
��|�d
d���|�d|�	dddd|j
d�����dS)Nr�rrrr�i��z	#%02x0000rr
r�r�)r�r��gradient�hidden�r��redbox)r�)rHr r
�_gradient_width�create_line�abs�
addtag_all�
itemconfig�addtag_withtag�create_rectanglerG)r3rkr�rr
s     r6rIzDownloaderGUI._init_progressbar�sB������A�g�J����Q�x�[�)9�)9�v���q�3�q��z�?�?�Q�.�4�3G�G�H�H�	�	�A�
�M�M��D�(�(�2�-���D�(�(�6�1�B�6�����*� �B��Q��U�Q�Y���"�)<�$<�=�

�
�
�
�
�	
���Z� � � �	���Z�x��0�0�0�	
����a�(�(��A�q�!�$�:N�q�:Q�(�R�R�	
�	
�	
�	
�	
r8c�8�|j}|�2|�ddddd��|�dd���dSt|d��t|d��}}|t|��zdzd	z}|�ddd||d	z��dS)
Nr�rr�r�r�r�rr	r)rH�coordsr�r )r3�percentrkr�rr�s      r6r�zDownloaderGUI._show_progress�s�������?�
�H�H�X�q�!�Q��*�*�*�
�L�L��8�L�4�4�4�4�4���'�
�O�O�S��8��-=�-=�6�E��#�e�*�*�$��+�a�/�A�
�H�H�X�q�!�Q���
�3�3�3�3�3r8c��|j}|js|�dd���dS|�dd���|�d��\}}}}|dkr#|�d|jdzdz
d��n|�dd	d��|j�d
|j��}||j	d<dS)Nr�r�r�r�i����r
rrr;r*�_progress_alive)
rHr�r��bbox�mover�r�r�rr�)r3rk�x1�y1�x2�y2r�s       r6rzDownloaderGUI._progress_alive�s������� �
	7�
�L�L��8�L�4�4�4�4�4�
�L�L��8�L�4�4�4��V�V�J�/�/�N�B��B���T�z�z����z�D�$8�1�$<��#A�1�E�E�E�E����z�2�q�)�)�)��h�n�n�S�$�*>�?�?�G�/6�D�M�+�,�,�,r8c�f���jr����dSd�jd<�fd�tt	�j����D��}�j���}|s|��j|dfg}t�jj	�jj
��}�jgksJ��jgksJ���
||�j�j�j�����d�_d�_��������dS)N�Cancelrc�T��g|]$}�j|dfdk��j|df��%Sr�r�r�s  �r6r^z4DownloaderGUI._download_threaded.<locals>.<listcomp>
r�r8r�rT)r�r�rCr
r�r�r�r�r�rr�r�r��_DownloadThreadr��startr��_monitor_message_queuer)r3rr�r��dss`    r6r�z DownloaderGUI._download_threadeds`�����	�� � �"�"�"��F�)1���f�%�
�
�
�
��S���-�-�.�.�
�
�
��
�K�,�,�.�.�	��	<�)�/��k�)�\�"9�:�;�F�
����d�h�&;�
<�
<���'�2�-�-�-�-��)�R�/�/�/�/���������$��&�	
�	
��%�'�'�'���� ����#�#�%�%�%�	
�������r8c��|jrN|j���|j�d��|j���dSdS)N�abort)r�r��acquirer�r��releaserJs r6r�zDownloaderGUI._abort_download,s^����	*���'�'�)�)�)��&�-�-�g�6�6�6���'�'�)�)�)�)�)�	*�	*r8c��eZdZd�Zd�ZdS)�DownloaderGUI._DownloadThreadc��||_||_||_||_||_t
j�|��dSrC)�data_serverr��lock�
message_queuerr��Threadr7)r3rr�rrrs      r6r7z&DownloaderGUI._DownloadThread.__init__3sD��*�D���D�J��D�I�!.�D���D�J���%�%�d�+�+�+�+�+r8c���|j�|j��D]�}|j���|j�|��|jr6|j�d��|j���dS|j�����|j���|j�d��|j���dS)N�aborted�finished)	rr�r�rrrr�rrr�s  r6r�z!DownloaderGUI._DownloadThread.run;s����'�5�5�d�j�A�A�
$�
$���	�!�!�#�#�#��"�)�)�#�.�.�.��:���&�-�-�i�8�8�8��I�%�%�'�'�'��F�F��	�!�!�#�#�#�#��I��������%�%�j�1�1�1��I�������r8N)rMrNrOr7r�r:r8r6rr2s2������	,�	,�	,�	 �	 �	 �	 �	 r8rr	c����fd�}�j���sdS�jD�]D}|dks|dkr�����d�_d�jd<�jdd�=�jdd�=�j���|dkr!|d����d��n+�j	�
d�jd��}|�jd	<dSt|t��r��|j����t|t��rU||j��|j���|jj����d��d�_dSt|t(��r*|d
|jjz���xjdz
c_���t|t.��r>�j�|jj��|d|jjz�����t|t4��r|d
|jjz����t|t6��r|d|jjz����@t|t8��r|d|jjz����ot|t<��r|d|jjz�����t|t>��rI�xjdzc_|d|jjz���� |jj�����t|tB��r3������ |jj����F�jr
d�j"d<�jdd�=�j����j	�
�j#�j$��}|�jd	<dS)Nc�F��|�jd<��|��dSr�r�r�s �r6r6z2DownloaderGUI._monitor_message_queue.<locals>.showLr�r8rrFr'rzDownload aborted!r	r
r:rzDownloading package %rr�r�r�zFinished installing %sr�zAborting download...)%r�rr�r�r�rCr�rr�r�r�r�r;r�r�r�r�rtr�rrfrir�rqr�rOr�r�r�r.r�rnr�ryrF�_MONITOR_QUEUE_DELAYr
)r3r6r�r�s`   r6r
z$DownloaderGUI._monitor_message_queueKs
���	�	�	�	�	�
�"�*�*�,�,�	��F��+�4	1�4	1�C��j� � �C�9�$4�$4��)�)�+�+�+�$)��!�0:��%�f�-��,�Q�Q�Q�/��.�q�q�q�1��#�+�+�-�-�-��)�#�#��D�,�-�-�-��'�'��-�-�-�-�"�h�n�n�S�$�2E�t�L�L�G�>E�D�M�":�;�����C��1�1� 
1��#�#�C�L�1�1�1�1��C��.�.�
1���S�[�!�!�!��;�*��L�L����0�0�0��#�#�D�)�)�)�$)��!�����C�!7�8�8�
1���0�3�>�3D�D�E�E�E�� � �A�%� � � ��C�!4�5�5�
1���+�+�C�K�N�;�;�;���-����>�?�?�?�?��C��1�1�
1���0�3�;�>�A�B�B�B�B��C�!6�7�7�
1���/�#�+�.�@�A�A�A�A��C�!2�3�3�

1���^�c�k�&:�:�;�;�;�;��C�!3�4�4�
1���-����>�?�?�?�?��C�!8�9�9�
1�� � �A�%� � ���:�S�^�=N�N�O�O�O�� � ���!2�3�3�3�3��C�!5�6�6�
1��)�)�+�+�+�� � ����0�0�0��
�%�	A�*@�D���'�
�$�Q�Q�Q�'���#�#�%�%�%��(�.�.��!:�D�<W�X�X��29��
�.�/�/�/r8�TrC)FrMrNrOrPr4rr8r9rrrkr�r�r�r�r�r�r�r7r�r�rGr2r7r�r�r�r�rDr$rBr5r�r�r�r>r?r3r��_rowsr�r�r<r�r�r�r�r�r@r=ror�r�r�r�r��dedentr�rvrur�rIr�rr�r�r�rrrr
r:r8r6r�r�s���������
�
�
�G�D��Q��Q�?�?�N�C�
������
��M�<��� �C�B�B�O�A������G�|�|�|�|�|�
�����G�|�|�|�|�|�
�����G�|�|�|�|�|�'�O�	��.���,���*�� �"2�	�J�#�K�(��&�O�&�O�$�I�/5�/5�/5�/5�b
�
�
�m.�m.�m.�^C&�C&�C&�J0�0�0����������� � � � ���
�
�
�(�(�(�	F�	F�	F�	F�	F�	F�>�>�>��D��E�'S�'S�'S�R���1�1�1�&�I�*0�*0�*0�X������")�)�)�
&�&�&�$�$�$�8�8�8����,���"�"�"�+�+�+��8�?�	���D�:U�U�U�-�-�-��O�
�
�
�(4�4�4�7�7�7�$*�*�*�X*�*�*� � � � � �)�*� � � �.��J:�J:�J:�J:�J:r8r�c��t|t��r7t|d��5}t|��cddd��S#1swxYwYt|��S)zz
    Calculate and return the MD5 checksum for a given file.
    ``file`` may either be a filename or an open stream.
    �rbN)r;r<r�_md5_hexdigest)r0r(s  r6r\r\�s���
�$����*�
�$��
�
�	*��!�&�)�)�	*�	*�	*�	*�	*�	*�	*�	*�	*�	*�	*�	*����	*�	*�	*�	*��$���s�A�A�	Ac��t��}	|�d��}|sn|�|���.|���S)NTr)rr!r2�	hexdigest)�fp�
md5_digestr+s   r6r%r%�sY�����J�!����	�"�"���	�����%� � � �	!�
���!�!�!r8c�v�t|||��D]&}t|t��rt|����'dS)z\
    Extract the contents of the zip file ``filename`` into the
    directory ``root``.
    N)r&r;r�r�)r.�rootrr�s    r6r0r0�sN��
�x��w�7�7�%�%���g�|�,�,�	%��G�$�$�$�	%�%�%r8c#�K�|rctj�dtj�|��dz��tj���	tj|��}nT#tj	$r}t|d��V�Yd}~dSd}~wt$r}t||��V�Yd}~dSd}~wwxYw|�|��|rt��dSdS)Nr�rzError with downloaded zip file)r��stdoutr"r)r*r,�flush�zipfile�ZipFile�errorr�r��
extractallr�)r.r+r�zfrs     r6r&r&�s�������
����"�'�-�-��*A�*A�!�*D�D�E�E�E��
�������
�_�X�
&�
&�����=�����8�%E�F�F�F�F�F����������������8�Q�'�'�'�'�'����������������M�M�$������
�������s$�)A>�>C�
B%�%
C�2C
�
Cc���g}ttj�|d����D�]5\}}}tj|j��}|�d|�dtj�|j��d��}td�|���D����}|�	dd|z��|�	dd|j
z��|�	ddt|j��z��|�	d	|��|�d
��s|�	d
|��|�
|����7tttj�|d������}	t��}
||	zD]f}|�d��|
vr%t!d
|�d��z���|
�|�d�����gt%jd��}|�
t%jd����|d�t+|d������|�
t%jd����|d�t+|	d������t-|��|S)a}
    Create a new data.xml index file, by combining the xml description
    files for various packages and collections.  ``root`` should be the
    path to a directory containing the package xml and zip files; and
    the collection xml files.  The ``root`` directory is expected to
    have the following subdirectories::

      root/
        packages/ .................. subdirectory for packages
          corpora/ ................. zip & xml files for corpora
          grammars/ ................ zip & xml files for grammars
          taggers/ ................. zip & xml files for taggers
          tokenizers/ .............. zip & xml files for tokenizers
          etc.
        collections/ ............... xml files for collections

    For each package, there should be two files: ``package.zip``
    (where *package* is the package name)
    which contains the package itself as a compressed zip file; and
    ``package.xml``, which is an xml description of the package.  The
    zipfile ``package.zip`` should expand to a single subdirectory
    named ``package/``.  The base filename ``package`` must match
    the identifier given in the package's xml file.

    For each collection, there should be a single file ``collection.zip``
    describing the collection, where *collection* is the name of the collection.

    All identifiers (for both packages and collections) must be unique.
    rVrrc3�$K�|]}|jV��dSrC)�	file_size)r\�zf_infos  r6rzbuild_index.<locals>.<genexpr>�s%����K�K�'�G�-�K�K�K�K�K�Kr8r"z%sr!r#rrr�rzDuplicate UID: %sr�rc�,�|�d��S�NrrZrss r6r%zbuild_index.<locals>.<lambda>	s��7�;�;�t�;L�;L�r8r�c�,�|�d��Sr9rZrhs r6r%zbuild_index.<locals>.<lambda>	s�����PT�AU�AU�r8)�_find_packagesr)r*r-rVr.r,r�infolistrsrWr\r[r�r��_find_collectionsr��addr�Elementrtr��_indent_xml)
r+�base_urlrV�pkg_xmlr3r�zipstatrr"r��uidsr_�top_elts
             r6�build_indexrF�s���>�H�-�b�g�l�l�4��.L�.L�M�M�!�!����V��'�"�+�&�&���D�D�F�D�D�R�W�]�]�2�;�%?�%?��%B�D�D���K�K�R�[�[�]�]�K�K�K�K�K�
�	���O�T�M�%9�:�:�:����F�D�7�?�2�3�3�3����J��}�R�[�'A�'A� A�B�B�B����H�f�%�%�%��{�{�5�!�!�	$��K�K��s�#�#�#�	���� � � � ��(�����d�M�)J�)J�K�K�L�L�K��5�5�D��;�&�!�!���8�8�D�>�>�T�!�!��0�4�8�8�D�>�>�A�B�B�B�������$��� � � � ��!�+�.�.�G��N�N�;�&�z�2�2�3�3�3��A�J���f�X�+L�+L�M�M�M�N�N�N��N�N�;�&�}�5�5�6�6�6��A�J���f�[�.U�.U�V�V�V�W�W�W�������Nr8rc��t|��dkr�|jpd���dz|zdz|_|D]}t||dz���|dd�D]+}|jpd���dz|zdz|_�,|djpd���dz|z|d_dSdS)z�
    Helper for ``build_index()``: Given an XML ``ElementTree``, modify it
    (and its descendents) ``text`` and ``tail`` attributes to generate
    an indented tree, where each nested element is indented by 2
    spaces with respect to its parent.
    rrr�r<Nr)r�rr@r@�tail)r?r�r]s   r6r@r@ 	s����3�x�x�!�|�|��H�N��)�)�+�+�d�2�V�;�d�B����	.�	.�E���v��}�-�-�-�-���"��X�	K�	K�E��*�*��1�1�3�3�d�:�V�C�d�J�E�J�J��B���*��1�1�3�3�d�:�V�C��B�����
�|r8c���tj�tj�|��d��d�|�d���kr6td�|�d��������t�fd�|���D����rtd��d��d����d	S)
zr
    Helper for ``build_index()``: Perform some checks to make sure that
    the given package is consistent.
    rrrz&package identifier mismatch ({} vs {})c3�V�K�|]#}|�ko|��dz��V��$dS�rN�r��r\r�uids  �r6rz!_check_package.<locals>.<genexpr>=	s?�����
V�
V��D�C�K�:�����c�	� :� :�:�
V�
V�
V�
V�
V�
Vr8�Zipfile �..zip does not expand to a single subdirectory rN)	r)r*r+r,r[r�r�r�namelist)rB�zipfilenamer3rNs   @r6�_check_packagerS0	s����
�'�
�
�2�7�=�=��5�5�a�8�
9�
9�!�
<�C��{�{�4���C����4�;�;�G�K�K��<M�<M�s�S�S�
�
�	
�
�
V�
V�
V�
V����
�
�
V�
V�
V�V�V�
��j�"%�#�#�s�s�s�
,�
�
�	
�
�
r8c�t�tjddd|gtjtj���}|���\}}|jdks|s|sJtdtj�|��d�dtj
|�������|���d	S)
z�
    Helper for ``build_index()``: Calculate the subversion revision
    number for a given file (by using ``subprocess`` to run ``svn``).
    �svnr�z-v)r-r�rz#Error determining svn_revision for rr�r)�
subprocess�Popen�PIPE�communicate�
returncoder�r)r*r,r�r�)r.rer-r�s    r6�
_svn_revisionr[E	s���
	��	��$��)�����	�	�	�A�
�}�}����V�V��|�q���F��&���j��w�}�}�X�&�&�q�)�)�)�8�=��+@�+@�+@�
B�
�
�	
��<�<�>�>�!��r8c#�K�tj|��D]h\}}}|D]_}|�d��rHtj�||��}tj|�����V��`�idS)z�
    Helper for ``build_index()``: Yield a list of ElementTree.Element
    objects, each holding the xml for a single package collection.
    �.xmlN)r)r^r%r*r-rr=rs)r+�dirname�_subdirsrZr.�xmlfiles      r6r=r=X	s�����
%'�G�D�M�M�;�;� ���5��	;�	;�H�� � ��(�(�
;��'�,�,�w��9�9��!�'��0�0�8�8�:�:�:�:�:��	;�;�;r8c
#�D�K�ddlm}g}tj|��D�]~\}}}d�|||����}|D�]-}|�d���rvtj�||��}|dd�dz}		tj|	��}
n(#t$r}td|	�d	|����|�d}~wwxYw	tj|���
��}n(#t$r}td|�d	|����|�d}~wwxYwtj�|dd���d
�|�d���kr)td|�d���d
��d����t!�fd�|
���D����rtd��d��d����||
|fV����|�d��r�tj�|��d}
tj�||
dz��}tj�|��s"t)j|�d|
dz�d|
�d�d�����/	|�d����o#t$rY��|wxYwdS)a�
    Helper for ``build_index()``: Yield a list of tuples
    ``(pkg_xml, zf, subdir)``, where:
      - ``pkg_xml`` is an ``ElementTree.Element`` holding the xml for a
        package
      - ``zf`` is a ``zipfile.ZipFile`` for the package's contents.
      - ``subdir`` is the subdirectory (relative to ``root``) where
        the package was found (e.g. 'corpora' or 'grammars').
    r)�
_path_fromrr]Nr;rzError reading file z!
rrzpackage identifier mismatch (z vs r�c3�V�K�|]#}|�ko|��dz��V��$dSrKrLrMs  �r6rz!_find_packages.<locals>.<genexpr>�	sS���������S�[�C�����s��)C�)C�%C������r8rOrPz
 exists, but z' cannot be found! This could mean that z can not be downloaded.r)�
stacklevelz.svn)�nltk.corpus.reader.utilrbr)r^r-r%r*r/r0r�r�rr=rsr,r[rrQr+r�warnings�warnr)r+rbrVr^�subdirsrZ�relpathr.�xmlfilenamerRr3rrB�resourcenamerNs              @r6r;r;d	sX�����3�2�2�2�2�2��H�#%�7�4�=�=�3�3����%��(�(�:�:�d�G�4�4�5�5���+	�+	�H�� � ��(�(�*
� �g�l�l�7�H�=�=��)�#�2�#�.��7��Y� ���5�5�B�B�� �Y�Y�Y�$�%P�;�%P�%P�Q�%P�%P�Q�Q�WX�X�����Y����Y�)�/��<�<�D�D�F�F�G�G�� �Y�Y�Y�$�%P�;�%P�%P�Q�%P�%P�Q�Q�WX�X�����Y�����g�m�m�K����$4�5�5�a�8���;�;�t�$�$��+�+�$�*�$+�K�K��$5�$5�$5�$5�s�s�s�<��������� "���
�
�������%�*�58�S�S�#�#�#�?����
�r�7�*�*�*�*�*��"�"�6�*�*�	
�!�w�/�/��9�9�!�<�� �g�l�l�7�L�6�4I�J�J���w�~�~�k�2�2���M�#�V�V�,��2G�V�V�0<�V�V�V�#$������	��N�N�6�"�"�"�"���	�	�	��D�	����e3�3sB�B$�$
C	�.C�C	�
&C4�4
D�>D�D�8J�
J�Jc�R�tt�����dSrC)r��_downloaderr�r:r8r6�download_shellrn�	s"���K� � �$�$�&�&�&�&�&r8c�R�tt�����dSrC)r�rmr�r:r8r6�download_guirp�	s"���+���'�'�)�)�)�)�)r8c�8�t���dSrC)rmr2r:r8r6r2r2�	s���������r8�__main__)�OptionParserz-dz--dir�dirz!download package to directory DIR�DIR)�destrv�metavarz-qz--quietrB�
store_truezwork quietly)rv�action�defaultrvz-fz--forcerz"download even if already installedz-ez--exit-on-errorrCzexit if an error occursz-uz--urlr��NLTK_DOWNLOAD_URLzdownload server index url)rvrzrv)r�)r�r�rBrrC)r�rBrrCr r=)]rPr>rr)�shutilrVr�r�r�rnrfr/�hashlibr�	xml.etreerr��tkinterrrrrr	r
rrr
r�r�nltk.draw.tabler�nltk.draw.utilrr�r��urllib.errorrr�urllib.requestrrprrSrdrfrnrqryr}r�r�r�r�r�r�r�r�r�r�r�r\r%r0r&rFr@rSr[r=r;rmrArnrpr2rM�optparsers�parser�
add_optionr�r[�
parse_argsr�r�r��
downloader�pkg_idrtrBrrC�rvr:r8r6�<module>r�s���<�<�~Y�t��������	�	�	�	�
�
�
�
�����
�
�
�
���������������������������!�!�!�!�!�!�	��G�W�W�W�W�W�W�W�W�W�W�W�W�W�W�W�W�W�W�W�W�W�W�,�,�,�,�,�,�%�%�%�%�%�%�'�'�'�'�'�'�'�������G��H�H�H�����-�,�,�,�,�,�,�,�"�"�"�"�"�"�����Z(�Z(�Z(�Z(�Z(�Z(�Z(�Z(�z&+�&+�&+�&+�&+�&+�&+�&+�\!�!�!�!�!�!�!�!�
%�%�%�%�%�.�%�%�%�%�%�%�%�%�/�%�%�%������+���������,���������,���������-���������)���������*���������'���������$����#�#�#�#�#�$�#�#�#�!�!�!�!�!�'�!�!�!�)�)�)�)�)�0�)�)�)�b
(�b
(�b
(�b
(�b
(�b
(�b
(�b
(�Jb�b�b�b�b�b�b�b�JP:�P:�P:�P:�P:�P:�P:�P:�r � � �"�"�"�%�%�%�%�����4C�C�C�L
D�
D�
D�
D� 
�
�
�*���&	;�	;�	;�A�A�A�V�j�l�l�����'�'�'�*�*�*�����z���%�%�%�%�%�%�
�\�^�^�F�
�����
�
0�����������
���
�
���������
���
1�
���������
���
&�
���������
��
���2�3�3�
(������'�'�)�)�O�W�d���W�-E�F�F�F�J��
��		�		�F��$�$�!�$�[��m��m�%�3�%���B��U�{�{�w�4�{�����		�		�	��� ���-��-�!�/�		�	
�	
�	
�	
�	
�w�s�*A(�(	A4�3A4

Youez - 2016 - github.com/yon3zu
LinuXploit