403Webshell
Server IP : 54.37.205.81  /  Your IP : 216.73.216.76
Web Server : nginx/1.22.1
System : Linux vps-249481fa 6.1.0-50-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.176-1 (2026-07-02) x86_64
User : debian ( 1000)
PHP Version : 8.2.32
Disable Function : NONE
MySQL : OFF  |  cURL : ON  |  WGET : ON  |  Perl : ON  |  Python : OFF  |  Sudo : ON  |  Pkexec : OFF
Directory :  /lib/python3/dist-packages/nltk/chunk/__pycache__/

Upload File :
current_dir [ Writeable ] document_root [ Writeable ]

 

Command :


[ Back ]     

Current File : /lib/python3/dist-packages/nltk/chunk/__pycache__/util.cpython-311.pyc
�

 T�c�P��8�ddlZddlmZddlmZddlmZddlm	Z	d�ZGd�d��Z
d	�Z	dd
�Zej
d��Zdd�Zd�Z	dd�Zd�Zej
dej��Zej
d��Zd�Zgd�dfd�Zd�Zedkre��dSdS)�N)�accuracy)�map_tag)�	str2tuple)�Treec���g}g}|D]M}|�|�����}|t|��z
}|t|��z
}�Nt||��S)a|
    Score the accuracy of the chunker against the gold standard.
    Strip the chunk information from the gold standard and rechunk it using
    the chunker, then compute the accuracy score.

    :type chunker: ChunkParserI
    :param chunker: The chunker being evaluated.
    :type gold: tree
    :param gold: The chunk structures to score the chunker on.
    :rtype: float
    )�parse�flatten�tree2conlltags�	_accuracy)�chunker�gold�	gold_tags�	test_tags�	gold_tree�	test_trees      �1/usr/lib/python3/dist-packages/nltk/chunk/util.pyrrsr���I��I��/�/�	��M�M�)�"3�"3�"5�"5�6�6�	��^�I�.�.�.�	��^�I�.�.�.�	�	��Y�	�*�*�*�c�h�eZdZdZd�Zd�Zd�Zd�Zd�Zd�Z	dd	�Z
d
�Zd�Zd�Z
d
�Zd�Zd�Zd�ZdS)�
ChunkScorea;
    A utility class for scoring chunk parsers.  ``ChunkScore`` can
    evaluate a chunk parser's output, based on a number of statistics
    (precision, recall, f-measure, misssed chunks, incorrect chunks).
    It can also combine the scores from the parsing of multiple texts;
    this makes it significantly easier to evaluate a chunk parser that
    operates one sentence at a time.

    Texts are evaluated with the ``score`` method.  The results of
    evaluation can be accessed via a number of accessor methods, such
    as ``precision`` and ``f_measure``.  A typical use of the
    ``ChunkScore`` class is::

        >>> chunkscore = ChunkScore()           # doctest: +SKIP
        >>> for correct in correct_sentences:   # doctest: +SKIP
        ...     guess = chunkparser.parse(correct.leaves())   # doctest: +SKIP
        ...     chunkscore.score(correct, guess)              # doctest: +SKIP
        >>> print('F Measure:', chunkscore.f_measure())       # doctest: +SKIP
        F Measure: 0.823

    :ivar kwargs: Keyword arguments:

        - max_tp_examples: The maximum number actual examples of true
          positives to record.  This affects the ``correct`` member
          function: ``correct`` will not return more than this number
          of true positive examples.  This does *not* affect any of
          the numerical metrics (precision, recall, or f-measure)

        - max_fp_examples: The maximum number actual examples of false
          positives to record.  This affects the ``incorrect`` member
          function and the ``guessed`` member function: ``incorrect``
          will not return more than this number of examples, and
          ``guessed`` will not return more than this number of true
          positive examples.  This does *not* affect any of the
          numerical metrics (precision, recall, or f-measure)

        - max_fn_examples: The maximum number actual examples of false
          negatives to record.  This affects the ``missed`` member
          function and the ``correct`` member function: ``missed``
          will not return more than this number of examples, and
          ``correct`` will not return more than this number of true
          negative examples.  This does *not* affect any of the
          numerical metrics (precision, recall, or f-measure)

        - chunk_label: A regular expression indicating which chunks
          should be compared.  Defaults to ``'.*'`` (i.e., all chunks).

    :type _tp: list(Token)
    :ivar _tp: List of true positives
    :type _fp: list(Token)
    :ivar _fp: List of false positives
    :type _fn: list(Token)
    :ivar _fn: List of false negatives

    :type _tp_num: int
    :ivar _tp_num: Number of true positives
    :type _fp_num: int
    :ivar _fp_num: Number of false positives
    :type _fn_num: int
    :ivar _fn_num: Number of false negatives.
    c��t��|_t��|_t��|_t��|_t��|_|�dd��|_|�dd��|_|�dd��|_	|�dd��|_
d|_d|_d|_
d|_d|_d|_d	|_dS)
N�max_tp_examples�d�max_fp_examples�max_fn_examples�chunk_labelz.*rgF)�set�_correct�_guessed�_tp�_fp�_fn�get�_max_tp�_max_fp�_max_fn�_chunk_label�_tp_num�_fp_num�_fn_num�_count�
_tags_correct�_tags_total�_measuresNeedUpdate)�self�kwargss  r�__init__zChunkScore.__init__rs�������
�����
��5�5����5�5����5�5����z�z�"3�S�9�9����z�z�"3�S�9�9����z�z�"3�S�9�9���"�J�J�}�d�;�;��������������� ������#(�� � � rc�4�|jr�|j|jz|_|j|jz
|_|j|jz
|_t
|j��|_t
|j��|_t
|j��|_	d|_dSdS)NF)
r-rrrr!r �lenr'r(r)�r.s r�_updateMeasureszChunkScore._updateMeasures�s����#�	-��}�t�}�4�D�H��}�t�}�4�D�H��}�t�}�4�D�H��t�x�=�=�D�L��t�x�=�=�D�L��t�x�=�=�D�L�',�D�$�$�$�	-�	-rc	���|xjt||j|j��zc_|xjt||j|j��zc_|xjdz
c_d|_	t
|��}t
|��}n#t$rdx}}YnwxYw|xjt|��z
c_|xj
td�t||��D����z
c_
dS)aU
        Given a correctly chunked sentence, score another chunked
        version of the same sentence.

        :type correct: chunk structure
        :param correct: The known-correct ("gold standard") chunked
            sentence.
        :type guessed: chunk structure
        :param guessed: The chunked sentence to be scored.
        �T�c3�,K�|]\}}||k�dV��dS)r6Nr7)�.0�t�gs   r�	<genexpr>z#ChunkScore.score.<locals>.<genexpr>�s3����"
�"
��1�a�q�A�v�v�A�v�v�v�v�"
�"
rN)
r�
_chunksetsr*r&rr-r
�
ValueErrorr,r2r+�sum�zip)r.�correct�guessed�correct_tags�guessed_tagss     r�scorezChunkScore.score�s��	
�
�
��G�T�[�$�:K�L�L�L�
�
��
�
��G�T�[�$�:K�L�L�L�
�
����q����#'�� �	-�)�'�2�2�L�)�'�2�2�L�L���	-�	-�	-�+-�,�L�<�<�<�		-����
	
���C��-�-�-������c�"
�"
���l�;�;�"
�"
�"
�
�
�	
����s�+B
�
B�Bc�:�|jdkrdS|j|jzS)z�
        Return the overall tag-based accuracy for all text that have
        been scored by this ``ChunkScore``, using the IOB (conll2000)
        tag encoding.

        :rtype: float
        rr6)r,r+r3s rrzChunkScore.accuracy�s(����q� � ��1��!�D�$4�4�4rc�l�|���|j|jz}|dkrdS|j|zS)z�
        Return the overall precision for all texts that have been
        scored by this ``ChunkScore``.

        :rtype: float
        r)r4r'r(�r.�divs  r�	precisionzChunkScore.precision��?��	
�������l�T�\�)���!�8�8��1��<�#�%�%rc�l�|���|j|jz}|dkrdS|j|zS)z�
        Return the overall recall for all texts that have been
        scored by this ``ChunkScore``.

        :rtype: float
        r�r4r'r)rHs  r�recallzChunkScore.recall�rKr��?c��|���|���}|���}|dks|dkrdSd||zd|z
|zzzS)a�
        Return the overall F measure for all texts that have been
        scored by this ``ChunkScore``.

        :param alpha: the relative weighting of precision and recall.
            Larger alpha biases the score towards the precision value,
            while smaller alpha biases the score towards the recall
            value.  ``alpha`` should have a value in the range [0,1].
        :type alpha: float
        :rtype: float
        rr6)r4rJrN)r.�alpha�p�rs    r�	f_measurezChunkScore.f_measure�sd��	
�������N�N�����K�K�M�M����6�6�Q�!�V�V��1��E�A�I��U��a��/�0�0rc�j�|���t|j��}d�|D��S)z�
        Return the chunks which were included in the
        correct chunk structures, but not in the guessed chunk
        structures, listed in input order.

        :rtype: list of chunks
        c��g|]
}|d��S�r6r7�r9�cs  r�
<listcomp>z%ChunkScore.missed.<locals>.<listcomp>����%�%�%���!��%�%�%r)r4�listr!�r.�chunkss  r�missedzChunkScore.missed�s7��	
�������d�h����%�%�f�%�%�%�%rc�j�|���t|j��}d�|D��S)z�
        Return the chunks which were included in the guessed chunk structures,
        but not in the correct chunk structures, listed in input order.

        :rtype: list of chunks
        c��g|]
}|d��SrWr7rXs  rrZz(ChunkScore.incorrect.<locals>.<listcomp>�r[r)r4r\r r]s  r�	incorrectzChunkScore.incorrect�s7��	
�������d�h����%�%�f�%�%�%�%rc�B�t|j��}d�|D��S)z�
        Return the chunks which were included in the correct
        chunk structures, listed in input order.

        :rtype: list of chunks
        c��g|]
}|d��SrWr7rXs  rrZz&ChunkScore.correct.<locals>.<listcomp>r[r)r\rr]s  rrAzChunkScore.correct��'���d�m�$�$��%�%�f�%�%�%�%rc�B�t|j��}d�|D��S)z�
        Return the chunks which were included in the guessed
        chunk structures, listed in input order.

        :rtype: list of chunks
        c��g|]
}|d��SrWr7rXs  rrZz&ChunkScore.guessed.<locals>.<listcomp>r[r)r\rr]s  rrBzChunkScore.guessedrerc�H�|���|j|jzS)NrMr3s r�__len__zChunkScore.__len__s#���������|�d�l�*�*rc�F�dtt|����zdzS)z`
        Return a concise representation of this ``ChunkScoring``.

        :rtype: str
        z<ChunkScoring of z chunks>)�reprr2r3s r�__repr__zChunkScore.__repr__s ��#�T�#�d�)�)�_�_�4�z�A�Arc���dd|���dzd�d�zd|���dzd�d�zd|���dzd�d�zd|���dzd�d	�zS)
a-
        Return a verbose representation of this ``ChunkScoring``.
        This representation includes the precision, recall, and
        f-measure scores.  For other information about the score,
        use the accessor methods (e.g., ``missed()`` and ``incorrect()``).

        :rtype: str
        zChunkParse score:
z    IOB Accuracy: rz5.1fz%%
z    Precision:    z    Recall:       z    F-Measure:    z%%)rrJrNrTr3s r�__str__zChunkScore.__str__s���
"�D�D�M�M�O�O�c�$9�D�D�D�D�
F�E�D�N�N�$4�$4�s�$:�E�E�E�E�
G�C�D�K�K�M�M�C�$7�B�B�B�B�
D�D�D�N�N�$4�$4�s�$:�C�C�C�C�	
E�	
rN)rO)�__name__�
__module__�__qualname__�__doc__r0r4rErrJrNrTr_rbrArBrirlrnr7rrrr3s�������<�<�|)�)�)�&-�-�-�
�
�
�:
5�
5�
5�&�&�&�&�&�&�1�1�1�1�&
&�
&�
&�	&�	&�	&�&�&�&�&�&�&�+�+�+�B�B�B�
�
�
�
�
rrc�T�d}g}|D]�}t|t��rwtj||�����r+|�||f|���f��|t|�����z
}��|dz
}��t|��S)Nrr6)
�
isinstancer�re�match�label�append�freezer2�leavesr)r:�countr�posr^�childs      rr=r=2s���
�C�
�F������e�T�"�"�	��x��U�[�[�]�]�3�3�
>��
�
��s�|�U�\�\�^�^�<�=�=�=��3�u�|�|�~�~�&�&�&�C�C��1�H�C�C��v�;�;�r�NP�S�/c��tjd��}t|g��g}|�|��D�]Y}|���}	|	ddkryt|��dkr%t
d|���d�����t|g��}
|d�|
��|�|
����|	ddkrMt|��d	kr%t
d
|���d�����|�	����|�|d�|	����t|	|��\}}|r|rt|||��}|d�||f����[t|��dkr t
dt|��d�����|dS)
aB
    Divide a string of bracketted tagged text into
    chunks and unchunked tokens, and produce a Tree.
    Chunks are marked by square brackets (``[...]``).  Words are
    delimited by whitespace, and each word should have the form
    ``text/tag``.  Words that do not contain a slash are
    assigned a ``tag`` of None.

    :param s: The string to be converted
    :type s: str
    :param chunk_label: The label to use for chunk nodes
    :type chunk_label: str
    :param root_label: The label to use for the root of the tree
    :type root_label: str
    :rtype: Tree
    z\[|\]|[^\[\]\s]+r�[r6zUnexpected [ at char �d����]�zUnexpected ] at char NzExpected ] at char )ru�compiler�finditer�groupr2r>�startrx�poprr)
�sr�
root_label�sep�
source_tagset�
target_tagset�WORD_OR_BRACKET�stackrv�text�chunk�word�tags
             r�tagstr2treer�?s���(�j�!4�5�5�O�
�*�b�
!�
!�"�E� �)�)�!�,�,�.�.���{�{�}�}����7�c�>�>��5�z�z�Q��� �!J������!J�!J�!J�K�K�K���b�)�)�E��"�I���U�#�#�#��L�L������
�!�W��^�^��5�z�z�Q��� �!J������!J�!J�!J�K�K�K��I�I�K�K�K�K��{��b�	� � ��&�&�&�&�%�d�C�0�0�	��c� �E�]�E�!�-���D�D�C��b�	� � �$���-�-�-�-�
�5�z�z�Q����9�s�1�v�v�9�9�9�:�:�:���8�Orz(\S+)\s+(\S+)\s+([IOB])-?(\S+)?�r~�PP�VPc��t|g��g}t|�d����D�]!\}}|���s�t�|��}|�t
d|d�����|���\}}}	}
|�|
|vrd}	|	dko|
|d���k}|	dvs|r't|��d	kr|�
��|	d
ks|r@t|
g��}|d�|��|�|��|d�||f����#|dS)a*
    Return a chunk structure for a single sentence
    encoded in the given CONLL 2000 style string.
    This function converts a CoNLL IOB string into a tree.
    It uses the specified chunk types
    (defaults to NP, PP and VP), and creates a tree rooted at a node
    labeled S (by default).

    :param s: The CoNLL string to be converted.
    :type s: str
    :param chunk_types: The chunk types to be converted.
    :type chunk_types: tuple
    :param root_label: The node label to use for the root.
    :type root_label: str
    :rtype: Tree
    �
NzError on line r��O�Ir��BOr��Br)r�	enumerate�split�strip�_LINE_RErvr>�groupsrwr2r�rx)
r��chunk_typesr�r��lineno�linervr�r��state�
chunk_type�
mismatch_Ir�s
             r�
conllstr2treer�usk��$�*�b�
!�
!�"�E�!�!�'�'�$�-�-�0�0�&�&�����z�z�|�|�	�����t�$�$���=��8�f�8�8�8�9�9�9�).�����&��s�E�:��"�z��'D�'D��E��c�\�E�j�E�"�I�O�O�4E�4E�&E�
��D�=�=�J�=��5�z�z�Q����	�	�����C�<�<�:�<���R�(�(�E��"�I���U�#�#�#��L�L�����	�b�	���$���%�%�%�%���8�Orc�N�g}|D]�}	|���}d}|D]O}t|t��rtd���|�|d|d||zf��d}�P�l#t
$r'|�|d|ddf��Y��wxYw|S)z�
    Return a list of 3-tuples containing ``(word, tag, IOB-tag)``.
    Convert a tree to the CoNLL IOB tag format.

    :param t: The tree to be converted.
    :type t: Tree
    :rtype: list(tuple)
    �B-z7Tree is too deeply nested to be printed in CoNLL formatrr6�I-r�)rwrtrr>rx�AttributeError)r:�tagsr}�category�prefix�contentss      rr
r
�s����D��3�3��	3��{�{�}�}�H��F�!�
�
���h��-�-��$�Q�������X�a�[�(�1�+�v��7H�I�J�J�J����

���	3�	3�	3��K�K��q��5��8�S�1�2�2�2�2�2�	3�����Ks�A(A1�1.B"�!B"Fc�
�t|g��}|D�]n\}}}|�)|rtd���|�||f���2|�d��r/|�t|dd�||fg�����v|�d��r�t	|��dksAt|dt��r&|d���|dd�krA|rtd���|�t|dd�||fg������ |d�||f����?|dkr|�||f����^td	|�����|S)
z1
    Convert the CoNLL IOB format to a tree.
    NzBad conll tag sequencer�r�r�rr�r�zBad conll tag )rr>rx�
startswithr2rtrw)�sentencer�r��strict�treer��postag�chunktags        r�conlltags2treer��s����
�B���D�$,�<�<� ��v�x����
,� �!9�:�:�:����T�6�N�+�+�+�+�
�
 �
 ��
&�
&�	<��K�K��X�a�b�b�\�T�6�N�+;�<�<�=�=�=�=�
�
 �
 ��
&�
&�	<��D�	�	�Q���!�$�r�(�D�1�1����8�>�>�#�#�x����|�3�3��F�$�%=�>�>�>��K�K��X�a�b�b�\�T�6�N�3C� D� D�E�E�E�E��R�����v��/�/�/�/�
��_�_��K�K��v��'�'�'�'��:�h�:�:�;�;�;��Krc�^�d�t|��D��}d�|��S)z�
    Return a multiline string where each line contains a word, tag and IOB tag.
    Convert a tree to the CoNLL IOB string format

    :param t: The tree to be converted.
    :type t: Tree
    :rtype: str
    c�8�g|]}d�|����S)� )�join)r9�tokens  rrZz!tree2conllstr.<locals>.<listcomp>�s"��<�<�<��S�X�X�e�_�_�<�<�<rr�)r
r�)r:�liness  r�
tree2conllstrr��s1��
=�<�.��*;�*;�<�<�<�E��9�9�U���ra<DOC>\s*(<DOCNO>\s*(?P<docno>.+?)\s*</DOCNO>\s*)?(<DOCTYPE>\s*(?P<doctype>.+?)\s*</DOCTYPE>\s*)?(<DATE_TIME>\s*(?P<date_time>.+?)\s*</DATE_TIME>\s*)?<BODY>\s*(<HEADLINE>\s*(?P<headline>.+?)\s*</HEADLINE>\s*)?<TEXT>(?P<text>.*?)</TEXT>\s*</BODY>\s*</DOC>\s*z#<b_\w+\s+[^>]*?type="(?P<type>\w+)"c��t|g��g}|�gStjd|��D�]1}|���}	|�d��r�t
�|��}|�td|��t|�d��g��}|d�|��|�|��nE|�d��r|�	��n|d�|����#ttf$r,}td|���d�d	���|�d}~wwxYwt|��d
krtd���|dS)
Nz<[^>]+>|[^\s<]+z<b_�XXXX�typer�z<e_z$Bad IEER string (error at character r��)r6zBad IEER stringr)rrur�r�r��
_IEER_TYPE_RErv�printrxr��
IndexErrorr>r�r2)r�r�r��piece_m�piece�mr��es        r�_ieer_read_textr�s���
�*�b�
!�
!�"�E�	�y��	��;�1�1�5�5�����
�
����	�����&�&�

(�!�'�'��.�.���9��&�%�(�(�(��Q�W�W�V�_�_�b�1�1���b�	� � ��'�'�'����U�#�#�#�#��!�!�%�(�(�
(��	�	�����
�b�	� � ��'�'�'����J�'�	�	�	��K�w�}�}���K�K�K�K����
�����	�����5�z�z�Q����*�+�+�+���8�Os�CD�E�/'E�E)	�LOCATION�ORGANIZATION�PERSON�DURATION�DATE�CARDINAL�PERCENT�MONEY�MEASUREc	�`�t�|��}|r�t|�d��|��|�d��|�d��|�d��t|�d��|��d�St||��S)ap
    Return a chunk structure containing the chunked tagged text that is
    encoded in the given IEER style string.
    Convert a string of chunked tagged text in the IEER named
    entity format into a chunk structure.  Chunks are of several
    types, LOCATION, ORGANIZATION, PERSON, DURATION, DATE, CARDINAL,
    PERCENT, MONEY, and MEASURE.

    :rtype: Tree
    r��docno�doctype�	date_time�headline)r�r�r�r�r�)�_IEER_DOC_RErvr�r�)r�r�r�r�s    r�ieerstr2treer�'s���8	���1���A��.�#�A�G�G�F�O�O�Z�@�@��W�W�W�%�%��w�w�y�)�)�����-�-�(����
�(;�(;�Z�H�H�
�
�	
��q�*�-�-�-rc�d�d}ddl}|j�|d���}|���t	��d}t|d���}|���t	d��t	|j�|����t	��dS)	Nzd[ Pierre/NNP Vinken/NNP ] ,/, [ 61/CD years/NNS ] old/JJ ,/, will/MD join/VB [ the/DT board/NN ] ./.rr~)rav
These DT B-NP
research NN I-NP
protocols NNS I-NP
offer VBP B-VP
to TO B-PP
the DT B-NP
patient NN I-NP
not RB O
only RB O
the DT B-NP
very RB I-NP
best JJS I-NP
therapy NN I-NP
which WDT B-NP
we PRP B-NP
have VBP B-VP
established VBN I-VP
today NN B-NP
but CC B-NP
also RB I-NP
the DT B-NP
hope NN I-NP
of IN B-PP
something NN B-NP
still RB B-ADJP
better JJR I-ADJP
. . O
)r~r�)r�z
CoNLL output:)�nltkr�r��pprintr�r�r�)r�r�r:�
conll_trees    r�demor�Rs���n�A��K�K�K��
���q�d��3�3�A��H�H�J�J�J�	�G�G�G�	�A�<�q�l�;�;�;�J�������
�/����	�$�*�
"�
"�:�
.�
.�/�/�/�	�G�G�G�G�Gr�__main__)r~rr�NN)r�r)r�rF)ru�nltk.metricsrr�nltk.tag.mappingr�
nltk.tag.utilr�	nltk.treerrr=r�r�r�r�r
r�r��DOTALLr�r�r�r�r�ror7rr�<module>r�s���
�	�	�	�.�.�.�.�.�.�$�$�$�$�$�$�#�#�#�#�#�#�������+�+�+�<z
�z
�z
�z
�z
�z
�z
�z
�~
�
�
�UY�.�.�.�.�f�2�:�8�9�9��2�2�2�2�j���8FK�!�!�!�!�H
�
�
��r�z���I�
�
����
�A�B�B�
����D
�
�
��(.�(.�(.�(.�V-�-�-�`�z����D�F�F�F�F�F��r

Youez - 2016 - github.com/yon3zu
LinuXploit