403Webshell
Server IP : 54.37.205.81  /  Your IP : 216.73.216.76
Web Server : nginx/1.22.1
System : Linux vps-249481fa 6.1.0-50-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.176-1 (2026-07-02) x86_64
User : debian ( 1000)
PHP Version : 8.2.32
Disable Function : NONE
MySQL : OFF  |  cURL : ON  |  WGET : ON  |  Perl : ON  |  Python : OFF  |  Sudo : ON  |  Pkexec : OFF
Directory :  /lib/python3/dist-packages/nltk/parse/__pycache__/

Upload File :
current_dir [ Writeable ] document_root [ Writeable ]

 

Command :


[ Back ]     

Current File : /lib/python3/dist-packages/nltk/parse/__pycache__/transitionparser.cpython-311.pyc
�

 T�c�z����ddlZddlZddlmZddlmZddlmZ	ddlm	Z	ddl
mZddlm
Z
ddlmZn#e$rYnwxYwdd	lmZmZmZGd
�d��ZGd�d
��ZGd�de��Zd�ZdS)�N)�deepcopy)�
itemgetter)�remove)�array)�sparse)�svm)�load_svmlight_file)�DependencyEvaluator�DependencyGraph�ParserIc�,�eZdZdZd�Zd�Zdd�Zd�ZdS)	�
Configurationa
    Class for holding configuration which is the partial analysis of the input sentence.
    The transition based parser aims at finding set of operators that transfer the initial
    configuration to the terminal configuration.

    The configuration includes:
        - Stack: for storing partially proceeded words
        - Buffer: for storing remaining input words
        - Set of arcs: for storing partially built dependency tree

    This class also provides a method to represent a configuration as list of features.
    c���dg|_ttdt|j������|_g|_|j|_t|j��|_dS)z�
        :param dep_graph: the representation of an input in the form of dependency graph.
        :type dep_graph: DependencyGraph where the dependencies are not specified.
        r�N)	�stack�list�range�len�nodes�buffer�arcs�_tokens�_max_address)�self�	dep_graphs  �=/usr/lib/python3/dist-packages/nltk/parse/transitionparser.py�__init__zConfiguration.__init__(sX���S��
��5��C�	��$8�$8�9�9�:�:�����	� �������,�,�����c��dt|j��zdzt|j��zdzt|j��zS)NzStack : z  Buffer : z
   Arcs : )�strrrr)rs r�__str__zConfiguration.__str__4sR����$�*�o�o�
��
��$�+���
��	
�
�$�)�n�n�
�	
rFc�6�|�dS|dkrdS|dur|dkrdSdS)zs
        Check whether a feature is informative
        The flag control whether "_" is informative or not
        NF��_T�)r�feat�flags   r�_check_informativez Configuration._check_informative>s8��
�<��5��2�:�:��5��5�=�=��s�{�{��u��trc�j�g}t|j��dk�rL|jt|j��dz
}|j|}|�|dd��r|�d|dz��d|vr9|�|d��r|�d|dz��|�|d��r|�d	|dz��d
|vrS|�|d
��r8|d
�d��}|D]}|�d|z���t|j��dkrh|jt|j��d
z
}|j|}|�|d��r|�d|dz��d}d}d}	d}
|jD],\}}}
||kr |
|kr
|
|kr|
}|}
|
|kr
|
|kr|
}|}	�-|�|	��r|�d|	z��|�|
��r|�d|
z��t|j��dk�r2|jd}|j|}|�|dd��r|�d|dz��d|vr9|�|d��r|�d|dz��|�|d��r|�d|dz��d
|vrS|�|d
��r8|d
�d��}|D]}|�d|z���t|j��dkr�|jd}|j|}|�|dd��r|�d|dz��|�|d��r|�d|dz��t|j��d
krS|jd
}|j|}|�|d��r|�d|dz��t|j��dkrS|jd}|j|}|�|d��r|�d|dz��d}d}d}	d}
|jD],\}}}
||kr |
|kr
|
|kr|
}|}
|
|kr
|
|kr|
}|}	�-|�|	��r|�d|	z��|�|
��r|�d|
z��|S)a/
        Extract the set of features for the current configuration. Implement standard features as describe in
        Table 3.2 (page 31) in Dependency Parsing book by Sandra Kubler, Ryan McDonal, Joakim Nivre.
        Please note that these features are very basic.
        :return: list(str)
        rr�wordT�STK_0_FORM_�lemma�STK_0_LEMMA_�tag�
STK_0_POS_�feats�|�STK_0_FEATS_��
STK_1_POS_i@B���r#�STK_0_LDEP_�STK_0_RDEP_�BUF_0_FORM_�BUF_0_LEMMA_�
BUF_0_POS_�BUF_0_FEATS_�BUF_1_FORM_�
BUF_1_POS_�
BUF_2_POS_��
BUF_3_POS_�BUF_0_LDEP_�BUF_0_RDEP_)rrrr(�append�splitrr)r�result�
stack_idx0�tokenr0r&�
stack_idx1�	left_most�
right_most�
dep_left_most�dep_right_most�wi�r�wj�buffer_idx0�buffer_idx1�buffer_idx2�buffer_idx3s                  r�extract_featureszConfiguration.extract_featuresLs������t�z�?�?�Q�����C��
�O�O�a�$7�8�J��L��,�E��&�&�u�V�}�d�;�;�
=��
�
�m�e�F�m�;�<�<�<��%���D�$;�$;�E�'�N�$K�$K���
�
�n�u�W�~�=�>�>�>��&�&�u�U�|�4�4�
;��
�
�l�U�5�\�9�:�:�:��%���D�$;�$;�E�'�N�$K�$K���g��,�,�S�1�1��!�9�9�D��M�M�.�4�"7�8�8�8�8��4�:����"�"�!�Z��D�J���!�(;�<�
���Z�0���*�*�5��<�8�8�?��M�M�,��u��"=�>�>�>� �I��J��M��N�#�y�
*�
*���Q����#�#��R���b�:�o�o�%'�
�)*���R���b�9�n�n�$&�	�()�
���&�&�}�5�5�
=��
�
�m�m�;�<�<�<��&�&�~�6�6�
>��
�
�m�n�<�=�=�=��t�{���a����+�a�.�K��L��-�E��&�&�u�V�}�d�;�;�
=��
�
�m�e�F�m�;�<�<�<��%���D�$;�$;�E�'�N�$K�$K���
�
�n�u�W�~�=�>�>�>��&�&�u�U�|�4�4�
;��
�
�l�U�5�\�9�:�:�:��%���D�$;�$;�E�'�N�$K�$K���g��,�,�S�1�1��!�9�9�D��M�M�.�4�"7�8�8�8�8��4�;���!�#�#�"�k�!�n����[�1���*�*�5��=�$�?�?�A��M�M�-�%��-�"?�@�@�@��*�*�5��<�8�8�?��M�M�,��u��"=�>�>�>��4�;���!�#�#�"�k�!�n����[�1���*�*�5��<�8�8�?��M�M�,��u��"=�>�>�>��4�;���!�#�#�"�k�!�n����[�1���*�*�5��<�8�8�?��M�M�,��u��"=�>�>�>��I��J��M��N�#�y�
*�
*���Q����$�$��R���b�:�o�o�%'�
�)*���R���b�9�n�n�$&�	�()�
���&�&�}�5�5�
=��
�
�m�m�;�<�<�<��&�&�~�6�6�
>��
�
�m�n�<�=�=�=��
rN)F)�__name__�
__module__�__qualname__�__doc__rr!r(rTr%rrrrsd��������
-�
-�
-�
�
�
�����d�d�d�d�drrc�@�eZdZdZdZdZdZdZd�Zd�Z	d�Z
d	�Zd
�ZdS)�
Transitionz�
    This class defines a set of transition which is applied to a configuration to get another configuration
    Note that for different parsing algorithm, the transition is different.
    �LEFTARC�RIGHTARC�SHIFT�REDUCEc��||_|tjtjfvr*t	dtj�dtj�d����dS)z�
        :param alg_option: the algorithm option of this parser. Currently support `arc-standard` and `arc-eager` algorithm
        :type alg_option: str
        � Currently we only support � and � N)�_algo�TransitionParser�ARC_STANDARD�	ARC_EAGER�
ValueError)r�
alg_options  rrzTransition.__init__�sg��
 ��
���)��&�
�
�
��*�#�0�0�0�2B�2L�2L�2L�N���
�	
�
rc��t|j��dkst|j��dkrdS|jddkrdS|jt|j��dz
}d}|jtjkr|jD]\}}}||krd}�|rE|j���|jd}|j�|||f��dSdS)a
        Note that the algorithm for left-arc is quite similar except for precondition for both arc-standard and arc-eager

        :param configuration: is the current configuration
        :return: A new configuration or -1 if the pre-condition is not satisfied
        rr5rTFN)	rrrrcrdrfr�poprC)	r�conf�relation�idx_wir'�
idx_parentrN�	idx_child�idx_wjs	         r�left_arczTransition.left_arc�s���
������!�!�s�4�:���!�';�';��2��;�q�>�Q����2���C��
�O�O�a�/�0�����:�)�3�3�3�.2�i�
!�
!�*��Q�	���&�&� �D���	��J�N�N�����[��^�F��I���f�h��7�8�8�8�8�8��2rc��t|j��dkst|j��dkrdS|jtjkrO|j���}|jd}||jd<|j�|||f��dS|jt|j��dz
}|j�d��}|j�|��|j�|||f��dS)z�
        Note that the algorithm for right-arc is DIFFERENT for arc-standard and arc-eager

        :param configuration: is the current configuration
        :return: A new configuration or -1 if the pre-condition is not satisfied
        rr5rN)	rrrrcrdrerjrrC)rrkrlrmrps     r�	right_arczTransition.right_arc�s���
������!�!�s�4�:���!�';�';��2��:�)�6�6�6��Z�^�^�%�%�F��[��^�F�#�D�K��N��I���f�h��7�8�8�8�8�8��Z��D�J���!� 3�4�F��[�_�_�Q�'�'�F��J���f�%�%�%��I���f�h��7�8�8�8�8�8rc��|jtjkrdSt|j��dkrdS|jt|j��dz
}d}|jD]\}}}||krd}�|r|j���dSdS)z�
        Note that the algorithm for reduce is only available for arc-eager

        :param configuration: is the current configuration
        :return: A new configuration or -1 if the pre-condition is not satisfied
        r5rrFTN)rcrdrfrrrrj)rrkrmr'rnrNros       r�reducezTransition.reduce�s����:�)�3�3�3��2��t�z�?�?�a����2���C��
�O�O�a�/�0����*.�)�	�	�&�Z��I��F�"�"�����	��J�N�N�������2rc��t|j��dkrdS|j�d��}|j�|��dS)z�
        Note that the algorithm for shift is the SAME for arc-standard and arc-eager

        :param configuration: is the current configuration
        :return: A new configuration or -1 if the pre-condition is not satisfied
        rr5N)rrrjrrC)rrkrms   r�shiftzTransition.shiftsM���t�{���q� � ��2������#�#���
���&�!�!�!�!�!rN)
rUrVrWrX�LEFT_ARC�	RIGHT_ARCr]r^rrqrsrurwr%rrrZrZ�s���������H��I��E�
�F�
�
�
����89�9�9�(���.
"�
"�
"�
"�
"rrZc�R�eZdZdZdZdZd�Zd�Zd�Zd�Z	d�Z
d	�Zd
�Zdd�Z
d
�ZdS)rdzl
    Class for transition based parser. Implement 2 algorithms which are "arc-standard" and "arc-eager"
    zarc-standardz	arc-eagerc��||j|jfvr td|j�d|j�d����||_i|_i|_i|_dS)z�
        :param algorithm: the algorithm option of this parser. Currently support `arc-standard` and `arc-eager` algorithm
        :type algorithm: str
        r`rarbN)rerfrg�
_algorithm�_dictionary�_transition�_match_transition)r�	algorithms  rrzTransitionParser.__init__+sm��
�d�/���@�@�@��*��$�$�$�d�n�n�n�6���
�$���������!#����rc��|j|}|j|}|d�dS|d|dkr|dSdS)Nr*�head�address�rel)r)rrnro�depgraph�p_node�c_nodes      r�_get_dep_relationz"TransitionParser._get_dep_relation;sL����
�+����	�*���&�>�!��4��&�>�V�I�.�.�.��%�=� ��4rc��g}|D]O}|j�|t|j����|�|j|���Pd�d�t|��D����S)z�
        :param features: list of feature string which is needed to convert to binary features
        :type features: list(str)
        :return : string of binary features in libsvm format  which is 'featureID:value' pairs
        rbc3�:K�|]}t|��dzV��dS)z:1.0N)r )�.0�	featureIDs  r�	<genexpr>z?TransitionParser._convert_to_binary_features.<locals>.<genexpr>Ss>����
�
�(1�C�	�N�N�V�#�
�
�
�
�
�
r)r}�
setdefaultrrC�join�sorted)r�features�unsorted_result�features    r�_convert_to_binary_featuresz,TransitionParser._convert_to_binary_featuresGs������	>�	>�G���'�'���T�5E�1F�1F�G�G�G��"�"�4�#3�G�#<�=�=�=�=��x�x�
�
�5;�O�5L�5L�
�
�
�
�
�	
rc�r�g}|jD]<}|j|}d|vr)|d}|d}|�|�||f���=|D]m\}}||kr|}|}|}t|dz|��D]H}tt|j����D]$}	|	|ks|	|kr||	f|vrdS|	|f|vrdS�%�I�ndS)Nr�r�rFT)rrCrr)
rr��arc_list�key�node�childIdx�	parentIdx�temp�k�ms
          r�_is_projectivezTransitionParser._is_projectiveWs �����>�	;�	;�C��>�#�&�D���~�~��	�?�� ��L�	��(��O�O�Y��$9�:�:�:��%-�	)�	)�!�Y���)�#�#���$�� �	��8�a�<��3�3�
)�
)���s�8�>�2�2�3�3�)�)�A��H���!�i�-�-��q�6�X�-�-�#(�5�5�5�5��q�6�X�-�-�#(�5�5�5�5��)�
)��trc�&�|j�|t|j��dz��||j|j|<t	|j|��dz|zdz}|�|�d����dS)z^
        write the binary features to input file and update the transition dictionary
        rrb�
zutf-8N)r~r�rrr �write�encode)rr��binary_features�
input_file�	input_strs     r�_write_to_filezTransitionParser._write_to_fileqs���	
��#�#�C��T�-=�)>�)>��)B�C�C�C�8;���t�/��4�5���(��-�.�.��4��F��M�	�����)�)�'�2�2�3�3�3�3�3rc�b�t|j��}d}g}|D�]G}|�|��s�|dz
}t|��}t	|j��dk�r|jd}|���}	|�|	��}
t	|j��dk�rL|jt	|j��dz
}|�	|||��}|�Utj
dz|z}
|�|
|
|��|�||��|�
|
����|�	|||��}|��d}|j}t|dz��D]/}||kr'|�	|||��}|�|||f|jvrd}�0|rVtjdz|z}
|�|
|
|��|�||��|�
|
�����tj}
|�|
|
|��|�|��|�
|
��t	|j��dk����It+dt-t	|����z��t+dt-|��z��|S)	z�
        Create the training example in the libsvm format and write it to the input_file.
        Reference : Page 32, Chapter 3. Dependency Parsing by Sandra Kubler, Ryan McDonal and Joakim Nivre (2009)
        rrN�:TF� Number of training examples : �) Number of valid (projective) examples : )rZrer�rrrrTr�rr�rxr�rqrCrrrryrsr]rw�printr )r�	depgraphsr��	operation�
count_proj�training_seqr�rk�b0r�r��s0r�r��precondition�maxID�w�relws                  r�!_create_training_examples_arc_stdz2TransitionParser._create_training_examples_arc_std{s���
�t�0�1�1�	��
���!�/	)�/	)�H��&�&�x�0�0�
���!�O�J� ��*�*�D��d�k�"�"�Q�&�&��[��^���0�0�2�2��"&�"B�"B�8�"L�"L���t�z�?�?�Q�&�&���C��
�O�O�a�$7�8�B��0�0��R��B�B�C���(�1�C�7�#�=���+�+�C��*�M�M�M�!�*�*�4��5�5�5�$�+�+�C�0�0�0� ��0�0��R��B�B�C���'+�� $� 1��!&�u�q�y�!1�!1�=�=�A� �B�w�w�'+�'=�'=�b�!�X�'N�'N��#'�#3�(*�D�!�}�D�I�'E�'E�7<���'�%�",�"6��"<�s�"B�C� �/�/��_�j�Q�Q�Q�%�/�/��c�:�:�:�(�/�/��4�4�4�$�!�&���#�#�C��*�E�E�E�����%�%�%��#�#�C�(�(�(�S�d�k�"�"�Q�&�&��V	�/�#�c�)�n�n�2E�2E�E�F�F�F�
�9�C�
�O�O�K�L�L�L��rc��t|j��}d}g}|D�]�}|�|��s�|dz
}t|��}t	|j��dk�rK|jd}|���}	|�|	��}
t	|j��dk�r�|jt	|j��dz
}|�	|||��}|�Utj
dz|z}
|�|
|
|��|�||��|�
|
����|�	|||��}|�Vtjdz|z}
|�|
|
|��|�||��|�
|
����gd}t!|��D]4}|�	|||���d}|�	|||���d}�5|rOtj}
|�|
|
|��|�|��|�
|
�����tj}
|�|
|
|��|�|��|�
|
��t	|j��dk��K���t+dt-t	|����z��t+dt-|��z��|S)	z�
        Create the training example in the libsvm format and write it to the input_file.
        Reference : 'A Dynamic Oracle for Arc-Eager Dependency Parsing' by Joav Goldberg and Joakim Nivre
        rrNr�FTr�r�)rZrfr�rrrrTr�rr�rxr�rqrCryrsrr^rur]rwr�r )rr�r�r��	countProjr�r�rkr�r�r�r�r�r�r'r�s                r�#_create_training_examples_arc_eagerz4TransitionParser._create_training_examples_arc_eager�s��
�t�~�.�.�	��	���!�1	)�1	)�H��&�&�x�0�0�
����N�I� ��*�*�D��d�k�"�"�Q�&�&��[��^���0�0�2�2��"&�"B�"B�8�"L�"L���t�z�?�?�Q�&�&���C��
�O�O�a�$7�8�B��0�0��R��B�B�C���(�1�C�7�#�=���+�+�C��*�M�M�M�!�*�*�4��5�5�5�$�+�+�C�0�0�0� ��0�0��R��B�B�C���(�2�S�8�3�>���+�+�C��*�M�M�M�!�+�+�D�#�6�6�6�$�+�+�C�0�0�0� �!�D�"�2�Y�Y�(�(���1�1�!�R��B�B�N�#'�D��1�1�"�a��B�B�N�#'�D���!�(�/���+�+�C��*�M�M�M�!�(�(��.�.�.�$�+�+�C�0�0�0� �!�&���#�#�C��*�E�E�E�����%�%�%��#�#�C�(�(�(�W�d�k�"�"�Q�&�&��Z	�/�#�c�)�n�n�2E�2E�E�F�F�F�
�9�C�	�N�N�J�K�K�K��rTc	�*�	tjdtj��d���}|j|jkr|�||��n|�||��|���t|j	��\}}tjddddd|d	�
��}|�||��tj|t|d����t!|j	��dS#t!|j	��wxYw)
z�
        :param depgraphs : list of DependencyGraph as the training data
        :type depgraphs : DependencyGraph
        :param modelfile : file name to save the trained model
        :type modelfile : str
        ztransition_parse.trainF)�prefix�dir�delete�polyr3rg�������?g�?T)�kernel�degree�coef0�gamma�C�verbose�probability�wbN)�tempfile�NamedTemporaryFile�
gettempdirr|rer�r��closer	�namer�SVC�fit�pickle�dump�openr)rr��	modelfiler�r��x_train�y_train�models        r�trainzTransitionParser.train�s!��	$�!�4�/�X�5H�5J�5J�SX����J���$�"3�3�3��6�6�y�*�M�M�M�M��8�8��J�O�O�O�������1�*�/�B�B��G�W�
�G������� ����E�
�I�I�g�w�'�'�'��K��t�I�t�4�4�5�5�5��:�?�#�#�#�#�#��F�:�?�#�#�#�#���s�C$C<�<Dc���g}tjt|d����}t|j��}|D�];}t|��}t
|j��dk�r�|���}g}	g}
g}|D]U}||j	vrJ|	�
|j	|��|
�
d��|�
d���Vtt|	����}
t|
��}t|��}tj|||
ffdt
|j	��f���}i}|�|��d}t!t
|����D]
}||||<�t|���t%d��d���}|D�]7\}}|j|}||jv�r
|j|}|�d��d}|tjkr6|�||�d��d��d	krn���|tjkr6|�||�d��d��d	krni��|tjkr|�|��d	krn=��|tjkr|�|��d	krn��*t=d
���t
|j��dk���t?|��}|j D]}|j |}d|d<d|d
<�|j!D]\}}}|j |} || d
<|| d<�|�
|����=|S)aZ
        :param depgraphs: the list of test sentence, each sentence is represented as a dependency graph where the 'head' information is dummy
        :type depgraphs: list(DependencyGraph)
        :param modelfile: the model file
        :type modelfile: str
        :return: list (DependencyGraph) with the 'head' and 'rel' information
        �rbrg�?r)�shapeT)r��reverser�r5z;The predicted transition is not recognized, expected errorsr#r�r�)"r��loadr�rZr|rrrrTr}rCrr�r�
csr_matrix�
predict_probar�itemsr�classes_rrDrxrqryrsr^rur]rwrgrrr)!rr��	modelFilerEr�r�r�rkr��col�row�datar��np_col�np_row�np_data�x_test�	prob_dict�	pred_prob�i�sorted_Prob�
y_pred_idx�
confidence�y_pred�
strTransition�baseTransition�new_depgraphr�r�r�r��childr�s!                                 r�parsezTransitionParser.parse"s�������D��D�1�1�2�2���t��/�/�	�!�Y	(�Y	(�H� ��*�*�D��d�k�"�"�Q�&�&��0�0�2�2��������'�)�)�G��$�"2�2�2��
�
�4�#3�G�#<�=�=�=��
�
�1�
�
�
����C�(�(�(���v�c�{�{�+�+���s������+�+���*��v�v�.�/��3�t�?O�;P�;P�7Q�����,�	�!�/�/��7�7��:�	��s�9�~�~�.�.�0�0�A�#,�Q�<�I�a�L�L�$�Y�_�_�%6�%6�J�q�M�M�SW�X�X�X��1<���,�Z��#�^�J�7�F���!7�7�7�(,�(>�v�(F�
�)6�)<�)<�S�)A�)A�!�)D��)�Z�-@�@�@� )� 2� 2�4��9L�9L�S�9Q�9Q�RS�9T� U� U�#%�!&�!&�!&��!&�,�z�/C�C�C� )� 3� 3�D�-�:M�:M�c�:R�:R�ST�:U� V� V�#%�!&�!&�!&��!&�,�z�/@�@�@�(�/�/��5�5��;�;� %�� <�+�z�/?�?�?�(���t�4�4��:�:� %���(�Y����O�d�k�"�"�Q�&�&�Z$�H�-�-�L�#�)�
!�
!��#�)�#�.�� ��U�� ��V���&*�i�
$�
$�"��s�E�%�+�E�2��!%��v�� #��u�
�
��M�M�,�'�'�'�'��
rN)T)rUrVrWrXrerfrr�r�r�r�r�r�r�r�r%rrrdrd"s���������"�L��I�$�$�$� 
�
�
�
�
�
� ���44�4�4�<�<�<�|>�>�>�@'$�'$�'$�'$�Rh�h�h�h�hrrdc��dS)a6
    >>> from nltk.parse import DependencyGraph, DependencyEvaluator
    >>> from nltk.parse.transitionparser import TransitionParser, Configuration, Transition
    >>> gold_sent = DependencyGraph("""
    ... Economic  JJ     2      ATT
    ... news  NN     3       SBJ
    ... has       VBD       0       ROOT
    ... little      JJ      5       ATT
    ... effect   NN     3       OBJ
    ... on     IN      5       ATT
    ... financial       JJ       8       ATT
    ... markets    NNS      6       PC
    ... .    .      3       PU
    ... """)

    >>> conf = Configuration(gold_sent)

    ###################### Check the Initial Feature ########################

    >>> print(', '.join(conf.extract_features()))
    STK_0_POS_TOP, BUF_0_FORM_Economic, BUF_0_LEMMA_Economic, BUF_0_POS_JJ, BUF_1_FORM_news, BUF_1_POS_NN, BUF_2_POS_VBD, BUF_3_POS_JJ

    ###################### Check The Transition #######################
    Check the Initialized Configuration
    >>> print(conf)
    Stack : [0]  Buffer : [1, 2, 3, 4, 5, 6, 7, 8, 9]   Arcs : []

    A. Do some transition checks for ARC-STANDARD

    >>> operation = Transition('arc-standard')
    >>> operation.shift(conf)
    >>> operation.left_arc(conf, "ATT")
    >>> operation.shift(conf)
    >>> operation.left_arc(conf,"SBJ")
    >>> operation.shift(conf)
    >>> operation.shift(conf)
    >>> operation.left_arc(conf, "ATT")
    >>> operation.shift(conf)
    >>> operation.shift(conf)
    >>> operation.shift(conf)
    >>> operation.left_arc(conf, "ATT")

    Middle Configuration and Features Check
    >>> print(conf)
    Stack : [0, 3, 5, 6]  Buffer : [8, 9]   Arcs : [(2, 'ATT', 1), (3, 'SBJ', 2), (5, 'ATT', 4), (8, 'ATT', 7)]

    >>> print(', '.join(conf.extract_features()))
    STK_0_FORM_on, STK_0_LEMMA_on, STK_0_POS_IN, STK_1_POS_NN, BUF_0_FORM_markets, BUF_0_LEMMA_markets, BUF_0_POS_NNS, BUF_1_FORM_., BUF_1_POS_., BUF_0_LDEP_ATT

    >>> operation.right_arc(conf, "PC")
    >>> operation.right_arc(conf, "ATT")
    >>> operation.right_arc(conf, "OBJ")
    >>> operation.shift(conf)
    >>> operation.right_arc(conf, "PU")
    >>> operation.right_arc(conf, "ROOT")
    >>> operation.shift(conf)

    Terminated Configuration Check
    >>> print(conf)
    Stack : [0]  Buffer : []   Arcs : [(2, 'ATT', 1), (3, 'SBJ', 2), (5, 'ATT', 4), (8, 'ATT', 7), (6, 'PC', 8), (5, 'ATT', 6), (3, 'OBJ', 5), (3, 'PU', 9), (0, 'ROOT', 3)]


    B. Do some transition checks for ARC-EAGER

    >>> conf = Configuration(gold_sent)
    >>> operation = Transition('arc-eager')
    >>> operation.shift(conf)
    >>> operation.left_arc(conf,'ATT')
    >>> operation.shift(conf)
    >>> operation.left_arc(conf,'SBJ')
    >>> operation.right_arc(conf,'ROOT')
    >>> operation.shift(conf)
    >>> operation.left_arc(conf,'ATT')
    >>> operation.right_arc(conf,'OBJ')
    >>> operation.right_arc(conf,'ATT')
    >>> operation.shift(conf)
    >>> operation.left_arc(conf,'ATT')
    >>> operation.right_arc(conf,'PC')
    >>> operation.reduce(conf)
    >>> operation.reduce(conf)
    >>> operation.reduce(conf)
    >>> operation.right_arc(conf,'PU')
    >>> print(conf)
    Stack : [0, 3, 9]  Buffer : []   Arcs : [(2, 'ATT', 1), (3, 'SBJ', 2), (0, 'ROOT', 3), (5, 'ATT', 4), (3, 'OBJ', 5), (5, 'ATT', 6), (8, 'ATT', 7), (6, 'PC', 8), (3, 'PU', 9)]

    ###################### Check The Training Function #######################

    A. Check the ARC-STANDARD training
    >>> import tempfile
    >>> import os
    >>> input_file = tempfile.NamedTemporaryFile(prefix='transition_parse.train', dir=tempfile.gettempdir(), delete=False)

    >>> parser_std = TransitionParser('arc-standard')
    >>> print(', '.join(parser_std._create_training_examples_arc_std([gold_sent], input_file)))
     Number of training examples : 1
     Number of valid (projective) examples : 1
    SHIFT, LEFTARC:ATT, SHIFT, LEFTARC:SBJ, SHIFT, SHIFT, LEFTARC:ATT, SHIFT, SHIFT, SHIFT, LEFTARC:ATT, RIGHTARC:PC, RIGHTARC:ATT, RIGHTARC:OBJ, SHIFT, RIGHTARC:PU, RIGHTARC:ROOT, SHIFT

    >>> parser_std.train([gold_sent],'temp.arcstd.model', verbose=False)
     Number of training examples : 1
     Number of valid (projective) examples : 1
    >>> input_file.close()
    >>> remove(input_file.name)

    B. Check the ARC-EAGER training

    >>> input_file = tempfile.NamedTemporaryFile(prefix='transition_parse.train', dir=tempfile.gettempdir(),delete=False)
    >>> parser_eager = TransitionParser('arc-eager')
    >>> print(', '.join(parser_eager._create_training_examples_arc_eager([gold_sent], input_file)))
     Number of training examples : 1
     Number of valid (projective) examples : 1
    SHIFT, LEFTARC:ATT, SHIFT, LEFTARC:SBJ, RIGHTARC:ROOT, SHIFT, LEFTARC:ATT, RIGHTARC:OBJ, RIGHTARC:ATT, SHIFT, LEFTARC:ATT, RIGHTARC:PC, REDUCE, REDUCE, REDUCE, RIGHTARC:PU

    >>> parser_eager.train([gold_sent],'temp.arceager.model', verbose=False)
     Number of training examples : 1
     Number of valid (projective) examples : 1

    >>> input_file.close()
    >>> remove(input_file.name)

    ###################### Check The Parsing Function ########################

    A. Check the ARC-STANDARD parser

    >>> result = parser_std.parse([gold_sent], 'temp.arcstd.model')
    >>> de = DependencyEvaluator(result, [gold_sent])
    >>> de.eval() >= (0, 0)
    True

    B. Check the ARC-EAGER parser
    >>> result = parser_eager.parse([gold_sent], 'temp.arceager.model')
    >>> de = DependencyEvaluator(result, [gold_sent])
    >>> de.eval() >= (0, 0)
    True

    Remove test temporary files
    >>> remove('temp.arceager.model')
    >>> remove('temp.arcstd.model')

    Note that result is very poor because of only one training example.
    Nr%r%rr�demor��s���r)r�r��copyr�operatorr�osr�numpyr�scipyr�sklearnr�sklearn.datasetsr	�ImportError�
nltk.parser
rrrrZrdr�r%rr�<module>r�s����
�
�
�����������������������	�������������������3�3�3�3�3�3�3���	�	�	��D�	����E�D�D�D�D�D�D�D�D�D�V�V�V�V�V�V�V�V�rl"�l"�l"�l"�l"�l"�l"�l"�^h�h�h�h�h�w�h�h�h�VM�M�M�M�Ms�5�=�=

Youez - 2016 - github.com/yon3zu
LinuXploit