| Server IP : 54.37.205.81 / Your IP : 216.73.216.76 Web Server : nginx/1.22.1 System : Linux vps-249481fa 6.1.0-50-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.176-1 (2026-07-02) x86_64 User : debian ( 1000) PHP Version : 8.2.32 Disable Function : NONE MySQL : OFF | cURL : ON | WGET : ON | Perl : ON | Python : OFF | Sudo : ON | Pkexec : OFF Directory : /lib/python3/dist-packages/nltk/__pycache__/ |
Upload File : |
�
T�c�E � � � d Z ddlZddlZddlmZ ddlmZmZmZm Z ddl
mZmZ G d� d� � Z
G d� d e
� � Z ej d
� � Zdd�Z G d
� de
� � Zdd�Zd� Zd� Zd� Zd� Zd� Zd� Zd� Zedk r e� � dS dS )zT
Module for reading, writing and manipulating
Toolbox databases and settings files.
� N)�StringIO)�Element�ElementTree�
SubElement�TreeBuilder)�PathPointer�findc �D � e Zd ZdZdd�Zd� Zd� Zd� Z dd �Zd
� Z dS )
�StandardFormatzT
Class for reading and processing standard format marker files and strings.
Nc �F � || _ |�| � |� � d S d S �N)� _encoding�open)�self�filename�encodings �./usr/lib/python3/dist-packages/nltk/toolbox.py�__init__zStandardFormat.__init__ s/ � �!������I�I�h������ �� c � � t |t � � r!|� | j � � | _ dS t j |d| j � � | _ dS )z�
Open a standard format marker file for sequential reading.
:param sfm_file: name of the standard format marker input file
:type sfm_file: str
�rN)�
isinstancer r r �_file�codecs)r �sfm_files r r zStandardFormat.open sK � � �h��,�,� D�!���t�~�6�6�D�J�J�J���X�s�D�N�C�C�D�J�J�Jr c �. � t |� � | _ dS )z�
Open a standard format marker string for sequential reading.
:param s: string to parse as a standard format marker input file
:type s: str
N)r r )r �ss r �open_stringzStandardFormat.open_string+ s � � �a�[�[��
�
�
r c # � K � d}d}t j |dz � � }t j |dz � � }t | j � � } t |� � }n# t
$ r Y dS w xY wt j ||� � }|� � � \ }} | g}
d| _ |D ]t}| xj dz
c_ t j ||� � }|� � � \ }} |r||� |
� � fV � |}| g}
�_|
�
| � � �u| xj dz
c_ ||� |
� � fV � dS )z�
Return an iterator that returns the next field in a (marker, value)
tuple. Linebreaks and trailing white space are preserved except
for the final newline in each field.
:rtype: iter(tuple(str, str))
�
z^%s(?:\\(\S+)\s*)?(.*)$u (?:)?� Nr � )�re�compile�iterr �next�
StopIteration�match�groups�line_num�join�append)r �join_string�line_regexp�first_line_pat�line_pat� file_iter�line�mobj�mkr�
line_value�value_lines�line_mkrs r �
raw_fieldszStandardFormat.raw_fields4 su � � � � ��0����K�2E�$E�F�F���:�k�B�.�/�/�� ���$�$� � �� �?�?�D�D��� � � ��F�F� ���� �x���-�-���+�+�-�-���Z�!�l����
�� /� /�D��M�M�Q��M�M��8�H�d�+�+�D�#'�;�;�=�=� �H�j��
/��K�,�,�[�9�9�:�:�:�:���)�l����"�"�:�.�.�.�.��
�
���
�
��K�$�$�[�1�1�2�2�2�2�2�2s �
A �
A(�'A(T�strictc # �� K � |�|�t d� � �t j d� � }| � � � D ]9\ }}|r|� d|� � }|r|� � � }||fV � �:dS )a�
Return an iterator that returns the next field in a ``(marker, value)``
tuple, where ``marker`` and ``value`` are unicode strings if an ``encoding``
was specified in the ``fields()`` method. Otherwise they are non-unicode strings.
:param strip: strip trailing whitespace from the last line of each field
:type strip: bool
:param unwrap: Convert newlines in a field to spaces.
:type unwrap: bool
:param encoding: Name of an encoding to use. If it is specified then
the ``fields()`` method returns unicode strings rather than non
unicode strings.
:type encoding: str or None
:param errors: Error handling scheme for codec. Same as the ``decode()``
builtin string method.
:type errors: str
:param unicode_fields: Set of marker names whose values are UTF-8 encoded.
Ignored if encoding is None. If the whole file is UTF-8 encoded set
``encoding='utf8'`` and leave ``unicode_fields`` with its default
value of None.
:type unicode_fields: sequence
:rtype: iter(tuple(str, str))
Nz'unicode_fields is set but not encoding.z\n+� )�
ValueErrorr# r$ r8 �sub�rstrip) r �strip�unwrapr �errors�unicode_fields�
unwrap_patr4 �vals r �fieldszStandardFormat.fields[ s� � � � �>