403Webshell
Server IP : 54.37.205.81  /  Your IP : 216.73.216.76
Web Server : nginx/1.22.1
System : Linux vps-249481fa 6.1.0-50-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.176-1 (2026-07-02) x86_64
User : debian ( 1000)
PHP Version : 8.2.32
Disable Function : NONE
MySQL : OFF  |  cURL : ON  |  WGET : ON  |  Perl : ON  |  Python : OFF  |  Sudo : ON  |  Pkexec : OFF
Directory :  /lib/python3/dist-packages/nltk/lm/__pycache__/

Upload File :
current_dir [ Writeable ] document_root [ Writeable ]

 

Command :


[ Back ]     

Current File : /lib/python3/dist-packages/nltk/lm/__pycache__/api.cpython-311.pyc
�

 T�cD ���dZddlZddlZddlmZmZddlmZddlmZddl	m
Z
ddlmZddl
mZGd	�d
e���Zd�Zd
�Zdd�ZGd�de���ZdS)zLanguage Model Interface.�N)�ABCMeta�abstractmethod)�bisect)�
accumulate)�NgramCounter)�	log_base2)�
Vocabularyc�D�eZdZdZd�Zed���Zed���ZdS)�	Smoothingz�Ngram Smoothing Interface

    Implements Chen & Goodman 1995's idea that all smoothing algorithms have
    certain features in common. This should ideally allow smoothing algorithms to
    work both with Backoff and Interpolation.
    c�"�||_||_dS)z�
        :param vocabulary: The Ngram vocabulary object.
        :type vocabulary: nltk.lm.vocab.Vocabulary
        :param counter: The counts of the vocabulary items.
        :type counter: nltk.lm.counter.NgramCounter
        N)�vocab�counts)�self�
vocabulary�counters   �-/usr/lib/python3/dist-packages/nltk/lm/api.py�__init__zSmoothing.__init__s�� ��
������c��t����N��NotImplementedError)r�words  r�
unigram_scorezSmoothing.unigram_score&���!�#�#�#rc��t���rr�rr�contexts   r�alpha_gammazSmoothing.alpha_gamma*rrN)�__name__�
__module__�__qualname__�__doc__rrrr�rrrrsc������������$�$��^�$��$�$��^�$�$�$rr)�	metaclassc�@�t|��t|��zS)z0Return average (aka mean) for sequence of items.)�sum�len)�itemss r�_meanr*/s���u�:�:��E�
�
�"�"rc�b�t|tj��r|Stj|��Sr)�
isinstance�random�Random)�seed_or_generators r�_random_generatorr04s.���#�V�]�3�3�!� � ��=�*�+�+�+rc�$�|std���t|��t|��krtd���tt|����}|d}|���}|t|||z��S)z`Like random.choice, but with weights.

    Heavily inspired by python 3.6 `random.choices`.
    z"Can't choose from empty populationz3The number of weights does not match the population���)�
ValueErrorr(�listrr-r)�
population�weights�random_generator�cum_weights�total�	thresholds      r�_weighted_choicer;:s���
�?��=�>�>�>�
�:���#�g�,�,�&�&��N�O�O�O��z�'�*�*�+�+�K���O�E� �'�'�)�)�I��f�[�%�)�*;�<�<�=�=rc�d�eZdZdZd
d�Zdd�Zdd�Zedd���Zdd�Z	d�Z
d	�Zd
�Zdd�Z
dS)�
LanguageModelzKABC for Language Models.

    Cannot be directly instantiated itself.

    Nc���||_|r9t|t��s$tjd|jj�d�d���|�t��n||_|�t��n||_	dS)apCreates new LanguageModel.

        :param vocabulary: If provided, this vocabulary will be used instead
            of creating a new one when training.
        :type vocabulary: `nltk.lm.Vocabulary` or None
        :param counter: If provided, use this object to count ngrams.
        :type counter: `nltk.lm.NgramCounter` or None
        :param ngrams_fn: If given, defines how sentences in training text are turned to ngram
            sequences.
        :type ngrams_fn: function or None
        :param pad_fn: If given, defines how sentences in training text are padded.
        :type pad_fn: function or None
        z$The `vocabulary` argument passed to z- must be an instance of `nltk.lm.Vocabulary`.�)�
stacklevelN)
�orderr,r	�warnings�warn�	__class__r r
rr)rrArrs    rrzLanguageModel.__init__Ps�����
��	�j��Z�@�@�	��M�?�t�~�7N�?�?�?��
�
�
�
�
&0�%7�Z�\�\�\�Z��
�(/��l�n�n�n�W����rc����js+|�td����j�|���j��fd�|D����dS)zeTrains the model on a text.

        :param text: Training text as a sequence of sentences.

        Nz:Cannot fit without a vocabulary or text to create it from.c3�L�K�|]}�j�|��V��dSr)r
�lookup)�.0�sentrs  �r�	<genexpr>z$LanguageModel.fit.<locals>.<genexpr>ts3�����D�D�t�4�:�,�,�T�2�2�D�D�D�D�D�Dr)r
r3�updater)r�text�vocabulary_texts`  r�fitzLanguageModel.fithss����z�	/��&� �P����
�J���o�.�.�.�����D�D�D�D�t�D�D�D�D�D�D�D�Drc��|�|j�|��|r|j�|��nd��S)z�Masks out of vocab (OOV) words and computes their model score.

        For model-specific logic of calculating scores, see the `unmasked_score`
        method.
        N)�unmasked_scorer
rGrs   r�scorezLanguageModel.scorevsL���"�"��J���d�#�#�7�%T�T�Z�%6�%6�w�%?�%?�%?�PT�
�
�	
rc��t���)a�Score a word given some optional context.

        Concrete models are expected to provide an implementation.
        Note that this method does not mask its arguments with the OOV label.
        Use the `score` method for that.

        :param str word: Word for which we want the score
        :param tuple(str) context: Context the word is in.
            If `None`, compute unigram score.
        :param context: tuple(str) or None
        :rtype: float
        rrs   rrPzLanguageModel.unmasked_score�s��"�#�#�#rc�H�t|�||����S)z�Evaluate the log score of this word in this context.

        The arguments are the same as for `score` and `unmasked_score`.

        )rrQrs   r�logscorezLanguageModel.logscore�s ������D�'�2�2�3�3�3rc�d�|r#|jt|��dz|n|jjS)z�Helper method for retrieving counts for a given context.

        Assumes context has been checked and oov words in it masked.
        :type context: tuple(str) or None

        �)rr(�unigrams)rrs  r�context_countszLanguageModel.context_counts�s2��7>�W�D�K��G���q�(�)�'�2�2�4�;�CW�	
rc�@��dt�fd�|D����zS)z�Calculate cross-entropy of model for given evaluation text.

        :param Iterable(tuple(str)) text_ngrams: A sequence of ngram tuples.
        :rtype: float

        r2c�X��g|]&}��|d|dd�����'S)r2N)rT)rH�ngramrs  �r�
<listcomp>z)LanguageModel.entropy.<locals>.<listcomp>�s3���K�K�K�e�T�]�]�5��9�e�C�R�C�j�
1�
1�K�K�Kr)r*�r�text_ngramss` r�entropyzLanguageModel.entropy�s5����E�K�K�K�K�{�K�K�K�
�
�
�	
rc�H�td|�|����S)z�Calculates the perplexity of the given text.

        This is simply 2 ** cross-entropy for the text, so the arguments are the same.

        g@)�powr_r]s  r�
perplexityzLanguageModel.perplexity�s ���3����[�1�1�2�2�2rrVc����|�gnt|��}t|��}|dkr�t|���jkr|�jdzd�n|����j������}�rR|sPt���dkr
�dd�ng����j������}�r|�Pt|��}t|t��fd�|D����|��Sg}t|��D]0}|���d||z|������1|S)a�Generate words from the model.

        :param int num_words: How many words to generate. By default 1.
        :param text_seed: Generation can be conditioned on preceding context.
        :param random_seed: A random seed or an instance of `random.Random`. If provided,
            makes the random sampling part of generation reproducible.
        :return: One (str) word or a list of words generated from model.

        Examples:

        >>> from nltk.lm import MLE
        >>> lm = MLE(2)
        >>> lm.fit([[("a", "b"), ("b", "c")]], vocabulary_text=['a', 'b', 'c'])
        >>> lm.fit([[("a",), ("b",), ("c",)]])
        >>> lm.generate(random_seed=3)
        'a'
        >>> lm.generate(text_seed=['a'])
        'b'

        NrVc3�D�K�|]}��|���V��dSr)rQ)rH�wrrs  ��rrJz)LanguageModel.generate.<locals>.<genexpr>�s1�����>�>��d�j�j��G�,�,�>�>�>�>�>�>r)�	num_words�	text_seed�random_seed)
r4r0r(rArXr
rG�sortedr;�tuple�range�append�generate)	rrfrgrhr7�samples�	generated�_rs	`       @rrmzLanguageModel.generate�s�����*$�+�B�B��i���	�,�[�9�9����>�>��y�>�>�T�Z�/�/��4�:�+��/�+�+�,�,��
�
�)�)�$�*�*;�*;�G�*D�*D�E�E�G��
J�'�
J�),�W����)9�)9�'�!�"�"�+�+�r���-�-�d�j�.?�.?��.H�.H�I�I���
J�'�
J��W�o�o�G�#���>�>�>�>�>�g�>�>�>�>�>� ���
��	��y�!�!�	�	�A�����
�
��'�)�3� 0����
�
�
�
��r)NNr)rVNN)r r!r"r#rrNrQrrPrTrXr_rbrmr$rrr=r=Is���������E�E�E�E�0E�E�E�E�
�
�
�
��
$�
$�
$��^�
$�4�4�4�4�	
�	
�	
�	
�	
�	
�3�3�3�5�5�5�5�5�5rr=r)r#r-rB�abcrrr�	itertoolsr�nltk.lm.counterr�nltk.lm.utilr�nltk.lm.vocabularyr	rr*r0r;r=r$rr�<module>rvs6�� ��
�
�
�
�����'�'�'�'�'�'�'�'������� � � � � � �(�(�(�(�(�(�"�"�"�"�"�"�)�)�)�)�)�)�$�$�$�$�$�'�$�$�$�$�6#�#�#�
,�,�,�>�>�>�>�b�b�b�b�b�g�b�b�b�b�b�br

Youez - 2016 - github.com/yon3zu
LinuXploit