| Server IP : 54.37.205.81 / Your IP : 216.73.216.76 Web Server : nginx/1.22.1 System : Linux vps-249481fa 6.1.0-50-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.176-1 (2026-07-02) x86_64 User : debian ( 1000) PHP Version : 8.2.32 Disable Function : NONE MySQL : OFF | cURL : ON | WGET : ON | Perl : ON | Python : OFF | Sudo : ON | Pkexec : OFF Directory : /lib/python3/dist-packages/nltk/translate/__pycache__/ |
Upload File : |
�
T�c�M � � � d Z ddlmZ ddlmZ ddlmZ ddlmZ d� Z G d� d� � Z
G d � d
� � Z G d� d� � Zd
S )a�
Common methods and classes for all IBM models. See ``IBMModel1``,
``IBMModel2``, ``IBMModel3``, ``IBMModel4``, and ``IBMModel5``
for specific implementations.
The IBM models are a series of generative models that learn lexical
translation probabilities, p(target language word|source language word),
given a sentence-aligned parallel corpus.
The models increase in sophistication from model 1 to 5. Typically, the
output of lower models is used to seed the higher models. All models
use the Expectation-Maximization (EM) algorithm to learn various
probability tables.
Words in a sentence are one-indexed. The first word of a sentence has
position 1, not 0. Index 0 is reserved in the source sentence for the
NULL token. The concept of position does not apply to NULL, but it is
indexed at 0 by convention.
Each target word is aligned to exactly one source word or the NULL
token.
References:
Philipp Koehn. 2010. Statistical Machine Translation.
Cambridge University Press, New York.
Peter E Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and
Robert L. Mercer. 1993. The Mathematics of Statistical Machine
Translation: Parameter Estimation. Computational Linguistics, 19 (2),
263-311.
� )�insort_left��defaultdict)�deepcopy)�ceilc �\ � d}| D ]&}t |j � � }t ||� � }�'|S )z�
:param sentence_aligned_corpus: Parallel corpus under consideration
:type sentence_aligned_corpus: list(AlignedSent)
:return: Number of words in the longest target language sentence
of ``sentence_aligned_corpus``
r )�len�words�max)�sentence_aligned_corpus�max_m�aligned_sentence�ms �:/usr/lib/python3/dist-packages/nltk/translate/ibm_model.py�longest_target_sentence_lengthr . s>