IBM model of word alignment
The IBM’s word alignment model is used to align the words from the sentences aligned parallel corpus. Given a parallel corpus(E,F) of S sentence pairs, let e(f) be the sth sentence pair in E(F) and ei(fj) denote the ith(jth) word among a total of I(J)words in e(f). Let there be a “NULL” word e0 to account for any unaligned words in f. let VE and VF denote the size of the respective vocabularies.With each fj we associate a hidden alignment variable aj whose value ranges over [0,I]. The set of alignments for a sentence (corpus) is denoted by a(A). The model parameters consists of a VE ×VF table T of the word translation probabilities such that te,f=P(f|e). Since f is conditional on e, we refer to e(e)as the “source” word (sentence) and f(f) as the “target”
…show more content…
Koehn, H. Hoang, A. Birch, C. Callison-Burch, M. Federico, N. Bertoldi, B. Cowan, W. Shen, C. Moran, R. Zens, C. Dyer, O. Bojar, A. Constantin, and E. Herbst, “Moses: Open source toolkit for statistical machine translation,” in Proc. ACL: Demo and Poster Sessions, Prague, Czech Republic, Jun. 2007, pp. 177–180.
[9] F. J. Och and H. Ney, “A systematic comparison of various statistical alignment models”, Comput. Linguist., vol. 29, no. 1, pp. 19–51, 2003.
[10] A. Stolcke, “SRILM—An extensible language modeling toolkit,”in Proc. ICSLP, 2002,vol.3.
[11] S. F. Chen and J. Goodman, “An empirical study of smoothing techniques for language modeling,” Comput. Speech Lang., vol. 13, pp. 359–394, 1999.
[12] [Online] Available: https://lindat.mff.cuni.cz/repository/xmlui/handle/11858/00-097C-0000-0001-BD17-1
[13] J. H. Clark, C. Dyer, A. Lavie, and N. A. Smith, “Better hypothesis testing for statistical machine translation: Controlling for optimizer instability,” in Proc. ACL:HLT, Portland, OR, USA, 2011, pp. 176–181.
[14] K. Papineni, S. Roukos, T.Ward, andW.-J. Zhu, “BLEU: A method for automatic evaluation of machine translation,” in Proc. ACL, Philadelphia, PA, USA, Jul. 2002, pp.