1 
                                                                                                               
                                                                                                               
                                                                                                               
                                                                                                               
                                                                                                               
                                                                                                              DEPARTMENT OF LINGUISTICS AND ENGLISH LANGUAGE 
                                                                                                                                                                                                                                              LANCASTER UNIVERSITY 
                                                                                                               
                                                                                                                                                                                                                                                                                                                                       
                                                                                                               What do ICAO Language Proficiency Test Developers 
                                                                                                                             and Raters Have to Say about the ICAO Language 
                                                                                                                                               Proficiency Requirements 12 Years after their 
                                                                                                                                                                                                                                                                                   Publication? 
                                                                                                                                        A qualitative study exploring experienced professionals’ opinions 
                                                                                                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                 Angela Carolina de Moraes Garcia  
                                                                                                                                                                                                                                                                                                                                       
                                                                                                                         Dissertation submitted in partial fulfillment of the requirements for the 
                                                                                                                                                                                                MA in Language Testing (by distance) degree 
                                                                                                               
                                                                                                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                                              October 2015 
                                                                                                                                                                                                                                                                                                                                       
                                                2 
           
                           Abstract 
              The  International  Civil  Aviation  Organization  (ICAO)  standard  and 
          recommended  practices  (SARPs)  related  to  the  language  use  for  aeronautical 
          radiotelephony communications were published in March 2003. Twelve years after 
          their publication, in the light of research suggesting the revision of the ICAO policy, it 
          is important to learn what experts who have been working with the ICAO language 
          proficiency requirements (LPRs) think are their strengths and weaknesses according to 
          their  experiences.  This  dissertation  investigates  experienced  test  raters  and  test 
          developers’ opinions about the ICAO LPRs. Six expert professionals were interviewed 
          in this qualitative analytic research and the data were analysed in accordance with the 
          thematic analyses method. The discussions included not only general features of the 
          policy  but  also  the  specific  features  of  the  assessment  criteria.  The  research  puts 
          forward suggestions of improvements to be made to the ICAO policy and recommends 
          ICAO to revise the LPRs at the earliest.  
                        
                                                3 
           
                         Acknowledgements 
              I  would like to express my deep sense of gratitude to all my MA tutors for 
          having  assisted  me  in  my  distance  learning  process,  especially  my  dissertation 
          supervisor,  Luke  Harding, who has carefully guided me during the writing of this 
          work,  and  Tineke  Brunfaut,  the  Director  of  Studies  of  the  programme,  who  has 
          promptly assisted me whenever I needed her help. Very special thanks to the research 
          participants, whose contribution was invaluable for the success of this study. Many 
          thanks to the National Civil Aviation Agency – Brazil, the company I work for, which 
          has  kindly  sponsored  my  study,  and  also  to  my  coworkers  for  their  continuous 
          encouragement and support. I am also thankful to my family members and friends, 
          who were very supportive and encouraging. I am especially grateful to my mother for 
          having  helped  me  transcribe  the  interviews  and  for  having  kindly  reviewed  this 
          dissertation. 
                        
                                                                                                                      4 
                          
                                                             Table of Contents 
                         1. Introduction         8 
                                 1.1 The ICAO language proficiency requirements  10  
                         2. Background           13 
                                 2.1 Literature review 13 
                                         2.1.1 Issues related to the ICAO LPRs in general       13 
                                                 2.1.1.1 Characterizing radiotelephony English as ELF           13 
                                                 2.1.1.2 Plain English versus phraseology                       14 
                                                 2.1.1.3 Lack of clarity in relation to the test construct      15 
                                                 2.1.1.4 Not  taking  into  consideration  technical  knowledge  of 
                                                 operations in the language proficiency test                    17 
                                         2.1.2 Issues related to reliability     17 
                                         2.1.3 Issues related to the ICAO rating scale          18 
                                                 2.1.3.1 Pronunciation           19 
                                                 2.1.3.2 Structure               20 
                                                 2.1.3.3 Vocabulary              21 
                                                 2.1.3.4 Fluency                 22 
                                                 2.1.3.5 Comprehension           23       
                                                 2.1.3.6 Interactions            24 
                                         2.1.4 So what is next?          25 
                                 2.2 Research question           26       
                         3. Methodology          28 
                                 3.1 Overall research design  28