Issues for debate
   • Implications of the use of evaluation as 
    a management tool
   • Changing evaluation "objects".
    –systems and policies, 
    –programmes, 
    –research 'collectives', 
    –individuals and
    –institutions. 
    Evaluation as a management tool
   • Characterizing evaluation activities
    - objects
    - aims
    - approaches 
    - articulation with decision making processes
   • Different objects (Chabbal's terminology (1987)) 
     – Actors: individuals & research collectives, 
     – Operators: institutions, procedures, programmes and agencies/services, 
      Policies
            Evaluation objects: changing focus
         •   From the beginning: actors at the core of the process
                    Researchers and research teams, research results and 
             projects
                    Organizing the Peer review process 
         •   OECD and the evaluation of national systems
                    Peer reviews and indicators
         •   The eighties and the focus on research operators
                    Programmes at the core of new methodological 
             developments
         •   The nineties (1) and the fashion of privatisation
                    Focusing on the performance of research institutions
         •   The nineties (2) up to … : the fashion of 'excellence'
                    and the growing between multiple missions and one sided 
                    criteria of performance
      Evaluation as a management tool
    • Different aims 
      1- Audits
      Objective: compliance to pre-established rules, often administrative.
      Main effect: sanctions. 
      2- Assessment of Performance 
      Problems: measures, differentiating outputs from outcomes. 
      Approaches: "Summative"/"ex-post". 
      Effects: hierarchy/positioning, gratifications/ rewards.
      3- Relevance of action/activities 
      Objective: the adequate "implementation structures". 
      Approach: "pro-active" (focusing on the definition of future action).
      Effects: changes/adaptations in the course of action
      4-  Appropriateness (overall strategy) 
      Objective: discussing the aims (against the changing environment) 
      and/or their translation into goals and objectives. 
      Effects: redefinition of the action and of its course. 
          Evaluation as a management tool
     • Approaches to evaluation
         - evaluation process versus tools mobilized
         - "characterization" (of a situation) versus "judgment"
         - relations between approaches and couples "object/aim" 
         --> typical articulations 
     • The articulation with decision making processes
         - 2 different approaches: support to the “boss” or as a means 
         for collective learning.
         - 4 major constraints -->  
           - Timing    --->     to feed back in the decision making process
           - Relevance --->     to address issues at stake
           - Robustness--->     to resist critics from evaluees
           - Credibility  --->  to be taken up by stakeholders & decision makers