Matrix Derivatives, Single Entry Matrix and derivatives
           of X,XTX,detX,lndetX and lndetXTX
                                Andersen Ang
                      Math´ematique et de Recherche op´erationnelle
                           Facult´e polytechnique de Mons
                                    UMONS
                                 Mons, Belgium
                          email: manshun.ang@umons.ac.be
                             homepage: angms.science
                                July 14, 2017
                                                                         1/17
  Overview
   1  Derivative involving matrix
   2  Sigle Entry Matrix
   3  Application of Single Entry Matrix in deriving matrix derivative
   4  Chain Rule with Frobenius inner product
   5  Summary
                                                                             2/17
  Derivative of matrix w.r.t. scalar
   For a matrix Y ∈ Rm×n,
                                       ∂y        ∂y          ∂y    
                                            11      12   ...     1n
                                        ∂x       ∂x           ∂x 
                                                                   
                                                                 
             y     y     ...  y        ∂y        ∂y          ∂y    
              11    12         1n       21         22   ...     2n 
           y      y     ...  y                                   
        ∂  21      22         2n= ∂x           ∂x           ∂x        (1)
            .      .    .     .                                  
       ∂x .        .     ..   .                                  
              .     .          .        .          .    .      .   
                                        .          .     ..    .   
             y     y     ...  y             .       .           .
              m1    m2         mn                                  
                                                                   
                                       ∂y       ∂y           ∂y    
                                            m1      m2   ...     mn
                                           ∂x     ∂x           ∂x
                                                                           3/17
  Derivative of sclar w.r.t. matrix
   For a matrix X ∈ Rm×n, ∂y is
                             ∂X
                                       ∂y        ∂y     ...   ∂y 
                                      ∂x        ∂x           ∂x    
                                       11          12           1n 
                                                                   
                                       ∂y        ∂y           ∂y 
                                                        ...        
                                                                   
                    ∂y                ∂x        ∂x           ∂x    
                                = 21             22           2n      (2)
            x     x     ...  x                                     
             11    12         1n       .          .     .      .   
                                  .             .      ..    .   
            x     x     ...  x         .          .            .   
       ∂ 21       22         2n                                  
          .       .    .     .                                   
          .       .     ..   .  
             .     .          .        ∂y        ∂y           ∂y 
           x     x      ...  x                           ...
             m1    m2         mn         ∂x      ∂x           ∂x
                                           m1       m2           mn
                                                                           4/17