Cross-Efficiency in DEA: A Maximum Resonated Appreciative Model

Accepted Manuscript Cross-Efficiency in DEA: A Maximum Resonated Appreciative Model Muhittin Oral, Gholam R. Amin, Amar Oukil PII: S0263-2241(14)00603-4 DOI: http://dx.doi.org/10.1016/j.measurement.2014.12.006 Reference: MEASUR 3161 To appear in: Measurement Received Date: 8 May 2014 Revised Date: 18 July 2014 Accepted Date: 3 December 2014 Please cite this article as: M. Oral, G.R. Amin, A. Oukil, Cross-Efficiency in DEA: A Maximum Resonated Appreciative Model, Measurement (2014), doi: http://dx.doi.org/10.1016/j.measurement.2014.12.006 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. Cross-Efficiency in DEA: A Maximum Resonated Appreciative Model Muhittin Oral 1 , Gholam R. Amin 2 *, Amar Oukil 2 1 Graduate School of Business, Ozyegin University, Istanbul, Turkey 2 College of Economics and Political Science, Department of Operations Management, Sultan Qaboos University, Muscat, Oman Abstract The occurrence of multiple optimal solutions is an important and interesting issue in data envelopment analysis (DEA), for it allows flexibility to estimate the optimal cross-efficiencies of all decision making units (DMUs). This paper uses the advantage of multiple optimal solutions, be in the cases of efficient and/or inefficient DMUs, to integrate both the first and second-order voices of all DMUs and proposes a most appreciative cross-efficiency DEA method. The paper deals with multiple optimal solution cases within the context of cross- efficiency models and suggest a model that is most appreciative for all DMUs being cross-evaluated by all others. The merits and appreciative superiority of the proposed method is proven theoretically; and illustrated practically through a ranking study chosen from the literature. Keywords: Data envelopment analysis; Cross efficiency; Multiple optimal solutions; Ranking; Maximum Resonated Appreciation *Corresponding author:

(Gholam R. Amin) 1. Introduction In the field of data envelopment analysis (DEA), one important issue is the frequent occurrences of multiple optimal solutions. Such cases exist for almost all efficient decision making units (DMUs) and for some inefficient ones. Several studies have paid attention to this property, see for instance Sexton et al (1986), Doyle and Green (1994, 1995), Oral et al (1991), Cooper et al (2007), Cooper et al (2009), Oral (2010, 2012), Oral et al. (2001), Amin et al (2011), and Wang et al (2012). Theory and applications of cross-efficiency stem from recognizing the advantage of having multiple optimal solutions from DEA models. In fact, cross-efficiency is needed and introduced, as an extension to the theory of DEA, to increase the power of individual voices in peer-evaluation processes. On the other hand, due to the frequent presence of multiple optimal solutions, the cross-efficiency scores through traditional self- efficiency DEA models cannot be obtained uniquely. For a comprehensive source of DEA issues, the reader is referred to Cooper et al 2002. To remedy this situation, the concepts of secondary goals have been introduced in cross-efficiency methods. A DEA model with secondary goal usually seeks to identify the optimal weights for inputs and outputs that maximize the average efficiency scores of all DMUs from the perspective of a given DMU. Sexton et al. (1986) and Doyle and Green (1994) suggested the use of secondary goals to deal with the issues of non-uniqueness of cross- efficiency scores. In recent years, there have been even more studies especially dealing with cross-efficiency issues in DEA. Liang et al (2008) proposed alternative secondary goals to extend the initial model of Doyle and Green (1994). Ramón et al (2010) discussed on the choice of weights in cross- efficiency context. Also, Wang and Chin (2011) proposed an ordered weighted averaging (OWA) based method for aggregation of cross-efficiency 2 scores of DMUs. Wu et al (2012) suggested a method to reduce the number of zero weights for inputs and outputs in cross-efficiency models. Despite sporadic and disordered theoretical contributions, some cross- efficiency techniques have been successfully used in several DEA applications: R&D project selection (Oral et al 1991, Oral 2010, 2012); project ranking (Green et al 1996), game theory (Liang et al 2008), corporate philanthropic project evaluation (Partovi, 2012), and academic performance evaluation (Oral et al 2014) are but few examples. The primary objectives of this paper in dealing with cross-efficiency issue are fourfold: (1) recognize the appreciative voices of all DMUs while evaluating themselves and others; (2) avoid any kind of aggregated measures, especially those central tendency ones; and (3) propose a cross-efficiency method that confirms to the objectives (1) and (2); and (4) illustrate that the proposed method is both theoretically and practically a sound one. In line with the first three objectives, this paper presents a secondary goal method for cross-efficiency in DEA theory that maximizes the second- order individual appreciative voices. The possibility of having multiple solutions for both efficient and inefficient DMUs is explicitly recognized and appreciatively used in cross-efficiency estimates. More precisely, we suggest a method called “Maximum Resonated Appreciative Model,” or simply MRA, that uses the advantage of multiple optimal solutions from self-efficiency DEA models and provides the highest cross-efficiency values for each and every DMU. The need for cross-efficiency evaluations, interestingly, stems from decision making contexts where the ranking of DMUs is of a primary concern. Given the fact that CCR is not sufficient to rank DMUs simply because the presence of several efficient DMUs with the self-efficiency scores of unity, one 3 needs to find a way to distinguish the efficient DMUs from one another for ranking purpose (Despotis 2002). Hence we have cross-efficiency evaluation methods. The studies of Saxton et al (1986), Doyle and Green (1994), Anderson et al (2002), Liang et al (2008), Wu et al (2009), Ramon et al (2010), Wang and Chin (2010), Wu et al (2011), Ramon et al (2011), Yang et al (2012), Contreras (2012), Ruiz and Sirvent ( 2012), and Zerafat Angiz et al (2012) are just few examples in this regard. The other reason for inventing cross-efficiency is the claim that CCR permits unrealistic weighting schemes and that is something to be avoided without requiring the elicitation of weight restriction from experts. (Anderson et al 2002; Contreras 2012). Consequently, it has been realized that a second-order voice is needed to come up with a more reliable ranking. Therefore, as a numerical application, we chose ranking a group of professors from a Business College to show that the proposed MRA is appreciatively superior to the well-known secondary goal Benevolent Model of Doyle and Green (1994), simply BCE henceforth. Although the application is restricted to ranking, we should note that the results of our research can be used in other DEA contexts, like rating, clustering, etc. The rest of this paper is organized as follows: The next section, Section 2, offers a geometrical illustration for the existence of multiple optimal solutions in self-efficiency DEA models as well as the benefit of using multiple optimal solutions to generate better cross-efficiency scores. Section 3 is devoted to a brief summary of self-efficiency and cross-efficiency DEA models, but focusing on CCR of Charnes et al (1978) and BCE of Doyle and Green (1994). Section 4 focuses on the cross-efficiency model of Oral et al (1991), which is renamed as MRA, by observing that MRA aims to maximize the secondary goal in a most resonated appreciative way. These superior properties of MRA, when compared with BCE, are ascertained by theoretical proofs. Section 5 is devoted to a numerical example where cross-efficiency is used to rank a group of professors. Section 6 offers some discussion whereas Section 7 4 concludes the paper with some remarks regarding the merits of MRA and possibilities for further research. 2. Motivation This section provides a geometrical interpretation for the existence of multiple optimal solutions in the case of self-efficiency DEA models. We use a two-dimensional example to show the existence of multiple optimal solutions as well as the potential use of these multiple optimal solutions in cross- efficiency calculations. Let us start with a geometrical illustration for multiple optimal solutions in self-efficiency DEA models through the numerical example given in Table 1. [Please insert Table 1 about here] Figure 1 shows the production possibility set (PPS) for the example given in Table 1. [Please insert Figure 1 about here] According to Figure 1, there are four efficient DMUs: A, B, C, and D; and two inefficient ones: E and F. Here, we illustrate the case of multiple optimal solutions geometrically. To start, let us first consider the efficient DMU-A shown on the efficiency frontier and the two line segments AA and AB of the frontier passing through A. These two line segments have the following equations. AA : 0.125 y 2 = 1 AB : 0.038462 y 1 + 0.115385 y 2 = 1 5 On the other hand, the self-efficiency DEA model for DMU-A can be written as follows (Charnes et al., 1978). max 2u1 + 8u 2 s. t. 2u1 + 8u 2 ≤ 1, 5u1 + 7 u2 ≤ 1 6u1 + 5u 2 ≤ 1, 7 u1 + 3u2 ≤ 1 2.5u1 + 3.5u 2 ≤ 1, 5u1 + 3u 2 ≤ 1 u1 ≥ 0 , u2 ≥ 0 This model has the following alternative optimal solutions. (u1* , u2* ) = ( 0 , 0.125) (u1* , u2* ) = ( 0.038462 , 0.115385) that correspond to the coefficients of line segments A A and AB equations, respectively. There are also two line segments passing through DMU-B; namely, AB and BD. The latter has the following equation. BD : 0.117647 y1 + 0.058824 y 2 = 1 If we use the self-efficiency DEA model for DMU-B we obtain two optimal solutions which, again, correspond to the supporting hyperplanes AB and BD. We have the same situation for the efficient DMU-D, i. e. two optimal solutions corresponding to line segments BD and DD , where DD can be represented by the equation below. DD : 0.142857 y1 = 1 Now, consider the efficient DMU-C. As one can observe from Figure 1 and equation BD, this DMU is on a single line segment and equivalently there is only one supporting hyperplane passing from it. Graphically, this means that the self-evaluation DEA model for DMU-C has a unique optimal solution, contrary to the cases of DMUs A, B, and D. On the other hand, the target for DMU-E, as the reference point, is the efficient DMU-B itself which has two supporting hyperplanes. So, the self-efficiency DEA model for DMU- E has also two alternative optimal solutions corresponding to line segments AB and BD. 6 Based on the above, we can divide our observations into two groups: DMUs A, B, D, and E in one group and DMUs C and F in the other. The DMUs in the latter are those DMUs having unique optimal solutions whereas the DMUs in the first group have multiple optimal solutions each. Now, let us explain how the cross-efficiency calculation process can benefit from the existence of multiple optimal solutions. The aim of a cross-efficiency DEA method is to use individual DMU voices in the evaluation of all other DMUs. In simple words, what are the efficiency scores of the other DMUs from the perspective of a specific base DMU? These cross-efficiency scores, in accordance with the appreciative theory (Cooperrider et al. 2008), should be optimally obtained by keeping the self-efficiency score of the base DMU unchanged. The word “appreciative” accentuates the positive, and avoids the negative, when an individual, unit, department, organization, or a situation is being assessed, evaluated, or considered. The concept of appreciative approach, like “appreciative inquiry”, takes the idea of the social construction to its positive extreme (Cooperrider et al 2008). According to Figure 1, the traditional cross-efficiency scores cannot be calculated uniquely when the base DMU is one of the DMUs having multiple optimal solutions. This is the main reason for the development of several alternative DEA models, popularly known in the DEA literature as “secondary goal” approaches. In this paper, we aim to suggest a cross- evaluation DEA model which can give the best appreciative individual voices in favor of other DMUs. To illustrate this, graphically, consider a cross- efficiency case where DMU-A is used as the base DMU to judge DMU-B. More specifically, this is the cross-efficiency of DMU-B from the perspective of DMU-A. According to the discussion above, the self-efficiency DEA model for DMU-A has two optimal solutions. The best appreciative voice of DMU-A in favor of DMU-B can be heard through the line segment AB. The reason is 7 clear since the base DMU-A keeps its own self-efficiency on this segment unchanged and gives the highest possible value to the cross-efficiency of DMU-B. So, the cross-efficiency of DMU-B from the perspective of DMU-A is equal to one. Mathematically, the cross-efficiency score of a DMU from the perspective of a base DMU is equal to 1 if and only if both DMUs have common optimal solution(s) from their respective self-efficiency DEA models. The appreciative voice of DMU-A in the cross-efficiency evaluation of other DMUs can be interpreted as one of the line segments A A and/or AB depending on the values given to their cross-efficiencies. The next two sections of this paper show how this idea of maximum resonated appreciation can be formally generalized as a superior cross- efficiency DEA method and then can be used in both DEA theory and practice. 3. Cross-Efficiency Models in DEA The primary objective of cross-efficiency models in DEA is to allow all DMUs to participate in the process of peer-efficiency evaluation. This entails the use of both self and cross-efficiency DEA models. Sexton et al. (1986) were the first researchers who proposed the concept of cross-efficiency, albeit in a rather simple form. Since then however there have been developments both theoretically and practically. All of the developed methods benefit from the existence of multiple optimal solutions in the self-evaluation DEA model. In an attempt in that direction, Sexton et al. (1986) constructed a secondary goal method for cross-efficiency in DEA context. One of the most well-known secondary goal models is the Benevolent Cross-Efficiency (BCE) due to Doyle and Green (1994). 8 In what follows, we present and interpret MRA. Assume that we have n DMUs, each of which with s outputs and m inputs. More specifically, let x j = ( x1 j ,… , x mj ) and y j = ( y 1 j ,… , y sj ) be the vectors of inputs and outputs of DMU-j ( j = 1,… , n ), respectively. Then the self-efficiency DEA model, or CCR (Charnes et al. 1978), for DMU-k ( k = 1,… , n ) can be written as follows. CCR Self-Efficiency DEA Model s Ekk* = max ∑y r =1 rk ur s. t. m ∑x i =1 ik vi = 1 s m ∑r =1 y rj ur − ∑x v i =1 ij i ≤ 0 j = 1,… , n ur ≥ 0 r = 1,… , s , v i ≥ 0 i = 1,… , m where, DMU-k is the one being evaluated; v = ( v1 ,… , v m ) and u = (u1 ,… , us ) are vectors of weights to be determined optimally for inputs and outputs, respectively, and Ekk* is the cross-efficiency score of DMU-k obtained from CCR. We know that the efficient DMUs that are on the intersection of at least two supporting hyperplanes of PPS have multiple optimal solutions (like DMUs A, B, and D in Figure 1) as well those “special” inefficient ones like DMU-E in Figure 1. Therefore, the conventional cross-efficiency scores from the perspective of these extreme efficient and “special” inefficient DMUs cannot be calculated uniquely. In fact, the conventional cross-efficiency for a given DMU may have different values depending upon the use of different software (Cooper et al. 2007). This is perhaps the main reason for which there are several DEA models for cross-evaluation in the literature. However, we 9 will focus on the most well-known secondary goal model of Doyle and Green (1994); that is, BCE. This model is shown below. BCE – Benevolent Cross-Efficiency Model s n  max ∑ ∑   y rj  ur r = 1  j =1 , j ≠ k   s. t. m  n  ∑ ∑   xij  v i = 1 i =1  j = 1, j ≠ k   s m ∑ r =1 y rj ur − ∑x v i =1 ij i ≤ 0 j = 1,… , n, j ≠ k s m ∑ r =1 y rk ur − Ekk* ∑x i =1 ik vi = 0 ur ≥ 0 r = 1,… , s, v i ≥ 0 i = 1,… , m Note that in the above formulation of BCE, we use, for the sake of simplicity, the variables u r and v i rather than u rk and v ik , for all r = 1,… , s, i = 1,… , m . The objective of BCE is to maximize the cross-efficiency values of all DMUs from the perspective of DMU-k. The last n constraints of BCE ensure that the self-efficiency of DMU-k remains unchanged. Assume ( u * , v * ) = (u1* ,… , u s* , v 1* ,… , v m* ) is an optimal solution from BCE. Then, the cross-efficiency of DMU-l from the perspective of DMU-k can be calculated as follows. ∑ s * r =1 y rl ur* B = l = 1,… , n, l ≠ k ∑ kl m x v* i =1 il i As we will deductively prove and inductively justify later, BCE does not necessarily give the most appreciative cross-efficiency scores when compared with those obtained from MRA. 10 4. A Most Appreciative Cross-Efficiency Although Oral et al (1991) are the first researchers who formulated MRA Model and also used it in a real-life context, they did not however provide any proof or explanation as to its merits in theory and its applications in practice. This paper focuses their cross-efficiency DEA model by showing its superiority, both theoretically and empirically. The optimal solutions from CCR for any DMU, say DMU-k, can be classified into two mutually exclusive and collectively exhaustive cases: Case 1 : I = {k : Ekk* can be obtained uniquely} , implying CCR Model has one single optimal solution for DMU-k, ∀k ∈ I . Case 2: J = { k : 1 ≤ k ≤ n} − I , implying CCR Model has multiple optimal solutions for DMU-k, ∀k ∈ J . For all DMU-k, k ∈ J , we can now suggest the following cross-evaluation DEA model. MRA – Maximum Resonated Appreciative Model s A kl* = max ∑y r =1 rl ur s. t. m ∑x v i =1 il i =1 s m ∑r =1 y rj ur − ∑x v i =1 ij i ≤ 0 j = 1,… , n, j ≠ k s m ∑r =1 y rk ur − Ekk* ∑x v = 0 i =1 il i ur ≥ 0 r = 1,… , s, v i ≥ 0 i = 1,… , m 11 where, A kl* is the cross-efficiency score of DMU-l from the perspective of DMU-k obtained from MRA, for all k ∈ J and l = 1,… , n, l ≠ k . Here again we use, for simplicity, the notations ur and v i rather than u rk and v ik . MRA seeks to find the best possible cross-efficiency score for DMU-l over all multiple optimal solutions of CCR. Actually, MRA takes into consideration the existence of multiple optimal solutions from the self- evaluation of CCR as a flexible tool to find the best cross-efficiency possible for DMU-l. In practical words, this means that DMU-k has the highest level of appreciation, maximum appreciation as a matter of fact, in the evaluation of the cross-efficiency of DMU-l. This is achieved by keeping the self-efficiency of DMU-k unchanged through the last n constraints in MRA. Furthermore, MRA needs to be solved n-1 times for each DMU-k ( k ∈ J ) in order to calculate the cross-efficiency scores of all n-1 DMU-l for l = 1,… , n, l ≠ k . The cross- efficiency score of DMU-l from the perspective of DMU k is the optimal value of MRA, that is s A kl* = ∑y r =1 rl ur* We do not need to use MRA for those DMUs belonging to Case 1, it suffices to substitute the unique optimal solution of CCR in the left-hand side expression corresponding to the DMU in question to get its cross-efficiency score. Repeating the same process for all other DMUs we complete the cross- efficiency scores for all DMUs. Now, we prove the appreciative superiority of MRA to BCE. Let us assume that DMU-k belongs to Case 2, or k ∈ J . First, we need to establish the fact that self-efficiency scores are greater than or equal to cross-efficiency scores ( Ell* ≥ A kl* ); indicating that the first-order appreciative voice ( Ell* ) is stronger than the second-order appreciative voice ( A kl* ). Hence 12 Theorem 1 : For any k ∈ J and l = 1,… , n and l ≠ k we have A kl* ≤ Ell* Proof: Let ( u * , v * ) be an optimal solution from MRA for DMU-l where u * = (u1* ,… , us* ) and v * = (v1* ,… , v m* ) . Moreover, let ∑ m v* xl = i =1 x il v i* = δ Also, let ( uˆ ,vˆ ) be an optimal solutions from CCR. Clearly, ( uˆ , vˆ ) = δ −1 ( u * , v * ) is a feasible solution since it is in the feasible region defined by CCR for DMU-l. Therefore, the objective value with this solution should be less than or equal to the optimal value from the CCR. That is u*yl δ −1 u * y l = ≤ Ell* v* xl This concludes that A kl* ≤ Ell* .■ Now, we shall prove that MRA outperforms BCE in terms of appreciation maximization. The theorem reads as follows: Theorem 2 : For any k ∈ J and l = 1,… , n and l ≠ k we have Bkl* ≤ A kl* Proof: Assume that ( uˆ * , vˆ * ) and ( u * , v * ) are the optimal solutions from BCE and MRA, respectively. 1 * * Obviously, ( u , v ) = ( uˆ , vˆ ) is a feasible solution from MRA, where α m α = vˆ * x l = ∑ x vˆ i =1 il * i Therefore, s s uyl = ∑r =1 y rl ur ≤ u * y l = ∑y r =1 rl ur* 13 The above inequality is true because (u * , v * ) is an optimal solution from MRA. By substitution we have 1 * uˆ * y uyl = uˆ y l = * l ≤ u * y l α vˆ x l which means that Bkl* ≤ A kl* for all k ∈ J l ∈ {1,… , n}, l ≠ k . This completes the proof. ■ Given the above two theorems, we can state the following two corollaries. Corollary 1: For any k ∈ J and l = 1,… , n and l ≠ k we have Bkl* ≤ A kl* ≤ Ell* Proof: The proof is straightforward using the results of Theorems 1 and 2. ■ The following corollary shows that BCE and MRA give the same cross- efficiency for DMU-l from the perspective of DMU-k when k ∈ I . Corollary 2: For any k ∈ I and l = 1,… , n and l ≠ k we have Bkl* = A kl* Proof: Since k ∈ I , CCR corresponding to DMU-k has a unique optimal 1 * * 1 * * solution, say ( u * , v * ) . Obviously, (u , v ) and (u , v ) are optimal α β solutions for BCE and MRA, respectively, where m  n  * m α= ∑  ∑ x ij  v i and β =  ∑x v il * i i =1  j = 1, j ≠ k  i =1 This implies that u * yl Bkl* = A kl* = v* xl This completes the proof. ■ 14 The above results indicate that when CCR for DMU-k has multiple optimal solutions then MRA outperforms BCE. 5. A numerical illustration In Section 4, we proved that Bkl* ≤ A kl* , implying that MRA is more appreciative than BCE in determining cross-efficiency scores. To assess the two secondary goal methods, BCE and MRA, we chose the ranking context described in Oral et al (2014), a case study describing how faculty performance evaluation can be done. As can be seen from Table 2, the dataset consists of 32 faculty members of a business school with one input (current annual salary in $1,000 -column 7) and five outputs – columns 2-6, namely, the average number of peer reviewed articles published in recognized academic journals, the average number of peer reviewed articles published in proceedings, the average monetary contributions to the school through research funding and consulting projects, the teaching scores of the faculty members, and the citizenship score of each faculty member. The last column includes the self-efficiency scores obtained from CCR. [Please insert Table 2 about here] CCR identifies eleven efficient professors (DMUs), as highlighted in Table 2. For all these efficient professors, CCR has multiple optimal solutions, implying that Case 2 applies. Consequently, the remaining professors belong to Case 1 and therefore we don’t need to use MRA nor CBE, since their cross- efficiency scores can be uniquely obtained by using the conventional method. In order to rank the professors, Oral et al (2014) suggested a non-DEA model based on an explicit count of the number of votes. A dummy variable ϕ ijk is 15 used to decide whether Professor-i is a better performer than Professor-j according to Professor-k, that is ϕ ijk = 1 if Eik > E jk and ϕ ijk = 0 otherwise, ∀k ≠ i ≠ j . The number of votes in favor of Professor-i is N i = ∑N j ij , where N ij = ∑k ϕijk represents the number of times Professor-i is judged to be more efficient than Professor-j, ∀i ≠ j . The above approach is applied to rank the sample of professors using both MRA and BCE models cross=efficiency matrices. The number of votes and the corresponding rankings from the highest to the lowest vote are given in Table 3. [Please insert Table 3 about here] First, we notice that the total number of votes counted through BCE is larger. This observation may suggest that MRA is less appreciative than BCE. To statistically assess whether this suggestion is supportable, a Wilcoxon signed- rank test was conducted (see, e.g., Anderson et al. 2011, Chapter 19, pp 820) with the following hypotheses: H0: m MRA − m BCE ≤ 0 Versus H1: m MRA − m BCE > 0 where m MRA and m BCE are the median numbers of votes obtained with MRA and BCE, respectively. Here, the question is whether the median difference between the pairs of number of votes is greater than zero. With the sum of the signed-rank values T= - 236, we have the test statistic z=- 2.21. Using the standard normal distribution, we obtain p-value=0.9863. Therefore, we have no reason to reject the null hypothesis and, hence, conclude that BCE supersedes MRA with respect to the number of vote counts. 16 With respect to the ranking, the results of MRA differ from that of BCE for almost 65% of the professors. Interestingly, the two methods rank most of the efficient DMUs among the best, with almost the same ranking positions for both. Meanwhile, for some pairs of DMUs, the ranking positions seem to be swapped with each other, such as between P21 and P15 for the 3rd and 4th ranking positions, between P26 and P13 for the 25 th and the 26 th positions and between P2 and P3 for the 21st and 22nd ranking positions. The ranking gaps are much wider for the remaining DMUs, like P20 that ranks 12 th with MRA and 16 th with BCE. Such discrepancy between the outcomes of MRA and BCE can be imputed to (1) the difference between the cross-efficiency matrices provided with the two approaches (2) the ranking criteria used by the non-DEA model (Oral et al., 2014). In an attempt to explain the latter point, let’s consider the following scenarios. Scenario 1: The cross-efficiencies of Professor-a and Professor-b from the perspective of Professor-k are, respectively, Eak = 0.765 and Ebk = 0.764. The appreciation gap Eak - Ebk is only 0.001 but nevertheless ϕ abk = 1 . Scenario 2: The cross-efficiencies of the same professors from the perspective of Professor-l are, respectively, Eal = 0.995 and Ebl = 0.005. Once again, ϕ abl = 1 even if the appreciation gap is now Eal - Ebl = 0.990. According to these scenarios, it appears that the definition of ϕijk , although conform to the democratic spirit, overlooks the appreciation gap between the assessed professors. One way to integrate such a feature in the non-DEA model is by redefining ϕijk as ϕijk = Eik - E jk if Eik > E jk and ϕ ijk = 0 otherwise, ∀k ≠ i ≠ j . As a result, instead of simply counting the number of votes in favour of Professor-i, we cumulate the amount of appreciation he/she receives 17 using the same formulae as in the non-DEA model. Table 4 gives the new results. [Please insert Table 4 about here] As can be observed from Table 4, more than 53% of the DMUs are equally ranked with MRA and BCE models. Again, the majority of efficient DMUs are ahead of the list, while for a few of them, the ranking gap seems too large between the two methods, as the case of P20 and P11. With respect to the amounts of appreciation, there is no doubt that the modified non-DEA model preserves the appreciative nature of DEA for at least two observations: (1) the total amount of appreciation is higher for MRA than for BCE, 694.08 versus 679.24, and (2) the individual appreciativeness score of each and every single professor is higher for MRA than for BCE, albeit the differences for some are rather small. Thus, the appreciative potential of MRA is once again emphasized. Although such an aspect may be diluted by the ranking approach, it remains rather vital for a rating process, where bonuses need to be assigned to the faculty members (Oral et al., 2014). 6. Discussion There are at least two fundamental issues that need to be taken into consideration while dealing with cross-efficiency calculations in decision making contexts, especially in the human resource management related areas like performance evaluation: (1) voicing appreciation and (2) avoiding aggregation. Below are brief explanations why these two issues are of serious concern. 18 6.1: Voicing Appreciation Cross-efficiency is needed to integrate the voices of all DMUs about all DMUs when peer-evaluation is in order and required. Again, in the spirit of CCR, one needs to develop a cross-efficiency model that will result in offering the best results for the DMUs being evaluated by a base DMU. The meaning of this statement is that every DMU being evaluated must be given the best chance to have the highest evaluation by the DMU in the evaluating position. This is nothing but maximizing the second-order appreciative voice for the DMU being evaluated. The second-order maximum voices create a decision making context in which every DMU evaluates every DMU positively at the highest levels possible. MRA is such a model, because it appreciates other DMUs most appreciatively. The other main feature of MRA is the fact that every single DMU has a voice and it counts, since no aggregation process takes place. 6.2: Avoiding Aggregation Any aggregation, especially through central tendency measures like simple average or weighted average, is against the very philosophy of DEA, and particularly contrary to the spirit of CCR. The basic idea behind CCR is to give a “voice” to all DMUs in their own efficiency evaluations. CCR did not opt for an aggregative model, for instance like a multiple regression or a similar one. To be consistent with the philosophy of CCR, cross-efficiency models should also avoid any kind of aggregations. Otherwise, it will be another procedure, but not in the spirit of original CCR formulation. From another perspective, aggregation means individual voices do not count in decision making process, but only their aggregation. There are decision making contexts where the appreciative voices, both first-order and second- order, of all DMUs need to be taken into consideration individually one by 19 one (Oral, 2010, 2012). Aggregation avoids appreciative voices totally; therefore cross-efficiency evaluations should avoid aggregation totally. This is even more so when participative or collective decision making is of prime importance. MRA does not employ any aggregation process and from this perspective it is very much in line with the spirit of CCR, which is really origin of all DEA models. 7. Concluding Remarks As concluding remarks, this paper has investigated the issue of cross- evaluation context in DEA attesting the importance of the role of multiple optimal solutions. It is shown that the results of classical secondary goal DEA models can be improved by considering the best individual appreciative voices in the modeling of cross-evaluation processes. This improvement has been shown by introducing a superior cross-evaluation DEA model – MRA, the merits of which are shown both theoretically and practically through a case study chosen from the literature. The studied case dealt with a group of professors from a Business College and it has been shown that the appreciation potential of MRA in ranking is significant. As further research, the results of this paper can be used in several decision making contexts including DEA clustering, project selection, country risk assessment, country competiveness, and human resource management. Acknowledgments The authors would like to express their sincere appreciation to the reviewers and the editor of Measurement for their constructive comments which contributed substantially to improving the current paper. 20 References Amin, GR. Emrouznejad, A. Rezaei, S. 2011. Some clarifications on the DEA clustering approach. European Journal of Operational Research, 215: 498- 501. Anderson, T.R., Hollingsworth, K., Inman, L. 2002. The fixed weighting nature of cross-evaluation model. Journal of Productivity Analysis. 17: 249- 255. Anderson, D.R., Sweeney, D.J., Williams, T.A. 2011. Statistics for Business and Economics. 11th edition, South-Western, Cengage Learning. Charnes A, Cooper WW, Rhodes E. 1978. Measuring the efficiency of decision making units. European Journal of Operational Research. 2: 429–444. Contreras, I. 2012. Optimizing the rank position of the DMU as a secondary goal in DEA cross-evaluation. Applied Mathematical Modeling. 36: 2642- 2648. Cooper, W.W, Seiford, L.M., Tone, H. 2002. Data Envelopment Analysis: DEA Solver Software. Kluwer Academic Publisher, NY. Cooper, W. W. Ruiz, J. L., Sirvent, I. 2007. Choosing weights from alternative optimal solutions of dual multiplier models in DEA. European Journal of Operational Research, 180: 443-458. Cooper, W. W. Ruiz, J. L., Sirvent, I. 2009. Selecting non-zero weights to evaluate effectiveness of basketball players with DEA. European Journal of Operational Research. 195(2):563-574. Cooperrider, D.L., Whitney D., Stavros J.M. 2008. Appreciative Inquiry Handbook: For Leaders of Change, Crown Custom Publishing and Berrett- Koehler Publishers. Despotis, D.K. 2002. Improving the discriminating power of DEA: Focus on globally efficient units. Journal of the Operational Research Society. 53: 314-323. 21 Doyle J.R., Green R.H. 1994. Efficiency and cross-efficiency in DEA: derivations, meanings and uses. Journal of the Operations Research Society. 45: 567–78. Doyle, J.R. & Green, H.R. 1995. Cross-evaluation in DEA. INFOR. 33:205-222. Green, R.H., Doyle, J.R., Cook, W.D. 1996. Preference voting and project ranking using DEA and cross-evaluation. European Journal of Operational Research. 90: 461-472. Liang, L., Wu, J., Cook, W.D., Zhu, J., 2008. The DEA game cross-efficiency mode and its Nash equilibrium. Operations Research 56 (5): 1278–1288. Liang, L., Wu, J., Cook, W.D., Zhu, J., 2008. Alternative secondary goals in DEA cross-efficiency evaluation. International Journal of Production Economics 113, 1025–1030. Oral, M., Kettani, O., Lang, P. 1991. A methodology for collective evaluation and selection of industrial R&D projects. Management Science 37(7): 871– 885. Oral, M., Kettani, O., Çınar, Ü. 2001. Project evaluation and selection in a network of collaboration: A consensual disaggregation multi-criterion approach. European Journal of Operational Research. 130: 332-346. Oral, M., Oukil, A., Malouin, J.L., Kettani, O. 2014. The appreciative democratic voice of DEA: A case of faculty academic performance evaluation. Socio-Economic Planning Sciences. 48(1): 20-28. Oral, M. 2012. Action Research contextualizes DEA in a multi-organizational decision-making process, Expert Systems with Applications. 39: 6503-6513. Oral, M., 2010. E-DEA: Enhanced data envelopment analysis. European Journal of Operational Research, 207: 916-926. Partovi, F.Y. 2011. Corporate philanthropic selection using data envelopment analysis. Omega 39: 522–527. 22 Ramon, N., Ruiz, J.L., Sirvent, I. 2011. Reducing differences between profiles of weights: A “peer-restricted” cross-efficiency evaluation. Omega. 39: 634-641. Ramón, N. Ruiz, J. L., Sirvent, I. 2010. On the choice of weights profiles in cross-efficiency evaluations. European Journal of Operational Research, 207: 1564-1572. Ruiz, J.L., Sirvent, I. 2012. On the DEA total weight flexibility and the aggregation in cross-efficiency evaluations. European Journal of Operational Research. 223: 732-738. Sexton, TR, Silkman RH, Hogan AJ. 1986. Data envelopment analysis: critique and extensions. In: Silkman RH, editor. Measuring Efficiency: An Assessment of Data Envelopment Analysis. San Francisco, CA: Jossey-Bass. Wang, YM., Chin, KS., Wang, S., 2012. DEA models for minimizing weight disparity in cross-efficiency evaluation. Journal of the Operational Research Society, 63(8): 1079-1088. Wang, Y-M, Chin K-S. 2011. The use of OWA operator weights for cross- efficiency aggregation. Omega. 39: 493-503. Wang, Y-M., Chin, K-S., 2010. Some alternative models for DEA cross- efficiency evaluation. International Journal of Production Economics. 128: 332-338. Wu, J, Liang,L., Zha, Y., Yang, F. 2009. Determination of cross-efficiency under the principle of rank priority in cross-evaluation. Expert Systems with Applications, 36: 4826-4829. Wu, J., Sun, J., Liang, L., Zha, Y. 2011. Determination of weights for ultimate cross-efficiency using Shannon entropy. Expert Systems with Applications. 38: 5162-5165. 23 Wu, J. Sun, J. Liang, L. (2012). Cross efficiency evaluation method based on weight-balanced data envelopment analysis model. Computers & Industrial Engineering, 63, 513-519. Yang, F., Ang, S., Xia, Q., Yang, C. 2012. Ranking DMUs by using interval DEA cross-efficiency matrix with acceptability analysis. European Journal of Operational Research, 223: 483-488. Zerafat Angiz, M. Mustafa, A. Kamali, MJ. 2012. Cross-ranking of Decision Making Units in Data Envelopment Analysis. Applied Mathematical Modelling, 37, 398-405. 24 Figure y2 A A B C F E D F y1 O D Figure 1: PPS for the Numerical Example 25 Tables: Table 1: Numerical Example DMUs Output 1 ( y 1 ) Output 2 ( y 2 ) Input ( x ) A 2 8 1 B 5 7 1 C 6 5 1 D 7 3 1 E 2.5 3.5 1 F 5 3 1 Table 2: Academic Performance Data Faculty Outputs Input CCR Article Proceeding Teaching Funding Citizenship Salary efficiency P1 1.12 0.87 5.00 176 3.60 220K 0.981 P2 0.66 0.16 4.50 124 2.00 185K 0.916 P3 0.69 0.23 4.10 132 3.10 184K 0.917 P4 0.42 1.03 3.60 62 3.50 155K 0.777 P5 0.34 0.81 3.70 57 1.50 152K 0.757 P6 0.1 0.75 2.80 36 3.80 172K 0.582 P7 0.54 0.65 3.10 87 2.20 154K 0.770 P8 0.16 0.71 2.50 101 0.60 147K 0.821 P9 0.92 0.62 2.20 12 4.40 205K 0.638 P10 0.20 0.10 3.80 8 4.60 138K 0.748 P11 0.10 1.30 2.70 12 4.60 135K 1.000 P12 0.25 0.70 3.50 8 3.50 129K 0.719 P13 0.41 0.32 4.10 17 3.20 132K 0.728 P14 0.30 0.25 4.20 54 3.20 135K 0.861 P15 0.66 0.66 4.60 72 3.80 138K 1.000 P16 0.80 0.20 4.10 62 3.00 134K 0.893 P17 0.90 0.90 3.30 123 2.30 147K 1.000 P18 1.00 1.30 3.60 95 3.70 142K 1.000 P19 0.75 1.60 2.50 65 2.10 126K 1.000 P20 0.85 0.90 3.50 16 3.90 122K 1.000 P21 1.00 0.80 2.90 87 3.40 118K 1.000 P22 0.60 0.00 4.40 34 2.70 104K 1.000 P23 0.50 0.40 4.00 27 3.10 100K 0.997 P24 0.20 0.40 3.40 18 2.50 98K 0.831 P25 0.00 0.60 3.60 20 1.20 95K 0.941 26 P26 0.40 0.40 2.50 10 1.80 92K 0.763 P27 0.35 0.60 3.80 18 4.10 92K 1.000 P28 0.20 0.20 4.10 9 1.40 91K 0.976 P29 0.20 0.00 4.50 7 1.00 93K 1.000 P30 0.25 0.20 3.30 5 0.00 75K 0.988 P31 0.00 0.20 3.30 5 0.00 75K 0.959 P32 0.00 0.00 3.60 6 0.00 73K 1.000 Table 3: Academic Ranking with Non-DEA Model Number of Votes Rank Faculty MRA BCE MRA BCE P1 544 546 P27 P27 P2 353 362 P18 P18 P3 356 343 P21 P15 P4 357 368 P15 P21 P5 260 276 P23 P23 P6 84 88 P22 P22 P7 277 292 P17 P17 P8 231 235 P29 P29 P9 56 43 P19 P28 P10 150 142 P28 P32 P11 298 233 P32 P25 P12 244 248 P1 P19 P13 271 276 P25 P1 P14 406 428 P20 P30 P15 754 786 P30 P16 P16 485 504 P16 P31 P17 672 696 P31 P24 P18 769 796 P24 P20 P19 580 553 P14 P14 P20 533 451 P4 P4 P21 764 774 P3 P2 P22 719 735 P2 P3 P23 740 752 P11 P7 P24 442 454 P7 P5 P25 542 565 P26 P13 P26 274 263 P13 P26 P27 810 833 P5 P12 P28 570 590 P12 P8 P29 587 598 P8 P11 P30 518 513 P10 P10 P31 457 471 P6 P6 P32 567 586 P9 P9 Total 14670 14800 27 Table 4: Academic Ranking with Modified Non-DEA Model Amount of Rank Faculty Appreciation MRA BCE MRA BCE P1 23.57 23.20 P27 P27 P2 20.48 20.14 P18 P18 P3 20.86 20.39 P21 P21 P4 20.30 20.17 P23 P23 P5 18.89 18.77 P15 P15 P6 13.75 13.25 P22 P22 P7 19.20 19.06 P17 P17 P8 18.32 18.01 P19 P28 P9 11.33 9.51 P28 P19 P10 16.43 15.66 P29 P29 P11 18.87 17.06 P1 P1 P12 18.08 17.66 P20 P25 P13 18.97 18.73 P25 P32 P14 21.34 21.14 P30 P30 P15 26.01 25.89 P32 P16 P16 22.63 22.36 P16 P20 P17 25.10 24.82 P31 P24 P18 26.88 26.73 P24 P31 P19 24.29 23.53 P14 P14 P20 23.36 21.78 P3 P3 P21 26.78 26.49 P2 P4 P22 25.69 25.45 P4 P2 P23 26.23 25.96 P7 P7 P24 21.71 21.56 P13 P5 P25 23.24 23.02 P5 P13 P26 18.58 17.82 P11 P8 P27 27.71 27.55 P26 P26 P28 23.94 23.64 P8 P12 P29 23.77 23.42 P12 P11 P30 22.96 22.39 P10 P10 P31 21.86 21.47 P6 P6 P32 22.95 22.63 P9 P9 Total 694.08 679.24 28 DEA : SELF-EFFICIENCY DEA : CROSS-EFFICIENCY NON-DEA MODEL DECISION Research Highlights This paper • provides an illustration of multiple optimal solutions for both efficient and inefficient DMUs, • deals with multiple optimal solution within the context of cross- efficiency models, and • introduces a model that is most appreciative for all DMUs being cross- evaluated by all others. • proves the appreciative superiority of the method theoretically and numerically for a case of ranking in DEA. 29