notes-groupDecisionMaking-rankingApplicants

grant applicants, admissions applicants, job applicants.

one thing i noticed reading business books was that you want to 'hire for strength' (and promote for strength), which means, rather than emphasizing the elimination of candidates with weaknesses, emphasize the choosing of candidates who have strengths. To rephrase, if you imagine a candidates attributes as an array, where each attribute is -10 to 10, and the population median is 0, don't worry about taking the "min" function over these arrays, instead, select the most important dimensions for this job and look at candidates with particularly high values in those -- if a candidate has a 7 in three of the most important dimensions but a -7 in another dimension, that's a good candidate.

i notice that, at least in the situations i've heard about or participated in in academia, they tend to do the other thing; if there's anything obviously wrong with any applicant you eliminate them. If five people review applications, and there's one application that excites one reviewer but that another reviewer thinks is unworthy, and if both have evidence to defend their position, that's bad -- there will be other applications with nothing obviously wrong with them and they'll be preferred even if they're not exciting, because this is a consensus-y process.

So, i'd suggest injecting some 'hiring from strength'-type thinking into committe application ranking procedures. My suggested method is pretty simple. If you can choose to accept n applications, then reserve some portion of these n slots for the top candidates of individual reviewers. E.g. maybe you say that any application which is the #1 or #2 favorite of any reviewer is accepted. How many tops you accept from each reviewer depends on how many slots you make available for this sort of thing (e.g. if you have 5 reviewers and you can accept 15 applicants, and you decide to reserve 5 slots for this process and to fill the other 5 the usual way, then each reviewer's #1 top gets accepted; if you had decided to dedicate 10 slots to this process, then each reviewers's #1 and #2 would have been accepted. You can also interpolate this process with a consensus-voting one by saying that an application must land in more than one person's top rankings to be chosen this way. E.g. you may say that an application must be chosen in a top spot by 2 of your five reviewers to get chosen this way, so if an application is #1 on one person's list and #3 on another person's list, they may get chosen (the procedure here is a little more complicated than i'm saying but it should be pretty obvious)).

Also, use reweighted score voting (a proportional voting method) for the rest of the slots.