A Procedure for Managing Conflict of Interest When Forming Thesis Juries

Conflicts of interest in thesis juries, when not identified and appropriately managed, can lead to an uncritical and unfair evaluations that can then undermine trust in the process and threaten academic credibility and institutional reputation. This Commentary presents and justifies the choices made in developing a practical procedure to identify and manage conflicts of interest in the formation of Masters and PhD juries in the School of Public Health (ESPUM) at the University of Montreal.

Thanks to Lise Lamothe and François Béland for their constructive feedback on the manuscript, and to members of the Graduate Studies Committee in the School of Public Health (ÉSPUM), Université de Montréal, who provided feedback on drafts of the procedure and declaration form presented in this paper.Thanks also to the peer-reviewers and to the editors for their very helpful suggestions for restructuring and clarifying the presentation of the paper.The procedure presented here is the practical culmination of more than a decade of research with members of the Research Ethics and Integrity Group (formerly the Conflict of Interest Research Group), which was supported by grants from the Quebec Fonds de recherche -société et culture (FQR-SC), the Ethics Office of the Canadian Institutes of Health Research (CIHR), and the Office of the Vice Principal Human Resources and Planning, Université de Montréal.

Introduction
In 2012, I published a case study in this journal, entitled Choosing Thesis Juries: The Costs of Taking a Strict Line on Conflicts of Interest [1], inspired by my first experiences as a professor in forming thesis juries for my Masters and PhD students at the Université de Montréal.The heart of the issue in this case study was the challenge of working with a very rigid rule for managing conflict of interest (COI) that prohibited the inclusion of jury members with whom the student or supervisor had published within the previous 5 years.The aim of this rule was clearly to avoid supervisors "stacking the jury" with their friends or those who might be perceived as giving an unduly favourable review, because such behaviour could 1) reduce the quality (e.g., critical review, objectivity, fairness) of the thesis evaluation process (e.g., by permitting lower quality theses to be accepted or even highly ranked), 2) undermine trust by members of the institution in their colleagues (or even entire departments) found have had problematic COI in their juries, and so 3) lead to an important loss of institutional reputation, from the perspective of both institutional members and the general public.Research institutions valorise academic integrity, scientific rigour and public credibility and so cannot permit COI situations to continue to be poorly managed without incurring great risk.
While having the advantage of being clear, this rule was limited in its scope because other potential COI where ignored and its application relied on a literature search to detect collaborations.More problematic, however, was that it resulted in the blanket exclusion from thesis juries those professors who were experts in the field but had "unfortunately" already collaborated with the supervisor and/or student.This situation is especially challenging in very specialized fields such as bioethics or public health where there is a limited pool of potential experts, and exacerbated in a context where the language of the thesis is not English, such as is the case at the Université de Montréal, a Frenchlanguage institution.The case study sought to highlight the diversity of COI that can and should be ISSN 1923-2799 2 / 21 taken into consideration in forming thesis juries (e.g., authorship and grant collaborations, personal and professional relations, practical considerations, issues of expertise and fairness), and that are part of the broader set of real and potential COI that professors face in their day-to-day activities [2,3].Our previous research examining institutional COI [4,5] and research integrity/misconduct policies [6], as well as focused searches of the Université de Montréal policies (whether the top-level policy University COI policy or those general documents produced by the Faculty of Graduate Studies), found general policies but very little clear guidance beyond the recommendation to avoid COI.Yet, as our research clearly showed, without an awareness that COI is an important problem (i.e., because it can bias decision making and undermine trust) and that COI situations must be identified so they can be managed appropriately (e.g., prevented, or the risks mitigated), even the best policies are "not worth the paper on which they are printed".

With the creation in
From awareness that there is a problem in need of management flows the obligation on the part of institutions to develop and implement clear, understandable and practical procedures that can effectively manage COI when they cannot be avoided, as is very often the case in academia.However, all too often the declaration of COI is seen as a panacea, which is particularly problematic [7].Empirical research by Cain and colleagues [8,9] has shown that disclosure of COI may lead to a form of "moral licensing" both on the part of individuals who disclose COI ("I've declared my COI so the problem is gone, right?") and others ("He disclosed his COI, so he's ethical, and the situation is fixed.").In practice, disclosure is still necessary, but it is not sufficient.Policies and procedures must also include evaluation mechanisms, such as review by other parties to verify and determine the level of risk posed by the COI and ensure that especially problematic COI are avoided (i.e., some jury members excluded), and that those COI that are more acceptable are managed appropriately (e.g., selection of different jury members to "balance-out" or mitigate the COI).
Awareness raising off all stakeholders involved is also necessary so that as jury members striving for objectivity or neutrality in their review of a thesis, they keep in mind that this is impossible and that biases may still remain (i.e., reflexivity).
Building on my team's previous work analyzing the language of institutional COI policies [10], and our more recent collaborations with the Office of the Vice-Principle Human Resources and Planning to develop COI educational resources and a new procedure for the annual Declaration of interests (www.interets.umontreal.ca),I developed a short and practical procedure to better manage the formation of thesis juries.To be effective, the procedure had to explain the nature of COI (e.g., personal, financial, institutional, ideological) and give clear examples (i.e., awareness raising), and provide a clear and detailed decision-making process (i.e., be practical).
A first draft of the procedure was produced in October 2014 and sent to the Vice-Dean of Education, Professor Lise Lamothe, and the director of the PhD in Public Health program, Professor François Béland, both of whom provided substantive feedback and additions.A revised version of the procedure, with the addition of a Declaration form, was then circulated to members of the Education Committee before approval in November and a subsequent Faculty-wide roll-out in December 2014.
During 2015, the operationalization of the procedure allowed for the identification of some practical difficulties (e.g., application to all diplomas or just research Masters and PhDs; issues about application of specific processes), which led to another round of revisions, culminating in the March 2016 version of the procedure.In February 2017, the procedure was launched in an online format (using the LimeSurvey platform) to facilitate completion of the questionnaire by jury members.
Both the Procedure and the associated Declaration of Interest form have been circulated to all professors, program directors, and administrative staff in the School of Public Health, and are also to be sent by supervisors (who are responsible for soliciting participation in juries) to all selected jury members during the process of forming a jury for their Masters or PhD students.
The aim of this Commentary is to share with the wider academic community a model for a practical procedure that could be used/adapted by other academic institutions to help more effectively manage the COI that are inherent in academe, specifically those that arise in the formation of Masters and PhD thesis juries.

The Procedure
Entitled Procedure for Managing Conflicts of Interest when Constituting Masters and Doctoral Thesis Juries in the ÉSPUM, the procedure is structured as follows: a Summary about the aims of identifying COI and the importance of their management; a Definition of COI, the one used by the University in its annual Declaration form; a list of pertinent Types of COI, adapted from a document produced by the University of Western Australia [11]; a table of the Relations between jury members and the Process for forming the jury.
The official procedure is in French (Annex 1), but an English version is also made available to jury members and so is presented here, in Boxes, preceded or followed by text to briefly explain the content, structure, and reasoning behind the procedure and its associated declaration form.

Summary
• The management of conflicts of interest (COI) in the composition of Masters and PhD juries is essential in order to maintain the integrity and credibility of the jury process and the evaluation.• Unaddressed COI bring with them important risks, namely bias/reduced objectivity in the evaluation process, but also a potential loss of trust by members of the academic community (internal and external to the ÉSPUM) in the institution and its processes, and in the quality of degree granted.• It is thus essential that real and apparent COI be identified, so that they can then be evaluated (e.g., severity, nature of the risk involved) and thus managed appropriately.providing educational content to faculty and potential jury members about the nature and importance of dealing with COI appropriately.

Definition (see: www.interets.umontreal.ca) A conflict of interest may arise when activities or situations place an individual or an organization in the presence of interests (personal, institutional or otherwise) that conflict with the interests inherent to the duties
and responsibilities of their status or function.These interests include business interests, financial or nonfinancial (e.g., religious beliefs, values) relating to the organization or to the individual, their family members, friends or associates, past, present or future.These conflicts risk altering the integrity of the decisions made and so can cause harm and undermine public trust in the organization and its members.
The definition is the one developed for the University in 2012, and which is also included at the top of the Annual Declaration form.

Types of pertinent conflicts of interest to consider Adapted from the "Conflict of Interest and Examiner Independence for Examination of Higher Degree by Research Thesis", Board of the Graduate Research School, University of Western Australia
The term 'Examiner' is understood to refer primarily to the external examiner of a PhD thesis and the internal jury member for a masters.The types of COI illustrated below are also relevant, but to a lesser degree, for the other members of the jury, such as the internal member (in the cases of a PhD), the President and the Dean's Representative (PhD).Note that the following list is aimed at helping to identify potentially problematic situations, so that they can then be evaluated and managed; it should not be read as a "check list" that leads to automatic exclusion of a jury member if one of the listed COI occurs.Cases of real or apparent COI need to be declared so that they can be evaluated and managed appropriately, when they cannot be avoided.
In the composition of a jury, an important concept is the notion of "critical distance" between the various jury members and the student/supervisor, and relative to the respective roles of the members (e.g., president, internal or external jury members).For example, while it would be expected that the external examinee in a PhD defense would, with few exceptions, have little or no recent collaborations with the student or their supervisor, such a requirement could be difficult to achieve (e.g., lack of expertise) and unreasonable for a jury member (often a colleague) for the evaluation of a master's thesis.That being the case, other members of the jury, e.g., the president (also often a colleague), could be chosen who have less close relations with the student/supervisor, to help ensure the necessary critical distance.
Even if as professors we may think that we can be completely objective in the evaluation of a student's work, regardless of our relationship with the student, there is good empirical evidence that this is often not the case; it is difficult to recognize one's own interests and biases.Furthermore, the perception by third parties that there is a COI can significantly undermine the credibility of and trust in the evaluation process.It is thus essential to ensure that the different members of the jury are transparent about their interests and relations and that a critical distance be maintained to ensure an expected degree of impartiality.
Based on practical considerations and following a reflection about the nature of the associated risks, a distinction was made between the requirements for the external examiner and those of other jury members.Specifically, if all jury members, with the exception of the supervisor of course, are required to have no COI whatsoever, regardless of the risk, then it becomes almost impossible to constitute a jury.The notion of "critical distance" was thus introduced to highlight the importance of ensuring the most objective evaluation possible of the thesis, within reason; depending on the different roles of jury members, different levels or degrees of critical distance are necessary.
Critical distance is a means of pointing to widely shared ideals of objectivity and impartiality in academe, while accepting that professors are still human beings and so are nonetheless inextricably subjective, even if when they have PhDs!Evaluations and judgments will never be perfectly impartial, but one can still aim towards impartiality as an ideal (while also recognizing that it is ultimately unattainable).Humility and reflexivity are required, as well as the recognition that certain relations will ISSN 1923-2799 5 / 21 (http://bioethiqueonline.ca/6/7) make impartial judgments all the more difficult.In cases where there are relatively close relationships between students and jury members, such as between a student and their supervisor, it is unreasonable to expect a high degree of impartiality, so the expected critical distance will be less.A supervisor may (will) often have a direct interest in seeing their student succeed, i.e., a COI, and so a source of bias.They may legitimately defend their student's thesis while trying to give a fair judgment of the quality of the work, but it is unreasonable to expect them to be as impartial or have the same critical distance as other jury members.By contrast, when the jury member is essentially a stranger (e.g., a distant colleague with no history of collaboration) to the student and their supervisor, then one can reasonably expect a high degree of impartiality and thus maximal critical distance.
In recognizing that in a thesis jury there are different roles and responsibilities -e.g., president, supervisor, internal and external examiners -I argue that it is reasonable to accept different degrees of critical distance.To offset to some extent the biases that may result from COI generated by various sorts of relationships, I propose various thresholds of "critical distance" for the different roles in the jury (see the table in the section on PhD Jury Composition, below).In this view, expectations for impartiality for jury members will generally flow from least to most critical distance -supervisor (least), president, internal jury member, external evaluator (most) -but can be modulated depending on particular types of COI (a risk analysis), issues of availability of expertise, etc.So it may be legitimate to accept an external evaluator who has also been a collaborator (but not too close) with the student student/supervisor, when 1) there are no other qualified evaluators available, and 2) other members, such as the president and internal jury member, are maximally critically distant.The goal is to create a jury that is as impartial as possible in order to ensure a fair review of the thesis, and if not all COI can be eliminated, then those that are tolerable can be counter-balanced.
To help the users of this procedure ask themselves the right questions in order to identify different types of COI, a list of potentially problematic COI was provided, grouped into Work, Personal and Business relations, as they pertain to any relations between the examiner (i.e., the jury members) and the student or the supervisor.• Examiner has co-authored a publication with the supervisor in the past 3 years.Note: mitigating circumstances may exist in that co-authorship may be less than significant (i.e., one of many authors on a large group project).Such circumstances may not be considered as a Conflict of interest, but must be declared.
• Examiner was a student of the supervisor within the past five years Note: Relationship should be declared in all circumstances.• Examiner is in a social relationship with the supervisor, such as co-Trustees of a Will, god-parent, and miscellaneous personal contacts which give rise to the perception that the examiner may be dealing with the supervisor in a less than objective manner The issue of co-authorship and other research collaborations amongst jury members was one of the instigating factors for developing a more nuanced procedure.The choice of 3 years for co-authorship and 5 years for supervision were somewhat arbitrary, but aimed at creating critical distance without undue constraint.Examiners found to be in one of these relations would not be automatically excluded; instead, the goal was to ensure declaration and documentation so as to enable an evaluation of their nature.For example, was the co-authorship a rare collaboration or part of a large team publication, where the examiner and supervisor had little in the way of real ongoing collaborations?Further, it was recognized that internal jury members (e.g., colleagues in the same Department or research centre) would more likely have collaborations, but these too should be disclosed so their nature could be evaluated, and not grounds for immediate exclusion.

PhD Jury Composition
Summary of the relations between jury members and the critical distance necessary to appropriately manage COI (with regards to the types of COI mentioned above).

Member/Role Relationship with student
Relationship with supervisor(s)

President
• May have been a professor • Often a colleague • Must be sufficiently distant from student and supervisor to ensure a neutral process Supervisor(s) • Close relation with student • NA • Try to be as objective as possible in the evaluation process Jury member (internal) • May have been professor • Often a colleague • Must be sufficiently distant from student and supervisor to ensure a neutral process External evaluator • Should not be in a close relationship • Should not be in a close relationship • Must be external to the university • Must be an expert in the subject matter • Must be free from real or apparent COI Dean's representative • May have been a professor • Should be from another department • Guarantor of the quality and rigour of the exam process • Does not have to be a content expert This table provides a summary of the roles and relationships between the different members of the jury, and the required (and realistic) critical distance that should be sought.It recognizes, for example, that the supervisor and the student will invariably have a close working relationship and so while the supervisor can evaluate their student, they are not impartial.By contrast the other jury members are held to a greater degree of critical distance.
ISSN 1923-2799 8 / 21 PhD jury nomination process Supervisor 1.The student's supervisor is responsible for creating a list of potential jury members, i.e., two people for each of the 3 categories of jury member: president, internal and external jury members: a. Ensure the greatest possible critical distance on the part of all potential jury members; b.Identify any real or apparent COI between the potential jury members, the supervisor and the student.To do so, completes the Declaration of conflicts of interest form; c.If the COI appear inevitable, whether or not they are acceptable, the supervisor must document these on the form for selecting jury members; d.Ensures the availability of the first choice of candidates (i.e., president, internal and external jury members); 2. Submits the candidate list and the Declaration of conflicts of interest form to the director of the option or of the program, in the case of programs without options.
Option or program director 3. The option or program director decides on the competence of candidates and the absence or presence of COI and, where applicable, the risks and their acceptability; • In the event that one or more candidates is not appropriate (e.g., because of expertise, unacceptable COI), the option or program director returns to the supervisor who recruits new candidates.
4. Where candidates are found to be appropriate, the director of the option informs the program director who chooses one candidate per category of jury member; in programs without a director for the option, the program director is responsible for evaluating the supervisor's proposals for forming the jury.
5. The graduate secretary asks the selected jury members (president, internal and external jury members) to complete the Declaration of conflicts of interest form • In the event that one or more candidates is not appropriate (e.g., because of expertise, unacceptable COI), the option or program director choses one of the other potential jury members.He/she may also return to the supervisor to recruit new candidates.
6.The jury nomination form is completed by the program director and submitted for signature to the Faculty superiors (Vice-Dean of Education).
7. The graduate secretary informs the selected jury members.
8. The jury processes can begin.
The decision to make the supervisor responsible for providing a list of potential jury members was again based both on pragmatic considerations and an evaluation of the risks or costs associated with other options, such as requiring the Option or Program director to make the selection.Most often, it is the supervisor who best knows the experts in the field who would be best placed to participate in jury.But to avoid obviously unethical situations where supervisors "stack the jury" with their friends to ensure an easy evaluation, the supervisor must provide a primary and secondary list of choices for each of position in the jury (President, Internal Jury Member, External Evaluator, Dean's Representative), and also disclose any potential COI with these members.This disclosure is then verified by the Option or Program director, and then also cross-referenced with the Declarations of the selected jury members.

Member/Role Relationship with student
Relationship with supervisor(s)

President
• May have been a professor • Often a colleague • Must be sufficiently distant from student and supervisor to ensure a neutral process Supervisor(s) • Close relation with student NA • Try to be as objective as possible in the evaluation process Jury member (internal) • May have been a professor • Often a colleague • Must be sufficiently distant from student and supervisor to ensure a neutral process

Masters jury nomination process
• For a Masters with thesis, see the criteria and procedure for the PhD and adapt accordingly.
• For a Masters with internship or directed study, even if the procedure is not directly applicable, the general principles aiming to minimize COI remain pertinent.
The PhD procedure was adapted for the research Masters, which do not include an external evaluator or Dean's representative.The professional Masters programs do not involve a formal jury, but instead rely on one or more evaluators (usually a colleague) to grade the student's final project (a report varying between 25-50 pages).Nonetheless, the general principles outlined in the procedure still apply.

Declaration of Conflicts of Interest Form
The Declaration of conflicts of interest forms (Annex 2), one for the supervisor and another for the jury members, have both a practical and psychological purpose: they both provide means to document real or potential COI so that they can be evaluated in the formation of a jury, and enable supervisors and potential jury members to reflect on their own COI and how these can be best managed.
The forms begin with the statement that "The purpose of this form is to provide information on the interests of potential jury members that could affect the evaluation process in the context of masters or doctoral juries" and then presents the Definition of COI described above, in the Procedure.
The supervisor and the potential jury members are asked to provide personal information (name and affiliation), and identify their role on the jury (president, supervisor, internal or external jury member, Dean's representative).They are then asked to fill in a table declaring all potential or real COI (based on the more detailed content outlined in the Procedure), and add any other COI that were not mentioned in the table.

Conflict of interest relative to your role as jury member
Declare all personal, professional or financial relations with the student or the supervisor that could be perceived to influence your roles as an examiner.(please check and provide any explanations)

Working relations
• co-authored a paper in the last 3 years • worked with the student on issues of analysis or synthesis • was a student of the supervisor in the last five years • provided funding for the student or holds a grant with the supervisor • employed or has been employed by ... • is negotiating to directly employ or be employed by ...

Personal relations
• is a family member • is a friend / partner / student mentor • has a hostile personal relationship • has an existing or previous emotional connection, is co-resident or a member of the same household Business relations • are in a business relationship (past or present), as a partner in a small business or employment • has a current working relationship, such as membership of a board or committee • is in a social relationship, such as co-administration of a will, sponsor, or through various personal contacts

Publication
• has published work critical or laudatory of the student's approach (naming the student / supervisor) Public forum • has spoken publicly in a fashion that is critical or laudatory of the student's work (naming the student / supervisor)

Research
• has a direct commercial interest in the research results

Conclusion
Participating in the development of a practical procedure for managing COI in the composition of Masters and PhD juries at the School of Public Health proved to be a very rewarding experience.In the space of a few months (and then validated over a period of 2 years), I was able to put into practical application the results of almost a decade's worth of ethics research on university COI, that was primarily focused on documenting and raising awareness about the problem, i.e., that COI are widespread in academia, can be very problematic if left unaddressed, but that they can be managed with the right tools.In building such a COI management procedure, I was able help my colleagues ISSN 1923-2799 12 / 21 (http://bioethiqueonline.ca/6/7) address a real-world ethical problem in their role as supervisors, potential jury members and program directors.As with any new policy or procedure, it will take time and sustained effort to create awareness about both the procedure and the importance of implementing it in practice.Nor are procedures a panacea; there will be unintentional mistakes, important COI may not be disclosed, and some people may even wilfully ignore the procedure, thinking it irrelevant ("What do you mean I'm not objective?I'm a university professor!").Surveillance by secretaries (reminding supervisors and jury members to complete the forms), program directors, and the Vice-Dean will be important to ensuring compliance with the COI procedure and thus its effective integration into the practice of forming thesis juries.
What is important, in the end, is recognizing that good (i.e., practical and pertinent), procedures can be developed and implemented to support and encourage ethical conduct, and this includes the management of an issue as sensitive and as widely misunderstood as conflicts of interest [12].

Membres du jury
Le but de ce formulaire est de fournir de l'information sur les intérêts des membres potentiels de jury qui pourraient influer le processus d'évaluation dans le contexte d'un jury de maitrise ou de doctorat.

Recherche
• a un intérêt commercial direct dans les résultats de la recherche
☐ Aucun autres lien, condition ou circonstance présentant un risque de conflit d'intérêts ☐ Oui, les relations, conditions ou circonstances suivantes sont présentes (expliquer ci-dessous): 2013 of a new faculty-level School of Public Health (École de santé publique) at the Université de Montréal -made up of the Departments of Social and Preventive Medicine, Health Management, Evaluation and Health Policy, and Environmental and Occupational Health, that were previously in the Faculty of Medicine -an entire administrative infrastructure (e.g., appointment of a Dean and Vice-Deans, creation of various policy committees) had to be developed.One of these structures, which I joined in 2013, is the Education Committee (Comité des études), chaired by a Vice-Dean and which brings together all the program directors in the School.The committee is responsible for oversight and approval of developments and changes to programs, but also for establishing policies and procedures.An issue that arose in 2014, following a series of major reforms to the Masters and PhD programs, was the need for standardized Faculty-wide procedures to guide the composition of thesis juries, and in particular, to deal with issues of COI.As my research team and I have been working on the ethical management of COI since early 2005, I volunteered to help create a practical procedure that could be used across all the programs in the School of Public Health.
Examiner has published work critical or laudatory of the student's approach (naming the student / supervisor) Public Forum • Examiner has spoken publicly in a critical or laudatory way about the student's work (naming the student / supervisor) Research • Examiner has a direct commercial interest in the outcomes of the research This section sought to bring attention to COI related to the subject matter of the student's work, particularly with regards to possible ideological COI that might lead to bias either in favour of or against the student and the evaluation of the content of the thesis.ISSN 1923-2799 7 / 21 (http://bioethiqueonline.ca/6/7)

•
Examiner holds a current grant with the supervisor.Note: Mitigating circumstances may exist (such as in first bullet), which should be declared.• Examiner has co-supervised with the supervisor in the past five years • Examiner holds a patent with the supervisor • Examiner had directly employed or was employed by the supervisor in the past five years Personal Relationship • Examiner is in negotiation to directly employ or be employed by the supervisor • Examiner is a relative of the supervisor • Examiner has a personal relationship of enmity with the supervisor • Examiner and the supervisor have an existing or a previous emotional relationship, are co-residents or are members of a common household Business Relationship • Examiner is in a business relationship (past or present), such as partner in a small business or employment, with the supervisor • Examiner has a current professional relationship, such as shared membership of a Board or Committee (including editorial and grant decision boards), with the supervisor.

3) Conflict of interest relative to your role as jury member
Declare all personal, professional or financial relations with the jury members (President, Internal Jury Member, External Evaluator, Dean's Representative) that could be perceived to have an influence on the jury members.(please check and provide any explanations) ISSN 1923-2799 11 / 21 (http://bioethiqueonline.ca/6/7)
Processus de nomination de jury de maitrise•