Infra Finder: A New Tool to Enhance Transparency, Discoverability and Trust in Open Infrastructure

This paper describes Infra Finder, a new tool built by Invest in Open Infrastructure to help institutional budget holders and libraries make more informed decisions around adoption of and investment in open infrastructure. Through increased transparency and discoverability, we aim for this tool to foster trust in the decision-making process and to help build connections between services, users


Introduction: Identifying the Problem
One of the ongoing conversations in the open scholarship space surrounds the intersection of trust and prestige and countering existing bias against open publications and open infrastructure.Existing and deeply-rooted bias against open access and open scholarship can result in potential authors perceiving that choosing an open access publication is more risky than the status quo of subscription publication (Cantrell & Collister, 2019;Collister & Cantrell, 2021).There are several routes to countering this bias to lift up open practices as more trustworthy and move the needle towards open as a default.One avenue was recently investigated in the "Equity in OA" report from the Open Access Scholarly Publishing Association (OASPA); this report contained key findings about increasing trust in open access publishing that could counter misconceptions about the quality of open access materials, concluding that "part of the solution to perceptions of low quality OA publishing is positively defining what good or trustworthy OA publishing is" (Legge, 2023).OASPA recommends principles of transparency and best practices that foster trust in open access and open scholarship, such as openly describing processes in all steps of publishing, being clear about ownership and governance, and openly disclosing financial information.These findings can be applied more broadly, for example to discussions about risk and perception of trust in open tools and new technology in general.Even when one has signed on to a call like David W. Lewis's call to academic libraries to dedicate 2.5% of its total budget to support open infrastructure (Lewis, 2017), there may be additional barriers to actually implementing the open infrastructure that aligns with the stated goals.Open source software is associated with a perception of higher risk that can often privilege commercial vendors in riskaverse environments, even when there are advocates in that environment who might be interested in supporting the ideas of the open source movement (Silic & Back, 2016).In particular, institutional procurement processes can be especially problematic for open source software adoption, because these processes are designed specifically to manage the exchange of money for goods and services, whereas no such exchange takes place unless a vendor-supported OSS option is under consideration (ICOLC Strategies for Open Collaboration in Library Consortia Task Force, 2022;OSS Watch, 2008;Teal et al., 2020;Teperek & Dunning, 2020;Thompson, 2009).
Inspired by these conversations, Invest in Open Infrastructure (IOI) built a new tool called Infra Finder1 , which is designed to assist advocates when making the case for adoption of open infrastructure in their local contexts.This tool is intended to help increase the transparency and ease of finding information about open infrastructure services, and to provide a foundation of information to get started with further conversations and explorations.To foster trust, the tool surfaces information that goes beyond typical market-driven indicators.These include, but are not limited to, governance structures, policies, pricing, user contributions and community, technological affordances (e.g., dependencies, maintenance, implementation support), and interoperability -many of which overlap with the factors mentioned by OASPA in their report.This information is provided by representatives of the service and checked and verified by the team at IOI to ensure the information is up-to-date and usable in context with the other information surfaced in the tool.In addition, because we know that deciding which infrastructure to trust with an organization's business and labour is a long and multi-step process, Infra Finder contains contact information and links to relevant resources to discover additional methods of information gathering, such as contacting current users of the tool or exploring extant use cases.
It is our goal to provide a useful resource for advocates to use when engaging in contexts such as institutional-level advocacy for open infrastructures.One example of this context was surfaced in a panel presentation with leaders in higher education (Carter et al., 2023), where the representatives discussed the problem of university-level procurement processes such as Requests for Proposals (RFPs) that privilege established, commercial vendors of services over open infrastructure providers.Beyond this privilege embedded into these processes, open infrastructure services must compete for adoption in a market with commercial providers with large staff and marketing departments.Furthermore, the processes for decision making at many institutions are created with commercial providers in mind; during interviews as part of a research project at Invest in Open Infrastructure, we heard many stakeholders share similar sentiments regarding the laborious and burdensome nature of due diligence, as well as the absence of a standard approach to the process even within one institution.These interviewees mentioned that there are two paths in these processes -"build" decisions, which engage IT governance and review processes, and "buy" decisions that engage IT governance and review as well as procurement processes (see also The Higher Education Leadership Initiative for Open Scholarship (HELIOS Open), 2023).When an institution has to engage with procurement processes, interviewees reported that the language and processes are built for vendors and require significant effort to document legal compliance, technical integrations, and specifications.This often led to decisions to adopt commercial offerings due to an urgent need to meet the needs of the research community, as well as the ease found with commercial companies' abilities to navigate often complex institutional procurement and approval systems (Invest in Open Infrastructure, 2020, Forthcoming).Even if there is an individual or a group at the organization who is willing to do the work to advocate for open infrastructure, the logistical and informational barriers may be difficult to overcome when combined with pre-existing biases, and the less laborious option becomes the easiest, or sometimes the only, choice.
An intersecting problem in this space relates to investment and funding for new open infrastructure tools.For example, when tools are needed to support a new discipline, method, or mode of scholarly inquiry, how does an interested party discover potential partners in the space in order to work together to acquire this funding?How could a funder discover and solicit proposals from promising resources that already exist, or identify gaps in the infrastructure landscape that would benefit from targeted investment?Prior research has shown that word of mouth and existing networks are powerful tools that have a large effect on discovery and decision making (e.g., Kim & Song, 2010), but this practice does tend to reinforce the usage of the same set of tools within networks and privilege those organizations already in a strong position.The situation when one's peers are all using a tool successfully creates a set of subjective norms, which can build the intention to trust a tool that is already in use in a community (see, for example, Ho et al., 2017 on the adoption of cloud computing tools).When these subjective norms come into contact with the built-in presuppositions of a procurement process, they can reinforce the dominance of particular players in a particular sector.
Ultimately, Invest in Open Infrastructure is designing Infra Finder to make information visible and to establish a shared set of parameters that can be identified across different providers, all with the aim of contributing to an environment of trust in open infrastructure services.We recognize that this is the start of the conversation, and that the landscape will inevitably change.In the rest of this paper, we describe the process that has brought us to the point of launching Infra Finder in the current landscape.

Landscape and Prior Work
The landscape of open infrastructure discovery contains many different types of resources and services.We found that these fell into three broad categories: discovery services, convening organizations, and guidance resources.
One example of an extant discovery services for open infrastructure is as the Scholarly Communication Technology Catalogue, or SComCat (Ballard, 2021).SComCat was developed in 2020 as part of the Next Generation Libraries Publishing project, and was a collaboration between a number of organizations.This catalogue aimed to provide an overview of functionality, organizational models, dependencies, standards, and adoption cases for each organization included.SComCat features a technology browser that includes 82 profiles, and the ability to search as well as filter by parameters like function, business form, and adoption level.As it is part of a project related to library publishing, most of the technologies included in this catalogue cater to that group.This catalogue builds on other discovery services that IOI evaluated as we prepared Infra Finder, such as the July 2019 Mind the Gap report (Maxwell et al., 2019), Bianca Kramer and Jeroen Bosman's list of 400+ tools for scholarly communication2 , and the outputs of the "Mapping the Scholarly Communication Infrastructure" project (Lewis, 2020;Skinner, 2019).
Another resource in the landscape of open infrastructure discovery are convening organizations.These include those routing funding to open infrastructure (such as the Global Sustainability Coalition for Open Science Services, or SCOSS) and membership organizations (such as Lyrasis), although there are many types of organizations in this space doing a wide variety of work to collaborate, connect, and support open infrastructure.In our preparation for creating Infra Finder, IOI consulted the web pages and networks for these organizations to discover services that could be invited to participate in Infra Finder.Additionally, during our intake process, we discovered that service providers were keen to name organizations like SCOSS in their Infra Finder entries as additional resources to facilitate discovery about their programs or to contribute funding.This is one way that we hope Infra Finder can bolster the work of these organizations and contribute to the ecosystem: by driving additional attention to existing initiatives and develop an understanding of how these organizations contribute to the knowledge base for an open infrastructure.
Finally, rubric and guidance documents exist in this open infrastructure landscape to help institutions make decisions about infrastructure adoption.Examples include the University of California's Scholarly Transformation Advice and Review Team (STAR) Criteria3 , and the HELIOS Shared Infrastructure Decision-Making Guide (2023).These complement resources that are intended to help evaluate infrastructures against a set of principles, such as the Principles of Open Scholarly Infrastructure (Bilder et al., 2020), the Community Health Analytics in Open Source Software (CHAOSS) metrics (Linux Foundation, n.d.), and the FOREST Framework (Lippincott & Skinner, 2022).While these resources do not always mention specific open infrastructures, they provide key insights into the kinds of information that potential adopters may be seeking, and thus provided valuable framing for the questions we posed for Infra Finder's intake form.
These resources are all important components of the open infrastructure discovery landscape that have helped bolster the health of the open infrastructure ecosystem.In preparing Infra Finder, IOI intends to complement and elevate these resources, whether building on the existing foundation to expand into new contexts or providing additional material that can assist with ongoing practices and development.

Prototype
IOI launched a prototype of a discovery tool in 2022 called the Catalog of Open Infrastructure Services (COIs).The development of this prototype was founded on research into the challenges facing open infrastructure services and the needs of their users, including a lack of information sharing and coordination (Enkhbayar & Jack, 2021).This prototype was designed as a proof-ofconcept and therefore contained only 10 service providers.Our intention was to evaluate the potential utility of such a tool and data displayed, test the intake and data collection processes, and experiment with ways of displaying information.After the launch of the prototype, Taimour Azizuddin conducted 12 semi-structured user interviews with stakeholders from philanthropic institutions and academic institutions, focused around the utility of the prototype, we discovered the need to surface some of the narrative and untold story behind the numbers, names, and indicators displayed (Azizuddin, 2023).These users wanted more details about the services they were exploring beyond the numbers that were displayed in the prototype and in other tools.In addition to basic information on history and technical affordances, users also wanted information about open values such as community governance.
As IOI began to develop the next version of the prototype, we also interviewed representatives from 26 of the infrastructure services that would be listed in the next version of the tool.Our goal was to more deeply understand their service and the highlights of their service's use and begin to surface some of the narrative that the potential users of the tool had outlined in our research, and to refine our intake process to best highlight information about each service.These service provider interviews were critical to understanding the needs of the services and the people who worked on them, and to engage with those who were providing us with information about their infrastructures.Service providers expressed a desire that aligned with the users from Azizuddin's interviews who wanted more context and details around the numbers -the service providers also wanted the opportunity to provide a more thorough and nuanced view of the open infrastructure services.They wanted to provide that additional context and understood that potential adopters could use the additional narrative to advocate for increased investment in and adoption of open infrastructure.These interviews also helped us better understand the many different ways that infrastructure services can operate.
From this pilot project, we knew that the intake form for the next version of the tool needed to include questions that allowed providers to share not only a description of their service and the mission, but also to give them space to describe their key achievements and funding needs.We also needed to develop questions that would elicit information about policies, governance, and technology that interested the potential users that we interviewed.Finally, the information needed to be flexible enough to allow many different kinds of organizations to participate and be able to accurately share their information, while not sacrificing the ability for users to compare across different options as they conducted their searches.

Designing and testing Infra Finder
At IOI, we are committed to developing Infra Finder iteratively and transparently, as we believe that this is critical not only to attract early adopters and advocates, but also in building users' trust in the tool and the data we're providing, and in ensuring that Infra Finder can serve the needs of our target users.In our development and design process, we took extra care to involve our target users (institutions) and open infrastructure services early on through introductory calls, focus groups, and product testing.
In order to deepen our understanding of the experiences of our potential users, we conducted focus groups in September 2023 with 13 library directors ("decision makers") and members of staff ("case makers") at libraries affiliated with different types of institutions with different levels of research support.We learned that there was an opportunity for the tool to unburden early-stage evaluation of infrastructure services through showcasing verified information around aspects such as costs (of implementation and maintenance), technical dependencies, and open value alignment.In these focus groups, we also learned that finding the "right" infrastructure is a difficult and time-consuming process.Participants described having to go to various websites, forums, and e-mail threads to find the information that they needed, and that different stakeholders from their organizations had different requirements for information that made them go back again searching all of these locations.
To gather this information, we designed a new intake form that included questions that specifically targeted the needs of the users, as well as information contained in the decisionmaking tools surveyed above.We invited a group of initial services from our network of contacts to participate in order to pilot our intake form and data collection process.To be included in Infra Finder, an infrastructure had to align with the following definition: "A service, protocol, standard or software that the academic ecosystem needs in order to perform its functions throughout the research lifecycle."In addition, we prioritized services that met one or more of the following criteria:  5) is operated by a non-profit or non-commercial entity; (6) are designed to be widely used and distributed.
These services in the first group were mostly in the repositories and scholarly data sharing space, since those were primarily the groups that responded with interest to our pilot COIs project; furthermore, they were the networks that IOI was heavily involved with and provided infrastructure in the categories that most closely aligned with the research that we had done.We invited 84 infrastructure services to participate, and 57 responded to our call and submitted their information in our form.Of these, 26 services also participated in an interview to ask questions, share feedback on the questions, and provide additional context for their service's responses.After receiving their responses, IOI's research team reviewed the answers and documented websites and manuscripts that served as additional information to support the responses provided.This review process was intended not only to verify that the information provided was current and accurate, but also to provide as many additional sources of information as possible.Where discrepancies existed, we worked with the service providers to update their information.
We also designed mechanisms to gather usage feedback from early users to ensure that we can continue to improve the tool and its user experience.Before launch, and based on the findings from the focus groups, our designer developed low-fidelity wireframes which we tested with 5 librarians.This test was to understand how they searched for information, and what was the most relevant information to present on the page.After reviewing the feedback and updating the designs, IOI shared the designs and data for Infra Finder with the 56 infrastructure service providers included in the initial release for their feedback and review in December 2023.The providers were able to update their own data in their entry and to offer key feedback on the presentation of information in our designs.These initial review periods were critical in helping further refine the user experience and design.
Infra Finder was intended for launch in January 2024.As part of our process of transparent and collaborative creation, we encountered many new discoveries as we worked through our data, alongside excellent feedback from our user communities on our initial prototypes.Consequently, we adjusted our approach to incorporate our discoveries and take many suggestions into account.As such, instead of launching in part in January 2024, we consulted with our team and our communities and decided to delay until April 2024.We were thankful to receive a positive response on this decision from our infrastructure service providers and those in our community who were watching for Infra Finder's release."Measure twice, cut once," was one such response from Chris Holdgraf at 2i2c (one of the organizations participating in Infra Finder); this advice typically given by carpenters was perfectly applicable to our situation.Rather than potentially risk wasting time, materials, and energy by rushing to finish the project, we decided to take the time to go back over our work to create the best outcome.Through this process, we ourselves experienced the sense of trust that can be fostered in transparent communication about a product.

Use Cases for Infra Finder
The several stages of input and feedback provided us with several opportunities to provide value for different use cases.In this first release, Infra Finder will build on the prototype and the related tools and services in the field to provide additional resources for two particular groups of users: the potential adopters of open infrastructure, and the providers of open infrastructure services.These are not the only groups that will be able to use or benefit Infra Finder, as the resource itself will be open and free to use; however, we designed and developed Infra Finder with these two particular use cases in mind to start the conversation and identify additional opportunities.

The utility of Infra Finder for institutions and networks
The most immediate use case for Infra Finder is for institutional adoption and procurement processes.We see it used and relied upon for gathering information in decision-making processes like Request for Proposals (RFPs).If a potential adopter can come to a curated catalog of relevant information that they can use to fill in their forms, then the tool has saved that user time; however, the tool also provides another benefit to interested adopters of open infrastructure services.We are intentionally building in methods to discover related services and conduct due diligence on the possible options.We envision that a potential adopter --for example, an Associate University Librarian for Digital Strategies --can search and filter by certain parameters relevant to their situation, see the other open infrastructure service providers in the space, and potentially discover additional services that can broaden their pool of potential choices.
This tool can alleviate the lack of transparency associated with the current discovery stage of procurement, which appears to be the least transparent part of the procurement process.Respondents typically relied on relationships and word of mouth to find open infrastructure services to work with and fund, reinforcing that personal recommendations are a strong trustbuilding tool (Kunkel et al., 2019).We have also heard in our conversations with institutions that having nearby, trusted peers adopting an open infrastructure encourages the institution to adopt that same infrastructure as they anticipate the ability to lean on their peers for knowledge exchange and community support (Azizuddin, 2023).Infra Finder can help improve the utility of these recommendations by including information about current users and adopters, and where that information is available, we asked providers to share it in their entries and we are exploring additional ways to share these details.Infra Finder also provides an opportunity for discovering networks of relationships through including information about governance, community engagement, and funders that support the service.These data are real examples of the relationships and networks that can build on the value of word of mouth recommendations.
Additionally, Infra Finder can also support awareness raising and education of leadership and staff on open values.With increasing momentum for institutions to support their researchers in adopting open data and data management best practices, alongside increasing pressure for libraries to shift budgets away from subscriptions and transformative deals with commercial publishers to support open scholarship, Infra Finder highlights various aspects of "openness" that infrastructure can have.Some of the topics that services can highlight in their entries are transparent governance activities, community engagement and participation, and commitments to accessibility, equity, and inclusion.This enables institutional decision makers to not only identify the tools that best fit their institutions' open priorities, but also provide additional dimensions for openness and transparency that institutions can consider adding to their decision-making and internal advocacy process.

The utility of Infra Finder for infrastructure services
The success of Infra Finder depends on the participation of open infrastructure service providers.As such, IOI designed an extensive intake form driven by our research into what facets are important to decision makers in a variety of contexts.We see that the utility of the tool for institutions and networks and for infrastructure services go hand-in-hand: the more that Infra Finder is used and relied upon, the more visibility and potential for adoption and additional funding exist for the infrastructure services.Nevertheless, we understand that collecting, providing, and updating the information in the intake form and the final entries in Infra Finder takes time and capacity that the creators and maintainers of open infrastructure services may not have.
We therefore are prioritizing engaging with these service providers individually.To do this, IOI hired an Engagement Coordinator dedicated to their participation in the development of the tool.The Engagement Coordinator acts as the bridge between the infrastructure service providers and the IOI tool development and research teams, ensuring that the infrastructures'

IJDC | Conference Paper
needs and feedback are centred and incorporated in the product development process.IOI's research team reviews the data and information provided to ensure thoroughness, completeness, and accuracy, and to identify areas for refinement and further investigation.We also provide space for the service providers to add their own highlights and narratives; for example, in the intake form, we ask infrastructure service providers to name their key achievements.This creates a free space for the service to highlight unique value propositions and achievements beyond the research-informed facets that we have included.One interesting outcome of this is that the first group of participating service providers have also pointed to other signals of trust that have proliferated in the community; some examples include the Principles of Open Scholarly Infrastructure (POSI) (Bilder et al., 2020), the Contributor Covenant for Open Source Communities (Ehmke, 2014), the Coalition for Diversity and Inclusion in Scholarly Communications (C4DISC, 2023), and inclusion in the SCOSS family (SCOSS -The Global Sustainability Coalition for Open Science Services, 2024).We intend to conduct additional research into these mentions of other trust-fostering organizations and principles to identify common threads and useful ways to display and showcase these signals.As these mentions are often in free-text fields at this time, they currently serve a referring mechanism for other infrastructure service providers to discover helpful resources for developing their own policies and practices and connecting with other like-minded organizations.
In addition to the intake form, we have built in a regular system of updates so that a service can update their entry to showcase new developments and highlights.At launch, this can be completed by a service provider at any time through a form that directs revisions directly to the Engagement Coordinator; as we iterate in Infra Finder, we intend to build more self-service update options to respond to the needs of the service providers.Additionally, we will conduct annual revision sprints with reminders and working sessions to ensure that the information in Infra Finder is current.
Additionally, open infrastructure providers exist in a number of different contexts, making the open infrastructure landscape complex and difficult to compare across different services.The funding and stability situation for a startup software project with one or two developers working on it as a side project will look very different from the situation of a service embedded in a university department, and that itself will look very different from a service associated with a research organization or a nonprofit providing tools to a particular community.The goal of Infra Finder is not to judge which situation is the best in terms of sustainability or stability, but to surface the multiplicity of contexts that can sustain and grow these services.The goal is also to connect services that may be in similar contexts or considering various challenges in order to foster a stronger, healthier network of collaborators.Through this process, we also intend to identify gaps and needs in the open infrastructure community to catalyse investment and collaboration.
All of these goals rely on the participation of service providers to share their information, update their entries on a regular basis, and engage with IOI and the other services showcased in Infra Finder.IOI has recognized the need for human infrastructure to support this community to work towards a common goal, and has invested in that aspect by first having our research team pre-populate the intake form with publicly available information to reduce the burden on service providers to provide information in another new context.Second, IOI has hired a person dedicated to engagement with this community to field questions, gather their input, and keep them updated on news and opportunities for updates and participation.We envision a future where the investment in labour and collaboration can lead to more robust sustainability, in financial resources and human effort, for the open infrastructure ecosystem.

Next Steps
The next steps for the new IOI tool after the launch include refining our intake form and data model, onboarding additional services into Infra Finder, conducting ongoing user testing and interviews, designing and testing the user experience and feedback collection mechanisms, developing a robust front-end architecture, and promoting the tool to potential users.

IJDC | Conference Paper
Regarding the intake form and data model, as we spoke with the infrastructure service providers and reviewed their responses to our questions, we found that some of our questions may have been more effective if we had offered more structured response options.For example, we asked service providers to name the key technologies that their service relied on.Some went to great lengths to share every different piece of code and software package that they used to build and power their service, while others focused on what technologies a user or implementer might need to know to run the software or host a version locally.The wide breadth of responses showed us that we need to our question, which will be an immediate improvement in clarity for this particular item.After launch, we plan to conduct additional user interviews and sessions and collect additional feedback about the information contained in Infra Finder and potential future adjustments that can make the entries more useful.We also anticipate that future developments in the policy or funding landscape may introduce additional categories of useful information to share.
We also recognize that our current approaches to verifying and updating information in Infra Finder and adding new infrastructure services are labour-intensive and not scalable.IOI plans to include a self-submission interface for new infrastructure providers to submit their information to Infra Finder at launch.We will also explore alternative mechanisms to verify the information submitted, including community curation.We recognize the importance of exploring ways to ensure that Infra Finder can continue to provide utility and value to institutions, infrastructure providers, and other potential users in the long term.
The first release of the tool will include 56 service providers mostly in the categories of data repositories and content sharing and the standards and tools that enable that sharing.IOI plans to expand Infra Finder to include digital collections, cultural heritage infrastructures, data and computing services, and other open infrastructure services based on referrals from our community and additional recommendations.Additionally, IOI's goal is to expand the geographic scope of Infra Finder; currently, the providers are mostly those from North America and Europe with a few infrastructure service providers from Africa and Latin America.We intend to build on our engagement with research and education networks in Africa and Latin America to discover the tools and services that are essential to their work and incorporate them into Infra Finder, and to take this to additional global areas in the future.We welcome recommendations and collaborations to further this goal.
In addition to continuing to develop and improve Infra Finder

Conclusion
Infra Finder from Invest in Open Infrastructure is a resource intended for the advocates, decision makers, and creators in the open infrastructure space, to help them identify services, increase adoption, and foster development and investment.IOI's approach to this tool has been to focus on trust-building mechanisms identified by research in the open scholarship and publishing communities.We intend for this resource to add to the toolbox of available sources of information that can help foster trust in open infrastructure in various contexts, most particularly in adoption and advocacy situations.We are developing Infra Finder to bolster the power of recommendations through surfacing adoption networks and investors in these services in order to enable conversations in the community about open infrastructure.IOI will launch (1) meets the definition of open source software; (2) primarily or exclusively distributes openly licensed (open access) content; (3) is free to use by anyone (free of charge or other restrictions); (4) is community-governed and is transparent in its operations and IJDC | Conference Paper finances; ( , we also plan to further our understanding of the open infrastructure funding landscape by conducting additional research into the characteristics of open infrastructure services, and the patterns and gaps in open infrastructure funding.This additional research aims to produce actionable, practical recommendations and guidance for institutions, funders, and other supporters of open infrastructure, and for open infrastructure services to ultimately increase investment in and adoption of open infrastructure to further equitable access to and participation in research.The research will be built upon the data and insights gathered in the Infra Finder development process.