Making vulnerable groups able to connect socially and digitally—opportunities and pitfalls

1 Introduction

This article discusses the impact of providing digital learning activities (DLAs) by public and civic organizations to refugees, immigrants, senior citizens, and young adults neither in education, employment, nor training (NEETs). Many, although not everyone, in these groups meet obstacles in obtaining human capital through education or other formal channels, and refugees are particularly vulnerable (2022). Research has shown that inequalities in access to computers, internet and digital skills persist, even in well-connected countries where most of the population is online (van Deursen et al., 2017). Research has also indicated that there is a strong correlation between educational background and digital competence among refugees. Refugees coming from some African countries (Somalia, Uganda, Kongo or Eritrea) as well as some Arabic speaking countries are likely to have the lowest digital skills (Proba, 2022).

The article discusses findings based on a project conducted in 2021–2023, funded by the EU program Erasmus+. The aim of the project was to develop ways for public and non-governmental organizations (NGOs) to work as connectors in local network building with the aim to promote digital and social inclusion for people with low digital skills. Social networks, both digital and physical, were seen as important for participation in society, and the aim was to make citizens “able to connect” and become more included in the local community. Two governmental institutions (libraries), each in a medium sized city in Denmark and Norway, and an NGO (a social entrepreneur) in a small city in Belgium, cooperated in planning and implementing learning activities in their respective cities to give adults with potential low digital skills opportunities to acquire new digital competences. The project aimed to develop digital learning opportunities for individuals by activating resources in the community, to explore how institutions such as libraries can work as connectors in local network building, and to explore how professionals can work closely together in planning activities by sharing ideas, knowledge, and experiences. The project was a pilot, and the aim was to learn from the activities, so that they were able to produce better services later.

Each partner implemented two learning activities each, in their respective cities. The activities did to some extent build upon earlier experiences, but all of them were new initiatives that had not been tried out before. The six DLAs were similar in the way that they introduced the participants to new digital tools. All workshops focused on creative use of various digital tools: digital printing on clothes and other items (Norway), digital self-portraits (Norway), podcast making (Denmark), digital textile printing (Denmark), digital photography (Belgium) and digital silk screen printing of photos (Belgium). The topics for the learning activities correspond with the professional background of the project partner representatives, who were mostly from information science, media studies, film production and similar occupations, and all of which were very creative and vital. Only one of the representatives from the two libraries was a librarian. The partners cooperated with local sub-partners which represented the target groups in recruiting participants and implementing the learning activities.

The learning activities were inspired by design thinking as a way of developing the activities; striving to explore important user needs, criticizing own assumptions, and creating new innovative ideas and solutions. Design thinking is “an analytic and creative process that engages a person in opportunities to experiment, create and prototype models, gather feedback, and redesign” (Razzouk and Shute, 2012, p. 330). Two researchers from Western Norway University of Applied Sciences were invited to research the project (the authors of this article). A process tracing method often used within case design was chosen (Bukve, 2019). We followed the development of the Erasmus+ project from the beginning to the end and analyzed the processes in light of theoretical models and explanations (Beach and Pedersen, 2019; Bukve, 2019). We followed the processes without being active in any of the interventions. The design thinking approach, with its possibilities to redesign along the way, created a good climate among project partners to learn from their mistakes and to constantly try to improve activities. This also provided very rich and detailed data on all parts of the process. This article reports on results from the project and pays particular attention to the activities that was redesigned, and the reasons behind the redesign, as this provide unique insights into the ways in which public institutions can work together with civil organizations to develop better educational services outside the official or formal educational system for people who face difficulties to be included in society because of low digital skills.

In total 98 adults participated in six learning activities. The selection criteria for participating in the learning activities were low digital skills. One third of the participants were refugees, the rest were other immigrants, senior citizens, and young adults neither in education, employment, or training (NEETs). Although these groups consist of heterogenous populations of people who may have nothing in common except for “not accumulating human capital through formal channels” (Mascherini et al., 2012, p. 3), some of these categories overlap. NEETs is a category of young people (15–29 years) that also can include refugees and immigrants, and vice versa. In this article we define refugees broadly as someone who is “unable or unwilling to return to their country of origin owing to a well-founded fear of being persecuted for reasons of race, religion, nationality, membership of a particular social group, or political opinion” (UNHCR, 2010, p. 3), and do not distinguish between refugees who were recently displaced and those who have lived for a long time in a new country.

The article discusses how the three partners shared information about the development of their respective DLAs and how each partner worked in joint efforts with local sub-partners in initiating, planning, designing, and implementing the learning activities in their respective local communities. We use the theoretical lenses of co-creation and co-producing from management theory and theories on social capital building as described by Putnam (2000). The two main questions to be discussed are: How did the three partners cooperate with local sub-partners in planning and implementing the learning activities, and how did they involve representatives from the target groups in co-creating and co-producing elements in the learning activities? The article will point to some experiences as well as barriers and challenges that the two libraries and the NGO met in their work and will point out some possibilities for further developments.

2 Conceptual departure and theoretical approach

This article presents empirical findings that will benefit several areas of research. By combining literature and theoretical approaches from several strands; digital inclusion, public organization, and management research—with a particular focus on libraries, and their role in educating refugees and other vulnerable groups, we provide new insights on how adults with potential low digital skills utilize digital technologies to achieve digital inclusion, and how public and non-public organizations can work toward assisting these groups.

2.1 Digital inclusion

While other researchers have emphasized the importance of addressing digital needs of marginalized communities, including efforts to understand the unique challenges faced by groups such as seniors, individuals with disabilities, and immigrants and ethnic minorities (Alam and Imram, 2015; Bertot, 2016; Chadwick and Wesson, 2016; De Vecchi et al., 2016; Gonzales, 2016; Hargittai and Dobransky, 2017; Gómez, 2020), there is little research on those providing such training for people with low digital skills, how they design it, and the content of this training. This article provides new perspectives on how public and civic organizations can provide training in digital competence to vulnerable groups, more specifically, training for adults with potential low digital skills.

Research on digital competence for vulnerable people belongs to a broad research field, under the umbrella of “digital inclusion.” Digital inclusion is a multifaceted research field that explores the access, adoption, and effective use of digital technologies (Chadwick and Wesson, 2016; Kumar et al., 2022; Proba, 2022), access to the internet (Norris, 2001), often with a focus on bridging the “digital divide” (van Dijk, 2005). The digital divide is characterized by disparities in access to digital technologies, with an emphasis on the urban–rural gap, socioeconomic disparities, and issues related to race, gender, age, and disability (Gonzales, 2016; Hargittai and Dobransky, 2017). However, digital inclusion extends beyond mere access, and includes both digital skills and literacy. While the digital divide pertains to the gap between those with and without access to internet and digital devices, “digital literacy” focuses on the skills and abilities needed once access is available (Reisdorf and Rhinesmith, 2020). Digital literacy is essential for individuals to utilize digital technologies effectively once they have access.

As dependence on digital devices and reliable internet increases all over the world, being digitally excluded is often interpreted as social exclusion. However, digital inclusion does not directly translate into social inclusion, which is a much more comprehensive concept, which can involve exclusion from one or several important areas, such as: formal citizenship rights, from the labor market, from participation in civil society and from social arenas (Aasland and Fløtten, 2001). The project we analyze in this article aimed to build digital literacy, with the larger aim to promote social inclusion by attempting to find ways to bridge the digital divide. By building digital competence that could possibly open doors to education and the labor market, and through building social networks through participating in learning activities, the aim was to promote both digital and social inclusion.

2.2 Libraries and digital inclusion

Research on public libraries emphasizes their societal role as being pillars of information as well as of digital inclusion, topics that are well rooted in the concern over rising inequality in societies (Bertot, 2016; Beyene, 2018; Noh, 2019; Strover, 2019; Farney, 2021). Libraries are presented as some of the most inclusive public institutions in society and as important sites for internet access and digital inclusion efforts. However, as more materials migrate to the internet, and as changes occur on how to access information, libraries are challenged to incorporate new ways of information-seeking into their operations and philosophy (Strover, 2019, p. 189).

The role of libraries in expanding internet access, working with digital literacy, and the library’s emerging role in efforts in bridging the digital divide is addressed by several researchers (Bertot, 2016; Beyene, 2018; Noh, 2019; Strover, 2019). Many researchers pin-point libraries as some of the most important public institutions to achieve digital inclusion in society. There is an increasing literature emphasizing that these efforts cannot be done in a vacuum. Efforts, such as the creation and implementation of ICT centers for instance, targeting populations that are digitally excluded, are best done together with other institutions (Bertot, 2016). These collaborations might be between non-profit organizations, private-public partnerships, and the use of public or other types of libraries (Bertot, 2016).

2.3 Digital competence building for refugees

Findings from a research project about newly arrived refugees in Norway and their digital competency showed a strong correlation between educational background and digital competence (Proba, 2022, p. 2). A considerable number of refugees did not have sufficient digital competency to describe their everyday life as digital. Other factors, such as gender, age, family situation and residence time in Norway, also had an impact on their use of, preferences, and habits regarding digital services (Proba, 2022, p. 43).

Refugees and immigrants granted asylum have in many European countries, such as Norway, “the right and are obliged to participate in an introduction program offered by the municipalities” (Røhnebæk and Bjerck, 2021, p. 745). The purpose of the programs is to increase the opportunities for newly arrived refugees and other immigrants to participate in working and social life and to increase their financial independence. However, the results vary considerably across countries, counties, and municipalities. Research from Norway has shown that actors from the private and voluntary sector contribute significantly to the public introduction programs by helping newly settled refugees to obtain basic qualifications and achieve economic independence in the longer term (Røhnebæk and Bjerck, 2021, p. 745). Many adult education centers across Europe collaborate with public libraries on organizing for example tours of the library for refugees (Proba, 2022). For many refugees, this will be their first encounter with the local public library. The language café stands out in this respect as being a very appreciative service among refugees (Proba, 2022). However, research has also shown that collaboration between the public, private and voluntary sector requires strong coordination efforts, facilitator competency and meeting points between the involved actors (Espegren et al., 2019).

2.4 Collaborations between public and civil sector—co-production and co-creation

Research on public management focusing on the use of public services, as well as user and community co-production of services, started in the late 1970s. An important output of this research was the recognition that public services require input from both professionals and users to be fully effective (Ostrom, 1996). The concept of co-production has lately seen an upsurge theoretically and the concept of co-creation has been introduced to the research field acknowledging that public services need multiple stakeholders at all phases in the decision making processes to improve public services, with users, user representatives, communities and civil sector organizations playing key roles (Bovaird et al., 2016; Osborne et al., 2016; Torfing et al., 2016; Brandsen et al., 2018; Honingh et al., 2020; Könings et al., 2021).

The literature on what co-creation and co-production means, the understanding, definitions and what distinguishes the terms has become voluminous and will not be elaborated fully upon here. Shortly, they can both be defined as “joint efforts of citizens and public sector professionals in initiating, planning, designing and implementing public services” (Brandsen et al., 2018, p. 3). The two concepts point to distinct phases in the decision-making process, the planning phase and the implementation phase. Both can be understood as “the voluntary or involuntary involvement of citizens in public services in any of the design, management, delivery and/or evaluation of public services (Strokosch and Osborne, 2016, p. 640, Eriksson, 2019). The main difference between them is that co-creation refers mainly to the planning or preparation phase of the service, while co-production refers to the implementation phase (Brandsen et al., 2018, p. 3).

In sum, co-creation and co-producing efforts relate to collaborations between institutions within the public, non-governmental or private sectors to improve services to “the increasing number of people facing social and structural barriers to full participation” (Mulvale et al., 2021). In this article we distinguish between co-creation and co-production in the same way, by trying to develop knowledge about how public and non-public agents can work toward assisting people with low digital skills.

2.5 Social capital building

The DLA’s that we address in this article aimed at building participants’ capabilities through forming new relationships and networks that possibly could promote social inclusion by opening doors to education and the labor market. The digital activities did not aim at replacing human interactions, on the contrary, the aim was to give participants an opportunity to gain more knowledge about how to use digital tools in a social setting. The capabilities that were sought built in the project are partly analyzed using Robert Putnam’s theory on social capital building, in combination with Woolcock’s concept of linking institutions. We found that the concepts bridging, bonding, and linking social capital were useful (Woolcock, 1998; Putnam, 2000). Bridging capital strengthens networks across individuals with heterogenous characteristics, whilst bonding capital strengthens networks between individuals with homogenous characteristics. Both types of social capital indicate that context is important for individuals, as a social structure is created making them act in a certain way and develop mutual trust and common norms (Putnam et al., 1993, p. 167). Linking capital is a third type of network building across institutions at macro-level, like state institutions or public municipal institutions, securing the connection between important institutions, aiming at producing empowerment for the local community (Woolcock, 1998).

However, the concept of social capital is contested and has been criticized for being vague and tending to oversimplify complex processes (Claridge, 2021). Therefore, an explanation of how we understand this concept is needed. We understand social capital as capability building through networking, such as co-creation and co-production, that might lead to innovation or developing skills, confidence, or a capacity to resolve future problems (Osborne et al., 2016, p. 645). An important question here is whether the DLA’s made it possible for participants to reach out to each other, providing new experiences and developing new relationships. In this perspective, Putnam’s theory on building social capital through bridging and bonding showed relevance (Putnam, 2000). The linking type of building social capital (Woolcock, 1998) addresses mainly connections on the institutional level, was equally important as a theoretical lens to understand the value of what was gained.

Furthermore, as we went more in-depth into the analysis of linking, bridging, and bonding, public management theories on co-creation and co-producing helped to clarify important decisions and organizational steps that the partners took in the process. These theories have proven relevant for research on the design and redesign of public services (Strokosch and Osborne, 2016; Brandsen et al., 2018), and we found it equally relevant for our analysis. Robert Putnam’s research emphasizes that a successful co-creation and co-production between public and civil sector might encourage citizens to develop trust through horizontal relationships as well as social capital through for instance bonding and bridging (Ostrom, 1996; Putnam, 2000).

Digital learning has many benefits, but also uncertainties and risks. According to Lember, the debate today is pervaded by a techno-optimism that may or may not have negative social impacts (Lember et al., 2019). Libraries often present themselves as public institutions partaking in community networks to bridge the “digital divide.” Although not discussing the role of libraries, Lember claims that different digital technologies in general unintentionally might alienate potential users (Lember et al., 2019). Being aware of this, our theoretical lenses focus on whether and how the digital learning activities affected behavioral change and built social capital, relying on co-production and co-creation through the project.

3 Methodology

We chose a case design approach, using a process tracing method that often is used within case studies (Bukve, 2019). The case in this project is the design and provision of DLA’s to vulnerable groups outside the formal educational system by the three partners in the Erasmus+ project, in three countries. Process tracing means using in-depth analysis of one or a small number of cases, enabling causal inferences to be made about how the processes work (Beach and Pedersen, 2019). We followed the decision-making processes by the three partners from start to end—through the phases of planning, implementation and evaluation. The data collection process involved a mixed-methods approach, incorporating qualitative and quantitative techniques to provide a comprehensive understanding of the research questions. The data was collected throughout all phases of the project (planning, implementation, and evaluation), and consists of participant observation, qualitative interviews, and surveys.

Participating observation was conducted during fieldworks to each partner institution in 2021 and 2022, and included meetings and conversations with project representatives of the institutions, as well as observations of project activities. We also conducted participating observation in 12 bi-monthly digital meetings with partner representatives in 2021–2023. Observation data provided important information on the organization of the project and the DLAs, as well as the cooperation between the partners. Notes from the observations are used in the analysis to describe the activities and to analyze interactions between the partners throughout the project.

Semi-structured interviews were conducted with representatives from each of the three partners in the project. The interview guide was designed to explore the project partners’ perceptions, experiences and strategies related to planning, implementing, and evaluating the learning activities. Interviews lasted approximately 1 h and were audio-recorded and transcribed for subsequent thematic analysis. We interviewed representatives from each of the three partners two times. In Norway, we interviewed all four representatives in the project in 2021 and 2022, conducting focus group interviews. In Denmark we interviewed the representative responsible for the first learning activity in 2021, and the coordinator of the learning activities in 2022. Belgium had two project representatives, and we conducted a focus group interview with them in 2021, and in 2022 we interviewed the person in charge of the activities. Citations from the interviews are used in the analysis.

All project partners filled out three self-reporting schemas for each learning activity, six schemas each, 18 schemas in total. The schemas describe the planning, implementation and evaluation phase of the learning activity. The schemas were designed as a survey for the institution representatives to fill in their own reflections. They provided valuable data on the progress and the process of each learning activity, as well as the obstacles in the implementation of the learning activity and choices and changes that had to be made along the way. The self-reporting schemas are cited in the analysis.

A simple survey with 15 questions was administered to all participants in four of the six learning activities to gather quantitative data on their perceptions of participating in the learning activities. N = 20, and the response rate was 31%. The survey was not administered in the first two activities (Norway and Denmark) because of time restraints. We used SurveyExact as a survey tool. We were particularly interested in aspects that could indicate network building or other indicators of bridging or bonding capital. As several participants had low digital skills as well as low skills in the native languages of Norway, Denmark, and Belgium, we took means to make it very easy to understand the questions and to handle the digital survey. We translated the survey into English, and we employed a 5-point Likert scale based on green, yellow, and red smileys, for it to be easy to understand the alternatives. The partner institutions helped us administer the survey to the participants after the learning activity, via email. Due to time restraints and concerns for anonymity, a digital survey was chosen. However, a paper survey that could have been filled out immediately after the learning activity would probably have generated more answers. Instead we chose a digital tool which provided anonymity, and the answers were only made available to the researchers, not the partner institutions. We do not know how many of the respondents were refugees as the survey did not ask about social background. Due to the weaknesses in the distribution and the low response rate, the findings can only represent weak indications. We still include results from the survey in the analysis, as it provides interesting trends relevant for the understanding of the project we analyze.

Interviews, self-reporting schemas and survey materials are analyzed using thematic analysis, a method designed to identify and analyze patterns (themes) in the data material (Braun and Clarke, 2006). We used manual coding and employed a combination of theory driven coding and data driven coding (Fereday and Muir-Cochrane, 2006). We first categorized information in line with Putnam’s (2000) theory on social capital building and theories about co-creation and co-producing processes, and then looked for new findings that could possibly elaborate or even challenge the first round of coding.

Since the authors of this article belong to Western Norway University of Applied Sciences (HVL), one of the partners in the project that is analyzed in this article, we include a discussion of the research positions of the authors. Process tracing method is a fruitful method when researching time limited projects. It is important to note that process tracing researchers are not actively involved in the activities. The process tracing method is, however, not neutral, as it was necessary to follow people and activities closely over time. The learning activities were solely developed by the three other partners, and the researchers were not involved in the planning or implementation. Our task was to research activities and to give research-based input along the way. In two meetings with the partners, one in the beginning of the project and one mid-way, we presented Putnam’s theory of social capital and the status of research in the field of community building with a focus on collaborations between public and civil sector. We also presented our preliminary research findings in another meeting with partners midway in the project.

Since both authors have been partners in the project from the beginning, our analysis may of course be affected. To be reflexive and conscious about how we interpret the data material, and not to “force” meaning to the data has been important to secure that we interpret the data in new ways and not “reinventing” what is already known to us (Sundet, 2014, p. 36). This included giving extra space for curiosity and new ways of interpretating the interviews, observations, and surveys, to prevent our preconceptions from blocking out meaningful interpretations. An advantage of being closely involved in processes we research is that this opens for further developments of what is the object of the research (Sundet, 2014), in this case DLA’s for adults with low digital skills. The article therefore suggests a few developments and lessons learned from this project toward the end.

Ethical approval for this study was obtained from the Norwegian Agency for Shared Services in Education and Research, ensuring compliance with ethical guidelines for research involving vulnerable groups. Informed consent was obtained from all participants, and their anonymity and confidentiality were rigorously maintained throughout the study.

4 Findings and discussion

We divide the findings in this article into two main sections following the organization process of the project we analyze: (1) the planning and preparation phase, and (2) the implementation phase. These faces will be analyzed using the lenses of organization and management theories as well as theories on social capital as outlined earlier.

4.1 Networking and co-creation in the planning phase

The three partners in the project had bi-monthly digital meetings throughout the whole project period. In the planning phase these meetings were used to share plans and ideas for their learning activities with each other and give feedback to each other’s planned activities. This type of cooperation was inspired by design thinking, which gave opportunities to be creative and flexible in designing and redesigning activities, and to learn from each other’s experience (Razzouk and Shute, 2012). The partner representatives expressed several times throughout the project period that they found this cooperation very inspiring and helpful when designing their respective activities (notes from digital meetings). They also planned to launch an international network of library professionals where they would share information about the project and the ways they worked with digital and social inclusion through the project (notes from digital meetings). Drawing on Woolcock (1998), this way of building network across institutions (linking social capital), was a central element of the project.

The project also aimed to develop digital learning opportunities for individuals by activating resources in the local community, by the way of co-creation, understood as joint planning efforts between representatives of both civil and public sector. Co-creation and co-producing (in the implementation phase) are inter-locked but belong to different phases in an organizational process. What they have in common is that citizens and public sector professionals work together during an organization process toward designing and delivering a new service, that is they work together in all the phases of the process: through initiating, planning, designing, and implementing the public services (Brandsen et al., 2018, p. 3).

The partners planned to collaborate with several sub-partners in their local communities as these would know the target group very well and have in-depth knowledge of their needs. All partners worked with one or several local sub-partners in the planning phase to recruit participants, such as other public libraries (Denmark), companies (Ltd’s) owned by a municipality (Belgium), public schools (Denmark, Norway, and Belgium), social entrepreneurs (Norway and Belgium), voluntary organizations (Denmark, Norway and Belgium), and other NGO’s (Norway and Belgium). In this way several informal networks started to develop through co-creation efforts to recruit participants. The project partners inspired each other on this issue, and shared experiences about their chosen partners in the planning phase (notes from digital meetings).

While the partners were successful in co-creating with the other project partners in developing the learning activities, as seen above, they found it more challenging to actively involve local institutions (sub-partners) who represented the target groups in co-creation processes when planning and developing the activities. Local partners mostly contributed to recruiting participants, as we will elaborate in the next section, and were less involved in the planning of activities, thus not involved in co-creating. Local sub-partners were to a little degree invited to meetings or asked for advice on behalf of the user groups that they represented when planning the content of the activities (interviews, Norway, Denmark and Belgium). Some activities were for instance dismissed already at the beginning of the planning process because they did not find local partners that were interested in the idea. For example, in Norway, they planned a workshop using a VR story telling tool. They tried both to contact a local secondary school, and an NGO who worked with minority women, but none of them were interested enough to carry out the partnership. The planned workshops also had to be canceled several times because of corona restrictions, and this eventually made the partners give up the effort of cooperation with them. The planned workshop was eventually not implemented because of this, and they decided to carry on with other learning activities instead (focus group interview, Norway 2021).

Even if most of the activities were not co-created with the target group and local sub-partners, the partners were highly aware of the problem. During conversations along the way in the project, also in the early phase of the planning process, it was a current topic for discussion about what local organizations would be the best partners, and why (notes from digital meetings). It was equally important for the partners to have the right mindset and preconceptions, not objectifying target groups: “We should not define the target groups as being challenged, seeing them without agency, defining their limitations or putting them a priori in a specific frame of understanding. This is something we must work with” (interview, Denmark 2021).

Co-creative activities and having the right mindset as discussed here, have long proven to be a challenge for many public organizations. Eriksson (2019) and Osborne et al. (2016) are among many researchers who argue that the logic of widespread New Public Management reforms have proved unfit for the public sector. As Eriksson puts it: “Standardized, ‘one-size-fits-all’ solutions have failed to address the diverse needs and prerequisites of some social groups” (Eriksson, 2019, p. 308). Eriksson (2019) furthermore states that “to target disadvantaged groups in society, it is necessary to integrate social context into this logic more than is the case today” (Eriksson, 2019, p. 308). Recognizing exactly this challenge, all the project partners planned for learning activities which would develop social skills for groups of people by seeking partnerships with representatives for these groups, but perhaps did not appreciate enough the need for involving them also in the earliest phase of the planning process.

4.1.1 Producing digital and social inclusion through networking

The aim of the project was to develop new ways for public and non-public organizations to work as connectors in local network building with the aim to promote digital and social inclusion for people with low digital skills. In Norway, for example, the first learning activity was a one-day workshop carried out in 2022. They introduced two soft-wares; “Procreate,” where the aim was that participants would learn how to “make a unique self-portrait from a scan, print and frame it,” and “My Heritage Deep Nostalgia,” where the participants would “bring an old photo and they will learn how to make the old photo “alive” with the help of My Heritage” (self-reporting schema, Norway). In Denmark a mobile podcast studio (a special built car) was set up in the street in front of a new library, where people in the street were invited to come in and learn about how to use the instruments, tell a story using the microphone, as well as invited to visit the library (interview, Denmark 2021).

The planning of the technical activities in all six learning activities in the project was closely linked to the aim of producing digital inclusion. The mentioned learning activity in Norway for example, aimed to develop “Basic digital skills, scanning photos, uploading to cloud services, using software, sharing content/ downloading video files. Also enhanced social skills, meeting others” (self-reporting schema, Norway). Combining developing basic digital skills with social skills was an aim in all the learning activities. The mobile podcast studio in Denmark had a resource focus, inviting people to share stories with other people on the street about their neighborhood. It was designed as a spontaneously and voluntarily invitation (interview, Denmark 2021).

The plans in all DLAs circled around various ideas in how to best promote digital competence to people who have, not only low digital skills, but often also low interest and low motivation in learning digital competence. One partner representative in Norway explains how they were thinking about this topic:

“We are planning let them get to know computers in a creative way, as a soft way to get familiar with a computer. We will introduce them to some very simple digital tools, that are easy to use, and by that show them that computers are not only useful for things like writing a job proposal.” (Focus group interview, Norway 2021).

In order to build social inclusion for the participants, they planned for a series of workshops for them to get to know each other and potentially build networks that could help them later in getting for example jobs: “We want them to get to know each other. The digital tool invites to share personal stories about each other, and we hope that this makes them get to know each other better. This is the social part of the workshop, that they get to know each other and build networks” (Interview, Norway 2021). Drawing on Putnam (2000), at the individual level, strengthening contact between individuals who are not familiar with each other (bridging) is one way of building networks. Strengthening contact between individuals who already have something in common (bonding) is another way. They were thus planning for both bridging and bonding activities in the DLAs.

All partners also made a point of how the technology could lead to inclusive elements. Several of the learning activities included the aspects of using technology as an invitation to present personal stories and involving the participants personally. For example, the above-mentioned learning activity in Denmark inviting people on the street to tell their own story:

Podcasts are an excellent tool […] to make those voices heard that we rarely hear. We want to explore the democratic potential of the podcast platform and produce podcasts featuring people from [name of unprivileged area in the city]. […] We want the participants to experience that their version of reality is valuable and that the podcast can provide an actual platform for distributing that story. The digital literacy that we want the participants to build is one that is focused on enhancing the perception of their own digital voice – because they DO have one. (Focus group interview, Denmark).

The partner in Denmark hoped that the technology would both strengthen the relationship between participants and to be a social activity by inviting the participants to present personal stories for each other, as well as a tool for self-expression. Mechanisms of digital inclusion and exclusion are highly social, as they entail a diversity of formal and informal support-seeking patterns, which in turn have an important influence on the adoption and use of digital skills (Asmar et al., 2020).

4.2 Networking and co-producing activities in the implementation phase

In the project we analyze in this article, the partners aimed at working more closely with local sub-partners than they had before, by co-creating and, in particular, co-producing activities; to not only cooperate in recruiting participants but also to co-produce social capital together with local sub-partners by designing activities to match the participants’ needs and implement them accordingly. Some of these intentions proved more challenging than others. In this section we will have a closer look at how the learning activities were co-produced in the project and what social capital was produced, and some of the challenges that were faced in the implementation phase.

4.2.1 Building networks at individual level—bridging and bonding social capital through co-producing

The project wanted to explore the possible role of libraries and NGOs as co-producers, or as drivers for digital and social inclusion by both generating digital literacy and connecting individuals together to form their own social networks. A small survey sent to the participants after the end of each learning activity revealed that they were overall very satisfied with the learning activities. Almost all participants reported that they enjoyed participating, that they learned something new, and that they gained more confidence. Many, but not everyone, reported that the digital tool they learned was relevant for them, indicating that they had gained at least some basic digital skills during the learning activities.

Many also reported that they had gained new friends during the activities, and a few said they had gained a new social network, something which indicates that the building of social network by the way of bridging and bonding among the participants was at least on its way for several of the activities mentioned in the survey. Bonding capital is, as mentioned before, a type of network building that strengthens networks of trust across individuals within homogenous social, economic, and/or demographic characteristics. Bridging capital is a type of network that strengthens networks of trust across individuals with heterogenous social, economic, and/or demographic characteristics. Several of the partners said in interviews that they observed that some of the participants became friends, one example is during one of the learning activities in Belgium, which had three workshop days over 3 weeks. The partner representative from Belgium told us that they had participants from many continents and countries: Europe, Africa, and the Middle East. He said: Between them they could be talking twenty languages. But by the end of workshops they usually all are having coffee together, you will also see that they come an hour earlier, they have coffee and talk (Interview, Belgium 2022). He observed that the participants started to talk more and more, and even started to hang out together inside and outside the building before and after the workshops. Further, the Belgian representative reported that it is seldom that refugees become friends with Belgian people who have lived in the area for a long time. For these participants it is more bonding than bridging, especially for those living in a refugee camp nearby as they obviously are isolated from society, he reflected (Interview, Belgium 2022). Bridging is a particular challenge regarding asylum seekers, due to their status as non-citizens or as marginalized groups in society (Strokosch and Osborne, 2016).

As discussed in the previous section, the project had high ambitions for promoting social inclusion by co-producing social capital through bridging and bonding activities in the DLAs but found that this was not as easy as intended. Most realized throughout the project that building networks and the bridging type of social capital takes more time than what they had planned for. The mobile podcast studio in Denmark made people very interested in the podcast technology itself and some young people started bonding with each other, talking about how “cool it was” (interview, Denmark 2021). However, many did not want to share their personal stories, as had been expected by the public library. Instead, people stopped and were curious about the technology, talking about it. Elderly people talked together about their childhood memories, as opposite to the modern digital world; when they wrote postcards and listened to the radio (interview, Denmark 2021).

The Belgian partner explained in an interview that the bridging type of building social capital would be the next step in future projects. He emphasized that this would be hard work, for the municipality and for the involved NGO’s. The future aim, he emphasized, was to create a “safe space” downtown in the “shady” areas in the city and build a new community center where people with different backgrounds could mingle (Interview, Belgium 2022). Part of the aim was to develop a youth center combined with a little “plaza for workshops with artistic endeavors,” he described (Interview, Belgium 2022). The Belgian partner, as well as the other partners, thus recognized that the timeframe of the project was too short, and that activities co-producing social capital and inclusion require more investments in time. By investing in more permanent activities where participants can meet over time, there is even greater potential for building trust and both forms of social capital.

Research has shown that many integration or inclusion measures, services and projects focus mainly on the individual level, and less on the institutional level (Ødegård et al., 2014). An explanation might be that institutions often are infused with values that are difficult to change (Selznick, 1949). We observed that the partners had high ambitions in promoting digital and social inclusion for people with low digital skills, and to make citizens “able to connect” and become socially included in the local community though participation in the DLAs. The project thus aimed to reduce a problem—social exclusion of some citizens—by implementing efforts at the individual level—learning activities for individuals with low digital skills. This proved problematic, as the partners realized throughout the project that they were not able to co-produce and build social capital in the ways that they had planned. Social inclusion requires work on both meso and macro levels; to change institutions from within, to change values, mindsets, traditions, and habits, as well as working for changing the system on a political level (Ødegård et al., 2014).

The partner in Belgium used a strategy that was not used by the public libraries, called “Asset-based Community Development Strategy” (ABCD)—which can be seen as co-producing social capital in practice. ABCD is described as a strategy to (re)discover and mobilize resources in a community on both micro and macro levels; on individual level mobilize talents and skills, on group level get individuals with common interests to create networks and at the institutional level, support individuals and groups. This strategy asks for using local capacities, finding useful and valuable qualities and resources in individuals and networks. A main key in the ABCD strategy is the focus on skills, and the claim that sustainable solutions come from within the community itself (Kretzmann and McKnight, 1993). This strategy might explain why the NGO in Belgium seemed to be more successful than the public libraries in co-producing by the way of bridging and bonding activities, due to their long experience and status as a social entrepreneur in the civil sector.

4.2.2 Libraries and NGOs as connectors in local network building—linking social capital

Through cooperation with civil society actors all three partners aimed to recruit participants and to develop relevant learning activities for local citizens (Osborne, 2018). Securing the connection between important institutions in the local community is as mentioned above regarding the ABCD-strategy, vital for producing empowerment for the local community and for linking people across organizations with those in positions of power or influence to make changes happen (Modood, 2012). Our analysis shows that libraries have the potential of taking up such a linking role, which is vital for social inclusion work to be beneficial for the community and the target groups. In the project all the partners functioned as a “spider-web,” connecting relevant institutions together in developing their learning activities. As one representative in Norway said it: As a library, all target groups are ours. We work to reach all people. But it is truly demanding to maintain all the communication channels with each target group so having partners that can reach out to them is vital (focus group interview, Norway 2021).

All three partners in the project cooperated with other local sub-partners such as schools, public welfare and social institutions, limited companies (Ltd’s), NGO’s and non-profit social entrepreneurs. The Belgian partner however, since being an NGO, had as mentioned above longer experience cooperating in the form of co-creating and co-producing with established partners and using the ABCD-method in the community than Denmark and Norway. While the first round of learning activities in Norway and Denmark were less integrated in the local community, in terms of co-creation and co-production, their second learning activity was to a greater extent a result of co-production. Now they cooperated more with local companies, some of them owned by the municipality itself, or with public schools and different NGO’s, such as public adult education schools or centers for refugee immigrant and, local labor market companies (Ltd’s) working with preparing long time unemployed citizens for the labor market. Libraries are natural meeting places in local communities, and observational data showed us that the two libraries in the project made a big effort to inform participants about other activities in the library and to educate them in how to use this public service. In all their learning activities, they included information about the library services, and when applicable also a guided tour to the library (interviews, Norway and Denmark 2022).

4.2.3 Co-producing with local sub-partners

Acting as a connector toward local sub-partners was, by far, the most challenging part of this project. As seen in the previous section, several plans for learning activities had to be canceled because of efforts of co-producing that did not work. When having many partners, some unforeseen events occurred, and we will look closer at some examples.

4.2.3.1 Recruiting participants proved difficult

To reach out to immigrants and refugees, the partners aimed to co-create together with what they saw as successful NGOs to recruit potential participants to the DLAs. For Norway in the first round, it was challenging to invite young female immigrants to learn how to do digital photoshopping using VR glasses. The reasons were manifold, partly because of restrictions due to the corona pandemic at the time, partly because of a change in the leadership of the NGO and partly because a project leader in the NGO changed her mind about the relevance of the digital technology for their members. She decided that learning to use sewing machines were more relevant for the target group than computers (focus group interview, Norway 2022). As one disappointed Norwegian interviewee put it, when commenting this: I think they were a bit critical because they thought that this is not the right path for our members to get a job (focus group interview, Norway 2022). They experienced that the library’s approach about being playful and creative did not resonate well with minorities whom they experienced were very rational and goal-oriented: They want to learn exactly what gives them a job and our perspective is a wrong approach (focus group interview, Norway 2022). Thus, they realized that they had to think differently to be able to recruit. They, as well as Belgium, suggested that one needed to convince them a bit: Okay, maybe learning photoshop is not right now, but you are getting a skill, and skills can be transferred… knowledge is like a vault, you keep them like putting money in a bank, you invest your knowledge (interview, Belgium 2022).

In one of the workshops in Norway, their strategy to “convince the participants” was to mix a creative activity (printing on clothes and other items), with information about (1) the library services—especially about their newly opened technology lab with 3D printer, podcast studio, video studio etc., and (2) practical help with digital services in the municipality, such as help with applications for day care for their children, housing allowance, municipal housing, etc., as well as “other digital issues the participants may have” (self-reporting schema, Norway).

This example illustrates well something that was common for all the six DLAs; they were first and foremost successful as creative activities that used technology in new ways. As pointed out in the previous section, because activities were not co-created and developed as an answer to an articulated need from the target groups, and representatives from the target groups were only partly involved in the planning, the activities “missed on the target” in several cases. This does not mean that the participants did not enjoy participating (the survey also shows that most were satisfied), but we believe that it made recruitment difficult, as well as making the workshops less relevant for digital and social inclusion for the participants. The survey also shows that the participants reported that only a few said they had gained a new social network.

Another issue that influenced recruiting participants was that the three partners wanted to avoid giving participants a feeling of being stigmatized. The partners in Norway reflected over the problem in this way: “Gradually we found it difficult to craft direct marketing for the target group – like posters. Who will participate if we promote an activity for “those with low digital skills”? (Self-reporting schema, Norway). They also experienced that many participants showed interest, but never showed up. This was the same situation in Denmark and Belgium. When we asked the partners about how many participants they expected to recruit, the Norwegian partner answered: “Can not answer – we do not know anything about how many and who will attend the workshops. […] Based on previous experiences, we will take what we get” (Self reporting schema, Norway). Again, these are challenges that we believe could be better addressed if the partnerships with the local sub-partners had worked out better.

Another important reason for the problems with recruiting participants was that the project was launched in 2021, in the middle of the Covid 19 pandemic. This had a significant impact on the activities in the project, which needed to be completely readjusted: “The biggest problem was the outreach, during corona was a very difficult challenge and the strategy to do through the neighborhood activation did not work at all” (self-reporting schema, Belgium). The pandemic made participation difficult as all activities were planned for physical participation, and because the target groups were people with no or low digital skills, digital participation was not an option. Also, throughout the projects it became clear that the pandemic had changed people’s habits, making them less prone to participate in public activities than before, and they faced difficulties in recruiting also after corona measures were lifted.

4.2.3.2 The importance of trust

Another challenge in the project was that they realized that the target groups needed to build trust over time, as one interviewee said: The participants need to know that the place is open and ready for them, and we need a partner with a continuous success for the outreach (Interview, Belgium 2022). Having successful co-production with a local sub-partner was eventually seen as vital to reach out to the target groups.

In Denmark, this caused problems, especially in the first learning activity, as they relied too much on their own staff in the library and had not established co-creation with local partners in their learning activity. The mobile podcast studio was developed and placed in an unprivileged suburb to invite the local community into the studio and record people’s experience of their community. The citizens were as mentioned above, to some extent interested the technology behind the podcast and got many curious questions, but did not experience that they were able to teach anyone to make a podcast, or to use the technological equipment properly (Interview, Denmark 2021). We believe that the main obstacle was that they did not involve the community beforehand by the way of co-creating, and that they were not able to relate to what the locals felt were their actual needs. One reason for the partners to not include their local partners in co-creating activities was that they did not have enough knowledge about the coordination of such activities with other partners. This was a pilot, and the aim was to learn from the activities, so that they were able to produce better services later.

While Denmark did not involve local sub-partners in co-creating activities in their first learning activity, they learned from this experience and included several sub-partners in their second learning activity. The second learning activity (printing on textile) included co-production with a local school for adults under the age of 25 and who needed a comprehensive education offer to get ready for a vocational education. Twelve of the 18 participants were refugees (self-reporting schema, Denmark). They also met most of the participants in the workshops at the library, which provided an advantage in building trust (interview, Denmark 2022). Also in Belgium and Norway, the second round of learning activities were implemented with the co-production of several sub-partners. The Belgian representatives elaborates:

To do outreach and recruiting for this target group is more effective to partner up with institutions that already have a relationship with the group. Education institutions and welfare institutions have proven to be the best kind of partner since they already have a relationship of trust and can guide them to your activity (Self reporting schema, Belgium).

Building trust thus was imperative to providing learning activities which were able to build both digital literacy and promoting social inclusion, but not always easy to do within the short timeframe of the project.

4.2.4 Digital inclusion is more than a technological issue

All partners in the project planned DLA’s that made it possible for participants to reach out to each other, providing new experiences and developing new relationships, thus aiming for digital and social inclusion. They recognized that digital inclusion is not just a technological issue; rather it entails a variety of formal and informal sources which can both enhance or constrain access to, and use of, technologies (Asmar et al., 2020). The implementation of activities, however, showed that these goals were ambitious and harder to achieve than anticipated. In this section we discuss some of the reasons for this.

4.2.4.1 Finding the best level of digital ambition

All three partners learned that they had to lower their ambitions and focus more upon technical learning than digital inclusion because of the short time frame of the project (interviews Norway, Denmark, and Belgium). Understanding digital inclusion as the ability of individuals and groups to access and use information and communication technologies, they experienced that they had to reduce the technological level and change the content to focus on more basic skills. As the representative from Belgium explained talking about what happened during their first learning activity, teaching photography:

This was a new experience because it usually are people who are into photography, that already have a camera and are familiar with it, but these participants are people coming for other reasons, for instance being on welfare benefits. They are just coming because this is an offering. They might not be particular motivated at first. But then something really interesting started to happen. At first I give like this all technical things and it was their face they had, it was like a truck run over them because it was way too much information. Then I told them just work with your phones, and I focused more on the semiotics of photography, like how to treat photography as a language and give them a new language to express themselves (Interview, Belgium 2022).

From helping the participants, he changed to see their resources and take it further from there. He explained:

They want a set of instructions. You gonna do this you gonna do that. And I began differently… If you keep just giving instructions, they just learn that, to follow instructions. That’s not how you solve problems. I had a student for example at the beginning, she did not even talk. By the end she was talking more and more, and she was actually downloading TikTok, which I do not approve, but ha ha (Interview, Belgium 2022).

Individuals develop various ways of coping with learning in a society in constant change. Indeed, rapid technological evolutions are progressively transforming all realms of society, requiring individuals to learn and update their skills at a faster rate than before (Asmar et al., 2020). This includes of course, that the participants choose carefully what technology they want to participate in, corresponding to what their subjective needs are. This corresponds with research showing that people demonstrate their agency in choosing which moments are the most beneficial to make use of their support to acquire digital skills (Asmar et al., 2020, p. 148).

Another example is from Denmark, and the podcast studio. Even with good intentions they were not able to teach the technology sufficiently in such a short period. This means that after the learning activity the participants could not produce a podcast, but the Danish partners were convinced that at least some of the participants could translate knowledge about this platform to other digital platforms and devices. Through their presence in the community over 3 days they argued that the activity reached out to people in their own neighborhood making positive publicity about the activities of their brand-new library (Interview Denmark, 2021). In terms of bridging and bonding, the activity lasted too short to give measurable results on this but had potential for further building networks by creating a link between people and the local library. Research has documented that supporting people to extend their social networking is crucial, not only for social inclusion, but also for digital inclusion. Asmar et al. (2020) finds that despite difficulties, social and/or digital, many people with low digital skills show high interest and motivation to engage with digital technologies. The question is to find the best level in which to support them.

4.2.4.2 Social capital building takes time

Continuity and predictability are important values for institutions that are working for vulnerable groups such as refugees and NEET’s. Both public and private institutions report that it is important to have long-term predictability and express concern as they experience that it is easier to get funding for new and fancy projects, but more difficult to get funds when a project is about to be consolidated and be part of a continuous operation as having a predictable economy as an important condition to deliver long-term projects and services. The project we analyze in this article was funded by Erasmus+, an EU’s program to support education, training, youth, and sport in Europe.¹ The project’s high ambitions in their goals and aim, as elaborated earlier, were not necessarily in sync with the granted amount, and led to some of the challenges we have elaborated. For example, building networks for social inclusion at the individual level proved challenging mostly because the learning activities were short term, with one or two sessions, the longest lasted for three sessions (Belgium). In order to build continuity and long-lasting networks which can promote social inclusion, more time is needed compared to what was possible in this project.

A major aim for the project was to build social capital among the participants, which would potentially help them become more integrated in society and to get access to jobs or education. This is an ambitious task that takes time (Ødegård et al., 2014). Furthermore, bonding social capital has shown to be easier to work with as it is easier to bond with people that have something in common. Bridging people who have little in common, is a long-term task that needs to involve stable institutions, such as NGOs and municipalities (Ødegård et al., 2014), and is therefore more challenging and time-consuming.

5 Conclusion

In this article we have indicated some key points in how public and civic institutions can make the services more available and useful for vulnerable populations, such as refugees. The article has used theories on social inclusion through capital building, by the way of co-creation and co-production as analytical lens to discuss how educational services outside the official or formal educational system can develop better facilities for people who face difficulties in inclusion in society because of low digital skills. Our findings indicate that it is demanding, but rewarding, to induce cooperation between various public and civic institutions, and that this increases the chances for providing high quality services that fit the actual needs of the users. We have pointed at pitfalls such as:

• Bridging social capital is difficult to achieve in short-term projects.

• Involving users or local sub-partners and NGO’s to co-create and co-produce in the planning and implementation phase is vital.

• There need to be sufficient time and efforts to plan activities and to recruit participants.

• Encourage participants to use the technology is important.

• Avoid stigmatizing/objectifying is important.

• Customize digital ambitions - finding the “right” level is important.

• Develop trust is vital.

• Social and digital inclusion takes time.

We have pointed to co-creation and co-producing activities between public institutions and NGOs as important organizational conditions for building digital and social capital among people with low digital skills. We have also argued that the social capital that was built as a result of the participation in the DLAs was first and foremost the bonding type—that is bonding people with similar backgrounds—and less bridging people with diverse backgrounds. This was partly because of structural reasons, beyond the control of the partners in the project. For example, integrating participants coming from the refugee camp in Belgium with other residents, by the way of bridging, was more or less impossible, due to the involuntarily isolated existence of the refugees. Bonding social capital may not lead to the type of networks necessary to enhance social inclusion in society but is useful for strengthening social bonds and potentially building social confidence.

Sufficient planning is vital to develop activities that are relevant to users, and to secure that activities are implemented and carried out in the best manner. The project partners tried to involve local partners in co-creating activities but failed to develop activities which actively involved representatives for the users in the planning process. Lack of user involvement in the planning phase led to problems in recruiting participants and to less relevant activities. Although the libraries functioned well as connectors between different actors, they do not necessarily have firsthand knowledge about the users and their needs. To recruit efficiently, they experienced that participants needed to be invited personally by someone they trust and know, otherwise they might not attend. Reaching out to groups of people is less efficient than individuals. This is something that requires time and knowledge about successful and trustworthy networks in the local community (Mulvale et al., 2021; Røhnebæk and Bjerck, 2021).

The project aimed at increasing social inclusion by raising digital skills, while at the same time building social networks. They learned that their digital ambitions needed to be customized better to the target group for the best result. In some of the learning activities they assumed that participants had higher digital skills than what they really had, making the skill difficult to learn and perhaps less useful. Maybe the partners were too techno-optimistic in the beginning, but they managed to alter the technical bit of their activities and make them more basic. They also changed their approach from giving instructions and mere help, to a mindset seeing the individual’s resources and encouraging them further to solve problems using their own creativity and capacity.

The project had high ambitions for building social capital among participants, at the same time as having only limited financial resources. This led to very short-term learning activities, most of them lasting only 1 day. This made bonding and bridging activities difficult. Although the participants may have developed digital skills through the learning activities, the activities did not live up to it’s potential in promoting digital and social inclusion.

The project was inspired by design thinking that gave opportunities to be creative and flexible in designing and redesigning activities. In sum, all the project partners gained knowledge about new ways of collaboration (such as co-creation and co-producing) between organizations in the public and civil sector and with the target group. In the future, this can potentially result in better cooperation between organizations, easier out-reaching and recruiting as well as better customized digital activities for their targets groups and the building of social capital among all participants.

As a conclusion, we argue that well established cooperation between instances in the public and civil sector is a vital condition for success for social inclusion, and that libraries and other public institutions can fill a role as connecting actors by the way of linking relevant actors together and reaching out to participants (Osborne, 2018). Co-creation and co-production are much discussed in public management literature today and are rather new theoretical approaches within both educational and social service provision research. This article has pointed to some ways in which co-creating, and co-producing may be implemented in practice and pointed to some success factors for the planning and implementation process. Short-term projects, like the ones presented in this article, are less likely to develop digital and social competence. Quality takes time and resources and must involve the users. This is demanding, but far from impossible.

We hope that this article will provide knowledge to make more public and civic agents successful in the implementation process in similar projects.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by Norwegian Agency for Shared Services in Education and Research. The studies were conducted in accordance with the local legislation and institutional requirements. The ethics committee/institutional review board waived the requirement of written informed consent for participation from the participants or the participants’ legal guardians/next of kin because it was considered by the Norwegian Agency for Shared Services in Education and Research that is was sufficient with oral consent from participants in the study.

Author contributions

RF: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Writing – original draft, Writing – review & editing. BR: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. The research is funded by EU’s Erasmus+ program.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

References