Yorick Last
Abstract
Documents are largely stored and shared digitally. Yet, digital documents are still commonly signed using (copies of) handwritten signatures, which are sensitive to fraud. Though secure, cryptography-based signature solutions exist, they are hardly used due to usability issues. This paper proposes to use digital identity wallets for securely and intuitively signing digital documents with verified personal data. Using expert feedback, we implemented this vision in an interactive prototype. The prototype was assessed in a moderated usability test (N = 15) and a subsequent unmoderated remote usability test (N = 99). While participants generally expressed satisfaction with the system, they also misunderstood how to interpret the signature information displayed by the prototype. Specifically, signed documents were also trusted when the document was signed with irrelevant personal data of the signer. We conclude that such unwarranted trust forms a threat to usable digital signatures and requires attention by the usable security community.
1 INTRODUCTION
It is increasingly common for documents to be signed on computers [57]. Signing on a computer often involves scanning or drawing a digital version of a physical signature or even inserting a typed-out name (in a handwriting or script font), as offered in the popular signing tool Adobe Acrobat Reader [2]. These kinds of signatures are referred to as “electronic signatures” [55]. Signing documents this way is similar to physically signing documents and, therefore, intuitive for users [38, 43]. However, electronic signatures cannot reliably confirm the signer’s identity or commitment [19], regardless of their form. Namely, they can be easily copied, leading to potential misuse and fraud. Governments and businesses, therefore, regularly continue using physical signatures to be certain about the identity of the signatory [16] or employ proprietary solutions such as Adobe Acrobat Sign [52] or DocuSign [18]. These digital signing solutions are generally closed-source, require data to be uploaded to external servers, and, above all, are costly. While their costs might be bearable for organizations, it is unreasonable to expect individuals to bear these costs. We are thus in need of secure, free, usable and trustworthy tools for digitally signing documents that can be used by organizations and individuals alike. This paper sets out to develop such a signature tool and presents a first interactive prototype and its evaluation.
Cryptography offers secure and reliable digital signatures [4, 39, 40]. These are rooted in public-key cryptography [15], and provide guarantees over the signer’s identity (“source authenticity’’) and warrant that a message has not been changed (“message integrity’’).
This cryptographic approach is already implemented in many tools for digitally signing email, and part of, e.g., PGP [20]. However, in the context of signing (and encrypting) emails, the HCI community has identified many usability and adoption problems with such solutions [23, 23, 33, 48, 49, 59], in particular issues related to usable key management [49]. It is extremely cumbersome to obtain and store cryptographic keys [33, 48], to verify the identity of key owners [23, 58], and to have your own identity verified [23]. Furthermore, users seem to have a poor understanding of general cryptographic concepts (see, e.g., [1, 61]), and many are unable to explain what digital signatures are [1, 60]. The lack of understanding of digital signatures may hinder the adoption of digital signature tools [1], and even lead to security issues [31].
Still, if the usability issues around key management are solved, cryptography-based signature applications could potentially provide secure, trustworthy, and usable signatures. A promising emerging solution to key management is digital identity wallets. Identity wallets are applications that contain personal data of a user — e.g., name, date of birth, or diplomas, verified by a trusted issuing party, such as a governmental organization [45]. Users can then disclose these credentials to requesting parties to prove properties about themselves, i.e., who (e.g., a verified name) or what (e.g., a verified diploma) they are [25]. Digital identity wallets will become widely available in Europe soon [12], with current pilots investigating their use as a mobile driver’s license, to open a bank account, or to collect prescriptions at pharmacies [13]. Identity wallets can be used to replace complex cryptographic key management with the more familiar process of authenticating oneself [7]: someone could sign a document with their name by proving who they are with the identity wallet. This process is likely more intuitive for both the signatory and the recipient. Namely, signatories interact with a QR code, a well-known interaction [see, e.g., 44] whereas recipients can recognize signatories by their credentials rather than cryptographic keys [see, e.g., 56].
Given the potential synergy between identity wallets and digital signatures, this paper explores how identity wallets can be used to provide a secure, usable, and trustworthy solution for digital signatures on documents and to what extent users understand the solution and the signatures they create. While digital identity wallets are becoming a popular topic among usability researchers [7, 29, 30], wallet-based signing has been explored only with a focus on cryptographic architecture [25] and as a potential means for battling fake news [28], leaving out usability aspects. In fact, digital document signatures, in general, have received very little attention from the HCI community so far. Exceptions are the work by Hölbl et al. [27], who investigated various digital signature tools and their usability, and research by Fitriana et al. [21], who have prototyped and tested a digital signature application for document legalization. However, as these studies focus on different signing mechanisms (e.g., logging into a platform with a password to sign), the potential of digital identity wallets for signing still remains unexplored.
To investigate this potential, we created IdentitySign — an interactive prototype for digitally signing PDFs with the identity wallet Yivi. This wallet is a front-runner of the European Digital Identity Wallet and is already actively used in the Netherlands (with > 150k registrations). The prototype was developed using expert feedback, subsequently evaluated and improved using a moderated usability test, and finally assessed with a large-scale unmoderated remote usability test.
Our results show that although participants were satisfied with wallet-based signing, they generally misunderstood the role and meaning of digital signatures. Specifically, participants trusted signatures even when this was not warranted (e.g., because the signature was placed by the wrong entity). Another finding is that the role of the digital identity wallet in the signing process was also often unclear to participants. These findings contribute to the highly relevant domain of both digital signatures and digital identity wallet apps.
2 THE IDENTITYSIGN PROTOTYPE
Our prototype for digitally signing PDF documents is called IdentitySign and takes the form of a JavaScript/TypeScript web application that runs client-side within the user’s browser. The source code of IdentitySign is available in the supplemental materials, and screenshots of all features are available in Appendix A. The prototype works with the identity wallet Yivi. In the Netherlands, Yivi can be filled with verified data from official issuers (e.g., a municipality). These data can be used to sign documents. IdentitySign also supports signing with demo credentials, to allow testing from outside the Netherlands and to facilitate anonymous participation in our usability studies.
We developed IdentitySign iteratively, moving from low-fidelity mockups to an interactive website with a fully functional front-end. An early version of IdentitySign was improved with the help of feedback from four UX experts, who assessed the usability using the 10 usability heuristics by Nielsen [41], identified UI legibility and consistency issues and suggested improvements, such as providing more explanation to users. The final prototype features three core functionalities, namely, (1) signing documents, (2) verifying signatures, and (3) creating signature requests. Because we were interested in the prototype’s usability and users’ understanding of the signatures, we implemented only the front-end of the application. IdentitySign does not (yet) make use of cryptography to create actual digital signatures. (Actual cryptographic operations are not relevant to our specific research as they happen ‘under the hood’ rather than affect the user flow/interface.) From a user’s perspective, IdentitySign appears to be fully functional. While all functionalities are provided on our IdentitySign website, the entire process runs client-side, ensuring documents remain confidential and are not uploaded to some cloud service as with, e.g., DocuSign [18].
2.1 Signing documents
To create a signature, users have to select the document they wish to sign and the personal data they wish to sign with on the IdentitySign website (see Fig. 1, step 1). For instance, a user might select a file “document.pdf” and decide to sign with their name and email address. Subsequently, users need to scan a QR code with Yivi (2), which prompts them to share the selected Yivi credentials (containing name and email address) with the IdentitySign application, thus proving who they are (3). Subsequently, the disclosed credentials show up in IdentitySign, and the user can click “Sign” to sign the PDF (4). Upon doing so, the PDF is signed by the client-side software and can be found in the download folder (5). The signed PDF contains a footer at the bottom of the PDF that informs the recipient about the signature and about where to verify the signature (6).
Figure 1: User flow of signing a PDF with IdentitySign and Yivi. Screens have been cropped to display the main elements.
2.2 Verifying signatures
A signature can be verified by selecting a signed digital document within IdentitySign.1 If the document contains a signature, IdentitySign will then display a ‘valid signature’ screen, which contains the personal data that was used to sign the document (see Fig. 2 for the final implementation). If the document does not contain a signature, a message informs the user of this.
Figure 2: Verification screen with a valid signature (final implementation, cropped to show the main part of the screen).
2.3 Requesting signatures
Signatures can be requested by determining who should sign which document with which credentials. This involves selecting the file that should be signed and the attributes (e.g., name, address) that the signer should sign with. IdentitySign then generates a signature request link and an accompanying message, which can be copied with one click and subsequently be shared together with the file that needs to be signed, e.g., via email or text message. Opening such a link will restrict file selection to a file of the same name and will pre-select the personal data used for signing.
2.4 Limitations and envisioned implementation
As mentioned, we have implemented only the front-end and user flow of the application, i.e., no actual (cryptographic) signatures are created yet. However, the technical feasibility of using Yivi to sign files digitally is demonstrated by the PostGuard project [7], where users prove who they are with Yivi to encrypt/decrypt emails. Like PostGuard, IdentitySign can use identity-based encryption [5, 6, 51], and ask users to authenticate themselves to a trusted third party, i.e., an IdentitySign server, with the credentials they want to include in their signature to sign documents.
While we have implemented the flow with Yivi, IdentitySign could equally work with different (future) identity wallets as long as they allow users to authenticate themselves via selective disclosure of attributes (e.g., name, address).
3 EVALUATION OF IDENTITYSIGN
We evaluated IdentitySign’s usability and users’ trust in the provided digital signatures with two (IRB-approved) complementary, subsequent studies. Study 1 (N = 15) was a moderated usability test and took place in person, whereas study 2 (N = 99) was unmoderated and carried out remotely about a month later. Both studies gathered qualitative and quantitative data. However, study 1 was more exploratory, focused more on qualitative aspects, used the ‘think-aloud’ method [36, 42], and allowed us to observe participants and ask clarifying questions. In contrast, study 2 focused more on obtaining quantitative data and potentially generalizable results. We kept methods and materials constant across studies as much as possible. Any differences are listed below.
3.1 Participants
To be eligible for participation in our studies, participants had to be at least 18 years of age, able to use a computer and smartphone, able to speak and read English, and they needed to be based in the European Union. Experience with digital identity wallets was not required.
For study 1, the moderated usability test, we recruited 15 participants from our own network. Of the 15 participants, 9 were between 18–35 years old (60%), 3 between the age of 36 and 55 (20%), and 3 over 55 years old (20%). Most participants either completed or were currently attending higher education (66.7%, N =10), followed by vocational education (20%, N =3) and primary or secondary education (13.3%, N =2). Two of the participants (13.3%) either studied or were employed in a field related to IT.
We recruited 100 participants via Prolific for study 2, the large-scale unmoderated usability test. For the analysis, we excluded one participant who was unable to finish due to technical difficulties. Participants of the second study were slightly younger overall, with most being in the 18-35 age group (77.8%, N =77), followed by the 36-55 (17.2%, N =17) and the >55 (5.1%, N =5) age group. Most participants either completed or were currently attending higher education (76.8%, N =76), followed by primary or secondary education (17.2%, N =17) and vocational education (5.1%, N =5). Field of study/employment was not recorded for this study.
3.2 Procedure
After an informed consent procedure, participants completed a pre-study questionnaire and received pre-task information about digital signatures, our prototype, and Yivi. Study 1 continued with a short demo of the identity wallet Yivi, upon which participants were handed a laptop running Firefox (for IdentitySign) and Thunderbird (for receiving/sending documents) and a mobile phone running Yivi [46] (with a demo credential with the name Robin Stevens inside the wallet). In study 2, participants received instructions about installing Yivi on their phones and loading the demo credential into Yivi. Participants were then asked to complete a number of tasks (see below), each followed by a post-task questionnaire. In the end, participants completed a post-study questionnaire. Finally, participants were thanked and then either presented with a gift voucher (study 1) or returned to Prolific for payment (study 2).
3.3 Tasks and materials
In both studies, we asked participants to complete signature-related tasks. These tasks were presented in the form of short motivating scenarios. The exact materials used for both studies can be found in the supplemental materials.
3.3.1 Study 1.
The moderated usability test included four tasks (A1, B1, C1, D1). Tasks A1 was designed to evaluate the signing process. To complete this task successfully, participants had to download a PDF and sign the document with specific personal data. Task B1 meant to evaluate signature requests from a recipients/signers perspective. To complete it, participants had to open a signature request received via mail, sign the attached document, and return it to the requestor. Task C1 was created to evaluate signature requests from a requestors’ perspective. Participants needed to obtain a signature on a sales agreement by using the signature request feature. This entailed generating the request and sharing it by email, together with the document that needed to be signed. They then received the requested signed document, enabling us to see whether participants would verify the signature on the received document with IdentitySign. Task D1 was included to evaluate the signature verification process. Participants were told to verify the certificate (diploma) of an electrician and decide whether to hire the electrician. Notably, the signature on the electrician’s certificate was placed by the electrician themselves instead of the institution that awarded the certification (thereby not providing any guarantees of the legitimacy of the diploma). We thus wanted to find out whether participants would consider who signed the document and identify a meaningless signature as such (instead of trusting the document simply because of the presence of a signature).
As this first study primarily served an exploratory purpose, we made two slight adjustments to the test materials in between test sessions. Because initially, most participants did not trust the electrician’s certificate in task D1 due to the certificate’s simplistic and unrealistic design, we switched to a realistic-looking certificate. This change seems to have mitigated the issue. As users were not alarmed by the fact that the electrician’s certificate was signed by the electrician themselves, we furthermore included warnings to not blindly trust signatures in the prototype and added help information on when not to trust a signature. However, we did not observe a difference in the subsequent task results.
After the first study, we implemented the functionality needed for setting up the Yivi app with demo credentials. This was necessary as participants in study 2 would not have access to a smartphone with the Yivi application pre-configured.
3.3.2 Study 2.
The unmoderated remote usability test included two of the tasks from study 1, namely adapted versions of task A1 and D1, which we refer to as task A2 and D2. Accordingly, task A2 focused on signing a document and task D2 on the signature verification process. The tasks were changed slightly to fit an unmoderated remote study. For instance, in task A2, a success code was added to correctly signed PDFs. (This code needed to be reported back to us, so we could check if participants had completed the task successfully despite the remote and unmoderated setting.) The core task of downloading and signing a document (task A1/A2) and verifying a signature (task D1/D2) remained the same in both studies. (The exact scenarios of A1 and A2 differed, with participants signing a building permit in study 1 and a lease agreement for a storage box in study 2.)
3.4 Measures and data
Both studies collected quantitative and qualitative data on usability, trust, and general impressions.
We collected demographics, participants’ experience with signing digital documents and existing digital signature tools, and their knowledge and understanding of digital signatures. Whereas we included Affinity for Technology Interaction (ATI) scale [22] in study 1, it was excluded in study 2 for survey length reasons.
For each task, we measured task success and factors expected to affect perceived usability with the After-Scenario-Questionnaire (ASQ) [37]. Overall usability was measured with the System Usability Scale (SUS) [9]. For these quantitative measures, descriptive statistics were computed.
Our questionnaires also included open questions (e.g., to record whether participants liked/disliked anything in particular) and asked participants to rate several aspects, e.g., trust in IdentitySign’s digital signatures, via self-designed scales.
In study 1, participants were observed, and audio was recorded and later transcribed.
4 RESULTS
This section presents quantitative results and qualitative insights.
4.1 Quantitative results
Quantitative data was evaluated for each study separately. We first present success rates and usability perceptions (ASQ ratings) per task, followed by results concerning the overall usability of the prototype (SUS).
4.1.1 Task A.
Downloading and signing a document was completed with a success rate of 11/15 (study 1; A1) and 73/99 (study 2; A2). Task A1 received an ASQ score of 4.9 (SD = 1.30), and task A2 a score of 5.65 (SD = 1.46) on a scale of 1 to 7, where higher scores represent higher task usability. As reflected by these results, most participants found this task relatively easy and did not encounter any major issues. However, some participants had issues in recognizing whether the signing process was completed (most common in study 1, probably due to the absence of the success code introduced in study 2) or ran into issues with setup (most common in study 2).
4.1.2 Task B.
Responding to a signature request was only featured in study 1. The task (B1) was completed successfully by 14/15 participants. It received an ASQ score of 5.3 (SD = 0.93). Some participants noted that they perceived this second task as easier because they were more familiar with how IdentitySign worked after the first task.
4.1.3 Task C.
Requesting, obtaining, and verifying a signature was only featured in study 1. This task (C1) had a success rate of 9/15, and an ASQ score of 5.9 (SD = 1.09). Problems that caused task failure included not sending a document to sign, not utilizing the requested functionality, or not verifying the received document.
4.1.4 Task D.
Verifying the signature of a diploma was completed successfully by 3/15 (in study 1; D1) and 19/99 (in study 2; D2). The ASQ scores were 5.6 (SD = 1.27) for D1 and 6.08 (SD = 1.22) for D2. In other words, most participants were not able to successfully complete this task. We found two common reasons of task failure. First, some participants did not check the signatory. Second, some participants checked the signatory, but failed to question whether the assurances of that signatory were sufficient for the document in question. In neither study, participants explicitly expressed confusion about the task or scenario itself. However, observing confusion for participants was difficult in study 2 due to the remote setup.
4.1.5 Overall usability.
The overall prototype got a SUS score of 73.5 (SD = 17.39) in study 1 and 76.4 (SD = 16.3) in study 2. According to Bangor et al. [3], these SUS scores can both be interpreted as “good”.
4.2 Qualitative results
We conducted a thematic analysis [8] of the combined qualitative data of both studies. In our coding, we marked participants’ comments on our prototype, Yivi, digital signatures in general, and their current method of signing documents. Positive and negative remarks on each of these topics were assigned separate codes. From these codes, themes were identified, which were subsequently reviewed and refined. This section presents the resulting themes, followed by suggested improvements. Figure 3 provides a visual summary of the themes.
Figure 3: Visual representation of thematic analysis themes.
4.2.1 Prototype is usable.
Participants were generally very positive about IdentitySign’s overall usability. Positive reactions to the prototype were mostly related to participants describing the prototype as easy to use and fast. As one participant in study 2 stated: “All the information needed to use IdentitySign was available on the front page and the process was very easy to follow.” Similarly, positive reactions about Yivi were mostly about its ease of use. As stated by another participant in study 2: “Yivi has a great UI and seems very straightforward.”
4.2.2 Unwarranted trust.
While participants were able to complete most tasks with IdentitySign successfully, a big problem we observered was unwarranted trust. Participants commonly trusted the signatures made by the prototype, either because of trust in the prototype in general or trust in the information displayed with the signature. Participants’ trust was, e.g., based on the visible banner on the PDF, motivated by the presence of a signature, or caused by the green checkmark and/or signer’s name displayed in the verification process. This was most notable for tasks D1 and D2, where participants had to verify the certification of an electrician. Most participants incorrectly assumed the certificate’s validity — even though the certificate was signed by the electrician themselves rather than by the institution responsible for the certification. One participant in study 2 stated “I got a signature as a result, so the electrician was certified.”
4.2.3 Lack of understanding of the prototype.
Our qualitative data revealed that the IdentitySign prototype was not entirely clear. Misunderstandings mostly arose from signature validity not being clear or the information contained within the prototype itself not being sufficiently clear. As stated by a participant in study 2: “It says the signature was found, but I am not sure what that means.” Some participants also noted that the signature did not contain sufficient information for them to judge whether they could trust it.
4.2.4 Use of Yivi is unclear.
The role of Yivi in the signing process was also not always clear to participants, with some thinking that Yivi served as an additional layer of verification/authentication. Two participants in study 1 described the role of Yivi as “Yivi is the 2-step verification to confirm the identity of the person making the signature.” and “Possibly for double identity check.”. WhileYivi is indeed used to prove one’s identity, there is no second/double check of the user’s identity.
4.2.5 Benefits are not clear or convincing.
We identified roadblocks in the way of adoption. Specifically, the benefits of using digital signatures, IdentitySign, or both were either unclear or unconvincing to some participants. These participants were hesitant to change their methods and often preferred their current method of signing over the prototype. As one participant in study 1 states: “I’m a bit on the fence. The program is easy to use and, I assume, very reliable. But I don’t think I would choose this over Acrobat.”
4.2.6 Aversion to further digitalization.
On a more general level, we observed an aversion to further digitalization, which also could be an obstacle to adoption. Some participants were hesitant to trust either electronic systems in general or digital signatures specifically. One participant in study 2 notes: “Not sure I would want to use it in the real world though. Too much information is being stored digitally for my liking. Who would be able to see the information?”, illustrating their reluctance to store personal information digitally. These participants were worried about the trustworthiness of electronic systems and/or preferred to stick to traditional, handwritten signatures. Some also mentioned a dislike for having to use another device, either because participants did not want to switch devices or carry a second device.
4.2.7 Suggested improvements.
Multiple participants indicated their desire for a visual representation of a signature instead of the banner used by the current prototype. Two common reasonings behind this were 1) the banner not being recognizable as a signature, and 2) the banner did not evoke the level of trust a signature would.
5 DISCUSSION
We have presented a prototype for digitally signing documents with an identity wallet. Our main goal was to create an easy-to-use, efficient method to sign digital documents securely. Our prototype was largely well-received, and our results suggest that this is achievable with an identity wallet. In fact, for many users, the ease of use and speed with which they could sign and verify documents were reasons to be willing to use IdentitySign in the future. Specifically, the signing-focused tasks (A1, A2 and B1) were successfully completed by most participants. However, our evaluation revealed serious issues, especially with signature verification (D1 and D2), which require further study and might also play a role in other signature apps.
The biggest issue we observed is that users trusted the signatures even in situations where this was not warranted, e.g., when a document was signed with irrelevant personal data. This unwarranted trust is likely rooted in users’ habit of not scrutinizing the legitimacy of a signature until the assumption of honest communication is lifted [34, 35]. If digital signatures can be used to mislead users, they are not secure in practice [14].
Participants seemed to over-trust signatures because of checkmarks and green text outputted by the prototype, i.e., automation bias [53], or because of the presence of a signature banner. Interestingly, a similar “inadequate reliance on visual cues as a proxy for proper digital verification” [24] was also observed by Gerhardt et al. [24] in the related context of visual digital certificates. Avoiding potentially misleading visual elements such as checkmarks and words such as “valid” could help, as users tended to interpret these as “trustworthy” [see, e.g., 26]. However, as Gerhardt et al. [24] point out, using visual cues to determine authenticity is common with analog documents. Hence, adapting users’ verification behavior might require additional effort [24]. As a next step, we plan a re-design that nudges users to be more conscientious when deciding whether to trust a signed document without harming overall usability by, for example, incorporating “security enhancing friction” [17].
Multiple participants indicated that they missed a visual representation of a handwritten signature. The absence thereof might negatively affect the perceived validity and trustworthiness of signatures made by IdentitySign [see 11]. Hence, future work could explore the effects of adding a visual representation of a signature, e.g., by displaying the attributes used for signing in a calligraphic manner. However, we worry that such visual additions might cause users to inadequately rely on these visual cues [24]. A key question for follow-up work is how to design the signature banner so it evokes associations with a signature but also prompts users to inspect and verify the actual (digital) signature.
This research is one of the first to look into use cases of digital identity wallets. Overall, Yivi and the interaction of IdentitySign with Yivi were well-regarded by participants. Those who disliked the use of Yivi mainly raised complaints about the need to use multiple devices and the reliance on QR codes. Still, this was also explicitly mentioned as a positive property by some. User experience spanning multiple devices comes with challenges that have been previously recognized and studied [10]. QR code alternatives, such as push notifications, could be explored in further studies.
Given the novelty of identity wallets, it is not surprising that the role of Yivi in the signing process was not well understood, even though users generally used it correctly. The introduction of a European Digital Identity might help overcome such problems and lower the initial adoption effort. Namely, if users already have an identity wallet installed, it no longer becomes necessary to install an additional mobile application specifically for signing documents.
Our concept assumes that the identity wallets themselves will provide users with a good user experience, something that has not been widely studied yet [30, 50]. More research into the usability of identity wallets is thus also desirable.
There are some limitations to our results. First, participants in study 1 might have reacted more positively as the moderator was also the developer of IdentitySign, which is undesirable [32, 47]. However, this effect seems to be marginal, given that study 2 produced similar results. Second, our design process has involved users rather late, possibly affecting our design solution space. In hindsight, users could have been involved earlier through, e.g., a participatory design [54] session. Third, as (some forms of) signatures hold legal value, it is desirable to include a legal perspective in our design process, too. Fourth, using Yivi demo credentials might have influenced our results. Namely, filling the Yivi app with one’s own real credentials can take slightly more effort. Still, it is reasonable to assume that users will have a filled identity wallet readily at hand in the near future. Fifth and last, it is unclear how our results generalize to future EU digital identity wallets, as their user experience might not necessarily be similar to Yivi.
6 CONCLUSION
Our prototype introduces a novel use case of digital identity wallets to create digital signatures. Based on our evaluation, we conclude that the concept of digital signatures using identity wallets is promising but that the current prototype should not be published. Arguably the most important issue is the unwarranted trust that the mere presence of a signature can induce. This poses the threat of digital signatures being used to legitimize fraudulent documents and has implications beyond merely the effectiveness and usability of this specific prototype.
We believe a solution to this problem exists, e.g., by stimulating users to consider who has signed the document and whether the assurances of that individual provide sufficient trust for the document in question. In this regard, a better balance must be struck between providing an easy and fast experience and promoting mindful interactions [17] with signatures. We will continue exploring solutions to this problem, and we look forward to the HCI community joining us in these endeavors.
ACKNOWLEDGMENTS
We thank the experts who reviewed the IdentitySign prototype in the early development phase: Leon Botros, Merel Brandon, Marianna De Sa Siqueira, Arnout Terpstra, and Lian Vervoort. Next, we thank the participants of our usability studies. Furthermore, we thank the CHI’24 reviewers for their thoughtful remarks and Bart Jacobs for his valuable feedback and suggestions.
A PROTOTYPE SCREENSHOTS
Figure 4: The prototype’s home (1) and signing screens (2,3).
Figure 5: The prototype’s signing (1), verifying (2), and requesting (3) screens.
Figure 6: The prototype’s requesting (1), about (2), and help (3) screens.
Footnotes
1 Signing happens digitally, and IdentitySign’s signatures on printed documents cannot be verified.
Footnote
Supplemental Material
Available for Download
This zip archive contains the source code for the prototype application. It is licensed under the GNU GPL version 3 and may be used within the boundaries of that license.
This folder contains full-size screenshots of the prototype (also used in the publication itself).
This folder contains the materials used for our two user studies. Files specific to one study are sorted in either the "Study_1" or "Study_2" folder. The materials for each task can be found in the "Tasks" folder.
- Ruba Abu-Salma, M. Angela Sasse, Joseph Bonneau, Anastasia Danilova, Alena Naiakshina, and Matthew Smith. 2017. Obstacles to the Adoption of Secure Communication Tools. In IEEE Symposium on Security and Privacy. IEEE Computer Society, 137–153.Google Scholar
Cross Ref
- Adobe. 2023. PDF Reader - Adobe Acrobat Reader. https://www.adobe.com/acrobat/pdf-reader.htmlGoogle Scholar
- Aaron Bangor, Philip T. Kortum, and James T. Miller. 2008. An Empirical Evaluation of the System Usability Scale. Int. J. Hum. Comput. Interact. 24, 6 (2008), 574–594.Google ScholarCross Ref
- Mihir Bellare and Sara K. Miner. 1999. A Forward-Secure Digital Signature Scheme. In CRYPTO(Lecture Notes in Computer Science, Vol. 1666). Springer, 431–448.Google Scholar
- Dan Boneh and Xavier Boyen. 2004. Efficient Selective-ID Secure Identity-Based Encryption Without Random Oracles. In EUROCRYPT(Lecture Notes in Computer Science, Vol. 3027). Springer, 223–238.Google Scholar
- Dan Boneh and Matthew K. Franklin. 2001. Identity-Based Encryption from the Weil Pairing. In CRYPTO(Lecture Notes in Computer Science, Vol. 2139). Springer, 213–229.Google Scholar
- Leon Botros, Merel Brandon, Bart Jacobs, Daniel Ostkamp, Hanna Kathrin Schraffenberger, and Marloes Venema. 2023. PostGuard: Towards Easy and Secure Email Communication. In CHI Extended Abstracts. ACM, 232:1–232:6.Google Scholar
- Virginia Braun and Victoria Clarke. 2012. Thematic analysis. American Psychological Association, Washington, DC, US. 57–71 pages.Google Scholar
- John Brooke 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4–7.Google Scholar
- Frederik Brudy, Christian Holz, Roman Rädle, Chi-Jui Wu, Steven Houben, Clemens Nylandsted Klokmose, and Nicolai Marquardt. 2019. Cross-Device Taxonomy: Survey, Opportunities and Challenges of Interactions Spanning Across Multiple Devices. In CHI. ACM, 562.Google Scholar
- Eileen Y. Chou. 2015. Paperless and Soulless: E-signatures Diminish the Signer’s Presence and Decrease Acceptance. Social Psychological and Personality Science 6, 3 (2015), 343–351.Google ScholarCross Ref
- European Commission. 2021. European Digital Identity. https://commission.europa.eu/strategy-and-policy/priorities-2019-2024/europe-fit-digital-age/european-digital-identity_en [Online; accessed 10-June-2023].Google Scholar
- European Commission. 2023. European Digital Identity Wallet Pilot implementation. https://digital-strategy.ec.europa.eu/en/policies/eudi-wallet-implementation [Online; accessed 14 September 2023].Google Scholar
- Lorrie Faith Cranor and Simson Garfinkel. 2005. Security and usability: designing secure systems that people can use. O’Reilly Media, Inc.Google Scholar
- Whitfield Diffie and Martin E. Hellman. 2022. New Directions in Cryptography. 42 (2022), 365–390.Google Scholar
- Sander Dijkhuis, Remco van Wijk, Hidde Dorhout, and Nitesh Bharosa. 2018. When Willeke can get rid of paperwork: a lean infrastructure for qualified information exchange based on trusted identities. In DG.O. ACM, 89:1–89:10.Google Scholar
- Verena Distler, Gabriele Lenzini, Carine Lallemand, and Vincent Koenig. 2020. The Framework of Security-Enhancing Friction: How UX Can Help Users Behave More Securely. In NSPW. ACM, 45–58.Google Scholar
- DocuSign. 2023. Electronic Signature and Contract Lifecycle Management. https://www.docusign.com/Google Scholar
- European Commission. [n. d.]. What is eSignature. https://ec.europa.eu/digital-building-blocks/wikis/display/DIGITAL/What+is+eSignatureGoogle Scholar
- Hal Finney, Lutz Donnerhacke, Jon Callas, Rodney L. Thayer, and Daphne Shaw. 2007. OpenPGP Message Format. RFC 4880. https://www.rfc-editor.org/info/rfc4880Google Scholar
- Gita Fadila Fitriana, Merlinda Wibowo, and Eko Aribowo. 2022. Design and Evaluation of Smart Digital Signature Application User Interface for Document Legalization in COVID 19 Pandemic. Scientific Journal of Informatics 9, 1 (2022), 63–72.Google ScholarCross Ref
- Thomas Franke, Christiane Attig, and Daniel Wessel. 2019. A Personal Resource for Technology Interaction: Development and Validation of the Affinity for Technology Interaction (ATI) Scale. Int. J. Hum. Comput. Interact. 35, 6 (2019), 456–467.Google ScholarCross Ref
- Simson L. Garfinkel and Robert C. Miller. 2005. Johnny 2: a user test of key continuity management with S/MIME and Outlook Express. In SOUPS(ACM International Conference Proceeding Series, Vol. 93). ACM, 13–24.Google Scholar
- Dañiel Gerhardt, Alexander Ponticello, Adrian Dabrowski, and Katharina Krombholz. 2023. Investigating Verification Behavior and Perceptions of Visual Digital Certificates. In USENIX Security Symposium. USENIX Association, 3565–3582.Google Scholar
- Brinda Hampiholi, Gergely Alpár, Fabian van den Broek, and Bart Jacobs. 2015. Towards Practical Attribute-Based Signatures. In SPACE(Lecture Notes in Computer Science, Vol. 9354). Springer, 310–328.Google Scholar
- Brian Hilligoss and Soo Young Rieh. 2008. Developing a unifying framework of credibility assessment: Construct, heuristics, and interaction in context. Inf. Process. Manag. 44, 4 (2008), 1467–1484.Google ScholarDigital Library
- Marko Hölbl, Boštjan Kežmah, and Marko Kompara. 2023. eIDAS Interoperability and Cross-Border Compliance Issues. Mathematics 11, 2 (2023).Google Scholar
- Bart Jacobs. 2024. The authenticity crisis. Computer Law & Security Review: The International Journal of Technology Law and Practice (2024). Forthcoming.Google Scholar
- Alina Khayretdinova, Michael Kubach, Rachelle Sellung, and Heiko Roßnagel. 2022. Conducting a Usability Evaluation of Decentralized Identity Management Solutions. In Selbstbestimmung, Privatheit und Datenschutz. Springer Fachmedien Wiesbaden, 389–406.Google Scholar
- Maina Korir, Simon Parkin, and Paul Dunphy. 2022. An Empirical Study of a Decentralized Identity Wallet: Usability, Security, and Perspectives on User Control. In SOUPS @ USENIX Security Symposium. USENIX Association, 195–211.Google Scholar
- Gianluca Lax, Francesco Buccafurri, and Gianluca Caminiti. 2015. Digital Document Signing: Vulnerabilities and Solutions. Inf. Secur. J. A Glob. Perspect. 24, 1-3 (2015), 1–14.Google ScholarDigital Library
- Jonathan Lazar, Jinjuan Feng, and Harry Hochheiser. 2017. Research Methods in Human-Computer Interaction, 2nd Edition. Morgan Kaufmann.Google Scholar
- Ada Lerner, Eric Zeng, and Franziska Roesner. 2017. Confidante: Usable Encrypted Email: A Case Study with Lawyers and Journalists. In EuroS&P. IEEE, 385–400.Google Scholar
- Timothy R. Levine. 2014. Truth-Default Theory (TDT): A Theory of Human Deception and Deception Detection. Journal of Language and Social Psychology 33, 4 (2014), 378–392.Google ScholarCross Ref
- Timothy R. Levine. 2022. Truth-default theory and the psychology of lying and deception detection. Current Opinion in Psychology 47 (2022), 101380.Google ScholarCross Ref
- Clayton Lewis and John Rieman. 1993. Task-centered user interface design. A practical introduction (1993).Google Scholar
- James R. Lewis. 1995. IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. Int. J. Hum. Comput. Interact. 7, 1 (1995), 57–78.Google ScholarDigital Library
- Aaron Marcus. 1998. Metaphor design in user interfaces. ACM SIGDOC Asterisk Journal of Computer Documentation 22, 2 (1998), 43–57.Google ScholarDigital Library
- Ralph C. Merkle. 1987. A Digital Signature Based on a Conventional Encryption Function. In CRYPTO(Lecture Notes in Computer Science, Vol. 293). Springer, 369–378.Google Scholar
- Ralph C. Merkle. 1989. A Certified Digital Signature. In CRYPTO(Lecture Notes in Computer Science, Vol. 435). Springer, 218–238.Google Scholar
- Jakob Nielsen. [n. d.]. 10 Usability Heuristics for User Interface Design. https://www.nngroup.com/articles/ten-usability-heuristics/ [Online; accessed 13 September 2023].Google Scholar
- Jakob Nielsen. 1994. Usability engineering. Morgan Kaufmann.Google Scholar
- Donald A Norman. 1986. Cognitive engineering. User centered system design 31, 61 (1986), 2.Google ScholarCross Ref
- Elif Ozkaya, H Erkan Ozkaya, Juanita Roxas, Frank Bryant, and Debbora Whitson. 2015. Factors affecting consumer usage of QR codes. Journal of Direct, Data and Digital Marketing Practice 16 (2015), 209–224.Google ScholarCross Ref
- Blaz Podgorelec, Lukas Alber, and Thomas Zefferer. 2022. What is a (Digital) Identity Wallet? A Systematic Literature Review. In COMPSAC. IEEE, 809–818.Google Scholar
- Privacy by Design Foundation and SIDN. 2023. Yivi (formerly known as IRMA) [Mobile application]. https://yivi.appGoogle Scholar
- Jeffrey Rubin and Dana Chisnell. 2008. Handbook of usability testing: How to plan, design, and conduct effective tests. John Wiley & Sons.Google Scholar
- Scott Ruoti, Jeff Andersen, Daniel Zappala, and Kent E. Seamons. 2015. Why Johnny Still, Still Can’t Encrypt: Evaluating the Usability of a Modern PGP Client. CoRR abs/1510.08555 (2015).Google Scholar
- Scott Ruoti and Kent E. Seamons. 2019. Johnny’s Journey Toward Usable Secure Email. IEEE Secur. Priv. 17, 6 (2019), 72–76.Google ScholarDigital Library
- Rachelle Sellung and Michael Kubach. 2023. Research on User Experience for Digital Identity Wallets: State-of-the-Art and Recommendations. In Open Identity Summit. LNI, Vol. P-335. Gesellschaft für Informatik e.V.Google Scholar
- Adi Shamir. 1984. Identity-Based Cryptosystems and Signature Schemes. In CRYPTO(Lecture Notes in Computer Science, Vol. 196). Springer, 47–53.Google Scholar
- Adobe Sign. 2022. e-Sign software: Electronic & digital signatures. https://www.adobe.com/sign.htmlGoogle Scholar
- Linda J. Skitka, Kathleen L. Mosier, and Mark D. Burdick. 1999. Does automation bias decision-making?Int. J. Hum. Comput. Stud. 51, 5 (1999), 991–1006.Google ScholarDigital Library
- Clay Spinuzzi. 2005. The methodology of participatory design. Technical communication 52, 2 (2005), 163–174.Google Scholar
- The European Parliament and The Council Of The European Union. 2014. Regulation (EU) No 910/2014 of the European Parliament and of the Council of 23 July 2014 on electronic identification and trust services for electronic transactions in the internal market and repealing Directive 1999/93/EC. https://eur-lex.europa.eu/eli/reg/2014/910/oj [Online; accessed 27-March-2024].Google Scholar
- Lorraine Tosi, Aurélien Bénel, and Karine Lan Hing Ting. 2015. Digital signature services for users, Improving user experience to support trust among work partners. In 11th Symposium on Usable Privacy and Security (SOUPS).Google Scholar
- Kallie Tzelios and Lisa A Williams. 2020. The Psychological Impact of Digital Signatures: A Multistudy Replication. Technology, Mind, and Behavior 1, 2 (11 2020).Google Scholar
- Alexander Ulrich, Ralph Holz, Peter Hauck, and Georg Carle. 2011. Investigating the OpenPGP Web of Trust. In ESORICS(Lecture Notes in Computer Science, Vol. 6879). Springer, 489–507.Google Scholar
- Alma Whitten and J. Doug Tygar. 1999. Why Johnny Can’t Encrypt: A Usability Evaluation of PGP 5.0. In USENIX Security Symposium. USENIX Association.Google Scholar
- Zarul Fitri Zaaba and Teo Keng Boon. 2015. Examination on usability issues of security warning dialogs. Age 18, 25 (2015), 26–35.Google Scholar
- Dimitrios Zissis, Dimitrios Lekkas, and Panayiotis Koutsabasis. 2011. Cryptographic Dysfunctionality-A Survey on User Perceptions of Digital Certificates. In ICGS3/e-Democracy(Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Vol. 99). Springer, 80–87.Google Scholar
Index Terms
- Digital Dotted Lines: Design and Evaluation of a Prototype for Digitally Signing Documents Using Identity Wallets
Recommendations
Delegation of signing rights using certificateless proxy signatures
A proxy signature scheme allows a proxy signer to sign messages on behalf of an original signer within a given context. It has lots of practical applications in distributed systems, grid computing, mobile agent applications, distributed shared object ...
Fair digital signing: The structural reliability of signed documents
The exchange of digitally signed data inherits all the problems related to the indeterminacy of human communication, which are further intensified by the legal implications of signing. One of the fundamental intrinsic weaknesses of digital signatures is ...
Identity-Based signcryption from identity-based cryptography
WISA'11: Proceedings of the 12th international conference on Information Security ApplicationsA signcryption scheme encrypts and signs data in a single operation which is more efficient than using an encryption scheme combined with a signature scheme. Identity-based cryptography (IBC) does not require users to pre-compute key pairs and obtain ...
Comments