Formal Specification and Verification of JDK’s Identity Hash Map Implementation

1 INTRODUCTION

Maps are versatile data structures and a common foundation for important algorithms, as they provide a simple modifiable association between two objects: the key and a value. A hash map realizes this association with a (constant time) hash function, which maps a key to a memory location in the managed memory space. Thus, the typical operations, i.e., lookup, update, and deletion of associations, achieve a constant run-time on average.

To optimize their performance, hash maps require complex memory layout and collision resolution strategies. The memory layout describes where and how associations are stored. The collision strategy handles the location resolution when the memory location is already occupied by a different key with the same hash. Furthermore, an implementation needs to decide when and how a restructuring of the memory layout is necessary to maintain the performance over time because the addition and removal of association leads to fragmentation. In this article, we present the specification and verification of the IdentityHashMap class of the Java SDK as it appears in the latest update of JDK7 and newer JDK versions (up to JDK17).¹ To our knowledge, this is the first case study, which formally verifies a real-world hash map implementation from a mainstream programming language library. In particular, it is part of the Java Collections Framework, which is one of the most widely used libraries. We formally specify the behavior of the implementation using the Java Modeling Language (JML). We show how we combined the results of two JML-based verification tools (JJBMC and KeY) to exploit their strengths and avoid the weaknesses. In detail, we firstly used JUnit tests with generated runtime assertion and bounded checks with JJBMC [2] to engineer and iteratively strengthen the required specifications and to gain confidence in them. Finally, we used KeY [1] to provide preciseness by the cost of performance and required user interaction. In the end, we were able to prove 15 methods of the class with KeY and achieved bounded proofs obtained by JJBMC for two additional methods (including the presumably most difficult method remove).

We discuss where during the different stages of the verification process, bounded verification can support the specifying person and can contribute to the results. Furthermore, we describe how various implementation choices of hash maps affect the verification performance with KeY. For this, we re-implemented commonly used hash map concepts in Java, specified them with JML, and compared the required runtime for automatic verification. We draw conclusions on the effects of data structure design decisions on the performance in KeY.

This article is an extended version of the conference article [8]. Main new contributions include a proof of the resize method in Section 4, we analyzed the remove method with JJBMC (Section 5.1), expanded the material on unit tests (Section 5.2), provided a more in-depth comparison to other hash map implementations and the challenges these bring in Section 7 and added the description of an overflow error in the capacity method of JDK7u80 (Section 6), which results in a performance problem.

Artifact. The case study with all artifacts is available at [9].

Related Work. The hash map/table data structure of a linked list has been studied mainly in terms of pseudocode of an idealized mathematical abstraction, see [21] for an Eiffel version and [22] for an OCaml version. Hiep et al. [12] and Knüppel et al. [16] investigate correctness of some other classes of the Collections framework using KeY, the latter mainly as a “stepping stone towards a case study for future research”. In [4], the authors specify and verify the Dual Pivot Quicksort algorithm (part of the default sorting implementation for primitive types) in Java. Software bounded model checking and CBMC/JBMC in particular have been used in several case studies; however, the verification target of those tools are normally rather low level/embedded code rather than complex data structures [6, 10, 19, 24].

2 PRELIMINARIES

The JML [18] is a behavioral interface specification language [11] for Java programs according to the of design-by-contract paradigm [20]. Listing 1 shows an excerpt of the specification for the hash map method get; the full specification is covered in detail in Section 4. JML annotations are enclosed in comments beginning with /*@ or //@. The listing contains a method contract (lines 5–10) covering the normal behavior case in which an exception must not be thrown. The requires and ensures clauses specify the pre- and post-condition, respectively; the framing condition is given in the assignable clause, which lists the heap locations modifiable by the method. The special keyword \nothing indicates that no existing heap location must be modified, but new objects may be allocated. \strictly_nothing specifies that the heap must not be modified at all. Multiple contracts for a method are separated with the keyword also. JML also supports class invariants (line 3), which need to be established before and after every method invocation. To conduct inductive proofs for loops, these can be annotated with loop specifications (lines 19–22). The loop invariants (maintaining) must hold when the loop is reached and after every iteration. In the example, the variable i is specified to remain in range between 0 and len and is always even. The loop variant expression (decreasing) computes to a natural number which must be strictly decreased in every loop iteration. The assignable clause specifies the heap locations all loop iterations are allowed to change. JML extends the Java expression language by first-order logic constructs like existential (\exists) and universal quantification (\forall). Also, the construct (\num_of int x; G; C) is relevant for the case study. It counts the number of values for x such that the guard G and the condition C are satisfied. For instance, (\num_of int i; 0<=i<a.length; a[i] != null) returns the number of non-null elements in array a. The identifier \result refers to the method’s return value in postconditions, and the expression \old(E) evaluates the expression E in the pre-state of the method invocation. JML ghost variables (line 17) behave like local Java variables during verification, but are not available at runtime and must therefore not have an impact on the effects and result of the method they are declared in. The special primitive JML value type \bigint refers to the mathematical integers \(\mathbb {Z}\). ² Finally, JML adds a few modifiers to the language like nullable which specifies that a field, method argument or return value may be null. Without the modifier, the value must not be null and, in the case of arrays, must not contain null values.

Listing 1.

View Figure

JML specifications can be used in different formal analyses, ranging from formal documentation, test case generation, runtime assertion checking to deductive verification. This article will focus on the deductive verification of JML-annotated programs using two tools implementing different deductive JML verification approaches: KeY and JJBMC.

KeY is a theorem prover for JML-annotated Java programs that supports automatic and interactive verification. KeY encodes method contracts as proof obligations in dynamic logic, a program logic similar to the weakest precondition calculus or Hoare logic. The programs inside the dynamic logic formulas are resolved by applying a series of inference rules, thus symbolically executing the code and hence producing the weakest preconditions in first order logic. Further inference rules are applied to discharge these resulting obligations. KeY possesses a powerful automatic strategy that can prove most obligations fully automatically. In case of more sophisticated heavyweight specifications (like the ones in the present hash map case study), the user can apply inference rules interactively to guide the proof.

The tool JJBMC [2] on the other hand combines modular deductive verification with bounded model checking. It translates JML specifications to Java statements using additional assumptions and assertions. The enriched Java sources are then submitted to the state-of-the-art Java bounded model checker JBMC [7]. In Section 5.1, we will report how the combination of bounded verification with modular concepts inside JJBMC helped engineering the specifications.

3 THE VERIFICATION SUBJECT: JDK’S IDENTITYHASHMAP

The IdentityHashMap is a hash table implementation of the interface java.util.Map of the Java Collections Framework. Figure 1 shows an overview of the class. Like any Map, it implements a modifiable mapping between keys and values, s. t. every key \(k_i\) is associated with exactly one value \(v_i\). In the IdentityHashMap, two keys \(k_1\) and \(k_2\) are considered equal if and only if \(k_1 = k_2\) (equality by reference, see Listing 1 line 26 and Listing 4 line 33). In contrast, the equality of keys in HashMap is defined by the equals method).

Fig. 1.

View Figure

The IdentityHashMap stores the key-value entries sequentially in a one-dimensional array (private field table). The class relies on the built-in function System.identityHashCode(o) which returns a hash code for the object o. The hash is the first candidate spot in table to look up the entry, or locating a free spot to store the entry.

When an entry \((k_1,v_1)\) is added (\(\mathtt {put}(k_1,v_1)\), cf. Listing 4), a hash \(h_1 \in \lbrace 0, 2, 4, \ldots , N-2\rbrace\) is calculated based on the hash of the key \(k_1\) and the length N of the table (line 28). The key \(k_1\) is then stored in table at the (even) index \(h_1\), and the value \(v_1\) is stored adjacently at (odd) index \(h_1+1\) (line 42). Item 1. in Figure 2 shows the case where an entry is added to an empty map. In case \(k_1\) was already present in the table, it would not be inserted a second time (this would break uniqueness), but its associated value would be overwritten with \(v_1\). While keys are unique, there is no guarantee that their hash values are. Collisions might occur: the calculated index in table can be already occupied by an entry with a different key. The new entry is then stored at the next free position in table (item 2. in Figure 2, where \((k_2,v_2)\) is stored at index 6, while its hash \(h_2\) is 4). If that index idx is taken as well, the next even index \(idx^{\prime }= idx+2 \mod {N}\) is calculated by nextKeyIndex and tried, until a free spot is found (item 3. in Figure 2). This ensures that there is no gap (empty slots) between the calculated index and the actual index of a key. This collision resolution strategy is called linear probing [17]. Section 7 discusses other strategies. IdentityHashMap supports using the null value as a key. To distinguish the null key from an empty slot in the table, a constant object reference, NULL_KEY, is used in place of null.

Fig. 2.

View Figure

The get(k) (Listing 1) method retrieves the value for a given key k. It searches the table with the same process that we described above for insertion: start at the hash of k (line 15+25) and move to the next key slot (two spots further, modulo N, see line 28) until k is found (line 26). The search also terminates when an empty element in the array is encountered: this means there is no entry with key k (line 27). To ensure termination, it is thus crucial that the array at all times contains at least one empty slot.

We do not discuss removing an entry (method remove) in detail, but only note that table needs to be rearranged as if the entry had never been added in the first place, so that remove introduces no gaps between the calculated and actual index of a key. For an example, see last items in Figure 2.

4 SPECIFICATION AND VERIFICATION OF THE IDENTITYHASHMAP

We now discuss the specification and verification of core parts of the IdentityHashMap. The full case study comprises several hundred lines of source code and specifications and over 1.4 million proof steps (Table 1 on page 13). An exhaustive exposition is therefore clearly not feasible. Instead, we focus on the core methods and highlight several of the main proof obligations and their proofs in this section.

Particularly with case studies of such a large size, it can be challenging, but is crucial, to make and keep the formal specifications manageable and understandable. Developers of the specification must quickly see which properties were formalized already and which remain to be fixed or added (if they turn out to be flawed during analysis). During the proof, one must understand the specifications sufficiently well to use them in proving the verification conditions. Clients of the IdentityHashMap should be able to use the class solely on the basis of the specifications (without looking at particular implementation details). To facilitate understandability, our specifications include comments in natural language that explain what the formal property expresses.

Listing 2.

View Figure

Some of the core properties maintained by the class invariant are, for example, that the table contains at least one empty spot (so that lookup methods terminate) (line 18 in Listing 2) and that all spots between the hash value and the actual index (including the wrap-around behavior as described in Section 3) in the table are occupied (lines 24 and 34 in Listing 2).

One could use the pure hash method from the Java code in the class invariant to refer to the hash of an object. But this can be inconvenient for the proof process: the hash method body must then be executed to derive that heap modifications do not alter hashes of existing objects (and that the result is deterministic, etc). We simplify this by introducing a new mathematical (deterministic) function dl_genHash that does not rely on the heap to refer to an object hash and adding a postcondition to hash that its return value is dl_genHash. Let us now discuss some of the main proof obligations that arise in the verification of this class.

Termination of get(..). Listing 3 shows the specification of the loop in the get method. This loop also appears (in slightly different forms) in many other core methods of the hash map: the three contains* methods, put, and remove. The main goal of this loop is to search for a given key. Because the loop guard is \(\mathit {true}\), the loop only terminates if a return statement is encountered (line 24 in Listing 1). Intuitively, if the given key is not present, the loop eventually hits the empty spot in the table, which the class invariant ensures to exist. If the key is present, eventually the condition item == k becomes \(\mathit {true}\).

Listing 3.

View Figure

We now prove termination formally, using the variant in the decreasing clause (line 21 in Listing 1). Suppose the loop invariant and the loop guard hold at the start of a loop iteration. If a return statement is hit in the iteration, then clearly the loop terminates promptly. Otherwise, we must show that the variant has decreased at the end of the iteration (with an updated value of i), but remains non-negative. The following cases (where i is the value at the start of the iteration) may be encountered in this order during the execution of the loop:

put(..) inner loop assignable clause. The assignable clause (Listing 3) is peculiar: the code (Listing 4, line 42) has an assignment to an array element (which is not dead code), yet the clause states that no locations are modified. This is due to the meaning of loop specifications: they must hold whenever the loop guard is checked. This; however, is not the case after leaving the loop by a return statement. Therefore, in our case the assignable clause does not have to hold for the loop iteration in which the return statement is reached, and this is the case whenever the assignment that modifies the table is reached.

This strong assignable clause is very useful to prove the remainder of the method: all facts true before the loop (this may include the class invariant) are still valid and can be exploited after the inner loop!

put(..) satisfies contract and preserves class inv. We distinguish three scenarios with respect to the put method and wrote a contract for each of them. A so-called exceptional contract for the case that the hash map is full (it has reached max capacity): in that case the map is not modified and an exception is thrown. Another contract for the case that the map already contains the given key: then the corresponding value is updated. And a contract for the case where the table does not contain the given key yet so that the new key/value pair must be added. We shall focus on the proof of this last contract and discuss the main reasoning to show formally that, assuming the class invariant and precondition hold initially, put preserves the class invariant and satisfies the postconditions of this contract. This is the proof obligation that must be proven at line 43.

Consider the postcondition on line 10 of Listing 4, about the preservation of old entries. The table is modified at table[i] and table[i+1] which are null, as per the loop guard. So clearly, none of the entries that were already present are overwritten. In particular, in the case where the table is not resized, the old entries are at exactly the same index as at the beginning of the method. If the table was resized, the postcondition in the contract of resize (not shown) guarantees that they are present. The second main postcondition on line 18 of Listing 4 is easy to establish: it says that there exists an index in the new table at which the new entry is stored. At line 43 of Listing 4 we know that i is that index.

Listing 4.

View Figure

Next, we focus on two of the class invariants. The invariants that there are no gaps (key indices with a null) between the hash of any key and its actual index in the table (lines 24 and 34) are satisfied for the new entry: this follows from the invariant of the inner loop in put, Listing 3, lines 7 and 15. For old entries, these properties remain true, because the method only overwrites a null entry, so it does not introduce new gaps. Hence, if there previously was no gap between an old key’s hash and its index, then certainly there is not one after inserting the new key either.

Finally, we discuss the invariant that the map maintains at least one empty spot in table (line 18). The main challenge here is that table[i] was previously null (i.e., it was an empty spot) and is now overwritten with the key object, so is there guaranteed to be an empty spot elsewhere? Note that the capacity of the table, i.e., the number of entries that can be stored, is \(\texttt {len}/2\) since every entry (key and value) occupies two indices. If the old size is smaller than len/2 - 1, where len is the new length of the table, we can establish the desired property from the previous class invariant: as the size is the number of non-null entries (line 11) there must have been at least two empty spots. We now show that the old size is indeed smaller than \(\texttt {len} - 1\) whenever we reach the return-statement on line 43. The if-statement prior to it must then have been false (otherwise control jumps back to the beginning of the outer loop with the continue statement). Hence, one of the following two cases is true:

resize(..) outer loop preserves its invariant. When the table would become too full when adding a new entry, put calls the (private) resize method, which is responsible for allocating a larger table. This method takes care of several cases. For example, if the table length is already at the maximum capacity and so is the size (i.e., all slots except one are in use), it throws an IllegalStateException. If the table length is at the maximum capacity, but the size is not, resize does nothing.

We will discuss the most common (and most complex) case here, where the maximum capacity is not yet reached and resize actually allocates a larger table and adds all the existing entries to it. In that case, the length of the table increases, and since the hash index of entries is calculated modulo the table length, all hashes have to be recalculated based on the new length. This may shuffle around all entries. The outer loop, depicted in Listing 5, takes care of adding the entries one by one, to the new table, based on their new hash.

Listing 5.

View Figure

Next, we discuss how we proved that the outer loop in preserves its invariant. The loop iterates over all entries in the old table and stores the index of the current entry in the variable \(\mathtt {j}\). It is sometimes convenient to be able to use an invariant in proving another invariant. We, therefore, prove the invariants in the order below.

5 LEVERAGING THE BENEFITS OF LIGHTWEIGHT ANALYSIS FOR COMPLEX PROOFS

To achieve a full-fledged modular proof of a piece of software as complex as the IdentityHashMap, one has to use powerful tools, which are able to deal with all the challenges the software to be verified brings with it. However, the power of these tools oftentimes comes at the cost of increased manual work and a high level of required expertise. Most of the time in modular verification is spent on finding the appropriate specifications. In the following, we distinguish between two types of specification: Property specifications describe the exported guarantees one wants to verify, and auxiliary specifications (like class invariants, loop-invariants and contracts of helper methods) partition the verification condition into smaller obligations. Additionally auxiliary specifications can help tools to find proofs. In the present case study, both categories posed challenges.

During the search for appropriate specifications, we applied two different lightweight Java verification analyses that helped us find the correct specifications and allow us to achieve partial proofs: unit testing and bounded model checking. In the upcoming sections, we will provide more detailed descriptions and examples how we applied those techniques to the IdentityHashMap case study, but also how these techniques can in general assist experts that need to find modular specifications.

5.1 Software Bounded Model Checking (SBMC)

We use JJBMC [3] with which modular and bounded verification techniques can be combined: methods (and loops) with specifications are treated modularly (exploiting user-given method contracts and loop invariants to abstract from the program flow) while unspecified constructs can be formally treated using bounded verification (performing loop unrolling and method inlining to obtain a finite program to analyze), enabling a formal (albeit bounded) analysis of partially specified programs. The bounded analysis is parameterized by the maximum number \(k \in \mathbb {N}\) of unwindings and unrollings to apply. For too small a value of k, specification violations may hence remain undiscovered by a bounded analysis.

A bounded verification tool can contribute in three different modes of operation to the overall proof attempt: First and foremost, we used bounded verification to find, refine and double-check the (auxiliary) specifications required by KeY. The second operation mode is obtaining bounded verification results. When—due to time limitations or other considerations regarding effort—a full-fledged proof is not feasible, a bounded verifier may still be able to prove a specification up to a bound. The third operation mode is a special case of the second mode: in many projects, there are parts of the code which operate on bounded data structures with a pre-known upper bound. If this upper bound is within reach for a bounded analysis and does not exceed the required resources, then results may be equally or even easier achievable with bounded model checkers than with heavyweight analysis tools.

In the following sections, we will explain in more depth how we used the bounded verifier to our advantage in this case study and give some ideas on how this might translate to a general approach for future verification efforts.

5.1.1 Modified Version of IdentityHashMap for SBMC.

JJBMC encodes JML specifications into Java code and invokes the bounded model checker JBMC to obtain its results. Due to its design principles, JBMC allows object references to alias in fewer places than Java. We hence had to slightly adapt the original code of the IdentityHashMap to model the verification goal more accurately and to leverage the full potential of JJBMC. The largest change we made was to replace the type of stored elements in the map. Instead of Object we used a special class HashObject, a completely generic object type with one final field of type integer, which represents the constant system hash value that does not change in the course of the program. The second major change is that instead of checking for reference identity, equality between system hash values is used (so two HashObjects are considered identical if they have the same hash value) to mimic aliasing between objects. Additionally, a few minor adaptations to the specification were made to ease the verification for JJBMC. Although some of these changes do change the behavior, we argue that the bounded results obtained in this study still increase the level of confidence w.r.t. the original specification of the original code. This argument was reinforced by this case study, where we were able to find bugs in the modified version that were identical to the original one.

An example for the adaptations made can be found in Listing 6 where the SBMC-version of the remove method and its specification is shown. Consider line 45: Here, the condition whether or not the item currently considered is the one to be removed, was adapted as pointed out. Notice how in addition to the fact that hashes are compared instead of objects, an extra null check is required to avoid NullPointerExceptions. In the specifications, similar null checks are introduced to preserve well-definedness.

Listing 6.

View Figure

5.1.2 SBMC for Engineering Specifications.

Coming up with appropriate specifications (a valid property specification with corresponding auxiliary specifications) is a challenging task because the specifications usually depend on each other in two directions: In modular verification, it is not possible to prove a method contract containing a method call without a specification of the called method. On the other hand, the inner method is difficult to specify while it is not clear what guarantees are needed at its call sites. It is thus very desirable to reduce these interdependencies and to step back from the design-by-contract paradigm for the inner method call. We achieved this by using a bounded analysis to check partially specified programs.

The workflow to engineer specifications is as follows: The user annotates a top-level API method \(m_0\) with the desired property specification together with candidate class invariants (but leaves inner methods unspecified). He then runs JJBMC to get feedback whether this specification is correct (within the given bound). If it is not, a concrete counterexample trace is produced and presented to the user, who can use it for debugging. Once a suitable specification has been found, the user can continue engineering the specification for a method \(m_1\) called by \(m_0\). By continuously checking the bounded correctness of \(m_1\) and the modular correctness of \(m_0\) (wrt. the contract for \(m_1\)), the user hones in on an appropriate specification (strong enough for the call sites and weak enough to be provable) for \(m_1\). The process then continues with the next nested method call, and also applies to (nested) loops. Using the bounded model checking analysis, we gained confidence in the specifications and avoided a few tedious refactorings otherwise needed for the proofs of the unbounded case.

JJBMC relies on the parsing front end of the Java verifier OpenJML [5] and inherits its syntax and well-definedness checks (like not calling a non-pure method in a contract). While really rather simple checks, these features add to the ability to find errors in a specification fast and easily.

As one example where this process helped us in the case study, reconsider the specification of get in Listing 1. In the first specification attempt, the conditions in line 7 missed the call to maskNull, making code and specification inconsistent. Using JJBMC we were able to spot and correct this flaw early on before the inner mechanisms of get had been looked at. It took JJBMC about 18 seconds to find that bug on a standard desktop PC. For most other methods, JJBMC is able to show the correctness regarding their specification for a bound up to \(k = 8\) (k being the size of the table, so for up to four mappings) in similar amounts of time. More complex methods can take up to several minutes to verify (e.g., remove and closeDeletion, see next subsection) or even exceed the memory resources of the workstation used (e.g., resize and put). We used this approach to come up with several parts of the specification, and while we do not have hard evidence, our subjective impression is that it allowed us to get to correct specifications faster than we would have without it. We spent about 0.14 estimated person months to verify the IdentityHashMap with JJBMC.

5.1.3 SBMC for Bounded Proofs.

While a bounded proof provides fewer guarantees than an unbounded deductive proof, it still increases the confidence in the correctness of the analyzed program. In particular, it allows one to continue work with other proof obligations in a method-modular setting with a raised level of trust in the assumptions made within that proof. The proofs in the IdentityHashMap case study were time-consuming, far from trivial and required also tedious repetitive work. With the available resources, this led to the situation that we were not able to obtain a closed, full deductive KeY proof for the remove method, which is probably the computationally most complex routine of the class. However, on the way towards a closed unbounded KeY proof for the remove method, JJBMC provided us with a bounded proof for the specified method up to four elements in the map (i.e., eight elements in the array). The method is shown in Listing 6 and it mainly states that after the method call (a) the element to be removed is no longer in the table (lines 32–35), (b) all other entries remain untouched (omitted in the listing), and (c) the return value is the value associated to the removed key or null if the key was not present (lines 9–18).

The method closeDeletion plays a central part in the removal process, as it re-establishes the linear probing property. In particular, since removing an entry from the table will introduce a new empty slot, this may cause a gap (a null value) between the calculated (hash) index and the actual index of some existing key in the map, which would violate the central linear probing class invariant (cf. lines 24 and 34 in Listing 2). closeDeletion relocates entries from within the table to close these irregular holes. A challenge for the verification of the method is that said class invariant is temporarily violated, and that it is necessary to assume a weakened version of this invariant where this property holds everywhere except for the spot at which the removed element was before. For its postcondition, the closeDeletion method must re-establish the full class invariant.

For the bounded proof, we started by providing the toplevel specification of remove and verified it using loop-unwinding and inlining of the call to closeDeletion. JJBMC needed about 150 seconds to verify this contract for \(k=4\). We, then proceeded to provide the specification for closeDeletion and verified it, which took about 10.5 minutes. This specification is auxiliary and must be established at the call site in remove. We verified that said specified precondition holds there without using the modular verification mode of JJBMC by adding the condition as an additional assertion to the program in line 52 (this can be done automatically using JJBMC). Small mistakes in the condition could thus be repaired.

The obtained bounded proof for the specification gives us some guarantees, and one can argue that if a hash map works correctly for up to four entries, it is likely to work for other numbers of entries as well according to the small scope hypothesis [13]. Yet, the bounded proof provides fewer guarantees than a full proof. To obtain a proof for the unbounded case, the same contracts ideas can be used in KeY, but it still remains to be shown that the postcondition of closeDeletion is strong enough to prove the postcondition of remove. Furthermore, loop invariants are still missing as it is one undeniable benefit of bounded verification that these are not needed.

5.1.4 SBMC for Proofs of Bounded Programs.

In the last two applications, bounded verification can only deliver a partial proof (up to the bound k). However, there are scenarios in which a full proof can be achieved by means of a bounded model checker. The premise for that is that the analyzed method does not exceed the loop count or recursion depth of k, and that this bound k is small enough for the analysis to finish with the allotted time and space resources. Depending on the scenario, the largest reasonable and reachable bound varies a lot. As mentioned earlier, the analyzed bound for this case study was \(k=8\), however, we have observed examples where bounds of up to nearly 100 are realistic options. The existence of such a bound may seem unlikely for realistically complex code, but bear in mind that the benefit for modular verification is that every method is analyzed separately. It is not uncommon that an application makes use of data structures with a fixed upper bound on their size.

If a proof with a bounded model checker is possible as an alternative to a heavyweight deductive verification tool, this may have advantages. It is always fully automatic and does not require manual interaction, and auxiliary specifications need not be written. Depending on the application, bounded model checkers may be a lot faster than the auto-mode of deductive verification, or may suffer from the state space explosion problem and take a very long time.

If a bounded loop or any subroutine is part of the method, this may prove to be a huge advantage. Last but not least, some program features are especially suitable for bounded model checkers as it fits their strengths. For example, bit-wise operations on integers or heavily branching control flow are two examples of neuralgic points for deductive verification tools, which can oftentimes be handled by bounded model checkers.

The present case study contains a few methods without loops that can be verified fully automatically with JJBMC, but they did not provide a benefit in comparison to the equally simple KeY proofs. As an example for this kind of application of a bounded model checker, we refer to another study [14]. That paper shows an example of a sorting algorithm that relies on a sorting network for its base case to sort small arrays. This is a perfect match for a bounded model checker, which is able to provide an unbounded proof for the sorting network. The proof provided by the bounded model checker may then be used as an assumption for other proofs (notice how JML acts as an interface between different tools in this case).

5.2 Unit Tests for Property Specifications

5.2.1 Unit Test Tooling.

Dynamic techniques that check whether specifications hold at run-time could be cheap to apply, provided those checks are generated automatically from the JML specifications. There are tools designed for this purpose: JMLUnitNG [26] aims to generate unit tests, and OpenJML [5] is a general analysis framework that includes support for run-time assertion checking. However, our application of these tools to this case study was unsuccessful: the semantics of the source code and specifications proved to be too complex and intricate to load the IdentityHashMap. In particular, this triggered exceptions, and we did not manage to get useful output of the tools (despite contacting the main developer of OpenJML).

Confronted with this problem, we instead manually wrote (ad-hoc) JUnit tests to perform checks on method contracts (both pre- and post-conditions) and a test method for the class invariant that checks all clauses. We can then call the test method whenever the class invariant should hold. Since the class invariant accesses private fields such as table, we used Java Reflection (Class.getDeclaredField(..)) to read the values of these fields. We handled quantifiers in JML specifications with for-loops (all quantifiers are bounded over the integers in our case study, so they can be translated routinely to for-loops).

Conducting these tests helped us to gain confidence in our specifications and even uncovered some errors in early versions of it. However, there are two main limitations: first, since JUnit tests operate at the granularity of entire methods, auxiliary internal specifications such as invariants and assignable clauses of loops are difficult to cover. Second, the manual translation of the JML specifications could be inconsistent with the actual specification due to a misunderstanding of the semantics of JML. Finally, as we use unit tests to discover errors quickly, one should keep in mind that writing and maintaining the unit tests is very time-consuming. We spent about 0.5 person months to develop the unit test framework.

5.2.2 A Unit Test Example: Testing Class Invariants.

Because class invariants must hold before and after execution of every method, testing the class invariants is part of every method’s unit test. We have written a utility class ClassInvariantTestHelper with a method protected static void assertClassInvariants(AbstractMap<?, ?> map) testing the class invariants of the IdentityHashMap and its inner classes (see Listing 7). In general, the basic structure of the unit tests for any method is as follows:

(1)	instantiate and initialize an IdentityHashMap
(2)	test if the class invariants hold (by invoking the assertClassInvariants method)
(3)	test if the specific pre-conditions of the method hold (if applicable)
(4)	execute the method under test
(5)	test if the specific post-conditions of the method hold (if applicable)
(6)	test if the class invariants hold (by, again, invoking the assertClassInvariants method)

Listing 7 shows the assertClassInvariants method. This method invokes four methods to test the class invariants of the main class and its three inner classes, respectively. On line 4 the assertIdentityHashMapClassInvariant method is invoked to check the validity of the class invariant of the main class. For this example, it suffices to zoom in on this method and ignore the other three.

Listing 7.

View Figure

Listing 8 shows a fragment of the assertIdentityHashMapClassInvariant method. The main goal of this method is to check the class invariant. To that end, the value of three private attributes of the IdentityHashMap are retrieved: MINIMUM_CAPACITY, MAXIMUM_CAPACITY and table (see lines 3–5). The getValueByFieldName method is a custom method that uses Java Reflection to access private attributes.

Listing 8.

View Figure

On lines 9–25 and 27–46, two invariant clauses are tested. The comment lines show the JML specification, followed by a short explanation. The Java code following the comments is essentially a translation of the JML specification to plain Java. The universal quantifiers in JML specifications are translated to for-loops. Note that these clauses are the last two in Listing 2 in Section 4.

6 DISCOVERED BUGS AND RECOMMENDATIONS

In this section, we discuss several issues that our analysis revealed.

Serialization. The IdentityHashMap supports serialization: writing a map to a stream (e.g., a file) with a writeObject method and reading a map from a stream with the readObject method. Effectively, readObject acts as a constructor: it creates a map object, so it should ensure that this object satisfies the class invariant. To fill the map with serialized entries, readObject uses a putForCreate method that does not resize (for performance reasons) but allocates a table based on the size stored in the stream. Suppose an attacker serializes a map with a single empty entry (satisfying the requirement from the class invariant that there is an empty slot) to a file. The attacker can tamper with the file using a hex-editor to overwrite the empty slot with a key. A victim who deserializes this rogue map then inadvertently enters an infinite loop in putForCreate. We suggest solving this by checking in the code whether the map to deserialize satisfies the class invariant, and if not, throw an exception to prevent infinite loops or construction of a map object that breaks the class invariant.

Listing 9.

View Figure

put in JDK7u80. The binaries distributed by Oracle for JDK7 (an older but still widely used JDK) uses source code from an old JDK7u80 update.³ The main difference between JDK7u80 and the IdentityHashMap in this paper (which is used in all newer JDK’s and the later source-only updates to JDK7) is in the put method. The JDK7u80 version resizes after adding a new entry, rather than before (see Listing 9), and there is no outer loop with a continue statement. Suppose put is called on a map that is filled to the maximum capacity. The last empty spot in the table is first overwritten and only then resize throws an exception. So, the map is left in an inconsistent state: it breaks the class invariant. If a client then calls get(k) on a key k not stored in the map, an infinite loop is triggered. In other words: this version of put breaks failure atomicity: put fails (as the table is full) so the operation should have been a no-op.

There is a way to fix this without resorting to continue statements: extract the code for the inner loop in put, get, etc., which searches for the index of a given key, or returns the index of its insertion point if the key is not present in the map, into a new private method search(k). The duplicated code for the loop can then be eliminated from the various methods by calling search. In put, call resize before modifying the table. This may shuffle around the existing keys: the hashes are recalculated based on the new table length. If a resize occurred, call search(k) again to obtain the new insertion point for the key. Now the entries can be safely inserted at the index returned by search.

Overflow error in capacity in JDK7u80. According to the Javadoc specification, the capacity method of the IdentityHashMap (version JDK7u80, see Listing 10) always returns the smallest power of two between MINIMUM_CAPACITY and MAXIMUM_CAPACITY,⁴ inclusive, that is greater than (3 \(\times\) expectedMaxSize)/2, if such a number exists. Otherwise, it returns MAXIMUM_CAPACITY. On line 11 of Listing 10 the result of (3 \(\times\) expectedMaxSize)/2 is assigned to the variable minCapacity. If this value is negative, it is assumed that overflow has occurred, and MAXIMUM_CAPACITY will be returned (see lines 14–15). However, the assumption that overflow will always result in a negative value is false.

Listing 10.

View Figure

Suppose we pass a value of 1,431,655,765 to the method capacity. We would expect 536,870,912 (MAXIMUM_CAPACITY) to be returned. This is, however, not the case: multiplying the integer value of 1,431,655,765 by 3 results in -1 (overflow) and dividing an integer -1 by 2 results in 0, which is not a negative value. Line 15 is not executed, and the control flow continues with the statement on line 17. The smallest power of two between 4 (MINIMUM_CAPACITY) and 536,870,912 (MAXIMUM_CAPACITY) is 4. The method will, therefore, unexpectedly return 4, instead of 536,870,912. Any value for expectedMaxSize between 1,431,655,765 and 1,610,612,736 will trigger the error, and results in an erroneous output by the capacity method.

This overflow error results in the following performance bug. The IdentityHashMap provides a constructor with an expectedMaxSize input parameter, and a putAll method that copies entries from an existing map to the IdentityHashMap. Both use the capacity method to reserve (a large) space in the table array for future items. If, due to the overflow error, a small space is reserved, putting new items in the map requires a bigger table to store new items. This results in an unanticipated number of calls to the resize method. The resize method allocates memory for a bigger table object and, as a consequence, a new hash is calculated for the keys of all the items already present in the map, and the items are put in a different position in table. This repeatedly allocating memory and reshuffling of items is a costly operation. We found that, in an average test run, the elapsed time for a map with an initially (too) small capacity, showed a decline in performance of about 45%. Later versions of the JDK use a different calculation for the capacity, which does not suffer from the undetected overflow and associated performance issue.

7 EMPIRICAL IDENTIFICATION OF VERIFICATION CHALLENGES

To learn more about the particular challenges imposed by the verification of hash tables, we not only verified the IdentityHashMap, but also investigated the contributing factors for the complexity of hash table verification endeavors in KeY. The underlying research question is how the choice of the conflict resolution strategy impacts the effort needed for the deductive verification with KeY. To this end, we considered two families of hash table implementations with different collision resolution strategies. For each strategy, we implemented four variants that operate on different data types (and thus have different practical complexity) by adapting textbook reference implementations [25]. The Java code and its JML specification are part of the material accompanying this case study [9]. The implementations are unoptimised, straightforward Java-typical realisations of the respective variant to be sure that the origin of the proof complexity are the chosen features and not some unrelated optimisations. To make the results more comparable and not influenced by user input, we have run KeY fully automatically without any user interaction (apart from providing method specifications and loop invariants). The specifications have a similar degree of abstraction and follow similar lines as the ones outlined in Section 4. By comparing the required number of proof steps for the different implementations, we can draw conclusions about the complexity of the verification and the strengths and weaknesses of the automated proof strategy in KeY for the different conflict resolution strategies and data types.

The two compared hash collision resolution paradigms are linear probing and separate chaining. They differ in situations in which two different keys map to the same hash (index) into the hash map. Linear probing is used in the IdentityHashMap, as described in detail in Section 3. Separate chaining is used in the HashMap class in the JDK. It allows storing multiple entries into one slot: each slot contains a bucket (i.e., a linear list) with all entries that are mapped by the hash function to the same index (slot). Listing 11 shows the most central class invariants for the separate chaining approach. The collision resolution strategy affects the algorithms for insertion and lookup routines, since these have to take conflicting keys with identical indices into account. Hence, the specifications for the two paradigms differ considerably, and in particular the class invariants capturing the properties of the hash structure must express different things—with different challenges both for the specifier and the automatic verification engine.

The four variants implemented for each conflict resolution strategy were chosen to understand the impact of involved data types and operations on them on the verification effort, and on the then obtainable degree of automation. The variants thus mainly differ in the data types used for values and keys. In the first variant NN (for number to number), keys and values are both of type int and the identity operator (==) is used to compare keys. The second variant NO (for number to object) is similar to NN, but values are arbitrary Objects. The third variant IO (for keys with identity to objects) uses keys of a specialized immutable class Key, while values are arbitrary Objects, and it uses the identity operator (==) to compare keys, like the IdentityHashMap does. The fourth variant EO (equality on keys to object), finally, is similar to IO, but uses the equals method to compare keys. The class Key stores an immutable integer value that is used as the return value from its hashCode method and which is used to decide equality of Keys. This encoding is very similar to the adaptation of IdentityHashMap for bounded model checking as outlined in Section 5.1.1.

Listing 11.

View Figure

Table 2 shows the required effort to prove the respective method contracts correct (in terms of man hours, all variants with code and specs together took about 300 hours). The numbers of the NN variants represent the absolute number of rule applications needed, whereas the other three variants (in italics) are stated as a ratio to the number in the NN column for the same method and hashing family. Thus, the relative overhead between the family members can be seen more easily. The exact numbers of steps and running times were not the main focus of the investigation, suffice it to say that the NN proof for addNewPair with more than 380,000 steps took 12 minutes to complete. The hash method computes the hash value of a key, and getIndex returns the index of a key (if present). addNewPair inserts a new key-value pair and is called by put when the key is not already in the hash table. Some proofs could not be closed automatically within 60 minutes (indicated as \(--\)) with KeY then becoming rather irresponsive.⁵ Since the KeY solver chooses its rule applications deterministically, the runs are repeatable.

It can be observed that in most cases the complexity grows within a family between the variants from NN to NO from NO to IO and from IO to EO. The variants that introduce equals instead of == experience a vast increase in complexity for the central lookup method getIndex. This can be explained by the fact that the built-in identity comparison is independent of the heap state (it only depends on the compared values) and is inherently transitive and symmetric. As mentioned, this study requires instances of the special class Key (and not Object) as keys. This is a considerable simplification for the verification implementing the same effects: (1) the relation induced by equals is by construction an equivalence relation compatible with hashCode and (2) there are no (or hardly any) framing issues with the heap that require reasoning that the hash code does not change. Although the class Key tries to mimic the properties of integers with identity comparison, and simplifies specifications and proofs considerably compared to the case general Object keys, the increased effort for proving IO and especially EO is still immense, as can be seen in Table 2. The mutability of data structures on the Java heap and the potential effects write operations can have an impact on the evaluation of the equal method, make verification tasks significantly more difficult.

The research question whether separate chaining or linear probing can be verified more efficiently with KeY cannot be answered conclusively as the experiments show, as either conflict resolution strategy has its strengths and weaknesses. In general linear probing seems to require considerably fewer rule applications for the verification, but there is the method addNewPair whose complexity is an order of magnitude larger than for the other methods in linear probing. The reason why the effort is unevenly distributed and why this question is so difficult to answer lies buried in the fundamental properties of the data structures: With separate chaining, the map is an array containing the buckets and there is an additional array for every bucket. In Java, these arrays could, in general, alias and if aliasing arises, changes in one bucket might manifest in several buckets. The invariants for separate chainging (cf. Listing 11) ensure that the two-dimensional structure of the data structure is preserved. The maintenance of these framing conditions explains the generally high numbers in the effort for the first resolution strategy (especially in those methods modifying the heap). The case study throughout the article shows that linear probing also has (different) challenging class invariants, as can be seen in (e.g., the invariant “there is no null value between the index of a key, and its hash value”). Showing that they are kept up is particularly challenging for the method addNewPair that introduces one new element to the data structure.

Hence, separate chaining seems to have a higher general verification overhead, whereas linear probing seems to be more susceptible to individual localized verification challenges. Bearing in mind that this study relied solely on fully automatic (modular) verification with KeY, it is likely that these difficulties can be overcome when interaction like in Section 4 would be applied.

In both paradigms, it can be observed that switching from integer keys and values to objects keys and values removes the capability of fully automatic proofs. In the light of this insight, the verification of the more general implementation class HashMap from the JDK appears computationally considerably harder than the present case study on IdentityHashMap; especially having to deal with equals and hashCode implementations based on their general contract (rather than relying on a simple implementation like done in this section) will make verification a lot more challenging.

Contrary to what one might expect, some numbers decrease for the more complex variants. This can be explained by the heuristic choices that are made by the KeY strategy. In some cases, good decisions are made earlier than in other cases, due to the presence/absence of certain expressions. The increase in complexity between NO and IO is also a result of different heuristics been chosen for integers and for Object variables by the proof automation. It came as a surprise to us that the addNewPair method (regardless of the resolution strategy) could not be proved using full automation anymore if integer identity gets replaced by object identity.

8 CONCLUSION

In this article, we specified and verified the core of the challenging, real-world implementation IdentityHashMap in KeY and discovered several issues. To speed up finding suitable specifications, we successfully leveraged model checking and unit testing. We extended our analysis with an investigation on the effect on the proof complexity in KeY of features and strategies used in other map implementations.

While the bounded verification provided us with guarantees for the remove method of IdentityHashMap, an unbounded proof for this method (and its helper methods) remains a challenging open task for future work that would round the case study off. Another direction for future work is verifying JDK’s HashMap class. In contrast to IdentityHashMap, keys are compared with equals(..) rather than by reference and collisions are resolved by a form of chaining.