Two SAT solvers for solving quantified Boolean formulas with an arbitrary number of quantifier alternations

In recent years, expansion-based techniques have been shown to be very powerful in theory and practice for solving quantified Boolean formulas (QBF), the extension of propositional formulas with existential and universal quantifiers over Boolean variables. Such approaches partially expand one type of variable (either existential or universal) for obtaining a propositional abstraction of the QBF. If this formula is false, the truth value of the QBF is decided, otherwise further refinement steps are necessary. Classically, expansion-based solvers process the given formula quantifier-block wise and use one SAT solver per quantifier block. In this paper, we present a novel algorithm for expansion-based QBF solving that deals with the whole quantifier prefix at once. Hence recursive applications of the expansion principle are avoided and only two incremental SAT solvers are required. While our algorithm is naturally based on the ∀\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\forall $$\end{document}Exp+Res calculus that is the formal foundation of expansion-based solving, it is conceptually simpler than present recursive approaches. Experiments indicate that the performance of our simple approach is comparable with the state of the art of QBF solving, especially in combination with other solving techniques.

performance of our simple approach is comparable with the state of the art of QBF solving, especially in combination with other solving techniques.

Introduction
Efficient tools for deciding the satisfiability of Boolean formulas (SAT solvers) are the core technology in many verification and synthesis approaches [49]. However, verification and synthesis problems are often beyond the complexity class NP as captured by SAT, requiring more powerful formalisms like quantified Boolean formulas (QBFs) [7]. QBFs extend propositional formulas by universal and existential quantifiers over Boolean variables [34] resulting in a decision problem that is PSPACE-complete. Applications from verification and synthesis [10,15,16,20,22,26], realizability checking [21], bounded model checking [18,52], and planning [19,44] motivate the quest for efficient QBF solvers (see [45] for a survey).
Unlike for SAT, where conflict-driven clause learning (CDCL) is the single dominant solving approach for practical problems, two dominant approaches exist for QBF solving. On one hand, CDCL has been successfully extended to QCDCL that enables clause and cube learning [23,37,51]. On the other hand, variable expansion has become very popular. In short, expansion-based solvers eliminate one kind of variables by assigning them truth values and solve the resulting propositional formula with a SAT solver. For QBFs with one quantifier alternation (2QBF), a natural approach is to use two SAT solvers: one that deals with the existentially quantified variables and another one that deals with the universally quantified variables. For generalizing this SAT-based approach to QBFs with an arbitrary number of quantifier alternations, expansion is recursively applied per quantifier block, requiring multiple SAT solvers realizing a counter-example guided extraction approach (CEGAR) [17]. As noted by Rabe and Tentrup [42], these CEGAR-based approaches show poor performance for formulas with many quantifier alternations in general.
We also propose an approach that is guided by counter-examples, but that deals with quantifier alternations in a different manner than available CEGAR approaches. Inspired by Counterexample-Guided Inductive Synthesis (CEGIS), we present a novel solving algorithm based on non-recursive expansion for QBFs with arbitrary quantifier prefixes using only two SAT solvers. In short, CEGIS is a generic framework initially devised in the context of syntax-guided synthesis [1]. It involves the interaction between two components: • The learner generates candidate solutions that are consistent with all currently found counterexamples or-if it does not find such a candidate solution-it has shown that the problem does not have a solution, i.e., it is unsatisfiable. • The verifier provides, given a candidate solution to a problem, a counterexample that disproves it, or it correctly proves that the candidate solution is indeed a valid solution.
We adopt the CEGIS paradigm for QBF solving as follows. Our approach instantiates all variables of the same kind (either the universal variables or the existential variables) at once with a candidate solution/counterexample and passes the resulting propositional abstraction of the QBF to a SAT solver. If the SAT solver finds the formula to be unsatisfiable, the truth value of the original QBF is decided, otherwise the model returned by the SAT solver is used as candidate solution/counter example for refining the propositional abstraction. In theory (i.e., from a proof complexity perspective), our approach of non-recursive expansion is equivalent to approaches that apply recursive expansion since both non-recursive and recursive expansion rely on the ∀Exp+Res proof system [6]. However, the non-recursive expansion has practical implications such as a modified search strategy. That is, the use of recursive or non-recursive expansion results in different search strategies for the proof. With respect to proof search, there is an analogy to, e.g., implementations of resolution-based CDCL SAT solvers that employ different search heuristics. In addition to the new algorithm, we also implemented a hybrid approach that combines clause learning with non-recursive expansion-based solving for exploiting the power of QCDCL. Our experiments indicate that this hybrid approach performs very well, especially on formulas with multiple quantifier alternations. This paper is structured as follows. After a review of related work in the next section, we introduce the necessary preliminaries in Sect. 3. After a short recapitulation of expansion in Sect. 4, our novel non-recursive expansion-based algorithm is presented in Sect. 5. The relation between our solving approach and ∀Exp+Res is explained in Sect. 6. Implementation details are discussed in Sect. 7 together with a short discussion of the hybrid approach. In Sect. 8 we compare our approach to state-of-the-art solvers.
This paper is an extended version of [9]. Besides a careful revision of the text, it contains more examples and illustrations, as well as an additional chapter relating our new solving approach to the ∀Exp+Res proof system. Furthermore, we added comprehensive experiments on the benchmark set used in QBFEval 2018.

Related work
Already the early QBF solvers Qubos [3] and Quantor [3] incorporate selective quantifier expansion for eliminating one kind of quantification to reduce the given QBF to a propositional formula. Qubos heuristically chooses which kind of quantifier to eliminate. If universal quantifiers are eliminated, subformulas of the form ∀x.φ are replaced by Dually, subformulas of the form ∃x.φ are replaced by φ[x/ ] ∨ φ[x/⊥]. For handling the blow-up, Qubos implements several simplification techniques. Qubos does not require the input QBF to be in prenex conjunctive normal form (PCNF), but it is able to process formulas of arbitrary structure. Even more, the expansion of existential variables destroys any PCNF structure. Quantor, in contrast, preserves the PCNF structure by expanding universal variables only. In both cases, the resulting propositional formula is then solved by calling a SAT solver once. Over 15 years ago, Qubos and Quantor impressively demonstrated the power of expanding universal variables but also showed its enormous memory consumption. As a pragmatic compromise, bounded universal expansion was introduced for efficient preprocessing [13,24,25,50].
The first approach which uses two alternating SAT solvers A and B for solving 2QBF, i.e., QBFs of the form ∀U ∃E.φ, was presented in [43]. Solver A is initialized with φ, B with the empty formula. Both propositional formulas are incrementally refined with satisfying assignments found by the other solver. If A finds its formula unsatisfiable, then the QBF is false. Otherwise, the negation of the universal part of the satisfying assignment is passed to solver B. If solver B finds its formula unsatisfiable, then the QBF is true. Otherwise, the existential part of the satisfying assignment is passed to solver A. Janota and Marques-Silva generalized the idea of alternating SAT solvers [33] such that one solver deals with the existentially quantified variables and one solver deals with the universally quantified variables exclusively. Solver A gets instantiations of φ in which the universal variables are assigned, and solver B gets instantiations of ¬φ in which the existential variables are assigned. The satisfying assignment found by one solver is used to obtain a new instantiation for the other. This loop is repeated until one solver returns unsatisfiable. This approach realizes a natural application of the counter-example guided abstraction refinement (CEGAR) paradigm [17]. A detailed survey on 2QBF solving is given in [4].
A significant advancement of expansion-based solving for QBF with an arbitrary number of quantifier alternations was made with the solver RAReQS [28,29], which recursively applies the previously discussed 2QBF approach [33] for each quantifier alternation. The approach turned out to be highly competitive. 1 For formalizing this solving approach the calculus ∀Exp+Res was introduced [6], and proof-theoretical investigations revealed the orthogonal strength of ∀Exp+Res and Q-resolution [35], the QBF variant of the resolution calculus that forms the basis for QCDCL-based solvers. Research on the proof complexity of QBF has identified an exponential separation between Q-resolution and the ∀Exp+Res system. There are families of QBFs for which any Q-resolution proof has exponential size, in contrast to ∀Exp+Res proofs of polynomial size, and vice versa. Hence these two systems have orthogonal strength.
Recent work successfully combines machine learning with this CEGAR approach [27]. Motivated by the success of expansion-based QBF solving, several other approaches [12,32,42,[46][47][48] have been presented that are based on levelised SAT solving, i.e., one SAT solver is responsible for the variables of one quantifier block. In this paper, we also introduce a solving approach that is based upon propositional abstraction but considers the whole quantifier prefix at once.

Preliminaries
The QBFs considered in this paper are in prenex normal form Π.φ where Π is a quantifier prefix Q 1 x 1 Q 2 x 2 . . . Q n x n over the set of variables X = {x 1 , . . . , x n } with Q i ∈ {∀, ∃} and x i = x j for i = j. The propositional formula φ contains only variables from X . Unless stated otherwise, we do not make any assumptions on the structure of φ. Sometimes Π.φ is in prenex conjunctive normal form (PCNF), i.e., Π is a prefix as introduced before and φ is a conjunction of clauses. A clause is a disjunction of literals, and a literal is a variable or the negation of a variable. The prefix imposes the order < Π on the elements of X such that we denote the set of universally (existentially) quantified variables of the prefix Π. If clear from the context we omit the subscript Π. We assume the standard semantics of QBF. A QBF consisting of only the syntactic truth constant ⊥ ( ) is Given a set X of variables, we call a total function σ : X → { , ⊥, } an assignment for X . If there is an x ∈ X with σ (x) = then σ is a partial assignment, otherwise σ is a full assignment of X . Informally, σ (x) = means that σ does not assign a truth value to variable By Σ X we denote the set of all full assignments σ : X → { , ⊥}. Let φ be a propositional formula over X . By σ (φ) we denote the application of assignment σ : X → { , ⊥, } on φ, i.e., σ (φ) is the formula obtained by replacing variables x ∈ X by σ (x) if σ (x) ∈ { , ⊥} and performing standard propositional simplifications. Let φ, ψ be propositional formulas over the set of variables X . If for every full assignment σ ∈ Σ X , σ (φ) = σ (ψ) then φ and ψ are equivalent. Let τ : X → { , ⊥, } and σ : Y → { , ⊥, } be assignments such that for every x ∈ X ∩ Y , τ (x) = σ (x) if τ (x) = and σ (x) = . Then the composite assignment of σ and τ is denoted by σ τ : X ∪ Y → { , ⊥, } and for every propositional formula φ over X ∪ Y , it holds that σ τ (φ) = τ σ (φ) = σ (τ (φ)) = τ (σ (φ)). Furthermore, we use the equality σ σ = σ for any assignment σ .
, the application of σ and τ on φ gives us

Expansion
In the following, we introduce the notation and terminology used for describing expansionbased QBF solving in general, and the algorithm introduced in the next section in particular. We first define the notion of instantiation that is inspired by the axiom rule of the calculus ∀Exp+Res [31] which is introduced in Sect. 6.
The instantiation of φ by σ , denoted by φ σ , is obtained from φ as follows: 1. All variables x ∈ X with σ (x) = are set to σ (x); 2. All variables x ∈ X with σ (x) = are replaced by x ω where annotation ω is uniquely defined by the sequence σ (x k 1 )σ (x k 2 ) . . . σ (x k m ) such that the set formed from the variables x k i contains all variables of X with x k i < Π x and σ (x k i ) = . Furthermore, All truth constants occurring in the formula (not in the annotations) are eliminated by standard simplification rules.
If we instantiate a QBF Π.φ with the full assignment σ : U Π → { , ⊥} of the universal variables, we obtain a propositional formula that contains only (possibly annotated) variables from E Π . The dual holds for the instantiation by a full assignment σ : E Π → { , ⊥} of the existential variables.
Note that a is not annotated because it occurs in the first quantifier block.
Sometimes we want to remove the annotations from an assignment or an instantiated formula. Therefore, we introduce the following notation. Let φ σ be an instantiation by assignment σ : X → { , ⊥, } and X σ the set of annotated variables. If we have an assignment τ : If we have an instantiated formula φ σ , then (φ σ ) −σ is the formula obtained by replacing every annotated variable x σ ∈ X σ by x. In general, (φ σ ) −σ = φ. input : QBF Π.φ with universal variables U and existential variables E output: truth value of Π.φ if isUnsat then return false; if isUnsat then return true;

Fig. 1 Non-Recursive Expansion-Based Algorithm
Example 3 Reconsider the propositional formula φ and assignments σ, τ from Example 2 above.
Proof By induction over the formula structure. For the base case let The induction step naturally follows from the semantics of the logical connectives.
Finally, we specify the semantics of a QBF in terms of universal and existential expansion on which expansion-based QBF solving is founded.
The lemma above has a dual version for true QBFs. This duality plays a prominent role in our novel solving algorithm.

A non-recursive algorithm for expansion-based QBF solving
The pseudo-code in Fig. 1 summarizes the basic idea of our novel approach for solving the QBF Π.φ with universal variables U and existential variables E.
First, an arbitrary assignment α 0 for the universal variables is selected in Line 1. The instantiation φ α 0 is handed over to a SAT solver. If φ α 0 is unsatisfiable, then Π.φ is false and the algorithm returns. Otherwise, τ : E α 0 → { , ⊥} is a satisfying assignment of φ α 0 . Let σ 1 denote the assignment τ −α 0 . Then α 0 σ 1 is a satisfying assignment of φ.
In the next round of the algorithm, the propositional formula φ α 0 ∧ φ α 1 is handed over to a SAT solver. If this formula is unsatisfiable, Π.φ is false and the algorithm returns. Otherwise, it is satisfiable under some assignment τ : This assignment is then used for obtaining a new propositional formula φ σ 1 ∨φ σ 2 . To show the validity of this formula, its negation is passed to a SAT solver. If this formula is unsatisfiable, Π.φ is true and the algorithm returns. Otherwise, it is satisfiable under the assignment ρ : This assignment is then used in the next round of the algorithm. In this way, the propositional formulas α∈Σ U φ α and σ ∈Σ E φ σ are generated.
The algorithm iteratively extends the sets A and S by adding parts of satisfying assignments of φ to S and parts of falsifying assignments to A. In particular, A is extended by assignments of the universal variables and S is extended by assignments of the existential variables. The order in which assignments are considered depends on the used SAT solver.

Example 4
We show how to solve the QBF ∀a∃x∀b∃y.φ with E = {x, y}, U = {a, b}, and φ = ((a ∨ x ∨ y) ∧ (¬a ∨ ¬x ∨ y) ∧ (b ∨ ¬y)) with the algorithm presented above. This formula can be solved in two iterations: Init: We start with some random assignment α 0 : U → { , ⊥}, for example with α 0 (a) = and α 0 (b) = ⊥. Iteration 1: The formula φ α 0 = (¬x ∨ y ⊥ ) ∧¬y ⊥ is passed to a SAT solver and found satisfiable under the assignment τ : Based on this assignment we obtain φ σ 1 = a. The formula ¬φ σ 1 is passed to a SAT solver. It is satisfiable and has the satisfying assignment ρ : is passed to a SAT solver in the second iteration. It is satisfiable and one satisfying assignment is τ : and σ 2 (y) = ⊥. Note that for any choice of τ , σ 2 = σ 1 . Next, we construct φ σ 1 ∨ φ σ 2 = a ∨ ¬a. This formula is a tautology, so its negation that is passed to a SAT solver is unsatisfiable, hence Π.φ is true.
The soundness of our algorithm immediately follows from Lemmas 2 and 3 : the algorithm returns false (true) if, in some iteration i, it finds that the current partial expansion α∈A i−1 φ α (respectively σ ∈S i ¬φ σ ) is unsatisfiable. Fig. 1 is sound. For showing that the algorithm also terminates, we argue that sets A i and S i increase in iteration i + 1. To this end, we have to relate the variables of the QBF, the annotated variables as well as their assignments. Before we give the proof, we first consider another example in which we illustrate how the different assignments are related.

Example 5
We show one possible run of the algorithm presented above for the QBF Φ := ∀a∃x∀b∃y.φ with and how it iteratively generates the sets Σ U and Σ E . Figure 2 shows the expansion trees that are implicitly built during the search. An expansion tree relates the variables of the partial expansion of Φ constructed from A i (left column) and S i (right column). Solid edges indicate that the variable on the top has been set by an assignment from A i or S i , and dotted edges indicate that the variable has to be assigned a value by the SAT solver. The order of the (annotated) variables in the expansion tree respects the order of the (original) variables in the prefix.
In the example above we saw that new assignments are generated in each iteration because A i and S i build models and counter-models of φ. The following definition formalizes the relationship between A i and S i .

Definition 2 Let Π.φ be a QBF over universally quantified variables U and existentially quantified variables E. Further, let
If for every assignment σ ∈ S, there exists an assignment α ∈ A such that ασ (¬φ) is true, then we say that A completes S. If for every assignment α ∈ A, there exists an assignment σ ∈ S such that ασ (φ) is true, then we say that S completes A.
We now show that S i completes A i−1 and A i completes S i if the algorithm does not terminate in iteration i because of the unsatisfiability of the respective expansion. Fig. 1.

(1) If α∈A i−1 φ α is satisfiable, then S i completes A i−1 , i.e., for every μ ∈ A i−1 , there is an assignment ν ∈ S i such that μν(φ) is true. (2) If σ ∈S i ¬φ σ is satisfiable, then A i completes S i , i.e., for every ν ∈ S i , there is an assignment μ ∈ A i such that νμ(¬φ) is true.
Proof By contradiction. For (1), assume there is an assignment μ ∈ A i−1 such that there is no assignment ν ∈ S i with μν(φ) is true. By assumption α∈A i−1 φ α is satisfiable, so there is a satisfying assignment τ with τ (2), assume that there is an assignment μ ∈ S i such that there is no ν ∈ A i with μν(¬φ) is true. The rest of the argument is similar as in (1).
Next, we show that the addition of new assignments A to a set A of universal assignments forces a set S of existential assignments to increase if some completion criteria hold.

If S completes A, and A ∪ A completes S, and α∈A∪A φ α evaluates to true under assignment τ , then there exists an assignment
Proof By induction over the number of variables in Π. Base Case. Assume that Φ has only one variable, i.e., Π = Qx. Note that |A | = 1 because x is outermost in the prefix and A is obtained from sub-assignments of ρ. If Q = ∀, then the elements of A are full assignments of φ, and S is either empty, or it contains the empty assignment ω : ∅ → { , ⊥}. Let A = {μ}. If S is empty, so is A (because S has to complete A). If τ is a satisfying assignment of φ μ , then ν = τ = ω is the empty assignment and ν / ∈ S. Otherwise, ω ∈ S. If there is an assignment α ∈ A, then φ α ∧ φ μ is a full expansion of Φ. If this full expansion is true, then ¬φ is unsatisfiable. Otherwise, φ α ∧ φ μ is unsatisfiable. In both cases, the necessary preconditions for the lemma are not fulfilled. If A = ∅, then μω(¬φ) is true. Then φ μ is unsatisfiable, again violating a precondition. If Q = ∃, then μ = ω and A = ∅. If S = ∅ and φ ω = φ has the satisfying assignment τ , then ν = τ and ν / ∈ S. Otherwise, if there is an assignment σ ∈ S, then ωσ (¬φ) is true, because A ∪ {μ} = {ω} completes S. Hence, if assignment τ satisfies φ μ , then ν = τ , so ν / ∈ S. Induction Step. Assume the lemma holds for QBFs with n variables. We show that it also holds for QBFs with n + 1 variables. Let Φ = QxΠ.φ be a QBF over existential variables E and universal variables U with Π = Q 1 x 1 . . . Q n x n and A ∪ A and S be as required (S completes A, A∪ A completes S, α∈A∪A φ α has a satisfying assignment τ , and σ ∈S ¬φ σ has a satisfying assignment ρ from which A is obtained).
If Q = ∀, then all assignments α ∈ A assign the same value t to x, i.e., α(x) = t, because these assignments are extracted from assignment ρ and since x is the outermost variable of the prefix of Φ, ρ(x) = t. Further, let A t = {α ∈ A | α(x) = t}. It is easy to argue that for Π.φ[x ← t] together with the assignment sets A t ∪ A and S the induction hypothesis applies, i.e., there is an assignment ν / If Q = ∃, assume that τ (x) = t. Let {σ ∈ S | σ (x) = t} ⊆ S t ⊆ S, and let A t ⊆ A such that the induction hypothesis applies to Π.φ[x ← t], A t ∪ A , and S t . Let τ t be those subassignments of τ that satisfy α∈A t φ α . Then there is an assignment ν that can be extracted from τ t with ν / ∈ S t . Since ν(x) = t, ν / ∈ S. This concludes the proof.
This property also holds in the other direction, i.e., adding a set S of new assignments to S will force the set A to increase.
where -C is a clause from the matrix of QBF Π.φ τ is an assignment to all universal variables U Π -[τ ] restricts τ to the universal variables that precede l in the prefix Fig. 3 The rules of the ∀Exp+Res [6,30,31] Lemma 7 Let Φ = Π.φ be a QBF over universally quantified variables U and existentially quantified variables E. Further, let S ∪ S be a set of existential assignments such that S ∩ S = ∅, S = ∅, let A be a set of universal assignments, α∈A φ α has the satisfying If A completes S and S ∪ S completes A and σ ∈S∪S ¬φ σ evaluates to true under assignment ρ, then there exists an assignment Proof The proof is analogous to the proof of Lemma 6.
Now that we have identified the relations between the sets of universal and existential assignments, we use them to show that the algorithm from Fig. 1 terminates. Fig. 1 terminates for any QBF Φ = Π.φ.

Proof By induction over the number of iterations i, we argue that sets
Base Case. Let i = 1 and A 0 = {α 0 }. S 0 ⊂ S 1 , because S 0 = ∅ and σ 1 ∈ S 1 is a satisfying assignment of φ α 0 (if φ α 0 is unsatisfiable, the algorithm terminates). A 0 ⊂ A 1 directly follows from Lemma 4. Induction Step. For i > 1, we argue that S i ⊂ S i+1 . By induction hypothesis the theorem holds for iteration i, i.e., A i = A i−1 ∪ A with A i−1 ∩ A = ∅ and A = ∅ and S i = S i−1 ∪ S with S i−1 ∩S = ∅ and S = ∅. Because of Lemma 5, S i completes A i−1 , and A i completes S i . Furthermore, if σ ∈S i ¬φ σ is satisfiable under some assignment ρ (otherwise the algorithm would terminate), by construction A ⊆ {(ρ| U σ ) −σ | σ ∈ S i }. Hence, Lemma 6 applies and if α∈A i φ α is satisfiable under some assignment τ (otherwise the algorithm would immediately terminate), then there is an assignment

The argument for A i ⊂ A i+1 is similar and uses the property shown in Lemma 7.
Note that the algorithm presented above does not make any assumptions on the formula structure, i.e., for a QBF Π.φ it is not required that φ is in conjunctive normal form. Without any modification, our algorithm also works on formulas in PCNF-as SAT solvers typically process formulas in CNF only, we focus on this representation for the rest of the paper.

Relation to the ∀Exp+Res calculus
The ∀Exp+Res calculus [6,30,31] yields the theoretical foundation of our algorithm for refuting a formula Π.φ in PCNF with universal variables U and existential variables E.
The ∀Exp+Res calculus consists of the two rules shown in Fig. 3. Given an assignment τ : U Π → { , ⊥} of the universal variables U Π and a clause occurring in a QBF Π.φ, then the axiom rule instantiates C such that all universal literals u are assigned value τ (u) and the remaining existential literals l are annotated by [τ ], i.e., by those universals that precede the variable of l in the prefix. In the notation introduced before, we can write the axiom rule by C τ Note that only clauses that do not contain τ (l) = are of interest for a refutation proof. Further, any occurrences of ⊥ are omitted in the proof.
The resolution rule corresponds exactly to propositional resolution, i.e., the annotated variables are seen as propositional variables. Resolution between two clauses is only possible, if one contains a literal x σ and the other clause contains a literal ¬x τ and σ = τ , i.e., the pivot literals must have the same annotation. Note that we represent clauses as sets of literals.
A derivation in ∀Exp+Res is a sequence of clauses where each clause is either obtained by the axiom rule or derived from previously derived clauses by the application of the resolution rule. A refutation of a PCNF Π.φ is a derivation of the empty clause. The application of the axiom rule instantiates the universal variables of one clause of φ. If enough of these instantiations can be found in order to derive the empty clause by the application of the resolution rule, the QBF Π.φ is false.
Our algorithm presented in Fig. 1 does not instantiate selected clauses of the input formula, but all clauses of the matrix φ at once using a particular assignment of the universal variables. Hence, when the SAT solver finds ψ ∀ = α∈A i φ α unsatisfiable for some A i , not necessarily all clauses of ψ ∀ are required to derive the empty clause via resolution, but only one minimal unsatisfiable core of ψ ∀ , i.e., a subset of the clauses such that the removal of any clause would make this formula satisfiable. This observation leads us to the following proposition.
Proposition 1 Let Π.φ be a false QBF. Further, let ψ ∀ = α∈A i φ α be obtained by the application of the algorithm in Fig. 1. Further, let ψ ∀ be a minimal unsatisfiable core of ψ ∀ . Then there is a ∀Exp+Res refutation such that all clauses that are introduced by the axiom rule occur in ψ ∀ .
When fully expanding universal a, we obtain the propositional formula For proving unsatisfiability of this formula, it is enough to consider the formula ((¬x ∨ y )∧ (¬y ) ∧ (x ∨ y ⊥ ) ∧ (¬y ⊥ )). The corresponding ∀Exp+Res proof is shown in Fig. 4. Since existential variable x occurs outermost in the prefix, it is not annotated during the applications of the axiom rule.
Currently our implementation supports the generation of refutation proofs for false formulas and checking them for correctness. In consequence, we are now able to efficiently check the correctness of the solving results for false formulas, because the correctness check is linear in the proof size. For such proofs, we designed a novel proof format, because to best of our knowledge recent QBF solvers implementing expansion-based approaches do not support any proof generation.
Conceptually, proof generation for true QBFs works dually: instead of refuting a set of clauses, a set of cubes (conjunctions of literals) is shown to be valid. For this purpose, the resolution rule has to be modified to operate on cubes instead of clauses. In practice, however, SAT solvers are used that operate on clauses, hence an extra transformation step introducing fresh variables is required. This is currently not supported by our checker and is subject to future work.

Implementation
The algorithm described in Sect. 5 is realized in the solver Ijtihad 2 The most recent version of Ijtihad is available at https://extgit.iaik.tugraz.at/scos/ijtihad The solver is implemented in C++ and currently processes formulas in PCNF available in the QDIMACS format. For accessing SAT solvers, Ijtihad uses the IPASIR interface [5], which makes changing the SAT solver very easy. The SAT solver used in all of our experiments is Glucose [2]. Although the base implementation does reasonably well, we have realized various optimizations to make Ijtihad even more viable in practice. Some of them are discussed in the following.
For solving a QBF Π.φ, the basic algorithm shown in Fig. 1 adds instantiations of φ to ψ ∀ = α∈A i−1 φ α and ψ ∃ = σ ∈S i ¬φ σ in each iteration i until the formula is decided. The calls to the SAT solver in Line 5 and Line 8 are done incrementally, i.e., we create two instances of the SAT solver and provide them with the clauses stemming from new instantiations of φ at each iteration. For simplicity, we omit indices of sets A and S and refer to an arbitrary iteration of the execution of the algorithm in the following discussion. Figure 5 relates set sizes of A and S as well as the accumulated time that one SAT solver needs to solve ψ ∀ with the time the other SAT solver needs to solve ψ ∃ for the formulas of the PCNF track of QBFEVAL'17 (preprocessed with Bloqqer [8]). In this paper, we also distinguish between true and false formulas. In Fig. 5a we see that for true formulas, set S tends to be larger than A, while for false instances the picture is less clear. Figure 5b shows the overall time needed for solving ψ ∀ (y-axis) and ψ ∃ (x-axis). In almost all cases, the solver that handles ψ ∀ needs more time than the solver that handles ψ ∃ . This may be founded on the observation that many QBFs have considerably more existential variables than universal variables [39], hence the instantiations added to ψ ∀ are much larger than the instantiations added to ψ ∃ .
In Line 1 of Fig. 1, the set of universal assignments A is initialized with one arbitrary assignment α 0 . Obviously, the set A may also be initialized with multiple assignments. In our current implementation, we initialize A with the assignments that set the variables of one universal quantifier block to ⊥ and the variables of all other universal quantifier blocks to . The impact of various initialization heuristics remains to be investigated in future work.
In Line 7 and Line 10 our algorithm increases the size of S and A in each iteration of the main loop, as argued in Theorem 2. In the worst case, this leads to an exponential increase in space consumption. Although we detect shared clauses among the instantiations, that alone is not enough to significantly reduce the space consumption. However, some of the assignments found in an earlier iteration could become obsolete after better assignments were found. It is therefore beneficial to empty either S or A and then reconstruct them from ψ ∀ and ψ ∃ , similarly to what is done in Line 7 and Line 10. We evaluated several heuristics for scheduling these set resets, and we found that resetting periodically and close to the memory limit works best. The regular resetting of one set has a similar effect as restarts in SAT solvers, and we observed a considerable improvement in performance, especially in terms of memory consumption. Our implementation periodically resets the set A, since experiments indicate that the resulting formula ψ ∀ is much harder to solve than ψ ∃ as seen in Fig. 5b. Besides the aforementioned imbalance between universal and existential variables, it is also likely due to the structure of ψ ∃ which is a conjunction of formulas in disjunctive normal form. Note that this reset of A does not affect the termination argument presented in Theorem 2, since the sets A and S still complete each other.
Finally, we extended the presented approach with orthogonal reasoning techniques like QCDCL [23] for exploiting the different strengths of ∀Exp+Res and Q-resolution, yielding a hybrid solver that smoothly integrates both solving paradigms. To this end, we implemented the prototypical solver called Heretic which pursues the following idea: The main loop of the algorithm shown in Fig. 1 (Lines 4-12) is extended in a sequential portfolio style such that a QCDCL solver is periodically called. After each call, all clauses that were learned through QCDCL are added to Π.Φ, making them available in further iterations. These new clauses potentially exclude assignments that would otherwise be possible and that could result in more iterations of the main loop.
The solver Heretic extends Ijtihad by additional invocations of the QCDCL solver DepQBF [38]. About every 30 seconds, DepQBF is called and run for about 30 seconds. The learned clauses are obtained via the API of DepQBF. Leveraging learned cubes is subject to future work.
The tables in the left column of Fig. 6 show the total numbers of solved instances (S), solved unsatisfiable (⊥) and satisfiable ones ( ), and total CPU time including timeouts. The plots in the right column of Fig. 6 visualize the runtimes of the respective solvers. In the first row, the results without any preprocessing are shown. Our solver Heretic is ranked third solving the most false formulas of all solvers. If the preprocessor QRATPre+ is applied, Heretic is ranked second. Only CAQE solves more formula instances. Also with the other two preprocessors, CAQE seems to be the solver that benefits most from the additional preprocessing step. In general, preprocessing has a considerable impact on the number of solved instances. With preprocessing enabled, Heretic solves up to 142 more formulas than without preprocessing. Also Ijtihad strongly benefits from preprocessing: alone it solves 151 formulas, and with a preprocessor it solves up to 229 formulas.
Notably Heretic, despite its simple design, significantly outperforms Ijtihad on the QBFE-VAL'18 benchmark set. Moreover, Heretic is ranked third and second on preprocessed instances and thus is on par with state-of-the-art solvers. On the considered benchmark set, the gap in solved instances between RAReQS and Heretic is considerably smaller than the one between RAReQS and Ijtihad.
A direct comparison of our solver Heretic with RAReQS is shown in Fig. 7a. Unlike our solver, RAReQS is based on a recursive implementation of expansion. While the plot looks very balanced for the whole benchmark set, the picture changes for formulas with four or more quantifier blocks, i.e., three or more quantifier alternations (see Fig. 7b and below).
On such formulas with many quantifier blocks, the strength of Heretic becomes apparent, cf. [39]. As shown in Tables 8a, 8c and 8e, Heretic outperforms all other solvers on original instances and on instances with preprocessing by QRATPre+ and Bloqqer. The only exception are instances preprocessed by HQSpre (Table 8g).
Moreover, on entire benchmark sets without and with preprocessing (Tables 6a, 6c, 6e, and 6g), Heretic significantly outperforms both DepQBF and Ijtihad. These results indicate the potential of combining the orthogonal proof systems ∀Exp+Res as implemented in Ijtihad and Q-resolution as implemented in DepQBF in a hybrid solver like Heretic.
Although RAReQS outperforms both Ijtihad and Heretic on instances preprocessed by Bloqqer (Table 6e) RAReQS failed to solve certain instances that were solved by Ijtihad or Heretic. Table 1 shows related statistics. E.g., on instances preprocessed by HQSpre   Table 8a) Table 1 Statistics related to  Tables 6a, 6c solved instances (more than any individual solver in Table 6g) that could have been solved by a hypothetical solver combining RAReQS and Heretic. This observation underlines the strength of expansion in general and, in particular, of the hybrid approach implemented in Heretic. Heretic solved a significant amount of instances not solved by RAReQS, and it clearly outperformed Ijtihad and DepQBF on all benchmarks (columns "I vs. H" and "D vs. H").

Conclusion
We presented a novel non-recursive algorithm for expansion-based QBF solving that uses only two SAT solvers for incrementally refining the propositional abstraction and the negated propositional abstraction of a QBF. We gave a concise proof of termination and soundness and demonstrated with several experiments that our prototype compares well with the state of the art. In addition to non-recursive expansion, we also studied the impact of combining Q-resolution and ∀Exp+Res in a hybrid approach. To this end, we coupled a QCDCL solver and non-recursive expansion to make clauses derived by the QCDCL solver available to the expansion solver. Experimental results indicated that the hybrid approach significantly outperforms our implementation of non-recursive expansion indicating the potential of combining expansion-based approaches with Q-resolution which gives rise to an exciting direction of future work. Further, our current implementation supports only formulas in conjunctive normal form while in theory, our approach does not make any assumptions on the structure of the propositional part of the QBF. We also plan to investigate how this formula structure can be exploited for efficiently processing the negation of the formula.
Funding Open access funding provided by Graz University of Technology.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.