Diophantine problems in solvable groups

We study the Diophantine problem (decidability of finite systems of equations) in different classes of finitely generated solvable groups (nilpotent, polycyclic, metabelian, free solvable, etc), which satisfy some natural"non-commutativity"conditions. For each group $G$ in one of these classes, we prove that there exists a ring of algebraic integers $O$ that is interpretable in $G$ by finite systems of equations (e-interpretable), and hence that the Diophantine problem in $O$ is polynomial time reducible to the Diophantine problem in $G$. One of the major open conjectures in number theory states that the Diophantine problem in any such $O$ is undecidable. If true this would imply that the Diophantine problem in any such $G$ is also undecidable. Furthermore, we show that for many particular groups $G$ as above, the ring $O$ is isomorphic to the ring of integers $\mathbb{Z}$, so the Diophantine problem in $G$ is, indeed, undecidable. This holds, in particular, for free nilpotent or free solvable non-abelian groups, as well as for non-abelian generalized Heisenberg groups and uni-triangular groups $UT(n,\mathbb{Z}), n \geq 3$. Then we apply these results to non-solvable groups that contain non-virtually abelian maximal finitely generated nilpotent subgroups. For instance, we show that the Diophantine problem is undecidable in the groups $GL(3,\mathbb{Z}), SL(3,\mathbb{Z}), T(3,\mathbb{Z})$.


Introduction
We study the Diophantine problem (decidability of finite systems of equations) in different classes of finitely generated solvable groups (nilpotent, polycyclic, metabelian, free solvable, etc), which satisfy some natural "non-commutativity" conditions. For each group G in one of these classes, we prove that there exists a ring of algebraic integers O that is interpretable in G by finite systems of equations (e-interpretable), and hence that the Diophantine problem in O is effectively polynomial time reducible to the Diophantine problem in G. A famous conjecture in number theory states that the Diophantine problem in any such O is undecidable, implying by the result above that the Diophantine problem in any such G would be also undecidable. In fact, we show that for many particular groups G as above, the ring O is isomorphic to the ring of integers Z, so the Diophantine problem in G is, indeed, undecidable. This holds, in particular, for free nilpotent or free solvable non-abelian groups, as well as for non-abelian generalized Heisenberg groups and uni-triangular groups U T pn, Zq, n ě 3. Then we apply these results to non-solvable groups that contain non-virtually abelian maximal finitely generated nilpotent subgroups. For instance, we show that the Diophantine problem is undecidable in the groups GLp3, Zq, SLp3, Zq, T p3, Zq.
The Diophantine problem (also called Hilbert's tenth problem or generalized Hilbert's tenth problem) in a structure R, denoted DpRq, asks whether there exists an algorithm that, given a finite system of equations S with coefficients in R, determines if S has a solution in R or not. The original version of this problem was posed by Hilbert for the ring of integers Z. This was solved in the negative in 1970 by Matiyasevich [42] building on the work of Davis, Putnam, and Robinson [8]. Subsequently the same problem has been studied in a wide variety of rings, most notably in Q and in rings of algebraic integers O (integral closures of Z in finite field extensions of Q), where it remains widely open. A longstanding conjecture (see, for example, [10,51]) states that Z is Diophantine in any such O (and thus DpOq is undecidable). This conjecture has been verified in some particular cases [63,64,18], and it has been shown to be true assuming the Safarevich-Tate conjecture [43]. We refer to [52,51,64] for further information on the Diophantine problem in different rings and fields of number-theoretic flavor. On the other hand, Kharlampovich and the second author showed in [33] that the Diophantine problem is undecidable for in free associative algebras for any field of coefficients, and in the group algebras (over any field of coefficients) of a wide variety of torsion-free groups, including toral relatively hyperbolic groups, right angled Artin groups, commutative transitive groups, and the fundamental groups of various graph groups. Moreover, they proved in [28] undecidability of the Diophantine problem in free Lie algebras of rank at least three with coefficients in an arbitrary integral domain. In [19] we studied the Diophantine problem in more general rings and algebras (possibly non-associative, non-commutative, and non-unitary), obtaining analogous results to the ones in this paper. Indeed, the present paper may be read as a continuation of [19].
Research on systems of equations and their decidability in groups has a very long history, it goes back to 1912 to the pioneering works of Dehn on the word and conjugacy problems in finitely presented groups. Within the class of solvable groups, it is known that the word and conjugacy are decidable for many such groups, including finitely generated nilpotent, polycyclic, metabelian, and free solvable groups. On the other hand, there is also a famous example, due to Kharlampovich, of a finitely presented solvable group with undecidable word problem [26].
The first results on the proper Diophantine problem in groups are due to Romankov. He showed in [56,55] that the Diophantine problem is undecidable in any non-abelian free metabelian group and in any non-abelian free nilpotent group of nilpotency class at least 9. Variations and improvements of these results were obtained subsequently in [2,16,66]. Recent work of Duchin, Liang and Shapiro [15] shows that DpN q is undecidable in any finitely generated nonabelian free nilpotent group N . We refer to a survey [59] for these and more results on equations in groups. Stepping outside of the realm of systems of equations, Noskov showed in [50], following the work of Malcev [39], Ershov [17] and Romanovskii [60], that the first-order theory of any finitely generated nonvirtually abelian solvable group is undecidable. Note, that much earlier Ershov [17] proved that any virtually abelian group has decidable elementary theory. The papers [57] and [5] contain results of a similar flavour to the ones of this paper: decidability of the universal theory of a free nilpotent group or a free solvable group of class at least 3 implies decidability of the Diophantine problem in the field of rational numbers Q, a major open problem.
In solvable groups systems of equations are fundamentally different from single equations. For instance, finite systems of equations are undecidable in the Heisenberg group (i.e. the free nilpotent group of nilpotency class 2 and of rank 2), while single equations are decidable [15]. This contrasts with most number theoretic settings, where the two notions are often used interchangeably since in all integral domains, whose field of fractions are not algebraically closed, every finite system of equations is equivalent to a single equation (see, for example, [52]). Much of the research regarding equations in solvable groups was focused so far on single equations (see [59]), indeed Romankov's aforementioned results [55,56] (and also Truss' [66]) actually prove that single equations are undecidable in the corresponding groups G. These are stronger results than just undecidability of the Diophantine problem in G. Allowing arbitrary finite systems of equations is fundamental to our approach. This makes the whole theory much more robust and brings to the table powerful general methods.
This line of results changes drastically outside of the class of solvable groups: the work of Makanin and Razborov [38,53] shows that DpF q is decidable for any free group F , and it further provides a description of the solution sets to arbitrary systems of equations in F (systems of equations are equivalent to single equations in F ). See also [24,12,6] for an entirely different approach. Analogous work has been done for other non-solvable groups, such as hyperbolic groups [54,7], partially commutative groups [3,14], and some free and graph products [4,13]. We refer to [27] for further results in this area.
Note, that there are finitely generated solvable non-virtually abelian groups with decidable Diophantine problem. The first such examples are due to Kharlampovich, López, and the second author, who proved in [34] that the Diophantine problem is decidable in the following metabelian groups: BSp1, nq, n ě 1 and A ≀ Z, where A is a finitely generated abelian group.
We would like to emphasize that in the case when the Diophantine problem in a group G is undecidable or open it is very interesting to consider decidability of equations or systems of equations of a particular type. In fact, it might be advantageous even in the case when the Diophantine problem in G is decidable, since decision algorithms for particular equations could be much more efficient than the general ones. To this end we would like to mention two results: it is shown in [36] that systems of quadratic equations are decidable in the first Grigorchuk group (the Diophantine problem in this group is wide open), and also that orientable quadratic equations are decidable in free metabelian groups [37] (though the Diophantine problem here is undecidable, see below).
We proceed to state the main results of the paper. In all of them, we consider certain types of groups, and we prove that for any such group G there exists a ring of algebraic integers O that is e-interpretable in G, which implies that DpOq is polynomial time (many-one) reducible to DpGq. We further conjecture that in this case, the ring Z is e-interpretable in G and the Diophantine problem in G is undecidable. In fact, we confirm this conjecture for many groups G of a particular type.
In Section 2.1 we introduce the notion of interpretation by equations (einterpretation) of one algebraic structure in another, which is the main technical tool of our method. We show that if a structure A is e-interpretable in a structure B then DpAq is polynomial time many-one reducible to DpBq, symbolically DpAq ď P DpBq. Reductions of this type are also called Karp reductions. All reductions in this paper are Karp reductions, so sometimes for brevity we refer to them simply as reductions.
In Section 4.1.2 we prove the following principal result, on which most of the other results of this paper are based on. Theorem 4.6. Let G be a finitely generated non-virtually abelian nilpotent group. Then there exists a ring of algebraic integers O e-interpretable in G, hence DpOq ď P DpGq (i.e., DpOq is Karp reducible to DpGq).
As we mention above, if G is virtually abelian then DpGq is decidable. For nilpotency class 2, we prove this result by considering the largest ring of scalars R of the bilinear map G{ZpGqˆG{ZpGq Ñ G 1 induced by the commutator operation r¨,¨s. It then follows from [19] that R is e-interpretable in G. By this same reference, there exists a ring of algebraic integers e-interpretable in R, and hence in G by transitivity. We refer to Section 3 for further details regarding these results and the notion of largest ring of scalars. Higher nilpotency class reduces to class 2 by the following nilpotent quotient argument: the third term γ 3 pGq of the lower central series of G has finite verbal width, hence it is e-definable in G. Therefore the class 2 nilpotent quotient G{γ 3 pGq is einterpretable in G, which provides the reduction.
Note that the nilpotent quotient argument is quite general: it works for any group G with e-definable subgroup γ 3 pGq or, more generally, γ i pGq, i ě 3.
Theorem 4.6 together with the nilpotent quotient argument yield the following Theorem 4.13. Let G be a finitely generated group such that for some i P N γ i pGq is e-definable in G (in particular, if γ i pGq has finite verbal width) and G{γ i pGq is not virtually abelian. Then there exists a ring of algebraic integers O that is e-interpretable in G, hence DpOq ď P DpGq.
There are many groups that satisfy the premises of Theorem 4.13. We mentioned some of them in the corollary below. Corollary 4.14. Let G be a finitely generated group which is either metabelian, or solvable minimax, or polycyclic, or virtually abelian-by-nilpotent, or (nilpotent minimax)-by-(abelian-by-finite). If G{γ i pGq is not virtually abelian for some i P N then there exists a ring of algebraic integers O that is e-interpretable in G, hence DpOq ď P DpGq.
The results above are all of the reducibility type, i.e., they show that the Diophantine problem for various groups G is at least as hard as the one for a suitable ring of algebraic integers O. It seems, it does not give us much unless we know the complexity of the Diophantine problem in O, which is conjectured to be undecidable [51,10]. One can show that for a wide variety of finitely generated nilpotent groups the ring O is, in fact, isomorphic to Z, where the Diophantine problem is known to be undecidable. Furthermore, we proved in [20] that the ring O in a random finitely generated nilpotent group of class c ě 2 is, indeed, isomorphic to Z. Our method of proving isomorphism O » Z is based on maximal rings of scalars of bilinear maps and centralizer small (csmall) elements in groups G, i.e., elements of infinite order g P G such that C G pgq " xgyˆZpGq. We further use the above methods to prove that the ring Z is e-interpretable in the nilpotent groups below. Let N c be a finitely generated non-abelian free nilpotent group of nilpotency class c ě 2. The next result is implicit in the aforementioned work of Duchin, Liang and Shapiro [15].
Theorem 4.7. The ring Z is e-interpretable in any finitely generated non-abelian free nilpotent group N c . Theorem 4.9. Let G " U T pn, Zq, n ě 3, be the group of upper uni-triangular nˆn matrices with integer entries. Then the ring Z is e-interpretable in G and DpGq is undecidable.
We also prove the following Proposition 4.11. Let G be a finitely generated nilpotent group such that G 1 {γ 3 pGq has torsion-free rank at most 2. Then the ring Z is e-interpretable in G, and DpGq is undecidable.
Combining this with a result from [15] we obtain that if G is a finitely generated non-virtually abelian nilpotent group of class 2 with infinite cyclic commutator subgroup, then DpGq is undecidable, while single equations are decidable in G. This applies in particular to any nonabelian generalized Heisenberg group, i.e. any group of the form H n " xa 1 , . . . , a n , b 1 , . . . , b n | ra i , b j s " ra 1 , b 1 s, ra i , a j s " rb i , b j s " 1, 1 ď i ď j ď ny N 2 for some n ě 1 (here xy N 2 denotes presentation in the variety of nilpotent groups of class 2). This result was already obtained in [15] for the classical Heisenberg group (n " 1).
We remark that, to some extent, the converse of Theorem 4.6 is also true: given a ring of algebraic integers O, or more generally, any associative commutative unitary ring R, the matrix groups SLpn, Rq, T pn, Rq and U T pn, Rq are e-interpretable in the ring R (matrix multiplication can be described by polynomials over R). Hence the Diophatine problems in these groups is Karp reducible to the Diophantine problem in R.
Finally, we briefly describe the maximal nilpotent subgroup argument. In Lemma 4.15 we prove that a maximal finitely generated nilpotent subgroup H of a fixed class c of a group G is e-definable in G. Thus Theorem 4.6 can also be carried over to groups that have such a (non-virtually abelian) subgroup H. With this in mind we denote by N max the class of groups where for all c ě 1, every set of c-nilpotent subgroups of G has a maximal element (with respect to inclusion). It turns out that N max consists precisely of groups where all abelian subgroups are finitely generated. In Section 4.3 we describe various types of groups that belong to N max , in particular, we note that groups GLpn, Oq, where O is a ring of algebraic integers, are there. This result allows, in particular, to extend Theorem 4.6 to the class of finitely generated virtually nilpotent groups (since the class N max is closed under finite extensions). Another application is the following Example 4.30. The Diophantine problem in the groups GLp3, Zq, SLp3, Zq, and T p3, Zq is undecidable.
From Theorem 4.16 and from the fact that any polycyclic group is (nilpotentby-abelian)-by-finite we obtain the following Note that, on the other hand, it is known that DpOq is undecidable for O the ring of algebraic integers of any quadratic field [9].
We finish the paper by studying relatively free groups in the product A d N c of the varieties A d and N c , where A d is the variety of all solvable groups of class d and N c is the variety of all nilpotent groups of class c, for c, d ě 1. We will refer to these as free solvable-by-nilpotent groups. Any such groups is isomorphic to F {pγ c pF q pdq q where F is a free group.

Theorem 4.34. Let G be a finitely generated nonabelian free (solvable-by-nilpotent) group. Then the ring Z is e-interpretable in G, and DpGq is undecidable.
The above includes all f.g. non-abelian free solvable groups, and it extends Romankov's result that f.g. free metabelian groups have undecidable Diophantine problem [56] (in fact Romankov proves the stronger result that single equations are undecidable in free metabelian groups of countable rank).

Theorem 4.33. Let G be a finitely generated nonabelian free solvable group.
Then the ring Z is e-interpretable in G, and DpGq is undecidable.

Interpretations by systems of equations
where I, J, K, L are sets of natural numbers; the A i are pairwise disjoint sets called sorts; the f j are functions of the form f j : 1 for some indices i t P I; the r k are relations of the form r k : A s 1ˆ¨¨¨ˆA sq Ñ t0, 1u for some indices s t P I; and the c ℓ are constants, each one belonging to some sort. The tuple pf j , r k , c ℓ | j, k, ℓq is called the signature or the language of A. We always assume that A contains the relations "equality in A i " for all sorts A i . If A has only one sort then A is a structure in the usual sense. One can construct terms in a multi-sorted structure in an analogous way as in uniquely-sorted structures. In this case, when introducing a variable x, one must specify a sort where it takes values, which we denote A x .
Let A 1 , . . . , A n be a collection of multi-sorted structures. We let pA 1 , . . . , A n q be the multi-sorted structure that is formed by all the sorts, functions, relations, and constants of each A i . Similarly, given a function f or a relation r we use the notation pA, f q or pA, rq to refer to the structure A together with the extra function f or relation r in the language, respectively. If two different A i 's have the same sort, then we view one of them as a formal disjoint copy of the other.
Diophantine problems and reductions. Let A be a multi-sorted structure. An equation in A is an expression of the form rpτ 1 , . . . , τ k q, where r is a signature relation of A (typically the equality relation), and each τ i is a term in A where some of its variables may have been substituted by elements of A. Such elements are called the coefficients (or the constants) of the equation. These may not be signature constants. A system of equations is a finite conjunction of equations. A solution to a system of equations^iΣ i px 1 , . . . , x n q on variables x 1 , . . . , x n is a tuple pa 1 , . . . , a n q P A x 1ˆ¨¨¨ˆA xn such that each Σ i pa 1 , . . . , a n q is true in A.
The Diophantine problem in A, denoted DpAq, refers to the algorithmic problem of determining if a given system of equations in A (with coefficients in A) has a solution in A. Sometimes this is also called Hilbert's tenth problem or a generalized Hilbert's tenth problem in A. An algorithm L is a decision algorithm for DpAq if, given a system of equations S in A, L determines whether or not S has a solution in A. If such an algorithm exists, then DpAq is called decidable, otherwise, it is undecidable. In this paper all structures are finitely generated, the coefficients of equations are given as terms (in the language of A) in the fixed set of generators.
Let A and B two structures. We say that Diophantine problem in A is reducible (or more precisely, many-one reducible) to the Diophantine problem in B if there is an algorithm that for every finite system of equations Σ in A constructs a finite system of equations Σ˚in B such that Σ has a solution in A if and only if Σ˚has a solution in B. All our reductions will be polynomial-time computable, i.e., the algorithm that transforms Σ to Σ˚works in polynomial time. These reductions are called polynomial-time reductions or Karp reductions. In this case we sometimes write DpAq ď P DpBq.
Interpretations by systems of equations. Interpretability by systems of equations (e-interpretability) is the analog of the classical notion of interpretability by first-order formulas (see [23,41]). In the e-interpretability one requires that only systems of equations are used, instead of arbitrary first-order formulas. As convened above, one is allowed to use any constants (not necessarily from the signature) in such systems of equations.
Let A be a structure with sorts tA i | i P Iu. A basic set of A is a set of the form A i 1ˆ¨¨¨ˆA im for some m and i j 's. From the viewpoint of number theory, an e-definable set is a Diophantine set. From the perspective of algebraic geometry, an e-definable set is a projection of an affine algebraic set.

Example 2.2.
Let G be a group generated by a 1 , . . . , a n . Then its center ZpGq is e-defined in G by the system of equations rx, a i s " 1 (i " 1, . . . , n) on the variable x.
We are ready to introduce the notion of e-interpretability. The tuple of maps φ " pφ 1 , . . . q is called an e-interpretation of A in M.
The next two results are fundamental and will often be used without referring to them. They follow from Lemma 2.7 of [19].

Proposition 2.5 (Reduction of Diophantine problems). Let A and M be (possibly multi-sorted) structures such that
A is e-interpretable in M. Then DpOq ď P DpGq. As a consequence, if DpAq is undecidable, then so is DpMq.
One of the principal features of e-interpretability is that it is compatible with taking quotients by e-definable congruence relations. Before we see this let us agree on some terminology. Remark 2.6. When we say that a subgroup H of a group G is e-definable in G, we mean that H is e-definable as a subset of G. Notice that in this case, the identity map H Ñ H constitutes an e-interpretation of H in G. Indeed, the graph of the group operation of H is e-defined in G by the equation z " xy, and similarly for the equality relation.
The following lemma may be read as an illustrative example of the notion of e-interpretability. It can be generalized to any suitable type of structure and the corresponding congrtuence relations.
Proof. Let Σpx, yq be a system of equations that e-defines N in G, so that g P G belongs to N if and only if Σpg, yq has a solution y. We check that the natural epimorphism π : G Ñ G{N is an e-interpretation of G{N in G. First observe that the preimage of π is the whole G, which is e-definable in G by an empty system of equations. Regarding equality in G{N , the identity πpg 1 q " πpg 2 q holds in G{N if and only if g 1 g´1 2 P N , i.e. if and only if Σpg 1 g´1 2 , yq has a solution on y. From this it follows that the preimage of equality in G{N , tg 1 , g 2 P G | πpg 1 q " πpg 2 qu , is edefinable in G by the system Σpx 1 , x 2 , yq obtained from Σpx, yq after substituting each occurrence of x by x 1 x´1 2 , where x 1 and x 2 are new variables. By similar arguments, the preimage of the graph of multiplication in G{N is e-definable in G: indeed, πpg 1 qπpg 2 q " πpg 3 q if and only if g 1 g 2 g´1 3 P N .

Varieties of groups
We will need the following terminology and results. A class of groups is a set of isomorphism classes of groups (in this paper any group is identified with its isomorphism class). Given two classes of groups S and T we let their product ST be the class of all S-by-T groups, i.e. those groups G for which there exists a normal subgroup N such that N P S and G{N P T . A variety of groups V is a class of groups for which there exist finitely many words w i px 1 , . . . , x n q, i " 1, . . . , m, on variables tx j | ju, such that K P V if and only if w i pk 1 , . . . , k n q " 1 for all k 1 , . . . , k n P K and all i " 1, . . . , m. Theorem 21.51 of [49] states that pRSqT " RpST q for any three varieties of groups. We let A, E n , F denote the classes of all abelian groups, all groups of exponent n, and all finite groups, respectively. The first two are varieties, while the third is not. H be a retract of a group G. The the Diophantine problem in H is polynomial time many-one reducible to the Diophantine problem in G. In fact the reduction is achieved through the identity map.

Proposition 2.8. Let
Proof. Indeed, let π : G Ñ H be a retract (i.e. a homomorphism G Ñ H which is identical on H). Let SpXq " 1 be an arbitrary finite system of equations with coefficients in H, then if x i Þ Ñ a i px i P Xq is a solution to this system in G, then x i Þ Ñ πpa i q is a solution of this system in H. Hence S has a solution in G if and only if it has a solution in H.
As an immediate consequence we have the following: Corollary 2.9. Let V be a variety of groups and X an infinite set. By F V pXq we denote the free group in V with basis X. Then for any finite subset X 0 Ď X the subgroup generated by X 0 in F V pXq is free in V with basis X 0 , it is a retract of F V pXq, and its Diophantine problem is polynomial time many-one reducible to the Diophantine problem in F V pXq.
We will need the following auxiliary result.

Lemma 2.10 ([22]
). Any finitely generated finite-by-abelian group G is abelianby-finite (i.e. virtually abelian). In fact, in this case the center ZpGq has finite index in G.
Let w " wpx 1 , . . . , x m q be a word on an alphabet tx 1 , . . . , x m u. The wverbal subgroup of a group G is the subgroup xwpGqy generated by wpGq " twpg 1 , . . . , g m q | g 1 , . . . , g m P Gu. One says that w has finite width in G if there exists an integer n such that every g P xwpGqy is equal to the product of at most n elements from wpGq˘1. In this case xwpGqy is e-defined in G by the equation which has variables x and ty ij , z ij | 1 ď i ď n, 1 ď j ď mu (note that some of the factors in (1) can be made trivial by taking wp1, . . . , 1q˘1). If w has finite width in G for any w, then G is said to be verbally elliptic. Observe that each term γ i pGq of the lower central series of G is w i -verbal, where w i " rx 1 , . . . , x i s. It is known that any finitely generated nilpotent, metabelian, or polycyclic group is verbally elliptic. More generally, any f.g. (abelian-by-nilpotent)-by-finite or (nilpotent minimax)-by-(abelian-by-finite) group is verbally elliptic. This includes the class of f.g. solvable minimax groups. These results are due to George, Romankov, Segal, and Stroud [21,58,62,65]. Proofs can be found in Theorems 2.3.1, 2.6.1, and Corollary 2.6.2 of [62], respectively. This same reference contains further results of this type for infinitely generated groups.
A group G is said to be minimax if it admits a composition series all whose factors are finite, infinite cyclic, or quasicyclic (a group is quasicyclic if it is isomorphic to Zr1{ps{Z for some prime p). If all the factors are cyclic (finite or infinite), then G is polycyclic.

Largest ring of scalars of bilinear maps and rings of algebraic integers
Let A and B be abelian groups, and let f : AˆA Ñ B be a bilinear map between them. We associate with such f a two-sorted structure pA, B; f q. The map f is said to be non-degenerate if whenever f pa, xq " 0 for all x P A, one has a " 0, and similarly for f px, aq. It is called full if the subgroup generated in B by the image of f is B. An associative commutative unitary ring R is called a ring of scalars of f if there exist faithful actions of R on A and B, which turn A and B into R-modules and such that f is R-bilinear with respect to these actions. More precisely, in this case f pαx, yq " f px, αyq " αf px, yq for all α P R and all x, y P A. Let R be a ring of scalars of f . Since R acts faithfully on A and B, there exist ring embeddings R ãÑ EndpAq and R ãÑ EndpBq. For this reason and for convenience, we always assume that a ring of scalars of f is a subring of EndpAq. We say that R is the largest ring of scalars of f if for any other ring of scalars R 1 of f , one has R 1 ď R when viewed as subrings of EndpAq. If f is full and non-degenerate then such ring exists and is unique [45], and we denote it Rpf q.
The notion of the largest ring of scalars of a bilinear map f was introduced by the second author in [45]. This ring constitutes an important feature of f , and it has been used successfully to study different first-order theoretic aspects of different types of structures, including rings whose additive group is finitely generated [46], free algebras [28,31,30,29], and nilpotent groups [47,48]. For us the most relevant property of Rpf q is that it is e-interpretable in pA, B; f q:

Theorem 3.1 (Theorem 3.5 of [19]). Let f : AˆA Ñ B be a full non-degenerate bilinear map between finitely generated abelian groups. Then the largest ring of scalars Rpf q of f is finitely generated as an abelian group, and it is e-interpretable in pA, B; f q. Moreover Rpf q is infinite if and only if B is.
Proof. A more general statement is proved in Theorem 3.5 of [19] for Ł-bilinear maps between Ł-modules, where Ł is an arbitrary Noetherian commutative ring. Our statement corresponds to the particular case Ł " Z, since the notions of Z-module and of abelian group coincide under the terminology used in [19] (see Paragraph 4 of Section 2.3, and Remark 1.5, both in [19]).
The previous result constitutes the first step towards e-interpreting rings of algebraic integers in different families of solvable groups (more generally, in structures that have a suitable bilinear map associated to them). The second step is given by the result below. By rank of a ring or an abelian group we refer to the maximum number of Z-linearly independent elements in it (considering the group with additive notation). Combining the previous two theorems, we obtain the following fundamental result.

Diophantine problems in solvable groups
In this section we present our main results regarding systems of equations in solvable groups. The next lemma deals with the case when the group is virtually abelian. [17], see also [50]). Any finitely generated virtually abelian group has decidable first-order theory (with constants). In particular, the Diophantine problem in such group is decidable.

Nilpotency class 2
In a nilpotent group G of class 2 the commutator operation r¨,¨s induces a full non-degenerate bilinear map between abelian groups: Here ZpGq denotes the center of G. By Theorem 3.1 the largest ring of scalars R " Rpf q of f exists and is e-interpretable in pG{ZpGq, G 1 ; f q. We denote it by RpGq.

Definition 4.2. The ring R " RpGq is called the largest ring of scalars of G.
Observe that if G is finitely generated, then both ZpGq and G 1 are e-definable in G (see Example 2.2 and Remark 2.11, respectively). It follows that the two sorted structure pG{ZpGq, G 1 ; f q is e-interpretable in G (indeed the preimage of the graph of f is e-defined in G by the equation z " rx, ys). Furthermore if G is not virtually abelian then G 1 is infinite due to Lemma 2.10. By Theorem 3.1, Corollary 3.3, and transitivity of e-interpretations we obtain the following

Proposition 4.3. Let G be a finitely generated nilpotent group of nilpotency class 2. Then the largest ring of scalars R of G is e-interpretable in G. If additionally G is not virtually abelian, then there exists a ring of algebraic integers
O that is e-interpretable in R, and also in G by transitivity of e-interpretations. Moreover, the rank of O is at most the rank of R.
Of particular interest is the case when G is a finitely generated free nilpotent group of nilpotency class 2. We shall need the following definition.

Definition 4.4. An element g in a group
G is said to be c-small (or centralizersmall) if C G pgq " tg t z | t P Z, z P ZpGqu and C G pgq{ZpGq is infinite cyclic (C G pgq denotes the centralizer of g in G).
We can now prove the following

Proposition 4.5. Let G be a finitely generated nilpotent group of class 2. If G has a c-small element then the largest ring of scalars of G is Z and it is is e-interpretable in G.
Proof. Denote Z " ZpGq. Let a be a c-small element of G, and let ψ : G{Z Ñ G 1 be the group homomorphism given by xZ Þ Ñ ra, xs. Notice that ψpxZq " f paZ, xZq, where f is the map (2). Note also that kerpψq " C G paq{Z " xaZy -Z. Let R be the largest ring of scalars of G. By definition, R acts on G{Z and G 1 , and furthermore ψ is R-linear. It follows that R stabilizes kerpψq -Z. Hence for all α P R there exists an integer t α such that αaZ " a tα Z. The map φ : R Ñ Z defined by α Þ Ñ t α induces a group embedding between the additive groups of R and Z (see the proof of Theorem 3.4 in [20]). On the other hand, Z is a ring of scalars of f (note that it acts faithfully on G{Z and G 1 ), and so Z embeds in R. It follows that R as a ring is isomorphic to Z. Finally, Theorem 4.3 implies that Z is e-interpretable in G.

Arbitrary nilpotency class
Suppose G is a finitely generated nilpotent group of nilpotency class at least 2. Then G{γ 3 pGq is e-interpretable in G and it is nilpotent of class 2 (see Remark 2.11). Now we can use the methods of the previous section, together with transitivity of e-interpretations, to obtain one of the main results of the paper.

Theorem 4.6. Let G be a finitely generated non-virtually abelian nilpotent group. Then there exists a ring of algebraic integers O e-interpretable in G,
and DpOq ď P DpGq. If otherwise G is virtually abelian, then DpGq is decidable.
Proof. The last statement of the theorem is a particular case of Lemma 4.1. Hence assume G is not virtually abelian, in which case G 1 is infinite by Lemma 2.10. This together with Corollary 9 of [61] makes G 1 {γ 3 pGq infinite as well. It follows that G{γ 3 pGq is not virtually abelian, since if G{γ 3 pGq had an abelian subgroup of G{γ 3 pGq of index say n, then G 1 {γ 3 pGq would be a finitely generated abelian group of exponent n 2 , thus finite (to prove this use the identity rx, ys k " rx k , ys " rx, y k s, which holds for any two elements x, y in a group of nilpotency class 2). Due to Proposition 4.3, the largest ring of scalars R of G{γ 3 pGq is e-interpretable in G{γ 3 pGq, and also in G by transitivity of e-interpretations. Since G{γ 3 pGq is not virtually abelian, this same proposition implies that there exists a ring of algebraic integers O that is e-interpretable in R, and so also in G.
We proceed to study the case of a finitely generated nonabelian free nilpotent group N c of arbitrary nilpotency class c ě 2. The next result is implicit in the work of Duchin, Liang and Shapiro [15], and it is made explicit in Corollary 3.3 of [20] (which follows the approach of [15]). The following is an immediate consequence of the above Theorem 4.7 and Corollary 2.9.

Corollary 4.8. The Diophantine problem is undecidable in any non-abelian free nilpotent group, not necessarily of finite rank.
Now we discuss the groups of uni-triangular matrices. Proof. Given k ą 0, denote by U T m pn, Zq the subgroup of G formed by all matrices of G with m´1 zero diagonals above the main one. Then U T 1 pn, Zq " U T pn, Zq. It is known that for any r, s ą 0 we have rU T r pn, Zq, U T s pn, Zqs " U T r`s pn, Zq (see Example 3.2.1 in [44]). It follows that γ 3 pU T pn, Zqqq " U T 3 pn, Zq. Let G " G{γ 3 pU T pn, Zqq.
Denote by t ij the transvection matrix with ones on the main diagonal, a one in its pi, jq entry, and zeros in all other entries. It is straightforward to check that, for all 1 ď i, j, k, ℓ ď n with i ‰ j, rt ik , t kj s " t ij and rt ik , t ℓj s " 1 if k ‰ ℓ. From these identities it follows that ZpGq " xt ij | j´i " 2y, C G pt 12 q " xt 12 , t 3,4 , . . . , t n´1,n , ZpGqy, and C G pt 23 q " xt 23 , t 45 , . . . , t n´1,n , ZpGqy.
Denote A " ZpC G pt 12 qq and B " ZpC G pt 23 qq. Again using the above identities we have that if n ą 5 then A " xt 12 yZpGq and B " xt 23 yZpGq. From rt 12 , t 23 s " t 13 P ZpGq we obtain H " xA, By " A¨B. Hence H is e-definable in G and t 12 is a c-small element in H. From Proposition 4.5 and transitivity of e-interpretations we conclude that the ring Z is e-interpretable in G, and by Lemma 2.7 and Remark 2.11, we also conclude that Z is e-interpretable in G for the case n ą 5.
Next we treat the case 3 ď n ď 5. If n " 5, then using our previous arguments we see that A " ZpC G pt 12 qq " xt 12 yZpGq and B " ZpC G pt 23 qq " xt 23 , t 45 yZpGq. However, t 45 belongs to the center of H " xA, By " A¨B. Therefore H " xt 12 , t 23 yZpGq and t 12 is a c-small element in H. The result then follows similarly as in the case n ą 5. If n " 4 then C G pt 23 q " xt 23 yZpGq. Hence t 23 is a c-small element of G, and then the result follows again from Proposition 4.5. Finally, the case n " 3 is already proved in Theorem 4.7, since then G is free nilpotent.
Remark 4.10. The notion of the largest ring of scalars of a nilpotent group G can be extended to any nilpotency class. This is achieved by considering a bilinear map which resembles the ring multiplication of the Lie ring of G, and which generalizes (2). We refer to Subsection 3.3 of [48] for further details, omitting a full explanation here due to its technicality.
It may be possible to prove that such ring of scalars R is always e-interpretable in G. If so, then the previous results can be approached by considering R directly instead of taking first the quotient G{γ 3 pGq. This approach would yield the same results presented above but with an overall more involved exposition. Nevertheless, it may be more adequate when studying finer aspects of systems of equations in G. We shall not pursue this approach in this paper.
We next turn our attention to nilpotent groups G for which G 1 {γ 3 pGq has "small rank". Recall that by rank of an abelian group or a ring we refer to its maximum number of Z-linearly independent elements (considering the group with additive notation). Proposition 4.11. Let G be a finitely generated nilpotent group. Suppose the rank of G 1 {γ 3 pGq is either 1 or 2. Then the ring Z is e-interpretable in G, and DpGq is undecidable.
Proof. Let R be the largest ring of scalars of G{γ 3 pGq. By Proposition 4.3, R is e-interpretable in G{γ 3 pGq, and thus in G as well, by Remark 2.11. By the same arguments as in the proof of Theorem 4.6, G{γ 3 pGq is not virtually abelian since otherwise G 1 {γ 3 pGq would be finite, contradicting the fact that its rank is 1 or 2. Hence by Proposition 4.3 there exists a ring of algebraic integers O that is e-interpretable in R with rank at most the rank of R. By transitivity, O is e-interpretable in G as well. Let K be the number field of which O is the ring of algebraic integers. It is well known that the rank of O coincides with the degree |K : Q| of the extension K{Q. In [9] Denef proved that if K is a quadratic field (i.e. if |K : Q| " 2), then Z is e-interpretable in O (see also [11]). Hence if we see that the rank of R is at most 2, then we will have proved that the ring Z is e-interpretable in O, and also in G by transitivity.
By definition, R acts faithfully by endomorphisms on G 1 {γ 3 pGq. We claim that if the rank of a nontrivial finitely generated abelian group A is at most 2, then any commutative, associative ring acting faithfully on it also has rank at most 2. The proof of the proposition will be finished once this claim is proved.
To prove the claim, first consider the case when A is torsion-free. Then either A " Z or A " Z 2 . If A " Z, then R ď EndpAq -Z has rank 1. If A " Z 2 , then R ď EndpZ 2 q is a commutative ring whose elements are 2ˆ2 integer matrices. Let X and Y be two such matrices. Assume that they are not proportional to the identity matrix I. Since X and Y commute, elementary calculations show that αX`βY`γI " 0 for some integers α, β, γ not all of them 0. This implies that the rank of R is at most 2 and proves the claim for the case when A is torsion-free. Now we reduce the general case to the case when A is torsion-free. Let A be a finitely generated abelian group of rank at most 2, and let T be the torsion subgroup of A, i.e. the set of all elements of A of finite order. Then A{T is a torsion-free abelian group of rank at most 2. Notice that for any a P T and r P R we have ra P T , hence R acts on A{T . Denote Ann R pA{T q " tr P R | rA Ď T u. Then R{Ann R pA{T q acts faithfully on A{T , and thus by the paragraph above R{Ann R pA{T q has rank at most 2. Since T is finite and A is finitely generated, HompA, T q is finite (because each homomorphism from HompA, T q is uniquely determined by its action on a set of generators of A). Hence Ann R pA{T q ď HompA, T q is finite. This implies that the rank of R is the same as the rank of R{Ann R pA{T q, which is at most 2, and finishes the proof of the claim.
Recall that the generalized Heisenberg group of rank n is the group admitting the following presentation H n " xa 1 , . . . , a n , b 1 , . . . , b n | ra i , b j s " ra 1 , b 1 s, ra i , a j s " rb i , b j s " 1, 1 ď i ď j ď ny N 2 , where xy N 2 denotes a presentation in the variety of nilpotent groups of class 2.

Corollary 4.12.
Let G be a finitely generated non-virtually abelian nilpotent group of nilpotency class 2. Assume that G 1 has rank one. Then the ring Z is einterpretable in G, and DpGq is undecidable. On the other hand, single equations in G are decidable. This result applies in particular to any generalized Heisenberg group H n with n ě 1.
Proof. The undecidability part is a particular case of the previous Proposition 4.11. The decidability part is a consequence of Theorem 3 of [15].

Nilpotent quotient argument
In this section we develop the nilpotent quotient argument mentioned in the introduction and then apply it to Diophantine problems of finitely generated verbally elliptic groups. Theorem 4.13. Let G be a finitely generated group such that for some i P N γ i pGq is e-definable in G (in particular, if γ i pGq has finite verbal width) and G{γ i pGq is not virtually abelian. Then there exists a ring of algebraic integers O that is e-interpretable in G, and hence DpOq ď P DpGq.
Proof. If for some i P N γ i pGq is e-definable in G (for example, if γ i pGq has finite verbal width) and G{γ i pGq is not virtually abelian, then G{γ i pGq is a finitely generated nilpotent not virtually abelian group e-interpretable in G. Hence the conclusions of the theorem hold after applying Theorem 4.13 and transitivity of e-interpretations.
If a group G is verbally elliptic, then γ i pGq has finite verbal width for any natural i, and the nilpotent quotient argument applies. All the groups mentioned in the result below are verbally elliptic. Corollary 4.14. Let G be a finitely generated group which is either metabelian, or solvable minimax, or polycyclic, or virtually abelian-by-nilpotent, or (nilpotent minimax)-by-(abelian-by-finite). If G{γ i pGq is not virtually abelian for some i P N then there exists a ring of algebraic integers O that is e-interpretable in G, hence DpOq ď P DpGq.
A definition of a minimax group can be found at the end of Subsection 2.3.

Groups with maximal nilpotent subgroups
In this section we outline an approach to the Diophantine problem in a group G via the maximal finitely generated nilpotent subgroups of G (if such exist). With this in mind we denote by N max the class of groups where for all c ě 1, every set of c-nilpotent subgroups of G has a maximal element (with respect to inclusion). Here, and below, for brevity we use the expression c-nilpotent as a replacement of "nilpotent of class at most c".
where te 1 , . . . , e m u is a generating set of H. Indeed, since H is c-nilpotent, every element x P H satisfies (3). Conversely, if x P G satisfies (3) then xH, xy is cnilpotent by the observation above. Then by maximality we have x P H. Hence H is e-definable in G.
The following is the main technical result of our approach to groups from N max . Proof. Let H be a c-nilpotent non-virtually abelian subgroup of G. By Lemma 4.17, H is contained in a subgroup K ď G that is maximal among all c-nilpotent subgroups of G, containing H. Note that K is also a maximal c-nilpotent subgroup in G. This K is finitely generated due to Condition 3, so it is einterpretable in G by Lemma 4.15. Moreover K is not virtually abelian, since if A is an abelian finite index normal subgroup of K, then A X H is normal in H and abelian, and it has finite index in H (because H{A X H embeds in K{A), a contradiction. The result then follows by Theorem 4.6 and by transitivity of e-interpretations.
We now provide some properties that guarantee the existence of maximal nilpotent subgroups. 1. For all c ě 1, every set of c-nilpotent subgroups of G has a maximal element.
2. All abelian subgroups of G are finitely generated.
3. All solvable subgroups of G are polycyclic.
Proof. Clearly 1 implies 2, since the set of all finitely generated subgroups of a not finitely generated abelian group has no maximal element. Note that if all abelian subgroups of a solvable group are finitely generated, then the group is polycyclic (see, for example, Theorem 21.2.3 from [25]). Therefore if Condition 2 holds then every solvable subgroup of G is polycyclic, and so 2 implies 3.
Suppose now Condition 3 holds. Assume that there exists a set S of cnilpotent subgroups of G with no maximal element, so that S contains an infinite strictly ascending chain of c-nilpotent subgroups N 1 ă N 2 ă . . . The group N " Ť i N i is c-nilpotent, thus it is solvable, and hence it is polycyclic, and therefore finitely generated. This contradicts to the fact that the chain is infinite and strictly ascending. Hence 3 implies 1.
Remark 4.18. The result above shows that the class N max coincides with the class A max of groups, where each abelian subgroup satisfies max. The class A max was extensively studied, especially in the case of solvable groups (see [35]).
We next provide some examples of groups from N max .   This is true because any solvable subgroup of GLpn, Oq acts faithfully on the additive group of O n , which is a finitely generated abelian group, and so it is polycyclic (see, for example, Theorem 21.2.2 in [25]). Indeed, all discrete solvable subgroups of GLpn, Rq are finitely generated [1]. This is true, in particular, for abelian subgroups of a discrete subgroup G of GLpn, Rq.
We remark that not all finitely generated subgroups of GLpn, Rq are discrete in the induced topology. For instance, Baumslag-Solitar groups provide examples of finitely generated solvable linear groups that are not polycyclic.

Example 4.24.
Let H 1 and H 2 be groups from N max . Then G " H 1˚H2 is also in N max .
Indeed, by Kurosh subgroup theorem, any subgroup A of G has the form F˚A 1˚¨¨¨˚Am for some m ě 1, some free group F , and some subgroups A 1 , A 2 each of them conjugates to a subgroup of either H 1 or H 2 . In particular, if A is abelian then either A is infinite cyclic or A is a conjugate of a subgroup of either H 1 or H 2 . In both cases A is finitely generated, given that all abelian subgroups in the groups H 1 and H 2 are finitely generated.
Indeed, let A be an abelian subgroup of G. The canonical projections π 1 pAq and π 2 pAq on G 1 and G 2 are abelian. Since G 1 , G 2 P N max the subgroups π 1 pAq and π 2 pAq are contained in some maximal abelian subgroups A 1 ď G 1 and A 2 ď G 2 , which are finitely generated. It follows that the abelian subgroup A 1ˆA2 of G is also finitely generated. To finish the proof it suffices to note that A ď A 1ˆA2 . Indeed, denote by g Ñḡ the canonical epimorphism G Ñ G{N . Let A be an abelian subgroup of G. Then the imageĀ of A in G{N is abelian, and hence finitely generated, sayĀ " xā 1 , . . .ā n y, for some elements a 1 , . . . , a n P A. The subgroup A X N is an abelian subgroup of N , so it is finitely generated, say A X N " xb 1 , . . . , b m y. Then A " xa 1 , . . . , a n , b 1 , . . . , b m y.
Example 4.27. [35] Let G P N max . If A is an abelian normal subgroup of G then G{A P N max .

it is closed under quotients by abelian normal subgroups.
The following refinement of Theorem 4.6 is an immediate consequence of Theorems 4.16 and 4.28.
Corollary 4.29. Let G be a finitely generated virtually nilpotent group that is not virtually abelian. Then there exists a ring of algebraic integers e-interpretable in G, and DpOq ď P DpGq. If otherwise, G is virtually abelian then DpGq is decidable.
Proof. Let G be one of the groups GLp3, Zq, SLp3, Zq, or T p3, Zq. Then, as mentioned above, G P N max . Observe that G contains a finitely generated nonvirtually abelian nilpotent subgroup U T p3, Zq of nilpotency class 2. Suppose we know that ZpGqU T p3, Zq is a maximal 2-nilpotent subgroup of G. Then it follows by Lemma 4.15 that ZpGqU T p3, Zq is e-definable in G. Notice that ZpGq is also e-definable in G. Therefore, the quotient group N " ZpGqU T p3, Zq{ZpGq is e-interpretable in G. Since ZpGq X U T p3, Zq " 1, the group N is isomorphic to U T p3, Zq. Thus, U T p3, Zq is e-interpretable in G. By Theorem 4.9 the ring Z is e-interpretable in U T pn, Zq, hence in G by transitivity of e-interpretations. Hence the Diophantine problem in G is undecidable.
It remains to prove that N is a maximal 2-nilpotent subgroup of G. Assume towards contradiction that this is not the case, and let x P G N be such that N x " xN, xy is 2-nilpotent. Since N x is infinite, so is ZpN x q, and thus ZpN x q contains an element g of infinite order. We can write g " g 0 g 1 for some g 0 P ZpU T p3, Zqq and some g 1 P ZpGq. Since all elements of ZpGq have order at most two, we have g 2 P ZpU T p3, Zqq " xt 1n p1qy, where by t ij pαq we denote the transvection matrix with ones on the diagonal, α in the entry pi, jq, and zeros everywhere else (1 ď i, j ď n, α P Z). Hence there exists α P Zzt0u such that t 1n pαq P ZpN x q. From the identity rt 1n pαq, xs " 1 we obtain that x P T p3, Zq (all entries below the main diagonal are zero), and the diagonal of x is pε, δ, εq for some ε, δ P t´1, 1u. Observe that if G " SLp3, Zq then we are done, since then ε " δ " 1 and N x " N , a contradiction.
Next, denote by y the matrix with diagonal p1,´1, 1q and with zeros everywhere else. Observe that xN x " yN x (this can be seen by multiplying x by transvections tt ij u 1ďiăjďn and by elements from ZpGq), which implies that y P N x . We now use y in order to prove that N x is not nilpotent. Indeed, let t p0q be the element from U T p3, Zq with entries t p0q 12 " t p0q 23 " 1 and t p0q 13 " 2. Define t pi`1q " rt i , ys " rt, y, i . . ., ys, for all i ě 0. It follows by induction on i that t piq 12 " t piq 23 " 2 i and t piq 13 " 2 2i`1 for all i ě 0. Therefore, N x cannot be 2-nilpotent, a contradiction.

Polycyclic groups
We next consider the Diophantine problem in polycyclic groups.

Theorem 4.31. Let G be a virtually polycyclic group that is not virtually metabelian. Then there exists a ring of algebraic integers O that is e-interpretable in G, and
DpOq ď P DpGq.
Proof of Theorem 4.31 Let G be as in the statement of the theorem, and let G 0 be a finite-index polycyclic normal subgroup of G. Any polycyclic group is (nilpotent-by-abelian)-by-finite (see §2 Theorem 4 of [61]). Thus there exists a chain of subgroups N H G 0 G such that G{G 0 and G 0 {H are finite, H{N is abelian, and N is nilpotent. If N is not virtually abelian then there exists a ring of algebraic integers O that is e-interpretable in G, by Corollary 4.28 and Examples 4.23, 4.26.
We claim that if N is virtually abelian, then G is virtually metabelian. The proof of the theorem will be complete once this claim is proved. We shall use two observations: 1. Virtually polycyclic groups of finite exponent are finite (this is well known for nilpotent groups, hence it holds for nilpotent-by-abelian, and for their finite extensions).
2. Any normal subgroup or a quotient of a virtually polycyclic group is again virtually polycyclic.
Now suppose N has an abelian normal subgroup A such that A{N is finite of order say n. Then H P pAE n qA " ApE n Aq (since the product of varieties is an associative operation -see Subsection 2.2). By Items 1 and 2 we obtain that H P ApFAq. Hence H P ApAFq. In particular H P ApAE m q for some m, and by the same reasons as before we have that H P pAAqF. Thus G 0 P ppAAqE k qE t for some k, t, and similarly as before we obtain G 0 P pAAqpE k E t q Ď pAAqF. Now the same argument yields G P pAAqF.
We refer to the introduction for comments regarding the Diophantine problem of polycyclic metabelian groups. See in particular Problem 1.11.

Free solvable-by-nilpotent groups
We finish the paper by studying systems of equations in finitely generated free solvable-by-nilpotent groups. Recall that by this we mean relatively free groups in the product A d N c of the varieties A d and N c , where A d is the variety of all solvable groups of class d and N c is the variety of all nilpotent groups of class ď c, for c, d ě 1. These are precisely the groups of the form F {pγ c pF q pdq q, where F is a free group.
The following auxiliary lemma is an immediate consequence of a result due to Malcev [40]. Proof. Let G " F {N 1 . By [40] we have that C G pgN 1 q " N {N 1 for any gN 1 P N {N 1 . Thus N {N 1 is e-definable in G, and consequently the quotient F {N is e-interpretable in G.
We start by studying free solvable groups. Theorem 4.33. Let G be a finitely generated nonabelian free solvable group. Then the ring Z is e-interpretable in G, and DpGq is undecidable.
Proof. Proceed by induction on the derived length d of G. If d " 2 then G is a f.g. free metabelian group. In this case G is verbally elliptic and G{γ 3 pGq is a finitely generated free 2-nilpotent group e-interpretable in G (see Remark 2.11). By Theorem 4.7, the ring Z is e-interpretable in G{γ 3 pGq, and thus in G by transitivity of e-interpretations. Now assume d ě 2. Note that G " F {F pd`1q for some free group F . Hence G{G pdq " F {F pdq is e-interpretable in G, by Lemma 4.32 taking N " F pdq . The quotient G{G pdq is a finitely generated free solvable group of derived length d´1, e-interpretable in G. Thus by induction the theorem holds for G{G pdq , and then it holds for G as well by transitivity of e-interpretations.
The previous results can be combined to prove the following generalization of Theorems 4.7 and 4.33.
Theorem 4.34. The ring Z is e-interpretable in any nonabelian free (solvableby-nilpotent) group G, and DpGq is undecidable.
Proof. We have G " F {pγ c pF q pdq q for some nonabelian free group F and some integers c ě 1 and d ě 1. We may assume that c ě 2, otherwise G is a free solvable group and the result follows by the previous Theorem 4. 33. Proceed by induction on d. If d " 1 then G is free nilpotent and the result is precisely Theorem 4.7. Hence suppose d ě 2 and let N " γ c pF q pd´1q . Then by Lemma 4.32, F {N is e-interpretable in F {N 1 " G. By induction the ring Z is einterpretable in F {γ c pF q pd´1q " F {N , and the result now follows by transitivity of e-interpretations.
The following is a consequence of the above Theorems 4.33, 4.34 and Corollary 2.9.