Geometric Loci and ChatGPT: Caveat Emptor! ^†

Abstract

We compare the performance of two systems, ChatGPT 3.5 and GeoGebra 5, in a restricted, but quite relevant, benchmark from the realm of classical geometry: the determination of geometric loci, focusing, in particular, on the computation of envelopes of families of plane curves. In order to study the loci calculation abilities of ChatGPT, we begin by entering an informal description of a geometric construction involving a locus or an envelope and then we ask ChatGPT to compute its equation. The chatbot fails in most situations, showing that it is not mature enough to deal with the subject. Then, the same constructions are also approached through the automated reasoning tools implemented in the dynamic geometry program, GeoGebra Discovery, which successfully resolves most of them. Furthermore, although ChatGPT is able to write general computer code, it cannot currently output that of GeoGebra. Thus, we consider describing a simple method for ChatGPT to generate GeoGebra constructions. Finally, in case GeoGebra fails, or gives an incorrect solution, we refer to the need for improved computer algebra algorithms to solve the loci/envelope constructions. Other than exhibiting the current problematic performance of the involved programs in this geometric context, our comparison aims to show the relevance and benefits of analyzing the interaction between them.

1. Introduction

Large Language Models (LLM) [1] have attracted a lot of attention in recent times. A LLM known as ChatGPT [2] was released at the end of 2022, and its accuracy has been rapidly tested in different fields of knowledge [3,4,5,6,7,8,9], given its question-and-answer dialogue procedure. Mass media were primarily concerned with plagiarism and student cheating (see, for instance, [10,11]). Concerning interaction mathematics and ChatGPT, various studies have been reported. For instance, in [12] the authors investigate the mathematical capabilities of ChatGPT on standard and ad hoc datasets; mathematical word problems are considered in [13]; and an extensive list of ChatGPT failures, related to mathematics and other issues, is given in [14], to mention just a few references on this interplay.

Some renowned authors [15] stated that these machine learning programs cannot really replicate human understanding, and conclude that

ChatGPT and its brethren are constitutionally unable to balance creativity with constraint. They either overgenerate (producing both truths and falsehoods, endorsing ethical and unethical decisions alike) or undergenerate (exhibiting noncommitment to any decisions and indifference to consequences). Given the amorality, faux science and linguistic incompetence of these systems, we can only laugh or cry at their popularity.

Moreover, although, at the time of writing issues, such as privacy and the need for state regulations involving AI tools, are being discussed, no severe objections are posed regarding the use of ChatGPT in mathematics, to the best of our knowledge. Indeed, the main developer of Mathematica wrote a laudatory welcome to the chatbot [16], announcing a Wolfram|Alpha plugin for ChatGPT. It is highly arguable that we should rely on hidden proprietary codes to perform computations in mathematics, as stated by J. Neubüser, the developer of GAP (cited in [17]).

With this situation two of the most basic rules of conduct in mathematics are violated: in mathematics, information is passed on free of charge, and everything is laid open for checking.

ChatGPT has also attracted a lot of attention from the education field. A review [18] found that fifty related scholarly articles written in English were published in January and February of 2023 alone, and concludes that the chatbot’s performance is unsatisfactory in mathematics education. A very commendable article [19] on ChatGPT and mathematical education, although recognizing the existence of contrary opinions, states that “ChatGPT lacks a deep understanding of geometry and cannot effectively correct misconceptions”. We will not discuss the machine understanding issue, which would lead us to a philosophical debate involving the Turing Imitation Game. Despite these negative precedents, we think—and will exemplify in this paper—that exploring a specific geometric issue (and this should be actually considered the key contribution of the program: to help humans to explore, not to fully replace human inquiry) with ChatGPT will promote the expansion of our geometric knowledge.

Indeed, this argument is not new. The emergence of chatbots in the field of mathematics can be seen as somewhat similar to the introduction of symbolic calculation programs. It is worth noting a couple of works that dealt with the topic at the end of the 20th century. In one of them, a founder and leader of this area, Buchberger, presents [20] a didactical principle, the White-Box/Black-Box Principle for Using Symbolic Computation Software in Math Education, stating that, when a mathematics topic is new to students, the use of symbolic software should be prohibited; when it has been thoroughly studied and hand calculations involving concepts from such topics have been made, “students should be allowed and encouraged to use the respective algorithms available in the symbolic software systems”. Similarly, the recommendation given in [21] is that, prior to using symbolic computation software, students should thoroughly study the main theoretical issues in the topic under consideration and resolve related problems by hand (in order to keep fundamental mathematical operations and skills). They conclude with two rules of thumb that we summarize as follows:

• If you know how to do it, use computer algebra. If not, do not use it.
• Caveat emptor!

It is from this educational perspective, and with the aim of contributing to the exploration of the mathematical abilities and shortcomings of ChatGPT, that we study its performance in dealing with a few (but representative for their presence in the curriculum or in some recent research papers (see references below)) plane geometric loci and envelopes of families of plane curves, a popular topic both in the educational and mathematical research contexts. We remark that the issue of loci and envelopes, while thoroughly studied either from a theoretical point of view [22,23,24], or within the context of dynamic geometry systems [25,26,27,28], has not been approached, as far as we know, with AI chatbots. It is true that LLMs have been tested in relation to formal theorem proving in Euclidean geometry (see, for instance [29,30]), but the topic of the algorithmic computation of loci/envelopes is not, technically speaking, part of the theories addressing the automated deduction of theorems in geometry. Computing loci/envelopes concerns, rather, finding some missing necessary conditions (the description of the locus of a point as the result of establishing some given constraints over that point).

In our experiment, we used the free 3.5 version of ChatGPT from 25 September 2023. Some elementary, albeit non trivial, loci constructions were posed to the chatbot, and its results are discussed and compared with those obtained with a fork version of the popular, free dynamic geometry program GeoGebra [31], called GeoGebra Discovery, which includes rigorous automated reasoning tools based on computational algebraic geometry algorithms implemented in Tarski 1.30 and Giac, two embedded computer algebra systems  [32,33,34,35]. In the case were GeoGebra Discovery does not give the right solution, we provide a specific analysis of the situation, describing the need for implementing improved computer algebra algorithms.

We envisage two types of users when searching for geometric loci and envelopes either with ChatGPT or GeoGebra, mainly the undergraduate, or even secondary education student, faced with a proposed task in these issues. Secondarily, we envisage researchers in other fields with collateral tasks involving these geometric problems. So, the need for free access, either to ChatGPT (via its webpage) or to GeoGebra (downloading this free, multi-platform program), is a prerequisite in this work.

It should be highlighted that only the web-accessible ChatGPT page, https://chat.openai.com/ accessed on 6 December 2023, will be used throughout this paper. Thus, we specifically exclude considering others LLMs, as well as, concerning ChatGPT, experimenting with variations in temperature and top-p parameters. We also refrain from considering related ChatGPT prompting techniques, such as few-shot learning, tuning, and chain-of-thoughts, with the exception of the construction in Section 2.1. The rationale behind our decision is two-fold: on the one hand, fine tuning the model would require accessing the GPT-3 API, which might involve costs, thus losing the required characteristic of being free of charge. On the other hand, while maintaining prolonged dialogues with the chatbot could be illuminating, it could also be frustrating for a student, as well as a possible new source of errors, as shown in the construction in Section 2.1.

Next, Section 2 describes some geometric loci with both systems and provides an “ad hoc” ChatGPT-assisted procedure for suggesting how the user can input a GeoGebra locus construction. Section 3 considers some envelopes and discusses their solutions using, again, both systems. Finally, we conclude, analyzing the behavior of this collection of representative cases, that ChatGPT is not mature enough for us to rely only on its use, and we point to some ways teachers can guide learners when using the chatbot for these geometric problems.

2. Geometric Loci

A classic problem in plane Euclidean geometry consists in finding the path of an construction element that is subject to given constraints. In cases where the element is a point, we call the problem a “geometric locus”. The usual difficulties when mentally visualizing various objects with different movements lead to the use of computer software allowing for the graphical visualization of loci, mainly dynamic geometry software. Besides the graphical representation of loci, we obtain full knowledge of such loci when finding their analytical description. Thus, although ChatGPT does not currently have sophisticated graphic abilities, we study its competence when finding the analytic determination of loci.

2.1. Simple Geometric Loci

ChatGPT correctly identifies simple loci as circles and ellipses. For instance, when asked “Find the locus of plane points equidistant, say 1, from one given, say A(0, 0)”, it finds the equation $x^2+y^2=1$ and states it is a circle centered at the origin with a radius of 1. Similarly, when asked about “Find(ing) the locus of points which sum of distances to a pair of given points is constant”, it returns “The locus of points in a plane such that the sum of their distances to two fixed points (called foci) is constant forms an ellipse”, and, since no particular coordinates nor distances are given, the found equation is $\sqrt{{(x-x_1)}^2+{(y-y_1)}^2}+\sqrt{{(x-x_2)}^2+{(y-y_2)}^2}=\mathrm{constant}$, with obvious coordinate assignments.

Nevertheless, ChatGPT does not give an accurate answer, at a first sight, to the following simple construction (a graphical representation made with GeoGebra is shown in Figure 1, suggesting that the sought locus is a circle):

• Question Consider a circle centered at $A(2,1)$ and passing through $B(3,2)$. Place a moving point in this circle, C, join it with B and compute the midpoint D of $BC$. What is the geometric locus of D?

• ChatGPT Let us denote the center of the circle as $A(2,1)$, and a point moving on the circle as $C(x,y)$. Since C lies on the circle passing through $B(3,2)$, the distance between A and C is equal to the radius of the circle.

The distance formula between two points $(x_1,y_1)$ and $(x_2,y_2)$ is given by

$$d=\sqrt{{(x_2-x_1)}^2+{(y_2-y_1)}^2}$$

For $A(2,1)$ and $C(x,y)$ on the circle, the distance is equal to the radius of the circle. Let r be the radius:

$$r=\sqrt{{(x-2)}^2+{(y-1)}^2}$$

Now, we want to find the midpoint D of the line segment $BC$, where $B(3,2)$ is the other endpoint of $BC$. The midpoint D is given by the average of the coordinates of B and C:

$$D\left(\right(x+3)/2,(y+2)/2)$$

Substitute the expression for r from the equation of the circle into the coordinates of C to obtain the coordinates of D:

$$D\left(\right(x+3)/2,(y+2)/2)$$

$$r=\sqrt{{(x-2)}^2+{(y-1)}^2}$$

Now, substitute the expression for r into the coordinates of D:

$$D\left(\right(x+3)/2,(y+2)/2)=\left(\right(x+3)/2,(y+2)/2)$$

Now, you have the coordinates of the midpoint D in terms of the variables x and y. If you simplify further, you might find a specific geometric shape for the locus of D, but the exact form depends on the simplification of the expression.

The computations are correct, although the last substitution is fruitless. Furthermore, eliminating the coordinates of C, $x,y$ in the following equations,

$$x_D=(x+3)/2,y_D=(y+2)/2,\sqrt{{(x-2)}^2+{(y-1)}^2}=2$$

the chatbot would obtain the sought locus. Nevertheless, no clue about the locus is given. Let us follow the advice on further simplification:

• Question Simplify the expression.
• ChatGPT Let us simplify the expression for the coordinates of the midpoint D:

  $$D\left(\right(x+3)/2,(y+2)/2)$$

Now, we know that $r=\sqrt{{(x-2)}^2+{(y-1)}^2}$, so substitute this into the coordinates of D:

$$D\left(\right(x+3)/2,(y+2)/2)$$

$$r=\sqrt{{(x-2)}^2+{(y-1)}^2}$$

$$(x+3)/2=1/2\sqrt{{(x-2)}^2+{(y-1)}^2}$$

Now, let us eliminate the denominators by multiplying both sides of the equation by 2:

$$(x+3)=\sqrt{{(x-2)}^2+{(y-1)}^2}$$

Square both sides to eliminate the square root:

$${(x+3)}^2={(x-2)}^2+{(y-1)}^2$$

Expand and simplify:

$$x^2+6x+9=x^2-4x+4+y^2-2y+1$$

Now, simplify further:

$$6x+9=-4x+y^2-2y+5$$

Move all terms to one side:

$$10x+4y^2-2y+4=0$$

So, after simplifying, the geometric locus of the midpoint D is given by the following equation:

$$10x+4y^2-2y+4=0$$

This is the equation that describes the locus of the midpoint D as the point C moves on the circle. The exact geometric shape of this locus might be evident with further analysis or manipulation.

There is a bizarre mistake in line 7 of the answer, where it is stated that $(x+3)/2=1/2\sqrt{{(x-2)}^2+{(y-1)}^2}$, which is obviously wrong. So the locus is not the correct one. Furthermore, it should be noted that is not identified as a parabola. We need a further interaction to obtain the right result:

• Question: Eliminate x and y in the equations $x_D=(x+3)/2, y_D=(y+2)/2,{(x-2)}^2+{(y-1)}^2=2$.

Finally, we obtain, after renaming the coordinates of D and suppressing the verbose output of ChatGPT,

$${(2x_D-5)}^2+{(2y_D-3)}^2=2.$$

ChatGPT does not have direct access to external services or computational tools, but it can help when describing GeoGebra procedural constructions. For instance,

• Question: Give a procedural description in GeoGebra’s input bar of the above construction, using the LocusEquation Command. Use the Execute command in GeoGebra to provide the description.
• ChatGPT: Certainly! Below is a procedural description for the construction using GeoGebra’s input bar. Note that GeoGebra does not have a direct Execute command, but you can input commands directly into the input bar.

• Open GeoGebra: Open the GeoGebra app on your device.
• Create Points A and B: Input $A=(2,1)$ in the input bar and press Enter. Input $B=(3,2)$ in the input bar and press Enter.
• Create the Circle: Input c = Circle(A, B) in the input bar and press Enter.
• Create a Moving Point C: Input C = PointOn[c] in the input bar and press Enter.
• Join Points B and C: Input $BC$ = Line(B, C) in the input bar and press Enter.
• Compute Midpoint D: Input D = Midpoint[$BC$] in the input bar and press Enter.
• Use the LocusEquation Command: Input LocusEquation($D,C,1$) in the input bar and press Enter. This command tells GeoGebra to create a locus equation based on the movement of point C with respect to the midpoint D, and 1 represents the constant sum of distances.
• Animate the Movement of Point C: Right-click on point C. Select “Animation Options”. Choose “Rotate” or “Translate” to make point C move around the circle.
• Observe the Locus: As point C moves around the circle, GeoGebra will dynamically display the locus of the midpoint D.

By entering these commands into the input bar, you can construct and observe the dynamic locus of the midpoint D as point C moves within the given circle in GeoGebra.

Thus, entering

  Execute({"A= (2, 1)", "B = (3, 2)", "c = Circle(A, B)",

  "C = Point[c]", "BC = Line(B, C)", "D = Midpoint[B,C]",

  "LocusEquation(D,C)"})

• in GeoGebra returns the locus graphic object and its equation (Figure 2). There are some mistakes in ChatGPT’s translation: PointOn should be just Point, Midpoint[BC] shoul be Midpoint(B,C), and LocusEquation should only accept one or two arguments. Moreover, GeoGebra indeed has the Execute command.

2.2. The Wallace–Simson Theorem

Consider a triangle $ABC$. The Wallace–Simson theorem states that the locus of all those points P is in its plane, such that the orthogonal projections of P on the three sides of the triangle are collinear is the circumcircle of $ABC$. See [26] for an algorithmic, algebraic geometry, approach, and generalizations.

Despite ChatGPT knowing this statement, we will investigate this result without mentioning its name to ChatGPT. Posing the question “Consider a triangle $A(0,0)$, $B(1,0)$, $C(1,1)$, and a point P. Trace the orthogonal projections from P to the sides of the triangle. Find the locus of point P such that the orthogonal projections are collinear”, the chatbot returns the equation of a parabola, $-x^2-x-y=0$, instead of the circumcircle of the triangle. ChatGPT adequately treats the problem, but there is a calculus mistake when computing the determinant for the collinearity of points. In order to improve the readability of this paper, the trace of ChatGPT interactions will appear in an appendix from now on.

As carried out above, ChatGPT can give a procedural description of the construction (see Appendix A), which can guide a non-expert GeoGebra user to compute the locus in the dynamic software. Yet, in the current state of the art, we think it is still not worth designing a ChatGPT–GeoGebra interface for automating the translation of constructions, given the impressibility of the chatbot when curating its answers, (thus making the interface probably unuseful), and the fact that it commits frequent errors when writing GeoGebra code. Finally, entering

Execute({"A = (0, 0)","B = (1, 0)","C = (1, 1)","P = (-1,-1)",

"f = Line(A, B)", "j = PerpendicularLine(P, f)",

"D = Intersect(f, j)", "g = Line(A, C)",

"k = PerpendicularLine(P, g)", "E = Intersect(g, k)",

"m = Line(B, C)", "n = PerpendicularLine(P, m)",

"F = Intersect(m, n)", "LocusEquation(AreCollinear(D, E, F), P)"})

• into the GeoGebra input bar, we obtain the sought graphic locus and its equation (Figure 3). Note that we could not use the Locus command; since P does not lie on any known object, it is subject to an a posteriori condition.

2.3. Sketchpad Classic Locus Construction

We consider a locus studied by Sutherland in his report on Sketchpad [36]. This locus is erroneously computed by ChatGPT, and GeoGebra returns an inaccurate answer. The construction can be defined as follows:

Let $A(0,0)$, $B(3,0)$ and $C(1,3)$ be three fixed points, and let a and b be two lines passing, respectively, through A and $E(-1,-1)$, and B and $D(5,-1)$. Let c be the circle with center C and radius 5. Consider a moving point G on the circle c. The line $CG$ intersects a in a point I and b in a point H. Find the locus of the intersection point J of lines $AH$ and $BI$.

ChatGPT correctly understands the task (see Appendix B), but finds that lines $AH$ and $BI$ are parallel, concluding that their intersection point I does not exist. When asked about a GeoGebra procedural description, it gives a step-by-step list, assuming user familiarity with GeoGebra. With a slight change, entering

Execute({"A = (0, 0)", "B = (3, 0)", "C = (1, 3)", "E = (-1, -1)",

"D = (5, -1)", "circleC = Circle(C, 5)", "G = Point(circleC)",

"a = Line(A, E)", "b = Line(B, D)", "I = Intersect(Line(C, G), a)",

"H = Intersect(Line(C, G), b)", "AH = Line(A, H)",

"BI = Line(B, I)","J = Intersect(AH, BI)", "LocusEquation(J, G)"})

• into GeoGebra, we obtain the locus (shown in red in Figure 4). But this conic is not the searched locus; dragging point G to make line $CG$ parallel to line a, makes line $BI$ change, approaching a limit position parallel to $CG$ and a. Thus, $BI$ and a do not intersect, I goes to infinity and J is undefined. The point $(1/2,-5/2)$ is not in the locus! Things are analogous when dragging point G to make line $CG$ parallel to line b, thus causing point $(7/2,-7/4)$ to be removed from the locus. GeoGebra is unaware of this subtlety and returns the whole ellipse as the locus. GeoGebra locus internals only deal with polynomials, that is, algebraic varieties. In order to correctly describe this locus, it would be necessary to manage different varieties, i.e., locally closed sets (see [22] for an in–depth discussion of this locus).

Table 1. Behavior of ChatGTP and GeoGebra with the proposed loci.

	ChatGPT	GeoGebra
Example 1	✔ (needs user interaction)	✔ equation, ✔ plot
Wallace–Simson	×	✔ equation, ✔ plot
Sketchpad	×	∼ equation, ✔ plot

summarizes the behavior of ChatGPT and GeoGebra when dealing with the loci in this section. ∼ means that the result is almost correct. Furthermore, given the difficulties of computer algebra systems when plotting isolated points, we give a ✔ even if a finite number of points is not included in the graphic object.

3. Envelopes of Families of Plane Curves

The informal idea of envelopes involves contact and tangency. A curve is the envelope of a family of curves if it touches every curve in the family. Formally, following [24], the envelope $E_1$ of a family of plane curves in the real $(x,y)$-plane $F(x,y,t)=0$, parametrized by $t\in \mathbb{R}$, is defined as the set

$$E_1=\{(x,y)\in {\mathbb{R}}^2:\exists t\in \mathbb{R},F(x,y,t)=\frac{\partial F}{\partial t}(x,y,t)=0\}.$$

If the family of curves depends on a point moving on an object, the family will be parametrized by the coordinates of this point. In this case, the family will be described by $F(x,y,t_1,t_2)=0$, where parameters $t_1$ and $t_2$ are constrained by the restriction of point $(t_1,t_2)$ in moving along a one-dimensional path in the plane, that is, by adding an extra equation $g(t_1,t_2)=0$. In this case, the defining condition in envelope $E_1$ is to be replaced by

$$E_1=\{(x,y)\in {\mathbb{R}}^2:\exists t_1,t_2\in \mathbb{R},F(x,y,t_1,t_2)=\frac{\partial F}{\partial t_1}\frac{\partial g}{\partial t_2}-\frac{\partial F}{\partial t_2}\frac{\partial g}{\partial t_1}=0\}.$$

For completeness, it should be highlighted that three other envelope definitions are given in [24]. A second notion of envelope $E_2$ is considered as the curve tangent to $F_t(x,y)=F(x,y,t)$, for each t. A new idea of envelope $E_3$ is also presented as the limit of intersection points of nearby curves $F_t$. We noted that ChatGPT sometimes uses this definition. Finally, the notion of envelope $E_4$ is outlined as the boundary of the region filled by curves $F_t$. Furthermore, it can be shown that $E_i\subseteq E_1$ for $i=2,3,4$ (see [37] for a detailed discussion).

3.1. The Envelope of Kissing Circles

Let $A(0,0)$ be the center of a circle passing through $B(1,0)$. Find the envelope of the family of circles centered at a moving point C lying on the first circle and with radius 1. Figure 5 shows the GeoGebra construction of the family of circles.

It seems easy to state that the envelope is the circle centered at A and with a radius of 2, following $E_2$ or $E_4$, but things are a bit more complicated. ChatGPT answers wrongly, again, stating that the envelope is the y-axis (see the Appendix C). GeoGebra correctly finds the exact envelope. Entering

 Envelope(D,C)

• in the input bar of the construction, shown in Figure 6, we obtain a long equation in two variables with a degree of 12 in the Algebra window. Factoring it with any computer algebra software or by inputting

 Factor(LeftSide(eq1)-RightSide(eq1))

• in GeoGebra, we obtain

  $$\begin{array}{cc}& (x^2+y^2)(x^2+y^2-4)\hfill \\ & \hfill (x^2+y^2-4y+4)(x^2+y^2+4y+4)(x^2+4x+y^2+4)(x^2+4x+y^2+4),\hfill \end{array}$$

  which corresponds to the point $(0,0)$, the circle centered at the origin with a radius of 2, and points $(\pm 2,0)$ and $(0,\pm 2)$, respectively. Note that the last four points are included in the circle and they appear due to the ideal–theoretic computations used by GeoGebra. Furthermore, the envelope graphic object does not include the origin. Finally, moving point B to have both non-integer coordinates, the equation is now a quartic, as expected, but GeoGebra cannot factorize the expression and the graphic object does not contain the isolated origin point. Thus, although GeoGebra behaves reasonably, some precautions should be taken.

3.2. The Deltoid of Steiner

In the preceding section, we tested the Wallace–Simson theorem as a locus problem with ChatGPT, obtaining a wrong answer, and with GeoGebra, which was successful. A result derived from this theorem studies the Simson lines of a triangle, that is, the lines containing the feet of the perpendicular projections to the sides of the triangle. Considering a point moving along the circumcircle of the triangle, we can find the envelope of the family of the Simson lines. As is well known, such envelopes exist and are the hypocycloid of three cusps, called the tricuspoid curve or Steiner curve, a plane algebraic curve of a degree of four.

ChatGPT returns a correct guideline to solve the problem (see Appendix D), but does not find the envelope. Instead, it recommends using computer algebra systems or specialized software.

For its part, GeoGebra computes an envelope (Figure 6), graphs it and declares its equation to be a quintic, against the expected quartic one. Note that x is a factor, so we have x and a polynomial of a degree of four, which is the true envelope. Why does the anomalous factor x appear? The Simson line l is defined as the line passing through E end G, and, when D and C coincide, so do the feet E and G, leaving the Simson line undefined. Despite the sophisticated algebraic geometry algorithms included in GeoGebra, it does not include the one described in [22], which is the only one that could distinguish between the two factors, thus allowing for the dropping of factor x.

Table 2. Behavior of ChatGTP and GeoGebra with the proposed envelopes.

	ChatGPT	GeoGebra
Kissing circles	×	✔ equation, ✔ plot
Steiner’s deltoid	×	× equation, × plot

summarizes the behavior of both systems with the two proposed envelopes. Despite the fact that GeoGebra deltoid results include the right answer, we give a × mark since there is a non-trivial extra factor in the equation and, regarding the plot, there is an infinite number of spurious points in the graphic object, the y-axis in Figure 6.

4. Conclusions

We tested, through some selected (but relevant, i.e., appearing in different research contexts and references) examples of plane geometric loci and families of plane curves, the abilities of ChatGPT when dealing with the computation of such loci or envelopes. Except in some simple cases, we showed that the chatbot is, at best, plausible, but not reliable. Either due to its making arithmetic errors, or by simply sketching a general guideline for solving the task, we cannot currently delegate ChatGPT for automating the search for loci. We also compared the ChatGPT results with those of GeoGebra, showing that the latter clearly surpasses the former for this particular task. However, we have also exhibited some cases showing that neither system is fully reliable.

Although it might seem that this statement is too audacious, given the small number of test cases we have analyzed, the failures we have observed are not due to unexpected errors or bugs in the design of these systems, but due to the inherent limitations in their algorithmic conception in addressing some subtle issues related to loci computations, such as, for instance, not being programmed to address, with mathematical formalism, the computation of loci or to output loci that are constructible (and not algebraic) sets. The required additional work will consist of training LLMs refined in this topic, while incorporating more powerful algorithms from algebraic geometry into GeoGebra. Until now, only specialized human knowledge has seemed capable of solving general problems in this field, with a reasonable success rate.

But this disappointing panorama can be also a source of interesting learning situations. As we hope to have clearly exemplified in the previous sections of this study, the analysis of ChatGPT’s answers and a comparison with those of GeoGebra Discovery in addressing a similar task, requires a deep and precise understanding of the basic geometry (locus, envelopes), computer algebra (e.g., Gröbner basis) and algebraic geometry (e.g., the inclusion of degeneracies, constructible sets) concepts and tools that are involved in each of the problems. And this understanding is predicated upon a previously untraveled path, as well as a non-guided exploration of the surprising, and often contradictory, outputs, whether contradictory or surprising when comparing ChatGPT and GeoGebra Discover results, or with respect to the expectations of human intuition. In summary, we think that the dialog between ChatGPT and GeoGebra Discovery can provide a rich context for developing diverse kinds of mathematical skills: problem solving, creativity, geometry, algebra, etc.

Yet, as a final remark, in connection with the title of this paper, Caveat emptor!, we would like to insist that, currently, using ChatGPT as a help for teaching situations, or even for doing mathematical research, could be valuable, but always under guidance and with caution. We reformulate the above-mentioned White–Box/Black–Box Principle for using AI chatbots in finding loci/envelopes as follows: users (students or researchers) should be aware that any results are to be taken with skepticism.