GPT-4 was just released. Here’s a preview of what it’s capable of. I tried throwing some mathematics problems at it to check out its capability.

Problem 1: Cheryl’s Birthday Problem

There’s an infamous logic problem from Singapore’s primary school Olympiad training. Here’s the full problem from the Wikipedia page (quoted verbatim).

And… here’s the output.

That’s correct: the first line of deduction tells us the month is July or August.

Correct!

Ok, but this is a well-known viral problem. Maybe let’s try something a little different.

Problem 2: Combinatorics

Here’s the problem.

I’d say it’s a decent inter-school level type of problem. Here’s the output:

A little of a bummer, since it incorrectly thought there must be four +1 and four -1 in each row. But the penultimate paragraph is impressive. So maybe a little nudging would help?

Result:

Extremely impressive!

Problem 3: Construction Problem

Here’s a rather tricky problem. [ Hint: the smallest solution is close to a million. ]

This is its reply:

The astute reader immediately sees the problem with this “solution”: 1118 has sum of digits = 11, which is a multiple of 11. I pointed it out; GPT-4 then responded with another 4-digit (wrong) example. I pointed out again and it responded with another wrong example. At this point, I gave up.

To avoid cluttering up the reader’s bandwidth, I won’t be sharing those wrong answers.

Problem 4: PSLE Coin Problem

This problem from PSLE 2021 again caused some controversy due to its perceived difficult:

Here’s part of the answer I got:

Right at the start it made the wrong assumption: that Helen and Ivan have the same number of 50-cent coins. I pointed this out:

Then GPT-4 realised its mistake:

But it still wouldn’t tell me who has more money. Guess wealth is a sensitive question even for a few dozen bucks.

I nudged it further:

And:

Goal!!

Problem 5: Group Theory

Here’s one which can be quite tricky if you’re unfamiliar with concrete examples of groups.

Long story short: it tried to come up with three examples, failed, and concluded that such an example is impossible. What impressed me was the diversity of its examples. It tried to use:

1. .
2. The free group .
3. The dihedral group of order 8.

The astute reader should immediately know why each of these examples fails.

This post is only tangentially related to mathematics. As an experiment on AI-assisted learning, I tried to write a web application in Javascript + WebGL. At the start of this experiment, I had some experience with Javascript but absolutely no knowledge of WebGL. My objective was to program a simple interface for manipulating a Rubik’s cube.

Within a few hours, I ended up with this:

https://lzw75.github.io/webgl/rubik_webgl.html

Instructions:

• Use the 4 arrow keys to rotate the entire cube.
• Hover your mouse cursor over the white “buttons” and use the mouse scroller to rotate the faces.

The following is a summary of how I picked up some basics of WebGL by constantly polling ChatGPT.

First I needed to know how to draw a cube, so I asked:

Help me write an HTML file using WebGL to display a cube. Allow the reader to vary the viewing angles.

ChatGPT replied with a piece of code to do the above. The resulting code let the user change the orientation of the cube via the 4 arrow keys.

Interestingly, the code also rotated the cube continuously:

var render = function () {
   requestAnimationFrame(render);

   cube.rotation.x += 0.1;
   cube.rotation.y += 0.1;

   renderer.render(scene, camera);
};
render();

I did not ask for this functionality but it turned out to be useful later.

My first idea was to have a single cube, with each face partitioned into 3 x 3 squares. But I wasn’t sure how to rotate each face with this geometry. Instead, I decided to have 27 cube objects. Next, I needed to have a different colour for each face. So I asked:

I want a cube with all six sides of different colors.

ChatGPT replied with the appropriate code, but the scene turned out black. So I responded:

I tried it but the cube isn’t appearing.

Happily, ChatGPT managed to point out the problem: it listed four possible causes, and I quickly identified the third one (lack of light source) to be the right one. ChatGPT even amended its original code to add a new light source, and lo and behold, it now worked!

Next I needed to add 27 cubes, so I asked:

How do I add 3 cubes of the same size in different parts of space?

With the reply, I could create the 27 cubes required to form a Rubik’s cube. But I recalled in many applications, one can form groups of objects for easier manipulation. So:

In WebGL, can I form a composite object comprising of three cubes?

ChatGPT suggested creating multiple THREE.Mesh objects, then putting them together in a THREE.Group object. That was exactly what I wanted.

Now to rotate a face of the cube, I needed to first identify which cubes to rotate. In each case, 9 cubes were chosen out of 27 for rotation. For that I needed the following.

In WebGL how do I access sub objects in a group object?

But ChatGPT didn’t give me what I wanted, because I had for some reason started a new conversation. The reply was ambiguous and unhelpful. So I tried again:

I have a WebGL object named “cubes” of from “THREE.Group()”. How do I access the third element in the list?

Note the grammatical mistake in the first sentence (that was completely unintentional!). Thankfully, ChatGPT correctly interpreted my intentions and gave me the following sample code (together with a rather long explanation and other variants):

let cube = cubes.children[2];

Exactly what I needed. My next thought was to put the 9 cubes in a separate group, then rotate the group. So I asked:

In WebGL, can a single cube belong to two “THREE.Group” objects simultaneously?

ChatGPT replied in the positive so I went about to edit my code, adding the 9 cubes to a separate group for rotation. But then I faced a problem:

In the following WebGL code:

function move1(all_cubes) {
var cube_list = all_cubes.children;
var tmp_group = new THREE.Group();
for(var i = 0; i < cube_list.length; i++) {
console.log(cube_list[i].position);
if (cube_list[i].position.x < -0.99)
tmp_group.add(cube_list[i]);
}
}

The object “cube_list” had some of its objects removed, when I added them to the object “tmp_group”. Why?

ChatGPT explained that the objects were being removed from the all_cubes group because they were being added to the tmp_group group. I could overcome this by cloning the cube. Finding this too much a chore, I decided against this and just rotated the 9 cubes manually. Next:

How do I rotate a 3D WebGL object with centre of rotation specified by (x, y, z)?

And so on…

Summary

Thus the general outline seems clear. At each stage I had a mental image of what I wanted to do, broke down the process into specific subtasks, then queried ChatGPT for a very concrete implementation of the said subtask. Other than the above examples, I asked:

• how I could access the mouse scroller in Javascript,
• how I could do animation in WebGL (this took a few tries with different wordings, as each time, ChatGPT would reply with a different approach),
• if I could access the contents of an array produced by a function (yes, elementary question), etc.

Here are some problems I encountered.

• ChatGPT was often down due to heavy usage.

So I had to revert to Google. After one gets used to ChatGPT’s useful replies, Google’s search results seemed frustratingly irrelevant in comparison.

• The reply was sometimes truncated.

This was rare, but fortunately when it happened, the existing reply was sufficiently useful for my purpose. In any case, people have reported we could just type ‘continue’ and ChatGPT would display the remaining response. [ Warning: this does not work for code, arguably the most important case. ]

• The answer worked but it wasn’t satisfactory to me.

For instance, ChatGPT once suggested using the TWEEN library to animate face rotations, but I considered it too much of a hassle. It took a few more queries for me to coax ChatGPT into giving me a solution using ‘render()’. In retrospect, the first reply already provided me a possible approach.

• The answer was in a different context.

Sometimes ChatGPT gave a reply using a gl.* framework, but what I really wanted was the THREE.* framework. This could be solved by further specification (e.g. ‘reformulate the reply for a THREE.Cube object type’).

• The answer just plain didn’t work.

In one instance, ChatGPT gave the following output:

THREE.Quaternion().setFromRotationMatrix(rotationMatrix);

Cubes.forEach(function(cube){
   cube.applyMatrix4(rotationMatrix);
   cube.position.sub(point);
   cube.position.applyMatrix4(rotationMatrix);
   cube.position.add(point);
   cube.quaternion.multiply(rotationQuaternion);
});

which was utterly strange since it performed a rotation, then repeat the rotation about a pivot point, before performing the rotation for a third time (using quaternion representation no less). This could be fixed by simply understanding what’s going on and editing the code on your own.

Conclusion

I have no illusions about my level of proficiency in WebGL, but this experiment proves that ChatGPT lets us get to a functional level in a remarkably short time.

The Main Point

ChatGPT is an AI language model that has been making the news recently. There are multiple articles on the internet on what the model is capable of so this will not be the focus of our post. Instead, we wish to consider how ChatGPT can be useful in the context of mathematics.

Now “in the context of mathematics” is rather broad. Broadly speaking, there are three aspects to it:

• learning mathematics (new concepts, techniques etc),
• teaching mathematics,
• researching mathematics.

This post is mostly about the first two cases. Here, we care not for the philosophical implications, future directions, or subtle flaws of the bot. Instead, our main question is:

How can ChatGPT help us, as mathematics practitioners?

The Query

People have reported getting conceptually deep and insightful answers from ChatGPT in the form of essays. So we ask it, out of curiosity:

Why are compact spaces interesting in point-set topology?

This is the reply we get:

Compact spaces are an important concept in point-set topology because they have many useful properties. For example, every compact space is closed and bounded, which means that all of its points are contained within some finite region of the space. This makes compact spaces well-suited for studying problems in which the points of interest are contained within a bounded region.

Compact spaces also have the property of being “small” in a certain sense. For example, every infinite compact space must contain a “small” infinite subset, meaning that it is possible to find an infinite number of points in the space that are contained within a finite region. This property makes compact spaces a good choice for studying problems in which the size of the space is a consideration.

Finally, compact spaces have the property of being “well-behaved” in many ways. For example, every continuous function defined on a compact space must achieve its maximum and minimum values, which makes them useful for optimization problems. Compact spaces also have many other desirable properties, such as the ability to be approximated by finite sets and the existence of convergent sequences, which makes them a useful tool for studying problems in many different areas of mathematics and physics.

Newcomers to ChatGPT would be impressed by its coherence and flawless grammar. However, if you’d been experimenting with ChatGPT for a while, such an impression fades rather quickly. So to us, at first glance, the above reply by ChatGPT actually seems underwhelming. The statements are ambiguous and ill-defined; a mathematics student who wrote the above in an essay would likely get a bad grade for the report. But since we emphasised the term “at first glance”, there’s more to the story than meets the eye.

The Critique

First, let us critique the above essay as a teacher would.

1. “…every compact space is closed and bounded” is largely true, if we replace the word “space” with “metric space”. In generic topological spaces, the term bounded does not make sense. Thus the first paragraph is largely sensible.
2. The second paragraph is confusing. If we were to interpret it literally, it seems to be a strictly weaker variant of the first paragraph. Or, does it mean to say “compact spaces are often infinite, despite being contained in a finite region”? This actually brings up a nice contrast: although structurally finite (bounded), a compact set can be infinite, even uncountably so.
3. The third paragraph is not bad. Replace the phrase “every continuous function” with “every real-valued continuous function” and you’re good to go. But it’s not clear what is meant by “approximated by finite sets”. Does it refer to the definition of compact spaces, where every open cover has a finite subcover? Also the part about “existence of convergent sequences” is terribly worded; it should be “every sequence has a convergent subsequence”, which is indeed a critical property of compact sets.

After some editing, we get the following.

Compact spaces are an important concept in point-set topology because they have many useful properties. For example, every compact metric space is closed and bounded, so heuristically the whole set is in a finite region of space. This makes compact spaces well-suited for studying problems where the domain of interest is bounded.

Compact spaces thus can be considered “small”, at least on a structural level. However as a set, it can be infinite, even uncountably so. For example, the closed interval [a, b] under the usual topology is compact, even though it is an uncountably infinite set.

Finally, compact spaces are also “well-behaved” in many other ways. For example, every continuous real-valued function on a compact space must achieve its maximum and minimum values, which makes them useful for optimization problems. Compact spaces can also be “approximated by finite sets”, in the sense that every compact metric space can be covered by finitely many ε-open balls, no matter how small ε>0 is. Also, every sequence in a compact space has a convergent subsequence, so one can imagine compact spaces to be constrained structurally.

It still wouldn’t win any award for best writing, but at least it’s more accurate (or so we’d hope!).

The Summary

Proceeding in this direction, ChatGPT has the following applications.

1. A lay person can get a rough guide on a deeply related topic, provided said person is willing to take a huge bag of salt with it.
2. An imposter can feign deep knowledge by querying ChatGPT. In its current incarnation, this can indeed hoodwink the general lay person, but it only takes a few seconds for a domain expert to find multiple inaccuracies, poorly worded statements, or even downright falsehoods. [ Needless to say, we do not recommend such a use. ]
3. A domain expert can save time by asking ChatGPT to write a first version of a report before editing it.

In this case, ChatGPT already has a concrete benefit. The “approximate by finite sets” can be a good motivation for the definition of compact sets, which is often uninspired (so what if every open cover has a finite subcover?).

One wonders what the next version of ChatGPT brings.