ChatGPT and beam bending redux
July 24, 2025 at 9:07 PM by Dr. Drang
A couple of years ago, I asked ChatGPT to solve two simple structural analysis problems. It failed pretty badly, especially on the second problem, where it wouldn’t fix its answer even after I told it what it was doing wrong. Earlier this week, I decided to give it another chance.
Here’s my statement of the problem:
A cantilever beam of length L has a concentrated load P applied at its midpoint. The modulus of elasticity is E and moment of inertia is I. What is the formula for the tip deflection?
There was, as usual, a lot of blather in ChatGPT’s response, but paring it down, its answer was
which is correct. If you want to see why this is the answer, you can go look at my two-year-old post and—oh, who am I kidding? Here’s the problem:
The key to determining the deflection at B is to recognize that the left half of the beam is a cantilever beam in its own right, with a length of and a load at its tip. And the right half of the beam doesn’t bend at all, it’s just carried downward and tilted by its connection to the left half of the beam.
The deflection and rotation at the end of the left half of the beam (Point A) have well known solutions:
and
So
which is just what ChatGPT said.
Unfortunately, ChatGPT got cocky and finished its response with this:
If you’d like a derivation step-by-step or a diagram to visualize the moment curves and deflection, I can provide that as well.
I told it to explain how it got its answer, and that’s when things went south.
First, it divided the beam into two regions, just as I did. It called the left half Region I, the right half Region II and it set the origin of its x-axis at the fixed end. It then said the bending moments were
in the left half and
in the right half. Since both of these are wrong, I wondered how it got the right answer, so I continued reading.
In both halves, it integrated the nearly
to solve for the deflection function, .
The integration has to be performed twice in each half of the beam, so you end up with four constants of integration. Remarkably, ChatGPT showed the values for only two of the constants, and said the others could be determined through “standard algebra.”
Of course, if you apply standard algebra and solve for and , you’ll get values that don’t lead to the correct tip deflection. You can’t, because you started with the wrong bending moments.
So I had ChatGPT go through the lengthy algebra to get the values of and and then plug them into its equation for the tip deflection. It took a while, but eventually ChatGPT had an epiphany:
Yes, something’s definitely wrong. It went off on a long quest to find an algebra error in its work. But there was no algebra mistake, so it decided that its initial correct answer of
was wrong, and the correct answer was
My response:
No, your original answer is correct. Your equations for the moments in both regions are wrong. What I don’t understand is how you got the right answer in the first place. I suspect you found it somewhere, and didn’t actually go through any of the (incorrect) analysis you showed me when I asked for your reasoning.
In other words, all the “reasoning” it had been showing me was not merely wrong, it was bullshit. That was not how it got the original answer. During its lengthy reply, ChatGPT said it pulled its original correct answer from a “standard solution.” Standard seems to be its favorite word for indicating that it’s doing things right and we shouldn’t question it.
I asked where it got the correct answer, and it said it came from “standard [sic] references in structural mechanics,” including
- Roark’s Formulas for Stress and Strain.
- AISC Steel Construction Manual.
- AISC Design Guide 3.
- Mechanics of Materials by Gere and Timoshenko.
- Mechanics of Materials by Beer, Johnston, DeWolf, and Mazurek.
I have all five of these books, and none of them have the answer ChatGPT gave. But three of them—Roark, the AISC manual, and Gere & Timoshenko—include general formulas for a cantilever beam with a concentrated load somewhere along its length. A reasonable person could use the formulas in those books to get the correct answer by plugging in as the location of the load.
Once again, though, ChatGPT puts its foot in its mouth by making something up. Before I could ask for more specific references within these books, it gave me this:
My reply:
I have the 6th edition of Roark, and Table 8 doesn’t cover this problem. Did you make up that reference?
Well. After some hemming and hawing, it said that its answer came from this general formula in Roark,
where it plugged in and simplified.
Which would be a good answer if Roark had that formula in it, but it doesn’t. In the formula ChatGPT just gave, a is the distance of the load from the fixed end of the beam. Roark gives a different formula because it takes a to be the distance of the load from the free end. So another bullshit answer.
Oh, and ChatGPT said this formula is in Table 27 of Roark. Table 27 has nothing to do with beam bending.
A glutton for punishment, I continued my Socratic dialog with a stochastic parrot:
Again, Table 27 in Roark doesn’t cover beams of any kind. The formula you’re using is from Gere and Timoshenko’s book, which I also have. Tell me where it is in that book.
Once again, ChatGPT told me I was right and said the formula was in Chapter 5 of Gere & Timoshenko. You will be shocked to hear that Chapter 5 doesn’t cover deflections of beams. Chapter 7 does, but the formula isn’t there, either. It’s in Appendix G, a fact I was never able to extract from the conversation. In fact, ChatGPT ultimately decided the formula wasn’t in Gere & Timoshenko at all.
After scrolling through many more words of apology and justification, I asked this:
Throughout this conversation, I have been the source of your knowledge. How can you use the person asking the question as your source?
More apologies followed, along with an offer to do the derivation correctly. I took a deep breath, said go ahead, and… it did the problem correctly! Well, except for the sign error I mentioned in a footnote earlier in the post. Its “explanation” of the sign error was even more hand-wavy than any of its other bullshit explanations, and I abandoned the conversation. If you’re of a masochistic bent, you can see it all here.
So what did I get out of this exercise?
- OpenAI seems to have done a lot more plagiarism in the past two years. I own my copies of the references ChatGPT cited. I bet OpenAI doesn’t.
The plausibility of the answers has improved, especially when it comes to mathematics. You may have noticed in the screenshots that ChatGPT is formatting equations nicely now, which makes it seem more authoritative. Two years ago, the formulas it presented looked like this:
δ = (P * L^3) / (3 * E * I)
Of course, plausible doesn’t mean correct.
- ChatGPT still believes that long answers sound more reliable. In fact, it may be giving even longer answers than before. To me, this is frustrating, but maybe its regular users like it. Which doesn’t say much for them.
- It still makes up references, the classic hallucination problem. And it doesn’t seem to care that I told it I had the books it was citing—it still happily give me bogus chapters, tables, and page numbers. Perhaps it didn’t believe me.
- I suppose it’s possible that I’d get better answers from a better model if I were a paying customer. But why would I pay for something that can’t demonstrate its value to me?
- The most disturbing thing is its inability to give a straight answer when asked how it arrived at a solution. Maybe if I were Simon Willison, I’d know better how to get at its actual solution process, but I don’t think I should have to have Willison-level expertise (which I could never achieve even if I wanted to) to get it to explain itself.
Maybe I shouldn’t put so much emphasis on ChatGPT’s inability to explain itself. After all, I didn’t ask it to explain how it came up with the SQL code I wanted a few months ago. But to me, the code it gave me was self-explanatory; I could read it and see what it did and why. And I could test it. Not so with the beam deflection problem. Being able to explain yourself and cite proper references is the way ensure this kind of answer is correct.
-
There’s a sign error if you define positive deflections and moments in the usual way, which ChatGPT does. ↩
-
Credit to Emily Bender, et al. ↩
Saving links to Tot
July 20, 2025 at 11:03 AM by Dr. Drang
I mentioned on Mastodon how much I enjoyed Myke Hurley’s interview of John Gruber on the most recent episode of Cortex. One part that particularly resonated with me was when Gruber spoke of all the online bookmark managers he’s used over the years and how none of them have stuck. It was like I was listening to myself. So when he said he’s shifted to using a Shortcut that saves the page he’s looking at to a list in Tot, I figured I’d give it a try.
I wrote some Tot Shortcuts last year: one for dictating short notes that includes the date, time and location; and another for cleaning up my shopping lists after a trip to the store. I still use them, and I still like Tot in general as a place to put ephemeral notes. I used Drafts for many years, but never cleared out the old notes; Tot’s seven-note limit forces me to keep temporary text temporary.
I keep the links in the 6th dot. Here’s what it looks like on the Mac:
For each link, it saves the date, title, and URL on successive lines. Blank lines separate the entries. It sounds as if Gruber’s system is more sophisticated, with an automation that runs every night to (I think) add a date header that applies to all the links saved on that day. That would make the list more compact, but I’m not going to take that step until I know I’ll actually keep using this system.
What is the system? It’s just this Shortcut, which you can make for yourself or download:
Step | Action | Comment |
---|---|---|
0 | ![]() |
Share Sheet input of web page |
1 | ![]() |
Get name of page |
2 | ![]() |
Get URL of page |
3 | ![]() |
Get today’s date |
4 | ![]() |
Construct the text |
5 | ![]() |
Add the text to the end of Tot dot 6 |
Not much to it, which is why I don’t feel bad about building it, even though my track record for saving links is pretty poor. My goal is to use this only for links to pages I intend to write about and to clear them when the post is written. Tot will help with that by getting crowded quickly.
To make it easy to use this Shortcut, I’ve added it to my favorites. It appears just below in that section of the Share Sheet on both my iPhone and iPad.
Running a Shortcut from the Mac Share Sheet takes more clicking than I’d like, so if I use this enough to get frustrated by the extra work, I’ll probably make a Keyboard Maestro macro that does the same thing.
Cauchy coup
July 19, 2025 at 10:03 PM by Dr. Drang
Bruce Ediger, who blogs at Information Camouflage, has been doing some interesting numerical experimentation recently; first in a post about estimating population size from a sample of serial numbers (you may have seen this Numberphile video on the same topic), and then a couple of videos about the Cauchy distribution. In the first, he looked at the sum of two random variables that are Cauchy distributed; in the second, he looked at the quotient of two normally distributed random variables, which—under certain conditions—is supposed to have a Cauchy distribution.
By numerical experimentation, I mean that Ediger’s generating sets of random numbers, doing some calculations on them, and then seeing how close his calculations are to certain theoretical results. It’s a good way to get a feel for how things work, and it’s the kind of analysis that’s really only possible because of the computational power we all have at our fingertips. This sort of playing around with thousands and thousands of numbers just wasn’t feasible when I was learning probability and statistics, and I’m jealous of today’s students.
What attacted my interest in Ediger’s post on the quotient of two normals was the possibility that I could do the problem analytically without too much effort—mainly because the same computational power that lets him mess around with a hundred thousand quotients of random samples also lets me do calculus and algebra without dropping terms or making other silly errors.
Cauchy from the quotient of two zero-mean normals
I started by making sure I could prove that the quotient of two zero-mean normals has a Cauchy distribution. The analysis in this section follows the structure outlined in Papoulis’s Probability, Random Variables, and Stochastic Processes.
We begin by looking at the general problem of the quotient of two random variables of any distribution. We’ll call the two random variables X and Y and their joint probability density function .
We’ll call our quotient Z and define it this way:
To figure out the probability density function of Z, we’ll first look at its cumulative distribution function, , which is defined as the probability that ,
This probability will be the volume under the joint PDF of X and Y over the region where . That’ll be this integral:
And the domain over which we take the integral will be the blue region in this graph,
where the region extends as far as extends in the upper left and lower right directions.
If this doesn’t seem obvious to you, don’t worry, it isn’t. But think of it this way:
- If , then will be less than when is to the left of the line.
- If , then will be less than when is to the right of the line. Making negative flips the direction of the less-than sign.
With the integration domain defined, we can write the expression for like this:
To get the PDF of Z, we need to differentiate with respect to z. This is done through the Leibniz rule. Normally that would leave us with six terms, three for each of the two integrals above. But luckily for us, four of those terms are zero, leaving
Now it’s time to consider the case when X and Y are jointly normal. And to get to a Cauchy distribution for Z, both X and Y will have to have zero means. With that restriction, the joint density function will look something like this:
To make the plot, I needed some actual numbers, so I’ve given X a standard deviation of 1, Y a standard deviation of 1.5, and the correlation coefficient a value of 0.25. The analysis doesn’t rely on these or any other particular values for the parameters.
We can see the joint PDF’s structure a little better with a contour plot:
The key is that it has this kind of symmetry,
and we can take advantage of that symmetry to simplify the expression for :
OK, now we have to get into the nitty gritty and actually do some integration. Which means it’s time to abandon “by hand” analysis and switch to Mathematica. Here’s the notebook that does the following:
- The integration to determine the form of . When writing stuff like this by hand, I would typically use , , and for the standard deviations of X and Y and their correlation coefficient, but to make the typing easier in Mathematica, I use
sx
,sy
, andr
. - Checks the result against Papoulis’s. It’s not uncommon for Mathematica to give answers that don’t look like what you’ll find in a book, even though they are algebraically equivalent.
Determines the shape and location factors, and , that put the result into the generic form for a Cauchy distribution that we see in the Wikipedia article:
Here’s the notebook:
So yes, the quotient of two zero-mean normals has a Cauchy distribution with a shape parameter of
and a location parameter of
Ediger’s problem
That was sort of a warmup to make sure I understood how to use the MultinormalDistribution
function. It’s time to move on to Ediger’s problem.
Ediger doesn’t take the quotient of two zero-mean normals. Instead, he takes the quotient of two independent () normals, one with zero mean and unit standard deviation, and the other with a mean of 1 and standard deviation of 3. This shouldn’t result in a Cauchy distribution, and it doesn’t. But to see how close it is, I made this Mathematica notebook:
Comparing the actual distribution of the quotient (in blue) with Ediger’s best fit of a Cauchy distribution (orange), we see this:
Definitely not the same but pretty damned close.
Finally
I’m not sure most people would consider this recreational mathematics, but it’s recreational to me. You should see the things in my notebooks that don’t get turned into blog posts.
-
A habit I picked up from my thesis advisor is to use capital letters for random variables and their lower case versions for particular instances of those random variables. ↩
Settings steganography
July 16, 2025 at 12:10 PM by Dr. Drang
Last week, I was going to be out with my MacBook Pro all day, and I wanted to make sure it was fully charged. I had noticed that it was typically charging up only to about 80%, and I assumed that was because Sequoia was doing some clever battery-life-lengthening thing. I wanted to turn the clever thing off so I could get the battery to 100% just for that day.
You will probably not be shocked to hear that I didn’t find the solution by simply opening System Settings and scanning the Battery panel—I had to do a Kagi search for it. It wasn’t that the toggle was buried several layers deep or that it was outside the Battery hierarchy. No, the problem was that Apple had put the toggle in a place where toggles—or any kind of control or data entry field—don’t belong.
Here’s the Battery settings panel:
The two popup menus in the Energy Mode section of the panel aren’t what I wanted, so I assumed the setting for turning off battery optimization would be found via the
button down at the bottom of the panel. But it wasn’t in the window that appeared when I clicked it.Well, I’ve already given away the secret with the red arrow in the first screenshot. You have to click the ⓘ button near the top right corner. That brings up this window with the toggle I was looking for.
Turning the toggle off brought up this further set of options:
Before I start complaining, let me say this final dialog box is nice. You can turn off optimized charging permanently if you want, but the default is to just turn it off for a day. The clear implication is that Apple thinks you should use non-optimized charging rarely, and it will help you turn it back on tomorrow if you forget.
Now for the complaint: none of this should be behind the ⓘ button. The “i” clearly means “information,” and when I click on it, I expect to be given information, not to give it. Just as you wouldn’t put toggles or other controls behind the ?⃝ button, you shouldn’t put them behind the ⓘ.
I know there’s always talk of Apple losing its way in user interface design. And that talk has been especially loud since the release of the xxxOS 26 betas. My Mastodon timeline has been filled with criticism of Liquid Glass and Alan Dye for weeks, and it’s tiresome to read the constant grousing. But then I run into this, and I need to do some grousing myself.