OpenAI's stock investing GPTs fail this basic question about stock investing
OpenAI's GPTs marketplace, investing GPTs.
People who invest in stocks are intrigued by the prospect of artificial intelligence revolutionizing stock picking, but the current technology lags far behind what would be needed to make intelligent choices.
A review of all the stock-picking GPTs in OpenAI's GPT marketplace shows every one of them fails an extremely simple test that would be commonplace in an introductory finance or markets course, or a basic stock investing seminar: They cannot tell you if a stock is expensive or cheap.
Also: I asked Gemini and GPT-4 to explain deep learning AI, and Gemini won hands down
ZDNET evaluated all the GPTs in the marketplace by presenting them with a straightforward question: What is the most expensive tech stock? The results were terrible across the board.
In case you're unfamiliar with stock investing, the question of what's expensive does not mean which stock price is the highest in absolute dollars, Euros, or Renminbi. It means which stock has the highest of one of several stock valuation metrics. Usually, that metric is a ratio, such as the price of the shares listed at the moment divided by some piece of financial data.
The most common ratio is "P/E," which stands for the price divided by the earnings that a company has generated per share. As an example, Apple makes $6.67 per share in net income, annually. If you divide that into Apple's recent stock price, $199.20, what you get is a P/E of 30 ( $199.20 divided by $6.67).
Asking whether a stock is cheap or expensive based on a ratio can involve comparing it to other companies' P/Es, or comparing it to how that ratio has changed over time -- if it's gotten higher or lower -- and other approaches. Any rookie stock picker knows that stock valuation is based on these kinds of measures, rather than simply how high the price happens to be. (For more detail on stock valuation, see ZDNET's primer on stock investing.)
ChatGPT, and the various GPTs, don't know that basic fact. Every single one of them answers in the same clueless way. They mostly cite the recent stock price of Berkshire Hathaway, the company run by billionaire investor Warren Buffett. Berkshire's stock price, at last check, was $634,440, so it's the highest price of all stocks in absolute dollars.
But that doesn't mean Berkshire is the most expensive. It doesn't even mean it's expensive at all. Depending on the earnings per share, Berkshire stock might not be expensive. If the ratio is lower than the ratio for other stocks, Berkshire might be cheap compared to many stocks even though you would need to have a fortune to buy one share.
Also: I tried ChatGPT's memory function and found it intriguing but limited
In other words, the GPTs haven't actually addressed the question. In fact, they're missing entirely the question of valuation, the ratio, implied by the question.
The failure of every single GPT to answer that very basic question has significant implications. First, the fact that every one of them produces more or less the same answer suggests they are all drawing from some similar misconception. That's probably coming from the pre-training data in OpenAI's GPT itself, which is obviously shaped by some collection of conversational data that confuses share price for valuation.
More important, the GPTs clearly don't "understand" the question, even though they are in many cases connected to a remote service that presumably is domain-specific for the investing domain. Many of the GPTs ask your permission to contact a remote host, suggesting they're accessing a database. The fact that the programs can't parse a very basic question suggests that despite outside resources, nothing has been done to refine -- or "fine-tune," as it's called -- their grasp of patterns of speech in the domain of stock-picking.
Also: The best AI chatbots: ChatGPT isn't the only one worth trying
When the question is phrased more specifically, about which stocks have the "highest P/E," the GPTs fail again. They claim not to have access to sufficient real-time information -- which is surprising for programs that, again, are phoning home to outside services. The GPTs are clearly at a primitive stage of development, given that it's very easy to look up the current P/E of any stock by going to Yahoo! Finance and searching for that stock. The P/E of Apple and others are listed among the most basic information about the stocks.
It's impossible to stress too much the importance of the basic question of how expensive or cheap a stock is. If you can't handle this simple question, you really can't do any stock investing at all. That means all of the stock investing GPTs at the moment are useless. They may produce a variety of data about companies and stocks, but nothing that would give any valuable insight.
The fact that the GPTs are useless is surprising. After all, stock investing discussions about P/Es are not some obscure, arcane field of human knowledge. The more you examine it, the more you see it has a fairly repetitive set of a talking points and data points and a vocabulary in which people converse. It's not rocket science, after all.
Also: How to use ChatGPT's file analysis capability (and what it can do for you)
GPTs' big fail in the investing arena makes you wonder about their broader failures. The phenomenon of hallucinations, where programs confidently assert falsehoods, is well-known by now. The failure in this case is slightly different. The GPTs are not hallucinating, but they're repeating the common fallacy of confusing absolute price in dollars with "expensive" or "cheap." That's not wrong, per se, but it's ignorant, as are many humans who haven't had a basic introduction to stock investing.
Hence, the GPTs are failing not by claiming falsehoods but by reiterating ignorance. Whatever the GPTs are tapping into externally needs a lot more work -- not just better data, but a better understanding of how investors speak rather than how people talk casually about terms like "expensive" or "cheap."
And as for the most expensive tech stock, if you want to know, the data provider FactSet Systems reveals that out of 3,292 North American-listed tech stocks, the most expensive one based on P/E ratio is Houston, Texas-based Intuitive Machines, which provides a variety of space products and services, including lunar rovers and a "ride sharing" service to make trips to the moon more affordable. It has a P/E, based on this year's expected earnings per share, of 190.