Extending ChatGPT: Can AI chatbot plugins really change the game?
Plugins have long been a fixture of complex software systems. As far back as the 1980s, I founded a company called Hyperpress that provided plugins for Apple's HyperCard (think of it as a web before there was a Web… or connectivity). My plugins added capabilities to HyperCard that weren't part of the original build.
Today, plugins enhance popular products like Photoshop and WordPress. WordPress offers roughly 60,000 plugins that modify its capabilities.
On the two main websites I operate, I use 49 separate plugins (which add roughly 49 new features) on one site, and 25 plugins (which add roughly 25 new features) on the other site. Neither of these sites would be the sophisticated professional e-commerce sites they are without the wide array of plugins that add features and value.
What are plugins?
Fundamentally, plugins are separate chunks of code that interact with the parent software platform. They do this via an API (application programming interface). All platforms that support plugins provide APIs that allow outside programmers to hook into the functionality of the platforms.
Often the breadth and reliability of the API determine the resilience and flexibility of the overall platform, especially when users rely on a great many plugins to expand the capabilities of the plugin-supporting platform they're using.
Done right, plugins benefit three constituencies: the platform provider (i..e., Adobe for Photoshop, the open source WordPress community, and OpenAI for ChatGPT), the developer of the plugin, and the users of the platform who get new capabilities.
Also: I asked ChatGPT to write a WordPress plugin I needed. It did it in less than 5 minutes
Platform providers often decide to compete with developers. If they see a plugin is particularly popular, they sometimes choose to include that functionality in the core product. They change APIs. Sometimes, if they offer a marketplace (like an app store, but for plugins), they are selective about who to allow into the marketplace and who to promote.
But when the dance between platform provider and plugin developer works, it's magical to behold. The original platform can be taken places nobody predicted, providing capabilities not otherwise possible.
Also: The best ChatGPT plugins (and how to make the most of them)
Plugin capabilities have been announced for Google's Bard, Microsoft's Bing AI, and ChatGPT. However, so far, only ChatGPT offers an extensive array of plugins you can actually put to use.
How plugins currently work in ChatGPT
Plugins for ChatGPT are only available to paying customers of ChatGPT Plus. That's a $20/month service that provides access to the GPT-4 dataset, plugins, and a special plugin called Advanced Data Analysis (more on that later).
For that $20/month, you get to use a very beta, very unfinished product. It's still amazing, but it's also very annoying. You're limited to 25 queries in three hours, so if you're trying to get a job done, you may well run out of queries right in the middle of your work time. Yes, I speak from very frustrated experience. You'll also need to turn them on in Settings.
Now that you've got the plugins enabled, prepare to be impressed.
Plugins that move the needle
I experimented with a lot of plugins. Because you can only use three plugins at one time, you really need to pick and choose a core library of plugins that you use regularly. Here's the list I came up that powered the examples I'm about to show you:
- Stories: This generates a story book from a prompt. I only used it once (because I don't have kids), but it was so impressive, it's something you need to see.
- MixerBox WebSearchG: This makes the entire current web available to ChatGPT, and it does it quite well. This truly expands the value of ChatGPT.
- World News: This scans news sources and provides up-to-date news summaries.
- AskYourPDF: You can feed ChatGPT a PDF and interact with the data in the PDF document.
- Advanced Data Analysis: This is a special add-on/plugin provided by OpenAI. If you run this, you can't run any of the other plugins. It allows you to use code to talk to ChatGPT, but it also interprets complex requests and substantially extends the queries you can ask ChatGPT.
Note that I'm not providing URLs to each of these individual plugins. The ChatGPT Plugin Store doesn't offer unique URLs for each plugin. But they're pretty easy to find. Just go to the ChatGPT Plugin Store and do a search for their titles. If you want to know how to enable plugins and access the Plugin Store, ZDNET's Steven Vaughan-Nichols has a great tutorial.
The plugin that's the reason why writers are on strike
Let's start with Stories. In a previous article, I showed you how I got ChatGPT to write a short Star Trek story (and how it mostly succeeded). Stories takes that idea and makes it real.
Within ChatGPT, you can give Stories a prompt that describes the story you want to be told. Here's what I fed it:
Using stories, tell the story of a group of friends who live on a starship (three are human, one is a robot). Tell of their adventure traveling to a planet populated by sentient dinosaurs and where creativity performed by the sentient dinosaurs is illegal, and all stories, entertainment, music, etc is composed by generative AI systems.
Stories then goes on to generate a full story book based on this premise. While the generated graphics were a bit weak (they could benefit from Midjourney-quality tech), the overall production is quite good. I gave the plugin a premise that involved some moral choices, and the AI came up with not only a good little story, but one that held together.
But stories takes it further. You can read the story online in digital form. Or you can go to the Stories site and order a hardcover copy. You can even publish the story on Amazon.
You can see how this sort of thing puts fear into the hearts of professional creatives, particularly those currently on strike. In less than five minutes, I had a fully usable story book. Written and illustrated traditionally, this 50-page story book (with a paragraph on each page) could have taken weeks or months.
I just took a sip of coffee, made up a premise loosely based on a typical Star Trek theme, and fed it to the AI.
When ChatGPT can read the web
As we all know, ChatGPT's knowledge base ends in 2021. But with the MixerBox WebSearchG plugin, we get a taste of what will happen when an AI can read the web. This also produced that "woah" feeling I sometimes get when I know I'm touching the future.
On July 10, 2023, I published a ZDNET article about challenges I was having with my Google Cloud storage enterprise plan. In that article, I coined the term "infraquake" and mentioned it in the two concluding paragraphs of the article.
Then, on July 11, I asked ChatGPT this question:
How does Gewirtz define an "infraquake"
I didn't tell ChatGPT which Gewirtz, nor did I tell it that an article had been published on ZDNET. And yet, it came up with a very clear (and, I might add, accurate) description of the intent behind the newly coined word "infraquake."
Clearly, ChatGPT can now access current data and process it for meaning using the plugin.
Also: How does ChatGPT actually work?
You can also see how ChatGPT's ability to retain context is mixed with the plugin's ability to access the web in this next example. I asked it:
Tell me about David Gewirtz's dog Pixel
Not only did it pull the information, it showed a picture of the little guy, and linked to an article where I wrote about choosing his name.
Understanding current events
While working on this special feature, I found three "killer apps" that I'm going to use regularly as part of my job. Let's discuss the first one first: creating briefings explaining current news with full background information.
In my job, I track a lot of news. I'm often asked by clients to provide perspective on tech news, technology trends, and some geopolitical issues. That means I spend a few hours every day keeping up with my reading, making sure I get a good understanding of what's going on.
But there's no way I can keep up with everything, and I can't really take a lot of time for those topics that are not on my main "beat." Even so, I'd like to have a solid understanding of the issues.
Also: What is generative AI? Here's everything you need to know
One example of this is the Ukraine/Russia war. While I've written about how the attacks impact Ukrainian developers and even covered previous Ukrainian security issues going back almost a decade, I have not been "fully briefed" on the issue of Ukraine's bid to be a NATO member.
I could have dug through a bunch of news articles and done background research, but I don't really have time to allocate to it. Instead, I asked ChatGPT, MixerBox WebSearchG, and World News to prepare me with a comprehensive briefing. I asked two questions:
You are a US policy advisor briefing a CEO on the NATO situation. You've been asked to explain why President Biden says that Ukraine is not ready to join NATO. Use World News and MixerBox WebSearchG to provide a clear briefing on both sides of the argument so that your client has a deep and current understanding of the issues, complexities, and political ramifications. Relate your answer to the US political climate as well.
and
Using the same plugins, is there an opposing view by the GOP for Ukraine NATO membership?
From these two questions, the AI gave me a comprehensive briefing on the NATO membership question, the foundational and political issues behind membership, and the position of both parties with regard to this issue.
My wife subscribes to a service called Blinkist. The company describes its service as "Blinkist offers the key insights from top nonfiction in a made-for-mobile format." It's essentially audible or readable Cliff's Notes for currently popular nonfiction books, and it lets her come up to speed on topics she cares about in about half an hour.
By combining ChatGPT with MixerBox WebSearchG and World News, I can essentially get a Blinkest "key insights" briefing on any currently unfolding world news issue. This is powerful stuff, but as with all press coverage, it's important to be aware that there may be bias, omissions, and inaccuracies in what the AI presents to you.
Using PDFs as source material for analysis
I recently had an analysis project where I had to dig through very long and very dry white papers to try to understand the relationships between some key technologies. Using the AskYourPDF plugin, I fed those PDFs to ChatGPT, and then asked questions related to the content of the PDFs.
It was extraordinary. I was able to ask ChatGPT to analyze various concepts contained in the PDFs. I could even get it to draw a table comparing items discussed in the PDFs, but which had not been compared to each other directly in the source documents.
I would never use ChatGPT as a substitute for reading all the background information on a project I'm tasked with investigating. But much of the analysis I do for my own learning process requires a lot of very tedious clerical work to construct tables and charts to increase my understanding of relationships within documents.
I also used this to examine some contracts. I fed it a contract document we had with a service provider and asked it to show me how the limitations differed between the parties, something normally quite time consuming to extract and determine. Here's the prompt I used:
Create a table comparing the limitations, itemizing each limitation listed. Show only where the limitations differ between the parties, summarize each different limitation in 8 words or less
And here's the table I got back:
Of course, without question, the results from ChatGPT can't be trusted as completely accurate. But a quick scan can definitely save time.
Also: 5 ways to explore the use of generative AI at work
Using ChatGPT and AskYourPDF, clerical research tasks that would normally take me half a day or more were reduced to mere minutes. That's a killer app.
Doing large-scale data analysis
Finally, I want to explore an add-on for ChatGPT from OpenAI that runs on its own. It's called Advanced Data Analysis, and it does so much more than interpret code. Advanced Data Analysis allows you to upload data to it, which ChatGPT can then analyze.
WARNING: Do not use this feature for the first time if you have anything else to do that day. You will get sucked in. It's more disruptive to productivity than kitten and puppy videos.
Ask me how I know. I mean, it's hard to believe anything this fun can really be legal.
This tool allows you to import data files (Excel, CSV, etc) into ChatGPT. It can then perform analysis on it and even generate basic graphics. It is dangerously addictive. Hours after I started, I found myself downloading dataset after dataset from data.gov and laughing maniacally about the power at my fingertips. It was not a pretty sight.
Also: AI gold rush makes basic data security hygiene critical
I think this is, ultimately, why ChatGPT Plus limits you to 25 queries every three hours. It's not to reduce the load on their infrastructure. It's for our own darned good. I sure needed it. I probably wouldn't have eaten all day if I hadn't been forced to step away from the computer by the query limit.
I'll save you any more disturbing analytics visions inside The Mind of David and, instead, show you a simple application: of my email contacts, what big PR firms do I regularly correspond with, and what big tech companies have the most representation. To do this, I exported my contacts from Google Contacts.
Using the email-related fields, list the top 20 domain names represented
Of the top 20 domain names, which are related to PR companies
Then, I had the AI list and draw a pie chart of which tech companies I had the most contact with. Here's what I asked:
Looking at the dataset, create a pie chart showing the relative representation of large multibillion dollar technology companies
And here's what I got back.
The format of the pie chart isn't ideal, but the information is there. And, again, we're talking minutes instead of hours.
But...we're still in the early days
Plugins are available, but they're very new. Some, like the ones I demonstrated above, have big advantages. But because they're so new, they also have a bunch of disadvantages and annoyances:
- There are almost 700 plugins in the ChatGPT plugin store. Most are uncurated (just about anything goes).
- While you can search on a keyword, they're otherwise uncategorized. Plugins like Pluginpedia and PlugFinder claim to help with that, but they're just not that reliable.
- Many plugins are…what's the word? Meh. Some do nothing more than access the provider's website. For example, there's a plugin for getting discount coupons. How is this better than RetailMeNot?
- Many plugins seem to be branding or PR exercises to get a foothold in a new marketplace early on. For example, there's an "AI clone" plugin of a specific Silicon Valley small startup CEO. Supposedly, you can ask everything you ever wanted to know about him. So, yeah. This is not exactly something most of us are likely to use.
- Many plugins don't work or don't do much. I tried to get the local food delivery plugin to tell me where I could get steak dinners in my town, and it recommended Subway. Yes, they have a steak sandwich, but I could have gotten better results from Yelp. I also found a bunch of plugins that just simply hung with no results.
- You can only run three plugins at once, and if you want to swap out the plugin set, you have to launch a new chat session in ChatGPT, losing all of your current discussion history. This is a big limitation. But even with only three plugins, you saw how I did get the plugin interface to do some magical stuff.
According to Pam Baker, author of ChatGPT for Dummies, "It's harder to see the magic now, given the 3 plugin/25 queries caps and the questionable value of some of the current plugins. But the caps are necessary so OpenAI can better manage model stability and strengthen the guardrails as it assimilates more capabilities."
Also: Microsoft embraces OpenAI's ChatGPT plugin standard
To be fair, we're still very early. That's why I'm not sharing the names of plugins that fell short. There's a good chance they'll get a lot better over time.
So, do plugins really change the game?
Yeah. They do. They really do. We're just in the early stages here, where I had to take extra time to pick and choose four that I think I'll use all the time (plus Stories, which shows an additional level of generative AI potential).
I find that if I make my primary three plugin set a combination of MixerBox WebSearch G, World News, and AskYouTPDF, I can do a whole lot. I can switch to Advanced Data Analysis if I want to do a more in-depth data analysis project. Note that I've found WebSearch G to be somewhat unreliable. If you run into issues, turn it off and try the WebPilot plugin instead.
As ChatGPT grows in its ability to handle plugins, as plugin discovery and curation get better, as we can use more and more plugins at once, it's pretty clear that the kind of chatbot we've just come to know in 2023 is in for a string of future upgrades, providing us with more and more assistance with our tasks and projects.
Also: GPT-3.5 vs GPT-4: Is ChatGPT Plus worth its subscription fee?
ChatGPT for Dummies author Baker shares her view of the future of plugins. She says, "Plugins add capabilities that will eventually enable us to mod ChatGPT on the fly." Her premise is that ChatGPT (and, by extension, other LLMs) will be able to grow their own capabilities.
She told ZDNET, "In an instant, ChatGPT will be able to morph into the perfect tool for any task. Eventually, ChatGPT will be able to automatically determine and select the plugins it needs to respond to every prompt. Where a plugin it needs doesn't yet exist, it will create it on the fly and seamlessly assimilate the new capability."
At this point, I can't tell whether we're creating Skynet or the Borg. Either way, enjoy the added capabilities plugins provide…while…you…still…can.
Resistance is futile.
You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter on Substack, and follow me on Twitter at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.