Oracle's new offerings bring AI to your data, and manage it too
Database giant Oracle wants you to know you'll probably have a better experience with generative artificial intelligence if it doesn't involve moving your data from where it lives -- which could very likely be inside an existing Oracle database -- especially if you're an enterprise user.
On Monday, at a partner event in Dubai, Oracle announced the general availability of OCI Generative AI Services, a managed service for AI first offered in beta back in September. The company also unveiled two new offerings, still in beta: OCI Gen AI Agents and OCI Data Science AI Quick Actions.
Also: How SuperDuperDB delivers an easy entry to AI apps
The company makes the pitch that building an enterprise generative AI application on top of the existing data store is going to be both more effective in terms of using unique data, but also more economical compared with buying lots of additional infrastructure.
The acronym "OCI" refers to Oracle's Oracle Cloud Infrastructure; that is, all the network and compute resources, and attendant software such as the Oracle Autonomous Database, that the company uses in data centers throughout the world to deliver cloud services. That includes what Oracle calls "super-clusters" of Nvidia GPU chips on which Oracle has spent billions of dollars.
"We're essentially bringing AI to the data," said Erik Bergenholtz, Oracle's vice president for strategy and operations, in a briefing with ZDNET before the announcement.
"Our Fusion applications, such as ERP and HCM, have exabytes of data -- we're bringing AI there," he said. "Our database, obviously, there are petabytes, exabytes of data there -- we're bringing generative AI there."
The virtues of building on top of Oracle's database, middleware, and Fusion suite of apps are practical ones, said Bergenholtz.
Companies could, observed Bergenholtz, try to buy additional software for the data management piece, such as a vector database like Pine Cone. "The drawback, of course, is you have yet another piece of infrastructure that raises the cost of cloud, and you now have to actually move and potentially synchronize data across your originating data store, whether that be apps or your Oracle database; and now you have an additional piece of infrastructure."
Using the OCI services, Bergenholtz added, "just eliminates that barrier, that friction, for our customers."
Also: AI will change software development in massive ways
"We don't want customers to move data, because the last thing they want is to move 500 terabytes just to get the advantage of generative AI," said Steve Zivanic, vice president of Oracle's database and autonomous services marketing, in the same briefing with Bergenholtz.
OCI Generative AI service, which becomes generally available this week, consists of pre-built large language models (LLMs), including Meta Properties' open-source Llama 2 70-billion-parameter model.
In addition to Meta, Oracle has partnered with venture-backed startup Cohere for the GenAI service. (Oracle is an investor.) Cohere has three models that will go into the Oracle service: Command, for mainstream text-language functions; Summarize, for document summary; and Embed, for multi-language functions.
Also: Meta-IBM alliance promotes 'open' approach to AI development
Customer data used in OCI to either train or fine-tune the models cannot be seen by other Oracle customers, Bergenholtz emphasized.
Since it went into beta testing, the service has added new capabilities, such as content moderation. "The most important thing is that we do this prior to actually submitting the prompts" to a language model, "and then we also evaluate the response that comes out of the model," noted Bergenholtz, "so, we don't just wait until the very end, because then you've already incurred the cost of processing that request."
The service also now integrates with the LangChain development framework for using LLMs.
The OCI Gen AI Agents product is meant to tie an LLM into other resources, such as the database of a customer's proprietary data. The first agent offered is for retrieval-augmented generation (RAG), an increasingly popular approach in the world of generative AI to hook up the language model to a source of truth such as a database.
Also: How LangChain turns GenAI into a genuinely useful assistant
An LLM "may not be as useful as it could be unless you're able to actually leverage the data that you have under management sitting in your various Oracle applications or Oracle databases," said Bergenholtz. Initially, the RAG agent service can tap into OCI's OpenSearch offering, a managed service based on the open-source OpenSearch platform. In the near future, the RAG offering will be amplified by being able to plug into Oracle's Database 23c AI Vector Search, as well as the MySQL Heatwave Vector Store.
The AI agents service starts beta testing this month.
The OCI Data Science Quick Actions offering comes out of Oracle's 2018 acquisition of startup DataScience, which brought to Oracle expertise in Jupyter Notebooks and statistical techniques relevant to machine learning.
The point of Quick Actions is a no-code approach to deploying and fine-tuning language models. Several frameworks are available for fine-tuning, for example, including distributed training with PyTorch, Hugging Face Accelerate, and Microsoft's DeepSpeed. Bergenholtz also emphasized its use of object and file storage to ease the organization of model "weights," the parameters that take up large amounts of memory and that give a neural network its shape.
Quick Actions will start beta testing next month.
Also: Hybrid AI and apps will be in focus in 2024, says Goldman Sachs CIO
The most common use cases that Oracle has seen working with beta customers for the OCI Gen AI service involve things such as providing automated responses to HR policy questions.
"The common question is, how many vacation days do I have left," explained Bergenholtz. "You need two pieces of data, the policy in the company, and how many vacation days you've used, then you need to compute the answer based on those two things.
"With RAG, you can very easily answer that question because the RAG system knows your identity," and it also knows the company policy. Similar sorts of early applications include healthcare insurance benefits questions, said Bergenholtz.
Another frequent application in these early days is customer support. For the customer support representative, using RAG to dip into customer data, the language model "can easily summarize the case that you called about, what the current situation is, and provide a script of the next steps to recommend or to walk through with the customer to be able to provide that much richer, better user experience," said Bergenholtz.
What about customers who don't want this newfangled GenAI stuff touching their precious data?
Also: Data is the missing piece of the AI puzzle. Here's how to fill the gap
"There are a number of ways we see this transpiring," said marketing VP Zivanic. By putting capabilities such as vector search into the Oracle database, and vector store in Heatwave, "we're bringing the technology directly to them," he said.
There will be, he conceded, "organizations [that] will do various, essentially, skunkworks projects on the side" with Gen AI to get comfortable with the technology. "But, I think, over time, as generative AI becomes more prevalent, the power of a converged database where everything is forged into one, essentially, database construct, that will prove advantageous versus subscribing to multiple databases and trying to get your answer."