Google expands BigQuery with Gemini, brings vector support to cloud databases

Google is adding new capabilities to its database and analytics platforms to help developers and organizations benefit from the power of generative AI.

Google has been busy in 2024 thus far, with multiple updates of its Gemini large language model (LLM) that haven’t all quite gone according to plan. Today Google announced that it is bringing the power of its Gemini models to its BigQuery analytics service, alongside new function updates for AI data preparation and retrieval augmented generation (RAG). Going a step further, Google is dramatically expanding its database capabilities for AI, with vector search support now coming to all of its cloud databases.

“We’re basically saying that vector indexing and vector search should be a primitive in any database,” Andi Gutmans, GM and VP for Databases, Google Cloud told VentureBeat. ” Databases are so critical as part of retrieval augmented generation and actually truly getting the real benefit of AI in the enterprise.”

Every Google database is now a vector capable database

Google has had vector support in some of its databases already. 

The Google AlloyDB database has previewed vector and AI capabilities and is now generally available. Google also has its Vertex AI Vector Search service, which is a purpose-built vector database. 

That lineup is now being expanded with preview support for vectors in the in-memory Memorystore for Redis database, the CloudSQL and Spanner relational databases, as well as the Firestore document database and the Bigtable key-value database.

Adding support for vectors across all Google databases is not a trivial operation and involves a lot of engineering work from Google. Gutmans noted that for AlloyDB which is based on the open-source PostgreSQL database, Google can benefit somewhat from the open-source pgvector technology which provides vector support. That said, even with AlloyDB Google has had to do a lot of work to ensure the best possible performance and features for users.

“We have to innovate on different work streams for different databases because each database has its own nuances of how it’s built,” Gutmans said.

With a vector search, there commonly also is a need in a database for an additional index, to help facilitate queries. Gutmans said that a big part of Google’s differentiation against other vendors that are building vector capabilities is how Google builds vector-capable indexes.

“We believe this is an area that is our strength because we’ve actually had to do it at a very large scale for ourselves for many years,” he said. “We’ve got quite a bit of experience doing this at Google scale, with billion user services.”

Though it hasn’t always been exposed to end users, Gutmans noted that Google internally has been using vector search capabilities for 12 years as part of its ad and search business units.

BigQuery gets a Gemini Pro boost

On the analytics side, Google is enhancing BigQuery with support for its latest Gemini Pro models.

“This unlocks a whole new set of analytical scenarios,” Gerrit Kazmaier, GM and VP for Data Analytics, Google Cloud said during a press roundtable detailing the news. 

The advanced capabilities include enhanced summarization and sentiment extraction, classification, enrichment as well as translation of structured and unstructured data. Kazmaier noted that the vast majority of data is unstructured and is often not fully utilized for enterprise data analytics because you couldn’t work with it in a meaningful way. 

“Now with Gemini Pro and BigQuery you can do basically all of the rich unstructured data analytics and combine it with your structured data,” he said.