Introduction#introduction

Semantic search is a powerful technique that allows you to find records in your database based on the meaning of text, rather than exact keyword matches. This can greatly enhance the search capabilities of your application, providing more relevant results to your users. In this post, we'll walk through how to implement semantic search in a Ruby on Rails application using the Neighbor gem, Anthropic's Claude API for summarization, and OpenAI for text embeddings.

Our Stack#our-stack

We'll be using the following technologies:

• A brand new Ruby on Rails 7.1 (The Neighbor gem is compatible with earlier versions of Rails)
• PostgreSQL with the pgvector extension
• Neighbor gem for easy vector operations w/ ActiveRecord
• Anthropic's Claude API for summarization
• OpenAI's API for text embeddings

How does this work?#how-does-this-work

When storing data that can be searched on via text embeddings, we need to create a standard prompt to feed an LLM to generate a summary of the content. Second, we’re going to pass that summary to a text embedding model to generate an array of floats (think of it as converting text to numbers). Lastly, we’re going to store those embeddings in our database.

At a high level:

• Create a summary of an incident via Anthropic’s API.
• Send that summary to OpenAI’s text embedding API to retrieve an embedding.
• Store the embedding on our ActiveRecord model (we’re going to use a simple Incident model)

Let’s get started.

Step 1: Setting Up#step-1-setting-up

First, add the Neighbor, OpenAI, and Faraday gems to your Gemfile:

Next, we need to choose an extension. Neighbor supports two: cube and vector. We'll use vector as it supports more dimensions and approximate nearest neighbor search.

Install pgvector in your PostgreSQL database, then run:

This sets up the necessary database extensions.

Step 2: Creating the Model#step-2-creating-the-model

Let's create an Incident model to store our IT incidents:

This creates a migration that includes a text column for our summary and a vector column for our embeddings. The 1536 specifies the number of dimensions, which matches OpenAI's text embedding model output.

Run the migration:

Now, update the Incident model to use Neighbor:

Step 3: Implementing the Anthropic Summarizer#step-3-implementing-the-anthropic-summarizer

I recommend creating a PORO (plain old ruby object) that is responsible for generating a consistent system and user prompt. By prompting an LLM with the same system prompt, and formatted incident information, we’ll get a more consistent format when we retrieve the summary blob of text.

Next, we’re going to create a small class that is responsible for sending the system and user prompt from our class above to Anthropic’s API to generate a summary of our incident. The Anthropic Claude API is very straightforward, and we haven’t discovered a need for a Ruby gem for it.

Step 4: Generating Embeddings#step-4-generating-embeddings

We'll use OpenAI's API to generate embeddings. Let's create a service to handle this similar to our Anthropic client previously.

ℹ️ Note: You’ll need to add an inflection in Rails so that this class will load correctly:

Step 5: Storing Summaries and Embeddings#step-5-storing-summaries-and-embeddings

We’re going to update our Incident model with a few simple methods that summarize the incident, grab the embeddings for that summary, and store it in our vector column in Postgres. In this tutorial we’re using Rails credentials, which if you’re following in a dummy Rails application you can modify with:

The format in the file is:

Our incident model gets a few simple methods to process our incident later for semantic searching.

Step 6: Finding Similar Incidents#step-6-finding-similar-incidents

Thanks to Neighbor, finding similar incidents is now very simple:

Testing with Sample Data#testing-with-sample-data

Let's create some sample data to test our semantic search. Add the following to your db/seeds.rb file:

Run the seeds with:

Using the Semantic Search#using-the-semantic-search

Now you can use your semantic search like this:

Indexing for Better Performance#indexing-for-better-performance

For better performance with large datasets, you can add an index. Create a migration with:

And then add the index to the generated file:

Run the migration:

Conclusion#conclusion

Adding semantic search to applications has become remarkably easy with the new APIs and libraries available to developers in the last two years. Semantic search, when used correctly, is a potentially more powerful way to display similar content to users – like how we store incident summaries and recommend them to users in FireHydrant.