Local AI: Wiring Up the Conversation Model with Rails

We tested our Ollama connection, now let’s actually wire it in. We will want create a Conversation model that have many Messages. Each message will be an individual piece of the conversation. Each message will have the message itself, and a role for who submitted it, the user, the AI, or the system. System Prompts are used to prime the AI.

rails g scaffold conversation title:string
rails g model message conversation:references role:string message:text
rails db:migrate

# app/models/conversation.rb
class Conversation < ApplicationRecord
	has_many :messages
end

We will update the conversation view to include the messages and a form to post another message in the thread:

<!-- app/views/conversations/show.html.erb -->
<div class="mt-8">
  <%= render "messages/form", message: @conversation.messages.build %>
</div>

<div class="divide-y-2 mt-8">
  <%= render @conversation.messages.order(created_at: :desc) %>
</div>

<!-- app/views/messages/_message.html.erb -->
<%= tag.div id: dom_id(message), class: "p-2" do %>
  <span class="font-semibold"><%= message.role %>:</span>
  <%= simple_format(message.message) %>
<% end %>

<!-- app/views/messages/_form.html.erb -->
<%= form_with(model: [message.conversation, message]) do |form| %>
  <div class="flex gap-x-2">
	<%= form.text_field :message, class: "grow" %>
	<%= form.submit nil, class: "bg-gray-300 hover:bg-gray-400 px-2 py-1 cursor-pointer" %>
  </div>
<% end %>

This will require a new nested route for the messages and a controller:

# config/routes.rb
resources :conversations do
	resources :messages, only: :create
end

# app/controller/messages_controller.rb
class MessagesController < ApplicationController
  def create
	@conversation = Conversation.find(params[:conversation_id])
	@conversation.messages.create({
	  role: :user,
	  message: params[:message][:message]
	})

	redirect_to @conversation
  end
end

This will store all the messages from the user, but never send them to the AI. Lets add a method to do that:

# app/models/conversation.rb
class Conversation < ApplicationRecord
	def update_thread
		messages = self.messages.order(:created_at).map { {
		  role: _1.role,
		  content: _1.message
		} }
	
		response = ollama_client.chat({
		  model: "llama3",
		  stream: false,
		  messages: messages
		}).first["message"]
	
		self.messages.create({
		  role: response["role"],
		  message: response["content"]
		})
	end
end

In this version, we are waiting for the entire response (stream: false), then creating a new message with the role from the assistant. This doesn’t change our interface, but is needed for the AI to keep track of things.

Now we just need to use it in the message controller:

class MessagesController < ApplicationController
	def create
		# …
		
		@conversation.update_thread
		redirect_to @conversation
	end
end

One thing to note is that this is slow. We can address this well by using Turbo to broadcast updates. But I think that is a topic for later.