OmniAI

OmniAI standardizes the APIs of various AI / LLM companies such as Anthropic, Google, Mistral and OpenAI for the generation of text, the conversion of text-to-speech, the conversion of speech-to-text, the generation of embeddings, and more. It offers a unified API regardless of the provider and task.

Examples

Example #1: Chat

require 'omniai/anthropic'

CLIENT = OmniAI::Anthropic::Client.new

CAT_URL = 'https://images.unsplash.com/photo-1472491235688-bdc81a63246e?q=80&w=1024&h=1024&fit=crop&fm=jpg'
DOG_URL = 'https://images.unsplash.com/photo-1517849845537-4d257902454a?q=80&w=1024&h=1024&fit=crop&fm=jpg'

CLIENT.chat(stream: $stdout) do |prompt|
  prompt.system('You are a helpful biologist with an expertise in animals that responds with the latin names.')
  prompt.user do |message|
    message.text('What animals are in the attached photos?')
    message.url(CAT_URL, 'image/jpeg')
    message.url(DOG_URL, 'image/jpeg')
  end
end

The animals in the photos are:

1. A cat (*Felis catus*).
2. A dog (*Canis familiaris*).

Example #2: Text-to-Speech

require 'omniai/openai'

CLIENT = OmniAI::OpenAI::Client.new

File.open(File.join(__dir__, 'audio.wav'), 'wb') do |file|
  CLIENT.speak('Sally sells seashells by the seashore.', format: OmniAI::Speak::Format::WAV) do |chunk|
    file << chunk
  end
end

Example #3: Speech-to-Text

require 'omniai/openai'

CLIENT = OmniAI::OpenAI::Client.new

File.open(File.join(__dir__, 'audio.wav'), 'rb') do |file|
  transcription = CLIENT.transcribe(file)
  puts(transcription.text)
end

Example #4: Tools

require 'omniai/google'

CLIENT = OmniAI::Google::Client.new

LOCATION = OmniAI::Tool::Property.object(
  properties: {
    city: OmniAI::Tool::Property.string(description: 'e.g. "Toronto"'),
    country: OmniAI::Tool::Property.string(description: 'e.g. "Canada"'),
  },
  required: %i[city country]
)

LOCATIONS = OmniAI::Tool::Property.array(
  min_items: 1,
  max_items: 5,
  items: LOCATION
)

UNIT = OmniAI::Tool::Property.string(enum: %w[celcius fahrenheit])

WEATHER = proc do |locations:, unit: 'celsius'|
  locations.map do |location|
    "#{rand(20..50)}° #{unit} in #{location[:city]}, #{location[:country]}"
  end.join("\n")
end

TOOL = OmniAI::Tool.new(
  WEATHER,
  name: 'Weather',
  description: 'Lookup the weather in a location',
  parameters: OmniAI::Tool::Parameters.new(
    properties: {
      locations: LOCATIONS,
      unit: UNIT,
    },
    required: %i[locations]
  )
)

completion = CLIENT.chat(tools: [TOOL]) do |prompt|
  prompt.user do |message|
    message.text('What is the weather in "London" in celcius and "Seattle" in fahrenheit?')
  end
end

puts(completion.text)

The weather is 24° celcius in London and 42° fahrenheit in Seattle.

Example #5: Embeddings

require 'omniai/mistral'

CLIENT = OmniAI::Mistral::Client.new

Entry = Data.define(:text, :embedding) do
  def initialize(text:)
    super(text:, embedding: CLIENT.embed(text).embedding)
  end
end

ENTRIES = [
  Entry.new(text: 'John is a musician.'),
  Entry.new(text: 'Paul is a plumber.'),
  Entry.new(text: 'George is a teacher.'),
  Entry.new(text: 'Ringo is a doctor.'),
].freeze

def search(query)
  embedding = CLIENT.embed(query).embedding

  results = ENTRIES.sort_by do |data|
    Math.sqrt(data.embedding.zip(embedding).map { |a, b| (a - b)**2 }.reduce(:+))
  end

  puts "'#{query}': '#{results.first.text}'"
end

search('What does George do?')
search('Who is a doctor?')
search('Who do you call to fix a toilet?')

'What does George do?': 'George is a teacher.'
'Who is a doctor?': 'Ringo is a doctor.'
'Who do you call to fix a toilet?': 'Paul is a plumber.'

Installation

gem install omniai

gem install omniai-anthropic
gem install omniai-mistral
gem install omniai-google
gem install omniai-openai

Usage

OmniAI implements APIs for a number of popular clients by default. A client can be initialized using the specific gem (e.g. omniai-openai for OmniAI::OpenAI). Vendor specific docs can be found within each repo.

Client

OmniAI::Anthropic

require 'omniai/anthropic'

client = OmniAI::Anthropic::Client.new

OmniAI::Google

require 'omniai/google'

client = OmniAI::Google::Client.new

OmniAI::Mistral

require 'omniai/mistral'

client = OmniAI::Mistral::Client.new

OmniAI::OpenAI

require 'omniai/openai'

client = OmniAI::OpenAI::Client.new

Usage with LocalAI

LocalAI support is offered through OmniAI::OpenAI:

Usage with LocalAI

Usage with Ollama

Ollama support is offered through OmniAI::OpenAI:

Usage with Ollama

Logging

Logging the request / response is configurable by passing a logger into any client:

require 'omniai/openai'
require 'logger'

logger = Logger.new(STDOUT)
client = OmniAI::OpenAI::Client.new(logger:)

[INFO]: POST https://...
[INFO]: 200 OK
...

Timeouts

Timeouts are configurable by passing a timeout an integer duration for the request / response of any APIs using:

require 'omniai/openai'
require 'logger'

logger = Logger.new(STDOUT)
client = OmniAI::OpenAI::Client.new(timeout: 8) # i.e. 8 seconds

Timeouts are also be configurable by passing a timeout hash with timeout / read / write / `keys using:

require 'omniai/openai'
require 'logger'

logger = Logger.new(STDOUT)
client = OmniAI::OpenAI::Client.new(timeout: {
  read: 2, # i.e. 2 seconds
  write: 3, # i.e. 3 seconds
  connect: 4, # i.e. 4 seconds
})

Chat

Clients that support chat (e.g. Anthropic w/ "Claude", Google w/ "Gemini", Mistral w/ "LeChat", OpenAI w/ "ChatGPT", etc) generate completions using the following calls:

Completions using a Simple Prompt

Generating a completion is as simple as sending in the text:

completion = client.chat('Tell me a joke.')
completion.text # 'Why don't scientists trust atoms? They make up everything!'

Completions using a Complex Prompt

More complex completions are generated using a block w/ various system / user messages:

completion = client.chat do |prompt|
  prompt.system 'You are a helpful assistant with an expertise in animals.'
  prompt.user do |message|
    message.text 'What animals are in the attached photos?'
    message.url('https://.../cat.jpeg', "image/jpeg")
    message.url('https://.../dog.jpeg', "image/jpeg")
    message.file('./hamster.jpeg', "image/jpeg")
  end
end
completion.text  # 'They are photos of a cat, a cat, and a hamster.'

Completions using Streaming via Proc

A real-time stream of messages can be generated by passing in a proc:

stream = proc do |chunk|
  print(chunk.text) # '...'
end
client.chat('Tell me a joke.', stream:)

Completion using Streaming via IO

The above code can also be supplied any IO (e.g. File, $stdout, $stdin, etc):

client.chat('Tell me a story', stream: $stdout)

Completion with Tools

A chat can also be initialized with tools:

tool = OmniAI::Tool.new(
  proc { |location:, unit: 'celsius'| "#{rand(20..50)}° #{unit} in #{location}" },
  name: 'Weather',
  description: 'Lookup the weather in a location',
  parameters: OmniAI::Tool::Parameters.new(
    properties: {
      location: OmniAI::Tool::Property.string(description: 'e.g. Toronto'),
      unit: OmniAI::Tool::Property.string(enum: %w[celcius fahrenheit]),
    },
    required: %i[location]
  )
)
client.chat('What is the weather in "London" in celcius and "Paris" in fahrenheit?', tools: [tool])

Transcribe

Clients that support transcribe (e.g. OpenAI w/ "Whisper") convert recordings to text via the following calls:

Transcriptions with Path

transcription = client.transcribe("example.ogg")
transcription.text # '...'

Transcriptions with Files

File.open("example.ogg", "rb") do |file|
  transcription = client.transcribe(file)
  transcription.text # '...'
end

Speak

Clients that support speak (e.g. OpenAI w/ "Whisper") convert text to recordings via the following calls:

Speech with Stream

File.open('example.ogg', 'wb') do |file|
  client.speak('The quick brown fox jumps over a lazy dog.', voice: 'HAL') do |chunk|
    file << chunk
  end
end

Speech with File

tempfile = client.speak('The quick brown fox jumps over a lazy dog.', voice: 'HAL')
tempfile.close
tempfile.unlink

Embeddings

Clients that support generating embeddings (e.g. OpenAI, Mistral, etc.) convert text to embeddings via the following:

response = client.embed('The quick brown fox jumps over a lazy dog')
response.usage # <OmniAI::Embed::Usage prompt_tokens=5 total_tokens=5>
response.embedding # [0.1, 0.2, ...] >

Batches of text can also be converted to embeddings via the following:

response = client.embed([
  '',
  '',
])
response.usage # <OmniAI::Embed::Usage prompt_tokens=5 total_tokens=5>
response.embeddings.each do |embedding|
  embedding # [0.1, 0.2, ...]
end

CLI

OmniAI packages a basic command line interface (CLI) to allow for exploration of various APIs. A detailed CLI documentation can be found via help:

omniai --help

Chat

w/ a Prompt

omniai chat "What is the coldest place on earth?"

The coldest place on earth is Antarctica.

w/o a Prompt

omniai chat --provider="openai" --model="gpt-4" --temperature="0.5"

Type 'exit' or 'quit' to abort.
# What is the warmet place on earth?

The warmest place on earth is Africa.

Embed

w/ input

omniai embed "The quick brown fox jumps over a lazy dog."

0.0
...

w/o input

omniai embed --provider="openai" --model="text-embedding-ada-002"

Type 'exit' or 'quit' to abort.
# Whe quick brown fox jumps over a lazy dog.

0.0
...

omniai

Runtime

OmniAI

Examples

Example #1: Chat

Example #2: Text-to-Speech

Example #3: Speech-to-Text

Example #4: Tools

Example #5: Embeddings

Installation

Usage

Client

OmniAI::Anthropic

OmniAI::Google

OmniAI::Mistral

OmniAI::OpenAI

Usage with LocalAI

Usage with Ollama

Logging

Timeouts

Chat

Completions using a Simple Prompt

Completions using a Complex Prompt

Completions using Streaming via Proc

Completion using Streaming via IO

Completion with Tools

Transcribe

Transcriptions with Path

Transcriptions with Files

Speak

Speech with Stream

Speech with File

Embeddings

CLI

Chat

w/ a Prompt

w/o a Prompt

Embed

w/ input

w/o input