# Redis::Objects - Map Redis types directly to Ruby objects

[![Build Status](https://app.travis-ci.com/nateware/redis-objects.svg?branch=master)](https://travis-ci.com/github/nateware/redis-objects)
[![Code Coverage](https://codecov.io/gh/nateware/redis-objects/branch/master/graph/badge.svg)](https://codecov.io/gh/nateware/redis-objects)
[![Donate](https://www.paypalobjects.com/en_US/i/btn/btn_donate_SM.gif)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=MJF7JU5M7F8VL)

## Important 2.0 changes

Several longstanding bugs have been addressed in this release. However, this may require some
code changes as part of the upgrade.

### Renaming of `lock` Method

The `lock` method that collided with `ActiveRecord::Base` has been renamed `redis_lock`.
This means your classes need to be updated to call `redis_lock` instead:

```ruby
class YouClassNameHere < ActiveRecord::Base
  include Redis::Objects
  redis_lock :mylock  # formerly just "lock"
end
```

For more details on the issue and fix refer to [#196](https://github.com/nateware/redis-objects/issues/196).

### New key naming method

A new method to determine the internal key naming scheme has been added to fix a
longstanding bug. (Refer to [#213](https://github.com/nateware/redis-objects/issues/231))
By default, backwards compatibility is maintained, but this means that in some cases,
Nested classes (for example `Dog::Behavior` and `Cat::Behavior`) would have key names
that collide (they would both start with `Behavior` as anything before `::` is stripped).

To ensure names are unique, you can set `Redis::Objects.prefix_style = :modern` after
you load the module. By default, it is set to `Redis::Objects.prefix_style = :legacy`.

### Migrating old keys

If your Redis::Object subclasses are nested such as `Dog::Behavior` and `Cat::Behavior`,
then you should upgrade them using the below proceedure. (only needed once)

Create a script like this:

```ruby
class Dog::Behavior < ActiveRecord::Base
  include Redis::Objects
  # ... your relevant redis_object definitions here (counters/sets) ...
end

Dog::Behavior.migrate_redis_legacy_keys
```

You need to find a time when you can temporarily pause writes to your redis server
so that you can run that script. It uses `redis.scan` internally so it should be able to
handle a high number of keys. For large data sets, it could take a while.

After migrating all of your redis keys, update `Redis::Objects.prefix_style = :modern` to
start using the new keys.

**Note: Your existing data in Redis will not be accessible after running `migrate_redis_legacy_keys`
until the `prefix_style` has been changed.**

---

# Overview

This is **not** an ORM ([Object-Relational Mapping](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping)).
People that are wrapping ORM’s around Redis are missing the point.

The killer feature of Redis is that it allows you to perform _atomic_ operations
on _individual_ data structures, like counters, lists, and sets. The **atomic** part is HUGE.
Using an ORM wrapper that retrieves a "record", updates values, then sends those values back,
_removes_ the atomicity, and thus the major advantage of Redis. Just use MySQL, k?

This gem provides a Rubyish interface to Redis, by mapping [Redis data types](http://redis.io/commands)
to Ruby objects, via a thin layer over the `redis` gem. It offers several advantages
over the lower-level redis-rb API:

1. Easy to integrate directly with existing ORMs - ActiveRecord, DataMapper, etc. Add counters to your model!
2. Complex data structures are automatically Marshaled (if you set :marshal => true)
3. Integers are returned as integers, rather than '17'
4. Higher-level types are provided, such as Locks, that wrap multiple calls

This gem originally arose out of a need for high-concurrency atomic operations;
for a fun rant on the topic, see [An Atomic Rant](http://nateware.com/2010/02/18/an-atomic-rant),
or scroll down to [Atomic Counters and Locks](#atomicity) in this README.

There are two ways to use Redis::Objects, either as an include within a [model class](#option-1-model-class-include) (to
tightly integrate with ORMs or other classes), or [standalone](#option-2-standalone-usage) by using classes such
as `Redis::List` and `Redis::SortedSet`.

# Installation and Setup

Add it to your Gemfile as:

```ruby
gem 'redis-objects'
```

Redis::Objects needs a handle created by `Redis.new` or a [ConnectionPool](https://github.com/mperham/connection_pool).

If you're using Rails, `config/initializers/redis.rb` is a good place for this.
However, there are **no** dependencies on Rails. Redis::Objects can be used in any Ruby code; Sinatra,
Resque, or Standalone - no problem.

The recommended approach is to use a `ConnectionPool` since this guarantees that most timeouts in the `redis` client
do not pollute your existing connection.

```ruby
require 'connection_pool'
Redis::Objects.redis = ConnectionPool.new(size: 5, timeout: 5) { Redis.new(:host => '127.0.0.1', :port => 6379) }
```

However, you need to make sure that both `:timeout` and `:size` are set appropriately
in a multithreaded environment.

Alternatively, you can set the `redis` handle directly:

```ruby
Redis::Objects.redis = Redis.new(...)
```

Redis::Objects will also default to `Redis.current` if `Redis::Objects.redis` is not set.

```ruby
Redis.current = Redis.new(:host => '127.0.0.1', :port => 6379)
```

Finally, you can even set different handles for different classes:

```ruby
class User
  include Redis::Objects
end
class Post
  include Redis::Objects
end

# you can also use a ConnectionPool here as well
User.redis = Redis.new(:host => '1.2.3.4')
Post.redis = Redis.new(:host => '5.6.7.8')
```

As of `0.7.0`, `redis-objects` now autoloads the appropriate `Redis::Whatever`
classes on demand. Previous strategies of individually requiring `redis/list`
or `redis/set` are no longer required.

# Option 1: Model Class Include

Including Redis::Objects in a model class makes it trivial to integrate Redis types
with an existing ActiveRecord, DataMapper, Mongoid, or similar class. **Redis::Objects
will work with _any_ class that provides an `id` method that returns a unique value.**
Redis::Objects automatically creates keys that are unique to each object, in the format:

    model_name:id:field_name

For illustration purposes, consider this stub class:

```ruby
class User
  include Redis::Objects
  counter :my_posts
  def id
    1
  end
end

user = User.new
user.id  # 1
user.my_posts.increment
user.my_posts.increment
user.my_posts.increment
puts user.my_posts.value # 3
user.my_posts.reset
puts user.my_posts.value # 0
user.my_posts.reset 5
puts user.my_posts.value # 5
```

Here's an example that integrates several data types with an ActiveRecord model:

```ruby
class Team < ActiveRecord::Base
  include Redis::Objects

  redis_lock :trade_players, :expiration => 15  # sec
  value :at_bat
  counter :hits
  counter :runs
  counter :outs
  counter :inning, :start => 1
  list :on_base
  list :coaches, :marshal => true
  set  :outfielders
  hash_key :pitchers_faced  # "hash" is taken by Ruby

  # Customized keys
  counter :player_totals, :key => 'players/#{username}/total'
  list :all_player_stats, :key => 'players:all_stats', :global => true
  set :total_wins,        :key => 'players:#{id}:all_stats'
  value :my_rank,         :key => 'players:my_rank:#{username}'

  def id; @id; end
  def username; "user#{id}"; end
end
```

Familiar Ruby array operations Just Work™:

```ruby
@team = Team.find_by_name('New York Yankees')
@team.on_base << 'player1'
@team.on_base << 'player2'
@team.on_base << 'player3'
@team.on_base    # ['player1', 'player2', 'player3']
@team.on_base.pop
@team.on_base.shift
@team.on_base.length  # 1
@team.on_base.delete('player2')
@team.on_base = ['player1', 'player2']  # ['player1', 'player2']
```

Sets work too:

```ruby
@team.outfielders << 'outfielder1'
@team.outfielders << 'outfielder2'
@team.outfielders << 'outfielder1'   # dup ignored
@team.outfielders  # ['outfielder1', 'outfielder2']
@team.outfielders.each do |player|
  puts player
end
player = @team.outfielders.detect{|of| of == 'outfielder2'}
@team.outfielders = ['outfielder1', 'outfielder3']  # ['outfielder1', 'outfielder3']
```

Hashes work too:

```ruby
@team.pitchers_faced['player1'] = 'pitcher2'
@team.pitchers_faced['player2'] = 'pitcher1'
@team.pitchers_faced = { 'player1' => 'pitcher2', 'player2' => 'pitcher1' }
```

And you can do unions and intersections between objects (kinda cool):

```ruby
@team1.outfielders | @team2.outfielders   # outfielders on both teams
@team1.outfielders & @team2.outfielders   # in baseball, should be empty :-)
```

Counters can be atomically incremented/decremented (but not assigned):

```ruby
@team.hits.increment  # or incr
@team.hits.decrement  # or decr
@team.hits.incr(3)    # add 3
@team.runs = 4        # exception
```

Defining a different method as the `id` field is easy

```ruby
class User
  include Redis::Objects
  redis_id_field :uid
  counter :my_posts
end

user.uid                # 195137a1bdea4473
user.my_posts.increment # 1
```

You can also define globals redis attributes that are accessed through the class itself.
No id needed/used for these.

```ruby
class Team < ActiveRecord::Base
  include Redis::Objects

  sorted_set :rank, :global => true
end

Team.rank['Yankees']   = 12
Team.rank['Red Socks'] = 5
Team.rank['Mariners']  = 7
Team.rank.members(:with_scores => true) # => [["Red Socks", 5], ["Mariners", 7], ["Yankees", 12]]
```

Finally, for free, you get a `redis` method that points directly to a Redis connection:

```ruby
Team.redis.get('somekey')
@team = Team.new
@team.redis.get('somekey')
@team.redis.smembers('someset')
```

You can use the `redis` handle to directly call any [Redis API command](http://redis.io/commands).

# Option 2: Standalone Usage

There is a Ruby class that maps to each Redis type, with methods for each
[Redis API command](http://redis.io/commands).
Note that calling `new` does not imply it's actually a "new" value - it just
creates a mapping between that Ruby object and the corresponding Redis data
structure, which may already exist on the `redis-server`.

## Counters

The `counter_name` is the key stored in Redis.

```ruby
@counter = Redis::Counter.new('counter_name')
@counter.increment  # or incr
@counter.decrement  # or decr
@counter.increment(3)
puts @counter.value
```

This gem provides a clean way to do atomic blocks as well:

```ruby
@counter.increment do |val|
  raise "Full" if val > MAX_VAL  # rewind counter
end
```

See the section on [Atomic Counters and Locks](#atomicity) for cool uses of atomic counter blocks.

## Locks

A convenience class that wraps the pattern of [using setnx to perform locking](http://redis.io/commands/setnx).

```ruby
@lock = Redis::Lock.new('serialize_stuff', :expiration => 15, :timeout => 0.1)
@lock.lock do
  # do work
end
```

This can be especially useful if you're running batch jobs spread across multiple hosts.

## Values

Simple values are easy as well:

```ruby
@value = Redis::Value.new('value_name')
@value.value = 'a'
@value.delete
```

Complex data is no problem with :marshal => true:

```ruby
@account = Account.create!(params[:account])
@newest  = Redis::Value.new('newest_account', :marshal => true)
@newest.value = @account.attributes
puts @newest.value['username']
```

Compress data to save memory usage on Redis with :compress => true:

```ruby
@account = Account.create!(params[:account])
@marshaled_value = Redis::Value.new('marshaled', :marshal => true, :compress => true)
@marshaled_value.value = @account.attributes
@unmarshaled_value = Redis::Value.new('unmarshaled', :compress => true)
@unmarshaled_value = 'Really Long String'
puts @marshaled_value.value['username']
puts @unmarshaled_value.value
```

## Lists

Lists work just like Ruby arrays:

```ruby
@list = Redis::List.new('list_name')
@list << 'a'
@list << 'b'
@list.include? 'c'   # false
@list.values  # ['a','b']
@list << 'c'
@list.delete('c')
@list[0]
@list[0,1]
@list[0..1]
@list.shift
@list.pop
@list.clear
# etc
```

You can bound the size of the list to only hold N elements like so:

```ruby
# Only holds 10 elements, throws out old ones when you reach :maxlength.
@list = Redis::List.new('list_name', :maxlength => 10)
```

Complex data types are serialized with :marshal => true:

```ruby
@list = Redis::List.new('list_name', :marshal => true)
@list << {:name => "Nate", :city => "San Diego"}
@list << {:name => "Peter", :city => "Oceanside"}
@list.each do |el|
  puts "#{el[:name]} lives in #{el[:city]}"
end
```

Note: If you run into issues, with Marshal errors, refer to the fix in [Issue #176](https://github.com/nateware/redis-objects/issues/176).

## Hashes

Hashes work like a Ruby [Hash](http://ruby-doc.org/core/classes/Hash.html), with
a few Redis-specific additions. (The class name is "HashKey" not just "Hash", due to
conflicts with the Ruby core Hash class in other gems.)

```ruby
@hash = Redis::HashKey.new('hash_name')
@hash['a'] = 1
@hash['b'] = 2
@hash.each do |k,v|
  puts "#{k} = #{v}"
end
@hash['c'] = 3
puts @hash.all  # {"a"=>"1","b"=>"2","c"=>"3"}
@hash.clear
```

Redis also adds incrementing and bulk operations:

```ruby
@hash.incr('c', 6)  # 9
@hash.bulk_set('d' => 5, 'e' => 6)
@hash.bulk_get('d','e')  # "5", "6"
```

Remember that numbers become strings in Redis. Unlike with other Redis data types,
`redis-objects` can't guess at your data type in this situation, since you may
actually mean to store "1.5".

## Sets

Sets work like the Ruby [Set](http://ruby-doc.org/core/classes/Set.html) class.
They are unordered, but guarantee uniqueness of members.

```ruby
@set = Redis::Set.new('set_name')
@set << 'a'
@set << 'b'
@set << 'a'  # dup ignored
@set.member? 'c'      # false
@set.members          # ['a','b']
@set.members.reverse  # ['b','a']
@set.each do |member|
  puts member
end
@set.clear
# etc
```

You can perform Redis intersections/unions/diffs easily:

```ruby
@set1 = Redis::Set.new('set1')
@set2 = Redis::Set.new('set2')
@set3 = Redis::Set.new('set3')
members = @set1 & @set2   # intersection
members = @set1 | @set2   # union
members = @set1 + @set2   # union
members = @set1 ^ @set2   # difference
members = @set1 - @set2   # difference
members = @set1.intersection(@set2, @set3)  # multiple
members = @set1.union(@set2, @set3)         # multiple
members = @set1.difference(@set2, @set3)    # multiple
```

Or store them in Redis:

```ruby
@set1.interstore('intername', @set2, @set3)
members = @set1.redis.get('intername')
@set1.unionstore('unionname', @set2, @set3)
members = @set1.redis.get('unionname')
@set1.diffstore('diffname', @set2, @set3)
members = @set1.redis.get('diffname')
```

And use complex data types too, with :marshal => true:

```ruby
@set1 = Redis::Set.new('set1', :marshal => true)
@set2 = Redis::Set.new('set2', :marshal => true)
@set1 << {:name => "Nate",  :city => "San Diego"}
@set1 << {:name => "Peter", :city => "Oceanside"}
@set2 << {:name => "Nate",  :city => "San Diego"}
@set2 << {:name => "Jeff",  :city => "Del Mar"}

@set1 & @set2  # Nate
@set1 - @set2  # Peter
@set1 | @set2  # all 3 people
```

## Sorted Sets

Due to their unique properties, Sorted Sets work like a hybrid between
a Hash and an Array. You assign like a Hash, but retrieve like an Array:

```ruby
@sorted_set = Redis::SortedSet.new('number_of_posts')
@sorted_set['Nate']  = 15
@sorted_set['Peter'] = 75
@sorted_set['Jeff']  = 24

# Array access to get sorted order
@sorted_set[0..2]           # => ["Nate", "Jeff", "Peter"]
@sorted_set[0,2]            # => ["Nate", "Jeff"]

@sorted_set['Peter']        # => 75
@sorted_set['Jeff']         # => 24
@sorted_set.score('Jeff')   # same thing (24)

@sorted_set.rank('Peter')   # => 2
@sorted_set.rank('Jeff')    # => 1

@sorted_set.first           # => "Nate"
@sorted_set.last            # => "Peter"
@sorted_set.revrange(0,2)   # => ["Peter", "Jeff", "Nate"]

@sorted_set['Newbie'] = 1
@sorted_set.members         # => ["Newbie", "Nate", "Jeff", "Peter"]
@sorted_set.members.reverse # => ["Peter", "Jeff", "Nate", "Newbie"]

@sorted_set.rangebyscore(10, 100, :limit => 2)   # => ["Nate", "Jeff"]
@sorted_set.members(:with_scores => true)        # => [["Newbie", 1], ["Nate", 16], ["Jeff", 28], ["Peter", 76]]

# atomic increment
@sorted_set.increment('Nate')
@sorted_set.incr('Peter')   # shorthand
@sorted_set.incr('Jeff', 4)
```

The other Redis Sorted Set commands are supported as well; see [Sorted Sets API](http://redis.io/commands#sorted_set).

<a name="atomicity"></a>
Atomic Counters and Locks

---

You are probably not handling atomicity correctly in your app. For a fun rant
on the topic, see [An Atomic Rant](http://nateware.com/an-atomic-rant.html).

Atomic counters are a good way to handle concurrency:

```ruby
@team = Team.find(1)
if @team.drafted_players.increment <= @team.max_players
  # do stuff
  @team.team_players.create!(:player_id => 221)
  @team.active_players.increment
else
  # reset counter state
  @team.drafted_players.decrement
end
```

An _atomic block_ gives you a cleaner way to do the above. Exceptions or returning nil
will rewind the counter back to its previous state:

```ruby
@team.drafted_players.increment do |val|
  raise Team::TeamFullError if val > @team.max_players  # rewind
  @team.team_players.create!(:player_id => 221)
  @team.active_players.increment
end
```

Here's a similar approach, using an if block (failure rewinds counter):

```ruby
@team.drafted_players.increment do |val|
  if val <= @team.max_players
    @team.team_players.create!(:player_id => 221)
    @team.active_players.increment
  end
end
```

Class methods work too, using the familiar ActiveRecord counter syntax:

```ruby
Team.increment_counter :drafted_players, team_id
Team.decrement_counter :drafted_players, team_id, 2
Team.increment_counter :total_online_players  # no ID on global counter
```

Class-level atomic blocks can also be used. This may save a DB fetch, if you have
a record ID and don't need any other attributes from the DB table:

```ruby
Team.increment_counter(:drafted_players, team_id) do |val|
  TeamPitcher.create!(:team_id => team_id, :pitcher_id => 181)
  Team.increment_counter(:active_players, team_id)
end
```

### Locks

Locks work similarly. On completion or exception the lock is released:

```ruby
class Team < ActiveRecord::Base
  redis_lock :reorder # declare a lock
end

@team.reorder_lock.lock do
  @team.reorder_all_players
end
```

Class-level lock (same concept)

```ruby
Team.obtain_lock(:reorder, team_id) do
  Team.reorder_all_players(team_id)
end
```

Lock expiration. Sometimes you want to make sure your locks are cleaned up should
the unthinkable happen (server failure). You can set lock expirations to handle
this. Expired locks are released by the next process to attempt lock. Just
make sure you expiration value is sufficiently large compared to your expected
lock time.

```ruby
class Team < ActiveRecord::Base
  redis_lock :reorder, :expiration => 15.minutes
end
```

Keep in mind that true locks serialize your entire application at that point. As
such, atomic counters are strongly preferred.

### Expiration

Use :expiration and :expireat options to set default expiration.

```ruby
value :value_with_expiration, :expiration => 1.hour
value :value_with_expireat, :expireat => lambda { Time.now + 1.hour }
```

:warning: In the above example, `expiration` is evaluated at class load time.
In this example, it will be one hour after loading the class, not after one hour
after setting a value. If you want to expire one hour after setting the value,
please use `:expireat` with `lambda`.

## Custom serialization

You can customize how values are serialized by setting `serializer: CustomSerializer`.
The default is `Marshal` from the standard lib, but it can be anything that responds to `dump` and
`load`. `JSON` and `YAML` are popular options.

If you need to pass extra arguments to `dump` or `load`, you can set
`marshal_dump_args: { foo: 'bar' }` and `marshal_load_args: { foo: 'bar' }` respectively.

```ruby
class CustomSerializer
  def self.dump(value)
    # custom code for serializing
  end

  def self.load(value)
    # custom code for deserializing
  end
end

@account = Account.create!(params[:account])
@newest  = Redis::Value.new('custom_serializer', marshal: true, serializer: CustomSerializer)
@newest.value = @account.attributes
```

---

## Under the Hood

Redis keys are prefixed to namespace keys with the same names.
By default the prefix is generated from the embedded class's name.
But you can also set a custom prefix using `redis_prefix=`.

If needed the redis key for a specific attribute of a class can be obtained by:
`MyClass.redis_field_key(attr_name, primary_id)`
or for globals
`MyClass.redis_field_key(attr_name)`

## Author

Copyright (c) 2009-2026 [Nate Wiger](http://nateware.com). All Rights Reserved.
Released under the [Artistic License](http://www.opensource.org/licenses/artistic-license-2.0.php).