Web sockets are by far the most efficient way to handle chat functionalities. As a framework, Rails is not very good at handling asynchronous events and therefore establishing a socket connection to a Rails application is almost impossible. There are many solutions that are designed to handle this kind of problem. As Andrew said, Frameworks such as Node.js with Socket.IO would suffice, or, if you want to stick with Ruby, Cramp, async_sinatra, or Goliath framework are all great solutions but it would be super awesome to continue.
If you wanted to use Rails for your application's logic and also have the benefits of some kind of asynchronous event handling with publishing and subscribing you will need to use something like Private Pub Gem which is built on top of Faye and makes it dead simple to publish and subscribe to real-time events in a Rails app.
I have tried to put together a Gmail-like instant message chat application in a rails 4 app. Check out this tutorial. Hope that helps shed some light on the same.