Streaming

Stream-processing response data

em-http invokes callback function when the request is fully parsed. However, in certain cases you want to connect to an endpoint and consume a long-lived stream of data, or process the response in small chunks. To address this, you can define a stream function on your connection object: anytime data is received by the connection, the data will be automatically passed to it.

EventMachine.run do
  http = EventMachine::HttpRequest.new('http://www.twitter.com/firehose').get
  http.stream { |chunk| print chunk }
end

Some API’s, such as Twitter Streaming API provide a JSON stream over a long-lived HTTP connection – a perfect use case for a stream callback. Add a Yajl (JSON) stream parser, and you’re off to the races!

Streaming data off disk

EventMachine provides a send_file like utility for efficient uploads of files directly off disk – all you have to do is give EventMachine the name of the file and it will do the rest.

EventMachine.run do
  http = EventMachine::HttpRequest.new('http://www.website.com/').post :file => 'largefile.txt'
  http.callback { |chunk| puts "Upload finished!" }
end