8  Threads, Fibers, and Continuations

Thread.new call and will continue with the following statement. The newly created



thread will exit when the block exits. The return value of the block becomes available

through the value method of the Thread object. (If you call this method before the thread

has completed, the caller will block until the thread returns a value.)

The following code shows how you might use threads to read the contents of multiple

files in parallel:

# This method expects an array of filenames.

# It returns an array of strings holding the content of the named files.

# The method creates one thread for each named file.

def readfiles(filenames)

# Create an array of threads from the array of filenames.

# Each thread starts reading a file.

threads = filenames.map do |f|

Thread.new { File.read(f) }

end

# Now create an array of

# method of each thread.

# until the thread exits

threads.map {|t| t.value

end



file contents by calling the value

This method blocks, if necessary,

with a value.

}



See §9.9 for much more about threads and concurrency in Ruby.



5.8.2 Fibers for Coroutines

Ruby 1.9 introduces a control structure known as a fiber and represented by an object

of class Fiber. The name “fiber” has been used elsewhere for a kind of lightweight

thread, but Ruby’s fibers are better described as coroutines or, more accurately, semicoroutines. The most common use for coroutines is to implement generators: objects

that can compute a partial result, yield the result back to the caller, and save the state

of the computation so that the caller can resume that computation to obtain the next

result. In Ruby, the Fiber class is used to enable the automatic conversion of internal

iterators, such as the each method, into enumerators or external iterators.

Note that fibers are an advanced and relatively obscure control structure; the majority

of Ruby programmers will never need to use the Fiber class directly. If you have never

programed with coroutines or generators before, you may find them difficult to

understand at first. If so, study the examples carefully and try out some examples of

your own.

A fiber has a body of code like a thread does. Create a fiber with Fiber.new, and associate

a block with it to specify the code that the fiber is to run. Unlike a thread, the body of

a fiber does not start executing right away. To run a fiber, call the resume method of

the Fiber object that represents it. The first time resume is called on a fiber, control is

transferred to the beginning of the fiber body. That fiber then runs until it reaches the

end of the body, or until it executes the class method Fiber.yield. The Fiber.yield



5.8 Threads, Fibers, and Continuations | 167



method transfers control back to the caller and makes the call to resume return. It also

saves the state of the fiber, so that the next call to resume makes the fiber pick up where

it left off. Here is a simple example:

f = Fiber.new {

puts "Fiber says Hello"

Fiber.yield

puts "Fiber says Goodbye"

}

puts "Caller says Hello"

f.resume

puts "Caller says Goodbye"

f.resume



#

#

#

#

#

#

#

#

#

#

#



Line 1:

Line 2:

Line 3:

Line 4:

Line 5:

Line 6:

Line 7:

Line 8:

Line 9:

Line 10:

Line 11:



Create a new fiber

goto line 9

goto line 11

goto line 2

goto line 4



The body of the fiber does not run when it is first created, so this code creates a fiber

but does not produce any output until it reaches line 7. The resume and Fiber.yield

calls then transfer control back and forth so that the messages from the fiber and the

caller are interleaved. The code produces the following output:

Caller says Hello

Fiber says Hello

Caller says Goodbye

Fiber says Goodbye



It is worth noting here that the “yielding” performed by Fiber.yield is completely

different than the yielding performed by the yield statement. Fiber.yield yields control

from the current fiber back to the caller that invoked it. The yield statement, on the

other hand, yields control from an iterator method to the block associated with the

method.



5.8.2.1 Fiber arguments and return values

Fibers and their callers can exchange data through the arguments and return values of

resume and yield. The arguments to the first call to resume are passed to the block

associated with the fiber: they become the values of the block parameters. On

subsequent calls, the arguments to resume become the return value of Fiber.yield.

Conversely, any arguments to Fiber.yield become the return value of resume. And

when the block exits, the value of the last expression evaluated also becomes the return

value of resume. The following code demonstrates this:

f = Fiber.new do |message|

puts "Caller said: #{message}"

message2 = Fiber.yield("Hello")

puts "Caller said: #{message2}"

"Fine"

end

response = f.resume("Hello")

puts "Fiber said: #{response}"



168 | Chapter 5: Statements and Control Structures



# "Hello" returned by first resume

# "Fine" returned by second resume

# "Hello" passed to block



response2 = f.resume("How are you?") # "How are you?" returned by Fiber.yield

puts "Fiber said: #{response2}"



The caller passes two messages to the fiber, and the fiber returns two responses to the

caller. It prints:

Caller said: Hello

Fiber said: Hello

Caller said: How are you?

Fiber said: Fine



In the caller’s code, the messages are always arguments to resume, and the responses

are always the return value of that method. In the body of the fiber, all messages but

the first are received as the return value of Fiber.yield, and all responses but the last

are passed as arguments to Fiber.yield. The first message is received through block

parameters, and the last response is the return value of the block itself.



5.8.2.2 Implementing generators with fibers

The fiber examples shown so far have not been terribly realistic. Here we demonstrate

some more typical uses. First, we write a Fibonacci number generator—a Fiber object

that returns successive members of the Fibonacci sequence on each call to resume:

# Return a Fiber to compute Fibonacci numbers

def fibonacci_generator(x0,y0)

# Base the sequence on x0,y0

Fiber.new do

x,y = x0, y0

# Initialize x and y

loop do

# This fiber runs forever

Fiber.yield y

# Yield the next number in the sequence

x,y = y,x+y

# Update x and y

end

end

end

g = fibonacci_generator(0,1)

# Create a generator

10.times { print g.resume, " " } # And use it



The code above prints the first 10 Fibonacci numbers:

1 1 2 3 5 8 13 21 34 55



Because Fiber is a confusing control structure, we might prefer to hide its API when

writing generators. Here is another version of a Fibonacci number generator. It defines

its own class and implements the same next and rewind API that enumerators do:

class FibonacciGenerator

def initialize

@x,@y = 0,1

@fiber = Fiber.new do

loop do

@x,@y = @y, @x+@y

Fiber.yield @x

end

end

end



5.8 Threads, Fibers, and Continuations | 169



def next

@fiber.resume

end

def rewind

@x,@y = 0,1

end

end



# Return the next Fibonacci number



# Restart the sequence



g = FibonacciGenerator.new

10.times { print g.next, " " }

g.rewind; puts

10.times { print g.next, " " }



#

#

#

#



Create a generator

Print first 10 numbers

Start over, on a new line

Print the first 10 again



Note that we can make this FibonacciGenerator class Enumerable by including the

Enumerable module and adding the following each method (which we first used in

§5.3.5):

def each

loop { yield self.next }

end



Conversely, suppose we have an Enumerable object and want to make an enumeratorstyle generator out of it. We can use this class:

class Generator

def initialize(enumerable)

@enumerable = enumerable

create_fiber

end



# Remember the enumerable object

# Create a fiber to enumerate it



def next

@fiber.resume

end



# Return the next element

# by resuming the fiber



def rewind

create_fiber

end



# Start the enumeration over

# by creating a new fiber



private

def create_fiber

# Create the fiber that does the enumeration

@fiber = Fiber.new do

# Create a new fiber

@enumerable.each do |x| # Use the each method

Fiber.yield(x)

# But pause during enumeration to return values

end

raise StopIteration

# Raise this when we're out of values

end

end

end

g = Generator.new(1..10)

loop { print g.next }

g.rewind

g = (1..10).to_enum

loop { print g.next }



#

#

#

#



Create a generator from an Enumerable like this

And use it like an enumerator like this

Start over like this

The to_enum method does the same thing



170 | Chapter 5: Statements and Control Structures



Although it is useful to study the implementation of this Generator class, the class itself

doesn’t provide any functionality over that provided by the to_enum method.



5.8.2.3 Advanced fiber features

The fiber module in the standard library enables additional, more powerful features

of the fibers. To use these features, you must:

require 'fiber'



However, you should avoid using these additional features wherever possible, because:

• They are not supported by all implementations. JRuby, for example, cannot

support them on current Java VMs.

• They are so powerful that misusing them can crash the Ruby VM.

The core features of the Fiber class implement semicoroutines. These are not true coroutines because there is a fundamental asymmetry between the caller and the fiber:

the caller uses resume and the fiber uses yield. If you require the fiber library, however,

the Fiber class gets a transfer method that allows any fiber to transfer control to any

other fiber. Here is an example in which two fibers use the transfer method to pass

control (and values) back and forth:

require 'fiber'

f = g = nil

f = Fiber.new {|x|

puts "f1: #{x}"

x = g.transfer(x+1)

puts "f2: #{x}"

x = g.transfer(x+1)

puts "f3: #{x}"

x + 1

}

g = Fiber.new {|x|

puts "g1: #{x}"

x = f.transfer(x+1)

puts "g2: #{x}"

x = f.transfer(x+1)

}

puts f.transfer(1)



#

#

#

#

#

#

#



1:

2:

3:

4:

5:

6:

7:



# 8:

# 9:

#10:

#11:

#12:



print "f1: 1"

pass 2 to line 8

print "f2: 3"

return 4 to line 10

print "f3: 5"

return 6 to line 13

print "g1: 2"

return 3 to line 3

print "g2: 4"

return 5 to line 5



#13: pass 1 to line 1



This code produces the following output:

f1:

g1:

f2:

g2:

f3:

6



1

2

3

4

5



You will probably never need to use this transfer method, but its existence helps explain the name “fiber.” Fibers can be thought of as independent paths of execution

5.8 Threads, Fibers, and Continuations | 171



within a single thread of execution. Unlike threads, however, there is no scheduler to

transfer control among fibers; fibers must explicitly schedule themselves with transfer.

In addition to the transfer method, the fiber library also defines an instance method

alive?, to determine if the body of a fiber is still running, and a class method current,

to return the Fiber object that currently has control.



5.8.3 Continuations

A continuation is another complex and obscure control structure that most programmers will never need to use. A continuation takes the form of the Kernel method

callcc and the Continuation object. Continuations are part of the core platform in Ruby

1.8, but they have been replaced by fibers and moved to the standard library in Ruby

1.9. To use them in Ruby 1.9, you must explicitly require them with:

require 'continuation'



Implementation difficulties prevent other implementations of Ruby (such as JRuby,

the Java-based implementation) from supporting continuations. Because they are no

longer well supported, continuations should be considered a curiosity, and new Ruby

code should not use them. If you have Ruby 1.8 code that relies on continuations, you

may be able to convert it to use fibers in Ruby 1.9.

The Kernel method callcc executes its block, passing a newly created Continuation

object as the only argument. The Continuation object has a call method, which makes

the callcc invocation return to its caller. The value passed to call becomes the return

value of the callcc invocation. In this sense, callcc is like catch, and the call method

of the Continuation object is like throw.

Continuations are different, however, because the Continuation object can be saved

into a variable outside of the callcc block. The call method of this object may be called

repeatedly, and causes control to jump to the first statement following the callcc

invocation.

The following code demonstrates how continuations can be used to define a method

that works like the goto statement in the BASIC programming language:

# Global hash for mapping line numbers (or symbols) to continuations

$lines = {}

# Create a continuation and map it to the specified line number

def line(symbol)

callcc {|c| $lines[symbol] = c }

end

# Look up the continuation associated with the number, and jump there

def goto(symbol)

$lines[symbol].call

end

# Now we can pretend we're programming in BASIC



172 | Chapter 5: Statements and Control Structures



i = 0

line 10

puts i += 1

goto 10 if i < 5

line 20

puts i -= 1

goto 20 if i > 0



# Declare this spot to be line 10

# Jump back to line 10 if the condition is met

# Declare this spot to be line 20



5.8 Threads, Fibers, and Continuations | 173



CHAPTER 6



Methods, Procs, Lambdas, and Closures



175



A method is a named block of parameterized code associated with one or more objects.

A method invocation specifies the method name, the object on which it is to be invoked

(sometimes called the receiver), and zero or more argument values that are assigned to

the named method parameters. The value of the last expression evaluated in the method

becomes the value of the method invocation expression.

Many languages distinguish between functions, which have no associated object, and

methods, which are invoked on a receiver object. Because Ruby is a purely objectoriented language, all methods are true methods and are associated with at least one

object. We have not covered class definitions in Ruby yet, so the example methods

defined in this chapter look like global functions with no associated object. In fact,

Ruby implicitly defines and invokes them as private methods of the Object class.

Methods are a fundamental part of Ruby’s syntax, but they are not values that Ruby

programs can operate on. That is, Ruby’s methods are not objects in the way that

strings, numbers, and arrays are. It is possible, however, to obtain a Method object that

represents a given method, and we can invoke methods indirectly through Method

objects.

Methods are not Ruby’s only form of parameterized executable code. Blocks, which

we introduced in §5.4, are executable chunks of code and may have parameters. Unlike

methods, blocks do not have names, and they can only be invoked indirectly through

an iterator method.

Blocks, like methods, are not objects that Ruby can manipulate. But it’s possible to

create an object that represents a block, and this is actually done with some frequency

in Ruby programs. A Proc object represents a block. Like a Method object, we can execute

the code of a block through the Proc that represents it. There are two varieties of Proc

objects, called procs and lambdas, which have slightly different behavior. Both procs

and lambdas are functions rather than methods invoked on an object. An important

feature of procs and lambdas is that they are closures: they retain access to the local

variables that were in scope when they were defined, even when the proc or lambda is

invoked from a different scope.

Methods have a rich and fairly complex syntax in Ruby, and the first four sections of

this chapter are dedicated to them. We begin by explaining how to define simple methods, and then follow this introductory section with three more advanced sections

covering methods names, method parentheses, and method parameters. Note that

method invocation is a kind of expression, covered earlier in §4.4. Further details on

method invocation are provided throughout the first four sections of this chapter.

After covering methods, we turn our attention to procs and lambdas, explaining how

to create and invoke them, and also detailing the somewhat subtle differences between

them. A separate section covers the use of procs and lambdas as closures. This is followed by a section on the Method object, which actually behaves much like a lambda.

The chapter ends with an advanced exploration of functional programming in Ruby.



176 | Chapter 6: Methods, Procs, Lambdas, and Closures