Jeremy Ashkenas

Code as literature. Give it a read.

Angela Veomett and I made a face for Stephen Hawking. His otherworldly mind is so well serviced by his computer-generated voice. We thought it would be a good idea to provide a computer-generated face to go along with it. It's kinda mesmerizing.

If you have a microphone on your computer, you can download and run the application for Mac, Windows, and Linux. There's a recorded lecture mp3 of Professor Hawking to play out loud while you watch.
A Face for Stephen Hawking

a_face_for_stephen_hawking.rb

This app really shows off the flexibility of Ruby-Processing. The main program is written in Ruby, but it uses the Minim Java library to do pitch, amplitude and beat detection, renders via OpenGL, and uses a pure-Ruby library for the algorithmic flocking. All of this in real-time on a laptop.
   1  # Description:
   2  # Stephen Hawking's otherworldly mind is well represented by his
   3  # computer-generated voice. Here, then, is a computer-generated face
   4  # for Stephen Hawking.
   5  
   6  require 'ruby-processing'
   7  
   8  class FaceForStephenHawking < Processing::App
   9    load_ruby_library "boids"
  10    load_java_library "minim"
  11    load_java_library "opengl"
  12    load_java_library "angela_audio"
  13    include_package "processing.opengl"
  14    include_package "javax.media.opengl"
  15    include_class "audioInterp.AudioIn"
  16    include Math
  17    attr_accessor :flare, :star
  18  
  19    def setup
  20      render_mode OPENGL
  21      hint DISABLE_ERROR_REPORT
  22      hint ENABLE_OPENGL_4X_SMOOTH
  23      sphere_detail 7
  24      color_mode RGB, 1.0
  25      no_stroke
  26      frame_rate 30
  27      shininess 1.0
  28      specular 0.3, 0.1, 0.1
  29      emissive 0.03, 0.03, 0.1
  30      
  31      @flocks = []
  32      @particles = []
  33      @color = 0.5
  34      @radius = 50
  35      @analyzer = AudioIn.new(self)
  36      @analyzer.set_beat_detect_sensitivity(1000)
  37      @flare = load_image "flare.png"
  38      @star = load_image "star.png"
  39      3.times do |i|
  40        flock  = Boids.flock(10, 0, 0, width, height)
  41        flock.goal width/2, height/2, 100
  42        flock.no_perch
  43        boids = flock.boids
  44        @flocks << flock
  45      end
  46      background 0.05
  47    end
  48    
  49    def draw
  50      configure_gl
  51      ambient_light 0.51, 0.51, 0.65
  52      light_specular 0.2, 0.2, 0.2
  53      point_light 0.1, 0.1, 0.1, mouse_x, mouse_y, 100
  54      @particles.each do |particle|
  55        particle.update(@particles)
  56        particle.render
  57      end
  58      @flocks.each { |flock| render_flock(flock) }
  59    end
  60    
  61    def sample_sound
  62      amp = @analyzer.get_amp_left
  63      limit = 2 + (amp * 500)
  64      color = (@analyzer.get_frequency_band_left.to_f / 60.0)
  65      color > @color ? @color += 0.01 : @color -= 0.01
  66      radius = 32 + (amp * 2000)
  67      @radius = (@radius + radius) / 2
  68      explode = @analyzer.beat_detect_left
  69      [limit, @color, @radius, explode]
  70    end
  71    
  72    def render_flock(flock)
  73      limit, color, radius, explode = *sample_sound
  74      flock.update(:goal => 185, :limit => limit, :separation => 7 + (limit / 7))
  75      fill sin(color * PI * 2).abs, cos(color * PI * 2).abs, color
  76      flock.each { |boid| render_boid(boid, radius, explode) }
  77    end
  78    
  79    def render_boid(boid, r, explode)
  80      (r / 13).to_i.times { @particles << Particle.new(boid.x, boid.y, boid.z) } if explode
  81      push_matrix
  82      translate boid.x-r/2, boid.y-r/2, boid.z-r/2
  83      r += boid.z
  84      oval(r/2, r/2, r/3, r/3)
  85      image(@flare, 0, 0, r, r)
  86      pop_matrix
  87    end
  88    
  89    def configure_gl
  90      pgl = g
  91      gl = pgl.gl
  92      pgl.beginGL
  93      gl.gl_depth_mask(false)
  94      gl.gl_disable(GL::GL_DEPTH_TEST)
  95      gl.gl_clear(GL::GL_DEPTH_BUFFER_BIT)
  96      gl.gl_enable(GL::GL_BLEND)
  97      gl.gl_blend_func(GL::GL_SRC_ALPHA, GL::GL_ONE)
  98      pgl.endGL
  99    end  
 100  end
 101  
 102  class Particle
 103    def initialize(x, y, z)
 104      @x, @y, @z = x, y, z
 105      @vel = [0,0].map { rand * 50 - 25 }
 106      @radius = 7 + rand * 30
 107    end
 108    
 109    def update(particles)
 110      @vel = @vel.map {|v| v + (rand * 3 - 1.5)}
 111      @x += @vel[0]
 112      @y += @vel[1]
 113      if @x < 0 || @x > $app.width || @y < 0 || @y > $app.height
 114        particles.delete(self)
 115      end
 116    end
 117    
 118    def render
 119      $app.push_matrix
 120      $app.translate @x, @y, @z
 121      $app.image($app.star, 0, 0, @radius, @radius)
 122      $app.pop_matrix
 123    end
 124  end
 125  
 126  $app = FaceForStephenHawking.new :title => "A Face for Stephen Hawking", 
 127        :width => 1400, :height => 700, :full_screen => true