Showing posts with label Processing. Show all posts
Showing posts with label Processing. Show all posts

2010-01-02

Merge sort in Scala and Processing (3/3)

(Part 1 2 3)


Odds and Ends


In the first two parts, I demonstrated visualization of the process of merge sorting an array of integers. In this part I will describe the two steps left undescribed: how to create a collection of stages and how to create a repeating iterator. I will also present a shuffling function that can be used to create the initial state.


The Stages class


The Stages class provides a map method which is used by the SortStates class to create a collection of states. The function that creates one of the states needs to 1) see the state array (which contains the previous state) and 2) be told how to rearrange it into the next state. The actual function passed to map is curried with the state array, so the only thing map needs to prepare is the collection of stages of rearrangement information.


class Stages (n: Int) {
   Define generate method 

  def map (f: List[Int] => Array[Int]) =
    generate() map f
}

Define generate method


The generate method creates the stages recursively, bottoming out when the parameter i exceeds n.


  private def generate (i: Int = 2): List[List[Int]] = {
     Define split subroutine 

    if (n <= i) {
       Base case 
    } else {
       Preceding cases 
    }
  }

Base case


In the base case, the state array (whose length is stored in the n field) is in a partially sorted state where two sublists of roughly equal length are individually sorted (e.g. Array(1, 3, 5, 0, 2, 4)) and are to be merged into a completely sorted array. The conversion function, of class StateFunc (described in part 2), will be able perform this operation if given a list containing the total length of the sublists, i.e. List(n).


      List(n) :: Nil

Preceding cases


In each of the cases preceding the base case, the state array consists of k pairs of sorted sublists that are to be merged into k sorted sublists (which will be merged in the next stage, and so on). Thus, in the general case Cx the stage information is a list of 2 × k integers where the sum of each pair is equal to the corresponding integer in the stage information of the following case Cx+1.


      val tail = generate(2 * i)
      split(tail.head) :: tail

Define split subroutine


The split subroutine uses h to create a list of integers whose size is twice that of h and where every element eh is mapped to two elements, the first being e/2 and the second being e-e/2.


    def split (h: List[Int]) =
      h flatMap { e => List(e/2, e-e/2) }

The RepeatingIterator class


The RepeatingIterator class trivially encapsulates a sequence and overrides the next method to reset the internal iterator every time it is exhausted. The hasNext method accordingly always returns true.


class RepeatingIterator[A] (xs: Seq[A]) extends Iterator[A] {
  private var iter = xs.iterator
  override def hasNext = true
  override def next () = {
    if (!iter.hasNext) iter = xs.iterator
    iter.next
  }
}

Shuffling the values


In the example, I passed an array of values in reverse order to SortStates. Alternatively, the array can be shuffled like this:

object Shuffle {
  private val rand = new scala.util.Random

  def apply[A] (xs: Array[A]) = {
    for (n <- 0 until xs.size) {
      val k = n + rand.nextInt(xs.size-n)
      val v = xs(k)
      xs(k) = xs(n)
      xs(n) = v
    }
    xs take xs.size
  }
}

And that's merge sort.

2009-12-18

Merge sort in Scala and Processing (2/3)

(Part 1 2 3)


For my merge sort demonstration, I wanted a factory object producing snapshots of progressive sorting in an array of integers. It would essentially be a (partial) function with the signature
(Array[Int]) ⇒ RepeatingIterator[Int].


The argument to the factory is an array of integers that represents the initial state, before sorting.


The class RepeatingIterator is derived from Iterator. It repeats infinitely by yielding the first item after the last, and so on.


  object SortStates {
    def apply (init: Array[Int]) = {
       Create stages 

       Define conversion function 

      new RepeatingIterator(Seq(init) ++
        (stages map convFn))
    }
  }


Create stages


The states of partial to complete ordering are mapped from an instance of Stages (which is a structure that will be described in part 3; suffice to say that it provides a method def map (f: List[Int] ⇒ Array[Int])).


      val size   = init.size
      val stages = new Stages(size)

Define conversion function


The function passed to map is an object whose apply method takes two arguments: a copy of the initial state, and one of the stages mapped over.


      val convFn = StateFunc(init take size) _

The StateFunc object


This object is a function (Array[Int])(List[Int]) ⇒ Array[Int] that takes as its first argument the previous state of the values. The second argument is a list that describes one stage in the merging process. After building a state with buildState, it is written back to the data parameter and finally returned.

object StateFunc {
   Define merge method 

   Define splitMerge method 

   Define buildState method 

  def apply (data: Array[Int])(stage: List[Int]) = {
    val state =
      buildState(data, stage).flatten.toArray

    for (i ← 0 until data.size)
      data(i) = state(i)

    state
  }
}

Define buildState method


The buildState method builds a new state by merging the sorted lists of the previous state. One merge operation is done for each element of the stage data structure. During each merge operation, the following constants are defined:


n
the length of the resulting sublist
s
the total length of the sublists that are already merged
r
a list of n values past the s first values of data

If the sublist length (n) is 1, it is already sorted and r is simply added to the result. Otherwise, r is passed to the method splitMerge.


  private def buildState (data: Array[Int], stage: List[Int]) = {
    for {
      i ← 0 until stage.size
      n = stage(i)
      s = (stage take i sum)
      r = (data drop s take n).toList
    } yield if (n == 1) r else splitMerge(r)
  }

Define splitMerge method


The splitMerge method splits its parameter (which has n values) into two sublists with n / 2 and n - n / 2 values, respectively. The subroutine merge is then applied to the two lists.


  private def splitMerge (r: List[Int]) = {
    val n   = r.size
    val n_2 = n / 2
    val xs1 = r take n_2
    val xs2 = r drop n_2 take (n - n_2)
    merge(xs1, xs2)
  }


Define merge method


The merge method is the same as in the mergeSort function in part 1.


  private def merge (a: List[Int], b: List[Int]): List[Int] =
    (a, b) match {
      case (_, Nil) ⇒ a
      case (Nil, _) ⇒ b
      case (ah :: at, bh :: _) if (ah <= bh) ⇒
        ah :: merge(at, b)
      case (_, bh :: bt) ⇒ bh :: merge(a, bt)
    }

And that's it for SortStates and StateFunc. In the final part I will show Stages and RepeatingIterator.

2009-12-17

Merge sort in Scala and Processing (1/3)

(Part 1 2 3)


Someone asked a question about merge sort the other day, and I was inspired to write a Scala implementation of a merge sort routine.


Assuming that we are sorting a list of integers, a recursive solution can be drawn up as follows.


  • in case the list has no elements, it is already sorted
  • in case the list has one element, it is already sorted
  • otherwise, we can make the list sorted by splitting it in two halves, sort each part, and the merge the parts back together

or

  def mergeSort (xs: List[Int]): List[Int] = {
    xs match {
      case Nil     ⇒ xs
      case List(_) ⇒ xs
      case _       ⇒
        val (xs1, xs2) = xs splitAt (xs.size/2)
        merge(mergeSort(xs1), mergeSort(xs2))
    }
  }

In this context, merging two sorted lists means (Xh means the first element, or head, of X, Xt means the rest, or tail, of X):

  • in case B is empty, take A
  • in case A is empty, take B
  • in case AhBh, take Ah and the result of merging At and B
  • otherwise, take Bh and the result of merging A and Bt

or

  def merge (a: List[Int], b: List[Int]): List[Int] =
    (a, b) match {
      case (_, Nil) ⇒ a
      case (Nil, _) ⇒ b
      case (ah :: at, bh :: _) if (ah <= bh) ⇒
        ah :: merge(at, b)
      case (_, bh :: bt) ⇒ bh :: merge(a, bt)
    }

This can be generalized to a sequence of anything that can be ordered:

  def mergeSort[A<%Ordered[A]] (xs:Seq[A]):Seq[A] = {
  def merge (xs1: Seq[A], xs2: Seq[A]): Seq[A] =
    (xs1, xs2) match {
      case (_, Seq()) ⇒ xs1
      case (Seq(), _) ⇒ xs2
      case (Seq(h1, t1 @ _*), Seq(h2, _*))
        if (h1<=h2) ⇒
          Seq(h1) ++ merge(t1, xs2)
      case (_, Seq(h2, t2 @ _*)) ⇒
        Seq(h2) ++ merge(xs1, t2)
  }
  xs match {
    case Seq()  ⇒ xs
    case Seq(_) ⇒ xs
    case _      ⇒
      val (xs1, xs2) = xs splitAt (xs.size/2)
      merge(mergeSort(xs1), mergeSort(xs2))
  }
}

scala> mergeSort("Hello world")
res1: Seq[Char] = List( , H, d, e, l, l, l, o, o, r, w)

Yawn.

Well, that's not very interesting, is it?

Then I thought, how about using Processing to visualize the sorting process?

A Scala/Processing applet could display a sequence of values as a dot-spread diagram, with position on the x-axis and value on the y-axis. It would basically look like this (n is the number of values, s is the pixel size of each dot):

    for (x ← 0 until n) {
      val y = f(x)
      ellipse(x * s, y * s, s-1, s-1)
    }

In Scala, objects and functions are interchangeable, so f acts as a partial function that maps x (for values in the range 0 ≤ x < n) to y values. With this definition, the loop in draw will display n dots on the screen, with a single dot in each column and row.

However, my goal was to show unsorted values getting more and more sorted, in effect an animated view of the sorting process. To do this, I need more than one state. While I'm at it, I'll define a state as an Array of integers, and the state structure as a Sequence (produced by the Seq object):

  val fStates: Seq[Array[Int]] = Seq(
    Array(4, 8, 7, 6, 2, 5, 9, 3, 1, 0),
    Array(4, 8, 7, 2, 6, 5, 9, 3, 0, 1),
    Array(4, 8, 2, 6, 7, 5, 9, 0, 1, 3),
    Array(2, 4, 6, 7, 8, 0, 1, 3, 5, 9),
    Array(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
  )

  var i = 0

  // ...

    val f = fStates(i)
    for (x ← 0 until f.size) {
      val y = f(x)
      ellipse(x * s, y * s, s-1, s-1)
    }
    i = (i + 1) % fStates.size

This works, but there's room for improvement. For one thing, I don't want to have to specify the values myself, and for another, the indexing exposes too much of the implementation. In the next part, I'll define a factory that produces states and provides them through an iterator, which will let me write the applet this way:

  val numVals = 70
  val dotSize =  6
  val fStates =
    SortStates(Array.range(numVals-1, -1, -1))

  override def setup {
    size(numVals * dotSize, numVals * dotSize)
    frameRate(1)
    smooth
    noStroke
    ellipseMode(CORNER)
  }

  override def draw {
    background(255)
    fill(0)
    val f = fStates.next
    for (x ← 0 until numVals) {
      val y = f(x)
      ellipse(
        x * dotSize,
        y * dotSize,
        dotSize-1,
        dotSize-1
      )
    }
  }

(In this example, the initial state has the values in reverse order.)

2009-11-13

Exing it up

I'm still working on translating Java Processing code to Scala.  I've gone through most of the basic Processing examples now.  To speed the process up a bit, I use a small set of Ex commands.  I load the Java code in a buffer, run :so ec (the name of the Ex file) and paste the text into a boilerplate Scala applet object.  This takes care of most of the common, simple changes and lets me concentrate on the more important aspects of translation.

To begin with, I like to change underlined identifiers, which some examples use, into droopyCamelCase such as foo_bar => fooBar.  I don't want to change certain constants like TWO_PI, though.  The command :%s/_\(\l\)/\u\1/gc does this for me.  If you're unfamiliar with Ex, this command means "in every line (%) search and replace (s) occurrencies of underline and a lowercase character (\l), which is stored as item #1 (\( ... \)) with that item (\1), translated into upper case (\u); do this for every occurence in the line (g), but ask first (c)".  If you are a Vim user, you're probably familiar with this, so I won't explain every command in detail in the following.

In most cases, I want to make structural changes by hand: it's a lot easier and more educating that way.  However, it is very handy to pre-translate the for structure (and also the switch structure, but I haven't gotten around to that yet).  I use two commands, one for cases where iteration includes the end value, and one for cases where it is excluded:

:%s/for\s*(\s*\i\+\s\+\(\i\+\)\s*=\s*\(\w\)[^;]*;[^<]*<=\s*\(\i\+\)[^;]*;\([^)]*\))/for (\1 <- \2 to \3 \/* TODO \4 *\/)/

:%s/for\s*(\s*\i\+\s\+\(\i\+\)\s*=\s*\(\w\)[^;]*;[^<]*<\s*\(\i\+\)[^;]*;\([^)]*\))/for (\1 <- \2 until \3 \/* TODO \4 *\/)/

Note that I just stick the third expression of the Java for (item #4) into a comment -- in case it's just a ++foo or the equivalent the action is implied in the Scala for and I just delete the comment, and in other cases I rewrite as required.

Java and Scala has different syntaxes for declarations: Java uses type name, while Scala uses name: type (like Pascal).  Also, Scala requires you to use the keywords val, var, and def for constants, variables, and methods, respectively (other declarations usually don't use them).  I use a rule of thumb here.  If the declaration is the first thing on the line, I assume it's a constant declaration and rewrite it using this command: :%s/^\(\s*\)\<float\>\s\+\(\w\+\)/\1val \2: Float/g (for float, which is used a lot in Processing code).  In other cases, I use :%s/\<float\>\s\+\(\w\+\)/\1: Float/g.  Currently, I only have commands for ints, floats, and booleans and translate other declarations manually.  If a declared identifier is later used for assignment, I try to restructure the code to be immutable or else just change the val to a var.

As for methods, in Processing they seem to be typed void as a rule.  A few methods, like setup and draw, are inherited from PApplet and need to be explicitly overridden.  In other cases, I just change the void to a def.  The commands are :%s/void \<\(setup\|draw\)\>/override def \1/ and :%s/void/def/.

Now I want to clear out unnecessary separators like semicolons and empty parentheses: :%s/;\s*$// and :%s/(\s*)// take care of (most of) those.  Some lines have an ending comment, which I like to rewrite (code; // comment  =>  // comment<newline>code), but I do that manually and then the semicolon is useful to position the cursor at the point of separation.

Finally, non-whole values are interpreted as Doubles by the Scala compiler, so I need to add an f to them unless it's already there: :%s/\(\.\d\+\)\([^f]\|$\)/\1f\2/g.

Many of the short example programs don't need much more work after these automatic translations.  I might add a few more rules in the future, but it's important to remember that the point of this is to simplify the translation procedure.  Simple, rule-of-thumb translations are good candidates for this kind of automatic processing, but anything that requires analysis is better to do manually.  The for commands are arguably too long, but they still do very simple substitutions and are very helpful.

2009-11-05

Processing Basics

I've been working on a few Scala/Processing projects to get the hang of it, and today I started translating some of the basic examples to Scala just to make sure I was getting it.  This one is the Basics/Array2D example:

size(200, 200)
background(0)
val maxDistance: Float = dist(width/2, height/2, width, height)
val distances = Vector.tabulate(width, height){ case (i, j) =>
  dist(width/2, height/2, j, i)/maxDistance * 255
}

for (i <- 0 until(height, 2) ; j <- 0 until(width, 2)) {
  stroke(distances(j)(i))
  point(j, i)
}

To me, that's a lot better than the original Java.  Less clutter, more higher-level constructs.  Even in this simple example, the Scala code does a better job of showing the intentions rather than the specifics of the algorithm.

2009-10-21

Scala and Processing

I'm trying out Processing together with Scala to write examples for my programming book.  There are at least two ways to use it: 1) Spde, which does some integration work but is a bit of a chore to run, and 2) simply using the Processing core.jar as a Java library from Scala, which is simple but probably has some compatibility problems that will surface eventually.

Processing: http://www.processing.org/
Spde: http://technically.us/spde/About
Using Processing from Scala described here: http://hipstersinc.com/blog/2008/1/23/scala_and_processing/