Papier-Mâché User Manual

Introduction

Papier-Mâché is a toolkit for building applications with tangible interfaces.

First Application: Hello World!

For RFID technology, here's a simple application that prints "Hello World!" when a tag gets added and "Goodbye World!" when a tag gets removed:

import edu.berkeley.guir.papier_mache.event.*;
import edu.berkeley.guir.papier_mache.rfid.RFIDManager;
     
public class HelloWorld {
   public static void main(String[] args) {
      final RFIDManager manager = new RFIDManager();
      final PhobListener tagListener = new PhobListener() {
         public void phobAdded(final PhobEvent event) {
            System.out.println("Hello World!");
         }
   
         public void phobUpdated(PhobEvent phobEvent) {}
   
         public void phobRemoved(final PhobEvent event) {
            System.out.println("Goodbye World!");
         }
      };
   
      manager.addPhobListenerForAllReaders(tagListener);
   }
}

Let's dive into the first section for a quick explanation of this application.

Managers, Generators and Listeners: The essentials

The general structure of a Papier-Mache application usually involves creating a DeviceManager or PhobGenerator, creating a listener, and then adding the listener to that manager or generator. For HelloWorld.java, first create a new RFIDManager. Next, create a PhobListener that prints "Hello World!" and "Goodbye World!" when the appropriate phob events get generated; a phob (physical object) for RFID technology represents a tag. As such, phobAdded events occur whenever a tag gets placed on the reader, and phobRemoved events occur whenever a tag is taken off the reader. Lastly, add this PhobListener to all readers so that these events may be generated.

Vision technology works similarly:

import edu.berkeley.guir.papier_mache.event.*;
import edu.berkeley.guir.papier_mache.input.CameraImageSource;
import edu.berkeley.guir.papier_mache.vision.VisionPhobGenerator;

public class HelloWorld {
   private static final int MIN_OBJECT_SIZE = 200;
   private static final int SLEEP_MILLISECS = 800;
     
   public static void main(String[] args) {
      final VisionPhobGenerator generator = new VisionPhobGenerator(
         new CameraImageSource(SLEEP_MILLISECS), MIN_OBJECT_SIZE);
      final PhobListener phobListener = new PhobListener() {
         public void phobAdded(final PhobEvent event) {
           System.out.println("Hello World!");
         }

         public void phobUpdated(PhobEvent phobEvent) {
         }
       
         public void phobRemoved(final PhobEvent event) {
           System.out.println("Goodbye World!");
         }
      };

      generator.addPhobListener(phobListener);
   }
}

The VisionPhobGenerator, instantiated with an ImageSource and an optional minimum object size, now replaces the RFIDManager. In addition, the VisionPhobGenerator adds the listener directly with addPhobListener. HelloWorld.java now prints "Hello World!" anytime a sizeable object gets registered in the vision system and "Goodbye World!" anytime an added object gets removed.

Associations: Mapping tags to meaningful objects and operations

Association Elements: Meaningful objects (nouns) and operations (actions)

AssociationNouns are objects that can have focus; AssociationActions operate on the AssociationNoun that has the application focus. For example, an AudioClip is an AssociationNoun whose phobAdded action is to start playing and its phobRemoved action is to stop playing. FastForwardElt is an AssociationAction that fast forwards the current focused clip.

Maps and Factories

The AssociationMap, a PhobListener, assigns associations to new tags and executes the associations to tags that have already been assigned; effectively, the map serves as a database for associations. The AssociationMap chooses how it assigns the association based upon an AssociationFactory that gets passed into its constructor. The factory may do something as simple as creating an instance of AssociationElt type A the first time, creating an instance of AssociationElt type B the second time, and so on.

The Marble Answering Machine assigns (and makes) recordings to new objects and executes (plays back) the recordings of objects that have already been assigned. This description falls in line with an AssociationMap model as a PhobListener.

import edu.berkeley.guir.papier_mache.Phob;
import edu.berkeley.guir.papier_mache.assoc.*;
import edu.berkeley.guir.papier_mache.event.PhobListener;
import edu.berkeley.guir.papier_mache.rfid.RFIDManager;

public class MarbleAnswering extends DefaultAssociationFactory {
   public static void main(String args[]) {
      final RFIDManager manager = new RFIDManager();
      final PhobListener assocMap = new AssociationMap(new MarbleAnswering());
      manager.addPhobListenerForAllReaders(assocMap);
   }

   public AssociationElt createAssociationEltForPhob(final Phob phob) {
      return new AudioClip();
   }
}

First, creating the manager has no difference. Using the given AssociationMap as the PhobListener requires passing in an AssociationFactory. Here the desired behavior of the factory is to return a new AudioClip upon each new phob, hence the reason for the single 'return new AudioClip()' line. Lastly, adding the phob listener has no difference.

Classifiers: A way to group tags

ObjectClassifiers implement the isMemberOfClass method, returning whether the particular Phob passed in is a member of this classification. For example, a TypeClassifier checks that the given phob is an instance of a particular class type. VisionPhobs may get filtered out by calling the isMemberOfClass method of a TypeClassifier initialized with VisionPhob.class.

ClassifierFactoryMap

The ClassifierFactoryMap is an AssociationFactory that keeps track of all added ObjectClassifiers and their counterpart AssociationFactories. Add different ObjectClassifier-AssociationFactory pairs by calling the addClassification method. The ClassifierFactoryMap's own createAssociationEltForPhob creation method iterates through all registered ObjectClassifiers until one gets accepted; the creation method of the accepted classifier's associated factory gets called on the same phob and returned.

Here's the Marble Answering Machine that allows the user to choose what type of media element they wish to associate.

import edu.berkeley.guir.papier_mache.Phob;
import edu.berkeley.guir.papier_mache.assoc.*;
import edu.berkeley.guir.papier_mache.assoc.ObjectClassifier.AcceptAllClassifier;
import edu.berkeley.guir.papier_mache.event.PhobListener;
import edu.berkeley.guir.papier_mache.rfid.RFIDManager;

public class MarbleAnswering extends DefaultAssociationFactory {
private static boolean firstTag = true;

public static void main(String args[]) {

	 AssociationMap.registerMediaElement(AudioClip.class);
	 AssociationMap.registerMediaElement(VideoClip.class);
	 AssociationMap.registerMediaElement(WebPage.class);


   final RFIDManager manager = new RFIDManager();
   
   final ClassifierFactoryMap classMap = new ClassifierFactoryMap();
   classMap.addClassification(new AcceptAllClassifier(), new UserAssociationFactory(classMap));
   final PhobListener assocMap = new AssociationMap(classMap);
   
   manager.addPhobListenerForAllReaders(assocMap);
}

public AssociationElt createAssociationEltForPhob(final Phob phob) {
   if (firstTag) {
      firstTag = false;
      return new PauseElt();
   }
      return new AudioClip();
   }
}
  

Events and Phobs

A Phob is the virtual representation of the physical object. One key difference between a phob and the actual physical object is that the phob knows the source that generated it. Phobs events know both the source that generated the phob and the phob itself. See the Class Reference for more details on specific phobs and phob generators.

Class Reference:

A. Classifiers (abstract class)

Given constants :
double TRUE, FALSE - used to compare

Methods to implement:
double isMemberOfClass(Phob phob) - returns TRUE (double constant) if phob is a member of this ObjectClassifer, else returns FALSE
String toShortString() - returns a short String description of an instance of this ObjectClassifier that can be displayed easily displayed in a table

Instantiable subclasses:
AcceptAllClassifier - accepts all Phobs (isMemberOfClass always returns TRUE)
IdentityClassifier - accepts only the Phob it was initialized with
MeanColorClassifier - accepts all VisionPhobs that have around the same color
ROIClassifier - accepts all VisionPhobs that have the same region of image
SameSourceClassifier - accepts only Phobs that were created from the same phob generator
SizeClassifier - accepts all VisionPhobs that have the same size
TypeClassifier - accepts only Phobs of the same class type it was initialized with

B. Factories (interface)

Methods to implement:
AssociationElt createAssociationEltForPhob(Phob phob) - returns a new instance of some AssociationElt

Instantiable subclasses:
ClassInstanceFactory - creates new instances of only the passed in class type
ClassifierFactoryMap - creates AssociationElts with the first registered accepted classifier's factory
VisualAnalogueFactory - creates AssociationWrappers used to build a visual representation of the phobs registered by the vision system
UserAssociationFactory - uses a ClassifierFactoryMap (passed in the constructor) to keep track of classifier-factory associations created through the GUI

C. Association Elements (abstract class)

C.i. Nouns (abstract class)

Methods to implement:
void phobAdded() - action that occurs when this AssociationNoun's phob gets added
void phobUpdated() - action that occurs when this AssociationNoun's phob gets updated
void phobRemoved() - action that occurs when this AssociationNoun's phob gets removed

Instantiable subclasses:
WebPage - on creation, a dialog prompts for a URL and a window pops up with that page; on phobAdded a window pops up with that page
(MediaClip) AudioClip - two constructors allow creation either by getting a file from the file system or starting a recording; on phobAdded, the clip gets played and on phobRemoved, the clip gets stopped
(MediaClip) VideoClip - on creation a file dialog prompts for a video file; on phobAdded, the clip gets played and on phobRemoved, the clip gets stopped

C.ii. Actions (abstract class)

Methods to implement:
void phobAdded(AssociationNoun focus) - action that occurs when this AssociationAction's phob gets added with the given focus
void phobRemoved() - action that occurs when this AssociationAction's phob gets removed

Instantiable subclasses:
FastForwardElt - on phobAdded, the focus's fastforward method gets called; on phobRemoved, the focus's resume method gets called
PauseElt - on phobAdded, the focus's pause method gets called; on phobRemoved, the focus's resume method gets called
ResetElt - on phobAdded, the focus's resetToBeginning method gets called
RewindElt - on phobAdded, the focus's rewind method gets called; on phobRemoved, the focus's resume method gets called

D. Phob Events

Given Methods:
Object getSource() - returns the PhobGenerator that generated the phob (note, the Phob also knows its source)
Phob getPhob() - returns the Phob associated with this event

Instantiable subclasses:
VisionPhob and TagPhob (for RFID and barcodes)

E. Phobs (abstract class)

Given Methods:
PhobGenerator getSource() - all Phobs know about the source from which they were generated

Instantiable subclasses:
VisionPhob and TagPhob (for RFID and barcodes)

E.i. VisionPhob

Ascertainable fields:
getBounds() returns the bounds (Rectangle)
getCenter() returns the center (Point2D)
getMajorAxisLength() returns major axis length (double)
getMajorAxisLength() returns minor axis length (double)
getMeanColorRGB() returns mean color (Point2D)
getTheta() returns theta (double)

E.ii. TagPhob

Given Methods:
getTagID() returns tag ID (String)

F. Phob Generators (abstract class)

Given Methods:
void addPhobListener(PhobListener listener) - adds the given listener to this generator
void removePhobListener(PhobListener listener) - removes the given listener to this generator

Instantiable subclasses:
VisionPhobGenerator, BarcodePhobGenerator and RFIDReader

F.i. VisionPhobGenerator

Constructor takes in an ImageSource and an optional minimum object size

F.ii. BarcodePhobGenerator

Constructor takes in a VisionPhobGenerator

F.iii. RFIDReader

Constructor takes in a unique reader ID (int)

Given Methods:
getReaderID() returns int

G. Device Managers (abstract class)

Given Methods:
void addDeviceListener(DeviceListener deviceListener) - adds the given device listener to this generator
void removeDeviceListener(DeviceListener deviceListener) - removes the given device listener to this generator

G.i. RFIDManager

Given Methods:
void addPhobListenerToAllReaders(PhobListener phobListener) - adds the given phob listener to all readers