Zach Schwartz Dot Com

Total Squares

2012-03-02T11:52:52Z

Update: We presented to Intel and ITP, put the prototype online at totalsquares.com, and Fred made the spiffy demo video above.

What?

Squares is a research project which explores the idea of technology having its own desires and learning its own tastes by using data from the Foursquare API and neural networks.

Squares are social organisms that need people to move them around. They develop preferences based on venue data from the foursquare api, and how much 'fun' that square had at the venue. This data is used to train a neural network (venue data as inputs, 'fun' score as output) after each outing.

When a user logs in, the neural net is executed on all nearby venues, which leads to the user's square suggesting places it wants to go. For instance, mine really likes the Key Foods near my apartment.

I'm working with Fred Truman to build the prototype, which is taking the form of a mobile web application for the moment.

Right now, 'fun' is determined in a simplistic way, depending on how much the user interacts with the square. In the future, we'd very much like to add a way for squares to interact with one another, to add another layer of depth.

Why?

Intel is funding a research group at ITP to explore "Vibrant Technology", which, very simply put, is the idea of viewing technology not as a tool to benefit people, but as a peer, with it's own desires. The group is lead by Heather Dewey-Hagborg from ITP and Maria Bezaitis from Intel.

Nerd Stuff

Squares backend is written in python using django
For the neural nets we're using FANN which is great, though I wouldn't reccomend the python bindings. It's pretty hacky. Or maybe it's fine, and all my trouble is due to not being a Swig expert.
Lots of javascript, css, and possibly sammyjs.
And of course, the Foursquare API.

~~We'll have a public demo up soon~~ Demo at totalsquares.com. Source on github.

De/bug

2012-03-02T11:36:20Z

I haven't tried making visual art since I was a kid. I think I was discouraged when I wasn't as good as my Mom.

But CSS is crazy powerful now. The above image wasn't intentional, but I wonder how long it'll be before someone creates an entire animated film, using CSS.

Nerdery for it's own sake? Maybe, but there's a lot of cool stuff one could do with a visual narrative made of interactive DOM elements. And between keyframes and 3d transforms it would look amazing.

New Site

2012-02-18T15:05:54Z

I never write anymore. This is bad, because writing is important, and I used to be fairly not terrible at it. I've built this site with the goal of blogging a lot more, or really, blogging at all. Which counts as writing. Maybe.

Nerdstuff

I've made the writing process easier and simpler. I'm writing this in ~~TextMate~~ ~~Sublime Text 2~~ Vim, using markdown.

The site is static, it's just html, no backend, no silly admin interface, no databases. I forked the python static site generator, pelican, changed a few things and added image support, which I haven't actually added to this site yet. And it's all hosted on Amazon's S3.

The Written World

2012-02-18T13:29:32Z

For Clay Shirky's class, we built an experiment, based on Andrew Badr's yourworldoftext.com.

We placed a free-form text canvas on top of Google maps and added geolocation to see what happens when we combined online chat with the pre-existing relationships that are inherent to physical location and real-world places.

Initially, we only allowed New York. We've removed that restriction, but haven't really been promoting it, so the rest of the world is pretty blank.

Check it out: WrittenWorld.org

The Future

I'm rewriting this project from scratch for my ITP masters thesis. I'll be adding features like zooming, location search, subworlds, and realtime cursors.

The original project was written in python using django, and a whole lot of javascript. My new version will be all javascript, as I'll be sampling this nodejs koolaid. It also won't use google maps, because I'm worried about their quotas, and being closed source, it's difficult to debug.

Spherification

2011-05-12T17:51:00Z

For my Design Frontiers in Biology and Materiality class I experimented with spherification of edible liquids (and one pureed delicious solid). Given that you're on the internet, and can't taste my spheres, the next best thing is to check out the photos I took.

Spherification uses a reaction between sodium alginate and calcium to create a thin skin around the sphere. The liquid you want to create spheres out of is mixed with the alginate, and then dropped into a bath of water and calcium (I used calcium chloride). Reverse spherification uses those same chemicals, but, you guessed it, in reverse. This is necessary for liquids that contain calcium already, as they would thicken too much if mixed with the alginate.

I bought all my chemicals from lepicerie.com. If you're going to try this, definitely be sure to purchase food grade chemicals.

Liquids I made into spheres: Water, red tea, apricot nectar, apple cider vinegar, wine (pinot noire), Sriracha hot sauce, and pureed bacon. I reverse-sphered: coffee , balsamic vinaigrette, and milk.

I used a ratio of about 1 gram of alginate per 200ml of liquid, which in retrospect was a little much. Wine, for instance, thickened a bit more than I'd have liked, so while the spheres were tasty, biting into them was a bit more like eating an alcoholic gummy bear than a caviar. The tea and apricot didn't thicken as much, and had the desired effect. Bacon failed miserably and looked disgusting. Sriracha was so thick to being with that I diluted it some, but it was still to thick to form spheres in the air, dropped out of a syringe into the calcium bath: hence Sriracha spaghetti.

After dropping them in the bath, I waited about one minute, before removing them, and rinsing them in a water bath. This last step is important, because calcium chloride does not taste good. In the future, I may experiment to take them out sooner in some cases, to get a thinner shell

For the reverse-spherification I used calcium lactate gluconate instead of the calcium chloride, with the same ratio. It worked very poorly, as you can see here. The problem with the reverse is that, instead of a thin shell, you get a very thick, clear shell. This could be avoided with an decreased alginate in the bath, but that would probably necessitate a thickener in the spheres, such as xanthan gum.

A debate seems to exist about whether or not it's necessary to blend the alginate with your liquids, or just gently mix it to avoid air bubbles, as well as how long to wait do degas your liquids. I come down firmly on the side of an immersion blender, and learned that waiting to degas isn't that important, at least in my experience. Spheres with air trapped in them looked more visually interesting than the ones I degassed by leaving them in the refrigerator overnight, and had no disernable difference in texture.

Budget Climb

2011-05-11T11:32:00Z

Budget climb is a project I worked on for 3D Sensing and Visualization class, that we've also entered in the datavischallenge. I worked with Fred Truman and Frankie Cheung to build a cityscape-style representation of the US federal budget from the last 26 years, broken up by type of spending.

The cool thing about our project isn't the graphics, which are fairly basic, but instead, the interaction. In order to explore the data environment, we have to physically use our arms to climb, which gives us a real sense of the scale of each type of spending. Climbing military spending, especially for the Bush years, for instance, takes a lot of physical effort.

Source code, and binary (runnable) applications (both kinect and non-kinect flavors) available at budgetclimb.com.

Update: We made it to the finals in the DataVisChallenge and won $500. Which will almost certainly be spent on Bon Chon Chicken.

Pong Head

2011-05-01T11:58:00Z

For Dano's Computational Cameras class (using skills from 3DSAV class), I built a version of the game Pong, where the users control the paddles with their bodies, and the loser's head becomes the ball.

I used a Kinect, Openframeworks, and OpenNi to track the users. The code for the game itself is super simple, with no win conditions, and the ball speed starts fairly slow (though it gets pretty dang fast). Source code is available here.

Neural Net Visualization

2011-05-01T09:23:00Z

For my machine learning class I created a neural network and a visualization to show how it learns.

What is a neural network? Basically, it's a computer model inspired by the way your brain works. The simplest version only has inputs and outputs. The interesting part is the synapses between the inputs and outputs, which start with small random weights (think multipliers), and get adjusted in small increments over many "epochs" in order to cause the network's output to approach the outputs desired.

Where it gets really interesting is when you start adding more layers, besides inputs and outputs, called hidden layers. This allows the neural net to, in a sense, hold abstractions, and generalize from the training data.

For this project, I took data from the last 70 years, and built a network with four layers, six inputs, four neurons in each of the two hidden layers, and one output. (Specifically, I built a feedforward back-propagating supervised network, which is what I'm discussing here. There are many other types of neural nets.)

As inputs, I used the number of Democrats and Republicans in the Senate and House, the active strength of the US Military, and the unemployment rate for the previous year. As output, I used the unemployment rate for that year, as that's what I wanted to predict.

The video above shows the neural net learning, by adjusting the synapses between the neurons, which are animated in gray, and then running through the training data, and then the testing data (i.e. data that it wasn't trained on, in this case, the most recent 15 years).

In the end, my neural network was much more accurate than I expected, the error on the testing data was 11%, meaning that, in theory, if you give the network inputs for a given year (House and Senate makeup, military, last year's unemp) it will predict the unemployment rate within 11% . So if the true rate was, say, 8.3%, the network would give you something between 9.21% and 7.387%. Not too bad. (Frankel and other economist friends, feel free to chime in about the various ways this is wrong and makes no sense.)

More details:
To do this, I used the excellent Encog library for java. I like python more than java, so messed with pybrain a lot, but Encog has every feature conceivable. And works with processing, which made visualizing easier.

Before I got to what you see in the visualization, I tried a lot of different, similar neural nets. First, I tried it without the temporal data, that is, without the last year's unemployment. This gave me a 17% error rate on the last 15 year data. Next, I tried it with a random selection of 15 years, and the error rate went down slightly. Finally, I added the extra input of last year's unemployment, and, with a ton of training (much more than in the video), achieved an 11% error rate on the test data. I also messed with a great amount of variation as far as number of neurons in the hidden layers, from 2 to 500, but 4 turned out to be best.

There were two areas in which I had trouble, one was data normalization using Encog, so I ended up normalizing the data myself using a python script I was using to aggregate the data anyway (neural nets like data to be between 0 and 1).

Second, was displaying the actual values moving through the network while it was being tested. I rewrote most of my code in attempt to add this (and learned how to use the iterable object correctly), as I think it would be really useful for understanding the way the network works, but I couldn't get the values for each layer to display correctly. With some time, I'd really like to add that.

Source code available here, though this doesn't really resemble what's in the video. Below is the console output for the most successful run of the network. Because the weights start randomly, each training is different from every other training.

Epoch #5447 Error:1.4999955997114394E-4 Neural Network Results for Training Data: 0.66,0.28,0.534,0.324,0.18,0.73, actual=0.4983663395292939,ideal=0.495 0.66,0.28,0.534,0.324,0.3859,0.495, actual=0.23436250492049088,ideal=0.235 0.57,0.38,0.444,0.418,0.9045,0.235, actual=0.08452120095631022,ideal=0.095 0.57,0.38,0.444,0.418,1.1451,0.095, actual=0.0815033122889647,ideal=0.06 0.57,0.38,0.486,0.38,1.2056,0.06, actual=0.08130640799774146,ideal=0.095 0.57,0.38,0.486,0.38,0.3025,0.095, actual=0.24327423747050617,ideal=0.195 0.45,0.51,0.376,0.492,0.1582,0.195, actual=0.20086019727739185,ideal=0.195 0.45,0.51,0.376,0.492,0.1445,0.195, actual=0.2008001851266592,ideal=0.19 0.54,0.42,0.526,0.342,0.1613,0.19, actual=0.27778673711288804,ideal=0.295 0.54,0.42,0.526,0.342,0.1459,0.295, actual=0.28814359093564434,ideal=0.265 0.48,0.47,0.468,0.398,0.325,0.265, actual=0.16153750109068066,ideal=0.165 0.48,0.47,0.468,0.398,0.3635,0.165, actual=0.13558489288725128,ideal=0.15 0.46,0.48,0.426,0.442,0.3555,0.15, actual=0.17602786013114474,ideal=0.145 0.46,0.48,0.426,0.442,0.3303,0.145, actual=0.2199931108910821,ideal=0.275 0.48,0.47,0.464,0.406,0.2935,0.275, actual=0.23021456016162994,ideal=0.22 0.48,0.47,0.464,0.406,0.2807,0.22, actual=0.253903192793502,ideal=0.205 0.49,0.47,0.468,0.402,0.2795,0.205, actual=0.2555298780817681,ideal=0.215 0.49,0.47,0.468,0.402,0.2599,0.215, actual=0.26123536179631845,ideal=0.34 0.64,0.34,0.566,0.306,0.2504,0.34, actual=0.27390713255535215,ideal=0.275 0.64,0.34,0.566,0.306,0.2476,0.275, actual=0.29709252965041594,ideal=0.275 0.64,0.36,0.524,0.35,0.2483,0.275, actual=0.2855653767340758,ideal=0.335 0.64,0.36,0.524,0.35,0.2808,0.335, actual=0.24978578065111492,ideal=0.275 0.67,0.33,0.516,0.352,0.27,0.275, actual=0.27536312822913894,ideal=0.285 0.67,0.33,0.516,0.352,0.2687,0.285, actual=0.2765217655180545,ideal=0.26 0.68,0.32,0.59,0.28,0.2656,0.26, actual=0.2554208345203682,ideal=0.225 0.68,0.32,0.59,0.28,0.3093,0.225, actual=0.1939832730643538,ideal=0.19 0.64,0.36,0.496,0.374,0.3375,0.19, actual=0.19772126359928496,ideal=0.19 0.64,0.36,0.496,0.374,0.3547,0.19, actual=0.16332311108949626,ideal=0.18 0.58,0.42,0.486,0.384,0.346,0.18, actual=0.16442127838161877,ideal=0.175 0.58,0.42,0.486,0.384,0.3066,0.175, actual=0.24245546463834086,ideal=0.245 0.54,0.44,0.51,0.36,0.2714,0.245, actual=0.2558851329275245,ideal=0.295 0.54,0.44,0.51,0.36,0.2324,0.295, actual=0.2954853700500669,ideal=0.28 0.56,0.42,0.484,0.384,0.2253,0.28, actual=0.2681779921075669,ideal=0.245 0.56,0.42,0.484,0.384,0.2163,0.245, actual=0.26151565006675204,ideal=0.28 0.61,0.37,0.582,0.288,0.2129,0.28, actual=0.3470994484698628,ideal=0.425 0.61,0.37,0.582,0.288,0.2081,0.425, actual=0.37127167300622715,ideal=0.385 0.61,0.38,0.584,0.286,0.2075,0.385, actual=0.3441559331912948,ideal=0.355 0.61,0.38,0.584,0.286,0.2062,0.355, actual=0.34953890305512214,ideal=0.305 0.58,0.41,0.554,0.316,0.2031,0.305, actual=0.33600392793467077,ideal=0.29 0.58,0.41,0.554,0.316,0.2063,0.29, actual=0.3332303554084988,ideal=0.355 0.46,0.53,0.484,0.384,0.2101,0.355, actual=0.38391068153643754,ideal=0.38 0.46,0.53,0.484,0.384,0.213,0.38, actual=0.4706693478188674,ideal=0.485 0.46,0.54,0.538,0.332,0.2163,0.485, actual=0.4305970376393928,ideal=0.48 0.46,0.54,0.538,0.332,0.2184,0.48, actual=0.42942131121472155,ideal=0.375 0.47,0.53,0.506,0.364,0.2207,0.375, actual=0.3791308632978942,ideal=0.36 0.47,0.53,0.506,0.364,0.2233,0.36, actual=0.3408292536305523,ideal=0.35 0.55,0.45,0.516,0.354,0.2244,0.35, actual=0.3180255945111218,ideal=0.31 0.55,0.45,0.516,0.354,0.2209,0.31, actual=0.3054461292681537,ideal=0.275 0.55,0.45,0.52,0.35,0.2203,0.275, actual=0.30308948074905867,ideal=0.265 0.55,0.45,0.52,0.35,0.2144,0.265, actual=0.3008652986844994,ideal=0.28 0.56,0.44,0.534,0.334,0.2077,0.28, actual=0.3167321964928064,ideal=0.34 0.56,0.44,0.534,0.334,0.188,0.34, actual=0.33105436361549073,ideal=0.375 0.57,0.43,0.516,0.352,0.1775,0.375, actual=0.35928649461540163,ideal=0.345 0.57,0.43,0.516,0.352,0.1678,0.345, actual=0.3048710453119955,ideal=0.305 0.48,0.52,0.408,0.46,0.1583,0.305, actual=0.2624632144627692,ideal=0.28 ------------------------------- ------------------------------- Neural Network Results for NEW DATA: 0.48,0.52,0.408,0.46,0.1538,0.28, actual=0.23930077726065127, ideal=0.27, percentage error=0.11370082496055091 0.45,0.55,0.414,0.452,0.1504,0.27, actual=0.24239373755821966, ideal=0.245, percentage error=0.010637805884817685 0.45,0.55,0.414,0.452,0.147,0.245, actual=0.22977833679284043, ideal=0.225, percentage error=0.021237052412624118 0.45,0.55,0.422,0.446,0.1451,0.225, actual=0.22768245012752356, ideal=0.21, percentage error=0.08420214346439793 0.45,0.55,0.422,0.446,0.1449,0.21, actual=0.2253461254045889, ideal=0.2, percentage error=0.12673062702294444 0.5,0.5,0.424,0.442,0.1451,0.2, actual=0.21840667496538055, ideal=0.235, percentage error=0.07060989376433804 0.5,0.5,0.424,0.442,0.1478,0.235, actual=0.22297073742023873, ideal=0.29, percentage error=0.23113538820607327 0.48,0.51,0.41,0.458,0.15,0.29, actual=0.24921051256244264, ideal=0.3, percentage error=0.16929829145852449 0.48,0.51,0.41,0.458,0.1494,0.3, actual=0.25960230146033725, ideal=0.275, percentage error=0.055991631053319176 0.44,0.55,0.404,0.462,0.1455,0.275, actual=0.24845814093718874, ideal=0.255, percentage error=0.025654349265926538 0.44,0.55,0.404,0.462,0.1456,0.255, actual=0.23292273464134336, ideal=0.23, percentage error=0.012707541918884128 0.49,0.49,0.472,0.398,0.1451,0.23, actual=0.24803874970006634, ideal=0.23, percentage error=0.07842934652202753 0.49,0.49,0.472,0.398,0.1474,0.23, actual=0.24857095999001486, ideal=0.29, percentage error=0.14285875865512113 0.58,0.4,0.514,0.356,0.1493,0.29, actual=0.26828403415745933, ideal=0.465, percentage error=0.4230450878334208 0.58,0.4,0.514,0.356,0.1506,0.465, actual=0.5537600529394734, ideal=0.48, percentage error=0.15366677695723627 Average Error for Test data 0.11466037 Hey there. End of program.

Spider Mother

2011-03-27T12:44:00Z

Our (video sculpture) take on Louise Bourgeois's Maman.

Video Self Portrait

2011-03-22T10:49:00Z

Here's my video self portrait for Video Sculpture class. It was the first time I'd edited video since high school. Turns out not all that much has changed as far as Premiere and FCP go.

Simple Kinect Test #3

2011-03-08T15:45:00Z

Spent a lot of time trying to get opengl shaders to work in OF. Failed. So I rendered the kinect mesh with triangles instead of quadrangles.

Attempt At Spherification

2011-03-07T16:35:00Z

For Design Frontiers class, I tried sperification of foods. I used Sodium Alginate, Calcium Chloride and Calcium Lactate Gluconate, and tried both sperification and reverse-sperification, with Root, a liquour, water, tomato juice, milk, and olive oil. I was really only successfull with the water and tomato juice, probably because I failed to use purified water for my water-bath. I'd like to try this again in a more controlled environment. More photos after the jump.

Simple Kinect Test #2

2011-03-01T11:39:00Z

Simply taking the closest point, using it to draw shapes, and moving them around. The black mask at the start is actually an artifact from youtube, oddly.

Ngram Nietzsche

2011-02-25T11:24:00Z

For Bit by Bit, I parsed Friedrich Nietzsche's Beyond Good And Evil as 3-grams, basically using Heather's code to put it back together. The text ends up about as, if not more, readable, as the original. An excerpt of the generated text follows below.

"Sure of our own virtues ? But then our organs themselves would be the cause of egoism as conducive to the highest and most successful type of man , the gregarious European man nowadays assumes an air as if by virtue of which he is not to speak of the future of humanity whom one must not resign oneself to one MAY NOT at all times can be at home"

" In short , the will " : -- that is certainly not the least of the world in its best springtime -- which , as is described by poets , intermediaries and blenders of the spirit , " in the case of Descartes , the " good " and almost incomprehensible MORAL world fleeing towards it , the will to overcome an emotion , for which the nationality - craze has induced and still induces among the neighbouring peoples , states , churches ) , he is BECOMING , he has been reserved for us nowadays ? 213 . It is the most delicate sensibility for the most spiritual festival -- laughter and arrogance , for the sake of his soul in misfortune which communicates to it . The noble type of man separates from himself for something from which tragedy itself no peace , until finally two primary types revealed themselves to be overcome on the contrary ! On the whole , speaking generally , and consequently also do not believe that severity , obtrusion of peculiar forms , incorporation , and a cash - box , modest also in politicis ! -- But why should one deal with such desires only as knowing or " humanising , " and " sin against Romanticism , besides being probably made from some rule , " and " sin against Romanticism , besides being probably made from some rule , " for the sake of his soul , he would like to have created it ; eventually , however , are now overlooked , or even with REGARD to us , inquisitive to a desk and a cash - box , modest also in politicis ! -- COUNTER TO this propensity for appearance , and that consequently unbelief in Catholic countries means something quite different from that which has always been -- transfigured . That which is called " convictions . " -- that is to say , in order to be a gold - dust of unconscious human vanity , women themselves ( or as I call it " Wagnerite " ; and in general prevail and make them furious , I found certain traits recurring regularly together , one might say arbitrary will , and also literally ) : he is now prescribed by the awe of authority from obtrusive touches and incivilities : something that is to say , " in every sense are required , as is appropriate in southern nations , eras , and a cash - box , modest also in politicis ! -- But why

Simple Kinect Test

2011-02-24T01:28:00Z

Getting going with the kinect and openframeworks. Worked reasonably well. Basically, I'm interacting with an imaginary red cube, that turns green when I touch it in 3d space. I plan to make this more, er, complicated.

if ((cur.x > redbox.x)&&(cur.x<redbox.x+redboxsize)){ if ((cur.y > redbox.y)&&(cur.y<redbox.y+redboxsize)){ if ((cur.z > redbox.z)&&(cur.z<redbox.z+redboxsize)) {redboxcolor=1;}}}

Is the important part of the code (done with in the point could loop).

YayTM Documented

2011-01-24T11:11:00Z

Over the weekend my YayTM got posted to Hackaday. Very cool except I'd failed to post documentation or source code, and the comments on this site were broken. I was notably called a "goober" and an "immature sociopath", among other things. Getting insulted by the Internet is cool, a rite of passage even, and nothing to be taken personally or seriously, but some of the comments were totally valid. I should have documented this thing. So here we go, after the nonjump.

The image above is of some sketches I did trying to figure out how the dance detection would work, and how to build the acrylic box. Source code of the actual processing sketch is available here, but it's totally and completely awful code that I can't recommend reading in good conscience, but be my guest. I should have written or found some sort of scripting/events system. Next time.

The bill dispenser was ten dollars used on ebay and was manufactured by ICT. I burned out the circuit board pretty quickly while messing around with it, but the motor continued to work, so with a little cajoling and the installation of my own LED and sensor, I was able to dispense dollar bills using an Arduino and Processing.

Below is a video of me using the YayTM. It was right at the end of the show (you can hear people cheering) so the bill dispenser is a little wonky and at the wrong angle, so dispensed two dollars instead of one.

Any other questions?

Track M+E

2011-01-18T13:51:00Z

For my Design for UNICEF class, I worked with Zevensuy Rodriguez and Margaret McKenna to plan a supply logistics tracking system for supplies in emergency situations. It consists of two components, a web application for the UNICEF community to monitor supply locations and progress, as well as a mobile application for use in the field. In short, it's similar to the tracking systems FedEx and UPS use, but simplified and much more flexible.

Last month, we presented Track M+E to staff at UNICEF HQ in New York, which was featured in this article. Video of the presentation will be available soon, and the slides for our presentation are after the jump.

The YayTM

2010-12-20T16:26:00Z

YayTM is a physical piece which calls attention to itself at predetermined times, and asks passersby to dance for it, in exchange for a dollar. Using face tracking to measure dancing prowess, the YayTM determines if the user is dancing to its standards and dispenses a dollar if it likes what it sees. Then, regardless of the success or failure of the dancer, and without their previous knowledge, the YayTM uploads video of the dancer to YouTube and displays that video here, for you to make fun of.

Check out the videos.
Or watch a first person view:

Lobbying By Industry Data Visualization

2010-11-23T13:51:00Z

I made this data visualization of lobbying data for my ICM class. It was due yesterday at 9:30am, and I turned in a much worse version of this, because I'd wasted a bunch of time messing with a deprecated Sunlight Foundation json method. In the end, I just downloaded 200MB of lobbying data from OpenSecrets.org and used the awesome, new Google Refine to clean up the data before writing my Processing sketch.

Full size version here.

Edit: Sam points out that I had cumulative totals, not the actual industry totals. Fixed

ICM Midterm

2010-11-17T20:28:00Z

My Intro to Computational Media midterm project used face detection and tracking to manipulate an, admittedly very simple, 3D scene rendered in processing. I used the OpenCV library, and moved the point of view for the scene based on the movement of the subject's face.

It fails if the subject's face isn't fully visible, or if the face is too in profile, or if the face is a black guy wearing glasses. It could also use a lot more smoothing, as what I'm using is fairly rudimentary. And a Microsoft Kinect probably wouldn't hurt matters.

I Stop Heart

2010-10-18T11:35:00Z

Color Lock Box

2010-10-10T20:22:00Z

The assignment for physical computing class was to create a 'stupid pet trick'. Basically, a device that one interacts with physically, that does something.

My original idea was to make a color based combination lock, but I ran into a lot of unrelated trouble and ended up simplifying the project into a sort of 'guess the color' game. If the user leaves the led/ping pong ball on the right color for two seconds, the lockbox activates and either opens or closes.

Problems I ran into: 1) I really had trouble figuring out an elegant way to map the potentiometer reading to the RGB values for the LED. It seems like there should be a smart way to do this, but I ended up using several if-statements and mapping within them, which felt...dirty and wrong. I need to study some more math, clearly.

Second, and way less my fault, the servo.h library for the Arduino messed up my LED's colors. It still worked, but the colors weren't pretty, which was a really big part of this project for me, so I ended up writing handrolling the servo controls, which worked fine. But still, I wonder why setting the a pin for the servo using the library screwed up my LED, even before I attached the motor.