Humanoids Project 1

by Luis Daniel Ballesteros

Goal:
The initial idea behind this project is to make the robot dance. Of course, simply executing a series of moves is boring. I wanted the robot to learn to dance, so I devised a simple probabilistic learning algorithm. The goal of my project, then, is to see results indicative of learning in the robot's dances.

Algorithm:
The algorithm is simple, yet complicated. As units of movement, I copied all of the preset move functions from the sample dances and numbered them move0 through move120. I then wrote about 30 of my own moves, combinations of movements that seemed interesting to add to the pool, for a total of 151 possible moves. Now all the robot needed was a way of invoking the moves that are best fit for the song playing.

To chose moves, I set up a model where good moves had a higher probability of coming up than bad moves. The way I did this was giving each move a score from 0 to 99. To chose a move, I do 100 samples numbered 0 to 99. For each sample, it randomly chooses a move whose score is at least as high as the sample number and gives that move a point. Thus a move with a score of 30 has fewer potential points than one with a score of 80. At the end of 100 samples, the move with the most points is executed.

At the end of the dance the robot takes input from the user to judge how good or bad the dance was. If the dance was good, all moved performed will receive a score bonus. If it was bad, all moves performed will be reduced score. The user can also choose to vote neutrally and not change the scores.

Also note that no move ever has 0 probability since not only do I not allow repeat moves in a dance, but even moves with score 0 are considered in the draw. This is good to revive moves that may have seemed bad in context, but are actually not that bad.

Results:
I was personally surprised with the results of the learning. In the beginning, every move had a score of 50, so they all had an even chance to be chosen. However, after a few runs, it became clear that the moves I liked were happening more frequently and the ones I didn't were not. For example, some of the moves were simply sleep instructions that caused the robot to stop moving for a bit, and some involved moving his wait. I really disliked it when he stopped, and really enjoyed it when he moved his waist. I could see that as I judged and reviewed his performance, the pauses almost went away and the waist moves were in every dance.

I also noticed that the more the robot learned, the harsher I had to be with my judging. After the point where it almost never called the wait move, I had to look for other things that I liked or disliked. It seems clear that the more dances you judge, the better they get overall. But as you will see below, doing many trails was difficult, long, and cumbersome. Generally I like the dance that robosapien does on average, though with a lot more trials I could effectively weed out all of the bad moves.

Although some move's probablilies went up into the 70s, and others into the 30s, since there are so many moves, the majority of them linger in the 40s and 50s. With a lot more trials I believe that thiss method would effectively seperate the bad moves from the good moves.
However, if you watch the videos, you will see that he does a lot of moves that move his waist, as per my taste in his dances. It's pretty exciting.

Obstacles:
Although there were many things the robot could do, the things that it couldn't were severe obstacles. For example, I could not find a way to create files inside the SD card, so I had no automatic way of saving the learning done in one session. I had to use a hack where calls to System.print are redirected into a log file that has debugging information, I would print the array of saved values to the log file and then copy it as a hard coded initial value inside the code, recompile, and copy the file over to the SD card again. This became very frustrating. If there is a way to create and write to files I would much appreciate it. The java mobile libraries don't include the functions necessary to do this that normal java has, so...

Also, I don't know why, nor do i know whose fault it is, but the dance would always fail on the 5th try. this means that from bootup, I could only do 4 dances before I had to turn it off and copy the saved learning and recompile. Worse yet, sometimes the log file would not be created so I would lose the data I had spent time learning. I never could figure out why the robot stopped working on the 5th dance because I had no way to do runtime debugging, so I never knew the state of the robot when it failed. No, I could not simply write the status information to the log file because whenever the robot froze, all I could do was shut the power off and the log file would never be written. I suspect it is a stack overflow or something of that sort, but I have no way to test anything.

Combined with all of these frustrations and obstacles, this project took me more than twice as long as it should have. Better documentation of things like how the log file is created would have been of utmost utility here. Not that I blame anyone for it, we're all still getting used to this new platform.

Experience:
Aside from the extreme frustration of the programming environment, using java, and losing my data frequently, this project was a lot of fun. I enjoyed watching the robot dance and it was cool to see the results of my learning. Overall i am glad that I did this project as I learned a lot of things from it. It was awesome to make all my friends burst out in laughter when they watched the robot dance. It really is quite entertaining. Furthermore, it's exciting to make him dance multiple times because the dance is never the same, so it's interesting to see what will come up based on what he's learned.

Todo:
It would be cool to install another learning algorithm that he not only learned which moves are best, but what order to do them in. Sometimes two moves can look individually bad when done in the wrong order, but in the right order they look good. This would be good to consider, but harder to implement and test. A good idea for version 2 if it ever gets to it.

Links:
Youtube video -
This is a quick run:



In this second one you see me press his right foot at the end of the dance. This is telling him that I liked the dance he did. I figured I should show this since I abruptly stop filming in the previous video.

Java source code - Link
A note on the java - I am primarily a C coder, and the lack of function pointer made my life very difficult here. do not laugh at my switch statement with 150 cases.... If you know how to do variable method calls in java, please leave a comment, I would love to know. I hated having to write a C program to write a giant java switch statement....