CouchDB vs. MongoDB Benchmark

Edit (9/1/10): this benchmark is old, silly, and should probably be ignored in favor of more recent and representative ones. I don’t want to take it down for historical purposes, but seriously people, it was never a good benchmark, it’s over a year old at this point, and both databases have changed a lot.

Edit (12/6/09): this is the #1 Google result for “mongodb benchmark”, so I figure I’ll do some community service: if you’re interested in benchmarks, you might want to look at the 3rd party ones listed on the mongodb.org website.


Felix Geisendörfer did a benchmark in PHP that was super-easy for me to port into MongoDB. You can see his post on his blog.

And now… comparing his results for CouchDB with mine for MongoDB’s (I did the graph in Open Office, which is why the quality sucks):

As you can see, MongoDB does, uh, slightly better.  Here are the numbers:

# of Inserts Couch Total Time (sec) Couch Time/Doc (ms) Mongo Total Time (sec) Mongo Time/Doc (ms)
1 .0015 1.46 .0005 .5
2 .0015 .75 .0004 .2096
3 .0017 .56 .0005 .1604
4 .0017 .44 .0005 .1190
5 .0018 .36 .0005 .1060
6 .0019 .32 .0006 .0931
7 .0021 .3 .0006 .0847
8 .0022 .27 .0007 .0789
9 .0023 .25 .0007 .0734
10 .0025 .25 .0007 .0721
50 .0072 .14 .0024 .0476
100 .0136 .14 .0044 .0442
500 .0687 .14 .0253 .0505
1000 .1361 .14 .0372 .0372
2500 .4686 .19 .0278 .0111
5000 .9165 .18 .0488 .0098
7500 1.5116 .2 .0835 .0111
10000 2.3111 .23 .1065 .0107
25000 6.8684 .27 .2711 .0108
50000 15.8227 .32 .5430 .0109
100000 35.3071 .35 .1.7697 .0177
250000 104.0009 .42 6.4533 .0258
500000 230.6021 .46 11.7684 .0235
750000 352.7959 .47 17.0473 .0227
1000000 487.3284 .49 18.4376 .0184

Please let me know if I made any mistakes, all the values were hand-copied.

I ran these tests using the PHP driver on Ubuntu 9.04 on my MacBook Pro.  You can see the test script I forked on Github.

A little analysis: Both DBs start with some overhead, but by 1000 inserts CouchDB seems to be chugging along nicely.  MongoDB takes slightly longer to hit its groove, hitting its peak around 10000.  They both slow a little near the end, as MongoDB starts spending most of its time allocating files and, although I know almost nothing about CouchDB’s structure, I’d guess it’s doing something similar.

Because we are the Mutha Flippin Win

I’ve been keeping records of funny and nice things people have said about MongoDB. A lot of them can be found at http://www.mongodb.org/display/DOCS/User+Feedback, but there are some good ones that weren’t quite… “right” for the official page. So, slightly off-color, biased, or weird quotes ahoy:

Mailing list:

“Guys at Redmond should get a long course from you about what is the software development and support :-)”
-kunthar

Twitter:

@mully It was a well orchestrated effort for @mongodb world domination. I was the shock, @jnunemaker was the awe.
-pengwynn

the @mongodb guys are great! They figure out segfaults, fix bugs and push new builds in practically real time!
-brondsem

Mongo looks like the Mutha Flippin Win. I need to carve some time out to play. http://www.mongodb.org/display/DOCS/Home
-thejohnny

IRC:

“limit=-10 is my hero”
-claymation

Blogs

“It actually has a kind of quirky, yet lovable syntax for defining criteria.”
-http://railstips.org/2009/6/3/what-if-a-key-value-store-mated-with-a-relational-database-system

Revisionist history

I’ve switched out the first ~10 cartoons, and replaced them with one-shot gags. So there are a bunch of new comics to read, starting with the first one.

I’ve always preferred doing The Far Side-/New Yorker-style cartoons, but when I started this site, I thought I should do an ongoing story with characters and plot an all. It turns out that I like killing my characters too much for that to really work out. I hope you enjoy their brief existences.

PHP Extension Wiki

I started a wiki on this site (http://www.kchodorow.com/php) to write down all the stuff I learn about writiing PHP extensions. If anyone else has experience with them, feel free to add or edit articles.

Some basics: a PHP extension is written in C. In fact, PHP itself is written in C, so there’s a lot of good source code to look at out there. There’s an excellent introduction to writing PHP extensions at Zend DevZone. However, it doesn’t go into a lot of the specifics, which is why I started the wiki. I had to figure out how to do a ton of stuff on my own, mostly by digging through the PHP source code and other extensions’ source code. No one should have to look through 500 undocumented C files to figure out how to create a PHP class in C. (However, if you like digging through source code, it’s all available to view on the web. Extensions are under pecl and PHP source is available under php-src.)

I feel like I have a pretty good handle on how to do almost anything with PHP in C, so if anyone has any questions or suggestions for an article, feel free to ask and I’ll try to write a page on it.

Upcoming pages I’d planning on writing:
– Throwing exceptions
– How to extend/implement other classes
– Using HashTable

From Russia with Bugs

My Windows partition got a virus. Ugh. Somehow I’ve managed to avoid this, I’ve never gotten a virus on any of my computers before. Anyway, being the geek I am, I decided to get it off my computer without any of this fancy-shmancy (expensive) anti-virus software. My journey:

1. I tried to edit my registry. I got the message: “Your administrator has turned off this function.” Basically, the virus, which had obviously installed itself with whatever permissions my user has, had made it impossible for me to open the registry editor. I’m cool with the registry only being editable by an admin, but then Windows should freakin prompt you for the admin password. And if the virus installed itself when I was logged in, it shouldn’t be able to do admin things!

2. I tried to delete all the virus’s files using Windows Explorer. The virus had deleted “Folder Options” from the menu, so I couldn’t delete hidden files and folders. Why the fsck would Windows let any program remove functionality willy nilly from Explorer?!

3. They could remove the “Show hidden files and folders” option from the GUI, but I’m a Linux programmer. I can just use the command line. Well, the Windows command line is awful. Or hard to use. Or both. I couldn’t make any headway with it. I couldn’t even make it do the equivalent of “ls -a”.

4. The virus put executables in system32, c:, and other folders that should be only writable by the administrator.

Anyway, that’s why I haven’t put up any new cartoons this week. I’m not sure what to do now. A combination of the virus and Windows itself has totally flummoxed me. I’d like to just get rid of Windows, but I need it for Photoshop.

Get on the bus, Gus

I booted into OS X today and tried running the unit tests for my PHP driver. It chugged away for a while, then gave me “bus error”. Great. Gotta love C error messages. I narrowed it down to when I insert 253 or more elements into MongoDB. Now, that number was suspiciously close to 256, but I couldn’t think why that would be. I tracked it to the encoding code, then to the _id class code, then to the _id generation code. Surprisingly, it was doing its “bus error” thing before any memory access, it was just reading a file. I suddenly realized that I wasn’t closing the file after reading it, fixed it, and it ran perfectly.

Got Mongo Working on Hostmonster!

This was written in April of 2009. It is very out of date. See http://rcrisman.net/article/11/installing-mongodb-on-hostmonster-bluehost-accounts for more up-to-date information (as of August 2010). Keep in mind that shared hosting with Hostmonster is very lame. They only lets you run a program for 5 minutes before killing it, so it’s fairly useless to install MongoDB unless you have a dedicated IP.

I finally got MongoDB working on this site, so I’m going to start switching stuff over from MySQL. I’m biased, but I think it’s just an easier database to use.

And, because I like writing tutorials… How I did it:

  1. Downloaded the binary I created of MongoDB for “legacy” Linux. I originally compiled this for a user on Mandriva 2006 (see previous post about VMWare), but it works fine for other old Linux distros, too.
  2. Run: 
    $ tar zxvf mongodb-linux-i686-old-linux-1.tgz
  3. Make a directory for the database to put files in: 
    $ mkdir /home/user/data
  4. Upload libjava.so, libjvm.so, and libverify.so. Make sure they have execute permissions and put them somewhere like /home/user/lib.
  5. Run: 
    $ export LD_LIBRARY_PATH=/home/user/lib

    replacing the path wherever you put the .so’s above.

  6. Start the database: 
    $ cd mongodb-linux-i686-old-linux-1
$ bin/mongod --dbpath /home/user/data --nojni run

I cheated a bit and didn’t install Java, so I had to use the –nojni option. If you install Java, you won’t need that (and you won’t need to upload the individual .so files).

Now, what good is a database if you can’t use it, right? So, I downloaded my PHP driver (go to its Github repository and click “Download” for the latest version). I then followed the install instructions and put the .so generated by make in /home/user/extensions.

I changed the options under “PHP Config” in Hostmonster’s CPanel to use php.ini in /home/user/public_html/php.ini, and then edited that file to use my extension.

I made a simple test page with:

Which connected me to MongoDB, showing:

localhost:27017

when I loaded the page!

VM of Death and Doom from Hell

On Friday at work, I wrote a short post extolling the virtues of VMWare. It was freakin awesome. I had to debug a problem a user was having on a 3-year-old version of an obscure Linux distro, so Eliot (my boss) suggested using VMWare. It was so cool. I found a site that actually still had the distro available, downloaded it, installed it on a virtual machine, and got the exact same error the user was getting! Awesome.

Then I went home, figuring I’d work on it more when I got there.

When I got home… my wireless internet didn’t work. The kernel module for it wouldn’t even load, and when I ran ifconfig, it printed two new interfaces: vmnet1 and vmnet2. It turns out VMWare does something to some types of kernel modules. Damn you, VMWare!

For some reason, I’ve always had computers with Atheros wireless cards, which are very touchy about things like working. So, I started trying to get it going again. This pretty much describes how it went: http://imgs.xkcd.com/comics/success.png. Around 11pm, I gave up, and decided to leave it until Sunday, when I could look at it with fresh eyes.

Today, I booted into Linux to give it another shot. Or, rather, I attempted to boot into Linux. But, I had somehow messed up the modules in a way that made my system unbootable.

So, now I’m reinstalling Ubuntu.

Here’s hoping I make it to shore before the sharks get me. I’m going to be running VMWare from the OS X side of things from now on.

Another comic

I finally put up another comic. I’ve been delayed for two reasons: first, I’ve been really busy with work. I’ve done some major work on the PHP driver, so it should now be easily installable on Linux and Mac (it’s also faster).

Second, I’ve been wanted to add a “one shot gag” section, which requires some fairly major rewriting of my site code, so I tried to get Git set up on Hostmonster (my hosting provider). I got it set up fine, but the money-grubbing jerks at Hostmonster block outgoing ssh unless you pay for a dedicated IP. So, I can’t put the source of my site on Github, which is sad. However, at least I have Git set up locally, so I don’t have to worry about breaking stuff.

Anyway, hope people enjoy the one-shots. More coming.

Zen and the Art of Sewing Machine Matinence

(Which is a terrible book, by the way.)

I read Schneier on Security, which is an interesting blog about security and squids. (Wait, is the plural of “squid” just “squid”?) See, every Friday he post something squid-related, and last week it was an 8-foot squid pillow. Well, obviously I had to make one for myself.

I went to one of the fabric stores on Orchard Street to get fabric. “How much per yard?” I asked the salesman as he pulled down a bolt.

“Five dollars.”

It had a stupid flowed pattern on it, and I wouldn’t want my 8′ stuffed squid to look ridiculous. “How about that one?” I asked.

“Five dollars.”

“And that one?”

“Five dollars.”

“Okay, I’ll take two yards. How much is a package of stuffing?”

“Five dollars.”

In retrospect, I should have asked for ermine.

I got home, cut the pattern out of newspaper, pinned it to the fabric, and cut out all the pieces. I pinned everything together and I pulled out my sewing machine. I put the first piece in and started up the machine. It went fwipafwipafwipafwipa. All was right with the world. As I started down the second seam, it went fwipafwipafwipaSCREEfwipaSCREEfwipaSCREECH-SCREECH-SCREECH.

I stopped.

I unscrewed the case and looked for parts that could be oiled. There were, as far as I could tell, two moving parts. I oiled them, put everything back together, and tried again. Now, not only did it screech, but the needle didn’t move up and down. I had apparently oiled something that once moved the needle through friction.

So now I have a paperweight. I’m going to have one more look at it and then find a sewing machine repair shop.

It annoys me that I can easily partition my hard disk or install more RAM on my computer, yet a machine built in 1918 has gotten the best of me.