P.T. Barnum on salary vs. equity

I’ve been reading P.T. Barnum’s autobiography and came across an interesting passage about when he was employed at Mr. Taylor’s (maybe of Lord & Taylor’s? Not sure) shop:

My employer manifested great interest in me, and treated me with the upmost kindness, but the situation did not suit me. The fact is, there are some persons so constituted that they can never be satisfied with labor for a fixed salary, let it be never so great. I am one of that sort. My disposition is, and ever was, of a speculative character, and I am never content to engage in any business unless it is of such a nature that my profits may be greatly enhanced by an increase of energy, perseverance, attention to business, tact, etc.

He also sounds like quite an ass and a racist, but it’s an interesting read.

Barnum & Bailey show poster

My favorite benefits that more companies should offer

Google is famous for its perks, but the ones that ended up being my favorite were not the free food:

Peer bonuses
Anyone at Google can nominate a coworker for a peer bonus. They write a couple sentences about what the person did and send it to their manager for approval. It’s a really nice way of getting quick recognition for going above and beyond for someone. I feel bad that I didn’t give more of these out, but it was a terrifically nice feeling to get them.
Anonymous mailing list for woman engineers
During my tenure, someone started an anonymous mailing list for woman engineers, where people discussed everything from robotics to travel to birth control. There was also, of course, lots of discussions about how to deal with the unpleasant situations women in tech invariably find themselves in. Having this platform was like having a place you could breathe free.

I am also proud of my contributions: when I was having a rough week a while ago, I started a thread about “tell me about a cool thing you’re doing” (I gave the example of having just regrouted my bathroom tile) and it became one of the most popular threads on the list. During the recent issues with The Document That Shall Not Be Named, someone revived this thread and people decided to make it a monthly topic, so I’m happy to have that as my (unattributable) legacy.

As part of regrouting: I was so proud of getting the caulk out from the bathtub in one long strip that I took a “I caught a fish”-style selfie with it.
Mail room
I hate going to the post office and I always feel like I’m doing it “wrong” (I’ll stand in the wrong line, use the wrong box, something). Google’s mail room had all sorts of envelopes and boxes and, most importantly, would let you drop off any envelope or package and they’d put on the right stamps on for free. They also had a hilarious wall of portraits of “famous postal workers” (Newman of Seinfeld, the guy from Cheers, and so on).


Many years ago, I was speaking at a PHP meetup and got talking to a guy who worked at Microsoft. He told me about Microsoft’s BizSpark program, which lets startups use Microsoft’s software for free. As MongoDB was about a dozen people at the time, I filled out the application and started using PowerPoint for my presentations (which was a vast improvement over Open Office).

One of my friends just asked on Facebook how they could get a cheap license for Microsoft Word, so I looked up the BizSpark program page and saw:

BizSpark homepage

MongoDB went from being one of the startups that BizSpark was aimed at to being one of the features of BizSpark. Pretty cool.

The next great frontier in ML: dogs in hats

I’ve been messing around with Keras, a ML library that makes it pretty easy to do AI experiments. I decided to try out image recognition, so I found a picture of Domino in a fez:

Then I wrote the following Python program which uses the existing ResNet50 image recognition library.

import numpy as np
from keras.preprocessing import image
from keras.applications import resnet50

# Load the image
img = image.load_img('fez.jpg', target_size=(224, 224))
input = image.img_to_array(img)
# Keras can process an array of images, but we're only passing one, so turn it into an array with one element.
input = np.expand_dims(input, axis=0)
# Normalize the RGB values into a 0-1 range, since ML algorithms get thrown off by big ranges.
input = resnet50.preprocess_input(input)

# Predict what's in the photo.
model = resnet50.ResNet50()
predictions = model.predict(input)
predicted_classes = resnet50.decode_predictions(predictions, top=10)

for _, name, liklihood in predicted_classes[0]:
    print("this is an image of {} {}".format(name, liklihood))

In addition to the obvious prereqs, you have to sudo apt-get install libhdf5-dev; pip install pillow certifi h5py.

This prints:

this is an image of Bouvier_des_Flandres 0.4082675576210022
this is an image of briard 0.3710797429084778
this is an image of Newfoundland 0.10781265050172806
this is an image of giant_schnauzer 0.04042242094874382
this is an image of Scotch_terrier 0.038422249257564545
this is an image of komondor 0.012891216203570366
this is an image of Tibetan_terrier 0.0026010528672486544
this is an image of affenpinscher 0.0024157813750207424
this is an image of standard_poodle 0.0021669857669621706
this is an image of Kerry_blue_terrier 0.002110496861860156

It’s pretty solid on “it’s a dog.” I’m disappointed in the lack of fez-related IDs. However, I like some of these as possible breeds for Domino:

Tibetan terrier is pretty close, they are relatives.
Newfies are adorable, but the size is a little off.
I have no idea where Komondor came from. They’re amazing looking, but pretty distinct.

So, still needs some work. But not bad for less than 30 lines of code.

Keeping your deps tidy

My coworker Carmi just published a blog post on the Bazel blog about how Java tracks dependencies. Bazel has some nice facilities built in to let you know when you need to add dependencies:

ERROR: /home/kchodorow/test/a/BUILD:24:1: Building libA.jar (1 source file) failed: Worker process sent response with exit code: 1.
A.java:6: error: [strict] Using type C from an indirect dependency (TOOL_INFO: "//:C"). See command below **
  C getC() {
** Please add the following dependencies:
  //:C  to //:A

He mentioned unused_deps, a great tool to go the other way: what if your BUILD file declares dependencies you’re not using? unused_deps lets you quickly clean up your BUILD files.

To get unused_deps, clone the buildtools repository and build it:

$ git clone git@github.com:bazelbuild/buildtools.git
$ cd buildtools
$ bazel build //unused_deps

Now go to your project and run it:

$ cd ~/my-project
$ ~/gitroot/buildtools/bazel-bin/unused_deps/unused_deps //... > buildozer-cmds.sh

This will print a bunch of info to stderr as it runs but, when it’s done, you should have a list of buildozer commands in buildozer-cmds.sh. For example, running this on the Bazel codebase yields:

buildozer 'remove deps //src/main/java/com/google/devtools/build/lib:auth_and_tls_options' //src/tools/remote_worker/src/main/java/com/google/devtools/build/remote:remote
buildozer 'remove deps //src/main/java/com/google/devtools/build/lib:build-base' //src/tools/remote_worker/src/main/java/com/google/devtools/build/remote:remote
buildozer 'remove deps //src/main/java/com/google/devtools/build/lib:concurrent' //src/tools/remote_worker/src/main/java/com/google/devtools/build/remote:remote
buildozer 'remove deps //src/main/java/com/google/devtools/build/lib:events' //src/tools/remote_worker/src/main/java/com/google/devtools/build/remote:remote

This is a list of shell commands, so now you need to execute this file. The buildozer tool also lives in the buildtools repository, so you just have to build that and then add it to your path:

$ cd ~/gitroot/buildtools
$ bazel build //buildozer
$ cd ~/my-project
$ chmod +x buildozer-cmds.sh
$ PATH=$HOME/gitroot/buildtools/bazel-bin/buildozer:$PATH ./buildozer-cmds.sh

This will run all of the buildozer commands and then you can commit the changes, e.g.,

$ git diff
diff --git a/src/tools/benchmark/javatests/com/google/devtools/build/benchmark/codegenerator/BUILD b/src/tools/benchmark/javatests/com/google/devtools/build/benchmark/codegenerator/BUILD
index 022a4037d..5d5cdf8d0 100644
--- a/src/tools/benchmark/javatests/com/google/devtools/build/benchmark/codegenerator/BUILD
+++ b/src/tools/benchmark/javatests/com/google/devtools/build/benchmark/codegenerator/BUILD
@@ -6,7 +6,6 @@ java_test(
     deps = [
-        "//third_party:junit4",
@@ -17,7 +16,6 @@ java_test(
     deps = [
-        "//third_party:junit4",
@@ -39,7 +37,6 @@ java_test(
     deps = [
-        "//third_party:junit4",
@@ -50,7 +47,6 @@ java_test(
     deps = [
-        "//third_party:junit4",

It’s a good idea to run unused_deps regularly to keep things tidy. For example, the Bazel project does not run it automatically and has nearly 1000 unneeded deps (oops). You might want to add a git hook or something to your CI to run for every change.

Messy closet vs. clean closet
unused_deps: the Container Store of build tools.

GitHub notification… notifier

Here is what my inbox look like each morning:

All those pink-tagged messages are emails from GitHub. Gmail cannot figure out which ones are important and GitHub’s notification stream, in my experience, is useless. It’s noisy and doesn’t clear the way I’d expect. The problem is, people actually do mention me on bugs. And I often have no idea.

So, I made my own notification system. It’s a Chrome extension that is a little ear that sits next to your location bar. If you have an unread GitHub notification, it turns red. If you’re all caught up, it’s green.

If this would be useful to you, please give it a try!

Download the Chrome extension here.

Feedback and suggestions welcome!

Life hacks

I was thinking about a couple of little things that have made my life a lot better in the last year and I figured I’d share:

Buying cheese powder
I love mac & cheese, particularly Annie’s sharp cheddar. However, 1) they always give too many noodles and not enough cheese and 2) Annie’s switched over to only selling either gluten free (which has a glue-like consistency) or organic (and I’m philosophically, or at least stubbornly, against organic food). However, it turns out that you can get cheese powder on Amazon. I can use as much as I want (I discovered that there is such a thing as “too much cheese powder”) and on any pasta I want.
Hemming my jeans
I took a jeans-making class last year and, although I doubt I’ll ever make another full pair of jeans (it was a lot of work), I am now pretty comfortable with modifying existing pairs. I recently got a pile of $10-a-pop jeans from Goodwill and spent 20 minutes hemming up the bottoms and now have jeans that fit me much better than the $80-a-pair jeans I used to get.

To hem jeans you need a sewing machine, iron, and the ability to sew a straight line, but other than that, it’s pretty straight-forward. Put them on, pin where you want the hem to fall, measure. Say it’s 5″ from the existing hem (shut up, I’m short). Draw a line with a Sharpie (or tailor’s chalk, but I’m assuming most people reading this don’t have a well-stocked sewing room), giving yourself 1.5″ for the new hem (so a line 3.5″ from the existing hem). Cut off just within that line. Fold over the hem .5″ Sew it down. Iron the shit out of it. No one likes to switch modes and iron, but it’ll look super jenky and handmade if you don’t. Then fold over the hem again, which shaves off the last .5″, and sew it down ~7/16″ from the edge (as far away from the edge as possible, while still catching your folded-over part).

If you want the overstitching to be visible, use some sort of triple stitch, but I usually just use the normal stitch and it looks fine. You will probably have to hand-crank the machine across the seams, since it’ll be trying to stitch through 12 layers of denim at that point. Then iron again and you’re done.

Cooking breakfast on weekends
I have discovered that scones and dutch babies are very easy to make. It’s very luxurious having hot pastries and good coffee while the Google Home plays jazz.

Scones are great because they need cold, even frozen, butter (pro tip: get a 4-pack and stick it in your freezer, it’ll last forever) and don’t even require eggs. Just throw together flour, sugar, baking powder, butter, and salt, then add cranberries, chocolate chips, maple syrup and walnuts, or sprinkle with cinnamon and sugar. Always put a little extra sugar on top before baking, because it’ll form a delicious crust.

Dutch babies are great because, again, it doesn’t matter what temperature the butter is: you put it in a pan to start with and stick it in the oven anyway. Also, the taste reminds me of something from my childhood, but I haven’t figured out what, yet. I’ll have to keep eating them until I figure it out.

How to Skylark – the class

I’ve heard a lot of users say they want a more comprehensive introduction to writing build extensions for Bazel (aka, Skylark). One of my friends has been working on Google Classroom and they just launched, so I created a build extensions crash course. I haven’t written much content yet (and I don’t understand exactly how Classroom works), but we can learn together! If you’re interested:

It’s free and you can get in on the ground floor of… whatever this is. If you’ve enjoyed/found useful my posts on Skylark, this should be a more serious business and well-structured look at the subject.

I’ll try to release content at least once a week until we’ve gotten through all the material that seems sensible, I get bored, or people stop “attending.”

Stamping your builds


By default, Bazel tries not to include anything about the system state in build outputs. However, released binaries and libraries often want to include something like the version they were built at or the branch or tag they came from.

To reconcile this, Bazel has an option called the workspace status command. This command is run outside of any sandboxes on the local machine, so it can access anything about your source control, OS, or anything else you might want to include. It then dumps its output into bazel-out/volatile-status.txt, which you can use (and certain language rulesets provide support for accessing from code).

For our example, let’s suppose we’re creating fortune cookie binaries. We want each binary to be “stamped” with a different fortune, so we’ll use the fortune command as our status. Then, our build will add “in bed” to the end of the fortune, since we’re all super mature here.

Let’s create a genrule:

    name = "cookie",
    srcs = [],
    outs = ["fortune-cookie"],
    cmd = "cat bazel-out/volatile-status.txt | grep -v BUILD_TIMESTAMP > $@; echo '...in bed.' >> $@",
    stamp = True,

The most important part here is the stamp attribute: this tells the genrule that it depends on the volatile-status.txt file. Without this attribute, the volatile-status.txt file is not a dependency of this rule, so it might not exist when the genrule is run.

cmd prints out this status. The status has a default “BUILD_TIMESTAMP” field as well, so we strip that out.

Now create a fortune-teller.sh script (and make it executable):


/usr/games/fortune -s | tr 'n' ' '

This generates short-ish fortunes and removes all of the newlines (Bazel status files are line-based, each new line is independent and written to volatile-status.txt in any order the status generator feels like).

Now we can build our fortune cookie by supplying the fortune teller to the build:

$ bazel build --stamp --workspace_status_command=$PWD/fortune-teller.sh //:cookie
INFO: Found 1 target...
Target //:cookie up-to-date:
INFO: Elapsed time: 0.467s, Critical Path: 0.09s

Things to note:

  • You must enable stamping on the command line.
  • You also must pass the full path to the script, otherwise Bazel won’t find it.

Now if your take a look at bazel-genfiles/fortune-cookie:

$ cat bazel-genfiles/fortune-cookie
Truth is the most valuable thing we have -- so let us economize it. 		-- Mark Twain
...in bed.


Using secrets with Google AppEngine

For side project #4323194 (implement a chrome extension that looks like this: 👂 and turns red when someone mentions you on GitHub), I needed to implement oauth from AppEngine to GitHub. As I’ve mentioned before, oauth is my nemesis, but for this project there didn’t seem to be a great way around it. It actually wasn’t as bad as I remember… maybe I know more about HTTP now? Either way, I only messed up ~16 times before I got it authenticating properly.

When you want an app to work with the GitHub API, you go to GitHub and set up a new application, tell it what URL it should send people to after login, and it gives you a “secret key” that no one else should know. Then you simply implement oauth’s easy, intuitive flow:

  1. Redirect a user to https://github.com/login/oauth/authorize when you want them to log in.
  2. GitHub will ask the person to log in, then redirect back to the URL you gave it when you set up your app.
  3. You POST the secret key you got to https://github.com/login/oauth/access_token.
  4. GitHub replies with an access token, which you can then use in the header of subsequent requests to access the API.

The problem here is #3: the secret key. In ye olde world of “I have a server box, I shall SSH into it and poke things,” I would simply set an environment variable, SOOPER_SECRET=<shhh>, then get that from my Java code. However, AppEngine prevents that sort of (convenient) nonsense.

So, I poked around and the thing I saw people recommending was to store the value in the database. This is an interesting idea. On the downside, it’s approximately a zillion times slower than accessing an environment variable. On the other hand, this is a login flow that makes three separate HTTP requests, I don’t think a database lookup is going to make a huge difference. On the plus side, it “automatically propagates” to new machines as you scale.

So I began working on a class to store the secret. Requirements:

  • Easy to set: I want to be able to visit a URL (e.g., /secrets) to set the value.
  • Difficult for others to set: I don’t want users to be able to override keys I’ve created, or create their own keys.
  • Perhaps most importantly: difficult for me to unintentionally commit to a public GitHub repo. I am super bad at this, so I need a completely brain-dead way to never, ever have this touch my local code, otherwise it will end up on GitHub.

What I decided on:

Create a servlet (/secrets) that takes the key/value to set as a query parameter. The servlet will only set the secret key for keys I’ve defined in code (so visitors can’t set up their own secret keys) and will only set the secret key if it doesn’t exist in the database, yet. Thus, after the first time I visit /secrets, it’ll be a no-op (and actually can be disabled entirely in production). Because the secret is given as a query parameter, it never hits my code base. It will appear in request logs, but I’m willing to live with that.

What this looks like in an AppEngine app:


And the Java code does some URI parsing and then:

  private void findOrInsert(String key, String value) {
    Entity entity = getEntity(key);
    if (entity != null) {
      // No need to insert.

    entity = new Entity(ENTITY_TYPE);
    entity.setProperty("key", key);
    entity.setProperty("value", value);

And the nice thing about using Google’s AppEngine datastore is that it’s easy (relatively) to write tests for all this.

You can check out the sources & tests at my git repo. (Note that the extension doesn’t actually work yet, right now it just logs in. I’ll write a followup post once it’s functional, since I think this might be relevant to some of my readers’ interests.)