Greg Detre

Pluviocabulary (rain words), Memrise-style

2011-04-03T14:48:00.000-05:00

I got distracted by the NY Times' Pluviocabulary list, and found myself playing with my new words - just thought I'd share. I particularly enjoyed being reminded of petrichor, and learning flaught and tirl.

The iPhone apps my cold, dead hands would cling most rigidly to

2011-01-10T17:23:00.000-05:00

Instapaper - combine this with the Instachrome extension, and whenever I see a webpage I want to read later, it'll be waiting with me as I wait for a train

Light - it's bright! No more torches. If you lived in Hanborough, you'd need this too.

Trainline - faster than my laptop and/or a speeding bullet for checking train times in the UK

PlainText - write notes on your laptop, have them appear on your phone instantly through Dropbox and vice versa. Oh, and Dropbox of course, too.

Dictionary.com - etymologies, pronunciations, the works

Remote - control Keynote presentations from your phone.

Glympse - let other people know where you are.

Skype - I can call Mia for free while walking the streets

iTrans Tube and Tube Status for planning London Underground journeys

Snaptell - red laser black magic. Point at a book, and have elves whisper about it to you.

Angry Birds - the most popular mobile game of all time

Spotify - all the music in the world on the go. Requires a Spotify subscription

Shazam - tells you the name of songs that are currently playing

Startup thinking - the people who have most influenced my thinking on startups.

2011-01-09T17:00:00.001-05:00

Steve Blank and Eric Ries

Steve Blank teaches entrepreneurship at the Haas Business School in Berkeley, but has a pretty serious pedigree as a tech entrepreneur himself. I'm ashamed to admit that I still haven't read The 4 Steps to the Epiphany, but I've read most of what he's posted online.

Eric Ries is a protege of Steve Blank's, applies and develops many of the same 'lean' ideas, and focuses specifically on web startups.

See the links for both Steve Blank and Eric Ries here.

----

Paul Graham

He's a tour de force and a hero of mine. One of the early proponents of 'release early, iterate often'.

http://www.paulgraham.com/start.html

http://www.foxbusiness.com/search-results/m/25897600/funding-tech-start-ups.htm

----

Joel Spolsky

These are my favorite as they relate to HR:

http://www.joelonsoftware.com/articles/GuerrillaInterviewing3.html
http://www.joelonsoftware.com/navLinks/fog0000000262.html

Where has all my disk space gone? (Mac)

2010-12-30T18:54:00.001-05:00

This is the best app I've found so far for visualizing where all my disk space has gone:

http://www.daisydiskapp.com/

I have a theory that inability to manage one's finances and one's hard disk space have the same psychological factors at their root.

P.S. It's free, but you can pay $20 to get it to stop nagging you.

Get your users to tell you what they really want

2010-12-21T17:00:00.000-05:00

We're building a site that we want people to love using, so we want as much feedback on it as we can get. From the get-go, we've had a big red UserVoice 'Feedback' button on the lefthand site of every page. But only a tiny proportion of our users ever suggested, voted or commented on an idea.

I'm not surprised. When you click the UserVoice 'Feedback' button, it brings up a list of the top few existing ideas to vote on, plus a vague invitation to 'go to our feedback forum'.

If the problem you're having is on that list, then you're in great shape. But if you have a particular suggestion to make, there is nowhere to do so. In this respect, the 'submit an idea' heading is misleading since this box doesn't really let you do that.

My guess is that not many people really notice the link at the bottom of the box to the feedback forum. This is a shame, because the forum itself is well-designed. It has a nice big box to add a new suggestion. It does a good job of minimizing duplicates by pointing out ideas that have already been submitted as you type. And the idea of giving users 10 votes to spend is enticing, and a rich source of information.

But despite that, we're scrapping the UserVoice Feedback button and disbanding the forum. Here's why:

- For the reasons described above, only a vanishingly small proportion of people who have something to say end up submitting/voting/commenting on an idea through UserVoice.

- We're trying to be thrifty, and our userbase is small. So we don't want to pay for the much more expensive plans - this cheaper plan doesn't allow us to transfer our users' login status from Memrise to UserVoice. As a result, many ideas get submitted anonymously, which makes it harder for us to engage in a conversation with a user [1]. Of course, people can create an account on UserVoice, but that's a huge pain.

- By emphasising the top ideas, there's a rich-get-richer effect. People vote most often on the ideas presented at the top of the list, and don't notice potentially more interesting ideas just below. UserVoice should hide how many votes each idea has already accrued until you've already voted - otherwise an idea's future popularity will be largely determined by its existing popularity.

So what did we do instead?

- We wanted to bring the barrier to entry down to zero. So we just dropped in a plain textbox, asking plainly what we can do to improve things. [2]

- The red Feedback button was ubiquitous, but I bet that made it effectively invisible. Our feedback box gets presented opportunely and prominently at the end of every learning session, at a moment when people will be most likely to have something they want to tell us.

- We get an email every time someone drops in a suggestion, with the user's email address, so we can respond quickly to them individually. We very much want to follow up with people that have made the effort to give us feedback.

I'd say we're getting at least 5 times as many suggestions as we were.

On the downside, we don't have a nice communal forum any more that allows people to vote or comment on one another's ideas - we might do something about that in the future.

Here's what I wish UserVoice would do:

- Make it much easier for people to suggest new ideas. Even if it were to bring down the overall quality, I think an increased volume of raw responses would be of greater value.

- Charge straightforwardly based on the number of new suggestions. This would set up the right incentives for UserVoice to make it really easy for users to make new suggestions. N.B. I don't have a problem with paying - I just don't want to be forced to start out with a $100/mo plan when we have a tiny userbase in order to get features that I consider essential to the user experience.

- Corollary: Don't withhold important features like single sign-on and white labeling the design for the exorbitant options.

[1] We compared UserVoice and GetSatisfaction pretty closely, and they both hike the prices if you want to be able to transfer login status across. In fact, GetSatisfaction didn't (at least at the time) allow anonymous suggestions, which felt like a huge barrier to entry to new submissions - this was what convinced me to try UserVoice in the first place.

[2] Hmm - perhaps this should be a command, e.g. 'Tell us what we can do to improve things'?

Storyville and Jenga iPhone apps

2010-12-20T17:00:00.000-05:00

Hey everyone,

Two friends have just released iPhone apps in time for Christmas:

Jenga, by Natural Motion. This is the official Jenga iPhone app - there's a fiendishly realistic physics model of the world behind the scenes, and it's a thing of beauty.
Storyville by Fatty Apps - I love the idea of receiving a short story each week, beamed straight to my outboard brain.

Buy. Them.

P.S. I'd be happy to put you in touch with Torsten or Atul if you have any direct feedback you'd like to give them.

Setting up your own domain name and website

2010-12-18T14:19:00.004-05:00

Someone asked me recently about getting their own domain name and setting up a website. I'm not very good at this stuff, but I have been through it once or twice, so I thought I'd at least offer this up in case it's useful.

Let's say you want to buy example.com, and set it up as a series of static informational pages about Example Business Inc, along with @example.com email addresses. There are (at least) 3 ways you can go:

1) the standard method

- Grab the domain from GoDaddy (or any other domain name registrar - they all do basically the same thing). It'll cost you $10-20 for a year or two

- Then you need to find a place to host your site (e.g. Rackspace, Dreamhost). You'd spend c. $10/month to rent space on a server, point your new domain name to the server's IP address, write and upload some html and images, and away you go.

- You then need to set up email addresses. If it's GoDaddy, I think you'll be able to set it up to forward your email to an existing account without too much trouble.

- This is what i had to do with Memrise because I wanted control over everything. Honestly, it was much much more complicated than i had anticipated to figure it all out.

option 2) use Google

- Use Google to register your domain name

- I think that'll automatically set you up with Google Apps (custom Gmail, Calendar, Sites, Blogger, Docs etc.) for free.

- Then you can set up the design and content of the pages of your website with Google Sites.

- So then you'd all use a custom Gmail interface to check your @example.com address, and have access to blog.example.com, calendar.example.com etc. I'm a big fan of Google Apps.

option 3) Weebly (or some equivalent competitor)

- Besides Google, there are a variety of companies that help build a site. I've heard good things about Weebly, but haven't closely investigated it for myself.

- Much like Google Sites, it looks like they'll do most of what you'd want: help with design templates, deal with the hosting, potentially help with domain names, and a bunch of other stuff. Nice!

Conclusions:

- As long as your needs are simple, I would consider the Google/Weebly approach, since I think it'll be the most straightforward.

- Down the line, if you decide that you want to build something more complicated and interactive, you can always hire a programmer and switch from Google/Weebly to your own hosting set up.

- If you have someone to help who enjoys techie stuff or has set up their own site before, then setting things up with GoDaddy and your own hosting will probably go smoothly. But otherwise, a company like Google or Weebly that'll do 90% of the work for you, so you can focus on building a great site :)

Breaking the seal

2010-11-05T16:42:00.000-05:00

I've wanted to try daily writing for an impossibly long time, but the first words didn't want to be dragged out.

In my case, they were unstoppered by a day in London. I pinballed from train platforms to coffee shops, oblivious bustle all around me, far away from the furrowed-browed finger-pecking at Memrise HQ. That context-shift provided a firebreak from the quotidian, and I was finally in the mood to mentally roll up my sleeves and rub my hands together.

Writing's like peeing - once you break the seal, the words just spill forth all evening.

I was able to decant a dozen half-thoughts that I queued up like toy soldiers, to be birthed one by one over the following week. It's now rather fun to receive a blog post from my previous self every day.

A letter to a prospective grad student

2010-11-04T09:00:00.001-05:00

Preface: I wrote this to a friend asking me for advice about whether to embark on a science PhD. At the time of writing I still had more than a year to go - so I could see the summit in the distance, but I was feeling grim about the steep icewall I had to climb to get there.

In retrospect, I think a lot about Jeff Bezos' advice: don't be proud of your talents - be proud of the things you really worked hard to achieve. For this reason, I'm more proud of (and glad about) my PhD than anything else I've yet done.

----

It's hard for me to summarize my thoughts on grad school, perhaps because it varied so much in so many ways. Grad school was wonderful when I was excited about it - for the first few years, there was literally nothing I wanted to do more than talk and think and write and program lab stuff. Every week was filled with new ideas, a sense of progress and discovery, and I bounded into the lab every morning.

I don't know what changed exactly, but at some point, I started to really lose enthusiasm. I'm perenially stymied by an inability to understand the source of my own motivations, and to make sense of my own emotions. So I don't really feel like I understand why the joy started to fade. Perhaps because I worked for years on ambitious experiments that didn't work out. Because I'd been in one place for years. Because I'm a little flighty. Because I thrive in a more competitive or fast-moving jobs. Because really I love AI and computers a little more than brains. Because I wanted to be my own boss. Because I lost confidence. Because I need to feel part of a team working towards a common goal. Because I needed more inter-personal contact with a range of different people. Because I'm not temperamentally suited to be a scientist. Because I need to be in a city. Because I felt obliged to finish it, after investing so much into it, long after I would have left a normal job. Because the specialization necessary can come to seem like a straitjacket. Because I got obsessed with new ideas. I don't know.

It seems to me that a PhD is the right move if one loves what one's doing, and one wants to be an academic. Of course, you can't know for sure in advance that both of those are true. But if you think they might be, then go for it! While I think we have some things in common, I don't expect the idiosyncracies of my experiences to apply closely to anyone else, so don't look too closely for parallels to yourself in my issues above.

Right now, starting a company feels like the job I've been looking for my whole life, but I wouldn't have the wherewithal to do it unless I'd been through the last few years.

I don't know where your path will lead. Like me, I think you get excited about a lot of things, and could happily set off in many different directions, including becoming a great and happy scientist.

This email doesn't really answer any of your questions. I'm sorry about that - I just don't want to give advice one way or the other, because I think you'll make the right choices without my advice, and because you'll make whatever choices you make into the right ones. You are a lucky guy, in this (technical) sense - http://gregdetre.blogspot.com/2009/10/i-dont-believe-in-luck.html

:)

Keep me posted.

When I am famous, I will decline interviews

2010-11-03T23:25:00.000-05:00

Reading the 5-page staged and glossy magazine interview in a hotel room with a famous actor has always filled me with a peculiar kind of existential dread. There's something a little horrifying about an hour of conversation in cold type, bereft of the intonation, expression, context and rapport that make anything one says out loud bearable. And at the end of it all, to be distilled, distorted, interpreted and weighed by the pen of a stranger... Who could have the strength of character to read about but not become their own caricature?

In contrast, the last page of the Sunday Times magazine features 'a life in the day of' a happy array of personalities and professions. I like the concreteness of a single day as a window into someone else's micro challenges and achievements. I realize that these days are probably fictionalized composites - but fiction makes for a sweet, concentrated and memorable pill. And at the end of it, there is no distillation, no weighing - just the reality of a daily rhythm.

When I am famous, I will decline interviews.

P.S. That said, I still remember being stopped in my tracks when a fashion photographer relative asked me sweetly 'what did you today?' in the midst of my PhD. My day had consisted of:

2 hours debugging a misplaced comma
so that I could finish the 3-day long project of rearchitecting my non-parametric statistics to work across-subjects
in order to get a better sense of whether results from the latest in a long line of experiments were actually better than chance
so that we could tell whether reminding people and distracting them at the same time was causing them to forget
to test our computational theory that half-remembering a memory actually weakens it
which would have deep implications for our understanding how the brain learns and self-organizes

But really, I'd been comma-hunting, and it seemed hard to fit that into a the kind of response usually expected from 'what did you do today?'.

But where does the wisdom come from?

2010-11-02T23:47:00.000-05:00

God, grant me the serenity

To accept the things I cannot change;

Courage to change the things I can;

And wisdom to know the difference.

With the support of those around me, I can struggle to be courageous.

I believe I can summon forth the serenity.

But where does the wisdom come from?

The muses are deaf, so speak up

2010-11-01T17:00:00.000-05:00

Good thoughts tend to shy away from short walks with a destination. They're kept at bay by the neuroses and instant replays that circle endlessly like tethered carrion.

Do you want to know the only way I've found to think while walking? Talk out loud. Loudly proudly aloud. Feel free to gesticulate. Close your eyes if traffic conditions permit. Tell yourself a story. Don't use your normal voice.

Why would talking out loud make such a colossal difference? Perhaps because repetition feels explicitly boring out loud, so we avoid re-treading the same paths. Perhaps because full sentences flush and flesh out our half-thoughts? Perhaps because serializing our massively parallel murmur squeezes the thoughts out one at a time with greater velocity, like putting your thumb on a hose.

The effect is so striking that I've wondered about potential neuroscientific explanations. It could be that different neural pathways are being activated - perhaps it is only by vocalizing that we recruit speech production areas, or only by hearing our own voice we recruit speech comprehension areas. Or just that there's less neural juice sluicing down the byways of my mind during my inner monologue, and the extra oomph required to speak gives the thoughts extra vivacity.

The explanation I favour? If I'm going to have to listen to myself, I want to be entertained.

P.S. For best results, wear a hat and learn to talk like Tom Waits.

How can your iPhone make you even more entertaining and interesting than you already are?

2010-10-31T14:00:00.002-05:00

Have you ever had a conversation with smart friends that got hung up on some disputed point of fact, or tip of the tongue memory failure? Don't you just wish someone would step in with the answer to unclog the free flow of ideas and happy banter? Disputes about facts and tip-of-the-tongue feelings *should* be a relic of the 20th century. So there are two things that are remarkable here:

- Through smartphones and search engines, we can marshal thousands of machines to produce the answer in the blink of your mind's eye.

- But we have to perform that instantaneous incantation with pudgy fingers and a 0.3G internet connection. I challenge anyone to find the name of an actor in under 2 minutes with an iPhone with crappy reception. While those 120 seconds creep past, you're coldly ignoring your friends, and the conversation is gasping on the table like a naked baby on a spacewalk.

Here's one technological solution to this social problem:

- At the beginning of the conversation, we all put our iPhones on the table, and fire up the Inforager app.

- Inforager is listening to us, uploading the audio of our conversation to voice-recognizing clouds.

- It runs dozens of google searches continually in the background, displaying result-snippet-bubbles that float past, driven by the whorls and eddies of our conversation. While we're talking about the beardy guy with the Greek name in The Hangover, a bubble for 'Zach Galifianakis - IMDb' looms large, only to be nudged offscreen as we move to debating whether the 'candied sunchokes' on the restaurant menu are likely to taste more like sunflowers or artichokes, while the other half of the table engages in a dialog on the nature of catnip.

In other words, the answers to questions we have are being provided in real time in response to our conversation. This frees us up to talk about what matters.

----

Technical notes:

- If multiple people at the same table were calling Inforager, it would use the multiple sound sources to do a better job of distinguishing voices and improving audio quality.

- Is it possible to use the phone (rather than the 3G connection) to upload the audio data? That would drain the battery much less.

- I made up the name Inforager.

A wiki for spaces. A town anyone can edit. School architecture founded on mnemonic principles

2010-10-30T23:33:00.002-05:00

When we think of wikis, we think of text, like the Wikipedia. But this notion of content that anyone can view and anyone can edit has barely unfurled its wings. What if we were to apply it to space?

For instance, imagine growing a World of Warcraft town as a community. Each person could design and improve upon the buildings, fill the walls with graffiti, neighborhoods would define themselves... the ease and pace of iteration might even generate new ideas about town planning.

Alternatively, let's build on Ed Cooke's fantastic plan for school architecture in the future [cached]:

Children, well known to be compulsive absorbers of information, crucially learn what they are interested in. Like all animals, they are interested in spaces.

I’d like to see schools’ spatial layout reflect the history of Western culture, and thereby implicitly teach it. A snake-like line of school buildings could begin at one end in Ancient times and run on, in temporally organized fashion, up to the computer science blocks of the present day. Key themes and figures from each epoch could provide the names for classrooms, which could also reflect some of the architecture, customs and furniture of the day.

Because in five years of school, everyone learns every detail of the spatial organisation of the buildings, and because memories always attach to the spaces in which they were first formed, merely attending such a school would give one a wonderfully detailed sense of the history and structure of Western civilisation. And it wouldn’t need to be prescriptive, for one could take advantage of the second source of childrens’ interest - things they have a role in - to redouble the effect. Each year-group could, over the course of five years, reconsider, re-design and re-build one of the twelve epochs/buildings.

Convincing someone to build a school organized on mnemonic principles is going to be tricky. But in the meantime, perhaps schools' online presence might take the form of a spatial wiki. Students could make changes ranging from decor to naming to overall organization, shaping their online school to their memories and vice versa. We love to deeply inhabit our environment by shaping it - what could be better than exercising our rich faculty for spatial navigation imaginatively?

Master Turkers

2010-10-29T20:07:00.000-05:00

Amazon's Mechanical Turk is an amazing service where one can create a simple task that can be micro-out-sourced to many people over the web, each of whom performs a small parcel of it. For instance, if you wanted 1000 people to highlight faces in photographs, think of synonyms for words, or provide from-the-hip feedback on your website, Mechanical Turk is ideal.

For reasons that are unclear to me, people seem to be willing to work for far below a minimum wage performing pretty dull tasks. As an experimental psychologist, I'm torn between feelings of data lust at the number of participants I could thus thriftily recruit, and concern about the quality of their data. What kind of person is willing to engage in dull tasks that must feel meaningless from a worm's eye view? Where's the incentive to do a good job?

It seems to me that there might be a market for tasks that require more effort, skill or thought, for which one would like to be able to cherry pick the participants. For this to work, you'd need a rich reputation scheme to Mechanical Turk, to pick out the Master Turks.

I'm picturing myself in holidays as an undergraduate. If someone was willing to pay more £10/hour (roughly what I was earning as a medical secretary), I (or my more talented peers) would have happily:

researched historical facts for a novel
proofread a doctoral thesis
helped with market research for a business plan
written a catchy jingle
filmed a youtube video using your product
written a program to generate verbal reasoning or arithmetic questions for an exam
provided summaries of white papers

You could imagine non-fixed-rate payment schemes, e.g.

a competition where the best submissions divide the spoils
an auction, so that more enjoyable tasks would be bid lower

And, deliciously, you could create a meta peer-review system where other Master Turkers' task is to rate the submissions you've received.

Stack Overflow is going to transform the programming job market by making answering people's questions satisfying, and then providing a metric of someone's expertise that will help them land a job.

There have been many precedents of this kind of idea, but it seems strange that none of them have taken off. This feels like a way to demonstrate one's abilities on potentially interesting tasks that would provide a portfolio of work to supplement a job application.

Brain orchestras and fMRI analyses

2010-10-28T21:52:00.005-05:00

[With help from David Weiss]

I spent much of my PhD working on algorithms for making sense of gigabytes of brain data from fMRI scanners, especially on a fairly new approach called Multi-variate Pattern Analysis (MVPA). I want to show you how the MVPA approach is useful for tackling certain kinds of questions.

Think of the brain as a kind of orchestra. You have lots of separate instruments playing at the same time, and you can subdivide them in lots of different ways, e.g.

You can subdivide the orchestra into parts by location - the 1st violins, the brass, the percussion etc.
Or you could organize them by what they're doing. Say the 2nd violins, the oboes and the trumpets have the melody, while the clarinets and the tubas have the harmony. [The harps are doing their own thing and the bassoonist is drunk.]

Likewise, there are all kinds of things going on at once in the brain.

You can subdivide the brain by location - frontal, temporal, parietal, occipital lobes.
Or you could organize the sub-parts by what they're doing - vision, language, executive control, motor etc.

Let's go back to thinking about how the multivariate approach differs in the kinds of questions it can address.

Standard univariate analysis is useful if you want to tell which instruments are involved in one case rather than another, e.g.

violins are more active in Beethoven than Mozart, but for trumpets it's the other way around

one part of the brain is more active when looking at houses than faces, but for another part it's the other way around

In contrast, a multivariate analysis might be useful if you want to know:

is this Mozart or Beethoven?
is this the brain of someone looking at faces or houses?

Now, let's introduce one more concept: dimensionality reduction is an attempt to boil down many instruments (or brain regions) into a few key themes/groups:

Take the famous da-da-da-dum of Beethoven's Fifth, where the entire orchestra is one voice - one could more or less describe the entire orchestra's activity in terms of just one theme/process. In contrast, for Bach or something more complex and interwoven, it might be very hard to summarize what's going in with less than 10 themes.

Likewise, maybe it's straightforward to summarize the brain's activity with just one or two processes when you're doing a very simple task like looking at faces vs houses, but if you're doing something more complicated (like watching a movie) then multiple processes are interacting in complex ways.

David Weiss's PACA algorithm boils down the brain's activity over time into just a few themes. Once you've summarized the 50,000 readouts we get from fMRI every few seconds into 50, it's much more feasible to try and compare different cognitive processes - just as it's much easier to compare Mozart and Beethoven by looking at the scores of a few key instruments than looking at the full orchestral scores.

PACA was inspired by a bunch of existing dimensionality reduction algorithms that could equally be applied to problems like voice, face or handwriting recognition.

But its magic involves adding a few constraints that are particularly relevant to the brain. Here's one example of a constraint: it doesn't allow its estimate of a theme's presence at a given moment to go below zero. Think of it like this - when was the last time you heard an anti-violin? Or had an anti-thought? In other words, PACA breaks the manifold streams of activity in the brain down to just a few that are all present to a greater or lesser degree at each moment.

P.S. If you hated this, you might also hate How to beat an fMRI lie detector.

Auto-links

2010-10-27T19:05:00.029-05:00

Most wikis require you to perform one of two contortions to create a link:

Use CamelCase. Much like a camel, this is robust, but tiring to finger.
Wrap things in ["symbols that are hard to type"].

In both cases, you need to know in advance that you plan to create a link, and be enough of a disciplined philistine to overcome the effort and overlook the ugliness.

Auto-links are the solution [1] - here's how they work. Say you create a page called 'Camel case'. Now, type Camel case anywhere else, and that 'Camel case' text will be turned into an auto-link as you go. In other words, the wiki notices that you've typed the name of an existing page in the midst of your text, and automatically creates a link for you. If you go back and edit the text, the link goes away. [2]

Links between pages become evident to readers without any extra effort on the part of the writer. If I type 'MySQL' and an auto-link appears, it's easy to see that a relevant page about it already exists.

Having used such a system for a long time, I have come to appreciate the tiny flash of satisfaction at seeing a link appear with no extra effort, confirming that the page does indeed exist [3], and making navigation while editing a breeze. Pages that I wrote years ago are now festooned with links to pages that were created long afterwards. Indeed, the most satisfying feeling of all is when an auto-link pops up to a page I'd forgotten I wrote. Lazy serendipity!

[1] see Per Sederberg's implementation in Emacs Freex mode, though we called them 'implicit links' back then

[2] To do this the way God intended requires running a regex containing all the pagetitles in your wiki over what you type on every keystroke - this is very nearly instantaneous for even 10k documents.

[3] For extra points, allow pages to have multiple aliases, so that (for instance) 'database', 'databases' and 'MySQL' all point to the same page.

Eroding our minds

2010-10-26T12:50:00.002-05:00

I said that I thought "there's something irresponsible about making money from advertising".

Matt Weber was right to point out that although people hate the idea of targeted ads, they can be genuinely useful. Though I don't think a very large proportion of the available advertising real estate offers the possibility for really great targeting.

[Of course, good advertising can be an art form in itself. And by funding most of our software and reading materials, advertising adds tremendous value to our lives.]

But even on the internet, most advertising still feels as though it's about increasing our familiarity with the brand.

Think of advertising in terms of cognitive fluency, i.e. how easy we find something to process. There are lots of ways to make something fluent - make it easy to read, easy to pronounce, write it in a simple font, or in high contrast.

Things that are fluent (easy to process) get processed faster. We tend to like fluent things better, find fluent statements more valid. We think companies with fluent names are more valuable.

Advertisers have (implicitly) known this for a long time. By incessantly dinging our minds with an advert over and over, we are gently having that brand branded upon our minds, making it easier to process, more familiar, and making us unwittingly and unjustifiedly like it more. Like the banks of a river worn smooth by the ceaseless flow, advertising erodes our minds.

If you are not paying for it, you're not the customer; you're the product being sold.

I refuse to consider an advertising-based business model

2010-10-25T08:38:00.002-05:00

"Chess is as elaborate a waste of human intelligence as you can find outside an advertising agency." - Raymond Chandler

How many Wikipedias might we build every day if we didn't watch advertising? [transcript]

"Shirky defines as a unit of attention "the Wikipedia": 100 million person-hours of thought. As a society we have been burning 2,000 Wikipedias per year watching mostly sitcoms"

There is something irresponsible about making money from advertising. The idea of monetizing people's attention makes me feel the same way I'd feel about burning books to stay warm.

Nietzsche

2010-10-24T11:21:00.000-05:00

“He who has a why to live for can bear with almost any how. " (Friedrich Nietzsche, quoted in Frankl, 1963, p. 121)

Communal interactive jukebox

2010-10-23T13:39:00.000-05:00

[I wrote this in 2003 - there are still pieces of this vision that haven't been realized]

why isn't there a little wireless didgeridoo that just sits next to a cd player (stereo audio input), with a wireless network card, and maybe an ip address or a network id or something that you can initially configure easily/remotely by plugging in a computer via a usb or something, that just sits there and plays whatever your laptop running winamp tells it to by wireless???

apparently these exist already :(
but they're pretty crap at the moment - they're proprietary, and are only just getting up to speed with 802.11b etc.

this doesn't exist though:
you could set a password to it, and then anyone with a laptop nearby who knew its id and had the password, could wrest control of it, e.g. at a party. better still, you could have a sort of queuing system/software for allowing different users to place requests, and people could vote whether they like what's playing and that person's reputation would go up - like slashdot karma - it would be a sort of communal interactive jukebox

House MD, gibberish and clown school

2010-10-22T16:27:00.004-05:00

I'd been watching a lot of House MD a while ago. Perhaps 1/3 of the show is taken up with mystical medical mumbo-jumbo that's gibberish to me, and yet it's still compelling. How can that be? It's like watching a soap opera in a foreign language.

Then I talked to a friend who went to clown school, who didn't find it at all surprising. He told me about his 40-minute graduation show that transfixed and amused audiences using only nonsense words.

House MD as a modern medical jabberwocky makes a certain amount of sense.

What's blowin' in the wind?

2010-10-20T21:17:00.003-05:00

[Thanks to Stephen Hartley-Brewer for the kernel of this idea]

My brother is my musical weather vane, my song-canary who knows what's good long before the rest of the world has cottoned on, and points out the things I'll like. I treasure his advice, partly because it's good, and partly because it comes from him.

But just for badness, pretend you don't have a brother with his ear to the ground. Instead, you have a big computer that you've fed the listening habits of the entire world. Ask it two questions:

1) who is listening to music that's hugely popular (or that I like) a year ago?

Those are your early adopters.

2) what are they listening to now?

Throw in some collaborative filtering to tailor the recommendations for me.

Are iTunes or Last.fm doing this to predict who's going to be big next year?

Are the record labels doing this to predict where they should put their marketing money?

Can bands buy this information to figure out which users to send promo albums to?

If there was a prediction market for music, could my brother get paid to tell people what he likes and dislikes?

Dropbocumentation

2010-10-20T17:37:00.000-05:00

Every time a programmer goes away for a few days, a piece of infrastructure they know best breaks. That's just Murphy's Algorithm.

If they they had only written a 100-word overview with some examples, that would have saved someone else a painstaking day figuring out how things should work, why they suddenly don't, and righting the world once more.

How do you make it likely that everyone writes down what they know while it's still fresh? Think of edits as conversions (in the analytics sense) - our funnel stretches from signup to viewing to editing, and we want to maximize the number of edits.

How do we optimize the 'edit' conversion rate for a wiki?

Editing should happen in the same mode as viewing. If you have to click 'edit', then wait for a page refresh, then scroll down inside a teeny textbox in a browser, then hit save to see your changes ... those steps create a barrier to entry. The conversion rate of views to edits will drop dramatically. Typos, inaccuracies, inscrutabilities and out-of-datenesses will accumulate.

It needs to be as available as possible. All and only your team can access and contribute to it, even if they're on a different computer or offline.

Consolidate everything in one or two searchable places. When it's hard to find something, you won't want to start looking. If you have to search one by one through a wiki, your email, a bug tracker, the version control commit log and comments in the codebase, you'll end up just tapping someone on the shoulder - the knowledge will never get planted in a way that it can grow.

No special knowledge. Wiki markups are confusing and confusable. WYSYWYG editors are a good start - but editing text in most browser textboxes feels like typing with chopsticks. And proprietary document formats are opaque and constricting.

No barrier to exit. I want to be able to easily (and ideally automatically) grab a dump of all our documentation, both as a backup and as an export.

After reviewing these possibilities over and over, these are the best solutions I've come up with for Memrise:

A few monolithic Google Docs. This has worked reasonably well, except that Google Docs still falters in an unwieldy and buggy way when dealing with even medium-sized documents. Boooo!
Etherpad clone. They seem pretty expensive for multi-user monthly subscriptions, and seem weak at linking and searching. Plus, they don't work offline, and I don't trust the companies behind them to be around in 5 years' time.
Text files in Dropbox. The main downside to this is that you can't easily inter-link text files, and they lack formatting which makes them ugly to read. But they have no barriers to entry whatsoever.

In an ideal world, someone would build a nice (optionally hosted?) wiki solution pulling and formatting Dropbox text files as webpages to give you the best of both worlds, perhaps combined with a few desktop apps and extensions to make offline viewing editing more pleasant.

Excretation over

2010-03-07T18:23:00.001-05:00

I just handed in my dissertation draft - Weakening memories by half-remembering them. This was the sunrise that birthed it (inspired by neurotomfoolery).

Mewling infant.

A hivemind with a sense of humor

2010-01-18T12:22:00.001-05:00

I'm a little obsessed by the notion of a noosphere, a humming hivemind - not a humdrum, roaring average, but rather a superlinear interwoven sum of wits.

The Bible has this quality, with its sea of voices that are unabashedly inconsistent and yet superhumanly wise. But what I find most unsettling about the Bible is its lack of humor. To my knowledge, there's not a jot of wit, humor or silliness in the whole thing. Perhaps this befits something with a purpose greater than simply sublime literature, but that dehumanizes it to me.

So, it's endearing and cheering to see that Google can giggle [from autocompleteme.com]:

Though more soberingly, see what boyfriends and girlfriends search for.

My two favorite music videos of all time

2010-01-11T21:25:00.003-05:00

This manic messiah, his impish accomplice and their merry band of virtuoso, ragtag disciples make me feel a little joyous every time.

Edward Sharpe & The Magnetic Zeros - Home (live @ kcrw)

And then there's Tom Waits. Bullfrog with a bullhorn, peacock with brass knuckles - this is fearless, peerless showmanship.

Tom Waits - Chocolate Jesus on Letterman

How to beat an fMRI lie detector

2010-01-04T00:48:00.005-05:00

In a not-so-distant dystopia, you might be placed in a brain scanner to test whether you're telling the truth. Here's how to cheat.

The polygraph

First, you'll need some background on old-school lie-detection technology. [This is a simplified story - see polygraphs for a richer account.] Polygraphs are seismographs for the nervous system. They measure physiological responses such as heart rate, blood pressure, sweatiness through skin conductance, and breathing. When you're anxious, angry, randy, in pain, or otherwise emotionally aroused, these measures spike automatically. The effort and stress of lying also causes them to spike.

Of course, if you're trapped in a windowless room on trial for murder, these measures will probably be pretty high to begin with. So you'll first be asked a few control questions to assess your baseline levels when lying and telling the truth, against which your physiological response to the important questions will be compared.

So, if you want to beat a polygraph, you need either keep your physiological responses stable when you lie (which is difficult), or you need to artificially elevate your baseline response when telling the truth. The age-old technique is to place a thumb-tack in your shoe, and press on it painfully with your toe when telling the truth, spiking your physiological responses, and providing a misleading control so that your lies don't seem higher relatively.

Functional magnetic resonance imaging

Now, on to fMRI. Simplifying again, the fMRI brain scanner takes a reading of the level of metabolic activity at thousands of locations around your brain every couple of seconds. Activity in a number of brain areas tends to be elevated when we lie, perhaps because we have to work harder to invent and keep track of the extra information involved in a lie, and override the default responses in the rest of the brain. Under laboratory conditions, accuracy at distinguishing truth from lie approaches 100%.

The modern machine learning algorithms used to make sense of the richer neural data are more sophisticated than those used in a polygraph. And they're measuring your brain activity (albeit indirectly), so it might feel as though there's no way to deceive them. But ultimately, they work in an analogous way to the polygraph, by comparing your neural response to the important questions with your neural response to the baseline questions. That means that they can be gamed in an analogous way - as you're being asked the baseline questions, wiggle your head, take a deep breath, do some simple arithmetic or tell a lie in your head. Each of these will elevate the neural response artificially. By disrupting the baseline response, you disrupt the comparison.

Possible flaws in this argument

This simplified account of how to cheat an fMRI lie detector has some issues.

Firstly, it rests on the idea that we'll still use some kind of comparison between baseline and important questions. In the case of most recent fMRI analyses, this is certainly true. Although they use modern machine learning classification algorithms to compare against baseline, they still seem subject to the same problems as the simpler statistical tests used in polygraphs.

Above, I suggested taking a deep breath, doing simple arithmetic or telling a lie in your head during the baseline questions. Taking a deep breath increases the BOLD response measured by fMRI throughout your brain. The idea behind doing arithmetic or telling a lie in your head is to engage the brain areas involved in internal mental conflict detection (between areas of the brain that are pulling in different directions), executive control (over the rest of your brain), and working memory whose activity changes when lying. As far as I know, all of the studies on lie detection seem to use naive participants, and no one has yet tested the efficacy of these counter-measures.

I have also assumed that the analysis would be run 'within subject'. In other words, the machine learning classifier algorithms would be making a comparison between baseline and important questions for the *same person*. However, there have been attempts to train the algorithms on a corpus of data from multiple participants beforehand, and then applied to a new brain. This approach is considerably and inherently less accurate (less than 90% as opposed to nearly 100%) since everyone's brain is different, and since brain activity will probably vary for different kinds of lies. Indeed, there appears to be variability in the areas that have been identified by different experiments.

There are alternative experimental paradigms to the basic questioning approach described here. For instance, one might show someone the scene of a crime, and look to see whether their brain registers familiarity. I haven't looked into this approach. But fundamentally, this familiarity assessment is much more limited in the kinds of questions that can be asked, and furthermore, you only get one chance to assess someone's familiarity (after which the stimulus is, by definition, familiar). That single response simply might not be enough data to go on.

All of the studies so far have employed 'willing' participants. In other words, the participants kept their heads still, told the truth when they were asked to, and lied when they were asked to. An uncooperative participant might move around more (blurring the image), show generally elevated levels of arousal that could skew their data, be in worse mental or physical condition, and come from a different population than the predominantly white, young, relaxed, intelligent and willing undergraduate participants. We don't know how these factors change things, and it's difficult to see how we might collect reliable experimental data to better understand them.

I haven't considered alternative imaging methodologies here (such as EEG or infrared imaging). Mostly though, fMRI appears to be leading the field in terms of accuracy and effort spent, and all of these arguments should apply to EEG and other methods equally.

Why am I writing this?

There are a number of fMRI-based lie detection startups attracting government funding and attempting to charge for their services. I don't begrudge them their entrepreneurial ambition, but I am dismayed by their hyperbolic avowals of success.

In truth, this is a new, mostly unproven technology that seems to work fairly well in laboratory conditions. But it's subject to the same sensitivity/specificity tradeoffs that plagues medical tests and traditional lie detection technologies. The allure of an ostensibly direct window into the mind with the shiny veneer of scientific infallibility is a beguiling combination.

Eventually, the limitations of this technology will be realized. I'd prefer to see this techno-myth punctured and caution exercised now, rather than after costly mistakes have been made. Cheeringly, the courts appear to take the same view (at least so far).

My credentials

I'm finishing my PhD in the psychology and neuroscience of human forgetting at Princeton. I've worked on the application of machine learning methods to fMRI for the last few years, was part of the prize-winning team in the Pittsburgh fMRI mind-reading competition, and lead the development of a popular software toolbox for applying these algorithms for scientific analysis. However, I have no expertise in the neuroscience of cognitive control, lie detection or law.

So I apologize if I'm wrong or out of date anywhere here. If so, I'd be glad to see this pointed out and to amend things.

The pain of forgetting

2010-01-03T22:31:00.002-05:00

There's an old adage that psychologists study their own deficiencies: I study the psychology of memory.

At times in the past, I was able to discourse fluidly on obscuranta ranging from the internals of software for text processing, Wildean views of suffering, or conspiracy theories behind the construction of the Egyptian pyramids... few wanted to listen, but at least I found my own thoughts interesting.

Painfully, painfully, I feel this rich arcana slowly seeping away. In my personal Hades, I would be doomed to fill a sieve with grains of sand by day, even as they pooled into a puddle of forgetting around my feet by night.

My frustration hardened into a hypergraphic compulsion to externalize everything I learned. I wrote notes on every article, lecture and conversation. I painstakingly heaped my newfound nuggets of knowledge into a gigantic paper haystack.

But then I had a new problem - I couldn't find any of it. Like a millionaire without the numbers to his Swiss bank account, I was rich and poor at the same time.

Things came to a head when I desperately tried to assimilate a pillar of textbooks for my neuroscience qualifying exams. Every brain area goes by a dozen names, can be organized by location or by lesion, by experiment or experimenter, by projections or inputs, by effect or atrophy, or equally along a dozen other dimensions.

But in my paper prison, each piece of information was confined to a single cell - a place for everything and everything in its place. In order to allow the informational inmates to run free, I needed a way to allow any nugget of knowledge to abide simultaneously in a multitude of homes.

And so I sought to build software to help me. After many musings in the shower, I constructed an elaborate infrastructure that incorporated: dynamically-generated hyperlinks to highlight associations; transclusion to include the same text in multiple places; tags to break down the trammels of tree-based hierarchies; and aliases to allow for multiple names.

Things are better now. I feel mnemonically empowered, or at least less mnemasculated. By granting conjugal visits from my conscious to my unconscious mind, this index-on-steroids means I can find things more easily.

But maintaining this index requires effort whenever I add a new piece. And my memory mansion grows so fast that even if I slept in a different room every night, I'd never return to the same one twice. I simply don't remember what's in there to look for it.

So in truth, even this sophisticated system is just a crude ropes-and-pulleys facsimile of my mind. A fixed hyperlink lacks all of the deep isomorphism, insight and spontaneity of an analogy. The ideas trapped there are dead and inert - they don't bump and bite and spark off one another like active, bustling, living thoughts. And the effort of exhuming them by typing laboriously into a laptop lacks all the rapid, happy spontaneity of immediate recollection.

I dream of distant-future neural prosthetics, a google gland hooked up to my hippocampus. But I am too impatient to wait.

This is the quest that led me to co-found Memrise. I have gone as far as I can efficiently *externalizing* my thoughts. Memrise's mission is to improve *internalization* - learning faster, forgetting slower.

I know that there is no silver bullet that will fix my memory. But I'm compelled to continue looking for tools and techniques that can boost it and shore it up.

Memory, the persistent effect of experience, provides the tools with which we think. We are the sum of our memories. When we forget, we erode.

Losing weight

2009-11-25T22:26:00.000-05:00

Whenever I want to lose weight, all I have to do is eschew solid food, shave my head and poop bulimically. I call it the neonate diet.

Our one true inalienable right

2009-10-21T19:29:00.001-05:00

In the future, our refusal to endorse assisted dying will be considered barbarism.

We as a society need to appreciate that a death of our own choosing, at our own time and on our own terms, may be the most noble way to end one's life.

One day in the future, I look forward to the publication of a beautiful book of joyous and life-affirming suicide notes.

I've felt strongly about this for a long time - I wrote this short story about suicide as an inalienable right ten years ago.

Why is it that chocolate and whisky help when I want a cigarette?

2009-10-20T22:38:00.000-05:00

What does it mean that I can sometimes quench the belly-fire-for-fire with chocolate or whisky? Is this some kind of interchangeable appetite for the appetitive? I just want something bad for me? Or that these are all different sources of endorphins, and OD'ing on one source of pleasures drowns out the baleful vacuum of an alternative, absent pleasure?

Word pools for psychology experiments

2009-10-19T13:08:00.003-05:00

Ah, you came to the right place for words, sonny boy. Roll up, roll up. We gots big words, little words, sad words, rude words, words you didn't know, words that haven't been invented yet, words your mother whispered to you in your sleep as a boy, words that cure warts, words that raze forts. A penny a word, first dollar is free!

Ahem.

Per collated a bunch of word pools at the hallowed Lab of Kahana, but it's 3 days' journey from here on foot. Unless you're a hyper-traveller, in which case this wordhole will whisk you there:

http://memory.psych.upenn.edu/WordPools

I based my experiments on the Toronto Noun Pool, since that has a 1000-odd imageable nouns which I'd then whittle down a dozen different ways by hand.

The MRC Psycholinguistic Database is pretty awesome if you want to filter automatically (e.g. frequency, imageability, length).

I don't believe in luck

2009-10-13T21:43:00.006-05:00

I once told someone glibly that I didn't believe in luck. That triggered a lengthy discussion about being hit by lightning and other instances of unpredictable mishap or serendipity - aren't all of these luck?

I'm closer now to articulating what I meant. I don't believe in systematic chance. Or to put it another way, some people are luckier than others, but this is a function of attitude rather than fortuity.

My dad likes to remind me that 'the harder I work, the luckier I get' [1,2]. There's some truth to this - but Lady Luck wants to be seduced not seed-sown; pan-handled not strip-mined.

Richard Wiseman is the standard-bearer of the scientific study of luck, and he writes beautifully on the four characteristics he has identified in lucky people - they:

are skilled at creating and noticing chance opportunities
make lucky decisions by listening to their intuition
create self-fulfilling prophesies via positive expectations
adopt a resilient attitude that transforms bad luck into good

Each of these principles fits within existing psychological research, but there's something satisfying about seeing them re-inferred and unified in one place through years of questionnaires and experimental studies on the hapful and the hapless.

For fun, substitute 'lucky' for 'entrepreneur' when reading the above 4 points.

On patents as a business tool

2009-10-12T23:58:00.006-05:00

This was posted as a comment here:

After doing some early research into patents, I concluded that:

they'd cost me $10k or so per patent
this would be a huge amount of time and effort for me
it would take at least a couple of years for the patents to be granted
I wouldn't have the money to enforce them
small companies would probably ignore them anyway

So I was confident in asserting that patents were a waste of time (for my needs).

In the 6 months since, a few things have conspired to change my mind:

we're close to raising investment, and investors care about patents because they provide evidence of value
likewise, they make you more acquirable
I've started to realize that patents' primary value is as a deterrent against large companies who might otherwise lumber into competition

It's this final point - that patents provide a watered-down 'mutually assured destruction' kind of deterrent against incumbents and larger companies that seems most important now. If you can protect yourself with patents, you make it much more likely that larger companies will partner with, license from or acquire you, rather than compete with you.

Self Control through software

2009-10-11T20:41:00.008-05:00

Leo Efstathiou asked me recently whether I'd rather be smarter, or have more willpower. It took only a moment's thought to realize that I'd rather have the self-control any time.

And so it was with a sense of wonder and optimism that I normally reserve for sunrises that I fired up Self Control: a Mac application that completely blacklists parts of the Internet. Like a gaoler with a blackjack, Self Control coshes any attempt to blunder down rabbit holes like Facebook or email for some time period you specify. It's absolutely and delightfully watertight.

The beauty of this is its potential long-term effect. I want to counteract the variable reinforcement schedule that email and blogs provide - with Self Control's help, I'm hoping to ensure zero reward from them for long enough to break the self-perpetuating cycle of reflexive refresh-pressing.

Google, here's some context

2009-10-01T00:30:00.001-05:00

So often, when I'm searching on Google, I want to give it some context. For instance, I'm looking for pages about Django (the Python-based web framework). They'll probably mention 'python', 'web', 'database' and 'programming'. I could feed in this query:

Django python web database programming

But I'd actually get a very restricted set of results back - just those that include *all* of those terms (give or take some Googley cleverness).

So I could instead do this:

Django python OR web OR database OR programming

This would certainly be progress. As long as the page mentioned one of those terms, I'd be golden. But there are two issues with this:

I want to be able to stipulate that it doesn't need to mention those exact terms, but rather stuff related to them.
I don't think the page would get bonus points for mentioning more than one of those terms, as it should

It's frustrating that I can't type something like:

Django context:(python web database programming)

Google Sets (or something sophisticated akin) should be able to fill in the rest.

Sssssssshhhhhhhhhhh... for now

2009-10-01T00:23:00.001-05:00

I suppose I must really care about mobile phones ringing, since this is the third piece I've written about it. Maybe it's just that I really care about auditory pollution. Or that it seems like a problem that affects billions of people and hasn't been given enough thought.

I often want to silence my phone for an hour or so, while in a meeting or class. However, I know that I'll forget to turn the ringer back on afterwards. This is a failure of prospective memory ('remembering to remember'), and it's something I feel I have almost no control over.

Wouldn't it be great if one could set one's phone to be silent for an hour, safe in the knowledge that soon after the meeting ended, you'd be back in business? Isn't this what we always want?

Mobile phones should be felt but not heard

2009-10-01T00:19:00.002-05:00

I can never hear or feel my phone when it rings, no matter how loudly or how insistently hornet-like the trilling and shrilling and buzzing and fuzzing.

For this reason, I got very excited when I heard that they now make a Bluetooth bracelet that buzzes when your phone rings.

Better still though, I'd like a sticky Bluetooth (Gluetooth?) doodad that you could affix to a watchstrap or a belt or a ring that stayed charged by dynamo from the kinetic energy of my movements - that would be much less obtrusive.

Emacs backup files

2009-10-01T00:11:00.001-05:00

Emacs loves to create billions of little ugly-twin backup files all over your hard disk that look like 'myfile~' and 'myfile.txt~'. Here are some better alternatives:

- add the following to your .emacs (this is the best solution)
(setq make-backup-files t)
(setq version-control t)
(setq backup-directory-alist
(quote ((".*" . "~/backup/emacs_autosave/"))))
; otherwise it keeps asking
(setq kept-new-versions 30)
(setq delete-old-versions t)

[i've forgotten which of these does what, but they're all in my .emacs file...]

make sure to create ~/backup/emacs_autosave first. this will create multiple snapshots of every file you edit and store them in that directory. this avoids having all the blah~ files in your current directory, and is useful if you want to revert to the way you had things 5 minutes ago.

- a quick alias to remove them (courtesy of Randy O'Reilly)
alias cleanup '/bin/rm *~ .*~ #* .#*'

- finally, put Dropbox into 'pack rat' mode, so that it stores every single version of your Dropbox files for eternity.

Alleviating tinnitus, and the shape of the auditory phenomenological landscape

2009-09-08T00:28:00.009-05:00

Tinnitus is a chronic condition where you hear a ringing in your ears - for acute sufferers it can be very loud and never stops. This is tortuously unpleasant.

I wonder if this has been tried - could you alleviate the symptoms of tinnitus by playing in a kind of psychologically out-of-phase sound, to cancel the ringing sound experienced? Of course, the normal physics of waves and phases won't hold true here, since the perceived sound isn't 'real' (i.e. external, based on moving currents of air).

However, I wonder if there might be psychological rules about sounds where some external sound of the right characteristics might cause a kind of neural interference, and disrupt the perception of the tinnitus sound.

One could imagine having a tinnitus subject navigate with gradient descent through a space of auditory parameters, rating the subjective intensity of the tinnitus sounds while listening to different external sounds. Eventually, you might find a point in the auditory parameter landscape where the tinnitus wasn't too annoying. With enough participants, you might learn something interesting about the shape of that landscape, and about the phenomenology (and neural representations) of audition.

In principle, I suppose, one could do this with non-tinnitus sufferers, but I'm assuming that the tinnitus sounds are constant and so would provide a fixed point of comparison.

UPDATE: hah! It looks like someone's trying to do something a little akin, though it uses a more physiological than phenomenological mechanism. I wonder what made them pick a low hum? See Teen inventors fight tinnitus

Open source the Drobo data format

2009-09-07T23:54:00.004-05:00

The Drobo is an amazing device - it allows you to pop in a handful of old hard disks, and it effectively pools them so that they show up as a single drive to your OS. It even distributes data redundantly across them to give you peace of mind with old disks. And it's hot-swappable.

When our lab was seeking an archival solution, this sounded perfect. But unfortunately, the Data Robotics people are trying to solve a hard problem, and there are a good number of unhappy people on the internet complaining about losing swathes of data. The really unfortunate part of this is that all the data on the hard disks that you add to the Drobo unit are stored in some kind of proprietary format that presumably facilitates the distributive algorithm at the center of Drobo's cleverness. As a result, the only way to read the data on those Drobo'd disks is with a Drobo. So if things get hosed, then there's no recourse but to send it off to them.

So here's my proposal. Open source the Drobo data format. Keep the hardware and the distributive algorithm proprietary. But make it very easy for other people to build apps that talk to Drobo boxes.

Maybe a cottage industry of hardware repair shops and that specialize in Drobo maintenance and repair will spring up. All to the good. Data Robotics' expertise and value lies in building products, not providing services. These repair shops customers with reassuring alternatives, become evangelists for the product, and might even add value by building good third-party add-ons.

Likewise, encourage a developer ecosystem. Someone might even come up with their own distributive algorithm that's better than Data Robotics'.

Paradoxically but importantly, by reducing the barrier to exit by making it easier for people to get their data off the Drobo, you actually reduce the barrier to entry.

The value of the Drobo is in the hardware, and maybe in the distributive algorithm, but not in the data format.

Mobile phone ringing

2009-09-07T22:39:00.003-05:00

Everyone hates it when a mobile phone rings in a cinema, classroom or restaurant. Especially if it's yours.

It's so easy to forget to mute the ringer. It's even easier to forget to turn the ringer back on at the end of the lecture. Turning ringers on and off seems beneath us, and beyond us.

Clearly, it would be better if our phone could decide when to ring for itself. This kind of 'context-awareness' is actually a very hard problem. Here's one simple algorithm that might go a long way towards helping.

If my phone can pick up lots of other mobile phones in close proximity, and they're not moving away, then assume it should be more silent. This covers most of the cases we'd want, where lots of people are sitting together, and no one wants to be disturbed. It excludes cases where we're walking down the street surrounded by lots of other people, but none of us are sticking around.

I can think of a few cases where this might fall down. If I'm anxiously awaiting a call from the hospital about a loved one, I want the phone to ring wherever I am. If I'm sitting in a noisy coffee shop, I want it to ring loudly, and no one will be particularly disturbed.

But on balance, this seems like a good heuristic. Instead of having a manual 'mute' button, we might just let the phone guess, and have a manual 'loud override' button for the above cases.

The problem is that often, we'd rather our devices be dumb but predictable than smart but surprisingly and unpredictably tricksy.

iThimble

2009-09-07T22:36:00.001-05:00

I have stubby, knobbly hands that look neanderthal to me as I prod sweatily at the smooth, cleanness of my iPhone.

Wouldn't it be cool if there was a thimble you could place on your thumbs/index fingers that would give you a restricted surface area with which to press on small buttons on your iPhone?

Then, I could genuinely use the device as a 21st century post-it note - I wish I could scratch and scribble on it with a horsehair finger-biro, rather than doodling and poodling all over it with my pudgy digits.

Assortative mixing for peer review

2009-09-07T22:15:00.003-05:00

[this is an idea proposed by Adrian de Froment]

Let's say you are a great scientific reviewer. You respond in a timely fashion, provide detailed and insightful comments, and your judgments about which papers should be published tend to match the judgments of other reviewers and of journal editors.

Being a good reviewer is like wetting yourself in a dark suit - you get a warm feeling, but no one ever notices. When it comes to having your own papers reviewed, your good karma is worth nothing. You're just as likely to be assigned a slow, careless reviewer as everyone else, despite all your contributions to the community.

In evolutionary biology and network theory, the term 'assortative mixing' refers to a bias in favor of connections between network nodes with similar characteristics. In other words, good reviewers should be reviewed by other good reviewers. We should build karma directly into the system.

Note that I'm not in any way suggesting that papers written by good reviewers should be more likely to be published. But perhaps they should be more likely to be well-reviewed - that is, rapidly, fairly and carefully. Likewise, if you perenially submit your reviews late, you would be more likely to be reviewed by someone similarly tardy. What better cure for your anti-social behavior than a dose of your own medicine?

I've used speed of response here as the dimension that determines who gets assigned whom as a reviewer, but the objective function determining what constitutes a 'good' reviewer could be anything.

If this assortative mixing of reviewers was adopted, it would reward good reviewers, punish bad reviewers, and almost certainly lead to an overall improvement in the standard of reviewing.

Picking scientific reviewers

2009-09-07T21:57:00.003-05:00

It's hard to find good reviewers for scientific papers. Because it's all anonymized (though that may be slowly changing), there's no easy way to tell who's a good reviewer and who's a bad reviewer. It's easier to define what makes for a bad than a good reviewer:

tardy in responding with their comments [not relevant for the points I make below]
don't read the paper carefully enough, and make fatuous criticisms
don't read the paper carefully enough, and miss gigantic flaws
misjudge the import of a paper, and reject it for being less interesting/novel than it actually is

To some degree, these are unchanging realities and limitations of human nature. But we all respond and improve with feedback. Wouldn't it be great if every reviewer had a public scorecard, summarizing their efficacy as a reviewer? (ideally aggregated across all the journals for which they've reviewed)

We can imagine various objective functions for what makes a good reviewer.

We could bin all a reviewer's reviews into a 2x2 boolean matrix: your review recommendation x whether the paper was published. If we adopt the parlance of signal detection theory, a good reviewer will have many 'hits', where they recommended publication, and the paper was indeed published, and many 'misses', where they recommended against publication, and the paper was indeed rejected. In other words, a good reviewer's recommendations will be predictive of whether a paper went on to be published in that journal or not. A bad reviewer's recommendations will often be at odds with the eventual fate of the paper.

We could swap in a variety of other dependent variables, besides just the boolean 'published in this journal or not', such as:

how many times the paper was cited
correlation of reviewer's numerical ratings of goodness with other reviewers
how many rounds of revisions were necessary for the paper to be published

Of course, there are problems and limitations to this approach. For instance, it says nothing of the usefulness of the reviewer's comments. And just because a paper eventually got published or was cited many times doesn't necessarily mean that it's genuinely good.

But on the plus side, it provides a measurement that reviewers can try to improve, which if optimized would be broadly good for the system. It's hard to game. It would provide journal editors with more information when deciding which reviewers to pay attention to. And if it were made public, it would provide a way to incentivize and recognize good reviewing.

Did you know that the word 'gullible' isn't in the dictionary?

2009-09-07T21:56:00.003-05:00

When I get rich enough that I can build products without caring whether the market size is 1, I'd like to print a dictionary missing the word 'gullible'.

Spacetime alarms

2009-09-07T21:54:00.005-05:00

Alarm clocks are temporal. They tell you when some time criterion has been reached. They're very useful.

But often, I really want an alarm with a spatial criterion - a location alarm. Let's consider some possible uses:

Beep shrilly if anyone tries to steal this device from its current location. [I think there are accelerometer-based programs for laptops that do this]

I'm snoozing on the train - wake me up when we get near Penn Station. [This is where I first came up with the idea].

Give me a kick if I'm still in my office when I'm supposed to have left for that meeting. After all, I don't need the alarm to go off if I'm already on my way to the meeting. [though that's a combination space + time criterion]

This would make for an obvious and delightful iPhone app. I haven't found one yet, but I haven't looked hard either.

UPDATE: there are some really superb suggestions in the comments, and in Hacker News that take this idea much further. I particularly liked these:

R.J.: Google Maps needs this so when I'm walking down the street I don't have to pay attention to street address numbers/keep my eyes glued to the screen.
Frankus: Maybe something like that could tell me when I'm close to the grocery store that I need to buy milk.
Frankus: A game where you try and assassinate your friends by setting imaginary time bombs to go off at a particular location when you think your friend will be there.

The wallet washkit for the man about town

2009-09-07T21:11:00.000-05:00

Have you ever charged out into the night like an angry banshee in search of entertainment, only to have your forwards momentum sapped from you by the bulkiness of your encumbrances? Then look no further than the Wallet Washkit For The Man About Town, available only at ThinkChic.com.

No thicker than an ordinary wallet, but packed with:

micro-toothbrush and paste, designed by NASA to function in zero-G, and employed by astronauts on Apollo
chewing gum that would make even Willy Wonka giggle with delight at its olfactory potency
oozing gel that turns your wallet into a roll-on deodorant
aphoristic reading material that fits on credit-card-sized pages.

Dice Man iPhone app

2009-09-07T19:49:00.001-05:00

Why isn't there a Dice Man iPhone app?

1) Set alarm clock for 5am.

2) Plot Google Maps route to nearest bar.

3) Call ex-girlfriend.

4) Email pornographic images to parents.

5) Download I Am Rich.

6) Throw iPhone into nearest river.

Emacs Freex gets a new home

2009-03-09T04:11:00.006-05:00

Emacs Freex mode is a minor mode for organizing and editing a massively-hyperlinked database of your notes and ideas. It's a personal wiki on steroids. Per Sederberg & I released the Emacs Freex code under the GPL two years ago.

I've just moved it to a new home on Google Code, and also recorded my first ever screencast, to demonstrate how it works.

There are probably still some teething problems with the installation instructions, and I think perhaps I could be clearer in the screencast. Do let me know what I can do to make it easier for new users.

If I were to write a novel

2009-01-17T14:25:00.004-05:00

How much time gets spent staring in a placidly bovine fashion at a concrete wall while waiting for subway trains every day? If we consider London, there are roughly a million people travelling a day, each waiting for maybe five minutes - that's 100,000 man hours a day. While chewing the cud in this manner, one finds oneself wearily reading and re-reading the same inane advertising drivel.

If I were to write a novel, I would spend all my meagre advertising budget on renting out all the billboards for a single platform for a day. I would display the first chapter of my book piecewise, starting at one end of the platform, so that people could begin reading, progressing crabwise in lockstep from billboard to billboard, hungrily seeking the next installment. At the very end of the platform, I'd hand out free copies of my book.

Adam Detre's recommendations on music in late 2008

2008-12-09T23:25:00.002-05:00

[this is quoted verbatim from an email from my brother]

sorry for the atrocious sepelling. i wrote this in the dark with a slightly broken keyboard...

Here are some of my favourite bands that i have listened to am listning to alot. Mainly in the last two months. Except Postal Service.

Try some or all.

I would love to listen to musics with you sometime.

Slow club-
http://www.myspace.com/slowclub
pretty. i liek them AT THE MOMENT. found them on the weekend.

Johnny Flynn
First person to open my eyes to folk. Love his voice.

Brendan Campbell.
Dont know much about him but he's kinda like folk meets acoustic reggae - only got a few of his songs but i lve them and i am too scared to buy more incase they arent as good

Fleet Foxes
Harkness almost to PET SOUNDS - Beachboys. Really pretty laid bak musics ) i even bought the CD cos te album rtwork is great

Listen to SWAY, he a rapper dude but SAY it TWICE is one of my favourite moern hip hop tracks.

LAURA MARLING has SUCH pretty voice. So easy to melt into your thoughts while listening to her. my favourite is New Romantic

NOWon a slightly different note

I ADORE KINGS OF LEON and their last two albyus especially - Bause of the times is one of my favouriets.

And Cajun Dance Party

Crystal Castles
Super funky electro - Lots of lovely Blip Blop noises

POSTAL SERVICE one of my favourite bands of last year (GIVE UP is thieir only alub i thitknk but some of the remixes of their songs are great

NEUTRAL MILK HOTEL introduced to m by BERGE. really good (semi-old apparentl clt following) indie band

LUDIVICO ENAUDI is BEEEUATIFUL modern classical musics. buy ANYTHING look at myspace of his.

MIA Sri Lanka rude gal elctro hip hop-sih in parts

SNTOGOLD similar genre to MIA

KANYE's new album isnt bad from the songs i heard... dont hate me for saying that

ADELE's album - 19 is BRILLIANT one of the nicest surpriswes of this year for me.Saw her live she was goooo

MGMT - first got turned onto the becuase radiohead were talkingabout them. They are nothing like radiohead. One of the best bands of this year again. Very happy-hippy-electro. NICE.

We only use 10% of our brains?

2008-11-29T18:44:00.005-05:00

The myth that we only use 10% of our brains is sticky and gets everywhere, much like glue-dipped belly button fluff. And just like glue-dipped belly button fluff, it's a nuisance, and can only be combatted with the even-stickier duct tape of truth. Rather than attempting to be exhaustive, I'll simply appeal to your intuitions to try and make sense of how it's nonsense.

The urban legend that we only use 10% of our brains is true in the same way it's true that we only use 10% of a piano. Have you ever seen even a concert pianist press more than ten keys at once? It wouldn't be too hard - one could probably manage 20 or 30 by adding elbows, and maybe even more with props. But it would sound terrible - all of the informative signal that goes into making music rather than noise is in the choice and timing of the keys that get pressed. It's the pattern of keypresses that matters.

The same is true of the brain. It may be that only (say) 10% of neurons are firing at a given moment, but that pattern of firing is what matters - the choice and timing of which neurons are active is what constitutes thought.

It is worth noting that, just like the piano, there are bits of your brain that are active more often than others - you can probably get rid of the very lowest and very highest notes without too much of a problem, though occasionally things might sound a little odd. The same is true for people and animals - you can lose a few thousand neurons heading a football or downing shots and no one will notice. But if you lose a big chunk of brain, or multiple keys in a row, then the music is going to sound pretty bad.

PyEPL blog

2008-11-03T22:32:00.000-05:00

PyEPL (the Python Experiment-Programming Library) is a versatile, high-level library for coding psychology experiments in Python. If you run human experiments and like coding, then I'd recommend it strongly.

PyEPL's adoption and utility would both receive a tremendous boost if there were a wider array of snippets and tutorials floating around, so I created a collaborative PyEPL blog to provide an unofficial home for such tutorials, snippets and discussion. See the announcement post for more info.

N.B. I wouldn't recommend PyEPL in its current form for low-level vision experiments, and it only runs on Mac and Linux. Both these concerns are slowly being addressed though, so things may have changed since the time of writing.

WriteRoom? DarkRoom? MushRoom!

2008-05-04T23:30:00.004-05:00

Mark Pilgrim said it best:

I’ve been peripherally following the latest fad of full-screen “writing-focused” text editors. Here’s what I’ve learned so far: in the beginning, there was WriteRoom (Mac OS X, $24.95). WriteRoom is “just about you and your text.” WriteRoom begat DarkRoom (Windows + .NET, $0), which is also “just about you and your text” but requires a 22 MB runtime environment. DarkRoom begat JDarkRoom (Java, $0), which is just about you, your text, and somebody else’s multi-megabyte runtime environment.

Here’s the basic problem: you’re writing a text editor. Stop doing that. It’s 2007. Saying to yourself “I’m gonna build my own text editor” is as silly as saying “I’m gonna build my own build system” or “I’m gonna build my own amusement park.”

Reading the change logs of these programs is like traveling back in time. Way back. Latest changes in JDarkRoom 8: Undo / Redo. Seriously. Version 8, and they now support undo. No offense, but what the fuck?

Anyway, so I had a nose around, and it turns out that Emacs has both undo/redo and very capable full screen capabilities. The EmacsWiki includes the following function, which I've renamed from 'write-room' and tweaked a little for the mac:

;; http://www.emacswiki.org/cgi-bin/wiki/WriteRoom
(defun mush-room ()
"Make a frame without any bling"
(interactive)
(let* ((frame (make-frame '(;; (minibuffer . nil)
(vertical-scroll-bars . nil)
(left-fringe . 235); big fringe to center
(right-fringe . 160)
(background-mode . dark)
(background-color . "black")
(foreground-color . "green")
(cursor-color . "green")
(border-width . 0)
(border-color . "black"); should be unnecessary
(internal-border-width . 20) ; whitespace
(cursor-type . box)
(menu-bar-lines . 0)
(tool-bar-lines . 0)
(mode-line-format . nil); dream on... has no effect
(fullscreen . t); does not work on all systems
(unsplittable . t)))))
(select-frame frame)
(setq mode-line-format nil); is buffer local unfortunately
(set-face-background 'fringe "black")
(set-frame-font gjd-gin-font)
(mac-toggle-max-window)))

I've also added the following auxiliary bits and bobs to make this work well:

(setf gjd-gin-font "-apple-courier ce-medium-r-normal--18-180-72-72-m-180-mac-centraleurroman")

(defun unfullscreen ()
(interactive)
;; restores mode line
(setq mode-line-format (default-value 'mode-line-format))
(set-face-background 'fringe "grey95")
)

(defun delete-frame-and-unfullscreen ()
(interactive)
(delete-frame)
(unfullscreen))

(global-set-key "\M-w" 'delete-frame-and-unfullscreen)

Switching from linux to mac

2008-05-04T23:12:00.005-05:00

So, I caved. I crumbled like a biscuit in a blender. I am now the smug owner of a MacBook Air, and it really is wonderful. I went 10 days without rebooting before needing a firmware upgrade, and both sleep and wireless work flawlessly.

Here are the bells and whistles that make my brain and fingers happy.

Spaces (OS X Leopard's virtual desktops doodad) isn't as good as KDE's, but it turns out to be fine once you turn off the auto-swoosh by typing these two commands into a Terminal:

defaults write com.apple.dock workspaces-auto-swoosh -bool NO

killall Dock

Before, switching to an application (such as Firefox) that didn't have a window open already on your current desktop would cause you to be whisked randomly to the first desktop which did have a Firefox/Terminal window open. Now, you can switch to Firefox, hit Cmd-N and a window pops up, ready to go with minimal context disruption.

MondoMouse makes it easy to move and resize windows. By default, moving a window involves awkward trips up to the title bar. The tiny resizing handle is even more awkward, and more awkwardly situated, and is often overpowered by an overeager Dock. Now, I just have to hold down Cmd and move the mouse to move the active window. Holding down Cmd and Alt resizes. Again, this doesn't work quite as nicely as KDE - for instance, the windows don't snap satisfyingly into place when they get close to other windows or the edge of the screen. MondoMouse is available for a trial period, after which it costs $15.

Various other niceties deserve a mention. I currently prefer the Carbon Emacs over Aquamacs, though there's not much to separate the two. Quicksilver, Expose and hot corners are great. For Dashboard widgets, I'd recommend iStat Pro, Word of the Day and maybe Album Art. Adding cdto to Finder is handy. Skim is the way forward for reading PDFs in full screen. Alarm Clock for timers. Cyberduck for accessing other computers over SSH (along with MacFuse and SSHFS).

Some things are just broken on the mac, and will probably never be fixed. Steve Yegge has described his heroic failure to civilize the mac's shiny silvery savagery with proper focus-follows-mouse behavior. In general, OS X's switching behavior is wrong in a few respects. The application (rather than the window) is the wrong level at which to switch, and so Cmd-Tab is always jarringly confusing to use. There are still kinks with the way that windows activate and bring themselves to the fore, especially when multiple desktops are involved, and I seem to lose a modal dialog to this problem about once every few days. MacPorts and Fink are workable but still disappointing - having two package systems devalues both, downloading binaries is preferable to compiling every time, and the packages are generally patchier than Debian's apt-get. Finally, X11 is very clearly a second-class citizen, but works tolerably well. Oh, and unbelievably, the regular expressions engine in Python 2.5 appears to be maddeningly sprained!

There were a few pleasant surprises. Two-finger two-dimensional scrolling is unutterably wonderful. The MacBook Air's colossal touchpad is very pleasant to use, even for nipple-lovers. OS X applications are pretty consistent about keyboard shortcuts. You can re-learn most muscle memories without too much effort so long as you don't have to switch back and forth between the old and the new too much in the first few weeks. Emacs is still Emacs.

Losing an old friend

2008-03-25T22:39:00.001-05:00

My trusty, indomitable Thinkpad T40 is showing telltale signs of domitability. I will feel its loss like the loss of an arm I'm using to cling to a mountain. Unfortunately, my Thinkpad crashes about once a day - hangs rigidly from a carefully-tied noose, requiring a hard reboot. I could send it off again to Lenovo, but it felt like a hasty hemispherectomy with a blunt knife the last time I needed to do that.

In pondering the purchase of my next outboard brain, I'm torn between the Lenovo X300 and the MacBook Air. If I refrain from superfluous eating or personal grooming, I think I can stretch my budget to $2200, which has to include warranty, spare power cable, external DVD drive etc. [Right now, the Lenovo X300 only exists in a flash drive model for $3000, but let's assume that if I pray and masturbate hard enough, they'll introduce a cheaper model with a standard hard disk].

Let's start with the MacBook Air pros. It's more beautiful than a prone supermodel, and almost as slim and weightless. Mac OS X probably works more reliably than anything else on the market, and with Parallels/Boot Camp, I should be pretty much covered for most eventualities. Keynote - nuff said. I don't need a DVD drive more than once every few months, and I can live reluctantly with the ports it offers, plus some dongles.

On the con side, I don't want to reward Steve Jobs' deviously effective strategy of locking Apple consumers into an all-Apple world. I hate barriers to exit in consumer products. I think open solutions foster greater innovation in the long run. And I resent proprietary software like I resent being told what (not) to do.

There's little to argue in favor of the Thinkpad specifically. I like its distinctiveness, its nipple mouse, and the fact that it has two mouse buttons and pageup/down keys.

Furthermore, I really like linux - I like that things are free in every sense. I love Debian-based package management - being able to automatically install and update everything with a single command or click is so weirdly, futuristically better than the Mac or Windows approach of downloading each application manually, each of which has its own update software - so much better that non-linux users literally don't seem to believe what they're missing. Finally, I love KDE. I like the fact that I have keyboard shortcuts for everything - I wear it like I wear my 5-year old walking boots that have moulded perfectly to my feet. I especially like focus-follows-mouse and being able to effortlessly move and resize windows by holding down Alt. These alone are very nearly enough to tie me to linux indefinitely.

But linux lets me down. My wireless is better than before, but it's still a little drunk and a little deaf. I lose minutes a day to niggles and imperfections (e.g. USB, sleep and projectors). I've never quite had 3D video stuff working, so I have to jump through fiery hoops to run my experiment presentation software (PyEPL). YouTube and video are pretty hit and miss. And, even after using Open Office for 2 years, I still don't like it as much as Microsoft Office.

So I'm left trading off my preference for linux's user interface, package management and freeness against beautiful hardware and reliability. Perhaps I'm getting old and impatient, but the reliability is really the kicker here.

Googling for 'boobies'

2008-03-17T14:28:00.000-05:00

You're drunk, you're horny and you're wandering around Atlantic City at 3am. It's time to put your money to even better use than gambling - clearly, you need a strip joint, stat.

Given your urgency, and your boozy clumsiness, you don't want to have to go far, but being a discerning consumer, you're willing to go a little bit if it's going to really improve things. You need a guide that's right at hand, quick to consult, and knows what they're talking about. Ideally, you want a randy and helpful local, but an iPhone app with very big buttons would be next up. If it had reviews from other discerning consumers, that would be an excellent way to decide between the various possibilities.

If I were to ever write such an app, I would be torn between calling it The Very Lonely Planet, Vagat's or The Muff Guide.

Email is the mind-killer

2008-03-11T05:10:00.000-05:00

I must not email.
Email is the mind-killer.
Email is the little-death that brings total obliteration.
I will face my email.
I will permit it to pass over me and through me.
And when it has gone past I will turn the inner eye to see its path.
Where the email has gone there will be nothing.
Only I will remain.

Running a psychology experiment

2008-02-16T17:25:00.008-05:00

Designing an elegant psychology experiment is a fiendish business. Even after you've done that, running and implementing it correctly requires considerable attention to detail. I've attempted here to catalogue all the checks I (should) run before collecting data in a brand new experiment. Where possible, I've tried to think of extra points to worry about when collecting fMRI data.

The most important advice I can give is to run yourself in your own experiment at least a couple of times before you run anyone else. This is a pain - running in the same experiment over and over again is tiring, but it's less of a pain than collecting 20 subjects' data only to find that the data are worthless.

N.B. Make sure you run these tests on the testing computer that you're going to use to actually collect the data - each machine is a complex ecosystem, and you can't generalize success from one to another.

Obvious bugs

Firstly, running yourself will help you spot relatively obvious bugs, such as:

- Stimuli that should only show up once occurring more than once

- Stimuli that should be randomized appearing in alphabetical or the same order each time

- Bugs that only occur when you actually interact with the other experiment, rather than just running it passively

- Display glitches

- Input bugs. If you can, add a little test program at the beginning of each experiment that requires the subject to press the appropriate buttons, speak into the microphone you're recording from etc. This confirms that they understand which buttons are which, and that everything is plugged in.

Secondly, running in your own experiment will give you a sense of what it feels like to be a subject in the experiment, and help you notice more subtle bugs:

- Is it impossibly long and tiring? My rule of thumb is that half an hour is usually too short - you might as well add some more trials to increase your power. More than 45 minutes starts to feel unbearable though. But this varies from experiment to experiment.

- Can you feel your brain working in the way you hope it should? Do the hard things feel hard in the right way?

- Can you sense that there's some strategy that you want to use, but that would distort your data? If there's an easy shortcut to doing your experiment, subjects will find and exploit it. In this case, you can directly instruct them to avoid it, but you'd be better off modifying the design so that they can't. For instance, if you don't want subjects to rehearse in between trials, add in some kind of distractor task to keep them busy and engaged.

- Is everything counterbalanced? Might there be lurking order effects (where one type of trial always occurs at the beginning or end of a phase, or always precedes/follows another type of trial)?

Timing glitches

It's very hard to make sure that every piece of your experiment starts when it should and lasts for as long as it should. Before you do anything else, sit down with a pen and paper and calculate exactly how long each piece of your experiment is supposed to take, and store these as variables somewhere. Better still, they should be calculated automatically from your parameters.

Now, add code to your experiment that automatically times how long each piece lasts and when things are being displayed, making sure this all gets logged. Compare how long things are taking with how long you've calculated they should take. If you don't trust your timing code, use a stopwatch to make sure it's at least approximately right.

Modern computers have hundreds of background processes running (virus checkers, software updaters, email checking, self-refreshing webpages etc). Write a list of everything running, and make sure that as much as possible gets turned off before you run your experiment. Ideally, as little of this should be installed on your testing room computer as possible, but it's hard to avoid on Windows.

Some experiment presentation programs (e.g. the Matlab Psych Toolbox, PyEPL) can self-calibrate their internal timing for each computer. Others have separate timing modes that are optimized for accuracy in duration vs onset (e.g. EPrime, PyEPL).

Avoiding disaster

There are a variety of ways in which things can be brought to a crashing halt:

- can subjects quit the experiment easily/accidentally? If possible, disable standard keyboard shortcuts like Alt-Tab, Alt-F4

- turn off the screensaver

- make sure that email notifications, software update warnings and the like won't pop up in the corner of the screen, distracting the subject

- if your experiment was to crash for some reason (e.g. a power cut), can you resume where you left off?

Logging

Log everything. If you're lucky, your experimental presentation software will do much of the work for you (e.g. PyEPL, EPrime). Either way, you should attempt to log enough data that you could reconstruct the exact stimuli, and all of the subject's interactions with the experiment. This might seem like overkill, but it's valuable for a number of reasons:

- You never know which analyses you might want to run in the future. You might suddenly care about reaction times, or the exact placement of the randomly moving dots on the screen - who knows? If you haven't logged all the data you need, you'll be out of luck.

- You may be worried that there's a bug somewhere. Being able to cross-check one log against another is key to determining if/where there's a problem.

- You might be logging the same information in multiple ways - that's fine. Depending on the analysis, it might be much easier to process in one form or another

- Don't just log the low-level details. Logging every keypress and pixel color will certainly capture all the information you could ever need, but it will create an enormous amount of work to make sense of it all afterwards. If you have the high-level variables available in your experiment code, you might as well record them too to make your life easier later.

fMRI-specific

The list of extra things to check for when running an fMRI experiment is pretty bewildering, but here's a handy subset:

- make sure you view things through the projector, not just on the monitor in the control room. Who knows what devilry the projector might wreak as a result of display resolution interpolation, longer video cables, dying bulbs and the like?

- check your button boxes carefully. Some of them number in ascending order, some in descending order, some of them are re-programmable...

- our scanner emits a '!' every time it starts to collect a new image. Some programs see this as a 'LEFT SHIFT' plus '1', others as '!'. Make sure your experiment knows how to start each run in sync with the trigger, and can't be set off by an inadvertent button box press.

- timing is critical with fMRI. If each of your stimuli take a few milliseconds longer than you intend, you could easily be out of sync by an entire image by the end of a long run, which would be enough to scupper all your analyses.

- it's difficult to see anything in the bottom half of the screen in our head-only scanner. Make sure subjects will be able to see what's going on during your experiment.

Analyze your data early

Once you've run yourself at least once, try analyzing your data. Since you aren't a naive subject, your data probably won't be publishable, but this is still a critical step for at least the following reasons:

- If you can run the analyses, then you know that you've get everything logged that you need.

- You can check that the most obvious, basic, uncontroversial effects are there. If they're not, then that's a serious problem. You can also confirm that subjects' performance isn't too far towards floor or ceiling.

- You might be able to check whether any of your stimuli are noticeably poorly-normed (i.e. they stick out when they shouldn't)

- Sometimes, grievous logical errors slip through the design phase. Running an analysis is a really good way to pick up on such confusions. To be honest, running an analysis on fake (i.e. synthetic) data would probably work just as well, but it can sometimes be more work to generate good fake data than to collect a small bucket of real data.

Ask your friends

Finally, once you're pretty sure that things are working, try running a few of your friends or colleagues as subjects before anyone else. You can be sure that they'll pay attention to your instructions, try hard, and you may be able to get useful feedback about how it feels to run in your experiment as a naive subject. That way, if the data from your first few subjects aren't the way you'd hoped, you can be more confident that it's not just because indolent or surly Psych 101 students were chatting amiably on the phone while doing the experiment.

Version control

If you're not using a version control system to keep track of your experiment scripts, you're making your life harder for yourself in a dozen ways. Here are the key benefits:

- you don't need to keep saving your files as experiment1.m, experiment2.m, experiment3.m... The version control system will keep track of all the different versions, so you can see what you've changed at every point

- if you're working on multiple computers, or there are multiple people all changing things, you can keep things synchronized across all these computers. No more carting around USB keys.

- your experiment is always backed up

To be honest, if you're not using version control for almost everything that you write or program on your computer, then you fall into the same category as people who want to be a world-class chef but refuse to abandon their tried-and-tested approach of an open fire and a flint axe.

It's not worth putting the experimental data into the version control system, since the data won't be frequently changed and updated. Instead, just backing these up to an external hard drive is probably sufficient.

Conclusions

Mostly, 'experience' results from having done things enough to make all the common mistakes. You could then say that 'competence' is when you institute procedures that make those mistakes unlikely to happen again in the future. My aim in writing this was to help you find ways to make the common mistakes unlikely without having to make them all first yourself, so that you can be competent without being experienced.

If you can think of anything I left out, please point it out in the comments.

Evangelizing Emacs (in terms of economics)

2008-02-07T21:56:00.000-05:00

As may have become apparent, I harbour somewhat fanatically evangelical feelings about Emacs, though i'm very aware of its failings and idiosyncracies too.

Let me try my pitch for why you should learn Emacs in terms of economics. As a computer user, you want the biggest bang for your buck, the most reward for effort spent mastering a tool - one way to ensure this is for that tool to be useful in a variety of situations. Keyboard shortcuts, customizations, regular expressions and the like all provide enormous economies of scale. That is, they provide value, but they have a high marginal cost initially. However, the more you use them, that cost gets amortized, and so the more you profit from them.

Anyway, Emacs has a huge huge barrier to entry. Getting the hang of it is a pain, mainly because it has a huge amount of jargon and all of its keyboard shortcuts seem to involve 3 keys. The good news is that with a little effort, you can change them all to something a little more reasonable. Furthermore, once you've got the hang of a few, then you get to reuse them for almost everything you do.

Perhaps I should just quote Neal Stephenson, who said it best a long time ago:

"I use Emacs, which might be thought of as a thermonuclear word processor. It was created by Richard Stallman; enough said. It is written in Lisp, which is the only computer language that is beautiful. It is colossal, and yet it only edits straight ASCII text files, which is to say, no fonts, no boldface, no underlining. In other words, the engineer-hours that, in the case of Microsoft Word, were devoted to features like mail merge, and the ability to embed feature-length motion pictures in corporate memoranda, were, in the case of Emacs, focused with maniacal intensity on the deceptively simple-seeming problem of editing text. If you are a professional writer--i.e., if someone else is getting paid to worry about how your words are formatted and printed--Emacs outshines all other editing software in approximately the same way that the noonday sun does the stars. It is not just bigger and brighter; it simply makes everything else vanish. For page layout and printing you can use TeX: a vast corpus of typesetting lore written in C and also available on the Net for free."

Sensing the buzz at a conference

2007-10-21T22:50:00.001-05:00

I go to a lot of conferences. Often, there'll be lots of posters hanging up at the same time as workshops and presentations and it's difficult to figure out what to go and see at a given moment. Imagine if everyone had an RFID tag in their badges (at their choice) that anonymously logged their movements. Wouldn't it be useful to know the whereabouts of people who like the same things I do? The schedules tend to be organised in blocks of an hour or two. Feeling at a loose end, I could peer at my laptop, and have the various sessions and posters ranked for me, based on guesses about which things I'd be most likely to enjoy.

Of course, when the session switches, the system's guesses about what I should see will be basically at chance. Within a short time though, based on the number of people and length of time spent in different places, the system can start to accumulate evidence for its recommendations. And it wouldn't be too hard to seed its guesses based on the text from the abstracts.

Conferences certainly aren't the only or even the best example of how to use this information. But if I was a company doing this, I'd make sure the RFID tags are entirely optional, anonymize and noisify the data to assuage privacy concerns, and provide an API to access the data for free, so that enterprising folks could easily build mashups and try out alternative algorithms. Something interesting might come out of it, and it would be a fun way of generating publicity.

Any idea how much 30,000 RFID tags would cost?

Create something shareable

2007-10-21T22:44:00.000-05:00

My friend Joseph Perla is taking a year off from his undergraduate degree. Joe has created a rare opportunity for himself to spend his time any way he pleases. To be honest, I'm a little jealous of him - for a long time, I'd dreamed of living alone for a few months in a sunny, comfortable cave with high-speed internet access near a supermarket and a pub, just to think and read and write and code.

We talked about how to fill this period meaningfully. It would be pretty easy to fill a year reading RSS feeds and masturbating, though this would leave you with little to show for your time. I want to have some more substantive impact on the world around me. To be able to look back on my time spent with pride. To do something worthwhile. Joe and I talked about how to decide what projects would be worthwhile, and this was the metric I suggested:

Create something shareable.

Add to the sum total of human knowledge. Create something that others can use.

To be shareable, it has to be useful. It has to be trustworthy - if it's flawed/buggy, or its conclusions are ill-founded then it could actually be of negative value. It's probably going to require some effort, since if it was easy and quick, then it would be easier for other people to generate it anew than to internalize your solution. It probably needs to synthesize or improve upon what already exists, maybe surprising us or overturning existing intuitions.

The internet provides the medium for distribution, advertisement and collaboration. Commoditized hardware, open source software and freely available information provide the tools and raw materials.

This validates a whole host of online activities. Write carefully considered opinion pieces. Build up a corpus of entertaining posts to create an online persona. Gather interesting tidbits. Develop your art in whatever form it appears. Maintain an existing piece of open source software or create a Debian package. Write a new application or library. Edit wikipedia articles. Release your photos under a Creative Commons license. Put your notes online. Provide instructions for building cool things. Share your tools. Run a controlled experiment on yourself. Curate a dataset. Write a scientific paper and make it easy for others to replicate it. [These links are just some of my favourite examples of each activity].

Emacs freex 0.3.1

2007-09-20T23:22:00.001-05:00

Woohoo! The first public release of Emacs freex minor mode is now available (v 0.3.1) (co-authored with Per Sederberg).

The help is currently scanty, but there's a starter tutorial and some installation instructions. If you do get it working, I'd love to hear from you.

UPDATE 090308: I'm in the process of moving all the Emacs Freex mode to Google Code.

Matlab cookbook

2007-09-20T22:45:00.000-05:00

I'm a big fan of O'Reilly's Python Cookbook. Since O'Reilly hasn't published one for Matlab, and it's the language I know best, I'd dreamed of writing one myself.

That's a gargantuan task. So, in the meantime, I've had a go at collecting and posting a few tips and scripts that I thought others might find useful. To the Matlab Cookbook!

Scroogled

2007-09-18T22:44:00.000-05:00

Cory Doctorow has a nice, dystopic piece imagining 'What if Google were evil?'. It also provides a little ammo to discharge in the direction of people who ask what good people should have to fear about the burgeoning infringements of our privacy. This is a tricky subject for me, since I spend much more time dreaming of cool uses for collaborative filtering than dwelling on the chills I get from abuses of people's private data.

Non-parametric programming

2007-09-18T22:30:00.000-05:00

Joel Spolsky (of Joel on Software fame) is in the midst of a World Tour, showing off the latest version of his company's bug tracking and project management software, FogBugz 6.0.

I went along to see him when he visited Princeton, mainly out of curiosity. It turns out that FogBugz is pretty awesome. Let me highlight a couple of its most endearing features:

- the built-in wiki has the best WYSYWG editor I've seen

- utilizes backlinking between cases as a simple but handy aggregating related bugs

- really low barrier to entry for creating new bug reports

- beautifully integrated with email and discussion lists

Fine. But what really appealed to me was their emphasis on planning, and on tools that enabled more accurate estimations of shipping dates ('evidence-based scheduling'). Firstly, they make it as easy as possible to log one's activities ('currently working on blah'), so that the system can keep track of how long is being spent on each case. Secondly, they require you to estimate up front how long you think that each case is going to take.

Now, in principle one's ship date should now be determined by the some function of the number of cases left in your milestones list, how long each is projected to take, and the number of developers at your disposal. In reality, of course, these estimates are probably wrong. One can look at the history of an individual's estimates, and compare them to how long things actually took. Perhaps I'm regularly off by a factor of 2 - so if I say something's going to take four hours, it's probably going to take the whole day. FogBugz builds a simple regression model, predicting actual duration from estimated duration for each developer, from which it can easily spot this factor-of-two trend.

If I'm reliably off by a factor of two, then that's actually a kind of good news. It means that the system can be confident of its adjusted estimates. It means that the system can guess when the team is *actually* going to meet its milestones, rather than when they think they're going to, by using the predicted durations from its regression model.

But things are probably more complicated. What if someone's estimates are sometimes off by a factor of two, sometimes a factor of four, sometimes even over-compensating in the opposite direction? The variability in the goodness of this person's estimates is more problematic for our projections. FogBugz's solution is to use a kind of non-parametric permutation statistic to estimate 5%, 50% and 95% confidence intervals for shipping dates. I think the idea is to create a bajillion synthetic potential futures for each developer, each created by shuffling the evidence from their track record of estimations. We create a bajillion projected ship dates, each a combination of a set of synthetic developer-futures. Then, when the team manager wants to know the likelihood of shipping in time for Christmas, he or she can tell where it lies amidst the distribution of bajillion projected ship dates - if Christmas is earlier than all but a handful of the bajillion projected ship dates, then maybe there's less than a 5% chance of shipping on time. If Christmas lies somewhere in the middle of the bajillion-strong distribution, then the system might estimate a 50% chance of shipping on time.

This idea of scrambling the past to generate a distribution is at the center of most non-parametric statistical tests, which avoid making assumptions about normal distributions, homescedasticity and other evil-sounding stuff that's unlikely to be true in the real world. Anyway, props to Fog Creek for attempting to inject some low-level statistical intelligence in a way that might very plausibly work.

Entangling the ground and cloud

2007-09-12T01:00:00.000-05:00

The cloud and the ground

Most of you will have an idea of what I mean by 'living in the cloud'. The cloud is the internet, the web, ssh servers, http protocols and text boxes in browsers, accessible from anywhere, hosted on a server or many servers somewhere. Wikis, blogs, and bookmarking services like del.icio.us are the cloud. We travel to the cloud on a flying TCP/IP rug, whisking us from anywhere in the world to wherever it is the cloud is. The cloud is available to us everywhere, as long as we have internet access. Its non-local ubiquity is its virtue.

In contrast, the ground is our hard disk, .doc files and desktop applications, our personal computer, local, physical, heavy, awkward, something we carry around with us. Distance means something on the ground, because if you don't have your laptop with you, you can't get to your ground-home.

Perhaps I'm being crystal clear about what I mean by 'cloud' and 'ground'. In case I'm not, we can boil the distinction down to this: the cloud is everything that we need the internet to access. The ground is everything that lives on our personal computer. The tradeoffs are obvious. The cloud is always there as long as you have internet. The ground is always there if you're willing to tote around your computer.

The filesystem as carpet

We talked earlier of the network whisking us to the cloud like a many-threaded flying rug, a wizardly tapestry of protocols. But if the network is a rug that whooshes us from one location to the other, then perhaps we can think of the filesystem as akin to an everyday carpet - the flat, underfoot, non-vehicular kind that sits there as you wander about, so permanent as to remain quite unnoticed. On the ground, all of our files rest on the homely, woven fundament of the filesystem carpet. If you want to edit a document of any kind, you're editing a file on the fileystem. This is what gives the ground its feeling of stable proximity. You can't fall off the ground.

But what if, in the future, the faded paisley pattern of your local filesystem carpet hid a kind of quantum entanglement, where "the quantum states of two or more objects have to be described with reference to each other, even though the individual objects may be spatially separated"? What if the carpet in our home on the ground and the carpet in our home in the cloud were entangled? A long-standing, two-way, lightning strike beanstalk linking cloud to ground, and ground to cloud. I would be able to pootle about on the ground, knowing that the cloud is synchronized. So that when I leave my laptop home at home, the cloud has mirrored everything I need. I want my toothbrush to always be there, waiting for me, in my home and my home-away-from-home.

Every file appearing to rest so solidly on the filesystem-carpet would also sit on the cloud-carpet above. And vice versa. Every time you edit a file on the filesystem-carpet, the changes are propagated automatically to the simulacrum on the cloud-carpet, and vice versa. You think you're editing C:\blog\my_new_post.doc, but when you save, http://gregdetre.blogspot.com/ updates. You just tagged a page with 'superduper' on http://del.icio.us/gdetre, and a new .xml file appeared in ~/del.icio.us/superduper/ at the same time. In this hopeful future, where our home on the ground changes in lockstep with our home in the cloud, the question of whether we live in one or the other becomes meaningless to the user - when you light one candle from another, which one carries the original flame? Equally both. Either. When I edit a document, on ground or cloud, I want that document to exist equally, indistinguishably on cloud and ground.

If and when the ground- and cloud-carpets do become temporarily untangled, while on a plane or because of a wireless malfunction, then any changes made in either cloud or ground are stored up to be replicated at the very first opportunity, bi-directionally, automatically, and in the background.

What would it be like to use an entangled filesystem?

Everyone will have some space of their own in the cloud. When you first install your operating system, it will ask you for your cloud-home Open ID address. When you post a comment on Slashdot, the sign-on process causes a new directory to appear on your hard disk. All of a sudden, ~/slashdot.org/article/492308/comments/2309109.txt appears on
the ground.

Every directory on your hard disk will be mounted with a cloud-address that automatically provides a ground-cloud mapping schema for that directory. All websites will provide a unique URL for every state or view into the system - it'll be the job of the ground-cloud mapping schema to map those URLs to files and directories, specify permissions, decide which file format your content will appear as, define the effect filesystem actions have, etc. So perhaps editing 2309109.txt would change the comment on the actual Slashdot page... unless the ground-mapping schema says that comments can't be modified once written, in which case 2309109.txt will appear as read-only once it has been moved from the 'drafts' directory where you created it.

Right now, the browser is the main gate through which we pass to get into the cloud, but it needn't be. No one wants to have to type a whole new wikipedia entry in a poxy text box. Much nicer to fire up MS Word or emacs, and work on c:\wikipedia\Carpets.wiki in comfort (effortlessly mounted as a wikipedia service).

We're so close already. On the one hand, tools like Subversion and Unison handle synchronizing pretty well. On the other hand, we can mount remote filesystems over SSH with Fuse SSHFS or Emacs Tramp, so that they almost feel local. So there are tools that sync, and there are tools that make the remote feel local, but there are no tools that do both: automatically sync the remote and make it feel local ... running in the background in perfect imitation of a filesystem.

Tim Bray sounds a stirring clarion call for 'a "Publish" button on everything'. That's a solution at the level of the interface. I'd much rather we build publishing into the very heart of the system. I don't need a "Publish" button. Just let me save my files to a ground-cloud entangled filesystem, and have them simultaneously published for me.

Updated website

2007-09-09T15:04:00.000-05:00

For some reason, Google has rubbed me from its mind like a pimple in the wind. If you google for 'Greg Detre', this blog now shows up, but my main page (www.gregdetre.co.uk) has vanished. So, for now, my site will live at:

www.princeton.edu/~gdetre

While I was at it, I revamped it somewhat.

My take on emacsclient

2007-05-15T00:16:00.000-05:00

Emacs is pretty lightweight relative to most modern editors, though by the time it loads all the modes and gets through all the uncompiled junk in my .emacs configuration, you wouldn't know it.

Emacsclient is the solution - when you open a file with emacsclient, it doesn't start up a whole new emacs - it just opens it in the running emacs, which is more or less instantaneous.

There are lots of webpages on this, so I won't go into detail. Unfortunately, although you'd hope that calling 'emacsclient' would work just like 'emacs', this isn't true:

I wanted to tell emacsclient to display an emacs frame, without feeding it any filename arguments to display. No dice.

If I don't have a running emacs server (necessary for emacsclient to connect to), it just gives you an error message, rather than taking matters into its own hands and opening up a new emacs instance.

If I have emacs running on one computer, and I ssh into that computer, then I want to be able to type emacsclient and have a window show up. No dice. You have to explicitly feed it a display.

These issues were annoying enough that I nearly stopped using emacsclient. Various people have offered shell script hacks that handle some of these issues and more, but none of them solved the ones that bothered me. Plus, I like Python and I hate shell scripts. So I offer up a teeny python script, emf-on-display. If you alias 'emf-on-display' to something handy like 'e', then it makes emacsclient behave in a way that I find much more consistent with emacs.

P.S. I have not dealt with the possibility that you're running this in the terminal, and you don't have an X11 display at all.

P.P.S. I suspect that it requires emacs 22 (or gnuclient) to work, since it relies on being able to pass 'make-frame-on-display' as elisp code to be evaluated.

Testing the legs of the Sansa Express

2007-04-14T13:23:00.001-05:00

My Sansa Express arrived yesterday.

We plugged it into Adrian's Windows XP laptop, dragged some
files across, and went to eject it. No eject button on the
disk drive. Didn't show up on the 'safely remove device'
list. Huh. So we yanked it out, it played happily and all
appeared to be well.

When I got home, I plugged it into my Ubuntu laptop. A
Konqueror window opened up, I dragged some files across, and
went to eject it. Again, there was no 'safely remove' option in the
'/media/usbdisk' context menu, so I figured it was the same
deal. I yanked it out, but now it refused to light up, it wouldn't turn
on, and when I popped it back in it didn't mount. The thing was
dead.

I've played this game before with my old iRiver. So I downloaded the Sansa Express manual, looking for info on how to reformat or reset it. The hard reset instructions said to hold the 'Select' button while pressing 'Volume up' simultaneously. I
tried that a bunch of times, but still no dice. I started to sweat a little at that point, but ended up pressing lots of pairs of buttons simultaneously, and one of them seemed to do the trick. It might have been 'menu' + 'volume up'.

After resetting, I retried things on my Ubuntu laptop, and this time put the computer to sleep to force a safe removal. I later found that manually entering 'eject /media/usbdisk' works too, though sometimes takes a while.

It looks like the Sansa Express tries to use MTP (Media Transfer Protocol) if you're running XP/Vista, and falls back to UMS/MSC (i.e. just a normal USB hard disk) in OSX/Linux. The latter mode requires a safe removal.

It has to be easy, and worth it, for you to add tags

2007-04-07T06:52:00.000-05:00

Whoever adopted the idea that "there's a place for everything, and everything in its place" when it came to organizing files and ideas on a computer suffered from a failure of imagination. Or maybe they were just over-wedded to the desktop and filing cabinet metaphors. Fortunately, the idea of 'tagging' (or 'labels' in Google's parlance) blew that whole banal tidiness away. In short, tagging lets you assign things to multiple categories, or if you prefer, put things in multiple places. Rashmi describes this well - tagging is popular because there's a lower cognitive cost when you can put things in multiple categories, rather than having to decide on just one.

We've only just started to scratch the surface of how categorization schemes could work. I'm going to propose a few ways in which things might grow from here, focusing on the restricted case where you're tagging your own files privately, ignoring all the interesting goodness that happens when those tags are available to others, delicious-style.

N.B. I'm going to use the term 'category' rather than 'tag', since it's easier to think of things belonging to categories than being labelled with a tag. The key notion is that things can belong to multiple categories simultaneously.

The more tags the better

Jon Udell has a great post on building up a taxonomy of categories by hand, starting with a smallish corpus of documents, and just letting the taxonomy emerge, combined with a little judicious weeding. The dataset he has in mind is pretty small, and so he's aiming for 15-40 categories. The kinds of datasets I have in mind are much larger.

For instance, I have a few thousand text files with notes on topics ranging from Ubuntu troubleshooting to the symptoms of schizophrenia to my travel arrangements for the summer. I could maybe try and shoe-horn things into a few tens of categories, with each category holding many items, and each item belonging to maybe one or two categories. But I very quickly found this to be unsatisfying. We want to be able to differentiate things more finely than that. For instance, how would I categorize a document containing hotel bookings in Florence last summer for the HBM conference? Just by 'travel'? Or also 'Florence', 'conference', 'hotel', 'HBM', and '2007'. Remember the argument about lower cognitive cost though - it's much less effort just to include all those categories. If I do that, I'll end up with many hundreds or even thousands of categories, some of which will have tens or hundreds of members and some of which might only have one or two members. I think one might raise two main objections to this approach:

Can you really be bothered to add a bunch of categories each time you write something?
How do you begin to find anything now? Sometimes filtering by a category doesn't help because it returns way too many members, and sometimes it doesn't help because it returns hardly any. Where's Goldilocks when you need her?

I'll address these in turn.

Can you be bothered to add a bunch of categories each time?

People are lazy. Any system that requires people to be assiduous book-keepers while they're writing is doomed. Dave Winer talks about how he should be categorizing all his posts, and yet he doesn't do it - and this makes him feel guilty. He knows that he won't be able to trust the categories to find that thing later. The value of the whole system has dropped. Squirrels wouldn't go to the effort of hoarding nuts for the winter if they knew that they wouldn't remember where those nuts are when they need them. So what's the point of hoarding nuts any more? All of a sudden, the system has broken down. We need to find a way to make the system less brittle.

Let's look at Dave Winer's guilty confession a little more closely:

"I have a very easy category routing system built-in to my blogging software. To route an item to a category, I just right-click and choose a category from a hierarchy of menus. I can't imagine that it could be easier. Yet I don't do it."

If you ask me, that's not easy enough. Navigating hierarchical menus with a mouse is slow and distracting. Blogger does it right - there's a 'labels' text box that you can tab to, into which you can write a comma-delimited list of tags. As you type, it auto-suggests - pressing 'return' fills in the rest of the tag and puts a comma and space after for you. So that's step 1.

But it should be even easier. What should happen is that the machine should automatically throw up a list of tags that it thinks might be appropriate for this post. It should put the ones it's most confident about to the left, and less confident ones to the right, with the cursor positioned at the end to make it easy for the user to delete false positives and add new categories it missed. And if you're feeling lazy, then you can just accept the machine's suggestions without glancing at them. The cost of a false positive is low, so it'll deliberately suggest too many. This brings us neatly to our second concern.

But then how do you find anything?

So now every document belongs to a bajillion categories, none of which is particularly useful on its own. But a conjunction of categories should narrow things down nicely. If I'm trying to find that hotel booking in Florence, I don't have to worry about remembering whether it's tagged with 'travel', 'hotel', 'Florence', '2007' or 'HBM conference', since it's tagged with all of them. So I'll try filtering by the conjunction of 'hotel'+'Florence'+'2007' and that'll probably winnow things down sufficiently for me to pick the file out manually (see also: make tags not trees). .

But maybe we never made a 'Florence' category. It seems like such a natural cue to use now, but at the time, 'Florence' didn't spring to mind as a salient category, despite our liberal categorizing policy. If the system auto-completes in a handy way, we'd already know this, and our fingers would already be backspacing and trying 'Italy' or 'HBM conference'. There are many points of failure, but there are also many points of entry. If we make it easy enough to cue for conjunctions of categories, then there's a very low cognitive cost to having to backtrack once or twice, since our brain effortlessly supplies us with so many possible cues to use.

We could make things even less brittle in lots and lots of ways. Perhaps the system notices that only one item in the whole database is tagged with 'Florence', so it's probably too restrictive a category. No matter. It could just ignore 'Florence', or suggest that we omit 'Florence' from our search. Better still, and less intrusively, it could now grep through all the files that match one or more of the tags to see if 'Florence' appears in the text, and automatically suggest any matches as partial matches.

Conclusions

I keep coming back to the same feeling - for the most part, people don't write notes because they don't think they'll be able to find those notes later when they need them - so why bother writing the notes in the first place?

All of these suggestions are geared towards:

Reducing the cognitive cost at both writing and retrieval. If it's less effort, you'll feel less lazy about adding category metadata.
Making the system less brittle, so that if you were lazy about your category metadata, you still have a good chance of finding things later. This is the key to ensuring that you don't end up losing faith and give up on writing things down in a structured way altogether.

Taken together, I hope that it will become easier to categorize your notes in a way that helps you find them later, which is going to make you much more likely to write them down in the first place.

Neuroscience notes

2007-04-07T04:07:00.001-05:00

The neuroscience notes for my PhD qualifying exam are now online as a single compressed tarball and as browsable individual webpages. If you use Emacs Muse, then you can also grab the muse files as a tarball (or by changing the .html to .muse) too.

The notes are in wiki form - in other words, I tried to give every brain region, tract, disorder, function and topic its own page. Emacs Muse automatically created the hyperlinks as I was typing (thanks to Per Sederberg's implicit linking patch. In reality, many of the pages are missing or sparse, since this is a pretty gargantuan task. There's also the possibility that things are inaccurate. For instance, I'm pretty sure there's some confusion regarding the nucleus reticularis vs nucleus reticularis pontis oralis, since I didn't initially realize that they're distinct brain regions with similar names...

Anyway, you're welcome to use, modify and distribute these in any way you'd like, though I'd appreciate a shout-out if you do. Consider them to be released with a Creative Commons Attribution 3.0 license, though I've been lazy and not included the license file. I'd doubly appreciate hearing about any flaws or confusions you find. I'm still working on these, and so I may one day release an updated version.

Obviously, to some degree this is reinventing the wheel. Most of these notes are from Kandel & Schwarz (4th edition), and you can find most of this stuff online, e.g. the wikipedia has some good pages on the brain, though the usual caveats apply. I also learned a lot about the pros and cons of wikifying knowledge along the way, but that's the topic of another post.

Collaborative filtering and how it's going to help us consume

2007-04-07T02:56:00.000-05:00

In the future, we will routinely employ some product that will probably be called Microsoft MyLife (1) to manage our reading, news, entertainment and shopping. What will it do? Let's start with the present and build forwards. For my money, amazon.com is the best site in the business. It takes the only shopping activity I enjoy, book-shopping, and manages to make it even better online.

Shopping with Amazon is so pleasurable and fruitful because it first leads me by the hand towards things that I'm genuinely interested in and then provides me with the 3rd-party reviews and ratings feedback that I always find myself hungering for when buying something. It's like having Virgil for a librarian. It's shopping by democracy, where your candidate always wins. But it's still pretty limited. I want to be able to head to the recommendations page and choose to be recommended books of a certain kind only, rather than having my interests in neuroscience, programming and sci-fi lumped indiscriminately together. I may want it to weight recent purchases heavily, or to only find books by authors I've never read. But the possibilities for tinkering with the recommendations parameters are sadly limited.

I dream of a 'How lucky do you feel, punk?' slider, that ranges from conservative to adventurous. Perhaps today I'm tired and I want something I'm certain I'll like. If I've bought the first 35 of David Gemmell's Waylander books, Amazon can be pretty sure I'll like the 36th, since there's no way to tell them apart. But maybe tomorrow I'll be high on redbull and tractor fluid, and I'll want something new and unexpected. Perhaps initial impressions indicate that I'll like some new author who's making waves, or perhaps Amazon's crazy collaborative filtering algorithm thinks that David Foster Wallace + David Sedaris = Tom Robbins, and recommends something to me out of left field accordingly. After all, I want help choosing a book, but part of the reason I like browsing for fiction arranged alphabetically is that you never know what' s going to catch your eye. I can choose on a given day whether to browse only for names I know, or to open myself up for something fresh and unexpected.

Secondly, I want to be able to ask for recommendations for someone else. Let's say it's my dad's birthday. I want to ask Amazon, 'What would I like if I was a middle-aged man who likes John Le Carre, Tom Peters and Hoagie Carmichael?'. I want to be able to create a persona for my dad, and for it to make some guesses. Even if they're terrible, maybe they'll give me ideas, or maybe I just need to give the system a little more information. At this point, things could get interesting. It would be pretty easy to integrate this with my dad's actual Amazon account, if he chooses to let me, so that it could take his purchasing history and wishlist into account as extra information. It would know what books he's bought recently, and so might remind me of some interests of his that i've forgotten, or of some burgeoning interests that I can sneakily anticipate. And i'm prepared to bet that it could do this with just a broad sprinkling of sample purchases to guide things. You can think of the adventurousness slider bar mentioned above as titrating from Marks & Spencer pullovers to gift vouchers at Stringfellows. The point is that i want to be able to tap my guide on the shoulder, shake my head, and point in a different direction. 'Yes' to the Herend china, but 'no' to the Chinese Hentai. The current collaborative filtering algorithm that they use to make recommendations works brilliantly, but is amazingly restrictive in the way that you can tweak it.

Let's say that Jeff Bezos reads this, slaps his forehead at the obvious genius of it all, and immediately engages a few of his platoons of elite Bonobo chimps trained from birth in arcane RDBMS lore to implement all of this. What next?

It knows what books I like. Why stop at books? Amazon sells everything except the kitchen sink. It probably does, in fact, sell legions of kitchen sinks too. But let's stick to books, music and films for now. It seems obvious to me that what I read, what I listen to, and what I watch are going to be predictive of each other. In fact, the broader the information the system knows about you, the better I would imagine it could triangulate on what you like and generalize to useful recommendations. It should be relatively effortless for Amazon to generalize from books to music to films, or vice versa, and I'd be astonished if they weren't already doing that. It's not so clear how your furniture purchases might be dictated by your reading habits, but it's not ridiculous either to think that a young 20-something male with money to burn who likes Friends might very easily be persuaded to buy a Lay-Z-Boy comfy chair (as featured on the series) if a few DVDs from the series (that he doesn't have) get bundled free.

Walmart are starting to use this kind of data-mining in all kinds of ingenious, insidious ways with their product placement, but I'm talking narrowcast, baby. I'm talking about a one-time offer for you and you alone, brought to you direct by the system. I don't really care all that much about Amazon knowing all this about me as long as they promise promise promise not to sell it, and as long as they continue to help me buy great shit cheaply without actually having to shop for it.

So they can tell me what DVDs and furniture to buy based on what I read. What if Amazon bought Ticketmaster.com tomorrow? Then, they could send me an email telling me that there's a crazy new concert/play/demonstration/sewing circle next week, and would I like tickets? It knows that I can't tell richly-developed fictional characters from a rotting horse's arse, and that I like Dan Brown and art, so it cobbles together a deal with lastminute.com to send me to Paris, Rome, London and Roslyn at highly discounted rates.

How does it know all this? Because other people who like the things I like - they liked that. Sure, I'm being shepherded, but if I could have my own personal shepherd who keeps pointing out great unsigned bands in intimate venues, movies from Chile that don't have Stephen Segal in them, and books that make me cry, then sign me up to be a sheep.

It's pretty easy to see where this is going. TV's going the way of the dodo, and even Tivo's a bit tovo. I don't want anyone to ever tell me that I have to watch the West Wing one episode at a time, once a god-damned-week. I want to buy 50 TV meals and watch them back-to-back without sleep. And there probably aren't that many people quite like me, but there are a few, and that's exactly what they like to do, so it shouldn't be too hard for my collaborative augmentation shepherd to have my TV meals frisbeed through my open window at regular intervals by a supermarket delivery man.

How far might the system be able to generalize across domains for a given person? If it knows about my book, music and film tastes, could it start to guess what kinds of plays I would like, or magazines I would read? Pretty soon, it could start recommending clothes and events and articles.

If you start to map individuals to their locations and movements, then you could start to make recommendations about where to shop or visit. What could be more useful than knowing where my dad goes to shop, if I'm trying to buy him a birthday present? Actually, I can think of one thing more useful than that - knowing where people looking for presents for their dad went when they went shopping... It could plan out routes, take me to little one-of-a-kind shops tucked away, either because I tell it about them, by keeping track of my credit record, following my movements with something like GPS.

Eventually, you could see how this could improve, or invade, every aspect of our life. All of the information you consume would be customized to your tastes, or if you prefer sometimes, to someone else's tastes. It seems critical though that you'd always be able to tweak the knobs when you're feeling adventurous, because it would be so easy for us to habitually tread the same well-worn paths, hearing only the opinions that we've trained the system we want to hear.

Footnotes

(1) The name's so catchy, trite, and alarmingly intrusive-sounding that I couldn't pass it up. There will probably be an open source version called GNU Memacs.

[Update: I think they already have a MyLifeBits project that focuses on collecting all the data amassed over your life together, but that's not really what's being discussed here. That's about retrieving information. This is about proactively suggesting new stuff from the cloud.]

Make tags not trees - filesystem idea based on tags instead of hierarchical directories

2007-04-06T23:32:00.002-05:00

Until recently, it was easier to find something amidst the five zillion pages on the web than it was to find something on your own hard disk. It would be faster to Google for something than to burrow through subdirectories looking for it.

Could this be because the files on my hard disk are poorly organized? Bah. Maybe so. But that's not my fault - it's more or less inevitable once you have a lot of files, because hierarchical filesystems require each file to live in a single location. If I download a paper on memory for a class, should I organize by:

the context, e.g. the name of the class, lumping together all my writings and reading materials from that context together - ~/psy330/reading/
or by the type - things I've written vs reading materials - ~/reading/psy330/
or by good/bad or date produced or something else entirely?

Whichever decision you make, there'll be times when you'll wish things were organized some other way. This is why tagging is so popular. It's because things inherently belong to multiple categories. And, because tagging is easy.
Google Desktop, Spotlight, Beagle and other offerings have helped considerably with all this. If you want to locate a single file, and you can't remember where you put it, then full-text search is the way to go. But let's consider the case where you have files that you want to treat as related, even if their contents aren't obviously similar. We want this all the time. Take the reading list for a particular course or project as an example. This is why we needed directories and filing cabinets in the first place.
My proposal here is to replace the hierarchical filesystem with a completely flat space and lots of tags. Each file would be tagged with one or more tags, just like on http://del.icio.us/. The 'save as' dialog would look a little different. Instead of a list of directories that you can burrow into, there'd be a list of tags. When saving a file, you'd select as few or as many as you like, give the file a name just as now, and you're done. To open a document, you filter using some tags, watching the list of files that match being winnowed down, and select from an alphabetized list. Or, use wildcards to winnow down by filename directly. Or some combination.
Converting an existing hierarchical filesystem would be easy in most cases. You could just grab all the subdirectory names in a path and treat them like unordered words in a bag. Let's keep the same '/' file separator we're used to, but change its implicit meaning from 'contains-this-directory' to 'and-also-this-tag', so:

~/reading/psy330/hippocampus/blah.pdf

would now be equally accessible from:

~/reading/psy330/hippocampus/blah.pdf
~/psy330/reading/hippocampus/blah.pdf
~/reading/hippocampus/psy330/blah.pdf

All these locations would end up meaning the same thing. In this way, a subdirectory is really a conjunction of tags. In our simple example of storing .doc and .pdf files for documents and reading materials for a class, we'd simply tag some of them 'doc' and some of them 'reading', and give them both the 'psy330' tag for the class.
Upon looking at this, it's clear you've lost some information, but I don't think it's information we'd miss much. The assumption underlying a lot of this is that where we now have hierarchy, we could manage just as well with intersecting sets, which would require considerably less effort to memorize.
There are, inevitably, unanswered questions and lurking gotchas.

I think we'd probably want to create a default/preferred way of expressing things, so that tags with more items or that are more discriminative go on the left, or something akin.
You shouldn't need to specify all the tags for a given file. Just enough to specify it uniquely, given its filename. So, if there are no other blah.pdf files in the 'reading' tag, then you should probably be able to access it straightforwardly at ~/reading/blah.pdf though this has the unfortunate implication that if you were to add a new blah.pdf that also had a 'reading' tag, the above location would become ambiguous.
If there are multiple blah.pdf files in the reading tag, then the system would need to prompt you with a list of tags that would help disambiguate them. Wikipedia's interface might have some lessons about disambigation that could be learned from.
At this stage, a tags-not-trees system seems better-suited for home directories ('My Documents' for Windows users) than system directories. In home directories, most of the organization is human-generated and needs to be human-readable, whereas /etc directories are mostly machine-generated to be uncomplicatedly machine-readable.
The only way metadata-entry systems work is if they require little work on the user's part. The nice thing about tagging is that it should be relatively easy for the computer to make guesses about which tags you'll want to put something in, based on your tagging of previous files. So when you click 'save as', it will prompt you with a list of tags that it thinks you'll want to use, ordered in terms of certainty. You delete a couple, add a couple more, and leave the rest in place.
This is not a trivial problem, but you'll have a large corpus from which to do your Bayesian learning (or whatever). And you can seed the corpus from day one with information from the existing file hierarchy, and with some clustering applied to the full text of the files.
This is the kind of problem that machine learning can really help with. There's a decent amount of data, it's getting feedback on each guess from the user and it's doesn't matter if it's occasionally off-base because it's only making suggestions.

I like this idea. I even think it might work, though I admit to feeling a little unsettled by the notion that all the files on my hard disk would effectively live in one place. Well, that's not strictly true. Our notion of 'space' in filesystems would have to warp a little. It's easy enough to imagine a filesystem now as a ramifying rabbit warren. This would require us to think of file locations in terms of boolean queries, and I can't come up with a nice metaphor. I think it's easy enough to grasp, but there's nothing outside the computer that implements tags, because they inherently incorporate the idea of superposition (one thing existing in multiple places).
I would love to see a FUSE implementation of this. It would have to be open source and run on Linux, and I'd consider trying it. The closest I've seen (from this list) are:

OpenomyFS - propietary and web-based. Otherwise, looks interesting
TagsFs - seems to be focused on mp3 tags
RelFS - a full relational database
LFS - the most interesting of the bunch

If it turns out that any of those projects are alive and easy to try, I'd be pretty gung-ho about it.

UPDATE: there are some great links and comments below, and also at:

John Cunningham Lily, RIP

2007-04-06T23:31:00.000-05:00

According to the wikipedia:

"[John Cunningham Lilly] is best remembered as a pioneer researcher into the nature of consciousness using as his principal tools the isolation tank, dolphin communication and psychedelic drugs, sometimes in combination."

A noble epitaph. I hope to be remembered as "a pioneer researcher into the nature of consciousness using as my principal tools drugs, dirty dancing and pounding techno music, sometimes in combination".

[Thanks to Sara Szczepanski for pointing this out]

The Pittsburgh EBC competition

2007-03-26T14:39:00.001-05:00

Try and picture the scene: you're in a narrow tube in almost complete darkness, there's a loud thumping noise surrounding you and you're watching episodes of the 90s sitcom, 'Home Improvement', with Tim The Tool Man Taylor and his family. There's a panic button in case you feel claustrophobic, but it's all over in less than an hour. It sounds a little surreal, but that's what it would have been like to be a subject whose functional magnetic resonance imaging (fMRI) brain data was used in last year's Pittsburgh Brain Analysis Competition.

After you've watched three episodes, kindly folk in glasses and white coats would take you out of the scanner bore, give you a glass of water and then over the next few days, they'd ask you to watch those same three episodes again over and over. On the second viewing, they'd ask you 'How amused are you?' every couple of seconds. On the third viewing, they'd keep wanting to know how aroused you are on a moment-by-moment basis. Then, 'Can you see anyone's face on the screen?', 'Is there music playing?', 'Are people speaking?' and so on, until you've watched every moment of every episode thirteen times, each time being asked something different about your experience.

Our job, as a team entering the competition, was to try and understand the mapping between your brain data and the subjective experiences you reported. For two of the episodes, we were given your brain data along with the thirteen numbers for every corresponding moment that described your arousal, amusement, whether there were faces on the screen, music playing, people speaking etc. Our team, comprising psychologists, neuroscientists, physicists and engineers, put together a pipeline of algorithms and techniques to whittle down your brain to just the areas we needed and smooth away as much of the noise and complexity as possible. Think of these first two episodes as the 'training' data. Then, we were given only the brain data for the third episode, the 'test' episode, from which we had to predict the reported experience ratings.

Our predictions were then correlated with the subjects' actual reports, and we were given a score. We ended up coming second in the whole competition, and we're hoping for the top spot in 2007. Much of this effort has had a direct payoff for our day-to-day research. We now routinely incorporate a lot of these machine learning techniques when trying to understand the representations used by different neural systems, and how they relate to behavior.

Members of the team: David Blei, Eugene Brevdo, Ronald Bryan, Melissa Carroll, Denis Chigirev, Greg Detre, Andrew Engell, Shannon Hughes, Christopher Moore, Ehren Newman, Ken Norman, Vaidehi Natu, Susan Robison, Greg Stephens, Matt Weber, and David Weiss

Statler pulchrifex

2006-08-31T23:30:00.000-05:00

My friend Matt Weber just had one of his fiction pieces published on the back page of Nature. I highly recommend it.

Push Singh

2006-08-31T23:29:00.000-05:00

Push Singh - memorial page and his blog

I didn't know him very well, but I'm sad he's not around any longer. He was an intruiging and inspiring figure.

Lawrence Lessig

2006-08-31T23:25:00.000-05:00

on free culture (from PresentationZen)

Does eating salmon lower the murder rate

2006-06-04T23:12:00.000-05:00

'Does eating salmon lower the murder rate?' (NY Times - from Pat Gunkel) (040616)

Gesch (2002) showed that feeding volunteers in a British prison fatty acids and other supplements led to a drop of more than a third in antisocial behavior (assaults and other violations) (N=231).

Also mentions an unnamed group in Finland working on Omega-3 fatty acids' effect on frontal cortex.

The world is your canvas

2006-04-18T23:22:00.000-05:00

Cool pavement art - (from ads - unknown source)

Ehren had a cool idea about this. He suggested painting massive fixed-perspective images just like these on roadside hillsides in the snow, like a huge alien flying saucer that would only look right from the perspective of passing drivers

[update] coffee cup manhole covers

The Turing tournament - a proposal for a reformulation of the Turing Test

2006-04-17T23:14:00.000-05:00

Introduction
Describing the Turing Tournament
Comparing the Turing Test and the Turing Tournament
Devising new rules, and non-linguistic competitors
But is it intelligent?

MH: Are you a computer?

Dell: Nope.

MH: You'd be surprised how many fall for that one.

Dell: Not me.

——

MH: What's fifty-six times thirty-three?

Dell: One thousand eight hundred forty-eight.

MH: You're pretty fast!

Dell: Those are my favorite numbers.

— from http://home.sprynet.com/~owl1/turing.htm

Introduction

The Turing Test was designed to be an operational test of whether a machine can think. In Stuart Shieber's words:

"How do you test if something is a meter long? You compare it with an object postulated to be a meter long. If the two are indistinguishable with regard to the pertinent property, their length, then you can conclude that the tested object is the given length. Now, how do you tell if something is intelligent? You compare it with an entity postulated to be intelligent. If the two are indistinguishable with regard to the pertinent properties, then you can conclude that the tested entity is intelligent (pg 1)."

In order for a machine to be deemed intelligent according to the Turing Test, we would determine whether human judges could reliably distinguish a human from the machine after some lengthy text-only conversation. I don't think a machine is going to pass it any time soon, and when it does, it'll be pretty self-evident that we're dealing with a machine that can think.

Anyone who disagrees that a full and proper Turing Test is a stringent enough test of intelligence should read Robert French's excellent discussion of the kinds of very human and culturally rooted subcognitive processes that would have to going on in the machine in order for it to pass. His criticism is that the Turing Test "provides a guarantee not of intelligence but of culturally-oriented human intelligence", i.e. that it sets the bar too high, or too narrowly. This is a subtler variant of the obvious point that human beings who don't speak English would fail a Turing Test with English-speaking judges. In other words, the Turing Test is a necessary but not sufficient test of intelligence, because you would have to have a certain subcognitive makeup in order to pass it, on top of being intelligent.

The beautiful thing about the Turing Test is that there's nothing about it that's specific to machines. Indeed, Turing's original idea for the Imitation Game, as he termed it, was based on a parlour game where the judge attempted to distinguish male from female players. This essay is an attempt to broaden the scope of the Turing Test from being a binary and culturally-rooted test of human intelligence to something vaguer and less unidimensional.

Let's make this idea somewhat more concrete, and considerably more vivid. Imagine that a small, slimy green-headed alien lands on your lawn right now, travelling in a spaceship the size of a Buick. Assume that the alien demonstrates its extraterrestrial credentials to your satisfaction by whisking you to its home planet and back before breakfast. It bats away the impact of a few .357 rounds with its forcefield and patiently replicates household objects for your amusement. It would seem niggardly to refuse a being that has mastered faster-than-light travel the ascription of intelligence when most humans can't tie their shoelaces in the morning without a dose of caffeine. So we might be moved to patch the Turing Test in some ad hoc manner to read:

"Any entity that cannot be reliably distinguished from a human after a lengthy text-only conversation, OR that has mastered faster-than-light travel and can withstand a .357 round at close up, can be considered to be intelligent."

It's clear that this lacks the pithy generality of Turing's original formulation, and we'd have to do quite a lot more work to restrict the scope of the above to exclude asteroids. Perhaps over time, our super-intelligent alien will learn to speak English with a flawless cockney accent, and will pass the standard Turing Test, rendering this discussion moot. But in the meantime, before it has learned to speak a human language, we are faced with a manifestly intelligent being that fails our gold standard test for intelligence. The background aim of this whole essay will be to consider a new version of the Turing Test that overcomes the inherent human- and language-specific parochialism of the original. That way, our intelligent alien might pass, without having to learn to speak English with a cockney accent.

Along the way, it may be that our reformulated test provides a more constructive goal and yardstick by which to direct and evaluate progress in AI research than the standard Turing Test. Perhaps its primary limitation is that it's difficult to restrict the difficulty or scope without losing everything that's interesting about the test. And since even our current best efforts are a long way from success, the gradient of improvement is almost flat in every direction, making it difficult to discern when progress is being made in the right direction. This makes it difficult for machines to bootstrap themselves by training against each other, requiring lots of labour-intensive profiling against humans. Finally, the current test is very language-orientated, and undesirably emphasizes domain knowledge,

Describing the Turing Tournament

I'll term this new version of the Turing Test the 'Turing Tournament', to reflect its competitive round robin form. Like the original Turing Test, the Turing Tournament will not yield a definitive, objective yes/no answer, but rather a ranking of the entrants, where the human players provide a benchmark. A lot of the details I'm proposing will probably need considerable refinement. Here are the organizing principles of a Turing Tournament:

The organizers of each tournament decide what the domain of play will be, e.g. a chessboard, text, a paint program, a 3D virtual reality environment, binary numbers, or some multidimensional analogue stimuli.
Every 'player' (within which I'm subsuming both human and machine variants) is competing in a round robin competition, and will play every other player twice, once as the 'teacher' and once as the 'student'.
Every bout will have two players, a teacher and a student. Play proceeds in turns, with the teacher going first. Play terminates when the allotted time has been exceeded, or when some terminating criterion specified by the teacher has been satisfied. Neither player will know the identity of the other player.
Before the bouts begin, every player is given access to the domain of play so that they can construct their own set of rules that will operate when they are the teachers in a bout.
The organizers of each tournament determine the scoring for bouts that terminate relative to bouts whose time elapses. We will consider some possible scoring systems later.

These sound like strange rules. What kind of games could be played? Why does each teacher get to set their own rules? Do teachers get rewarded or punished if a student is able to reach criterion for their bout?

I think the easiest way to illustrate what I have in mind is with a concrete example. Imagine the following scenario:

A big room with lots of people sitting at computers. The people are the human players. The machine players are sitting inside a big server at the back of the room.
The domain for this competition is a Go board, a 19x19 checkerboard with black and white pieces. Although all bouts in this tournament will take place on a Go board, the rules and goals of each bout will be up the teacher of that bout.
Let us peer over the shoulder of a human player, currently in the role of student, trying to determine what the rules of the bout are, and play so that the bout terminates before running out of time. Neither we nor they know whether the other player is human or machine.
The board is blank initially.
As always, the teacher makes the first move. They place a horizontal line of 19 black pieces in the bottom row of the board.
Now it is the student's turn. They have no idea how the bout is scored, what the aim is, what constitutes a legal move, how many moves there will be or whether there will be multiple sub-bouts. All of that is up to the teacher.
Working on the assumption that the teacher wants the student to play white, the student lays down a single white piece in the top left corner.
The teacher removes the white piece, and replaces it with a horizontal row of white pieces just above the existing horizontal black line, and another horizontal row of black pieces above that. So now there are three rows of pieces filling up from the bottom of the board: black, white and then black.
The student decides that the removal of their white piece in the corner was a signal that its future moves should consist of placing an entire row of pieces on the board at a time. The student tries placing an entire row of white pieces in the top row of the board.
The teacher again removes all the student's pieces, and replaces them with another row of white pieces and another row of black pieces. The bottom of the board consists of black, white, black, white and black stripes.
The student reasons that its next move should be to place a row of white pieces above the most recent row of black pieces to continue the stripy pattern.
Gratifyingly, the teacher leaves the row of white pieces in place, and adds a black row above it, as expected.
The two players continue to take turns until all but the top row has been filled with alternating black and white rows.
Now, it is once more the teacher's turn, and the student wonders whether the last row will be filled in. Instead, the board blanks again, and the teacher places a vertical column of white pieces on the right hand side.
The student tries tentatively to place an adjacent column of black pieces, deciding that this sub-bout involves creating black and white vertical stripes, with the black and white players reversed.
As it turns out, this assumption appears to be correct, since the teacher does not remove the student's pieces, and together they quickly build up an alternating vertical stripe that moves leftwards.
When only the last column remains to be filled in by the teacher, the bout has reached criterion, and the student moves on to the next bout, with a different player.
Upon inspecting the scores later, our human player (the 'student' in the bout just described) finds that they had scored highly on that bout, but not as high as some. Some of the machine players had failed to see a pattern at all, and had been putting pieces down more or less at random. These players did the worst, since the scoring for this tournament is a function of the total number of turns taken to finish the bout, as well as the number of errors made by the student. (need to clarify???)
Like our hero, the best players at this bout had also quickly deduced that the pattern involved stripes. Their extra insight came after a few turns, where they tried placing multiple stripes down at once. As it turns out, there was nothing in the rules set by the teacher prohibiting this, and so they finished more quickly, earning a higher score.

It seems reasonable to imagine that most humans would quickly figure out the stripy pattern, and some would eventually think to lay down multiple stripes at a time. Might a machine? Perhaps soon.

This is intended as a toy example. The rules of the bout are pretty simple, but I think they would discriminate somewhat between intelligent and not-so-intelligent players. The key point to note is that each player would play twice against every other player, once as the teacher and once as the student playing within the teacher's rules. Perhaps some bouts are too hard, and some are too easy. But en masse, the rankings should discriminate quite finely between players, even between human players. The exact details of the scoring, especially how teachers are scored, and how teachers pre-specify their rules, are clearly going to be crucial. It will suffice for now to say that students should probably get points for satisfying the criterion of a bout quickly, and teachers should be rewarded for devising discriminative games, that is, games that only intelligent players can solve. I will defer further discussion of these topics until later.

Comparing the Turing Test and the Turing Tournament

In discussing the idea of an Inverted Turing Test (more below) Robert French states that:

"All variations of the original Turing Test, at least all of those of which I am currently aware, that attempt to make it more powerful, more subtle, or more sensitive can, in fact, be done within the framework of the original Turing Test."

Is the same true of the Turing Tournament? I think the answer is both yes and no. In fact, you could think of a Turing Tournament as a kind of generalization of the Turing Test. That is, the original Turing Test could be treated (more or less) as a Turing Tournament where the domain of play is restricted to text, and the bouts terminate if the teachers/judges are satisfied they are talking to a human. It wouldn't be quite the same, since here the players would double up as judges and the judges would double up as players. In other words, the machines would also themselves be making judgements about the humanness of both each other and the humans - an 'Inverted Turing Test'. In its current formulation, where every player plays every other player as both teacher and student (i.e. judge and player), a Turing Tournament would really be a strange hybrid of both the Inverted and standard Turing Tests.

The idea of an Inverted Turing Test has been proposed before:

"Instead of evaluating a system's ability to deceive people, we should test to see if a system ascribes intelligence to others in the same way that people do ... by building a test that puts the system in the role of the observer ... [A] system passes [this Inverted Turing Test] if it is itself unable to distinguish between two humans, or between a human and a machine that can pass the normal Turing Test, but which can discriminate between a human and a machine that can be told apart by a normal Turing Test with a human observer."

French ingeniously showed that this Inverted Turing Test could be simulated within a standard (if somewhat convoluted) Turing Test. In contrast, it seems clear that an unrestricted Turing Tournament could not be fully simulated by a Turing Test because the potential domains of play are limitless. So although one might imagine instantiating the Go domain by communicating using grid references within a standard Turing test, it seems clear that there would be no way to run a domain of play such as a 3D virtual reality environment within a standard Turing Test using text alone. The principle advantage of widening the domain of play from text-only in this way is to allow players to pass some kinds of Turing Tournaments without speaking English, or any language at all. As a result, it seems reasonable to think of the Turing Tournament as (more or less) a superset of the Turing Test, or if the reader prefers, at least a redescription of it with unrestricted domains of play. I find this Ouroborean quality quite pleasing. [is it really ouroborean???] Either way, we can agree that most of the original Test's merits and stringency should still be present in the Tournament version, depending on the way a particular Tournament's domain of play and restrictions are set up. This does raise the important question of whether a Tournament victory would be as convincing a demonstration of intelligence as a victory in a standard Turing Test - I will return to this below.

French also shows that the Inverted Turing Test could be passed by a simple and mindless program that would take advantage of the very subcognitive demands that make the original test so parochial and difficult to pass. In short, the machine could have a pre-prepared list of questions that have been shown to weed out machines in the past, such as

"On a scale of 0 (completely implausible) to 10 (completely plausible), please rate:

'Flugblogs' as a name Kellogg's would give to a new breakfast cereal.
'Flugblogs' as the name of a new computer company.
'Flugblogs' as the name of big, air-filled bags worn on the feet and used to walk on water.
'Flugly' as the name a student might give its favorite teddy bear.
'Flugly' as the surname of a bank accountant in a W.C. Fields movie.
'Flugly' as the surname of a glamorous female movie star."

By pre-testing lots of humans and machines to figure out what kinds of things people say, and machines fail to say, a simple but well-prepared machine could draw up a 'Human Subcognitive Profile'. By comparing this to the responses of players, it would be an extremely effective judge in an Inverted Turing Test. There are two reasons why this strategy would not work in a Turing Tournament:

a) In the above specification, none of the players know which domain they will be playing in until the competition begins officially (after which the designer is barred from tweaking his machine). As a result, it would be impossible for the designer to create Human Subcognitive Profiles for every possible domain that the machine might find itself playing in a Tournament.

This same effect could perhaps be wrought in a standard Test by restricting the domain of conversation, but not telling the players before the competition begins what that domain will be.

b) In order to be successful, players have to be good as both teachers and students. As mentioned above, this is akin to holding both a standard and Inverted Turing Test. Even if the domain was known in advance, and even if it was possible to draw up a Human Subcognitive Profile for that domain somehow, such a machine would be exposed as a student.

Lastly, French asks whether the standard Turing Test might be modified to forbid the kind of subcognitive questions that underly its cultural and species-specific parochialism. He concludes that the kinds of questions that probe "intelligence in general ... are the very questions that will allow us, unfailingly, to unmask the computer".

He may well be right. However, it may be that moving out of the text domain will dramatically reduce the scope of possible subcognitive shibboleths that human teachers could employ. Having said that, there will still be many possibilities for rules that would place human student-players at a big advantage. For instance, in the case of the Go domain, a cunning human teacher could choose to play by the rules of Connect4, which other humans might be much quicker to fathom. In the case of some kind of Photoshop canvas domain, humans could spell out words cursively, outwitting even the most seasoned OCR software. If there's any kind of free-text involved, any of the subcognitive tricks designed for the standard Turing Test might be employed. In the case of a 3D virtual environment, human student-players will have a huge edge, though perhaps 2D or high-D worlds would level the playing field. One might hope that imaginative specification of domains could minimize such advantages, and that after 10 years of such competitions, machine programmers will almost certainly know to build in pre-loaded expert knowledge of Connect4, for instance, but the problem will clearly still remain.

[N.B. In order to ensure that the scales aren't conversely weighted too heavily against human players, it seems reasonable to allow all human players the use of a laptop throughout the Tournament.]

Maybe instead we should accept the possibility of subcognitive shibboleths, and embrace their utility instead as a means of cataloguing different kinds of conceptual schemes. There is a presumption inherent in the standard Turing Test that smartness can be measured on a one-dimensional continuum ranging from rocks to rocket scientists. In the case of the aliens that have travelled 4 million light years in a space ship built out of genetically-engineered quantum nanobits and powered by fermented mango juice, we could be pretty sure they're intelligent, even if they were never to get the hang of English. It's just that their conceptual schemes are different. In this case, we may find that there are cases where they think more like machines than like humans. Or possibly more like dolphins, or African grey parrots, or white mice or marmosets. If we're able to set up a domain in a Tournament that everyone can play in, then we can expect that human student-players may not necessarily come out on top in all respects, even within the animal kingdom. We will return to this idea when we discuss Turing Tournaments between groups of individuals.

Devising new rules, and non-linguistic competitors

Besides extending the domain of play beyond text, the principle innovation of the Turing Tournament is in casting every player as both student and teacher.

It is clear enough what is required of the student player. When the bout begins, they have some idea of the kinds of interactions, puzzles and patterns that the domain presents. By interacting with the teacher player, they have to somehow fathom what the aim (i.e. terminating criterion) of the current bout is, and attempt to satisfy that. It might involve placing pieces on the board in some complex pattern, learning the structure of a maze, guessing at the next number in a sequence or optimizing some noisy function. Depending on the tournament, they may or may not be given feedback after each move:

If they're given a running score, they can attempt to learn how to maximise that reward.
If no reward is given, but the teacher corrects incorrect moves, then the learning by imitation can be seen as a kind of supervised mapping or reconstructive learning problem.
There may even be cases where no feedback is given whatsoever, such as when the bout requires the student to guess the next number in some sequence.

It is the teacher's job to come up with new and inventive rules for bouts that challenge the student-players, and also to perhaps lead the student in the right direction. For the Tournament to work as intended, teachers should be intending to come up with the most discriminative bout rules they can.

Getting the incentive structure for the teachers right is therefore key. I expect that early scoring structures will contain loopholes that ingenious machine designers can exploit, but that over time, scoring structures that serve their purpose in a robust way will emerge. If our goal is to discriminate humans from machines, then this simple scoring system may work:

If the student 'wins' (i.e. satisfies the terminating criterion) a bout, whether human or machine, then they get a point, otherwise they get nothing.
If a human student wins a bout, then the teacher gets a point, otherwise they get nothing
If a machine student wins a bout, then the teacher loses a point, otherwise they get nothing.
Total player score: the sum of the player's scores as a student and their scores as a teacher
[There may need to be some weighting/normalization if the number of human and machine players is unequal.]

In effect, we're rewarding players that seem human, and can devise rules that discriminate whether other players are human. This Tournament setup is the combo standard/Inverse Turing Test described above, that would not differ all that wildly in principle from the standard Turing Test if played in a text domain. Such Tournaments would encourage the kinds of subcognitive or culturally-rooted human-parochialism that we're trying to avoid.

Perhaps this more general scheme will work instead:

If the student wins, whether human or machine, then they get a point, otherwise they get nothing.
To calculate the student's score: at the end of all the bouts, count the number of bouts that each player won as a student. Calculate the mean number of bouts won. For each student, subtract this mean value from the number of bouts they won. This will mean that a very average player will have zero student points, a good player will have a positive number of points, and a poor player will actually have negative student points.
In other words:

c_m = sigma_n^N( W_nm ) - sigma_n^N( sigma_m^N( W_nm )) / 2N

where:

c_m = the overall student score for player m

N = the number of players

W_nm = 1 if student m won in their bout with teacher n

W_nm = 0 if student m lost in their bout with teacher n
To calculate the teacher's score: if the student wins a bout, then add the student's student score (which may be negative) to the teacher's teacher score. If the student loses the bout, then the teacher gets nothing.
In other words:

p_n = sigma_m^N( W_nm * c_m )

where:

p_n = the overall teacher score for player n

N = the number of players

W_nm = 1 if student m won in their bout with teacher n

W_nm = 0 if student m lost in their bout with teacher n

[It may be that W_nm should be -1 if the student lost]
Total player score: the sum of the player's teacher score and student score
[There may need be some normalization to ensure that the teacher and student scores are weighted equally.]

What's the point of all this complexity? If you're a teacher, then you do best if you can design your rules such that only above-average players (whether human or machine) win in your bouts. There's actually a penalty if you make your rules so easy that everyone can figure them out, and you'll get zero points if no one can figure them out at all. When you're a student, you want to be as smart as you can, and when you're a teacher, you want to be as discriminative as you can. En masse, the community of competitors are striving to do as well as they can and to evaluate each other as well as they can.

Inventively devising rules to favour intelligent over non-intelligent participants requires sufficient representational power to understand, let alone manipulate, your own rules, a rich theory of mind, as well as a generative good taste. Consider a Tournament played in the simple domain consisting solely of letterstring analogy problems, where the student is faced with problems such as:

"I change abc into abd. Can you 'do the same thing' to ijk?"

or in non-linguistic terms:

abc —> abd; ijk —> ?

Reasonable responses include ijl, ijd, or even abd.

Let us imagine that a player as cunning as Douglas Hofstadter has devised the following problem:

abc —> abd; mrrjjj —> ?

Peer at this for a moment - you won't appreciate that this is somewhat fiendish unless you try it for a while yourself. Any ideas?

There's no obvious pattern to the letters chosen on the right hand side, so mrrkkk seems kind of lame, and abd always feels lame. Well, how about if you try this one first:

abc —> abd; abbccc —> ?

Though your first thought may have been abbddd, doesn't abbdddd seem so much nicer? It's as though the successorship sequence of letters needs to be reflected in the increasing length of the letter groups (to use the FARG's terminology). Now, let us turn back to:

abc —> abd; mrrjjj —> ?

Doesn't mrrjjjj seem like a nice, reasonable solution now? Would you have considered it so nice before the previous example. Probably almost as nice. Would you have thought of it on your own, without the previous example? Probably, given some head-scratching.

The point of this digression is to point out how an imaginative teacher can guide, plant ideas, manipulate, prime, coax and lead the student by example in such a way that an intelligent player would almost certainly get the right answer, but there are almost no extant machines that would stand a chance. Besides having the sheer representational flexibility to deal with even barebones analogies such as the one above, a really intelligent player would be using the first few turns to gauge the teacher, get a sense of what kinds of solutions are admissible, and would probably be relying on Gricean maxims wherever possible.

What if your alien doesn't know anything about Gricean maxims? Or doesn't understand concepts like tournaments, rules, intelligence, machines, scores or games? We've finished outlining how a Tournament might be run that might require less domain knowledge and linguistic ability than the standard Turing test. But one striking pragmatic problem remains, which becomes apteacher when we consider our newly-arrived green visitor. If the alien doesn't speak English, how are we going to explain the idea of the Turing Tournament to him so that he can participate?

Following Minsky, I think that we will be able to converse with aliens to some degree, provided they are motivated to cooperate, because we'll both think in similar ways in spite of our different origins. Every evolving intelligence operates within spatial and temporal constraints, suffers from a scarcity of resources (and presumably, competition), must develop symbols and rules, and must have thought about computation and machine learning in order to be able build spaceships. Perhaps notions of games, intelligence, scores and tournaments are only relevant in a society of individual entities that compete with each other for resources, and that maybe a hive mind or single monolithic being or other unimaginable entity might not need such concepts. In that case, you wouldn't have any more luck using the standard Turing Test on such a being.

Will we have much more luck with machines? Not unless we start small. At the moment, the state of the art in artificial intelligence wouldn't do very well in most of the domains we've discussed, and would struggle especially when trying to generate new rules of its own. Sadly, very few researchers have focused on generative heuristics to curiously discover things that are interesting simply for their own sake, such as Lenat's Automated Mathematician that sought interesting mathematical concepts. In order to stand a chance in a Turing Tournament, much work needs to be done on curiously discovering interesting things that could serve as the basis for a rule set in a new domain. Good, that is, discriminative, rule sets for a Turing Tournament bout might consist of a difficult but ultimately guessable sequence of numbers based on a funny arithmetical pattern, or the kind of letterstring analogy problem that elicits an 'aha'. Better still, teacher players that can lead an intelligent student player down a suggestive road towards the terminating criterion through tutorial or warm-up sub-bout problems will be at a tremendous advantage, where half the problem for the student consists of figuring out what their goal is supposed to be.

But is it intelligent?

Let us recall Shieber's pithy test for intelligence:

"Now, how do you tell if something is intelligent? You compare it with an entity postulated to be intelligent."

We've replaced that with an intellectual obstacle course. As teachers devising rules for their bouts, we are effectively asking players to define their own micro-test of intelligence (since being able to do this is surely a sign of good taste?). They must then be able to convey the parameters of that test such that other intelligent student players can figure out how to pass it, perhaps by creating lead-up sub-bouts, internalizing what the student player is probably thinking and so guiding the student players' intuitions in the right direction. Finally, as students, the players must demonstrate in turn that they can flexibly assimilate what their goal should be, and then be able to get to it.

So although we might imagine some narrow machines that could best humans in certain kinds of puzzles or computation, but it seems less likely that a brute force machine player would also do well on Bongard problems, letterstring analogies, or be able to devise ingenious, fun and discriminative rules for bouts. This new generative aspect is intended to tap into the kind of creative, generative, playful, inventive or aesthetic faculty that humans display, as well as the ability to form a rich internal model of the student player's state of confusion, and guide them towards a solution. In this respect, it borrows the idea of a dialog or gentle interrogation from the original Test, but allows for the translation of that dialog to new domains.

Bringing this back to Turing's original question, we can finally ask, 'if a machine were to score higher than some of the humans in a Turing Tournament, would we definitely be willing to call it intelligent?' The answer could depend on a few factors:

Let us assume that the Tournament is well-planned, that the human competitors are well-chosen, that no independent experts can find any scoring loopholes or weaknesses in the organization of the Tournament, and that the result is replicable. If any of these conditions are not met, we will not consider the Tournament to be well-run.
If the domain is too restrictive, then there may be a dearth of interesting rule sets that can be devised. In this case, a good player won't do much better than a poor player, and this wouldn't be an interesting result.
Even if the domain is a rich one, such as letterstring analogy problems, it could be that a highly specialized program like Copycat could outperform many humans. Unless the success is relatively domain-general, then you've shown what you probably knew already, i.e. the machine is exhibiting some domain-specific proto-intelligence.
At that point, we would probably want to analyze the machine's performance. Did it do better as a teacher or student? Was it simply very good at certain kinds of problems? Was there some simple trick to its way of devising problems which, once exposed, would clue in future human players in a rerun of the Tournament?
Could it pass a standard Turing Test?

Let us imagine that a machine is designed which is a poor teacher player, but a good student player, particularly in a couple of abstract limited-interaction domains like letter strings, number sequences, Go boards and cryptography, but that it can't pass the Turing Test. Is it intelligent? Somewhat? We've forfeited the no-frills and no-free-parameters yes/no answer that you get from a Turingf Test, but we now have a much richer set of data with which to try and place this machine in the space of all possible minds. We have a more finely-graded multi-dimensional scale. Our machines can bootstrap themselves by competing amongst themselves without human intervention - specialist teacher machines that are good at discovering generative heuristics can be used to train specialist student machines that are good at problem solving, and vice versa. So in forfeiting our neat yes/no answer, we've gained a great deal.

Perhaps most importantly for the field of AI, we can now attempt to scale the enormous subcognitive iceberg of the mind incrementally, using ever more complex Turing Tournaments as yardsticks. In time, perhaps this will lead back towards the Turing Test as the final summit.

Paul Graham, Joel Spolsky and Steve Yegge and the Law of Increasing Returns

2006-04-16T23:08:00.000-05:00

I've read almost everything these three guys (PG, JS, SY) have written. I think it's because I get an unshakable feeling of rightness and convergence when I read their stuff that I've been trying to pin down. Some fairly obvious commonalities between them include:

They have brains the size of planets, write really well and have strong views about programming languages.
They feel similarly about procrastination, that programmers need nice work environments, good tools and their own office, and about the evils of task-switching.
PG & SY are both smug lisp weenies, and make pretty convincing cases that Lisp is the way forward and although JS isn't as convinced, he may be changing his mind
PG & JS are both successful entrepreneurs, though they don't always agree on the details.

But most of all, I think the key tenet that binds them together is an awareness of the Law of Increasing Returns. They each buy into the idea that:

a really smart person
a powerful programming language
a beautifully-architected office
an uninterrupted 3-day period

is worth 10

Joes
Blubs
cubicles
half-hour slots between errands.

PG's essay on taste is perhaps the most ardent tribute to the Law of Increasing Returns. He catalogues the hallmarks of good design, and though he doesn't say it, the key point of all this is that they add non-linearly. I'm still thinking about this.

He doesn't say much about how one can hone one's taste. I think there's a Vonnegut quote to the effect that the only way to learn to tell good painting from bad is to look at thousands and thousands of good ones, and it will become obvious to you.

Interestingly, while I was trawling for links for this essay, I noticed that the three of them read each other:

Joel uses Reddit, a startup that grew out of Y Combinator. Update: hah! The first time I went to Joel's reddit page, I saw a link to Stevey's blog rants
PG actually appears in JS's documentary
Stevey's Being the averagest post was inspired directly by Paul Graham's Beating the averages

I feel a little less clever now that it's clear that a lot of other people are reading all of them too:

Direct comparison between PG and JS
Google search for all three names

Procrastination

2006-04-15T22:59:00.000-05:00

Reading about how to avoid procrastinating is amongst my very favourite ways of procrastinating. It's a lot easier than whatever you're supposed to be doing, and neatly neutralises the guilt that you'd otherwise feel with a seductive promise that in the long run, this will prove to be the most useful hour you've ever spent.

There appear to be at least two main schools of thought regarding procrastination. There are certainly those who treat it as an evil that can be combatted, either head-on or deviously, but there are also those that embrace some degree of procrastination in the service of sifting project-wheat from errand-chaff.

There are people who spring out of bed at 5am, chanting 'get thee behind me, Slashdot', who are all too willing to tell you how to 'maximise your productivity'. Steve Pavlina's intoxicating account of how he ostensibly graduated from college in CS in three semesters is the best example of this. Look how easy life is if you don't waste any time whatsoever, he whispers to you. He's either making it all up, or a superman, but he does tell an interesting story. And his polyphasic sleep experiment is worth a read.

Then there's this bit of mental judo for using procrastination as a force for good. Basically, the idea behind 'structured procrastination' is this:

"Procrastinators seldom do absolutely nothing; they do marginally useful things, like gardening or sharpening pencils or making a diagram of how they will reorganize their files when they get around to it. Why does the procrastinator do these things? Because they are a way of not doing something more important. If all the procrastinator had left to do was to sharpen some pencils, no force on earth could get him do it. However, the procrastinator can be motivated to do difficult, timely and important tasks, as long as these tasks are a way of not doing something more important."

Continue procrastinating. But instead of reading The Onion, procrastinate by doing something you'll have to do eventually. This may not be the thing you should be doing most of all, but it's better than nothing. And it won't feel as much like work, because you still get to feel that you're avoiding the thing you're not supposed to be avoiding. Everyone's a winner.

In opposition to this idea, Paul Graham argues that there are good and bad forms of procrastination:

"There are three variants of procrastination, depending on what you do instead of working on something: you could work on (a) nothing, (b) something less important, or (c) something more important. That last type, I'd argue, is good procrastination."

He and Joel Spolsky are in remarkably close agreement on this (and related issues). Difficult and important things, like research, need big chunks of time and get completely minced by interruptions and any kind of task-switching. If blowing off a few errands means that you don't get knocked out of the zone, and work solidly on a hard problem for three days straight, then that's the way to be. And often, the things that you're procrastinating about will disappear of their own accord - that's a sure sign they weren't that important to begin with.

I think there's a final point to remember about procrastinators, as people. It is possible to be very successful and still procrastinate horrendously. For this to work, you need constructive panic. People who constructively panic thrill a little in the throes of that total focus you get when you realize that you have no time left to waste. You have exactly as much time remaining as you need to get things done, assuming you sleep as little as humanly possible, and view the whole world through a hole the size of a pinprick with the unblinking eye of your deadline staring back at you. Procrastination brought you here, and constructive panic will get you out.

House MD

2005-06-30T20:38:00.000-05:00

House MD is the finest show on TV. And Batman Begins was really good.

I loved Ender's Game and Speaker for the dead even more, but this article about his views reminded me that he's a troublesome man and it's a strange book. It's weird, because I loved the books, and yet now I have this niggling doubt that I was duped somehow.

Things that are still crap about linux

2005-06-30T20:37:00.000-05:00

This should be taken with a pinch of salt because I'm still running RedHat 8, though some things don't appear much better for the Enterprise version

package manager - I hate downloading new software, because of dependency hell

X11 - I hate trying to SSH into my lab workstation from anywhere, because I invariably get weird authentication weirdnesses

why doesn't it have a nice task manager like windows?

have they sorted out wireless yet?

what about a media player for .wmv files etc?

there should only be one copy/paste clipboard

why isn't konqueror designed to behave exactly like Windows Explorer?

Update (April 2007): Ubuntu's solved most of these issues, but there are still niggles.

Google desktop search

2005-06-30T20:34:00.000-05:00

Why does the google desktop search suck so much? It's the only google product I hate. It regularly fails to find things that definitely do exist, its index doesn't update very fast so it's continually telling me that files that it's produced for me in the search results don't exist, it doesn't seem to do a very good job of searching inside PDFs, there's no way to tell it where to update, almost all of the plugins that I tried to download didn't work somehow...

Why isn't there an easy way to tell google or mozilla or something to download absolutely everything I view in my browser to my hard disk for local searching later? Am I the only one that depends on the information that I view online, but doesn't trust it to be there when I really need it? e.g. doing a presentation when the wireless doesn't like my MAC address, on a train, in my mum's house etc.

Update: the Slogger Firefox extension does exactly this.

Google as my permanent online user id?

2005-06-30T20:32:00.000-05:00

Google is surely now in a position where I just log into google on a public terminal and it automatically logs me into whatever sits require logins, tracks my online presence, prefetches things I'm likely to need (having saved them on a server farm somewhere local to me) and allows me to sync all this with my home PC when I get back

Wardriving in Windows

2005-06-30T20:30:00.000-05:00

Why doesn't Windows come with wardriving wireless software?

Update: I think that there's a check button in the 'View wireless connections' options that says it can automatically connect to unsecured connections that comes close.

Hard disk acceleromotors

2004-09-06T20:52:00.000-05:00

Why don't hard drives all have inertia/accelerometers, to park the heads whenever the drive moves at all - or are they safe for small amounts of movement?

Update: Thinkpads and Powerbooks have this now.

Philosophy as debugging

2004-09-06T20:31:00.000-05:00

Philosophical argument is kind of like debugging a program. You try and zero in on the source of the error, which is why you try and modularise the argument, provide test cases, see where you agree and disagree, and often it comes down to wrongful initialisation, a step accidentally added or omitted, or a failure to see the implications of some interaction you'd never fully considered.

Cringely on the RIAA

2004-08-24T13:28:00.001-05:00

Richard pointed this Cringely piece out to me. Interesting idea for getting round the RIAA. Nobody's done it though, so perhaps it wouldn't have worked out legally after all.

It does seem too easy somehow. And not the kind of progressive, gradual solution that wouldn't lead to a destructive implosion in the music industry and subsequent dark ages. Or maybe it would lead to a flowering of new songs and unfettered artists. I don't know.

For a moment I thought my wisdom was spreading by word of mouth

2004-08-24T13:26:00.000-05:00

David M%27s Page for Computer Science 828J

Brain and Computers

These people reference papers I wrote! Unfortunately, they don't appear to have actually read them.

Electronic voting

2004-08-20T10:42:00.000-05:00

I've been thinking a lot about direct democracy - see the post elsewhere. But right now, the whole electronic voting idea is in bad shape.

http://www.thenation.com/docprint.mhtml?i=20040816&s=dugger
http://www.blackboxvoting.org/

Michael Behe, 'Darwin's Black Box'

2004-08-20T10:31:00.000-05:00

I've been reading a provocative book by Michael Behe called Darwin's Black Box. In short, he's arguing that Darwinism goes a long way to explaining why the various forms of life are the way they are, but is completely unable to address some of our questions and issues, specifically how life arose in the first place and how a number of low-level biochemical structures and systems came to be.

His argument hinges around the idea of irreducible complexity. Take the example of a mousetrap - a mousetrap is irreducibly complex because you need a base, a hammer, a spring, a catch to hold the hammer back and some cheese to tempt the mouse with. If you get rid of any one of those components, you have a wholly non-functional mousetrap. In fact, if the base isn't sturdy enough, the hammer heavy enough but not too heavy, the spring springy enough etc., then the thing probably won't work either. It is irreducibly complex because it requires a number of special components to be together, configured correctly and each meeting certain criteria, otherwise you have a paperweight that would not pass on its genes. There is no way to start with just one or even two of those components lying around and progress through a monotonically beneficial series of minor mutations to get to a mousetrap.

He tries to show in a series of unnecessarily detailed but very readable chapters that e.g. the clotting of blood, the immune system, cellular cilia and flagella (propulsion mechanisms) and vesicular transport are all examples of very complicated and irreducibly complex mechanisms at too small a level for science in Darwin's time to know about. It would be like an internal combustion engine evolving. And he points out that even if you got a working combustion engine going, unless it goes at a certain minimum speed (say tens or hundreds of revolutions per minute), you might as well not bother.

Sometimes he overstates his case. The immune system example is interesting, but you might imagine that the original immune system started with just a handful of hardwired responses and *somehow*, god knows how, it became more and more general. But his point still stands that no one has a really good, convincing answer or even reassuringly specific set of speculations about how the antibodies, t cells, b cells, the cells that puncture unwelcome invaders etc. all came to be at once. Without a minimum functioning set of interacting components, you have roadkill - and it's impossible to imagine how to get from rocks past roadkill in a series of small, always-beneficial mutations.

I haven't finished the book yet, but very nearly. What I'm curious about is where he's going to leave the reader. It really seems as though he's pushing for god, or at least some kind of godly 'intelligent designer' to step in, since he's quite certain that no form of evolution of natural search could do the job. I'm reluctant to accept this conclusion. I can't help but feel that this tells us that the DNA programming language is cleverer still than we thought. On the one hand, nature's machines are gradually transformable, robust to damage, distributed and parallel like the brain, and yet on the other hand, they can involve long strings of essential components that interract in irreducibly complex and fragile ways.

My only response at the moment is to try and think about how these flagella and immune systems and biochemical cascades get represented in DNA. On the one hand, they have to be represented in a precise, error-free, symbolic way, and on the other hand, they have to be mutable and robust so that tiny copying errors lead to beneficial mutations much more often than chance would have us believe. Because, at root, though Behe fails to really put it in these terms, he's arguing that an evolutionary search through a space large enough to encompass the kinds of biochemical mechanisms he discusses would simply be too too large unless there was some very powerful representation or clever pruning or mid-life self-organisation going on. He briefly discusses Kaufmann's ideas about complexity and catalysed self-organisation, but he doesn't really understand them, and neither do I, and he says that they're mathematical models rather than actual, nitty-gritty biochemical stories.
I keep coming back to the idea of DNA as a very clever, very high level programming language (at least some of the time) - in such a hypothetical language, if we had rules to say that any legal source code constructs a relatively viable creature, and the mutations are such that only legal source code can come from legal source code, then you could mess with it all you like and you'd still have lots of organisms that more or less worked. It wouldn't be like adding random characters in the middle of your C++ source code, and hoping they fix your bug and double the performance. However, it looks as though the mutations are more often than not random - the cosmic rays aren't very discriminating. And it makes no sense to imagine that there could be a programming language whose legal source code only ever built viable organisms. That would be like a language that only ever produced true and interesting statements. But one might imagine that if DNA is a high-level programming language, then it could have found ways to program its own copying mechanisms to ensure that a mutation gives rise to legal code more often than not (e.g.by ensuring that the crossover points happen in roughly the right places), and that that source code operates at the level of objects (like classes in C++ or Java) so that you can build increasing levels of abstraction. So you might imagine a line of DNA++ that says 'if A and B then not C' or 'if B>5, then do X'. That way, messing with the contents of the slots, i.e. which objects go where, you could relatively rapidly build up a few lines of code that do something cool, e.g. only start to clot when there's tissue damage and not that much clotting already. When that works, you treat that paragraph as an object/function by marking the start and end with a tag that says "useful code - don't mutate unless you're feeling really capricious", and then build a more sophisticated clotting mechanism around or on top of that that facilitates healing once the immediate danger of bleeding to death is over.
In the case of bacteria, you have absolutely shitloads of them wandering around, and if you're mutating at the level of swapping functions and objects in and out, rather than individual characters, then you could search through a pretty large source code space pretty efficiently, building increasing levels of abstraction in small steps. Moreover, if one bacterium solves the problem of vesicular transport, and the other one figures out how to propel itself, you just combine the two bits of code together.

What's the alternative? God? A planet that's much older than we realise? Some proto form of life that solved all the biochemical problems for us, then died out or went to live like a hermit in the middle of a volcano? Aliens? Could our hindbrains be unconsciously directing things from above? Nah.

Google goes public

2004-08-19T22:46:00.000-05:00

I didn't buy any Google shares because I don't have much money to risk, but I think I would have if I had. I have faith in Google, that goes beyond the reasoned.

Google motto
-
Never settle for the best
Focus on the user and all else will follow
It's best to do one thing really, really well
Fast is better than slow
Democracy on the web works
You don't need to be at your desk to need an answer
You can make money without doing evil
There's always more information out there
The need for information crosses all borders
You can be serious without a suit
Great just isn't good enough

Peer pressure

2004-08-19T22:34:00.000-05:00

"I remember friends over describing one guy as the type "who would put his foot in the toilet and pee down his leg in order to be quiet about it"

Stolen wholesale from kur05hin.

Fahrenheit 9/11

2004-08-19T22:31:00.000-05:00

I went to see this and thought it was great. I wandered out wondering how anyone could vote Bush after having seen it. It seems quite possible that if enough people see this, it could swing the election. But Moore's journalism is regrettably far from impeccable.

http://www.moorewatch.com/

Having said that, the Left needs someone vocal, persuasive and with wide reach. I'm not sure that I agree with this quote, but I am starting to agree that what objective journalism there is has failed to get the message across.

"Some people will say that words like scum and rotten are wrong for Objective Journalism - which is true, but they miss the point," wrote Thompson. "It was the built-in blind spots of the Objective rules and dogma that allowed Nixon to slither into the White House in the first place. He looked so good on paper that you could almost vote for him sight unseen. He seemed so all-American, so much like Horatio Alger, that he was able to slip through the cracks of Objective Journalism. You had to get Subjective to see Nixon clearly, and the shock of recognition was often painful."

Googling for Michael Moore

[broken link]