I've spent twenty years making a decent living inventing. The question I'm asked most often is some version of "Where do the ideas come from?"
Even after two decades, I still don't have a satisfying answer. All I can really do is quote my friend Chris Luomanen and say that I've learned to smell promising ideas. As with most pursuits, practice, exposure, and intention produce discernment.
But show don't tell, right? Rather than bloviating about invention, I'm going to write up a project I think embodies the principles of a healthy invention process. It starts, as most interesting ideas do, with random input.
1. A new challenge
I started Direct Objects at ATAP in 2018 with a clear idea of the theme I wanted the team to pursue—Google's smarts in everyday objects—but had no defined project ideas. My boss had offhandedly boasted to a friend in the gaming industry that ATAP's Jacquard tags could easily power a magic wand game accessory, no problem. The Jacquard team was understandably busy working on things that weren't completely asinine, so making good on his swagger fell to me.
2. A coherent story
The more I thought about wands, the more they seemed like the Platonic embodiment of the not-a-deviceness I wanted my team to pursue. Wands don't have buttons or screens or apps or Bluetooth—they tend to be made of found organic materials—yet they still act at a distance upon the world. Magic, in my case (and with apologies to Arthur C. Clarke), would be indistinguishable from sufficiently advanced technology. That provided a satisfactory narrative and conceptual jumping off point for Direct Objects.
3. A technical insight
A magic wand is essentially a laser pointer that writes weird letters on a faraway surface. And 2D stroke recognition is largely a solved (and low-latency) problem!
Wands help magicians cast spells. Given that Jacquard tags have only a gyro and an accelerometer—no mic, no camera, no other motion sensors—our wand's powers would have to rely exclusively on wand motion, and thus on reconstructing wand motion using drifty IMU data.
Viewed through the lens of interaction design, magic is idiosyncratic and immediate. Initial experimentation led us to conclude that a heuristic approach to spell recognition was unlikely to yield consistent results across users, and that an ML model trained on 3D IMU data would struggle to distinguish between more than a handful of distinct gestures without very noticeable latency.
