Information Engineer

Few have heard of Numenta, a small company located in Menlo Park, CA, that was founded in March 2005 by Jeff Hawking (inventor of the Palm Pilot and author of the book, “On Intelligence”) and Dileep George (in addition to being a co-founder of Numenta, he is also the principal architect), that is devoted to modeling neocortex through Hierarchical Temporal Memory (HTM). The concept of HTM is discussed in Hawking’s book, “On Intelligence”, but even more informative is a recently released white paper entitled “Hierarchical Temporal Memory: Concepts, Theory, and Terminology” from the Numenta website that describes more details about HTM. Here is an excerpt:

Hierarchical Temporal Memories (HTMs) are unlike traditional programmable computers. With traditional computers, a programmer creates specific programs to solve specific problems. For example, one program may be used to recognize speech and another completely different program may be used to model weather. HTM, on the other hand, is best thought of as a memory system. HTMs are not programmed and do not execute different algorithms for different problems. Instead, HTMs “learn” how to solve problems. HTMs are trained by exposing them to sensory data and the capability of the HTM is determined largely by what it has been exposed to. HTMs are organized as a tree-shaped hierarchy of nodes, where each node implements a common learning and memory function. HTMs store information throughout the hierarchy in a way that models the world. All objects in the world, be they cars, people, buildings, speech, or the flow of information across a computer network, have structure. This structure is hierarchical in both space and time. HTM memory is also hierarchical in both space and time, and therefore can efficiently capture and model the structure of the world. HTMs are similar to Bayesian Networks; however, they differ from most Bayesian Networks in the way that time, hierarchy, action, and attention are used. HTMs can be implemented with software on traditional computer hardware, but it is best to think of an HTM as a memory system.

Numenta is an example of a small, agile company densely-packed with talent and motivation. It’s an excellent idea that they are proposing (i.e., extending Bayesian networks and Belief Propagation to model human neocortex) that is theoretically sound, biologically plausible, and that can be implemented in computer and related technologies. It is useful to compare the small, agile Numenta to the monolithic Allen Brain Project and the Blue Brain Project. My contention, born of experience, is that giant projects with too much money encourage laziness and dull thought. You see this at the Allen Brain Project and the Blue Brain Project. On the other hand, projects like Numenta, which are lean, mean, and hungry for results, are almost guaranteed to succeed. My prediction is that Numenta will be a big success.