"I shot the cat with my proton gun."

I often listen to lectures and audiobooks when I drive more than 2 hours because I don't always have the privilege of enjoying a good conversation with a passenger.  Recently I was listening to some philosophy of science podcasts on my iPhone while driving from Boston to New York when the following sentence popped into my head:

"I shot the cat with my proton gun."

I had just listened to three separate Podcasts (one about Kant, one about Wittgenstein and one about Popper) when the sentence came to my mind.  What is so interesting about this sentence is that while it is effortless to grasp, it uses two different types of concepts in a single sentence, a "proton gun" and a "cat."  It is a perfectly normal sentence, and the above illustration describes the sentence fairly well (photo credits to http://afashionloaf.blogspot.com/2010/03/cat-nap-mares.html for the kitty, and http://www.colemanzone.com/ for the proton gun).

Cat == an "everyday" empirical concept

"Cat" is an everyday "empirical" concept, a concept with which most people have first hand experience (i.e., empirical knowledge).  It is commonly believed that such everyday concepts are acquired by children at a young age -- it is an exemple of a basic level concept which people like Immanuel Kant and Ludwig Wittgenstein discuss at great length.  We do not need a theory of cats for the idea of a cat to stick.

Image from shadowpaw99

Proton Gun == a "scientific" theoretical concept

On the other extreme is the "proton gun." It is an example of a theoretical concept -- a type of concept which rests upon classroom (i.e., "scientific") knowledge.  The idea of a proton gun is akin to the idea of Pluto, an esophagus or cancer -- we do not directly observe such entities, we learn about them from books and by seeing illustrations such as the one below.  Such theoretical constructs are the the entities which Karl Popper and the Logical Positivists would often discuss.  

While many of us have never seen a proton (nor a proton gun), it is a perfectly valid concept to invoke in my sentence.  If you have a scientific background, then you have probably seen so many artistic renditions of protons (see Figure below) and spent so many endless nights studying for chemistry and physics exams, that the word proton conjures a mental image.  It is hard for me to thing of entities which trigger mental imagery as non-empirical.  

How do we learn such concepts?  The proton gun comes from scientific education!  The cat comes from experience!  But since the origins of the concept "proton" and the concept "cat" are so disjoint, our (human) mind/brain must be more-amazing-than-previously-thought because we have no problem mixing such concepts in a single clause.  It does not feel like these two different types of concepts are stored in different parts of the brain.

The idea which I would like you, the reader, to entertain over the next minute or so is the following:

Perhaps the line between ordinary "empirical" concepts and complex "theoretical" concepts is an imaginary boundary -- a boundary which has done more harm than good.  

One useful thing I learned from Philosophy of Science, is that it is worthwhile to doubt the existence of theoretical entities.  Not for iconoclastic ideals, but for the advancement of science!  Descartes' hyperbolic doubt is not dead.  Another useful thing to keep in mind is Wittgenstein's Philosophical Investigations and his account of the acquisition of knowledge.  Wittgenstein argued elegantly that "everyday" concepts are far from "easy-to-define." (see his family resemblances argument and the argument on defining a "game.")  Kant, with his transcendental aesthetic, has taught me to question a hardcore empiricist account of knowledge.

So then, as good cognitive scientists, researchers, and pioneers in artificial intelligence, we must also doubt the rigidity of those everyday concepts which appear to us so ordinary. If we want to build intelligent machines, then we must be ready to break down own understanding of reality, and not be afraid to questions things which appear unquestionable.

In conclusion, if you find popular culture reference more palatable than my philosophical pseudo-science mumbo-jumbo, then let me leave you with two inspirational quotes.  First, let's not forget Pink Floyd's lyrics which argued against the rigidity of formal education: "We don't need no education, We don't need no thought control." And the second, a misunderstood, yet witty aphorism which comes to us from Dr. Timothy Leary reminds us that there is a time for education and there is a time for reflection.  In his own words:  "Turn on, tune in, drop out."

Kant's Intuitions, the intentional stance, and reverse-engineering the mind

Immanuel Kant (1724-1804), Daniel Dennett (1942-), Josh Tenenbaum (~1971-)

“Thoughts without content are empty, intuitions without concepts are blind. The understanding can intuit nothing, the senses can think nothing. Only through their unison can knowledge arise.” -- Immanuel Kant

“We live in a world that is subjectively open. And we are designed by evolution to be "informavores", epistemically hungry seekers of information, in an endless quest to improve our purchase on the world, the better to make decisions about our subjectively open future.” -- Daniel Dennett

"For scientists studying how humans cometo understand their world, the central challenge is this: How do our minds get so much from so little? We build rich causal models,make strong generalizations, and construct powerful abstractions, whereas the input data are sparse, noisy, and ambiguous—in every way far too limited. A massive mismatch looms between the information coming in through our senses and the outputs of cognition." -- Josh Tenenbaum

---

Organizing by space (space, time, and physics)

There are two faculties of understanding which it is unlikely we have acquired from experience.  The first is that of understanding objects as extended bodies in a 3D space and thus occupying some volume.  I believe it is Kant argued best against the hardcore British Empiricists, who proclaimed that experience is the sole originator of knowledge.  Experiences are the pen strokes, which fill the Empricisit’s tabula rasa.  Kant argued (against Hume) that the concept of a spatially extended object is not acquired from experience – the very notion of experience requires that we already possess the notion of an object in order to have a meaningful percept.  It is as if the Empiricists failed to acknowledge that to make strokes on a sheet of paper, we need to already have a pen.  Kant’s intuitions are the pens of experience.  The requirement of having suitable intuitions for grouping percepts into experiences is what Kant described as a form of transcendental idealism.  “Objectness” is a faculty of human understanding, not something acquired from experience.  If you are a vision researcher, being aware of this can have drastic implications on your research programme.

It has also been argued that there are some primitive notions of object dynamics, aka folk-physics, which can are possessed by very young children.  Given the uniformity of human experience (at least I have no ostensible reason to double that my colleague’s experiences significantly differ from my own), and the diversity in our individual upbringing, it is also unlikely that folk-physics is learned from experience.  However, I don't want to make any strong claims regarding folk-physics.  I feel safe to say that Quantum Mechanics is another story -- it requires years of mathematics and thousands hours of deliberate problem solving to grasp.

Organizing by mind (psychology, mind, and intent)

The second faculty of understanding, which can be found in many aspects of human intelligence, is that of understanding the world in terms of cognitive agents.  Humans have an amazing capability when it comes to attributing stuff with having a mind.  This way of thinking about the world is so common and uniform among children all over the world, that the differences in their upbringing cannot be reconciled with the uniformity of their capability to project humanness onto objects.  Consider the following video (thanks J. Tenenbaum's videos/lectures for pointing this out).

We cannot just view this video os triangles, dots, and lines.  Each one of understands the story in terms of a narrative based on agents and their intent.  We are stimulated by the external world, we take as input sense-data, and the brain helps us make sense of it -- it turns the hodgepodge of data into experience.  But the brain is a mold, it conforms percepts to some shape defined by the mold.  These molds are the faculties of understanding which let us understand things, it is like the faculties of understanding are basis vectors onto which we project all input sense data.  The data is weak and noisy, the priors are strong, and understanding is the result of their union.  An experience without a proper basis is blind, it is just a ball of percepts.  These faculties allow us to have experience.  The experiences, coupled with memory, allow us to obtain understanding – where understanding is the relationship between a given experience and past experiences, either in the form of direct associations between currently-experienced-objects and previously-experienced-objects, or rules abstracted away from previously-experienced-objects being directly applied to the current sense data.

What I am talking about is what philosopher Daniel Dennett refers to by the “intentional stance.”  Given my background in AI and philosophy of mind, it is very likely that Dennett and I have had the same influences.  I like to juxtapose my ideas with those of the classical philosophers such as Descartes, Locke, Kant, Wittgenstein and Pinker -- I’m not sure how Dennett motivates his philosophy nor do I know against whose ideas he juxtaposes his own stance.

---

At MIT, J. Tenenbaum is pushing these ideas to the next level.  I only wish there was more perception in his work -- toy worlds just don't do it for me.  I want to build intelligent machines, and really cannot afford to sidestep the issue of perception.  Here is a great talk by Josh Tenenbaum on reverse engineering the mind from NIPS 2010. Video is on videolectures.net, just click the link.

Implications for Artificial Intelligence and Machine Vision

Following Josh Tenenbaum, I think that a criticism of classical machine learning is long overdue.  Machine Learning, as a field, has been spewing out hardcore empiricists.  “Let me download your features, my machine learning algorithm will take care of the rest,” they say.  It is like the glory is in the mathematics, which manipulates N-D vectors.  But I argue that intelligence isn’t “in the calculus,” it is what the primitives in the calculus actually represent.  As an undergraduate I proclaimed, “I am not a mathematician, I am a physicists.  I care about the structure of the world, not the structure of proofs. “  As a graduate student I proclaimed, “The glory isn’t in the manipulation of vectors, the glory is understanding the what/why of encoding information about the world into vectors.  I am a computer vision researcher, not a machine learning researcher.”  That is why the view of the world as coming from K different classes is wrong – this is merely a convenient view if the statistician’s toolbox is at your disposal.  It is all about structuring the input to match a researcher’s high-level intuitions about the world. 

Computer Vision is Artificial Intelligence

Computer vision is a diverse field and its researchers have multifaceted interests and aspirations.  It should not be surprising that no two vision researchers think about the field in the same way.  Different academic backgrounds foster alternative and potentially incommensurable interpretations.  It is as if W.V.O Quine's thesis that no observation can be "theory-independent" directly applies to vision: a researcher in computer vision cannot uphold a view on his own field that is objective and independent of their own predispositions, upbringing, and educational program.  While I cannot speak clearly about the long-term goals of the entire body researchers in vision, today I would like discuss my own take on computer vision.  I do not offer the world an objective account of why computer vision intrigues me, but by sharing with the world the reasons why I find vision exciting, perhaps together we can break the boundaries of machine intelligence.

Cognitive Science is a computational study of the mind: McGill Cognitive Science

One of the biggest accomplishments in the field of Artificial Intelligence was when Deep Blue, a chess playing program developed at IBM, beat the world chess champion, Garry Kasparov.  But this was in the early days of artificial intelligence -- when computer scientists still weren't sure on what it means for a machine to be intelligent.  Chess is a well-known thinking-man's game, and at first glance it seems that a machine can only be worthy of being dubbed intelligent if it performs competitively on intelligent-people activities such as chess.

Chess: Human vs. Machine: Slate article about Deep Blue

Given the plethora of tasks that humans can effortlessly perform in daily life, is engineering a machine to rival humans on just one such task bringing researchers any closer to building truly intelligent machines?

The problem with chess is that it has a "finite universe problem" -- there is a finite number of primitives (the chess pieces) which can be manipulated by choosing a move from a finite set of allowable actions.  But if we think of normal life (going to work, eating dinner, talking to a friend) as a game, then it is not hard to see that most everyday situations involving humans involve a sea of infinite objects (just look around and name all the different objects you can see around you!) and an equally capacious space of allowable actions (consider all the things you could with all those objects around you!).  Intelligence is what allows us to cope with the complexities of the universe by focusing our attention on a limited set of relevant variables -- but the working set of objects/concepts we must consider at any single instant is chosen from a seemingly infinite set of alternatives.

I believe that everyday human-level visual intelligence is greatly undervalued by people -- and there is a very good reason for this!  The ability to make sense of what is going on in a single picture is such a trivial and autonomous task for humans, that we don't even bother quantifying just how good we are at it.  But let me reassure you that automated image understanding is no trivial feat.  The world is not composed of 20 visual object categories and the space of allowable and interpretable utterances we could associate with a static picture is seemingly infinite.  While the 20 category object detection task (as popularized by the PASCAL VOC) does have a finite universe problem, the grander version of the vision master problem (a combination of detection/recognition/categorization where you can interpret an input any way you like) is much more complex and mirrors the structure of the external world well.

Robotics Challenge: Build a Robot like Bender

Any application which calls for automated analysis of images requires vision.  A robot, if it is to be successful interacting with the world and performing useful tasks, needs to perceive the external world and organize it.  While some see vision as just one small piece of the "Robotics Challenge" (build a robot and make it do cool stuff), it totally unclear to me where to draw the boundary between low-level pixel analysis and high-level cognitive scene understanding.  Over the years, I have been thinking more and more about this problem, and I've convinced myself that the interesting part of vision is precisely at the boundary between what is commonly thought of as low-level representation of signal and what is considered high-level representation of visual concepts.  While some view computer vision as "applied mathematics" or "applied machine learning" or "image processing in disguise", I passionately believe the following:

Computer Vision is Artificial Intelligence

I am not promulgating the thesis that all aspects of machine intelligence are visual, but I want to assure you that there are enough high-level semantic capabilities which must be set in place for vision to work, that it is not worthwhile to think of vision as smaller problem than general purpose intelligence.  I believe that once we have made progress on vision (not in the narrow-universe setting) to the point where generic visual scene understanding is effectively solved, there won't be much left that needs to go into the "ethereal" mind which cognitive scientists want to empower machines with!  The only way to make machines truly understand scenes, objects, and their interactions is to make machines know something about the fabric of human life, and it is important for machines to learn this for themselves from real-world experience.  This goes beyond representing object appearance because folk physics, folk psychology, causality, spatio-temporal continuity, etc are all faculties which vision systems will need (at least the vision systems I want to ultimately build) for general purpose scene understanding.  I don't want to undermine the efforts of cognitive scientists (which work on many of the theories/ideas I've delineated before), but perhaps only to convince them that I have been a cognitive scientist all along.  I don't think placing a label on myself, by calling myself as either a cognitive scientist, a computer vision researcher, or AI researcher is very conducive to good research.

Beyond Categories =? Doing without Concepts

The term "beyond category," from my limited knowledge, was originally coined to describe the music of Duke Ellington. It is a term of praise that acknowledges that one's style is inimitable and transcends barriers.

"Beyond Categories" was the first part of my NIPS 2009 paper's title. To "go beyond" means to transcend, to abandon or do without some limitation and strive higher -- there is nothing magical about my use of the term. I used the term category to refer to object categories, as are commonly used in computer vision, artificial intelligence, machine learning, as well as psychology, philosophy, and other branches of cognitive science. One of my research goals is to go beyond the use of categories as the basis for machine perception and visual reasoning. It has been argued by Machery that the term category is roughly equivalent to the term concept as used in psychology literature. In some sense the title of Machery's recent book, "Doing without concepts," is analogous to the phrase "Beyond categories" but to reassure myself I'll have to finish reading Machery's book.

So far the first chapter has been a delightful exposition into the world of concepts, a term dear to researchers in machine perception (AI) as well as human categorization (psychology). I look forward to reading the rest of the book, which I accidentally found while looking for Estes' book on categorization. I had already digested/assimilated some of Machery's work, in particular his paper titled Concepts are not a natural kind, so seeing his name on a book at the CMU library piqued my interest. In this 2005 paper, Machery argues that the debate between prototypes vs. exemplars vs. theories in the literature on concepts is not well-founded and there is no reason to believe a single theory should prevail. I'll attempt to summarize some of his take-home messages and their relevance to computer vision once I finish this book.