Is This HARD Or Is This HART?

So how do we humans perceive things around us so uniquely? Hofstadter mentions the phenomenon of conceptual slips of tongue in which we replace a word we meant to say with a conceptually similar one in the very same context. We don't only seem to have one specific word tightly connected to some meaning, but might use different ones to express the same or analogous meaning. Also when we look at other languages, there might be several translations for one term, fitting more or less in a specific context. These two phenomena thrown together occur in my daily life as an exchange student. When every now and then a German word slips into my English sentences or when I throw pieces of English vocabulary or even syntax at my parents on the phone. For example, I will "make" pictures while I am here in the US and as soon as I come home, I will say "Fotos nehmen" (take photos) instead of "machen" (make).

Can machines acquire meaning in a sense humans think they can? Grasp intrinsic properties? Highlight certain facts while disregarding others in certain circumstances or specific contexts? - To answer these questions we have to look at the understanding of meaning in the first place. The meaning of a thing might be all facts that are to know about it, its functional role, associations one might connect with it, in all: its perception in the world.

When computer programs are given only partial facts and properties of an entity, they cannot possibly grasp or understand the meaning of it, according to Hofstadter. But what are we given in the world? A human can never acquire all possible impressions, facts and angles of an object for example, but we attribute her the notion of understanding what the object is with just little knowledge about it, and be it just its name.

I have the impression, that humans are not that different then, let's say, google with a few extras. One can ask the search engine for example "what is a horse?" and it will bring up the internal definition of horse in first place under the search term "define:horse". From there we can switch to picture results of horses, videos of horses, scientific articles about horses, maps of places where horses are present, and so on ... All this information can be and might already be connected to an apt meaning of the term horse. The pictures would suffice to identify 99% of all horses shown to the engine in form of an uploaded picture and some kind of recognition software (which google posesses). The videos could identify the typical motion of horses and the common sounds a horse would make. The facts and common terms associated with horses could help to spot conversations via instant messaging or email, or even voice chat about horses.

Even though google has never perceived a horse like we humans do, it will be able to tell from the context that something is a horse, or the other way around develop a context around the term horse in order to draw an analogy. From my point of view, the perception of the object does not play the biggest role in "understanding" the meaning of it, as we cannot understand it any deeper than computers.

Conceiving Meaning - Meaning Conceiving

Chalmers, French, and Hofstadter point out in their paper High-level perception, representation, and analogy: A critique of artificial intelligence methodology that the major flaw in most artificial intelligence programs lies in the absents of true conception in some modality. In contrast to the human brain does none of the presented programs acquire or rather conceive the intrinsic properties of the given data, neither does any filter out irrelevant, incomplete or partially incorrect data from the vast stream of outside stimuli like a real world observer. Instead, all the data was prerefined and perfectly suiting for the task at hand making it in fact almost impossible not to come up with the anticipated conclusion/analogy/result.

I must admit that logically concluding an analogy or alike from a bunch of connected, alas meaningless predicates sounds much less impressive than the chance of having discovered a way to artificially draw real world analogies. Maybe Hofstadter is a little too harsh on the quite enthusiastic and sensationalist science colleagues since their main goal seemed to be a program which could logically deduce facts and relations logically from real world data. It was not in their interest to compute these findings as much human-like as possible, contrary to Hofstadter's approach. Admittedly, the citations and conclusions from the papers Hofstadter refers to sound very provocative and sensational, portraying the entire artificial intelligence department as some kind of science fiction lab. Therefore studies like Numbo, Copycat and alike sound rather minimalistic and unimportant even though their implications might be much more revealing than examples which might be more applicable to actual situations but lack a foundation of understanding the underlying concepts of intelligence altogether.

Imitating Intelligence Is Not Real Intelligence

The preface to chapter 4 in Hofstadter's book is intended to warn us about the dangers of the ELIZA Effect and the anthropomorphism of computer programs in general. Hofstadter in particular ventilates his dislike for the sensational press some of his colleagues receive with rather unsophisticated and misinterpreted AI programs like ACME, ARCS, SME and the program developed by Scott French. All of them were supposedly used to draw analogies from real life data to imitate human intelligence.

According to Hofstadter all these attempts failed right in the beginning, with the assumption that the programs indeed understood some kind of meaning of the terms (or rather strings) they handled. All approaches so far have only shown, that provided with the essential (and only the essential) data and some kind of rule based, logical system analogies and conclusions can be drawn by a computer. Hofstadter argues though, that the data could be arbitrarily exchanged with different or nonsense predicates and variables, yielding gibberish or even wrong results.

One of the reasons why these refuted scientific discoveries are still around and misinterpreted is a phenomenon called ELIZA Effect. This effect goes back to the chatbot ELIZA developed by Josef Weizenbaum in 1966 which very simply imitated a dull psychiatrist by rephrasing most of the human input into questions, generating a rather unsophisticated dialog mainly driven by the creativity of the inputting human himself. Even though the communication with such a chatbot is pretty obviously shallow and might be downright stupid, some humans ascribe human-like intelligence, emotional motivation and genuine interest to the "answer-questions" ELIZA and other chatbots generate.

Other examples of imitation of human-like text generation that came to my mind when I read the preface to chapter 4 were the Chomskybot, which generates very complicated, scientific-sounding text of arbitrary length including real words and syntax and the PARROT which generates anything between strings of arbitrary letters up to entire nonsense stories readable by humans. Rumor has it that someone actually handed in a slightly altered Ph.D. thesis generated by the Chomskybot and got away with it. PARROT is purely based upon probabilistic occurrences of letters or n-grams within a given corpus.

Numbo the Human Computer

Hofstadter's Numbo is not necessarily supposed to deliver fast and accurate results to the number problems, it's tackling, but rather to mimic the way we come about with a solution. So instead of brute-force trail-and-error of combinations which are doomed to be wrong from the beginning (at least from the human perspective) Numbo tries to juggle with the numbers in a probabilistic way to combine previous particles to likely bonds. This way solutions are found in a much smarter way, disregarding the enormous overhead created in the process.

Hofstadter is not really satisfied with "smart", he wants to compare it to human cognition 1:1. Therefore he compared the trace of Numbo runs to the mental protocols of human players (as good as they could remember or introspect). The comparison reveals certain similarities but also yields different results, since Numbo comes up with easy solutions in a way, humans would not think of first. So it does not seem to be as "smart" after all. Or do we just come up with weird and complicated solutions?

We are given a tiny peek under the hood of the mysterious Numbo-machine and discover that the complexity of human thought process was tried to be modeled by mere probabilistic functions in most cases. In my eyes, this is one fundamental flaw which cannot be eluded that easily. Humans have association with every number, context and method they use. The seeming randomness in their actions might be a result of the current environment and situation they are in. A human might use one technique to solve a problem once and another technique the next time, not because of statistic equivalent of the two approaches, but because all prior events taught her/him to use the one most promising one at the time.

The codelets from the coderack, Hofstadter uses, are supposed to stand for the thought processes we use in turns to come to a solution. They are supposed to contain as few prior knowledge as possible. This is what I see as another fundamental flaw in Hofstadter's approach. I have the notion that we literally learn the results of addition, subtraction and multiplication by heart (at least the easy ones). And from there, we draw analogies between problems like 3+2=5 and 30+20=50 or 24+7=31 and 64+7=71. But there is even a more basic level humans operate on. We associate a certain context and knowledge around numbers or operants and therefore pick those which pop up more easily to us.

In my opinion, Hofstadter is quite on the right track, but still misses a lot of essential processes, naturally going on in the human brain.

The Beauty of Math in TV Shows

In the beginning of the third chapter, Hofstadter remembers the involvement of a Belgian colleague of his during the year 1986, Daniel Defays. He was part of the NARG but at the same time wrote a program called Numbo which would solve the sort of mental math task involved in crypto problems. The idea is to have a series of smaller integers (bricks) and one bigger target number and having to combine an arbitrary number of the bricks with the help of addition, subtraction or multiplication so they would equal the target integer.

The task is very similar to the crypto problem but the (human) computer is forced to partially use different strategies, since the number may be quite higher as in ordinary crypto problems. One of these strategies is rounding, so approximating a value close to the target and trying to go from there. Another is using analogies of the kind that 20*30=600 is similar to 2*3=6 which might be very obvious to the average mathematician among us but will not strike an infant immediately, not to speak of modeling such prior general knowledge to computer programs.

But it was exactly this kind of shortcuts we use in everyday situations that inspired Hofstadter and Defays in optimizing their programs accordingly. I must admit that these strategies, as subconscious as they might seem, are very vital to the speed and accuracy of our thought processes not necessarily only to solving math problems or figuring out words. It is very neat to think, somebody would realize such mechanisms in a real problem solving problem (even though, we are of course let alone in the dark about how this realization came about).

Gloms are Happy when its Cold

In the last part of the second Chapter, Hofstadter carries on his metaphorically enriched idea of gloms, according to which atomic particles (letters) form weak and strong bonds when given the occasion and develop syllable-like structures and statistically likely word formations. He describes this as a parallel at the bottom, but serial at the very top approach in which particles may bind in no particular but rather random or semi-randomly prioritized order. At the end of such a chain of events stands a word-like top-level glom using all the available particles.

From reading this passage about finding a possible solution to a Jumble, I get the notion, that it is not even important that Jumbo spits out a genuine word but rather delivers something, that humans would recognize as pronounceable and possible word. This "word" is then not correct or incorrect in the absolute sense, but might be subject to simple rearrangements to come up with the right solution. Hofstadter seems to feel that this approach is closer to the way how we ourselves solve Jumbles in our heads: Finding possible subgoals and working on the produced structure from there instead of taking all letters apart completely again and starting from scratch. This sounds like a very clever approach to me.

In particular I liked the metaphorical use of temperature and entropy of the particles and gloms to describe their intrinsic state of being in form of happiness or confidence. Similar to physical particles, gloms bond more loosely or hardly at all bound to each other, but collide very frequently with others when hot and in motion. If being in a stable and promising combination with other particles the glom will cool down and therefore break less easily. All this might really describe possible analogues in our minds.

Beyond Brute-Force Dictionaries

In the pattern of letting the reader tip-toe in the dark while shouting: “A little bit more left, than you went backwards just before!” Hofstadter carries on with the Jumble puzzles throughout the second chapter. Jumbo, the program which he leaves us so unclear about, supposedly can solve simple and more complex anagrams by bonding letters, syllables and word parts together and forming ever more probable (but not necessarily more meaningful) chunks.

In metaphorically rich language, the author explains how such bonding between such atomic (letter) or molecule-like entities (syllables or intrinsically well fitting pairs or triplets) would carry out. That potential partners could first spark on sight of each other and eventually bond together if no other potential partner in sight would exert even more attraction. This way, all elements mingle first on a very small and detailed level and would then try to bond again with other more evolved structures further up in the hierarchy.

Yet, it does not seem quite plausible, how Hofstadter wants to realize such ranks of attraction level between the potential bonding partners. He suggests a very subjective approach, by bonding the first atomic elements with the help of his own intuitions. I would have rather suggested a probabilistic approach which had the underlying knowledge of how often certain letters proceed others in a certain language. This can be easily done by consulting just a small sample text and analyzing what letters normally occur in the environment of others.

Maybe Hofstadter is going to go into this, and I am being unjust to him. But I would have wished that he went into more detail of the actual realization than using two metaphors over like 5 pages to describe a process that most people have understood from the previous text.

Numbers Go Up, Words Come Down!

Foreword to this post: The reader might not take the lighthearted humor and hidden allusions contained in this post for a mischievous criticism of Douglas Hofstadters work but rather as a funny commentary about the complementary reading experience.

As well written the first chapter of "Fluid Concepts and Creative Analogies might have been and as thrilling Douglas Hofstadter know how to present the topic at hand, in the end it is merely sequences of numbers and their hidden pattern which seem to absorb the authors interest. Hence, I was relieved to find that he found practical application for such specialized cognitive ability by transferring his expertise to the realm of linguistics. - Well not quite yet, but at least since page 87 we are now juggling with words, or rather letters. But better than 86 pages about numbers anyway.

The task of rearranging letters in a word to form meaningful anagrams naturally involves almost the very same processes as in finding the underlying pattern in sequences of numbers. The proximity to everyday usage and implicitness for humans to learn a language, though, spawns totally new approaches to finding solutions for such kind of problems. Also, this way we are able to apply the found methods to other, more practical problems and formulate strategies which do not ground on sheer numeric templates but rather understandable words, found in a dictionary.

Take for example: P A R L E N I A (Try it!!!exclamation mark!!!)

After some time of memorizing the single letters used in this word, one might easily come up with the partial PLAIN and then we would only have A R E left. There are not too many combinations of order we could put the remaining three letters in: ARE or ERA. Both would make sense alone, but would not really form a proper word, put together with PLAIN. So we shuffle again and come up with other partials.

To stay in the tradition of the author I will not reveal which word I have originally come up with, solely to mock the baffled reader.

Mee-To-Phone-Me-None

In one of the last sub-sections of chapter one, Hofstadter explains the “Me-Too”-Phenomenon, which I haven't actually thought about but struck my interest immediately. The concept of this problem (or rather effect) is that people develop their own independent line of thought during a conversation. Either we come up with our own imagery which by definition has to differ from the other person's, we might even think into a different direction, though starting from the same original thought or we just think in analogous to the described story. The outcome, especially but not exclusively of the latter, can be the so-called “Me-too”-Phenomenon, where we answer according to the presented circumstances but just reflect/project them on our situation.

Example: “My dog used to literally eat my homework.” - “Yeah, mine too.”
Clearly in this example the two persons do not mean the first ones homework but theirs individually in analogy. The “Me-Too”-Phenomenon shows how easy it is for humans to think in analogies, sometimes even without noticing it. Applying the same ability to pattern-matching algorithms for example inevitably has to be a tough job, though.

Making a Program more Human

In the current part of Hofstadters text, he explains his approaches to more complex sequences that may be easy to human cognition but are generally hard to crack by computer algorithms. Sequences for example where patterns or numbers move around in lawful manners (e.g. singlers or bouncing doublers) are generally easy to spot by the human eye, but make hardly any sense if deciphered by counting the occurrences of certain numbers or interchanging sequences.

It is their notion of figure and ground according to the Gestalt laws which make these features so salient to us, having to cope with similar effects in nature. Other such features like plateaus (strings of the same number "1111"), up- and down-runs (continuous up- or down-counting sequences, "2345", "98765") and palindromes (symmetrical sequences, "014410") are also salient to the eye, but had hardly any mathematical relevance in the existing sequence-seeking programs before.

Hofstadters suggests some kind of bottom-up approach for finding such 'islands' (as he calls them) first and making sense of them, by finding their connections in the next step. This approach is by far more natural than the previous methods, the author briefly touched in the first sub-chapters. In my opinion, the notion of similarity should also play an important role among the other mentioned features. Even though this criteria would be much harder to implement, due to its broad definition, similarities among islands or slightly deviating sub-sequences reveal most of the connections between them and therefore useful hints about the generation process itself.

After this part of the book, I am actually sorry Hofstadter does not let us in on the actual realization of Seek-Whence program, because I imagine some details of it even more useful than the mere description of problems it can deal with.


[This blog post has been edited after the assigned deadline, because the original post was embarrassingly short and general.]

Perceiving Patterns

Isn't it surprisingly human to seek for patterns in a chaotic world? Douglas Hofstadter tries to do exactly this with naturally or synthetically occurring sequences of numbers in his book "Fluid Concepts and Creative Analogies". Recognizing patterns in the natural environment around us is a viable part of cognition and Hofstadter goes as far as calling it the potential key to intelligence. It comes into play in everyday use as grouping of letters in written language, sounds to meaningful spoken words, perception of geometric structures according to the Gestalt Laws.

Yet, Hofstadter concentrates on stripped down, purely mathematical sequences of numbers which can mostly be explained by repeating the same generation pattern (in his words: templates) over and over. And even though the mathematical construct standing behind such generative sequences, including arithmetic expressions and numbers, was arbitrarily thought up by humans, this kind of miraculous sense-seeking works for the most random sequences. Therefore it is easy to believe in some kind of higher power like Mathgod ("Zahlengott"), Einstein supposedly described as cagey but not wicked.

No matter how random something might be, if you break down the problem in finite space, there always seems to be a pattern emerging from it. I am just not that sure whether there actually can be patterns to all irrational sequences. Because once you thought you figured out the template behind the nth number of permutations, there might be a ‘n+1’th packet that screws up the previous template all together.

The Thinking Machine

For a long time, intelligence was supposed to be the one thing that set us humans apart from the other animals. But as we come to think about it, there is no such clear-cut definition of the term intelligence. It clearly involves the ability to solve problems in general but also to comprehend one's own environment and reflect this comprehension in appropriate behavior. To come up with entirely new ideas or to draw analogies is certainly the strength of humans. Therefore creativity makes up a great deal of the intelligent human being.

But the most ingenious idea the human mind has ever come up with is the idea of a machine or device, which extends its own ability to calculate, think recursively, memorize, communicate, etc... Inventing the computer and similar devices might turn out to be the biggest revolution in intelligence so far, but it might also mark a turn point from which intelligence could regress.