Is This HARD Or Is This HART?

So how do we humans perceive things around us so uniquely? Hofstadter mentions the phenomenon of conceptual slips of tongue in which we replace a word we meant to say with a conceptually similar one in the very same context. We don't only seem to have one specific word tightly connected to some meaning, but might use different ones to express the same or analogous meaning. Also when we look at other languages, there might be several translations for one term, fitting more or less in a specific context. These two phenomena thrown together occur in my daily life as an exchange student. When every now and then a German word slips into my English sentences or when I throw pieces of English vocabulary or even syntax at my parents on the phone. For example, I will "make" pictures while I am here in the US and as soon as I come home, I will say "Fotos nehmen" (take photos) instead of "machen" (make).

Can machines acquire meaning in a sense humans think they can? Grasp intrinsic properties? Highlight certain facts while disregarding others in certain circumstances or specific contexts? - To answer these questions we have to look at the understanding of meaning in the first place. The meaning of a thing might be all facts that are to know about it, its functional role, associations one might connect with it, in all: its perception in the world.

When computer programs are given only partial facts and properties of an entity, they cannot possibly grasp or understand the meaning of it, according to Hofstadter. But what are we given in the world? A human can never acquire all possible impressions, facts and angles of an object for example, but we attribute her the notion of understanding what the object is with just little knowledge about it, and be it just its name.

I have the impression, that humans are not that different then, let's say, google with a few extras. One can ask the search engine for example "what is a horse?" and it will bring up the internal definition of horse in first place under the search term "define:horse". From there we can switch to picture results of horses, videos of horses, scientific articles about horses, maps of places where horses are present, and so on ... All this information can be and might already be connected to an apt meaning of the term horse. The pictures would suffice to identify 99% of all horses shown to the engine in form of an uploaded picture and some kind of recognition software (which google posesses). The videos could identify the typical motion of horses and the common sounds a horse would make. The facts and common terms associated with horses could help to spot conversations via instant messaging or email, or even voice chat about horses.

Even though google has never perceived a horse like we humans do, it will be able to tell from the context that something is a horse, or the other way around develop a context around the term horse in order to draw an analogy. From my point of view, the perception of the object does not play the biggest role in "understanding" the meaning of it, as we cannot understand it any deeper than computers.

Conceiving Meaning - Meaning Conceiving

Chalmers, French, and Hofstadter point out in their paper High-level perception, representation, and analogy: A critique of artificial intelligence methodology that the major flaw in most artificial intelligence programs lies in the absents of true conception in some modality. In contrast to the human brain does none of the presented programs acquire or rather conceive the intrinsic properties of the given data, neither does any filter out irrelevant, incomplete or partially incorrect data from the vast stream of outside stimuli like a real world observer. Instead, all the data was prerefined and perfectly suiting for the task at hand making it in fact almost impossible not to come up with the anticipated conclusion/analogy/result.

I must admit that logically concluding an analogy or alike from a bunch of connected, alas meaningless predicates sounds much less impressive than the chance of having discovered a way to artificially draw real world analogies. Maybe Hofstadter is a little too harsh on the quite enthusiastic and sensationalist science colleagues since their main goal seemed to be a program which could logically deduce facts and relations logically from real world data. It was not in their interest to compute these findings as much human-like as possible, contrary to Hofstadter's approach. Admittedly, the citations and conclusions from the papers Hofstadter refers to sound very provocative and sensational, portraying the entire artificial intelligence department as some kind of science fiction lab. Therefore studies like Numbo, Copycat and alike sound rather minimalistic and unimportant even though their implications might be much more revealing than examples which might be more applicable to actual situations but lack a foundation of understanding the underlying concepts of intelligence altogether.

Imitating Intelligence Is Not Real Intelligence

The preface to chapter 4 in Hofstadter's book is intended to warn us about the dangers of the ELIZA Effect and the anthropomorphism of computer programs in general. Hofstadter in particular ventilates his dislike for the sensational press some of his colleagues receive with rather unsophisticated and misinterpreted AI programs like ACME, ARCS, SME and the program developed by Scott French. All of them were supposedly used to draw analogies from real life data to imitate human intelligence.

According to Hofstadter all these attempts failed right in the beginning, with the assumption that the programs indeed understood some kind of meaning of the terms (or rather strings) they handled. All approaches so far have only shown, that provided with the essential (and only the essential) data and some kind of rule based, logical system analogies and conclusions can be drawn by a computer. Hofstadter argues though, that the data could be arbitrarily exchanged with different or nonsense predicates and variables, yielding gibberish or even wrong results.

One of the reasons why these refuted scientific discoveries are still around and misinterpreted is a phenomenon called ELIZA Effect. This effect goes back to the chatbot ELIZA developed by Josef Weizenbaum in 1966 which very simply imitated a dull psychiatrist by rephrasing most of the human input into questions, generating a rather unsophisticated dialog mainly driven by the creativity of the inputting human himself. Even though the communication with such a chatbot is pretty obviously shallow and might be downright stupid, some humans ascribe human-like intelligence, emotional motivation and genuine interest to the "answer-questions" ELIZA and other chatbots generate.

Other examples of imitation of human-like text generation that came to my mind when I read the preface to chapter 4 were the Chomskybot, which generates very complicated, scientific-sounding text of arbitrary length including real words and syntax and the PARROT which generates anything between strings of arbitrary letters up to entire nonsense stories readable by humans. Rumor has it that someone actually handed in a slightly altered Ph.D. thesis generated by the Chomskybot and got away with it. PARROT is purely based upon probabilistic occurrences of letters or n-grams within a given corpus.

Numbo the Human Computer

Hofstadter's Numbo is not necessarily supposed to deliver fast and accurate results to the number problems, it's tackling, but rather to mimic the way we come about with a solution. So instead of brute-force trail-and-error of combinations which are doomed to be wrong from the beginning (at least from the human perspective) Numbo tries to juggle with the numbers in a probabilistic way to combine previous particles to likely bonds. This way solutions are found in a much smarter way, disregarding the enormous overhead created in the process.

Hofstadter is not really satisfied with "smart", he wants to compare it to human cognition 1:1. Therefore he compared the trace of Numbo runs to the mental protocols of human players (as good as they could remember or introspect). The comparison reveals certain similarities but also yields different results, since Numbo comes up with easy solutions in a way, humans would not think of first. So it does not seem to be as "smart" after all. Or do we just come up with weird and complicated solutions?

We are given a tiny peek under the hood of the mysterious Numbo-machine and discover that the complexity of human thought process was tried to be modeled by mere probabilistic functions in most cases. In my eyes, this is one fundamental flaw which cannot be eluded that easily. Humans have association with every number, context and method they use. The seeming randomness in their actions might be a result of the current environment and situation they are in. A human might use one technique to solve a problem once and another technique the next time, not because of statistic equivalent of the two approaches, but because all prior events taught her/him to use the one most promising one at the time.

The codelets from the coderack, Hofstadter uses, are supposed to stand for the thought processes we use in turns to come to a solution. They are supposed to contain as few prior knowledge as possible. This is what I see as another fundamental flaw in Hofstadter's approach. I have the notion that we literally learn the results of addition, subtraction and multiplication by heart (at least the easy ones). And from there, we draw analogies between problems like 3+2=5 and 30+20=50 or 24+7=31 and 64+7=71. But there is even a more basic level humans operate on. We associate a certain context and knowledge around numbers or operants and therefore pick those which pop up more easily to us.

In my opinion, Hofstadter is quite on the right track, but still misses a lot of essential processes, naturally going on in the human brain.

The Beauty of Math in TV Shows

In the beginning of the third chapter, Hofstadter remembers the involvement of a Belgian colleague of his during the year 1986, Daniel Defays. He was part of the NARG but at the same time wrote a program called Numbo which would solve the sort of mental math task involved in crypto problems. The idea is to have a series of smaller integers (bricks) and one bigger target number and having to combine an arbitrary number of the bricks with the help of addition, subtraction or multiplication so they would equal the target integer.

The task is very similar to the crypto problem but the (human) computer is forced to partially use different strategies, since the number may be quite higher as in ordinary crypto problems. One of these strategies is rounding, so approximating a value close to the target and trying to go from there. Another is using analogies of the kind that 20*30=600 is similar to 2*3=6 which might be very obvious to the average mathematician among us but will not strike an infant immediately, not to speak of modeling such prior general knowledge to computer programs.

But it was exactly this kind of shortcuts we use in everyday situations that inspired Hofstadter and Defays in optimizing their programs accordingly. I must admit that these strategies, as subconscious as they might seem, are very vital to the speed and accuracy of our thought processes not necessarily only to solving math problems or figuring out words. It is very neat to think, somebody would realize such mechanisms in a real problem solving problem (even though, we are of course let alone in the dark about how this realization came about).

Gloms are Happy when its Cold

In the last part of the second Chapter, Hofstadter carries on his metaphorically enriched idea of gloms, according to which atomic particles (letters) form weak and strong bonds when given the occasion and develop syllable-like structures and statistically likely word formations. He describes this as a parallel at the bottom, but serial at the very top approach in which particles may bind in no particular but rather random or semi-randomly prioritized order. At the end of such a chain of events stands a word-like top-level glom using all the available particles.

From reading this passage about finding a possible solution to a Jumble, I get the notion, that it is not even important that Jumbo spits out a genuine word but rather delivers something, that humans would recognize as pronounceable and possible word. This "word" is then not correct or incorrect in the absolute sense, but might be subject to simple rearrangements to come up with the right solution. Hofstadter seems to feel that this approach is closer to the way how we ourselves solve Jumbles in our heads: Finding possible subgoals and working on the produced structure from there instead of taking all letters apart completely again and starting from scratch. This sounds like a very clever approach to me.

In particular I liked the metaphorical use of temperature and entropy of the particles and gloms to describe their intrinsic state of being in form of happiness or confidence. Similar to physical particles, gloms bond more loosely or hardly at all bound to each other, but collide very frequently with others when hot and in motion. If being in a stable and promising combination with other particles the glom will cool down and therefore break less easily. All this might really describe possible analogues in our minds.

Beyond Brute-Force Dictionaries

In the pattern of letting the reader tip-toe in the dark while shouting: “A little bit more left, than you went backwards just before!” Hofstadter carries on with the Jumble puzzles throughout the second chapter. Jumbo, the program which he leaves us so unclear about, supposedly can solve simple and more complex anagrams by bonding letters, syllables and word parts together and forming ever more probable (but not necessarily more meaningful) chunks.

In metaphorically rich language, the author explains how such bonding between such atomic (letter) or molecule-like entities (syllables or intrinsically well fitting pairs or triplets) would carry out. That potential partners could first spark on sight of each other and eventually bond together if no other potential partner in sight would exert even more attraction. This way, all elements mingle first on a very small and detailed level and would then try to bond again with other more evolved structures further up in the hierarchy.

Yet, it does not seem quite plausible, how Hofstadter wants to realize such ranks of attraction level between the potential bonding partners. He suggests a very subjective approach, by bonding the first atomic elements with the help of his own intuitions. I would have rather suggested a probabilistic approach which had the underlying knowledge of how often certain letters proceed others in a certain language. This can be easily done by consulting just a small sample text and analyzing what letters normally occur in the environment of others.

Maybe Hofstadter is going to go into this, and I am being unjust to him. But I would have wished that he went into more detail of the actual realization than using two metaphors over like 5 pages to describe a process that most people have understood from the previous text.