The Grammar Of Thought

| September 3rd, 2008

Update: Found this interesting book related to this post – The Language Instinct [link]

I have just started to scratch the surface of Natural Language Processing for my next project (involving NLP and Twitter – details to follow) and I already have a dozen questions bothering me. I shall attempt to put forth a few of the ideas and questions in this post. Lets talk briefly about the structure of language. Language has different levels of structure:

  1. dicourse – group of sentences
  2. sentences
  3. phrases
  4. words
  5. and so on…

Between the ‘sentences’ and ‘words’ lies the syntactic structure of language. This syntactic structure is built using the parts of speech of the words: nouns, verbs, etc. Words are grouped into phrases whose formation is governed by the grammar rules, for example:

Sentence -> ‘Noun Phrase’ . ‘Verb Phrase’
‘Noun Phrase’ -> Determiner . Adjective . Noun
‘Verb Phrase’ -> Verb . ‘Noun Phrase’

A sentence is grammatically correct if it adheres to the grammar of the language (like described above). With just the above knowledge about language (something you might have learnt in the 5th grade) we can see that for a candidate sentence to make sense in some language, it has to be composed of meaningful components and these components have to be in some specific order for it to logically make sense.

Grammar of Thought

This has led me to ponder if an analogous grammar exists for ‘thought’. Our thoughts can also be broken down into meaningful components and the components here also have to follow some implicit ordering for the ‘thought’ to make sense. If you think about the way you think, you will notice that as you run from one thought to another there is some logical connection between them just as between the sentences in a paragraph. If we could somehow get a formal representation of this grammar, wouldn’t it enable machines to think?

Language and Thought

There is enough literature out there which links the structure of language with the structure of thought. Benjamin Whorf states in his writings:

the structure of a human being’s language influences the manner in which he understands reality and behaves with respect to it

Thus, human cognition is based on the structure of language which in turn is the grammar defining the language. Hence a machine capable of generating sequence of grammatically correct sentences which also fit together logically (discourse), should have some ability of cognition. Even the Turing test uses natural language as a test for some level of cognition. Is this perspective of Natural Language Processing as a means of provisioning cognition to a machine, correct? Could this be another path for achieving artificial intelligence? I would love to get an answer to this from NLP experts out there.

Or is it just one of my other posts which don’t make sense because its 3am and I’m half asleep?

  • Manvesh

    I’ve been reading up on Frame-based semantic memory, and the grammer of coherent thought seems to be based on a set of learnable production rules that are used to fill up these frames. The most important part of language is definitely understanding of verbs and actions.

    The methods by which these meta-production rules are learned, modified, and adapted still eludes me.

  • aman

    both ….

  • Atul Kulkarni

    Andy, my views about your post might be a bit philosophical but just give them some thought and let me know what do you think about them.

    Lets answer these questions,
    Is Language the only way of capturing the thoughts that come to one’s mind? I think NO. Language sure is one way of representation, but it is not the only way. So by modelling a language we might be able to track the thoughts that were “expressed” by the person. But the possibility of some other thought came like a flash in one’s mind he just did some gesture (looked some place, read something, or may be touched something) is pretty high, which intern triggered and chain of thoughts that resulted in a sentence that he/she expressed to the world again (via Twitter, if I understand your plan correctly). We surely have two of these thoughts but we do not have the behavioural or thought sequence that could be more important in understanding the cognitive sequence. The point I am trying to make is that Language surely has a potential of expressing things around but the assumption that all of us will express every thought that comes to one’s mind is a little bit of a over kill I think.

    Second, grammar of language “in use” keeps changing. If you don’t believe me, check for yourself the language used by kids today did you use something remotely close to this? I am not talking about the “Wren and Martin” text bookish language used in the technical papers, I am talking about the common man of tomorrow, who is hell bent up on using the phonetic acronyms, mixing two or more languages that he or she knows in to his or her discourse and is more keen on expressing the thoughts. Again, the point is language is a changing phenomena, it changed less frequently in the past but now mixing of cultures and inter mingling of people from various places and for many more reason that might be beyond me. But, I think while modelling a language the very assumption that it has a grammar is misleading. I DO NOT say language has no grammar but for our purpose of analysis it is kind of irrelevant. So I would suggest use the bag of words approach to define statistical models for language. I think there are enough papers and other technical articles on this. These are just my thoughts, and I will be happy to know if someone disagrees with them and wishes to discuss it.

  • Anand Kishore

    Atul, I agree with you on most of your arguments but the only point I want to bring across is that NLP could we viewed in this perspective rather than just tools for machine translation and likewise.

    I agree that Language is not the only way of capturing thoughts. We usually have a photographic representation of what we see in our heads. But we use language to bind the various abstract representations together. For example, you have a abstract representation of your friend John and the same for a grocery store. So when you think the following “When John goes to the grocery store, I’ll run away with his bike” – you don’t visualize John actually walking into the grocery store and you running away with his bike. This entire thought sequence is processed as language in your head – try it out to see what I’m trying to say. Yes, thoughts flash miraculously in our head – and I dont have an explanation for that – but the onus is not how they were initiated but rather in what form are they initiated. You are right in saying that Language is not the only way (or probably even the right) way of representing thoughts but if we can build machines which use language it would be easier for Humans to understand the output produced by such machines.

  • Anand Kishore

    For those interested, check out this great book ‘The Language Instinct’ []