search
2011.09.22 - Turing test 2011
about two weeks ago i came across an interesting topic – new chatterbot, that is said to (probably) pass Turing test. it's name is Cleverbot (can be |talked online). authors claim that 59.3% of tested people decided chatterbot is a human being!
since i'm interested in Artificial intelligence i was curious about the results and the test itself. and so i've read more. next thing i've noticed that, on the same test, humans got 63.3%! i've decided to talk with the bot and try it out myself. bot got confused after the third sentence.
here is a log of a short talk i've made today:
| who | conversation | my comment |
|---|---|---|
| H: | hello. | |
| AI: | Hello. | ok - simple |
| H: | What is the weather like at your place? | |
| AI: | Sunny. | very short answer - perhaps a 'weather' keyword found a match? |
| H: | Do you have any pet? | |
| AI: | Yes, I do. Do you? | nice. |
| H: | Me too. it's a rabbit. and yours? | |
| AI: | Elizabeth. | fail - bot probably thinks i'm asking her name… |
| H: | This is not the answer to my question. | |
| AI: | Yes. | standard trick on humans - searching deeper context in trivial answers. |
from this point it just got worse and worse. and so we see some casual chatterbot style: do not say too much, try to make human express himself (notice the average length of my pars and the ones generated by the bot), use general expressions whenever possible.
also please notice that i didn't try any sophisticated language structures, abstract concepts, nor tricky questions requiring combining knowledge and context from many sentences before. the most “advanced” thing i did is to ask a question with a reference the the very previous sentence.
i have recalled in my memory excellent s-f movie - Blade Runner. there was a man trying to capture 6 androids, that are so hard to distinguish from humans that the only way to tell a difference is to watch their reaction on emotions. since robots do not have an emotions, they do not show remorse nor guilt and their pupil does not change size, when owner's faced with a morally-difficult question.
so what are my conclusions?
- clever bot did NOT pass Turing test.
- 36.7% humans FAILED Turing tests.
- AI is still an open field with much to learn.
and so it looks like that intelligence of talkers was very poor. do people tend to pay less and less attention to the world and interactions? one could say: “dude! you failed Turing test!”…
