Friday, December 18, 2015

Q&A - 18/12

The Guardian

OpenAI, a new non-profit artificial intelligence company that was founded on Friday, wants to develop digital intelligence that will benefit humanity [..] We need both [code and data] to make predictions [.. O]pening the benefits of AI to all requires that everyone has a source of high-quality data.

Sharing is Caring

A representative sample of data accompanying each approach would be fine too; I don't understand, even that is not in the cards?

MIT Technology Review

[I]s deep learning based on a model of the brain that is too simple? Geometric Intelligence [..] is betting that computer scientists are missing a huge opportunity by ignoring many subtleties in the way the human mind works. In his writing, public appearances, and comments to the press, [AI expert] Marcus can be a harsh critic of the enthusiasm for deep learning. But despite his occasionally abrasive approach, he does offer a valuable counterperspective. Among other things, he points out that these systems need to be fed many thousands of examples in order to learn something. Researchers who are trying to develop machines capable of conversing naturally with people are doing it by giving their systems countless transcripts of previous conversations. This might well produce something capable of simple conversation, but cognitive science suggests it is not how the human mind acquires language.

Good Point

Not that a system must always be built to reflect its biological counterpart exactly; our planes are built differently from a bird. Maybe we can rephrase the last sentence in the paragraph above: we didn't even know what thinking even was in order to copy it incorrectly / differently :). It's no wonder that the main thread of initial AI research revolved around various forms of rote learning; because that's what we thought learning was: rote. ML systems do generalize obviously, but a lot of data is needed just so these systems can generalize properly (maybe the previous sentence will be considered an oxymoron one day).

Plus, from another angle / technologically speaking deep learning might turn out to be an evolutionary dead end. I always felt the approach to be a little hodge podge, the statistical properties of a network do not flow out of a model naturally. [geek] an analogy here would be the difference between K-Means clustering and mixture models - the latter is probabilistic, has statistical properties, the former is not [/geek]. If my memory serves correctly, even the neural network expert Hinton himself tried to submit an article to a journal once called "There Is Something Deeply Wrong with Deep Learning"; Characteristically the paper was rejected (H. mentioned this as an example of the shortcomings of the peer-review process). But this kind of talk puts me on ... say, "yellow alert" when it comes to these new neural net based new approaches.


Researchers at the Massachusetts Institute of Technology, New York University and the University of Toronto reported a new type of “one shot” machine learning on Thursday in the journal Science, in which a computer vision program outperformed a group of humans in identifying handwritten characters based on a single example.

The program is capable of quickly learning the characters in a range of languages and generalizing from what it has learned. The authors suggest this capability is similar to the way humans learn and understand concepts.

The new approach, known as Bayesian Program Learning, or B.P.L., is different from current machine learning technologies known as deep neural networks.


Guns and Butter

Peter Schiff "We had a lot of problems that happened in 70s, under Nixon and Ford. But those problems started in the 60s, the great ...