Science Magazine: Missing data hinder replication of artificial intelligence studies

Science Magazine: Missing data hinder replication of artificial intelligence studies. “Last year, computer scientists at the University of Montreal (U of M) in Canada were eager to show off a new speech recognition algorithm, and they wanted to compare it to a benchmark, an algorithm from a well-known scientist. The only problem: The benchmark’s source code wasn’t published. The researchers had to recreate it from the published description. But they couldn’t get their version to match the benchmark’s claimed performance, says Nan Rosemary Ke, a Ph.D. student in the U of M lab. ‘We tried for 2 months and we couldn’t get anywhere close.'”