Kaylin Walker: Tidy Text Mining Beer Reviews. “BeerAdvocate.com was scraped for a sample of beer reviews, resulting in a dataset of 31,550 beers and their brewery, beer style, ABV, total numerical ratings, number of text reviews, and a sample of review text. Review text was gathered only for beers with at least 5 text reviews. A minimum of 2000 characters of review text were collected for those beers, with total length ranging from 2000 to 5000 characters.”
Library of Congress: Digital Scholarship Resource Guide: Text analysis (part 4 of 7). “Clean OCR, good metadata, and richly encoded text open up the possibility for different kinds of computer-assisted text analysis. With instructions from humans (“code”), computers can identify information and patterns across large sets of texts that human researchers would be hard-pressed to discover unaided. For example, computers can find out which words in a corpus are used most and least frequently, which words occur near each other often, what linguistic features are typical of a particular author or genre, or how the mood of a plot changes throughout a novel. Franco Moretti describes this kind of analysis as ‘distant reading’, a play on the traditional critical method ‘close reading’. Distant reading implies not the page-by-page study of a few texts, but the aggregation and analysis of large amounts of data.”
British Library: BL Labs 2017 Symposium: Data Mining Verse in 18th Century Newspapers by Jennifer Batt. “Dr Jennifer Batt, Senior Lecturer at the University of Bristol, reported on an investigation in finding verse using text and data-mining methods in a collection of digitised eighteenth-century newspapers in the British Library’s Burney Collection to recover a complex, expansive, ephemeral poetic culture that has been lost to us for well over 250 years.” A ~23 minute video of her presentation and her slide deck is available at the URL I linked to.
Science: Want to analyze millions of scientific papers all at once? Here’s the best way to do it. “There is long-standing debate among text and data miners: whether sifting through full research papers, rather than much shorter and simpler research summaries, or abstracts, is worth the extra effort. Though it may seem obvious that full papers would give better results, some researchers say that a lot of information they contain is redundant, and that abstracts contain all that’s needed. Given the challenges of obtaining and formatting full papers for mining, stick with abstracts, they say. In an attempt to settle the debate, Søren Brunak, a bioinformatician at the Technical University of Denmark in Kongens Lyngby, and colleagues analyzed more than 15 million scientific articles published in English from 1823 to 2016.”
Dato Capital: Dato Capital Announces First Tool for Extracting Company Information from Documents (PRESS RELEASE). “The Company Information Extractor can process documents by entering a website URL, uploading a file or entering text directly. Accepted formats include PDF, Word, Excel, HTML and TXT files. The system scans the document and searches for mentions of companies and directors against a daily updated database of 14 million companies and 12 million directors from the United Kingdom, Spain, Luxembourg, Panama, Gibraltar, BVI, Cayman Islands and the Netherlands.” The direct link for the tool is https://en.datocapital.com/CompanyInformationExtractor .
Kris Shaffer: Mining Twitter data with R, TidyText, and TAGS. “One of the best places to get your feet wet with text mining is Twitter data. Though not as open as it used to be for developers, the Twitter API makes it incredibly easy to download large swaths of text from its public users, accompanied by substantial metadata. A treasure trove for data miners that is relatively easy to parse. It’s also a great source of data for those studying the distribution of (mis)information via digital media.”
Inside Big Data: The Advantages of Mining Full-Text Articles over Abstracts. “Given their easy accessibility through databases such as PubMed, many researchers use article abstracts to identify a collection of articles (or ‘corpus’) for use in text mining. But, while abstracts provide some valuable pieces of information, there are major advantages to taking steps using and mining full-text articles instead.”