newspaper3k

Full Changelog

Closed issues:

• ImportError: cannot import name 'Image' #183
• Won't let me import #182
• Install on Mac - El Capitan Failed - "Operation not permitted" #181
• Downgrades to old versions of required packages upon installation #174
• Handling 404, 500, and other non-200 http response codes to prevent scraping error pages #142
• Libray downgrading in installation #138

Merged pull requests:

• Don't scrape error pages #190 (yprez)
• Added Hebrew stop words for language support #188 (alon7)
• Fix installation and build #187 (yprez)
• Fix installation docs #184 (yprez)
• Travis CI integration #180 (yprez)
• requirements.txt - Use minimal instead of exact versions #179 (yprez)
• Handle lxml raising ValueError on node.itertext() - Python 3 #178 (yprez)
• Handle lxml raising ValueError on node.itertext() #144 (yprez)
• Parse byline fix #132 (davecrumbacher)

Full Changelog

Closed issues:

• Critical leak in newspaper.mthreading.Worker #177
• HTMLParseError #165
• Take local paths to .html files #153
• Wall Street Journal Full Text is not Correctly Scraped #150
• Article HTML Returning Null #131
• No articles #130
• Loading Pages that use heavy javascript #127
• Login handling for premium websites #126
• Installation of nltk is failing #121

Merged pull requests:

• Support urls with dots #176 (alexanderlukanin13)
• upgrade beautifulsoup4 to 4.4.1 for python 3.5 #171 (AlJohri)
• Updated requests version #170 (adrienthiery)
• Turkish Language added #169 (muratcorlu)
• Add macedonian stopwords #166 (dimitrovskif)
• Issue#95 added graceful string concatenation #157 (surajssd)
• fix for "jpeg error with PIL, Can't convert 'NoneType' object to str implicitly" #154 (hnykda)
• bugfix in article.py, is_valid_body #149 (ms8r)
• Fixed typo #139 (Eleonore9)
• Correct link for the Python 3 branch #136 (jtpio)
• Add python3-pip install step for Ubuntu #135 (irnc)

Full Changelog

Closed issues:

• is there any kind of documentation on centos 7? #114
• Add extraction publishing date from article. #3

Merged pull requests:

• bumping nltk to 2.0.5 - see #824 in nltk #125 (hexelon)

Full Changelog

Closed issues:

• Getting rate limiting issue? #116
• newspaper.build( ) error #111
• Allow lists in Parser.clean_article_html() #108

Merged pull requests:

• Fix incorrect log call while generating articles #115 (curita)
• Allow lists in clean_article_html() - fixes #108 #112 (ecesena)
• Fixed nodeToString() to return valid HTML #110 (ecesena)
• Fixed empty return in top_meta_image #109 (ecesena)

Full Changelog

Implemented enhancements:

• Fulltext extraction improvement #1 #105

Closed issues:

• Tags h1 in article_html - indented behavior? #107

Merged pull requests:

• Fulltext extraction improvement #1 #106 (codelucas)

Full Changelog

Closed issues:

• Metatags on Vice.com #103
• Can't extract images from german newspapers #96
• article_html misses many of the images #89

Merged pull requests:

• Integrate UnicodeDammit, deprecate parser_class, deprecate encodeValue, refactor, scaffolding for more unit tests #104 (codelucas)

Full Changelog

Closed issues:

• UnicodeDecodeError: 'utf8' codec can't decode byte 0xcc #99
• TypeError: Can't convert 'bytes' object to str implicitly #98
• [Parse lxml ERR] Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration. #78
• UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 11: ordinal not in range(128) #77
• article.text and keywords error #47

Merged pull requests:

• Huge bugfix to aid lxml DOM parsing + remove unhelpful and excess exception messages and added tracebacks to exception logging #102 (codelucas)
• Decode bytestring returned from lxml's toString early on before sending it out to outer code #101 (codelucas)
• Fixed #78: Remove encoding tag because lxml won't accept it for unicode #97 (mhall1)

Full Changelog