java - Jsoup returns 404 error -


jsoup-1.8.1

try {
document document = jsoup.connect(url).get();
return document.getelementsbytag("title").text();
} catch (exception e) {
system.out.println(e);
return null;
}

org.jsoup.httpstatusexception: http error fetching url. status=404, url=http://ja.wikipedia.org/wiki/%e3%83%aa%e3%83%b3%e3%82%b4

decoded url here http://ja.wikipedia.org/wiki/りんご

when run in main function in local, runs expect. if execute in servlet, returns 404 error. non encoded url can executed correctly.

wikipedia doesn't allow bots crawling add useragent , referrer

doc = jsoup.connect(url)       .useragent("mozilla/5.0 (windows; u; windowsnt 5.1; en-us; rv1.8.1.6) gecko/20070725 firefox/2.0.0.6")       .referrer("http://www.google.com")       .get(); 

Comments

Popular posts from this blog

java - Plugin org.apache.maven.plugins:maven-install-plugin:2.4 or one of its dependencies could not be resolved -

Round ImageView Android -

How can I utilize Yahoo Weather API in android -