java网络编程-爬虫+模拟浏览器-创新互联
网络爬虫+模拟浏览器(获取有权限网站资源):
获取URL
下载资源
分析
处理
public class http {
public static void main(String[]args) throws Exception
{
//http+s更安全
//URL.openStream()打开于URL的连接,并返回一个InputStream用于从连接中读取数据
//获取URL
URL url=new URL("https://www.jd.com");
//下载资源
InputStream is = url.openStream();
BufferedReader br=new BufferedReader(new InputStreamReader(is,"UTF-8"));;
String msg=null;
while((msg=br.readLine())!=null)
{
System.out.println(msg);
}
br.close();
}
}
获取有权限网络资源:
public class http {
public static void main(String[]args) throws Exception
{
//.openConnectio,,返回一个URLConnection实例表示由所引用的远程对象的连接URL
//URLConnection的子类有HttpURLConnection和JarURLConnection
URL url=new URL("https://www.jd.com");
//下载资源
HttpURLConnection conn=(HttpURLConnection)url.openConnection();
conn.setRequestMethod("GET");//模拟浏览器得get请求
conn.setRequestProperty( "User-Agent","Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/18.17763");
BufferedReader br=new BufferedReader(new InputStreamReader(conn.getInputStream(),"UTF-8"));
String msg=null;
while((msg=br.readLine())!=null)
{
System.out.println(msg);
}
br.close();
}
}
创新互联www.cdcxhl.cn,专业提供香港、美国云服务器,动态BGP最优骨干路由自动选择,持续稳定高效的网络助力业务部署。公司持有工信部办法的idc、isp许可证, 机房独有T级流量清洗系统配攻击溯源,准确进行流量调度,确保服务器高可用性。佳节活动现已开启,新人活动云服务器买多久送多久。
文章题目:java网络编程-爬虫+模拟浏览器-创新互联
本文网址:http://pwwzsj.com/article/diiepj.html