http://www.ip138.com/
网络爬虫又名“网络蜘蛛”，是通过网页的链接地址来寻找网页，从网站某一个页面开始，读取网页的内容，找到在网页中的其它链接地址，然后通过这些链接地址寻找下一个网页，这样一直循环下去，直到按照某种策略把互联网上所有的网页都抓取完为止的技术。
整理常见的网络爬虫IP2016-09-24
http://www.360doc.com/content/16/0924/18/34658207_593332134.shtml
最近整理了一下各平台爬虫的信息，记录一下（有异议的地方请指正）。有些个人的恶意IP并没统计。以下整理的爬虫IP都是根据对应的User-Agent排序的，所以可能统一号段的IP分属不同UA（当然UA也是可以随意伪造的），加上网上搜集的一些但是UA不确定所以都放到每个分组的最后边。
百度爬虫IP列表
爬虫Agent：Mozilla/5.0 (Windows NT 6.1; WOW64; Baidu Transcoder) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11
61.135.169.75
61.135.169.55
61.135.169.22
220.181.51.40
220.181.51.39
111.13.13.74
111.13.13.73
111.13.13.72
111.13.13.71
111.13.13.6
111.13.13.5
111.13.13.4
111.13.13.3
101.254.184.206
1.95.9.244
爬虫Agent：Mozilla/5.0 (compatible; heritrix/3.1.1 +http://www.baidu.com)
59.50.71.83
211.89.227.50
211.89.227.16
211.89.227.15
183.224.87.36
124.119.30.90
117.25.173.119
116.113.28.179
爬虫Agent：Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
61.191.191.87
61.184.241.206
61.183.41.95
59.49.46.165
58.222.20.226
58.218.204.136
45.113.253.53
27.151.30.176
27.151.28.24
222.216.28.45
222.186.3.229
220.181.51.81
220.181.51.76
220.181.51.75
220.181.51.74
220.181.51.110
220.181.51.109
220.181.51.107
220.181.108.97
220.181.108.96
220.181.108.87
220.181.108.86
220.181.108.84
220.181.108.83
220.181.108.81
220.181.108.80
220.181.108.79
220.181.108.78
220.181.108.77
220.181.108.76
220.181.108.75
220.181.108.186
220.181.108.184
220.181.108.181
220.181.108.179
220.181.108.178
220.181.108.177
220.181.108.175
220.181.108.174
220.181.108.173
220.181.108.172
220.181.108.171
220.181.108.170
220.181.108.168
220.181.108.166
220.181.108.165
220.181.108.161
220.181.108.158
220.181.108.157
220.181.108.156
220.181.108.155
220.181.108.153
220.181.108.152
220.181.108.151
220.181.108.150
220.181.108.149
220.181.108.147
220.181.108.146
220.181.108.145
220.181.108.144
220.181.108.143
220.181.108.142
220.181.108.141
220.181.108.140
220.181.108.139
220.181.108.123
220.181.108.122
220.181.108.120
220.181.108.119
220.181.108.117
220.181.108.116
220.181.108.115
220.181.108.114
220.181.108.113
220.181.108.112
220.181.108.111
220.181.108.110
220.181.108.109
220.181.108.108
220.181.108.107
220.181.108.106
220.181.108.105
220.181.108.104
220.181.108.103
220.181.108.101
220.181.108.100
219.153.68.137
219.138.135.236
202.46.52.201
202.46.51.156
202.46.51.132
183.61.171.152
183.61.171.137
183.2.242.150
180.97.35.165
180.76.15.9
180.76.15.8
180.76.15.7
180.76.15.6
180.76.15.5
180.76.15.34
180.76.15.33
180.76.15.32
180.76.15.31
180.76.15.30
180.76.15.29
180.76.15.28
180.76.15.26
180.76.15.24
180.76.15.23
180.76.15.22
180.76.15.21
180.76.15.19
180.76.15.18
180.76.15.17
180.76.15.163
180.76.15.162
180.76.15.161
180.76.15.158
180.76.15.157
180.76.15.156
180.76.15.155
180.76.15.154
180.76.15.153
180.76.15.152
180.76.15.151
180.76.15.150
180.76.15.15
180.76.15.149
180.76.15.148
180.76.15.147
180.76.15.145
180.76.15.144
180.76.15.143
180.76.15.142
180.76.15.141
180.76.15.14
180.76.15.138
180.76.15.137
180.76.15.136
180.76.15.13
180.76.15.12
180.76.15.11
180.76.15.10
14.29.49.206
124.237.78.217
124.232.151.230
124.228.91.179
123.125.71.98
123.125.71.94
123.125.71.92
123.125.71.91
123.125.71.90
123.125.71.89
123.125.71.87
123.125.71.86
123.125.71.85
123.125.71.84
123.125.71.83
123.125.71.82
123.125.71.81
123.125.71.78
123.125.71.74
123.125.71.73
123.125.71.70
123.125.71.69
123.125.71.60
123.125.71.58
123.125.71.57
123.125.71.56
123.125.71.55
123.125.71.53
123.125.71.52
123.125.71.50
123.125.71.49
123.125.71.47
123.125.71.45
123.125.71.44
123.125.71.43
123.125.71.42
123.125.71.41
123.125.71.39
123.125.71.38
123.125.71.34
123.125.71.33
123.125.71.32
123.125.71.31
123.125.71.30
123.125.71.29
123.125.71.28
123.125.71.27
123.125.71.26
123.125.71.23
123.125.71.22
123.125.71.20
123.125.71.19
123.125.71.18
123.125.71.17
123.125.71.15
123.125.71.14
123.125.71.13
123.125.71.12
123.125.71.116
123.125.71.115
123.125.71.114
123.125.71.113
123.125.71.111
123.125.71.110
123.125.71.109
123.125.71.108
123.125.71.107
123.125.71.106
123.125.71.105
123.125.71.103
123.125.71.102
123.125.71.100
123.125.67.152
123.125.67.148
122.10.89.22
120.39.244.111
117.34.115.181
117.25.139.66
117.25.139.42
117.25.139.35
117.25.139.34
116.9.156.37
116.55.241.24
115.47.26.143
115.231.26.31
113.113.120.28
112.124.57.11
爬虫Agent：Baiduspider+(+http://www.baidu.com/search/spider.htm)
61.146.178.41
61.146.178.186
爬虫Agent：Baidu-YunGuance-VSBot(ce.baidu.com)
111.206.36.17#网络搜集
180.149.130.*
180.76.5.66百度云观测IP
安全检测服务User-Agent：
User-Agent:Baidu-YunGuanCe-ScanBot(ce.baidu.com） 可用性检测
其他服务User-Agent：
User-Agent:Baidu-YunGuanCe-SLABot(ce.baidu.com） 可用性检测
User-Agent:Baidu-YunGuance-PerfBot(ce.baidu.com) 速度检测
User-Agent:Baidu-YunGuance-VSBot(ce.baidu.com) 认证服务
User-Agent:Baidu-YunGuance-RSBot(ce.baidu.com) 即时速度检测
180.97.106.36
180.97.106.37
180.97.106.161
180.97.106.162
115.239.212.7
115.239.212.8
115.239.212.9
115.239.212.10
115.239.212.11
115.239.212.6
115.239.212.4
115.239.212.5
115.239.212.65
115.239.212.66
115.239.212.67
115.239.212.68
115.239.212.69
115.239.212.70
115.239.212.71
115.239.212.72
115.239.212.134
115.239.212.135
115.239.212.136
115.239.212.137
115.239.212.138
115.239.212.139
115.239.212.132
115.239.212.133
115.239.212.193
115.239.212.194
115.239.212.195
115.239.212.196
115.239.212.197
115.239.212.198
115.239.212.199
115.239.212.200google爬虫列表
爬虫Agent：Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview) Chrome/27.0.1453 Safari/537.36
45.79.76.220
爬虫Agent：Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Search Console) Chrome/27.0.1453 Safari/537.36
66.249.84.239
66.249.84.233
爬虫Agent：Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
66.249.79.26
66.249.79.142
66.249.79.138
66.249.75.7
66.249.75.23
66.249.75.15
66.249.69.48
66.249.69.15
66.249.65.55
66.249.65.162
203.208.60.158
203.208.60.152
爬虫Agent：Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
66.249.79.69
66.249.79.64
66.249.79.25
66.249.79.24
66.249.79.231
66.249.79.210
66.249.79.194
66.249.79.187
66.249.79.180
66.249.79.174
66.249.79.171
66.249.79.168
66.249.79.146
66.249.79.127
66.249.75.8
66.249.75.24
66.249.75.16
66.249.69.254
66.249.69.213
66.249.69.209
66.249.69.206
66.249.69.197
66.249.69.192
66.249.69.19
66.249.69.188
66.249.69.176
66.249.69.140
66.249.69.137
66.249.69.134
66.249.69.128
66.249.69.11
66.249.66.67
66.249.66.64
66.249.66.254
66.249.66.197
66.249.66.186
66.249.66.183
66.249.66.180
66.249.66.168
66.249.66.166
66.249.66.162
66.249.66.158
66.249.66.154
66.249.66.150
66.249.66.142
66.249.66.139
66.249.66.138
66.249.66.135
66.249.66.134
66.249.66.126
66.249.65.61
66.249.65.60
66.249.65.59
66.249.65.57
66.249.65.56
66.249.65.186
66.249.65.183
66.249.65.180
66.249.65.168
66.249.65.166
66.249.64.50
66.249.64.140
66.249.64.137
66.249.64.134
54.159.17.214
203.208.60.157
203.208.60.156
203.208.60.155
203.208.60.154
203.208.60.153
203.208.60.151
203.208.60.150
203.208.60.149
122.155.168.168
爬虫Agent：Mozilla/5.0 (compatible; Google-Site-Verification/1.0)
66.249.82.245
66.249.82.242
66.249.82.239
66.249.82.208
66.249.82.205
爬虫Agent：DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
66.249.66.192#网上搜集
67.221.235.*
66.249.68.*
66.249.67.*
203.208.60.*
66.249.72.*
66.249.71.*360爬虫列表
爬虫Agent：Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider
42.236.49.171
42.236.10.*
爬虫Agent：Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html)
182.118.25.*
182.118.22.*
182.118.21.*
182.118.20.*
爬虫Agent：Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider
180.153.187.*
180.153.186.*
180.153.185.*
爬虫Agent：Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1; 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html)
101.226.169.*
101.226.168.*
101.226.167.*
101.226.166.*360爬虫这个倒是挺规矩的，基本上都是统一号段的IP都用一个UA。
必应爬虫列表
爬虫Agent：Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
40.77.167.95
40.77.167.39
40.77.167.19
207.46.13.183
207.46.13.151
207.46.13.64
207.46.13.13
157.55.39.212
157.55.39.203
157.55.39.188
157.55.39.169
157.55.39.131#网上搜集
65.52.108.142
74.125.75.3
74.125.64.81
74.125.44.82
74.125.158.86
74.125.156.82
65.52.108.146soso爬虫列表
#网上搜集
124.115.4.*
124.115.0.*sogou爬虫列表
#网上搜集
220.181.94.231
220.181.94.229
220.181.94.223
220.181.125.71
220.181.125.69
220.181.125.45
123.126.50.76
123.126.50.70
220.181.94.237
220.181.94.235
220.181.94.233
220.181.94.225
220.181.94.224
220.181.94.213
220.181.125.43
220.181.125.162
220.181.125.108
202.85.214.159
123.126.50.81
123.126.50.78
123.126.50.77
123.126.50.75
123.126.50.74
123.126.50.72
123.126.50.71
123.126.50.69
123.126.50.68
123.126.50.66yahoo爬虫列表
#网络搜集
72.30.142.223
67.195.37.168
67.195.37.154
202.160.180.198
202.160.180.176
202.160.180.118
202.160.179.16
202.160.179.127
202.160.178.70
110.75.176.30
110.75.176.29
110.75.176.28
110.75.176.27
110.75.176.26
110.75.176.25
110.75.173.176
110.75.173.175
110.75.173.174
110.75.173.173
110.75.173.172
110.75.173.171
74.6.18.249
203.209.252.21
202.160.189.241
202.160.189.234
202.160.188.215
202.160.184.15
202.160.182.11
202.160.181.190
202.160.180.8
202.160.180.73
202.160.180.66
202.160.180.59
202.160.180.53
202.160.180.45
202.160.180.39
202.160.180.32
202.160.180.27
202.160.180.191
202.160.180.187
202.160.180.17
202.160.180.165
202.160.180.163
202.160.180.16
202.160.180.158
202.160.180.155
202.160.180.154
202.160.180.148
202.160.180.138
202.160.180.136有道爬虫列表
#网络搜集
61.135.217.27
61.135.249.9
61.135.249.89
61.135.249.78
61.135.249.75
61.135.249.62
61.135.249.6
61.135.249.59
61.135.249.37
61.135.249.32
61.135.249.249
61.135.249.239
61.135.249.236
61.135.249.232
61.135.249.231
61.135.249.229
61.135.249.220
61.135.249.216
61.135.249.213
61.135.249.210
61.135.249.191
61.135.249.187
61.135.249.175
61.135.249.171
61.135.249.146
61.135.249.144
61.135.249.143
61.135.249.137
61.135.249.122
61.135.249.112
61.135.248.228
61.135.248.226
61.135.248.219
61.135.248.210
61.135.248.203
61.135.220.9
61.135.220.3msn爬虫列表
#网络搜集
207.46.204.38
207.46.204.37
207.46.204.35
207.46.204.128
207.46.199.244
207.46.199.242
207.46.199.213
207.46.194.95
207.46.194.91
207.46.194.88
207.46.194.85
207.46.194.78
207.46.194.67
207.46.194.55
207.46.194.140
207.46.194.130
207.46.194.129
207.46.204.44
207.46.204.43
207.46.204.42
207.46.204.40
207.46.204.39
207.46.204.34
207.46.204.31
207.46.204.30
207.46.204.138
207.46.204.137
207.46.204.133
207.46.204.129
207.46.199.249
207.46.199.246
207.46.199.240
207.46.199.238
207.46.199.229
207.46.199.218
207.46.199.216
207.46.199.215
207.46.195.167
207.46.194.96
207.46.194.94
207.46.194.90
207.46.194.89
207.46.194.87
207.46.194.83
207.46.194.82
207.46.194.81
207.46.194.77
207.46.194.75
207.46.194.74
207.46.194.73针对爬虫IP，至于是喷杀虫药还是放进来自己喂，就看各位施主了。
此篇文章，可以理解为我闲的纯记录所用，不过目测会有人用到。
...
送给每个来过的人，不管你是刚开始学习的小白，还是独领风骚的老司机：不是每个人都能认同你，即使是你身边的人。人嘛，都这样，谁说不是呢？要一直相信在以后的日子里一定还会有来自各方诸多的刁难、抱怨，你不可能把所有人的嘴巴都堵上，也不可能让所有人的观点都和你一致。你要做的就是尽自己最大的努力，用实际行动去回敬那些曾反对、刁难过你的人。
我可以对一个人无限的好，前提是值得。 ——慕冬雪图片描述
好帅的混血狼（拿来镇楼）！
最近整理了一下各平台爬虫的信息，记录一下（有异议的地方请指正）。有些个人的恶意IP并没统计。以下整理的爬虫IP都是根据对应的User-Agent排序的，所以可能统一号段的IP分属不同UA（当然UA也是可以随意伪造的），加上网上搜集的一些但是UA不确定所以都放到每个分组的最后边。
百度爬虫IP列表
爬虫Agent：Mozilla/5.0 (Windows NT 6.1; WOW64; Baidu Transcoder) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11
61.135.169.75
61.135.169.55
61.135.169.22
220.181.51.40
220.181.51.39
111.13.13.74
111.13.13.73
111.13.13.72
111.13.13.71
111.13.13.6
111.13.13.5
111.13.13.4
111.13.13.3
101.254.184.206
1.95.9.244
爬虫Agent：Mozilla/5.0 (compatible; heritrix/3.1.1 +http://www.baidu.com)
59.50.71.83
211.89.227.50
211.89.227.16
211.89.227.15
183.224.87.36
124.119.30.90
117.25.173.119
116.113.28.179
爬虫Agent：Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
61.191.191.87
61.184.241.206
61.183.41.95
59.49.46.165
58.222.20.226
58.218.204.136
45.113.253.53
27.151.30.176
27.151.28.24
222.216.28.45
222.186.3.229
220.181.51.81
220.181.51.76
220.181.51.75
220.181.51.74
220.181.51.110
220.181.51.109
220.181.51.107
220.181.108.97
220.181.108.96
220.181.108.87
220.181.108.86
220.181.108.84
220.181.108.83
220.181.108.81
220.181.108.80
220.181.108.79
220.181.108.78
220.181.108.77
220.181.108.76
220.181.108.75
220.181.108.186
220.181.108.184
220.181.108.181
220.181.108.179
220.181.108.178
220.181.108.177
220.181.108.175
220.181.108.174
220.181.108.173
220.181.108.172
220.181.108.171
220.181.108.170
220.181.108.168
220.181.108.166
220.181.108.165
220.181.108.161
220.181.108.158
220.181.108.157
220.181.108.156
220.181.108.155
220.181.108.153
220.181.108.152
220.181.108.151
220.181.108.150
220.181.108.149
220.181.108.147
220.181.108.146
220.181.108.145
220.181.108.144
220.181.108.143
220.181.108.142
220.181.108.141
220.181.108.140
220.181.108.139
220.181.108.123
220.181.108.122
220.181.108.120
220.181.108.119
220.181.108.117
220.181.108.116
220.181.108.115
220.181.108.114
220.181.108.113
220.181.108.112
220.181.108.111
220.181.108.110
220.181.108.109
220.181.108.108
220.181.108.107
220.181.108.106
220.181.108.105
220.181.108.104
220.181.108.103
220.181.108.101
220.181.108.100
219.153.68.137
219.138.135.236
202.46.52.201
202.46.51.156
202.46.51.132
183.61.171.152
183.61.171.137
183.2.242.150
180.97.35.165
180.76.15.9
180.76.15.8
180.76.15.7
180.76.15.6
180.76.15.5
180.76.15.34
180.76.15.33
180.76.15.32
180.76.15.31
180.76.15.30
180.76.15.29
180.76.15.28
180.76.15.26
180.76.15.24
180.76.15.23
180.76.15.22
180.76.15.21
180.76.15.19
180.76.15.18
180.76.15.17
180.76.15.163
180.76.15.162
180.76.15.161
180.76.15.158
180.76.15.157
180.76.15.156
180.76.15.155
180.76.15.154
180.76.15.153
180.76.15.152
180.76.15.151
180.76.15.150
180.76.15.15
180.76.15.149
180.76.15.148
180.76.15.147
180.76.15.145
180.76.15.144
180.76.15.143
180.76.15.142
180.76.15.141
180.76.15.14
180.76.15.138
180.76.15.137
180.76.15.136
180.76.15.13
180.76.15.12
180.76.15.11
180.76.15.10
14.29.49.206
124.237.78.217
124.232.151.230
124.228.91.179
123.125.71.98
123.125.71.94
123.125.71.92
123.125.71.91
123.125.71.90
123.125.71.89
123.125.71.87
123.125.71.86
123.125.71.85
123.125.71.84
123.125.71.83
123.125.71.82
123.125.71.81
123.125.71.78
123.125.71.74
123.125.71.73
123.125.71.70
123.125.71.69
123.125.71.60
123.125.71.58
123.125.71.57
123.125.71.56
123.125.71.55
123.125.71.53
123.125.71.52
123.125.71.50
123.125.71.49
123.125.71.47
123.125.71.45
123.125.71.44
123.125.71.43
123.125.71.42
123.125.71.41
123.125.71.39
123.125.71.38
123.125.71.34
123.125.71.33
123.125.71.32
123.125.71.31
123.125.71.30
123.125.71.29
123.125.71.28
123.125.71.27
123.125.71.26
123.125.71.23
123.125.71.22
123.125.71.20
123.125.71.19
123.125.71.18
123.125.71.17
123.125.71.15
123.125.71.14
123.125.71.13
123.125.71.12
123.125.71.116
123.125.71.115
123.125.71.114
123.125.71.113
123.125.71.111
123.125.71.110
123.125.71.109
123.125.71.108
123.125.71.107
123.125.71.106
123.125.71.105
123.125.71.103
123.125.71.102
123.125.71.100
123.125.67.152
123.125.67.148
122.10.89.22
120.39.244.111
117.34.115.181
117.25.139.66
117.25.139.42
117.25.139.35
117.25.139.34
116.9.156.37
116.55.241.24
115.47.26.143
115.231.26.31
113.113.120.28
112.124.57.11
爬虫Agent：Baiduspider+(+http://www.baidu.com/search/spider.htm)
61.146.178.41
61.146.178.186
爬虫Agent：Baidu-YunGuance-VSBot(ce.baidu.com)
111.206.36.17#网络搜集
180.149.130.*
180.76.5.66百度云观测IP
安全检测服务User-Agent：
User-Agent:Baidu-YunGuanCe-ScanBot(ce.baidu.com） 可用性检测
其他服务User-Agent：
User-Agent:Baidu-YunGuanCe-SLABot(ce.baidu.com） 可用性检测
User-Agent:Baidu-YunGuance-PerfBot(ce.baidu.com) 速度检测
User-Agent:Baidu-YunGuance-VSBot(ce.baidu.com) 认证服务
User-Agent:Baidu-YunGuance-RSBot(ce.baidu.com) 即时速度检测
180.97.106.36
180.97.106.37
180.97.106.161
180.97.106.162
115.239.212.7
115.239.212.8
115.239.212.9
115.239.212.10
115.239.212.11
115.239.212.6
115.239.212.4
115.239.212.5
115.239.212.65
115.239.212.66
115.239.212.67
115.239.212.68
115.239.212.69
115.239.212.70
115.239.212.71
115.239.212.72
115.239.212.134
115.239.212.135
115.239.212.136
115.239.212.137
115.239.212.138
115.239.212.139
115.239.212.132
115.239.212.133
115.239.212.193
115.239.212.194
115.239.212.195
115.239.212.196
115.239.212.197
115.239.212.198
115.239.212.199
115.239.212.200google爬虫列表
爬虫Agent：Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview) Chrome/27.0.1453 Safari/537.36
45.79.76.220
爬虫Agent：Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Search Console) Chrome/27.0.1453 Safari/537.36
66.249.84.239
66.249.84.233
爬虫Agent：Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
66.249.79.26
66.249.79.142
66.249.79.138
66.249.75.7
66.249.75.23
66.249.75.15
66.249.69.48
66.249.69.15
66.249.65.55
66.249.65.162
203.208.60.158
203.208.60.152
爬虫Agent：Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
66.249.79.69
66.249.79.64
66.249.79.25
66.249.79.24
66.249.79.231
66.249.79.210
66.249.79.194
66.249.79.187
66.249.79.180
66.249.79.174
66.249.79.171
66.249.79.168
66.249.79.146
66.249.79.127
66.249.75.8
66.249.75.24
66.249.75.16
66.249.69.254
66.249.69.213
66.249.69.209
66.249.69.206
66.249.69.197
66.249.69.192
66.249.69.19
66.249.69.188
66.249.69.176
66.249.69.140
66.249.69.137
66.249.69.134
66.249.69.128
66.249.69.11
66.249.66.67
66.249.66.64
66.249.66.254
66.249.66.197
66.249.66.186
66.249.66.183
66.249.66.180
66.249.66.168
66.249.66.166
66.249.66.162
66.249.66.158
66.249.66.154
66.249.66.150
66.249.66.142
66.249.66.139
66.249.66.138
66.249.66.135
66.249.66.134
66.249.66.126
66.249.65.61
66.249.65.60
66.249.65.59
66.249.65.57
66.249.65.56
66.249.65.186
66.249.65.183
66.249.65.180
66.249.65.168
66.249.65.166
66.249.64.50
66.249.64.140
66.249.64.137
66.249.64.134
54.159.17.214
203.208.60.157
203.208.60.156
203.208.60.155
203.208.60.154
203.208.60.153
203.208.60.151
203.208.60.150
203.208.60.149
122.155.168.168
爬虫Agent：Mozilla/5.0 (compatible; Google-Site-Verification/1.0)
66.249.82.245
66.249.82.242
66.249.82.239
66.249.82.208
66.249.82.205
爬虫Agent：DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
66.249.66.192#网上搜集
67.221.235.*
66.249.68.*
66.249.67.*
203.208.60.*
66.249.72.*
66.249.71.*360爬虫列表
爬虫Agent：Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider
42.236.49.171
42.236.10.*
爬虫Agent：Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html)
182.118.25.*
182.118.22.*
182.118.21.*
182.118.20.*
爬虫Agent：Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider
180.153.187.*
180.153.186.*
180.153.185.*
爬虫Agent：Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1; 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html)
101.226.169.*
101.226.168.*
101.226.167.*
101.226.166.*360爬虫这个倒是挺规矩的，基本上都是统一号段的IP都用一个UA。
必应爬虫列表
爬虫Agent：Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
40.77.167.95
40.77.167.39
40.77.167.19
207.46.13.183
207.46.13.151
207.46.13.64
207.46.13.13
157.55.39.212
157.55.39.203
157.55.39.188
157.55.39.169
157.55.39.131#网上搜集
65.52.108.142
74.125.75.3
74.125.64.81
74.125.44.82
74.125.158.86
74.125.156.82
65.52.108.146soso爬虫列表
#网上搜集
124.115.4.*
124.115.0.*sogou爬虫列表
#网上搜集
220.181.94.231
220.181.94.229
220.181.94.223
220.181.125.71
220.181.125.69
220.181.125.45
123.126.50.76
123.126.50.70
220.181.94.237
220.181.94.235
220.181.94.233
220.181.94.225
220.181.94.224
220.181.94.213
220.181.125.43
220.181.125.162
220.181.125.108
202.85.214.159
123.126.50.81
123.126.50.78
123.126.50.77
123.126.50.75
123.126.50.74
123.126.50.72
123.126.50.71
123.126.50.69
123.126.50.68
123.126.50.66yahoo爬虫列表
#网络搜集
72.30.142.223
67.195.37.168
67.195.37.154
202.160.180.198
202.160.180.176
202.160.180.118
202.160.179.16
202.160.179.127
202.160.178.70
110.75.176.30
110.75.176.29
110.75.176.28
110.75.176.27
110.75.176.26
110.75.176.25
110.75.173.176
110.75.173.175
110.75.173.174
110.75.173.173
110.75.173.172
110.75.173.171
74.6.18.249
203.209.252.21
202.160.189.241
202.160.189.234
202.160.188.215
202.160.184.15
202.160.182.11
202.160.181.190
202.160.180.8
202.160.180.73
202.160.180.66
202.160.180.59
202.160.180.53
202.160.180.45
202.160.180.39
202.160.180.32
202.160.180.27
202.160.180.191
202.160.180.187
202.160.180.17
202.160.180.165
202.160.180.163
202.160.180.16
202.160.180.158
202.160.180.155
202.160.180.154
202.160.180.148
202.160.180.138
202.160.180.136有道爬虫列表
#网络搜集
61.135.217.27
61.135.249.9
61.135.249.89
61.135.249.78
61.135.249.75
61.135.249.62
61.135.249.6
61.135.249.59
61.135.249.37
61.135.249.32
61.135.249.249
61.135.249.239
61.135.249.236
61.135.249.232
61.135.249.231
61.135.249.229
61.135.249.220
61.135.249.216
61.135.249.213
61.135.249.210
61.135.249.191
61.135.249.187
61.135.249.175
61.135.249.171
61.135.249.146
61.135.249.144
61.135.249.143
61.135.249.137
61.135.249.122
61.135.249.112
61.135.248.228
61.135.248.226
61.135.248.219
61.135.248.210
61.135.248.203
61.135.220.9
61.135.220.3msn爬虫列表
#网络搜集
207.46.204.38
207.46.204.37
207.46.204.35
207.46.204.128
207.46.199.244
207.46.199.242
207.46.199.213
207.46.194.95
207.46.194.91
207.46.194.88
207.46.194.85
207.46.194.78
207.46.194.67
207.46.194.55
207.46.194.140
207.46.194.130
207.46.194.129
207.46.204.44
207.46.204.43
207.46.204.42
207.46.204.40
207.46.204.39
207.46.204.34
207.46.204.31
207.46.204.30
207.46.204.138
207.46.204.137
207.46.204.133
207.46.204.129
207.46.199.249
207.46.199.246
207.46.199.240
207.46.199.238
207.46.199.229
207.46.199.218
207.46.199.216
207.46.199.215
207.46.195.167
207.46.194.96
207.46.194.94
207.46.194.90
207.46.194.89
207.46.194.87
207.46.194.83
207.46.194.82
207.46.194.81
207.46.194.77
207.46.194.75
207.46.194.74
207.46.194.73针对爬虫IP，至于是喷杀虫药还是放进来自己喂，就看各位施主了。
此篇文章，可以理解为我闲的纯记录所用，不过目测会有人用到。
...
送给每个来过的人，不管你是刚开始学习的小白，还是独领风骚的老司机：不是每个人都能认同你，即使是你身边的人。人嘛，都这样，谁说不是呢？要一直相信在以后的日子里一定还会有来自各方诸多的刁难、抱怨，你不可能把所有人的嘴巴都堵上，也不可能让所有人的观点都和你一致。你要做的就是尽自己最大的努力，用实际行动去回敬那些曾反对、刁难过你的人。
我可以对一个人无限的好，前提是值得。
想要了解更多java知识更多学习资料的那就来关注我们一起学习交流官方群253772578
加好友 
发短信
Post By：2019/3/14 0:35:38 [只看该作者]
Powered By Dvbbs Version 8.3.0
