com.virjar:dungproxy-webmagic5

proxy ip resource pool

License	License The Apache License, Version 2.0
Categories	Categories webmagic Business Logic Libraries Web Crawling
GroupId	GroupId com.virjar
ArtifactId	ArtifactId dungproxy-webmagic5
Last Version	Last Version 0.0.1
Release Date	Release Date Jun 2, 2017
Type	Type jar
Description	Description proxy ip resource pool

Download dungproxy-webmagic5

Filename	Size
dungproxy-webmagic5-0.0.1.pom
dungproxy-webmagic5-0.0.1.jar	13 KB
dungproxy-webmagic5-0.0.1-sources.jar	9 KB
dungproxy-webmagic5-0.0.1-javadoc.jar	50 KB
Browse

How to add to project

Apache Maven

<!-- https://jarcasting.com/artifacts/com.virjar/dungproxy-webmagic5/ -->
<dependency>
    <groupId>com.virjar</groupId>
    <artifactId>dungproxy-webmagic5</artifactId>
    <version>0.0.1</version>
</dependency>

Gradle Groovy

// https://jarcasting.com/artifacts/com.virjar/dungproxy-webmagic5/
implementation 'com.virjar:dungproxy-webmagic5:0.0.1'

Gradle Kotlin

// https://jarcasting.com/artifacts/com.virjar/dungproxy-webmagic5/
implementation ("com.virjar:dungproxy-webmagic5:0.0.1")

Apache Buildr

'com.virjar:dungproxy-webmagic5:jar:0.0.1'

Apache Ivy

<dependency org="com.virjar" name="dungproxy-webmagic5" rev="0.0.1">
  <artifact name="dungproxy-webmagic5" type="jar" />
</dependency>

Groovy Grape

@Grapes(
@Grab(group='com.virjar', module='dungproxy-webmagic5', version='0.0.1')
)

Scala SBT

libraryDependencies += "com.virjar" % "dungproxy-webmagic5" % "0.0.1"

Leiningen

[com.virjar/dungproxy-webmagic5 "0.0.1"]

Dependencies

compile (2)

Group / Artifact	Type	Version
com.virjar : dungproxy-client	jar	0.0.7
us.codecraft : webmagic-core	jar	0.5.3

Project Modules

There are no modules declared in this project.

DungProxy

DungProxy是一个代理IP服务,他包括一个代理IP资源server端和一系列适配中心IP资源得客户端。server负责代理IP资源的收集维护。client则是一系列方便用户使用得API,他屏蔽了代理IP下载、代理IP选取、IP绑定、IP切换等比较复杂逻辑。用户只需要引入client即可方便使用代理IP服务

特别声明 :本项目服务器已下线,默认客户端无法获取代理ip,最近半年没有启用计划。你可以自建server,或者就不要使用被本项目了
特别声明2: dungproxy的停止维护，他的思想会在另一个重构项目上实现。将会有极致的性能优化效果，感兴趣的可以参考dungpxoy的下一代项目： phantom

广告:dungproxy-client后续主要对vscrawler提供支持,vscrawler中,代理的切换策略会变得更加灵活,vscrawler地址: vscrawler,适合抓取得爬虫框架

client使用手册

演示视频

java client设计说明

server 部署手册

server设计说明

请注意,在没有遇到封IP的情况下,不要尝试使用本工具。使用代理的效果肯定比不上不使用代理

qq 群,人肉文档支持 😁

569543649(暗号：DungProxy)

案例

代理IP爬虫我们通过代理访问封堵我们的代理资源发布网站,以及访问国外的代理IP网站

##maven坐标

<dependency>
    <groupId>com.virjar</groupId>
    <artifactId>dungproxy-client</artifactId>
    <version>0.0.6</version>
</dependency>

##快速开始

请注意,虫0.0.6开始,dungproxy默认关闭代理IP池功能,打开方式IpPoolHolder.init(DungProxyContext.create().setPoolEnabled(true));

IpPoolHolder.init(DungProxyContext.create().setPoolEnabled(true));
HttpClientBuilder httpClientBuilder = HttpClientBuilder.create();
httpClientBuilder.setRetryHandler(new DunProxyHttpRequestRetryHandler(null))
        .setRoutePlanner(new ProxyBindRoutPlanner());
CloseableHttpClient closeableHttpClient = httpClientBuilder.build();

HttpGet httpGet = new HttpGet("http://www.baidu.com");
CloseableHttpResponse response = closeableHttpClient.execute(httpGet);

String string = IOUtils.toString(response.getEntity().getContent());
System.out.println(string);

serverList

115.159.40.202 收集&分发
123.56.155.209 验证
114.215.144.211 地址同步&数据备份&域名下线&域名刷新
121.42.249.101 域名验证,端口验证

贡献者

邓维佳 [email protected]
符灵通
韦轩 [email protected]

项目主页

oshina http://git.oschina.net/virjar/proxyipcenter
github https://github.com/virjar/dungproxy

附成功率报告

0.0,0.5198585434285783,0.8378301099889928,0.9400409853285343,0.9801515822006206,0.9925873249283991,0.9972593143317614,0.9988682347546196,0.9995183318895711,0.9887699715297846,0.9958894463864819,0.9984336853917545,0.999438087300657,0.9997943217790684,0.9948285473649975,0.9981447501706506,0.9992930623991622,0.9997463873391322,0.9999080978683261,0.9999689596505754,0.9999891950680941,0.99999575678408,0.9999984929759771,0.9999994140932955,0.9999997980894105,0.99999992756492,0.9999999737515539,0.9999999903922192,0.9999999964832412,0.9999999986997503,0.9999999995652269,0.9999999998392518,0.9999999999417493,0.9999999999791029,0.999999999992726,0.9999999999973899,0.9999999999990352,0.9999999999996324,0.999999999999876,0.9999999999999537,0.9999999999999848,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.9931055091413006,0.9975513470096478,0.9990946604171477,0.9996719302685132,0.9998787028970538,0.9999551528660793,0.9999837486535864,0.9999940514816233,0.9907703757056113,0.996787231333771,0.9987634179379771,0.9995473709726613,0.9998376201069609,0.9999405637075748,0.999978024641138,0.9999919563079458,0.9999971143391688,0.9999989437548095,0.9999996133800998,0.9999998613007688,0.9999999482007912,0.9999999812294165,0.9999999930599585,0.99999999745972,0.9999999990794738,0.9999999996630575,0.9999999998779014,0.9999999999574986,0.9999999999836411,0.9999999999938907,0.9999999999977639,0.9999999999991901,0.999999999999701,0.9999999999998905,0.9999999999999604,0.9999999999999852,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.9944173386145141,0.9979565653825354,0.9992742520964651,0.999726221294359,0.9998987758983674,0.999962948705086,0.9999865736474982,0.999995134665551,0.9999981829713698,0.9999993415596684,0.9999997589895435,0.9952465122444977,0.9983286299419042,0.9993631292560765,0.9997692158634081,0.9999129397661486,0.9999681331387721,0.9999885678177818,0.9933730783142507,0.9975250814610045,0.9990940997727108,0.9996684112188451,0.9998786277820938,0.9999546718119718,0.9999843793541556,0.9999940477979176,0.999997799294481,0.9999992025259007,0.999999702171909,0.9999998920754338,0.9999999600970889,0.9999999855403012,0.9999999947072823,0.9909561773889444,0.9965538774574086,0.9987637094042421,0.9995382896222231,0.9998292919893257,0.9999393712038522,0.9999773573102716,0.9999917949228336,0.9999969663397921,0.9999989440037695,0.9999995935511915,0.9999998512265907,0.9999999466277925,0.9999999802667128,0.9999999926303215,0.9999999974603191,0.9933102814367898,0.9975513466518494,0.9991126765451901,0.9996719302205751,0.9998787028793293,0.9999560453176217,0.9999840720530522,0.9999941698562871,0.9999978659788471,0.9999992420793089,0.9999997225765148,0.9999998974283151,0.999999962455446,0.999999986394904,0.9999999949697931,0.999999998177194,0.9999999993260547,0.9999999997533137,0.999999999911501,0.9999999999676064,0.9958705032768045,0.9984732026728573,0.9994467313716991,0.9997954401905358,0.9999251244940113,0.9999723163063023,0.9999898668727805,0.9999962909477195,0.9999986423669098,0.9999995030623811,0.9999998199237126,0.9999999340862545,0.9999999763535561,0.9999999910796947,0.9999999967675306,0.9999999988519532,0.9999999995922592,0.9999999998430611,0.9999999999431296,0.9999999999793922,0.9999999999923038,0.9999999999972117,0.9999999999989694,0.9999999999996233,0.9999999999998664,0.9999999999999493,0.9999999999999816,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.996112160819253,0.998576925122178,0.9994791085706636,0.9998074110005767,0.9999295061976379,0.9999747104684054,0.999990743213541,0.9999965429209048,0.9999987472512718,0.9999995460389157,0.9999998338355612,0.9999999403887901,0.9999999786145801,0.9999999920133095,0.9999999970766127,0.9999999989406461,0.9944731648444961,0.9979769995881996,0.999259516422872,0.9878967617022839,0.9955698233488997,0.986499083690786,0.9950582279940048,0.9983475858091686,0.9770230230247468,0.9915047306286532,0.9969215520956365,0.9988618065717961,0.9995957582947466,0.9998505398607936,0.9951449668637738,0.9982229008541884,0.9993751555793664,0.9997666429280523,0.9999145837646026,0.9936708112343886,0.9976362740717923,0.9991520173470694,0.9996833087686858,0.9998852399594692,0.9937223158628861,0.9977251429317433,0.9991920599103001,0.999695215389647,0.9998884389754886,0.9999591650570029,0.9999850530902086,0.9999946378215567,0.9999980174437069,0.9999992887613044,0.999999737033975,0.9999999084633374,0.9999999654690166,0.9999999899691459,0.9910466127842311,0.9967879875778556,0.9988124238110345,0.9995696556200414,0.99984248003905,0.9999434899821583,0.9999791065715793,0.9999924288061834,0.9999972564112188,0.999998995757775,0.9999996433325131,0.9999998681294592,0.9999999517311173,0.9999999823320276,0.9999999935329507,0.9999999976565226,0.9999999991335468,0.9999999996860217,0.9999999998850732,0.9999999999583534,0.9999999999844469,0.9999999999943072,0.9999999999979582,0.9999999999992376,0.999999999999727,0.9960332219356826,0.9892587997248634,0.9959070965901755,0.9984714467730808,0.99944050008359,0.9997931362986404,0.9999242811950669,0.9999722844685519,0.9999901565292704,0.9999963969713623,0.9999986943632418,0.9999995123913074,0.9999998215194489,0.9999999353236423,0.9999999763263616,0.9999999912471543,0.9999999967961756,0.9959931523095088,0.9985625502119659,0.9937778348638515,0.997722486327427,0.9991746927747652,0.9996948594586644,0.9998871804982198,0.9999599309338331,0.9999853334258979,0.9999945773328716,0.9999980740811253,0.999999280738094,0.999999749629883,0.9999999074307477,0.9999999661166598,0.9933771677956033,0.9974508978533579,0.99906694617312,0.999661887401981,0.9998749897516493,0.9999556012583497,0.9999834186558969,0.9999939913848784,0.9999978006525393,0.9999991868360598,0.9999997082788208,0.9999998932206133,0.999999962076146,0.9999999859784267,0.9999999948158086,0.9999999981024179,0.9999999992984079,0.9999999997457626,0.9999999999078709,0.9999999999662776,0.9999999999876569,0.9999999999954823,0.99999999999838,0.9999999999994071,0.9999999999997813,0.9999999999999202,0.9999999999999711,0.999999999999988,0.999999999999988,0.999999999999988,0.9815282446861404,0.9931014591929813,0.9974749109570383,0.9990849783882245,0.9996750201788889,0.9998834140423959,0.9999555752568439,0.9999839017161797,0.9999941664324075,0.9999979072175571,0.9999992416342033,0.9999997196096888,0.9999998983943983,0.9999999624333982,0.9999999865230457,0.9999999950669998,0.9999999982124185,0.9999999993456876,0.999999999774516,0.9999999999132121,0.9999999999685513,0.9999999999883725,0.9999999999957433,0.9999999999984419,0.9999999999994412,0.9999999999998019,0.999999999999927,0.9999999999999729,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.9957440987661081,0.9984422025071314,0.9918826028644199,0.997028770121419,0.9989124337710816,0.9995978945322618,0.9998528163942046,0.999946664779772,0.9999810574197315,0.9999929963666606,0.9999975864626239,0.9999990895252532,0.999999656535559,0.999999875538097,0.9999999553495046,0.9999999836564751,0.9823609353063151,0.9933458983800882,0.9896282407685568,0.9962415646785386,0.9947364519390186,0.9980926374680361,0.9992947915421901,0.9997418708971023,0.9999073966237526,0.9999661041693824,0.9999878399284675,0.9999954586476351,0.9999983377181614,0.9999993915510867,0.99999977728802,0.9999999184802124,0.9999999707549143,0.9999999891872251,0.9999999960817532,0.9999999985366749,0.9999999994643752,0.9999999998078455,0.9999999999282372,0.9999999999737327,0.999999999990386,0.9999999999965159,0.9999999999987108,0.9999999999995282,0.9999999999998275,0.9999999999999382,0.9999999999999778,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.999999999999988,0.9941296321806177,0.9977854821276086,0.9992055454830941,0.9997092039531421,0.9998924840829657,0.9999610392401919,0.9999858817108515,0.99104271725649,0.9955675902135412,0.9619967629380355,0.9853727030695897,0.986102058626122,0.9882034932210207,0.9956384818230752,0.9984035432901943,0.986885069844016,0.9864814005730658,0.9949002888810807,0.9981333407993255,0.9993167423624176,0.9997499056072227,0.9999093727902337,0.9999674876427729,0.9829705154374193,0.9818833088549812,0.964407356032325,0.9560115923229564,0.9565530888209208,0.9767900674314974,0.991589369900746,0.9883552868964597,0.9957376583992563,0.9984398451245733,0.9957743201168843,0.9984992040900298,0.9943617345596348,0.9978943092543041,0.9992596197476494,0.9997422787320508,0.9712026018646892,0.964060189732713,0.9828494126870249,0.988430405237557,0.9958075026001051,0.9984654103609347,0.9922666023299214,0.9971976330813765,0.977330392559146,0.9849031847409812,0.9863785354635458,0.985766701975479,0.980122213643618,0.9927240873206447,0.9973098794414911,0.9990153288736624,0.9996502834005532,0.9998732724901792,0.9919377716011576,0.9904261432534526,0.9826351346951551,0.9935796946430028,0.9977197591187532,0.9991484114800226,0.9996851424850823,0.9919531503625639,0.9931667536065933,0.9974480265525199,0.9929023348228876,0.9974020249976482,0.9952385859462736,0.9982918508136663,0.9993684466202853,0.9997779400172597,0.9946096634350637,0.9980070328150358,0.999277804459655,0.9997382965447552,0.9932813876617174,0.9975407705957116,0.9990907500034549,0.9996671850949613,0.9960304364023519,0.9985759258168041,0.9994734775682607,0.9998072757615779,0.9999294566957907,0.9949255781926359,0.9981795598148883,0.9993269293099957,0.9820552982535289,0.9934316588062897,0.997595774694585,0.9991199757825108,0.9996713423888334,0.9998797006851078,0.9999559665601161,0.9999840435135364,0.9999942756376443,0.9999978835335805,0.9999992253048411,0.9959928727426361,0.998605146592817,0.9994894385416352,0.9998168369459494,0.9999349476604994,0.9999761887398677,0.9999914577509621,0.9999969964665933,0.9775725817513997,0.9918729311938529,0.9970252299771976,0.9989220265841658,0.9996093725981043,0.9998626518091599,0.9920949257035242,0.9970477371159285,0.9989193763044893,0.9852548174422615,0.9946027863058892,0.9980244452351932,0.9992912730911482,0.99974058303023,0.994858075715835,0.9980796749470642,0.9993041279356195,0.9997427154737857,0.999905825542496,0.9857526173828748,0.9874293081429601,0.995444733025862,0.9983658057275412,0.9957223018543101,0.9984498818911477,0.9941823156977176,0.9978490296909567,0.9992126753017715,0.9997118136973652,0.9998945144929242,0.9999613888928746,0.9999860084151991,0.9803918886274569,0.9892636007208649,0.9922604685397289,0.9970803651031683,0.9989420060110192,0.9834563322889287,0.987646246262438,0.9856079786924044,0.9946788431768954,0.9980127379958749,0.9992725978359068,0.9997337472828296,0.9999044822909618,0.9999643275475237,0.994783797341897,0.9981097941172139,0.9939313933680033,0.9978009067691612,0.9818202956720512,0.993278424506248,0.9975396859849773,0.9990994455008142,0.9901637637499369,0.9963996194160728,0.9986821442653909,0.9995127496767684,0.9998216506234019,0.996047531995776,0.9985532688826066,0.9943296165001937,0.9915599859556714,0.9968794766656325,0.9988462500383072,0.999577690200278,0.9905293586284023,0.9965334389653944,0.9987311265483371,0.9995401957669111,0.9998316967800353,0.9999383955783524,0.9999778995186043,0.9999922292672723,0.999997097908893,0.9999989377407974,0.9999996150669895,0.9900998591020683,0.9963762282481955,0.9986735823414473,0.9995241499229793,0.9902198633822134,0.9964201536958202,0.9986896604758826,0.9995203733561246,0.999826196725239,0.9999363823804179,0.9999767138937582,0.9999914765320136,0.9936351482095453,0.9936053540690842,0.9976593527782928,0.9991345933508526,0.9996895368378163,0.9998874968374482,0.9999584042464751,0.9999847746089503,0.999994537916879,0.9999980007009271,0.9999992681918797,0.9999997321345602,0.9999999029330602,0.9954338676999618,0.9983117655592456,0.999382051595032,0.9997760727895586,0.996145668929652,0.993159526213005,0.9932828231427802,0.9975658830679235,0.9991179441356155,0.9996771390268256,0.9998853324337312,0.9999584476826283,0.9999852422451092,0.9999944328340291,0.9999979622372046,0.9857164459051347,0.9946656027476313,0.9980474380844335,0.9895082324296613,0.990000842295555,0.9963399848946816,0.994502504141554,0.9980670298431553,0.9945649464035722,0.9980105946071537,0.9992644578649643,0.9931106397024655,0.997503488607215,0.9990861960898837,0.999672174402831,0.993519350316208,0.9976515938970377,0.9991490033417136,0.9939924956759805,0.9977788476031012,0.9992031653587204,0.999702410724053,0.9998921619735817,0.9999601290855676,0.9838550085087501,0.9941495068549757,0.989618258755203,0.98970892066199,0.9961950829653331,0.9888062773094508,0.9959027354760147,0.9985152659867356,0.9994565393329564,0.9997970368530269,0.999925708924121,0.990367118892925,0.9964740539751709,0.9987093897213287,0.999527594898026,0.9905098719947811,0.9965263062272669,0.9987285157355003,0.9908609604850459,0.9967213851315627,0.9987631863204776,0.9995472861931658,0.9998359491843045,0.9999405525748848,0.9947748350932993,0.9981065464491052,0.9993069347636738,0.9997463157088922,0.9999071433449808,0.9999660114611608,0.9907588146498246,0.9967178930821905,0.9948347587079032,0.9981093546366124,0.9993009723748705,0.9997492250294521,0.999908208250398,0.9999664012509837,0.9949389658656506,0.9981474978265399,0.9993420638188307,0.9997639665150221,0.9999144680697657,0.999968692547323,0.9999886550551998,0.9999958889094613,0.999998539901735,0.9999994655568136,0.9999998043765094,0.9999999298204061,0.9934433828692167,0.9928706649999219,0.997416528490018,0.9990448140147363,0.9953979660932016,0.9983155067544758,0.9993771929227091,0.9997765690212377,0.9999182170357309,0.9999706605007891,0.9999894745046025,0.9999961858549934,0.9999986178605781,0.9999995091182367,0.9909560031469149,0.9967879267094684,0.9988004053597858,0.999551994250856,0.9998408859131416,0.9999423415072751,0.999978895126914,0.9999921969029212,0.9943011886274344,0.9979140507308253,0.999244110354175,0.9997233199431946,0.9903748275316145,0.9886108429256356,0.995789091081759,0.9984586711497677,0.9994243687300041,0.9997893003385333,0.9999228771096079,0.9999720528225851,0.9999899739976778,0.9999963301588961,0.9999986701522736,0.9999995083158817,0.9999998253730678,0.9911360892621954,0.996787966779806,0.9988360490405632,0.9995696528335256,0.9961078362145527,0.9986176559389954,0.9994837438699163,0.9998166460522431,0.9999328865252205,0.9999756799547188,0.9999912752251561,0.9999968064502366,0.9930347335957687,0.9974504872304675,0.9990573695675374,0.9996549667758504,0.9998762199181922,0.9932654279762879,0.9975595795463412,0.9991156599157789,0.9996827445002144,0.9998838742266319,0.9999579192691784,0.9999845970915751,0.9999943620373668,0.9999979363233373,0.9999992521813239,0.9999997370600683,0.999999903755481,0.9999999647713935,0.9999999876132978,0.9953411877597083,0.9927506642668947,0.9920639881526955,0.9970658212153123,0.9989367357130294,0.9996147027748117,0.9920063761155261,0.987429311039136,0.9954447340753552,0.9788075529078131,0.951664151096509,0.9584593081819253,0.9300661452676633,0.9572867861606998,0.951726934608022,0.9360500288164569,0.9765922423233319,0.9914320036536534,0.9968638362373266,0.9988749065888896,0.99958817942458,0.9998507677470883,0.9999469985514752,0.9999811759623617,0.9862865921583606,0.9905194317978966,0.9868268154243834,0.9951294832401638,0.9909852576541364,0.9967659765293803,0.9988042856740564,0.9995579089755364,0.9998429865915774,0.9999425280145033,0.9999793820230384,0.9999922221435041,0.9999971815224471,0.999998957925316,0.9999996185669636,0.9999998603831719,0.9999999488957252,0.9999999812941823,0.9999999932215357,0.9999999975436742,0.9999999991009055,0.9999999996741938,0.9900532429845458,0.99643161785325,0.9986938567282669,0.99952190932021,0.9941307412908728,0.99785166149316,0.9992136386264941,0.9997092588943122,0.9998967401520353,0.999961821773828,0.9999860255344888,0.9999948848936714,0.9999981277056545,0.9999993283175911,0.9999997491506123,0.9999999109081272,0.9999999663912804,0.9933771676536601,0.9975266086832053,0.999121548324843,0.9996719296772234,0.9998799156516514,0.9999564847923754,0.9999840720266723,0.9912190179181748,0.9968813392499082,0.9988116433088321,0.9995693727878836,0.9925957098611389,0.997262414489905,0.991141173758232,0.9967573830897929,0.9988130973405039,0.9995698996882741,0.9998425693759164,0.9999423753000778,0.9999793272369989,0.9999923566668264,0.9999971740331951,0.9999989861892193,0.999999639934137,0.9933101479425649,0.9976000269626516,0.9991215322501462,0.9996784523927882,0.9998834801447365,0.9999573499645728,0.999984388707677,0.9999942857621202,0.9999979500268882,0.9844379760799525,0.9899302781738375,0.996351014582559,0.9986372342866878,0.9995011836753505,0.9952531454444552,0.9982970740088551,0.9993766740124462,0.9997695379084842,0.9999181488958183,0.9999700398487006,0.9956468300116,0.9984065989971854,0.9994283701027161,0.9997886514852077,0.9999234132122222,0.9999713975704105,0.9999897389270755,0.9999963556689806,0.9999986525828131,0.9999995068017324,0.999999819473483,0.9999999339214563,0.9999999758131162,0.999999991146818,0.994583146096028,0.9980172562831943,0.9992742516751928,0.9942075178698714,0.9979009665620845,0.9952725909533258,0.9982344815819711,0.9993602255282891,0.9997610670872694,0.9999125428265464,0.9999689386390241,0.9999885156942611,0.9999959620123713,0.9999985219659342,0.9922990329437613,0.9971811969983374,0.9926803083563042,0.9973475485689576,0.999009404134866,0.999637409876229,0.999868607485193,0.9939647031056379,0.991299859863794,0.9968788395365419,0.9932299268802661,0.9976195799620308,0.999102018326607,0.9996713096656671,0.9998784734417882,0.9999555173493704,0.9999840419248106,0.9999940998266428,0.9999978619422746,0.9961113860469649,0.9986049663640071,0.999479004766925,0.9998092988950452,0.9999308952557899,0.999974449927963,0.9999908339551505,0.9999966449311438,0.9999987842169284,0.9999995504889653,0.9999998354644241,0.9999999397746577,0.999999977955577,0.9999999920916005,0.9999999971628749,0.9999999989615209,0.999999999623684,0.9999999998608646,0.9999999999495813,0.9823342088881588,0.9935337491188889,0.9976331430507269,0.9921741008625983,0.9968326084041378,0.9988522259160524,0.999575633903839,0.9998462216014222,0.9999442750114035,0.999979396820158,0.9999926825578617,0.9999972945247693,0.9999990097085675,0.9999996375213088,0.9999998673210756,0.9999999524016623,0.9999999824014845,0.9999999936865628,0.9999999977350649,0.999999999179251,0.9999999996934786,0.9999999998878036,0.9999999999593433,0.999999999985119,0.9999999999944978,0.9999999999980262,0.9999999999992784,0.9999999999997358,0.9999999999999035,0.9999999999999651,0.9999999999999872,0.999999999999988,0.9957440987661081,0.9984264671789207,0.9994467313732873,0.9997954401911227,0.9999258732492842,0.9959662920102507,0.9985382970962939,0.9994649694639102,0.9998099781189103,0.9999297432787594,0.9908396230460184,0.9966131371480695,0.9987477764249867,0.9995370158313538,0.999830532820778,0.9999398118985618,0.9934215973104892,0.9975677695572931,0.9991361670544174,0.9996773892505452,0.999884263942819,0.9999584798865065,0.9999846487834827,0.9999944371486942,0.9954775476756822,0.99839380214803,0.9994001424749466,0.9997804327456744,0.9999204349709941,0.9999705824506564,0.9999892322255388,0.9999960980598405,0.9999985573370784,0.9999994719387136,0.9999998124533243,0.9999999327179495,0.9999999751238314,0.9999999911649595,0.9999999967984288,0.9999999988281212,0.9941296317808197,0.9977405182907763,0.9991646026463721,0.9996911288391495,0.9998880737341831,0.9999594410532049,0.9999853025726075,0.9935686640288658,0.9908672232184321,0.9965547819098205,0.9987515493686928,0.9995337482832963,0.9998327329903409,0.9999393871161886,0.999978035586985,0.9999920407113347,0.9999971157765059,0.999998954838113,0.9911357800822919,0.9924776629263262,0.997218769039471,0.9989819795198962,0.9996347868857579,0.9998663201887306,0.9868618564707553,0.990763872280729,0.9880186950002033,0.9955254105493472,0.998362155547141,0.999394440363918,0.9997783455886242,0.9999188673168508,0.9999703028140334,0.9999891298694915,0.9999960211806795,0.9942450251055726,0.9978722152175995,0.9992132949034063,0.9756680712329404,0.9823189411570813,0.9935928790291814,0.9976071691768492,0.9991329050662162,0.999682615211281,0.9998861387473369,0.9999583230991074,0.9999847449063894,0.9999944161423691,0.9999979561275181,0.9914846963856354,0.9593610042494845,0.9494282308659925,0.9813021181227779,0.9931559705196303,0.9974695594601586,0.9990737769247494,0.9996643626752325,0.981144135708021,0.9795732696635742,0.9923713458528737,0.968806740098156,0.9884669273193774,0.995692809309905,0.998407503946863,0.9994170949412015,0.9997866378965883,0.9999219025697493,0.9999714138147578,0.9999895365316879,0.991195600801369,0.9967447526745669,0.9987964385859074,0.9854997405531756,0.994584671039291,0.9980376363171297,0.9992744559865959,0.9997370831518164,0.9999037639304983,0.9999654754738908,0.9999876143850305,0.9999955117997109,0.9999983405793329,0.9999993925983677,0.9962274178055062,0.9815932306074785,0.9932625271039788,0.9880486278523888,0.9956254112713822,0.9983825848945016,0.9994079737621243,0.9997897354790294,0.9999222589748514,0.9999715442705515,0.9999896884398999,0.999996300745368,0.9999986322759252,0.9999995093313161,0.9999998167527733,0.9999999329255885,0.994246428194341,0.9860636800376085,0.9878643792254371,0.9955131013283663,0.9983243036161273,0.9993927745200547,0.992956612942657,0.9932425636805442,0.9975265597629844,0.9990762584216363,0.9996618807073405,0.9998762374036783,0.9999551518982353,0.9999834183275857,0.9999940513532489,0.9934412521711322,0.9975992861763789,0.9991212610984128,0.9943188873589871,0.9978783073553441,0.9992233918738137,0.9798840447490501,0.9867072453385319,0.9951830776704002,0.9925009346357766,0.9972550995473726,0.9990053248841552,0.9996359167385416,0.9998680664138763,0.9999531423600655,0.9952771536866086,0.9982712855064354,0.9886916460283101,0.9957767337208402,0.9925399420124431,0.9972693775087801,0.9990203938296629,0.9750380726276326,0.9813485359083878,0.9859857853113156,0.9949216407445313,0.9981781472620839,0.9993331429768851,0.9997655269312155,0.9999133083572396,0.9909250787706482,0.9966782853355496,0.9936964460917903,0.991926946624903,0.9876597498641608,0.99561722670809,0.9983957632306927,0.9913874099702901,0.9967835052617072,0.9897109155132989,0.9963088084556905,0.9986352570878274,0.9995152967954501,0.9998207908598141,0.9939467918711073,0.9978284218882458,0.9929934865997215,0.997383317514472,0.9893861608439757,0.9826375058610678,0.9937712347841463,0.997765441082152,0.9991820791675866,0.9997065722937332,0.9745405569071953,0.9703435586009809,0.9736985239116832,0.9902755647762136,0.9965462616633736,0.9987484618697944,0.9995599476710615,0.9998405373495979,0.9716790820735808,0.981003150387865,0.9929763016764196,0.9347822427406679,0.9283856701528608,0.9444145743122359,0.9399032288787406,0.9646742903315438,0.9807714114708845,0.988677278709067,0.976883923802092,0.9913669712358225,0.9843888661734336,0.9943995323148342,0.9979293411120951,0.9992266828683621,0.9997197715105621,0.9998963912221412,0.9999620758364639,0.9999862573443332,0.9999949189329002,0.981906917788588,0.9933773467573078,0.9975266755720195,0.9990946832688979,0.9922814632383329,0.9972030182587442,0.998955427226558,0.9927269851975954,0.9973109508724863,0.999015721052057,0.9939473128267248,0.9977845207429726,0.9901452421904203,0.9752159497117332,0.9811272360467523,0.9932294280592201,0.930566437990624,0.9386676924606001,0.9646142737647273,0.9811773119590746,0.993040694371035,0.997503360952561,0.9786195089268541,0.9921740487949251,0.9899867889441561,0.9962978191046126,0.9986718489060692,0.9956697544191332,0.99843084017103,0.9923853581550242,0.9972406668693514,0.9826511290160742,0.9841072945139535,0.9940046890719004,0.9978055223043265,0.9834022307017992,0.98558184071098,0.9948275098993578,0.9942377951441244,0.9978695420865743,0.9992201835019645,0.994190581550422,0.997915881020692,0.9390292294791946,0.921118102558564,0.9584047876485344,0.9521249040823103,0.9625271150011233,0.9181913810318992,0.9479946372361273,0.9312680481587812,0.9743310710614016,0.9151073687844544,0.933478237095046,0.9353775957909424,0.8909603119400451,0.9135563139542665,0.8919419467071148,0.906687345138291,0.899334237107918,0.897702274996367,0.9041859939926851,0.8751880167595938,0.8879772542752551,0.9009089402579908,0.8466019005475292,0.8606044229949087,0.86381921930426,0.803342625321617,0.8311104648355713,0.9005136635448013,0.8964003629656829,0.8824806343688509,0.893044007236567,0.8912151652805442,0.8799484891234312,0.8835447979439851,0.8812713968885995,0.9201963832678028,0.8793304845967644,0.8754817198766288,0.8247221296792143,0.808071473173296,0.8637397334782119,0.8723845844850885,0.8951037123925635,0.8881119212444085,0.911667650917853,0.9287476830325039,0.9200462070937331,0.8712556475590546,0.9007849211132689,0.9136727899665046,0.9160675416113979,0.8937345310859519,0.9092525687291498,0.9255072211390772,0.8926653795824956,0.8983908793748487,0.9412386948469575,0.914224547629815,0.9273402958112914,0.9085005238184248,0.9171681822941494,0.9047017967156588,0.9104738266446232,

成功率计算模型:使用webmagic作为爬虫,开启50个线程,使用dungproxy作为代理层,使用自动预热模式(进行预热,在运行时动态加载IP),每次访问统计进行一次成功率计算,没隔100次访问执行一次成功率采样

附本次测试IP池稳定后,每个IP的使用以及评分统计(failedCount:当前IP总失败次数,referCount当前IP总使用次数)

{"www.so.com":[{"authenticationHeaders":[],"avgScore":0.9989294294407398,"cloud":false,"domain":"www.so.com","failedCount":2,"ip":"163.121.188.2","port":8080,"referCount":106},{"authenticationHeaders":[],"avgScore":0.9191596292565115,"cloud":false,"domain":"www.so.com","failedCount":6,"ip":"5.2.69.103","port":1080,"referCount":88},{"authenticationHeaders":[],"avgScore":0.9998457198643247,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"94.182.181.38","port":80,"referCount":145},{"authenticationHeaders":[],"avgScore":0.9991826835683485,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"94.177.242.47","port":8080,"referCount":94},{"authenticationHeaders":[],"avgScore":0.9368936149497572,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"94.177.190.206","port":8080,"referCount":31},{"authenticationHeaders":[],"avgScore":0.9638746101455065,"cloud":false,"domain":"www.so.com","failedCount":1,"ip":"167.114.195.82","port":8080,"referCount":68},{"authenticationHeaders":[],"avgScore":0.891605287587417,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"185.46.151.26","port":8080,"referCount":56},{"authenticationHeaders":[],"avgScore":0.8652065589803551,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"78.157.94.35","port":3128,"referCount":20},{"authenticationHeaders":[],"avgScore":0.9676382076994349,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"149.56.198.196","port":8080,"referCount":63},{"authenticationHeaders":[],"avgScore":0.8641442034857438,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"185.128.36.10","port":8080,"referCount":28},{"authenticationHeaders":[],"avgScore":0.9919381387784261,"cloud":false,"domain":"www.so.com","failedCount":3,"ip":"183.131.151.208","port":80,"referCount":169},{"authenticationHeaders":[],"avgScore":0.8249148619961763,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"60.206.113.252","port":3128,"referCount":26},{"authenticationHeaders":[],"avgScore":0.6179186338401451,"cloud":false,"domain":"www.so.com","failedCount":52,"ip":"111.177.124.140","port":9797,"referCount":133},{"authenticationHeaders":[],"avgScore":0.9036956245698365,"cloud":false,"domain":"www.so.com","failedCount":12,"ip":"120.24.73.165","port":3128,"referCount":169},{"authenticationHeaders":[],"avgScore":0.9413395927658398,"cloud":false,"domain":"www.so.com","failedCount":2,"ip":"94.177.234.171","port":8080,"referCount":66},{"authenticationHeaders":[],"avgScore":0.9999586306410454,"cloud":false,"domain":"www.so.com","failedCount":1,"ip":"162.105.80.111","port":3128,"referCount":153},{"authenticationHeaders":[],"avgScore":0.7516396918580303,"cloud":false,"domain":"www.so.com","failedCount":49,"ip":"210.212.230.22","port":80,"referCount":173},{"authenticationHeaders":[],"avgScore":0.9323860160175971,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"139.199.4.166","port":3128,"referCount":30},{"authenticationHeaders":[],"avgScore":0.7527414302886897,"cloud":false,"domain":"www.so.com","failedCount":7,"ip":"91.247.250.209","port":8080,"referCount":35},{"authenticationHeaders":[],"avgScore":0.9528038199814195,"cloud":false,"domain":"www.so.com","failedCount":6,"ip":"61.191.41.130","port":80,"referCount":98},{"authenticationHeaders":[],"avgScore":0.6157304504125917,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"42.201.208.110","port":8080,"referCount":18},{"authenticationHeaders":[],"avgScore":0.5272000143039738,"cloud":false,"domain":"www.so.com","failedCount":44,"ip":"190.104.245.39","port":8080,"referCount":157},{"authenticationHeaders":[],"avgScore":0.7741665295715451,"cloud":false,"domain":"www.so.com","failedCount":11,"ip":"66.68.92.218","port":80,"referCount":50},{"authenticationHeaders":[],"avgScore":0.773202957664545,"cloud":false,"domain":"www.so.com","failedCount":25,"ip":"58.216.14.22","port":808,"referCount":89},{"authenticationHeaders":[],"avgScore":0.5593222654725396,"cloud":false,"domain":"www.so.com","failedCount":3,"ip":"119.159.235.163","port":8080,"referCount":10},{"authenticationHeaders":[],"avgScore":0.5686957650272573,"cloud":false,"domain":"www.so.com","failedCount":17,"ip":"12.126.194.42","port":28080,"referCount":49},{"authenticationHeaders":[],"avgScore":0.7803247171811785,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"45.55.148.194","port":8080,"referCount":26},{"authenticationHeaders":[],"avgScore":0.5522297503429354,"cloud":false,"domain":"www.so.com","failedCount":2,"ip":"123.138.216.91","port":9999,"referCount":6},{"authenticationHeaders":[],"avgScore":0.4739111561916609,"cloud":false,"domain":"www.so.com","failedCount":6,"ip":"182.253.107.26","port":8080,"referCount":11},{"authenticationHeaders":[],"avgScore":0.5099609103728013,"cloud":false,"domain":"www.so.com","failedCount":9,"ip":"27.202.74.202","port":9000,"referCount":24},{"authenticationHeaders":[],"avgScore":0.5558061208656564,"cloud":false,"domain":"www.so.com","failedCount":27,"ip":"103.76.50.181","port":80,"referCount":73},{"authenticationHeaders":[],"avgScore":0.5481880213024437,"cloud":false,"domain":"www.so.com","failedCount":3,"ip":"167.114.167.10","port":80,"referCount":9},{"authenticationHeaders":[],"avgScore":0.4891960676052673,"cloud":false,"domain":"www.so.com","failedCount":8,"ip":"41.79.137.239","port":8080,"referCount":16},{"authenticationHeaders":[],"avgScore":0.3354872560897466,"cloud":false,"domain":"www.so.com","failedCount":11,"ip":"35.164.169.247","port":3128,"referCount":15},{"authenticationHeaders":[],"avgScore":0.5,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"89.38.150.204","port":3128,"referCount":0},{"authenticationHeaders":[],"avgScore":0.5126056328312815,"cloud":false,"domain":"www.so.com","failedCount":8,"ip":"177.130.0.32","port":8080,"referCount":19},{"authenticationHeaders":[],"avgScore":0.34250882086070605,"cloud":false,"domain":"www.so.com","failedCount":75,"ip":"218.56.132.156","port":8080,"referCount":154},{"authenticationHeaders":[],"avgScore":0.6417861433649079,"cloud":false,"domain":"www.so.com","failedCount":57,"ip":"117.102.100.5","port":80,"referCount":149},{"authenticationHeaders":[],"avgScore":0.519825921305331,"cloud":false,"domain":"www.so.com","failedCount":7,"ip":"5.2.64.150","port":1080,"referCount":18},{"authenticationHeaders":[],"avgScore":0.5,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"118.123.245.153","port":3128,"referCount":0},{"authenticationHeaders":[],"avgScore":0.5,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"189.45.201.66","port":8081,"referCount":0},{"authenticationHeaders":[],"avgScore":0.5644444444444444,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"139.59.17.121","port":8080,"referCount":3},{"authenticationHeaders":[],"avgScore":0.302285018675011,"cloud":false,"domain":"www.so.com","failedCount":70,"ip":"218.56.132.155","port":8080,"referCount":152},{"authenticationHeaders":[],"avgScore":0.5,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"186.103.239.190","port":80,"referCount":0},{"authenticationHeaders":[],"avgScore":0.3581138087092813,"cloud":false,"domain":"www.so.com","failedCount":80,"ip":"218.56.132.154","port":8080,"referCount":165},{"authenticationHeaders":[],"avgScore":0.9977229706755345,"cloud":false,"domain":"www.so.com","failedCount":2,"ip":"223.68.1.38","port":8000,"referCount":122},{"authenticationHeaders":[],"avgScore":0.4685078708733425,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"92.63.194.17","port":3128,"referCount":7},{"authenticationHeaders":[],"avgScore":0.46422079205233957,"cloud":false,"domain":"www.so.com","failedCount":65,"ip":"218.56.132.157","port":8080,"referCount":143},{"authenticationHeaders":[],"avgScore":0.9836486514436177,"cloud":false,"domain":"www.so.com","failedCount":24,"ip":"94.20.21.38","port":3128,"referCount":117},{"authenticationHeaders":[],"avgScore":0.8794902579299114,"cloud":false,"domain":"www.so.com","failedCount":6,"ip":"112.74.72.1","port":8080,"referCount":42},{"authenticationHeaders":[],"avgScore":0.6915196956561499,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"5.2.69.104","port":1080,"referCount":8},{"authenticationHeaders":[],"avgScore":0.8591965376409871,"cloud":false,"domain":"www.so.com","failedCount":1,"ip":"200.75.9.35","port":3128,"referCount":23},{"authenticationHeaders":[],"avgScore":0.7931180177896435,"cloud":false,"domain":"www.so.com","failedCount":12,"ip":"118.123.245.151","port":3128,"referCount":80},{"authenticationHeaders":[],"avgScore":0.8660750228733539,"cloud":false,"domain":"www.so.com","failedCount":20,"ip":"91.194.42.51","port":80,"referCount":154},{"authenticationHeaders":[],"avgScore":0.9200026569761887,"cloud":false,"domain":"www.so.com","failedCount":21,"ip":"139.224.24.26","port":8888,"referCount":129},{"authenticationHeaders":[],"avgScore":0.9979959334736238,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"69.181.163.138","port":80,"referCount":81},{"authenticationHeaders":[],"avgScore":0.8331544461327645,"cloud":false,"domain":"www.so.com","failedCount":3,"ip":"185.46.151.29","port":8080,"referCount":30},{"authenticationHeaders":[],"avgScore":0.9982107007266005,"cloud":false,"domain":"www.so.com","failedCount":17,"ip":"79.188.42.46","port":8080,"referCount":180},{"authenticationHeaders":[],"avgScore":0.9945952487769604,"cloud":false,"domain":"www.so.com","failedCount":3,"ip":"47.88.136.236","port":3128,"referCount":105},{"authenticationHeaders":[],"avgScore":0.9846998024735502,"cloud":false,"domain":"www.so.com","failedCount":1,"ip":"94.177.233.44","port":8080,"referCount":66},{"authenticationHeaders":[],"avgScore":0.9999930027431303,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"1.82.216.134","port":80,"referCount":163},{"authenticationHeaders":[],"avgScore":0.7429535375543829,"cloud":false,"domain":"www.so.com","failedCount":8,"ip":"124.88.67.52","port":843,"referCount":87},{"authenticationHeaders":[],"avgScore":0.9450273268006774,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"138.68.62.213","port":3128,"referCount":33},{"authenticationHeaders":[],"avgScore":0.9063791359910273,"cloud":false,"domain":"www.so.com","failedCount":12,"ip":"101.200.46.171","port":80,"referCount":97},{"authenticationHeaders":[],"avgScore":0.6680514228010003,"cloud":false,"domain":"www.so.com","failedCount":21,"ip":"41.207.49.136","port":8080,"referCount":71},{"authenticationHeaders":[],"avgScore":0.8549239170955845,"cloud":false,"domain":"www.so.com","failedCount":7,"ip":"62.210.51.150","port":80,"referCount":70},{"authenticationHeaders":[],"avgScore":0.9982651314391758,"cloud":false,"domain":"www.so.com","failedCount":6,"ip":"109.197.188.7","port":8080,"referCount":149},{"authenticationHeaders":[],"avgScore":0.970455418275663,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"94.177.241.47","port":8080,"referCount":42},{"authenticationHeaders":[],"avgScore":0.9610658462673219,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"94.177.252.138","port":8080,"referCount":38},{"authenticationHeaders":[],"avgScore":0.9610658462673219,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"109.197.188.6","port":8080,"referCount":38},{"authenticationHeaders":[],"avgScore":0.873488587869113,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"192.241.142.248","port":8080,"referCount":51},{"authenticationHeaders":[],"avgScore":0.7453348801995705,"cloud":false,"domain":"www.so.com","failedCount":79,"ip":"218.56.132.158","port":8080,"referCount":212},{"authenticationHeaders":[],"avgScore":0.9939560104026408,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"91.225.8.201","port":8080,"referCount":65},{"authenticationHeaders":[],"avgScore":0.9829873464175343,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"124.88.67.32","port":843,"referCount":50},{"authenticationHeaders":[],"avgScore":0.99899473653177,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"94.177.240.61","port":8080,"referCount":91},{"authenticationHeaders":[],"avgScore":0.9184338586561616,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"94.177.198.64","port":8080,"referCount":57},{"authenticationHeaders":[],"avgScore":0.982770333336033,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"183.240.87.107","port":80,"referCount":119},{"authenticationHeaders":[],"avgScore":0.999379792038749,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"195.154.235.156","port":8080,"referCount":98},{"authenticationHeaders":[],"avgScore":0.8251324711039021,"cloud":false,"domain":"www.so.com","failedCount":2,"ip":"177.54.147.140","port":3128,"referCount":21},{"authenticationHeaders":[],"avgScore":0.9539620491713829,"cloud":false,"domain":"www.so.com","failedCount":2,"ip":"119.28.19.61","port":3128,"referCount":62},{"authenticationHeaders":[],"avgScore":0.9144186277602359,"cloud":false,"domain":"www.so.com","failedCount":6,"ip":"5.2.69.101","port":1080,"referCount":67},{"authenticationHeaders":[],"avgScore":0.997068325860943,"cloud":false,"domain":"www.so.com","failedCount":2,"ip":"118.193.23.162","port":3128,"referCount":90},{"authenticationHeaders":[],"avgScore":0.9807332071442386,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"208.72.148.86","port":8080,"referCount":71},{"authenticationHeaders":[],"avgScore":0.7678507275612682,"cloud":false,"domain":"www.so.com","failedCount":17,"ip":"201.162.127.178","port":8080,"referCount":67},{"authenticationHeaders":[],"avgScore":0.9999839864913866,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"222.186.161.215","port":3128,"referCount":151},{"authenticationHeaders":[],"avgScore":0.5491890919067214,"cloud":false,"domain":"www.so.com","failedCount":2,"ip":"111.40.84.73","port":9999,"referCount":7},{"authenticationHeaders":[],"avgScore":0.6694853882030178,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"144.217.53.86","port":3128,"referCount":7},{"authenticationHeaders":[],"avgScore":0.9897176080290997,"cloud":false,"domain":"www.so.com","failedCount":6,"ip":"138.197.132.90","port":3128,"referCount":92},{"authenticationHeaders":[],"avgScore":0.8459124105841036,"cloud":false,"domain":"www.so.com","failedCount":8,"ip":"124.239.177.85","port":8080,"referCount":56},{"authenticationHeaders":[],"avgScore":0.9991826835683485,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"158.69.157.32","port":3128,"referCount":94},{"authenticationHeaders":[],"avgScore":0.8728866965217581,"cloud":false,"domain":"www.so.com","failedCount":10,"ip":"111.40.84.73","port":9797,"referCount":167},{"authenticationHeaders":[],"avgScore":0.9377580818624824,"cloud":false,"domain":"www.so.com","failedCount":1,"ip":"183.63.110.202","port":3128,"referCount":148},{"authenticationHeaders":[],"avgScore":0.878752971516588,"cloud":false,"domain":"www.so.com","failedCount":5,"ip":"5.2.69.102","port":1080,"referCount":60},{"authenticationHeaders":[],"avgScore":0.8306324594255207,"cloud":false,"domain":"www.so.com","failedCount":9,"ip":"118.97.239.146","port":8080,"referCount":59},{"authenticationHeaders":[],"avgScore":0.9075869882244004,"cloud":false,"domain":"www.so.com","failedCount":3,"ip":"118.123.245.179","port":3128,"referCount":37},{"authenticationHeaders":[],"avgScore":0.840949822585196,"cloud":false,"domain":"www.so.com","failedCount":9,"ip":"190.90.218.253","port":8080,"referCount":58},{"authenticationHeaders":[],"avgScore":0.9998439467315429,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"58.217.195.141","port":80,"referCount":118},{"authenticationHeaders":[],"avgScore":0.9999657956017117,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"138.197.21.105","port":8080,"referCount":140},{"authenticationHeaders":[],"avgScore":0.9943589430424647,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"146.185.139.216","port":8080,"referCount":66},{"authenticationHeaders":[],"avgScore":0.9999950442236331,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"120.132.71.212","port":80,"referCount":168},{"authenticationHeaders":[],"avgScore":0.635991356486265,"cloud":false,"domain":"www.so.com","failedCount":5,"ip":"101.53.101.172","port":9999,"referCount":17},{"authenticationHeaders":[],"avgScore":0.9968058017747291,"cloud":false,"domain":"www.so.com","failedCount":1,"ip":"149.56.206.53","port":8080,"referCount":90},{"authenticationHeaders":[],"avgScore":0.7526930266155621,"cloud":false,"domain":"www.so.com","failedCount":3,"ip":"87.249.140.34","port":3128,"referCount":19},{"authenticationHeaders":[],"avgScore":0.9042965990329533,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"109.197.188.8","port":8080,"referCount":128},{"authenticationHeaders":[],"avgScore":0.773805214619254,"cloud":false,"domain":"www.so.com","failedCount":11,"ip":"118.123.245.150","port":3128,"referCount":60},{"authenticationHeaders":[],"avgScore":0.8253714711499458,"cloud":false,"domain":"www.so.com","failedCount":29,"ip":"60.21.221.228","port":80,"referCount":144},{"authenticationHeaders":[],"avgScore":0.8278652596331652,"cloud":false,"domain":"www.so.com","failedCount":39,"ip":"202.73.51.102","port":80,"referCount":138},{"authenticationHeaders":[],"avgScore":0.7664423622821643,"cloud":false,"domain":"www.so.com","failedCount":6,"ip":"167.114.171.178","port":80,"referCount":62},{"authenticationHeaders":[],"avgScore":0.7890524665792302,"cloud":false,"domain":"www.so.com","failedCount":7,"ip":"203.153.20.5","port":3128,"referCount":40},{"authenticationHeaders":[],"avgScore":0.7953028957501438,"cloud":false,"domain":"www.so.com","failedCount":9,"ip":"42.157.7.43","port":9999,"referCount":87},{"authenticationHeaders":[],"avgScore":0.9814605428855783,"cloud":false,"domain":"www.so.com","failedCount":9,"ip":"58.67.159.50","port":80,"referCount":171},{"authenticationHeaders":[],"avgScore":0.9999939046117935,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"1.82.216.135","port":80,"referCount":165},{"authenticationHeaders":[],"avgScore":0.9909837835363732,"cloud":false,"domain":"www.so.com","failedCount":2,"ip":"94.181.46.187","port":8080,"referCount":203},{"authenticationHeaders":[],"avgScore":0.9164914719891025,"cloud":false,"domain":"www.so.com","failedCount":2,"ip":"5.2.69.100","port":1080,"referCount":32},{"authenticationHeaders":[],"avgScore":0.9868573114334943,"cloud":false,"domain":"www.so.com","failedCount":5,"ip":"101.254.188.198","port":8080,"referCount":146},{"authenticationHeaders":[],"avgScore":0.9742074423246004,"cloud":false,"domain":"www.so.com","failedCount":7,"ip":"203.91.121.74","port":3128,"referCount":175},{"authenticationHeaders":[],"avgScore":0.8559114563267592,"cloud":false,"domain":"www.so.com","failedCount":8,"ip":"195.110.59.16","port":3128,"referCount":44},{"authenticationHeaders":[],"avgScore":0.9572828994453212,"cloud":false,"domain":"www.so.com","failedCount":3,"ip":"199.48.160.69","port":3128,"referCount":78},{"authenticationHeaders":[],"avgScore":0.9999999814608163,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"108.165.33.6","port":3128,"referCount":249},{"authenticationHeaders":[],"avgScore":0.5,"cloud":false,"domain":"www.so.com","failedCount":0,"ip":"138.197.126.97","port":3128,"referCount":1},{"authenticationHeaders":[],"avgScore":0.5026694512087367,"cloud":false,"domain":"www.so.com","failedCount":22,"ip":"31.145.83.210","port":9090,"referCount":54},{"authenticationHeaders":[],"avgScore":0.6012177120330112,"cloud":false,"domain":"www.so.com","failedCount":4,"ip":"123.139.56.234","port":9999,"referCount":14},{"authenticationHeaders":[],"avgScore":0.5965207652087141,"cloud":false,"domain":"www.so.com","failedCount":7,"ip":"118.123.245.154","port":3128,"referCount":26}],"www.tianya.cn":[{"authenticationHeaders":[],"avgScore":0.5,"cloud":false,"domain":"www.tianya.cn","failedCount":0,"ip":"111.40.84.73","port":9797,"referCount":1},{"authenticationHeaders":[],"avgScore":0.5,"cloud":false,"domain":"www.tianya.cn","failedCount":0,"ip":"111.177.124.140","port":9797,"referCount":0}]}

本次测试相关源码:SuccessRateTest.java,SuccessRateTestDownloader.java

平常使用的时候,一般会增加重试,这样成功率会更高。需要注意的是,因为使用的是自动预热模式,刚开始二十分钟左右IP不足,大量使用的是本地IP,半个小时之后IP才基本稳定

已知用户,如果您也在使用dungprxoy 欢迎提交您的公司信息或者软件信息到 [email protected]


中关村科技软件股份有限公司	量富征信
摸象大数据

TODO

扩充服务器IP源
排查IP刷新线程在IP严重不足的时候,线程数量不能达到峰值的原因
云代理partner关系维护,支持partner统一限流和封禁部分完成
dump时机使用模型来均衡
禁止使用 org.apache.commons.lang3.StringUtils.isNoneEmpty 完成
事件回调机制设计

Versions

Version
0.0.1 Jun 2, 2017

com.virjar:dungproxy-webmagic5

License

Categories

GroupId

ArtifactId

Last Version

Release Date

Type

Description