之前用bash写了一个Dnspod动态域名解析工具,可不是并发请求的，子域名多了，添加新域名的时候，真的要很长时间。发现有个golang轮毂dnspod-go,拿过来配上轮胎，又是一个好轮子！还是一样的套路，只不过请求都变成用协程去执行了，这下子更新起来就块多了，近40个子域名，双线更新的情况下，所需时间也只有不到8秒。real 0m 7.04s user 0m 0.54s sys 0m 0.18s配置文件还是跟bash版的一样{ "token":"yourApiToken", "domains":{ "xxxx.xx":[ "@", "www", "aaa" ] }, "ips":{ "ip-chinanet":"ifconfig |grep inet|grep netmask|grep 192|awk '{print $2}'", "ip-chinamobile":"ifconfig |grep inet|grep netmask|grep 10|awk '{print $2}'", "ip-chinaunicom":"ifconfig |grep inet|grep netmask|grep 12|awk '{print $2}'" } }运行命令则为dns -c dns.json项目地址
之前抓取IPTV播放列表都是用Openwrt的端口镜像，配合台式机去抓包的，但是每次这样弄都好麻烦了，路由器，机顶盒，电脑都要用网线连接在一起，主力机变成XPS 15后，没有网口就显得更麻烦了。此时就想起tcpdump了，这货不是能把所有数据包全抓下来吗，在路由器搞一搞就好了！家里用的是软路由，弄个tcpdump还是很简单的，但是老人家里用的是K2P，还没硬改过的，装这个包和对应依赖空间有压力啊。。。Google一番，没找到能用的tcpdump静态链接版本，又是DIY时刻了。K2P刷的是高恪的固件，基于openwrt 14.07的，找到对应SDK和Toolchain，参照之前的高恪插件编译教程弄起来了，却发现没有tcpdump这个包，搜索一番得知在base这个repo，SDK默认是没有添加这个repo的.Openwrt 14.07也算是年代久远了，费劲一番才找到这个repo的源git://git.archive.openwrt.org/14.07/openwrt.git添加后就是常规操作./script/feeds update ./script/feeds install先按照官方动态链接的方式编译一遍看会不会出错。make -j16 package/network/utils/tcpdump compile成功编译后，看到依赖库libpcap把静态链接库也编译出来了。接下来就是尝试编译静态链接的tcpdump了只需要对Makefile稍作改动 49 CONFIGURE_ARGS += \ 50 --without-crypto 51 52 ifeq ($(CONFIG_IPV6),y) 53 CONFIGURE_ARGS += \ 54 --enable-ipv6 55 endif 56 57 TARGET_CFLAGS += -ffunction-sections -fdata-sections 58 TARGET_LDFLAGS += -Wl,--gc-sections 59 60 CONFIGURE_VARS += \ 61 BUILD_CC="$(TARGET_CC)" \ 62 HOSTCC="$(HOSTCC)" \ 63 td_cv_buggygetaddrinfo="no" \ 64 ac_cv_linux_vers=$(LINUX_VERSION) \ 65 ac_cv_header_rpc_rpcent_h=no \ 66 ac_cv_lib_rpc_main=no \ 67 ac_cv_path_PCAP_CONFIG="" 68 69 MAKE_FLAGS :=改成 49 CONFIGURE_ARGS += \ 50 --without-crypto 51 52 ifeq ($(CONFIG_IPV6),y) 53 CONFIGURE_ARGS += \ 54 --enable-ipv6 55 endif 56 57 TARGET_CFLAGS += -ffunction-sections -fdata-sections 58 TARGET_LDFLAGS += -static -Wl,--gc-sections 59 60 CONFIGURE_VARS += \ 61 BUILD_CC="$(TARGET_CC)" \ 62 HOSTCC="$(HOSTCC)" \ 63 td_cv_buggygetaddrinfo="no" \ 64 ac_cv_linux_vers=$(LINUX_VERSION) \ 65 ac_cv_header_rpc_rpcent_h=no \ 66 ac_cv_lib_rpc_main=no \ 67 ac_cv_path_PCAP_CONFIG="" 68 69 MAKE_FLAGS :=再次执行编译命令就可以了。make -j16 package/network/utils/tcpdump compile编译好的静态链接tcpdump和tcpdump-mini二进制附上。在路由器上开始抓到包tcpdump -i br-lan -s 0 -w iptv.pcap host 192.168.2.113完了用scp把抓包文件传回到Mac上，直接用Charles就可以打开。我这边都是rtsp协议的直播，所以直接查找关键词rtsp就可以找到列表页了，找到以后就是用正则提取一下电视台名称和播放地址。用到的正则也很简单ChannelName="(.*?)".*ChannelURL="(?:.*?((?:rtsp|http).*?(?:smil|m3u8).*?END.*?))"，最后就是根据个人习惯调整一下列表的顺序了。广东电信这里抓到的播放列表在此。
自从入了魅族魅蓝E3的坑，等了一年也没等到BootLoader解锁的方法，也受够了那个没几秒就一次的WiFi断流。苦苦寻觅，终于找到价格在2k上下，性能过得去，非异形屏，非背面指纹，拍照还可以，最重要一点，可以解锁BootLoader，可以装xposed的机子 -- 二手 华为 Mate 10。机子到手随即在深水宝让人给解锁BootLoader了，开启了Magisk->Root->Xposed的折腾之路。或许是距离我太久没碰过华为手机了（上一台华为手机还是C8850），发现刷入Magisk，安装Xposed的方法跟其他机型有很大区别，爬文了解了之后，算是把Magisk+Root+Xposed搞定了。装上最熟悉的Nova Launcher却发现不能设置为默认桌面，华为什么脑残设定嘛！心想祭出冰箱冻结就好了，没想到事情却没这么简单，使用冰箱添加支付宝扫码快捷方式后，发现没有任何反应，冰箱的快捷方式权限已经给了啊!再去论坛爬文，发现还是华为脑残设定。。。这下子没这么好办了，冰箱管不到这块了。爬文良久没找到解决办法，只能硬着头皮自己来了。既然默认应用设置把第三方桌面灰了不让我点，那就从这里入手，用开发者助手找出这里的资源ID，包名，将系统设置的apk传到Mac上JEB静态反编译走起。拿着0x7f0a07a8就到JEB去搜索，找到位于com.android.settings.applications.PreferredSettings这个类的onCreateView方法里，但是这个方法看不出什么有价值的线索，JEB字体太小，看的眼睛都疼了，还是把dex拉出来，用jadx直接把dex还原成Java源文件吧。庆幸的是，这些代码并没有做混淆，还原成Java源文件后，方法名和变量名都是原始命名，这样看起来就容易多了。用IntelliJ IDEA打开源码看，舒服多了，翻翻这个类，发现PreferredSettings.buildPreferredActivitiesList这个方法有点嫌疑，build什么List，可惜jadx反编译失败了，还是要回到jeb去看。Jeb反编译也不算很好，出来一堆goto,耐着性子找亮点。发现这有重大嫌疑，Anti?Protection?Launcher?进去看看呗！在这个类里，看到了华为桌面的包名com.huawei.android.launcher了，先写个hook，把AntiMalProtectionUtils.checkLauncherPermisson始终返回true试试！XposedBridge.hookAllMethods(loadPackageParam.classLoader.loadClass("com.android.settings.AntiMalProtectionUtils"), "checkLauncherPermisson", new XC_MethodHook() { @Override protected void beforeHookedMethod(MethodHookParam param) throws Throwable { param.setResult(true); } });重启一看，红色提示文字消失了，第三方桌面可以选择了，但是应用快捷方式还是添加不了，常规返回Home时，还是回到的华为桌面。难道跟AntiMalProtectionUtils.getHwDefLauncherPkg这个方法有关?public static String getHwDefLauncherPkg() { return "com.huawei.android.launcher"; }改成返回Nova桌面的包名试试？XposedBridge.hookAllMethods(loadPackageParam.classLoader.loadClass("com.android.settings.AntiMalProtectionUtils"), "getHwDefLauncherPkg", new XC_MethodHook() { @Override protected void beforeHookedMethod(MethodHookParam param) throws Throwable { param.setResult("com.teslacoilsw.launcher"); } });结果还是不行，再仔细看AntiMalProtectionUtils.checkLauncherPermisson这个方法，按照它的正常流程走的话，ActivityManager.checkUidPermission应该要返回0,那就hook这个方法试试吧！XposedBridge.hookAllMethods(loadPackageParam.classLoader.loadClass("android.app.ActivityManager"), "checkUidPermission", new XC_MethodHook() { @Override protected void beforeHookedMethod(XC_MethodHook.MethodHookParam param) throws Throwable { if (param.args[0] == "com.huawei.permission.sec.SDK_LAUNCHER") param.setResult(0); } });重启一试，成了！！！反手就把华为桌面丢冰箱里了，没想到的是，这个老流氓，重启后竟然自动解冻了，还又成了默认桌面了！这下就尴尬了，难道每次重启都要手动重新选一次桌面？这用户体验极差啊！依稀记得看到过什么Launcher disabled之类的方法的，再翻翻去~~~看到这个方法，解法很明显了，int res = this.mPm.getApplicationEnabledSetting(hwDefLauncherPkg);中res的返回值不是2和3就好了。XposedBridge.hookAllMethods(loadPackageParam.classLoader.loadClass("android.app.ApplicationPackageManager"), "getApplicationEnabledSetting", new XC_MethodHook() { @Override protected void beforeHookedMethod(XC_MethodHook.MethodHookParam param) throws Throwable { if (param.args[0] == "com.huawei.android.launcher") param.setResult(1); } });重启，设置Nova为默认桌面，冻结华为桌面，再重启，华为桌面没解冻了，默认桌面还是Nova，终于搞定了！然而还是高兴的太早，屏幕手势的底部上滑失效了。。。算了，我投降了。我不冻结华为桌面了，你一边凉快去吧。。。最终达成效果：自由选择第三方桌面，自由使用Android 7开始的应用快捷方式。 完整Xposed源码在此
自从高恪魔改固件的消息在恩山公开了,然后再有人拿去贩卖后,作者就说了不再发布新版本了,随着koolproxy的更新,Telegram群里的呼声也越来越高,还是在GitHub上放出了源码了,但是不提供ipk,也不提供魔改固件.一大堆小白先是求别人编译好的ipk,没得到后就真的硬着头皮请教如何自行编译了.在群里教了若干个群友后,觉得这真的太累了,每个人的编译环境都不一样,遇到的问题也不一样,眼看群里很多人就差这么一点点了,愣是弄不出来,我都替他们着急了...所以,还是给他们弄个教程吧.遂录制了这个使用Docker镜像作为编译环境的视频,能确保每个人的编译环境都一致,后面的是就好办了嘛~PS1. 视频录制用的是asciinema,Linux/Mac 用户可以安装并下载cast文件在本地回放.PS2. 视频编译的是K2P的插件,用到的SDK和Toolchain自行到这里下载.
使用反向代理从外网管理家里的NAS服务器
把EA6500V2升级成了一个X86的NAS,各种服务就多起来了,但是在内网,各个服务都监听的80/443端口,我外网IP撑死也就两个,电信的家用宽带80端口又不能用,443端口的话,即使咗做端口映射,IP也不够分啊!还好之前写了个ddns.sh进行动态域名解析,那从外网管理各种服务就当然要通过域名来搞了~怎么搞?Nginx反向代理了解一下~撸起袖子加油干~一个服务一个子域名!路由器一个ESXI一个Ubuntu Server管理的Webadmin一个个人网盘一个你现在看到的站点一个放静态资源的站点一个......先来一个ESXI的,server { server_name xxx.1km.fun; listen 443 ssl http2; ssl on; ssl_certificate /usr/local/share/ssl/1km.fun.pem; ssl_certificate_key /usr/local/share/ssl/1km.fun.key; access_log off; error_log off; location / { #Proxy Settings proxy_redirect off; proxy_set_header Host $host:$server_port; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Host $host:$server_port; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass //192.168.2.254; proxy_connect_timeout 90; proxy_send_timeout 90; proxy_read_timeout 90; proxy_buffer_size 128k; proxy_buffers 32 32k; proxy_busy_buffers_size 256k; proxy_temp_file_write_size 256k; } }加好配置文件nginx -s reload,用手机断掉WIFI一试,成了!在用电脑打开,试试在网页打开虚拟机的显示器,囧了. location /ticket { proxy_redirect off; proxy_pass //192.168.2.254; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_set_header Host $http_host; }打开调试器看,原来用的WebSocket,想到之前用Websocket伪装的梯子搞过Websocket转发,搬过来...加上这个location,果然好了!刚好koolshare lede发新版了,打算升级一下,上传镜像的时候竟然失败了,经验告诉我,肯定是Nginx不受这么大的文件,赶紧补上proxy_max_temp_file_size 0;和client_max_body_size 4g;解决掉~最终nginx的配置文件就是这样子了client_max_body_size 4g; server { server_name esxi.1km.fun; listen 443 ssl http2; ssl on; ssl_certificate /usr/local/share/ssl/1km.fun.pem; ssl_certificate_key /usr/local/share/ssl/1km.fun.key; access_log off; error_log off; location /ticket { proxy_redirect off; proxy_pass //192.168.2.254; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_set_header Host $http_host; } location / { #Proxy Settings proxy_redirect off; proxy_set_header Host $host:$server_port; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Host $host:$server_port; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass //192.168.2.254; proxy_max_temp_file_size 0; proxy_connect_timeout 90; proxy_send_timeout 90; proxy_read_timeout 90; proxy_buffer_size 128k; proxy_buffers 32 32k; proxy_busy_buffers_size 256k; proxy_temp_file_write_size 256k; } }然后还值得一说的就是配置koolshare lede时,要隐藏掉X-XSS-Protection,proxy_hide_header X-XSS-Protection ;否则会在控制台报这样一个错误,而且软件中心用起来也会有问题.Error parsing header X-XSS-Protection: 1; mode=block, 1; mode=block: expected semicolon at character position 13. The default protections will be applied.根本原因是为了安全,在nginx.conf中已经配置过了,lede自己也有这个响应头,输出到浏览器的时候就重复了. add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;"; add_header X-Robots-Tag none; add_header X-Download-Options noopen; add_header X-Permitted-Cross-Domain-Policies none; add_header X-Content-Type-Options "nosniff" always; add_header X-XSS-Protection "1; mode=block" always; add_header Referrer-Policy "same-origin" always;
在路由器上搭建NextCloud作为个人网盘(续)-将数据加密备份至云端
上次在EA6500V2上把Nextcloud网盘搭起来后用了一段时间,那速度实在不能忍...打开web版界面,要生成缩略图的缘故,基本上每打开一个页面都要崩一次...还是下狠手从某宝搞了个小马路由V5NAS版准系统回来,淘了个MLC芯片的265G MSATA二手固态硬盘.从准备报废的ThinkPad E520 扯下来8G内存,装上一路累积起来的几个硬盘,一台X86NAS总算是装好了.装上ESXI虚拟化系统,Ubuntu Server先来一个,Koolshare Lede 软路由再来一个~用作Nextcloud备份的硬盘直通映射给了Ubuntu,但是我这没用做RAID,数据可靠性不够高啊!想起前段时间撸到的无限容量Google Drive,瞬间露出一个邪恶的笑容,把数据备份一份到Google Drive不就好了,放狗搜之~貌似没人这样干过,但是找到一个工具rclone,这货支持Google Drive,还支持加密,简直就是不要再合适了!参照文档先添加一个remote,取名Googledrive,类型选择drive,选择没有浏览器,然后复制授权链接,到一个登录了无限容量Google Drive的浏览器打开,确认授权.再建一个remote,类型选择crypt,remote填入刚生成的Googledrive及加密文件存放路径,选择加密模式为只在文件名后面加上.bin,设置密码,盐,并记录备份,选个强一点的加密强度,确认信息无误后,一个加密的备份网盘就弄好了!使用rclone copy命令先试着备份一个小文件,上传完毕,Google drive网页端看到文件存在,大小相差不大,直接下载下来,无法打开,嗯,这就对了,加密了嘛~再使用rclone copy命令拷贝到本地的另一目录,文件正常打开,这就对了,就是要这个效果!接下来就是配置crontab来实现自动备份了.*/15 * * * * for user in 用户1 用户2 用户3;do for drive in gdCrypted gdCrypted2;do screen -dmS ${user}_sync_nextCloud_${drive} bash -c "rclone -v copy /media/3T/nextcloud/${user}/files ${drive}:NextCloud/${user}/files"; done; done;安全起见,撸了两个无限容量的Google drive作为备份网盘,这下子算是见了个性价比最高的个人网盘了~
最近下载电影喜欢上用BT方式了,下载工具当然还是aria2,但是如果下载的种子是由BitComet这软件创建的话,多半会出现一堆名字为____padding_file_1_如果您看到此文件，请升级到BitComet(比特彗星)0.85或以上版本____这样的文件,简直就是流氓啊!治这流氓,本来是想通过再YAAW上动手脚,自动取消选择这些文件的.但是想想这效率也太低了,而且也不见得只会用这个WebUI啊,另外如果下载的是磁力链,连选择的机会都没有呢!那还是从aria2源码上动手脚吧,以流氓的方式对付流氓吧,嘿嘿.先看aria2的文档,是可以通过传入需要下载的文件的索引值进行控制的,找到src/bittorrent_helper.cc这个文件就是对BT种子文件进行解析的,就在此处动刀吧!思路就是,如果添加任务都时候传递了索引参数,那不进行任何流氓操作,如果只是添加任务,没说要下载指定的文件,那么流氓来了,所有文件名过滤一下,记录不是"流氓文件"的文件索引值,"生成"一个新的索引参数,当成是用户参数传给aria2,达到忽略"流氓文件"的目的.源码在此brew Formula参见上一篇aria2魔改文章~
很久之前就使用BaiduExporter下载百度网盘的东西了,放狗一搜,教程也是一抓一大把了.下载文件不限速那时候还好,后来限速了BaiduExporter也跟着斗智斗勇了好几回,始终服务器是人家了,几个回合下来,该限速的还是限速了.然后就有了aria2线程数破解版,我当然也马上跟进了,还研究了下brew的Formula文件要怎么改.效果是有了,可是这连接数一上去,同时下载多个文件的时候,刷刷的就一堆任务失败了.稍微改了下YAAW,给加上了个失败任务重试按钮,可这也不算个事啊,总不能守在显示器前不停点重试吧,更严重(JS不熟,瞎改)的是,这WebUI在Chrome下吃内存,放个几个小时就吃了几个G的内存,耗不起啊....不能忍了,还是从源头下手,既然都可以破解下载线程了,能不能再aria2层面弄个自动重试?根据报错信息的HTTP错误码,在源码目录使用ag一搜,发现在src/HttpResponseCommand.cc这文件有线索!!参照这原版的参数,魔改增加了一个max-http-forbidden选项,设置为-1的话,始终忽略状态码为400,403的错误状态,就是说,只有返回这两个状态码的话,会自动重试!代码嘛,就在这里.PS1. 俗话说,不写文档的程序员都不是好程序员,改好程序,当然还是要把文档改好,然后就在这里踩坑了.改源码的时候是git clone下来的代码,改好文档源码,编译安装后,man命令出来的文档死活没有我新加上去的选项.快被逼疯的时候,重新在release页面下载了一份源码下来,用Beyond Compare对比了一下,捶胸了!release源码包直接提供编译好的man文档!!!就奇怪,为什么以前直接用brew安装,明明没有装sphinx-doc这文档构建工具,也没见报错,就是这原因,赶紧把为man文档弄个patch补丁提交代码,推送完事!PS2. Linux用户看这里,有个用Docker编译静态链接的repo.PS3. 在Mac下能不用brew麽?不能!所以附上这个!class Aria2 < Formula desc "Download with resuming and segmented downloading" homepage "//aria2.github.io/" url "//github.com/aria2/aria2/releases/download/release-1.34.0/aria2-1.34.0.tar.xz" sha256 "3a44a802631606e138a9e172a3e9f5bcbaac43ce2895c1d8e2b46f30487e77a3" revision 1 bottle do cellar :any sha256 "a5244c4733c43fdd2441e97abe12211cc718a383d5e7c3be2117cec7d87f9424" => :mojave sha256 "8fe4633e41f67b4a80ad80f6c3423641d39e091779636c7b62e046c50331fe87" => :high_sierra sha256 "04b6207d99d9882c41f11178a70f61c5aebc43e9db0d8ea87c2d870de2f7c646" => :sierra sha256 "82f36d7a6cb88b292430a5ea05389e6e066f7e059df2468a2639cdd7844988c4" => :el_capitan end depends_on "pkg-config" => :build depends_on "openssl" => :build depends_on "c-ares" => :build depends_on "libssh2" patch :DATA def install # Fix "error: use of undeclared identifier 'make_unique'" # Reported upstream 15 May 2018 //github.com/aria2/aria2/issues/1198 inreplace "src/bignum.h", "make_unique", "std::make_unique" inreplace "configure", "-std=c++11", "-std=c++14" args = %W[ --disable-dependency-tracking --prefix=#{prefix} --without-appletls --with-libssh2 --with-openssl --without-gnutls --without-libgmp --without-libnettle --without-libgcrypt ] system "./configure", *args system "make", "install" bash_completion.install "doc/bash_completion/aria2c" end test do system "#{bin}/aria2c", "//brew.sh/" assert_predicate testpath/"index.html", :exist?, "Failed to create index.html!" end end __END__ diff --git a/doc/bash_completion/aria2c b/doc/bash_completion/aria2c index fba97156..a77cd1aa 100644 --- a/doc/bash_completion/aria2c +++ b/doc/bash_completion/aria2c @@ -78,7 +78,7 @@ _aria2c() esac case $cur in -*) - COMPREPLY=( $( compgen -W '--rpc-save-upload-metadata --rpc-save-upload-metadata=false --on-download-start --metalink-language --rpc-secret --torrent-file --enable-peer-exchange --enable-peer-exchange=false --http-proxy-passwd --bt-tracker-timeout --ftp-type --seed-time --keep-unfinished-download-result --keep-unfinished-download-result=false --bt-tracker-connect-timeout --bt-max-open-files --no-netrc --no-netrc=false --force-sequential --force-sequential=false --metalink-base-uri --private-key --ftp-passwd --allow-overwrite --allow-overwrite=false --rpc-allow-origin-all --rpc-allow-origin-all=false --bt-detach-seed-only --bt-detach-seed-only=false --dht-entry-point6 --summary-interval --lowest-speed-limit --bt-tracker-interval --proxy-method --metalink-preferred-protocol --enable-http-keep-alive --enable-http-keep-alive=false --metalink-version --stderr --stderr=false --bt-lpd-interface --force-save --force-save=false --rpc-secure --rpc-secure=false --listen-port --rpc-private-key --server-stat-of --server-stat-timeout --bt-load-saved-metadata --bt-load-saved-metadata=false --https-proxy-user --piece-length --dry-run --dry-run=false --truncate-console-readout --truncate-console-readout=false --save-not-found --save-not-found=false --async-dns-server --bt-max-peers --max-overall-upload-limit --rpc-user --optimize-concurrent-downloads --optimize-concurrent-downloads=true --optimize-concurrent-downloads=false --optimize-concurrent-downloads=A:B --dir --split --on-download-pause --auto-file-renaming --auto-file-renaming=false --http-proxy --save-session-interval --daemon --daemon=false --https-proxy --min-tls-version --save-cookies --out --rlimit-nofile --max-file-not-found --on-download-stop --certificate --bt-min-crypto-level --remove-control-file --remove-control-file=false --enable-dht --enable-dht=false --file-allocation --follow-metalink --on-bt-download-complete --ftp-proxy --show-files --show-files=false --timeout --bt-hash-check-seed --bt-hash-check-seed=false --ftp-pasv --ftp-pasv=false --check-certificate --check-certificate=false --always-resume --always-resume=false --load-cookies --bt-remove-unselected-file --bt-remove-unselected-file=false --bt-stop-timeout --version --max-concurrent-downloads --quiet --quiet=false --max-download-result --content-disposition-default-utf8 --content-disposition-default-utf8=false --max-resume-failure-tries --header --rpc-listen-all --rpc-listen-all=false --all-proxy-user --server-stat-if --dht-file-path6 --save-session --bt-external-ip --max-tries --conditional-get --conditional-get=false --ftp-reuse-connection --ftp-reuse-connection=false --gid --dscp --max-download-limit --bt-prioritize-piece --check-integrity --check-integrity=false --log-level --remote-time --remote-time=false --uri-selector --rpc-listen-port --index-out --bt-tracker --referer --ssh-host-key-md --console-log-level --connect-timeout --stream-piece-selector --dht-message-timeout --select-file --download-result --disable-ipv6 --disable-ipv6=false --rpc-max-request-size --rpc-passwd --stop-with-process --https-proxy-passwd --continue --continue=false --no-file-allocation-limit --netrc-path --ftp-proxy-user --enable-color --enable-color=false --metalink-location --allow-piece-length-change --allow-piece-length-change=false --max-connection-per-server --no-conf --no-conf=false --rpc-certificate --metalink-os --enable-http-pipelining --enable-http-pipelining=false --http-passwd --user-agent --enable-dht6 --enable-dht6=false --dht-file-path --http-auth-challenge --http-auth-challenge=false --bt-enable-hook-after-hash-check --bt-enable-hook-after-hash-check=false --peer-id-prefix --max-mmap-limit --enable-mmap --enable-mmap=false --use-head --use-head=false --bt-require-crypto --bt-require-crypto=false --show-console-readout --show-console-readout=false --conf-path --log --no-proxy --dht-entry-point --dht-listen-port --http-user --retry-wait --on-download-complete --help --help=#basic --help=#advanced --help=#http --help=#https --help=#ftp --help=#metalink --help=#bittorrent --help=#cookie --help=#hook --help=#file --help=#rpc --help=#checksum --help=#experimental --help=#deprecated --help=#help --help=#all --max-overall-download-limit --event-poll --http-accept-gzip --http-accept-gzip=false --metalink-file --all-proxy --disk-cache --hash-check-only --hash-check-only=false --dht-listen-addr6 --human-readable --human-readable=false --ftp-user --all-proxy-passwd --bt-exclude-tracker --pause-metadata --pause-metadata=false --http-proxy-user --deferred-input --deferred-input=false --metalink-enable-unique-protocol --metalink-enable-unique-protocol=false --stop --peer-agent --max-upload-limit --multiple-interface --realtime-chunk-checksum --realtime-chunk-checksum=false --http-no-cache --http-no-cache=false --ca-certificate --bt-force-encryption --bt-force-encryption=false --bt-save-metadata --bt-save-metadata=false --seed-ratio --follow-torrent --pause --pause=false --checksum --auto-save-interval --async-dns --async-dns=false --bt-enable-lpd --bt-enable-lpd=false --parameterized-uri --parameterized-uri=false --ftp-proxy-passwd --enable-rpc --enable-rpc=false --min-split-size --bt-seed-unverified --bt-seed-unverified=false --input-file --interface --enable-async-dns6 --enable-async-dns6=false --reuse-uri --reuse-uri=false --socket-recv-buffer-size --bt-request-peer-speed-limit --on-download-error --bt-metadata-only --bt-metadata-only=false ' -- "$cur" ) ) + COMPREPLY=( $( compgen -W '--rpc-save-upload-metadata --rpc-save-upload-metadata=false --on-download-start --metalink-language --rpc-secret --torrent-file --enable-peer-exchange --enable-peer-exchange=false --http-proxy-passwd --bt-tracker-timeout --ftp-type --seed-time --keep-unfinished-download-result --keep-unfinished-download-result=false --bt-tracker-connect-timeout --bt-max-open-files --no-netrc --no-netrc=false --force-sequential --force-sequential=false --metalink-base-uri --private-key --ftp-passwd --allow-overwrite --allow-overwrite=false --rpc-allow-origin-all --rpc-allow-origin-all=false --bt-detach-seed-only --bt-detach-seed-only=false --dht-entry-point6 --summary-interval --lowest-speed-limit --bt-tracker-interval --proxy-method --metalink-preferred-protocol --enable-http-keep-alive --enable-http-keep-alive=false --metalink-version --stderr --stderr=false --bt-lpd-interface --force-save --force-save=false --rpc-secure --rpc-secure=false --listen-port --rpc-private-key --server-stat-of --server-stat-timeout --bt-load-saved-metadata --bt-load-saved-metadata=false --https-proxy-user --piece-length --dry-run --dry-run=false --truncate-console-readout --truncate-console-readout=false --save-not-found --save-not-found=false --async-dns-server --bt-max-peers --max-overall-upload-limit --rpc-user --optimize-concurrent-downloads --optimize-concurrent-downloads=true --optimize-concurrent-downloads=false --optimize-concurrent-downloads=A:B --dir --split --on-download-pause --auto-file-renaming --auto-file-renaming=false --http-proxy --save-session-interval --daemon --daemon=false --https-proxy --min-tls-version --save-cookies --out --rlimit-nofile --max-http-forbidden --max-file-not-found --on-download-stop --certificate --bt-min-crypto-level --remove-control-file --remove-control-file=false --enable-dht --enable-dht=false --file-allocation --follow-metalink --on-bt-download-complete --ftp-proxy --show-files --show-files=false --timeout --bt-hash-check-seed --bt-hash-check-seed=false --ftp-pasv --ftp-pasv=false --check-certificate --check-certificate=false --always-resume --always-resume=false --load-cookies --bt-remove-unselected-file --bt-remove-unselected-file=false --bt-stop-timeout --version --max-concurrent-downloads --quiet --quiet=false --max-download-result --content-disposition-default-utf8 --content-disposition-default-utf8=false --max-resume-failure-tries --header --rpc-listen-all --rpc-listen-all=false --all-proxy-user --server-stat-if --dht-file-path6 --save-session --bt-external-ip --max-tries --conditional-get --conditional-get=false --ftp-reuse-connection --ftp-reuse-connection=false --gid --dscp --max-download-limit --bt-prioritize-piece --check-integrity --check-integrity=false --log-level --remote-time --remote-time=false --uri-selector --rpc-listen-port --index-out --bt-tracker --referer --ssh-host-key-md --console-log-level --connect-timeout --stream-piece-selector --dht-message-timeout --select-file --download-result --disable-ipv6 --disable-ipv6=false --rpc-max-request-size --rpc-passwd --stop-with-process --https-proxy-passwd --continue --continue=false --no-file-allocation-limit --netrc-path --ftp-proxy-user --enable-color --enable-color=false --metalink-location --allow-piece-length-change --allow-piece-length-change=false --max-connection-per-server --no-conf --no-conf=false --rpc-certificate --metalink-os --enable-http-pipelining --enable-http-pipelining=false --http-passwd --user-agent --enable-dht6 --enable-dht6=false --dht-file-path --http-auth-challenge --http-auth-challenge=false --bt-enable-hook-after-hash-check --bt-enable-hook-after-hash-check=false --peer-id-prefix --max-mmap-limit --enable-mmap --enable-mmap=false --use-head --use-head=false --bt-require-crypto --bt-require-crypto=false --show-console-readout --show-console-readout=false --conf-path --log --no-proxy --dht-entry-point --dht-listen-port --http-user --retry-wait --on-download-complete --help --help=#basic --help=#advanced --help=#http --help=#https --help=#ftp --help=#metalink --help=#bittorrent --help= #cookie --help=#hook --help=#file --help=#rpc --help=#checksum --help=#experimental --help=#deprecated --help=#help --help=#all --max-overall-download-limit --event-poll --http-accept-gzip --http-accept-gzip=false --metalink-file --all-proxy --disk-cache --hash-check-only --hash-check-only=false --dht-listen-addr6 --human-readable --human-readable=false --ftp-user --all-proxy-passwd --bt-exclude-tracker --pause-metadata --pause-metadata=false --http-proxy-user --deferred-input --deferred-input=false --metalink-enable-unique-protocol --metalink-enable-unique-protocol=false --stop --peer-agent --max-upload-limit --multiple-interface --realtime-chunk-checksum --realtime-chunk-checksum=false --http-no-cache --http-no-cache=false --ca-certificate --bt-force-encryption --bt-force-encryption=false --bt-save-metadata --bt-save-metadata=false --seed-ratio --follow-torrent --pause --pause=false --checksum --auto-save-interval --async-dns --async-dns=false --bt-enable-lpd --bt-enable-lpd=false --parameterized-uri --parameterized-uri=false --ftp-proxy-passwd --enable-rpc --enable-rpc=false --min-split-size --bt-seed-unverified --bt-seed-unverified=false --input-file --interface --enable-async-dns6 --enable-async-dns6=false --reuse-uri --reuse-uri=false --socket-recv-buffer-size --bt-request-peer-speed-limit --on-download-error --bt-metadata-only --bt-metadata-only=false ' -- "$cur" ) ) ;; *) _filedir '@(torrent|meta4|metalink|text|txt|list|lst)' diff --git a/doc/manual-src/en/aria2c.rst b/doc/manual-src/en/aria2c.rst index 6722e0d2..3badb2db 100644 --- a/doc/manual-src/en/aria2c.rst +++ b/doc/manual-src/en/aria2c.rst @@ -185,6 +185,17 @@ HTTP/FTP/SFTP Options The maximum number of connections to one server for each download. Default: ``1`` +.. option:: --max-http-forbidden=<NUM> + + If aria2 receives "forbidden" status from the remote HTTP + servers NUM times without getting a single byte, then force the + download to fail. Specify ``0`` to disable this option. This options + is effective only when using HTTP servers. The number of retry + attempt is counted toward :option:`--max-tries`, so it should be + configured too. + + Default: ``0`` + .. option:: --max-file-not-found=<NUM> If aria2 receives "file not found" status from the remote HTTP/FTP @@ -1927,6 +1938,13 @@ based on the last error encountered. 32 If checksum validation failed. +101 + If resource was forbidden. + +102 + If aria2 saw the specified number of "resource was forbidden" error. + See :option:`--max-http-forbidden` option. + .. note:: An error occurred in a finished download will not be reported @@ -2185,6 +2203,7 @@ of URIs. These optional lines must start with white space(s). * :option:`lowest-speed-limit <--lowest-speed-limit>` * :option:`max-connection-per-server <-x>` * :option:`max-download-limit <--max-download-limit>` + * :option:`max-http-forbidden <--max-http-forbidden>` * :option:`max-file-not-found <--max-file-not-found>` * :option:`max-mmap-limit <--max-mmap-limit>` * :option:`max-resume-failure-tries <--max-resume-failure-tries>` diff --git a/doc/manual-src/pt/aria2c.rst b/doc/manual-src/pt/aria2c.rst index 668a3331..945efb3b 100644 --- a/doc/manual-src/pt/aria2c.rst +++ b/doc/manual-src/pt/aria2c.rst @@ -189,6 +189,16 @@ Opções HTTP / FTP O número máximo de conexões para um servidor em cada download. Padrão: ``1`` + +.. option:: --max-http-forbidden=<NÚMERO> + + Se aria2 recebe çódigo de retorno "proibido" de um servidor + remoto de HTTP um NÚMERO de vezes sem obter nenhum byte, então o + download é forçado a falhar. + Especificar ``0`` para desabilitar esta opção. Esta opção só é válida + para servidores HTTP. + Padrão: ``0`` + .. option:: --max-file-not-found=<NÚMERO> Se aria2 recebe çódigo de retorno "arquivo não encontrado" de um servidor @@ -1670,6 +1680,13 @@ retorno ou saída com base no último erro encontrado. 30 Se aria2 não pode passar uma requisição JSON-RPC. +101 + Se o recurso foi proibido. + +102 + Se aria2 viu o número especificado de erro "recurso foi proibido". + Ver opção :option:`--max-http-forbidden` option. + .. note:: Um erro ocorrido em um download completado não será reportado como um @@ -1898,6 +1915,7 @@ URI. Estas linhas opcionais precisam iniciar com um ou mais espaços. * :option:`lowest-speed-limit <--lowest-speed-limit>` * :option:`max-connection-per-server <-x>` * :option:`max-download-limit <--max-download-limit>` + * :option:`max-http-forbidden <--max-http-forbidden>` * :option:`max-file-not-found <--max-file-not-found>` * :option:`max-resume-failure-tries <--max-resume-failure-tries>` * :option:`max-tries <-m>` diff --git a/doc/manual-src/ru/Makefile.am b/doc/manual-src/ru/Makefile.am index cdebe6f3..18ec0bac 100644 --- a/doc/manual-src/ru/Makefile.am +++ b/doc/manual-src/ru/Makefile.am @@ -126,7 +126,6 @@ text: man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man - sed -i -e '1i .\\" -*- mode: troff; coding: utf-8 -*-' $(BUILDDIR)/man/aria2c.1 @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." diff --git a/doc/manual-src/ru/aria2c.rst b/doc/manual-src/ru/aria2c.rst index ebebe4f3..272fc633 100644 --- a/doc/manual-src/ru/aria2c.rst +++ b/doc/manual-src/ru/aria2c.rst @@ -194,6 +194,18 @@ HTTP(S)/FTP, они тут же могут выгружаться в BitTorrent- загрузки. По умолчанию: ``1`` +.. option:: --max-http-forbidden=<NUM> + + Если aria2 получает статус "forbidden" (найден) с + удаленных HTTP-серверов NUM раз без получения, хотя бы одного байта, + тогда принудительно отказывается от загрузки. Укажите ``0``, чтобы + отключить этот параметр. Этот параметр действенен только, когда + используются HTTP-серверы. Количество повторных попыток засчитывается в + :option:`--max-tries`, таким образом, этот параметр также должен быть + сконфигурирован. + + По умолчанию: ``0`` + .. option:: --max-file-not-found=<NUM> Если aria2 получает статус "file not found" (файл не найден) с @@ -2035,6 +2047,13 @@ Content-Disposition. 32 Если проверка контрольной суммы не удалась. +101 + Если ресурс был запрещен. + +102 + Если aria2 увидела указанное количество «ресурсов было запрещено», ошибка. + Смотрите параметр :option:`--max-http-forbidden` option. + .. note:: Ошибка, произошедшая в завершенной загрузке, не будет передана как код @@ -2297,6 +2316,7 @@ URI. Эти дополнительные строки должны начина * :option:`lowest-speed-limit <--lowest-speed-limit>` * :option:`max-connection-per-server <-x>` * :option:`max-download-limit <--max-download-limit>` + * :option:`max-http-forbidden <--max-http-forbidden>` * :option:`max-file-not-found <--max-file-not-found>` * :option:`max-mmap-limit <--max-mmap-limit>` * :option:`max-resume-failure-tries <--max-resume-failure-tries>` diff --git a/doc/xmlrpc/aria2rpc b/doc/xmlrpc/aria2rpc index f52b7f6a..a9eb018f 100755 --- a/doc/xmlrpc/aria2rpc +++ b/doc/xmlrpc/aria2rpc @@ -43,6 +43,7 @@ OptionParser.new do |opt| options["dry-run"]=val||"true" } opt.on("--lowest-speed-limit SPEED"){|val| options["lowest-speed-limit"]=val} + opt.on("--max-http-forbidden NUM"){|val| options["max-http-forbidden"]=val} opt.on("--max-file-not-found NUM"){|val| options["max-file-not-found"]=val} opt.on("-m","--max-tries N"){|val| options["max-tries"]=val} opt.on("--no-proxy DOMAINS"){|val| options["no-proxy"]=val} diff --git a/src/HttpResponseCommand.cc b/src/HttpResponseCommand.cc index 6009dc2f..1df02b09 100644 --- a/src/HttpResponseCommand.cc +++ b/src/HttpResponseCommand.cc @@ -239,6 +239,9 @@ bool HttpResponseCommand::executeInternal() } if (statusCode >= 300) { + if (statusCode == 403) { + grp->increaseAndValidateHttpForbidden(); + } if (statusCode == 404) { grp->increaseAndValidateFileNotFoundCount(); } diff --git a/src/HttpSkipResponseCommand.cc b/src/HttpSkipResponseCommand.cc index a722d774..9aed0623 100644 --- a/src/HttpSkipResponseCommand.cc +++ b/src/HttpSkipResponseCommand.cc @@ -213,6 +213,14 @@ bool HttpSkipResponseCommand::processResponse() return prepareForRetry(0); } throw DL_ABORT_EX2(EX_AUTH_FAILED, error_code::HTTP_AUTH_FAILED); + case 400: + case 403: + if (getOption()->getAsInt(PREF_MAX_HTTP_FORBIDDEN) == 0) { + throw DL_ABORT_EX2(MSG_RESOURCE_FORBIDDEN, + error_code::RESOURCE_FORBIDDEN); + } + throw DL_RETRY_EX2(MSG_RESOURCE_FORBIDDEN, + error_code::RESOURCE_FORBIDDEN); case 404: if (getOption()->getAsInt(PREF_MAX_FILE_NOT_FOUND) == 0) { throw DL_ABORT_EX2(MSG_RESOURCE_NOT_FOUND, diff --git a/src/OptionHandlerFactory.cc b/src/OptionHandlerFactory.cc index 4339c912..a8b2a7c1 100644 --- a/src/OptionHandlerFactory.cc +++ b/src/OptionHandlerFactory.cc @@ -440,7 +440,7 @@ std::vector<OptionHandler*> OptionHandlerFactory::createOptionHandlers() { OptionHandler* op(new NumberOptionHandler(PREF_MAX_CONNECTION_PER_SERVER, TEXT_MAX_CONNECTION_PER_SERVER, - "1", 1, 16, 'x')); + "10", 1, -1, 'x')); op->addTag(TAG_BASIC); op->addTag(TAG_FTP); op->addTag(TAG_HTTP); @@ -501,7 +501,7 @@ std::vector<OptionHandler*> OptionHandlerFactory::createOptionHandlers() } { OptionHandler* op(new UnitNumberOptionHandler( - PREF_MIN_SPLIT_SIZE, TEXT_MIN_SPLIT_SIZE, "20M", 1_m, 1_g, 'k')); + PREF_MIN_SPLIT_SIZE, TEXT_MIN_SPLIT_SIZE, "1M", 1_k, 1_g, 'k')); op->addTag(TAG_BASIC); op->addTag(TAG_FTP); op->addTag(TAG_HTTP); @@ -861,6 +861,16 @@ std::vector<OptionHandler*> OptionHandlerFactory::createOptionHandlers() op->setChangeOptionForReserved(true); handlers.push_back(op); } + { + OptionHandler* op(new NumberOptionHandler(PREF_MAX_HTTP_FORBIDDEN, + TEXT_MAX_HTTP_FORBIDDEN, "0", -1)); + op->addTag(TAG_ADVANCED); + op->addTag(TAG_HTTP); + op->setInitialOption(true); + op->setChangeGlobalOption(true); + op->setChangeOptionForReserved(true); + handlers.push_back(op); + } { OptionHandler* op(new NumberOptionHandler(PREF_MAX_FILE_NOT_FOUND, TEXT_MAX_FILE_NOT_FOUND, "0", 0)); @@ -971,7 +981,7 @@ std::vector<OptionHandler*> OptionHandlerFactory::createOptionHandlers() } { OptionHandler* op( - new NumberOptionHandler(PREF_SPLIT, TEXT_SPLIT, "5", 1, -1, 's')); + new NumberOptionHandler(PREF_SPLIT, TEXT_SPLIT, "15", 1, -1, 's')); op->addTag(TAG_BASIC); op->addTag(TAG_FTP); op->addTag(TAG_HTTP); diff --git a/src/RequestGroup.cc b/src/RequestGroup.cc index 379029ec..e64c1487 100644 --- a/src/RequestGroup.cc +++ b/src/RequestGroup.cc @@ -139,6 +139,7 @@ RequestGroup::RequestGroup(const std::shared_ptr<GroupId>& gid, numStreamConnection_(0), numStreamCommand_(0), numCommand_(0), + httpForbiddenCount_(0), fileNotFoundCount_(0), maxDownloadSpeedLimit_(option->getAsInt(PREF_MAX_DOWNLOAD_LIMIT)), maxUploadSpeedLimit_(option->getAsInt(PREF_MAX_UPLOAD_LIMIT)), @@ -1255,6 +1256,17 @@ void RequestGroup::updateLastModifiedTime(const Time& time) lastModifiedTime_ = time; } } +void RequestGroup::increaseAndValidateHttpForbidden() +{ + ++httpForbiddenCount_; + const int maxCount = option_->getAsInt(PREF_MAX_HTTP_FORBIDDEN); + if (maxCount > 0 && httpForbiddenCount_ >= maxCount && + downloadContext_->getNetStat().getSessionDownloadLength() == 0) { + throw DOWNLOAD_FAILURE_EXCEPTION2( + fmt("Reached max-http-forbidden count=%d", maxCount), + error_code::MAX_HTTP_FORBIDDEN); + } +} void RequestGroup::increaseAndValidateFileNotFoundCount() { diff --git a/src/RequestGroup.h b/src/RequestGroup.h index 6698f93d..3c2ac9cf 100644 --- a/src/RequestGroup.h +++ b/src/RequestGroup.h @@ -155,6 +155,8 @@ private: int numCommand_; + int httpForbiddenCount_; + int fileNotFoundCount_; int maxDownloadSpeedLimit_; @@ -414,6 +416,8 @@ public: void updateLastModifiedTime(const Time& time); + void increaseAndValidateHttpForbidden(); + void increaseAndValidateFileNotFoundCount(); // Just set inMemoryDownload flag true. diff --git a/src/error_code.h b/src/error_code.h index 0d268c45..2af3875f 100644 --- a/src/error_code.h +++ b/src/error_code.h @@ -76,6 +76,8 @@ enum Value { JSON_PARSE_ERROR = 30, REMOVED = 31, CHECKSUM_ERROR = 32, + RESOURCE_FORBIDDEN= 101, + MAX_HTTP_FORBIDDEN=102, }; } // namespace error_code diff --git a/src/message.h b/src/message.h index 4705d7fb..b28d4a77 100644 --- a/src/message.h +++ b/src/message.h @@ -144,6 +144,7 @@ #define MSG_VERIFICATION_FAILED _("Checksum error detected. file=%s") #define MSG_INCOMPLETE_RANGE _("Incomplete range specified. %s") #define MSG_STRING_INTEGER_CONVERSION_FAILURE _("Failed to convert string into value: %s") +#define MSG_RESOURCE_FORBIDDEN _("Resource forbidden") #define MSG_RESOURCE_NOT_FOUND _("Resource not found") #define MSG_FILE_RENAMED _("File already exists. Renamed to %s.") #define MSG_CANNOT_PARSE_METALINK _("Cannot parse metalink XML file. XML may be malformed.") diff --git a/src/prefs.cc b/src/prefs.cc index 937e927d..5d2554d6 100644 --- a/src/prefs.cc +++ b/src/prefs.cc @@ -262,6 +262,8 @@ PrefPtr PREF_SERVER_STAT_OF = makePref("server-stat-of"); // value: true | false PrefPtr PREF_REMOTE_TIME = makePref("remote-time"); // value: 1*digit +PrefPtr PREF_MAX_HTTP_FORBIDDEN = makePref("max-http-forbidden"); +// value: 1*digit PrefPtr PREF_MAX_FILE_NOT_FOUND = makePref("max-file-not-found"); // value: epoll | select PrefPtr PREF_EVENT_POLL = makePref("event-poll"); diff --git a/src/prefs.h b/src/prefs.h index e1f83978..6a199404 100644 --- a/src/prefs.h +++ b/src/prefs.h @@ -215,6 +215,8 @@ extern PrefPtr PREF_SERVER_STAT_OF; // value: true | false extern PrefPtr PREF_REMOTE_TIME; // value: 1*digit +extern PrefPtr PREF_MAX_HTTP_FORBIDDEN; +// value: 1*digit extern PrefPtr PREF_MAX_FILE_NOT_FOUND; // value: epoll | select extern PrefPtr PREF_EVENT_POLL; diff --git a/src/usage_text.h b/src/usage_text.h index d73b50d9..f890f7d4 100644 --- a/src/usage_text.h +++ b/src/usage_text.h @@ -468,6 +468,15 @@ " connection to HTTP/FTP/proxy server. After the\n" \ " connection is established, this option makes no\n" \ " effect and --timeout option is used instead.") +#define TEXT_MAX_HTTP_FORBIDDEN \ + _(" --max-http-forbidden=NUM If aria2 receives `forbidden' status from the\n" \ + " remote HTTPservers NUM times without getting\n" \ + " a single byte, then force the download to fail.\n" \ + " Specify 0 to disable this option.\n" \ + " This options is effective only when using\n" \ + " HTTPservers. The number of retry attempt is\n" \ + " counted toward --max-tries, so it should be\n" \ + " configured too.") #define TEXT_MAX_FILE_NOT_FOUND \ _(" --max-file-not-found=NUM If aria2 receives `file not found' status from the\n" \ " remote HTTP/FTP servers NUM times without getting\n" \ diff --git a/Makefile.in b/Makefile.in index c6aac63..41df3e3 100644 --- a/Makefile.in +++ b/Makefile.in @@ -435,7 +435,6 @@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ -runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ diff --git a/deps/Makefile.in b/deps/Makefile.in index 34d60d8..77c66df 100644 --- a/deps/Makefile.in +++ b/deps/Makefile.in @@ -380,7 +380,6 @@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ -runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ diff --git a/doc/Makefile.in b/doc/Makefile.in index 0a40fdf..d918349 100644 --- a/doc/Makefile.in +++ b/doc/Makefile.in @@ -415,7 +415,6 @@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ -runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ diff --git a/doc/manual-src/Makefile.in b/doc/manual-src/Makefile.in index ca44a54..f725c84 100644 --- a/doc/manual-src/Makefile.in +++ b/doc/manual-src/Makefile.in @@ -380,7 +380,6 @@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ -runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ diff --git a/doc/manual-src/en/Makefile.in b/doc/manual-src/en/Makefile.in index 17c7266..7a47463 100644 --- a/doc/manual-src/en/Makefile.in +++ b/doc/manual-src/en/Makefile.in @@ -351,7 +351,6 @@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ -runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ diff --git a/doc/manual-src/pt/Makefile.in b/doc/manual-src/pt/Makefile.in index 7f734a5..9b22809 100644 --- a/doc/manual-src/pt/Makefile.in +++ b/doc/manual-src/pt/Makefile.in @@ -353,7 +353,6 @@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ -runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ diff --git a/doc/manual-src/ru/Makefile.in b/doc/manual-src/ru/Makefile.in index 2f31c1a..94a96ea 100644 --- a/doc/manual-src/ru/Makefile.in +++ b/doc/manual-src/ru/Makefile.in @@ -353,7 +353,6 @@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ -runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ @@ -729,7 +728,6 @@ text: man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man - sed -i -e '1i .\\" -*- mode: troff; coding: utf-8 -*-' $(BUILDDIR)/man/aria2c.1 @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." diff --git a/lib/Makefile.in b/lib/Makefile.in index a4c9d19..a8a0f4f 100644 --- a/lib/Makefile.in +++ b/lib/Makefile.in @@ -320,7 +320,6 @@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ -runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ diff --git a/src/Makefile.in b/src/Makefile.in index 0929012..16fa625 100644 --- a/src/Makefile.in +++ b/src/Makefile.in @@ -1429,7 +1429,6 @@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ -runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ diff --git a/src/includes/Makefile.in b/src/includes/Makefile.in index f05efbd..a607a7c 100644 --- a/src/includes/Makefile.in +++ b/src/includes/Makefile.in @@ -370,7 +370,6 @@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ -runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ diff --git a/test/Makefile.in b/test/Makefile.in index d95da8f..4a6f116 100644 --- a/test/Makefile.in +++ b/test/Makefile.in @@ -963,7 +963,6 @@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ -runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ diff --git a/doc/manual-src/en/_build/man/aria2c.1 b/doc/manual-src/en/_build/man/aria2c.1 index 1cf4c10..0f63692 100644 --- a/doc/manual-src/en/_build/man/aria2c.1 +++ b/doc/manual-src/en/_build/man/aria2c.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH "ARIA2C" "1" "May 15, 2018" "1.34.0" "aria2" +.TH "ARIA2C" "1" "Aug 14, 2018" "1.34.0" "aria2" .SH NAME aria2c \- The ultra fast download utility . @@ -237,6 +237,18 @@ Default: \fB1\fP .UNINDENT .INDENT 0.0 .TP +.B \-\-max\-http\-forbidden=<NUM> +If aria2 receives "forbidden" status from the remote HTTP +servers NUM times without getting a single byte, then force the +download to fail. Specify \fB0\fP to disable this option. This options +is effective only when using HTTP servers. The number of retry +attempt is counted toward \fI\%\-\-max\-tries\fP, so it should be +configured too. +.sp +Default: \fB0\fP +.UNINDENT +.INDENT 0.0 +.TP .B \-\-max\-file\-not\-found=<NUM> If aria2 receives "file not found" status from the remote HTTP/FTP servers NUM times without getting a single byte, then force the @@ -2251,6 +2263,13 @@ Reserved. Not used. .TP .B 32 If checksum validation failed. +.TP +.B 101 +If resource was forbidden. +.TP +.B 102 +If aria2 saw the specified number of "resource was forbidden" error. +See \fI\%\-\-max\-http\-forbidden\fP option. .UNINDENT .sp \fBNOTE:\fP @@ -2623,14 +2642,16 @@ of URIs. These optional lines must start with white space(s). .IP \(bu 2 \fI\%max\-download\-limit\fP .IP \(bu 2 +\fI\%max\-http\-forbidden\fP +.IP \(bu 2 \fI\%max\-file\-not\-found\fP .IP \(bu 2 \fI\%max\-mmap\-limit\fP -.IP \(bu 2 -\fI\%max\-resume\-failure\-tries\fP .UNINDENT .INDENT 2.0 .IP \(bu 2 +\fI\%max\-resume\-failure\-tries\fP +.IP \(bu 2 \fI\%max\-tries\fP .IP \(bu 2 \fI\%max\-upload\-limit\fP diff --git a/doc/manual-src/pt/_build/man/aria2c.1 b/doc/manual-src/pt/_build/man/aria2c.1 index f10dd1c..6346681 100644 --- a/doc/manual-src/pt/_build/man/aria2c.1 +++ b/doc/manual-src/pt/_build/man/aria2c.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH "ARIA2C" "1" "mai 15, 2018" "1.34.0" "aria2" +.TH "ARIA2C" "1" "ago 14, 2018" "1.34.0" "aria2" .SH NAME aria2c \- Utilitário para download super ultra rápido . @@ -219,6 +219,16 @@ Padrão: \fB1\fP .UNINDENT .INDENT 0.0 .TP +.B \-\-max\-http\-forbidden=<NÚMERO> +Se aria2 recebe çódigo de retorno "proibido" de um servidor +remoto de HTTP um NÚMERO de vezes sem obter nenhum byte, então o +download é forçado a falhar. +Especificar \fB0\fP para desabilitar esta opção. Esta opção só é válida +para servidores HTTP. +Padrão: \fB0\fP +.UNINDENT +.INDENT 0.0 +.TP .B \-\-max\-file\-not\-found=<NÚMERO> Se aria2 recebe çódigo de retorno "arquivo não encontrado" de um servidor remoto de HTTP / FTP um NÚMERO de vezes sem obter nenhum byte, então o @@ -1894,6 +1904,13 @@ temporária ou manutenção. .TP .B 30 Se aria2 não pode passar uma requisição JSON\-RPC. +.TP +.B 101 +Se o recurso foi proibido. +.TP +.B 102 +Se aria2 viu o número especificado de erro "recurso foi proibido". +Ver opção \fI\%\-\-max\-http\-forbidden\fP option. .UNINDENT .sp \fBNOTA:\fP @@ -2195,12 +2212,14 @@ URI. Estas linhas opcionais precisam iniciar com um ou mais espaços. .IP \(bu 2 \fI\%max\-download\-limit\fP .IP \(bu 2 -\fI\%max\-file\-not\-found\fP +\fI\%max\-http\-forbidden\fP .IP \(bu 2 -\fI\%max\-resume\-failure\-tries\fP +\fI\%max\-file\-not\-found\fP .UNINDENT .INDENT 2.0 .IP \(bu 2 +\fI\%max\-resume\-failure\-tries\fP +.IP \(bu 2 \fI\%max\-tries\fP .IP \(bu 2 \fI\%max\-upload\-limit\fP diff --git a/doc/manual-src/ru/_build/man/aria2c.1 b/doc/manual-src/ru/_build/man/aria2c.1 index 726b30c..70f05d8 100644 --- a/doc/manual-src/ru/_build/man/aria2c.1 +++ b/doc/manual-src/ru/_build/man/aria2c.1 @@ -1,7 +1,6 @@ -.\" -*- mode: troff; coding: utf-8 -*- .\" Man page generated from reStructuredText. . -.TH "ARIA2C" "1" "мая 15, 2018" "1.34.0" "aria2" +.TH "ARIA2C" "1" "авг. 14, 2018" "1.34.0" "aria2" .SH NAME aria2c \- сверхбыстрая утилита загрузки . @@ -248,6 +247,19 @@ BitTorrent\-загрузки. .UNINDENT .INDENT 0.0 .TP +.B \-\-max\-http\-forbidden=<NUM> +Если aria2 получает статус "forbidden" (найден) с +удаленных HTTP\-серверов NUM раз без получения, хотя бы одного байта, +тогда принудительно отказывается от загрузки. Укажите \fB0\fP, чтобы +отключить этот параметр. Этот параметр действенен только, когда +используются HTTP\-серверы. Количество повторных попыток засчитывается в +\fI\%\-\-max\-tries\fP, таким образом, этот параметр также должен быть +сконфигурирован. +.sp +По умолчанию: \fB0\fP +.UNINDENT +.INDENT 0.0 +.TP .B \-\-max\-file\-not\-found=<NUM> Если aria2 получает статус "file not found" (файл не найден) с удаленных HTTP/FTP\-серверов NUM раз без получения, хотя бы одного байта, @@ -2360,6 +2372,13 @@ aria2. .TP .B 32 Если проверка контрольной суммы не удалась. +.TP +.B 101 +Если ресурс был запрещен. +.TP +.B 102 +Если aria2 увидела указанное количество «ресурсов было запрещено», ошибка. +Смотрите параметр \fI\%\-\-max\-http\-forbidden\fP option. .UNINDENT .sp \fBПРИМЕЧАНИЕ:\fP @@ -2736,14 +2755,16 @@ URI. Эти дополнительные строки должны начина .IP \(bu 2 \fI\%max\-download\-limit\fP .IP \(bu 2 +\fI\%max\-http\-forbidden\fP +.IP \(bu 2 \fI\%max\-file\-not\-found\fP .IP \(bu 2 \fI\%max\-mmap\-limit\fP -.IP \(bu 2 -\fI\%max\-resume\-failure\-tries\fP .UNINDENT .INDENT 2.0 .IP \(bu 2 +\fI\%max\-resume\-failure\-tries\fP +.IP \(bu 2 \fI\%max\-tries\fP .IP \(bu 2 \fI\%max\-upload\-limit\fP diff --git a/src/bittorrent_helper.cc b/src/bittorrent_helper.cc index c0b1f60..0a0a631 100644 --- a/src/bittorrent_helper.cc +++ b/src/bittorrent_helper.cc @@ -216,6 +216,8 @@ void extractFileEntries(const std::shared_ptr<DownloadContext>& ctx, fileEntries.reserve(filesList->size()); int64_t length = 0; int64_t offset = 0; + int64_t fileIndex = 1; + std::string selectedFileIndex = ""; // multi-file mode torrent->mode = BT_FILE_MODE_MULTI; for (auto& f : *filesList) { @@ -293,7 +295,23 @@ void extractFileEntries(const std::shared_ptr<DownloadContext>& ctx, fileEntry->setSuffixPath(suffixPath); fileEntry->setMaxConnectionPerServer(maxConn); fileEntries.push_back(fileEntry); + + std::string filename = strjoin(pathelem.end()-1, pathelem.end(),' ',std::ptr_fun(util::encodeNonUtf8)); + if( filename.find_first_of("_____padding_file_")!=std::string::npos && filename.find("BitComet")!=std::string::npos ){ + A2_LOG_NOTICE(fmt("ignoring file: %s",filename.c_str())); + }else{ + selectedFileIndex += std::to_string(fileIndex)+","; + } + offset += fileEntry->getLength(); + fileIndex += 1; + + } + if(option->get(PREF_SELECT_FILE).empty()){ + A2_LOG_NOTICE(fmt("SYSTEM SET PREF_SELECT_FILE: %s",selectedFileIndex.c_str())); + option->put(PREF_SELECT_FILE,selectedFileIndex.c_str()); + }else{ + A2_LOG_NOTICE(fmt("GOT USER PREF_SELECT_FILE: %s",option->get(PREF_SELECT_FILE).c_str())); } } else { diff --git a/src/bittorrent_helper.h b/src/bittorrent_helper.h index a888601..b9ca4bd 100644 --- a/src/bittorrent_helper.h +++ b/src/bittorrent_helper.h @@ -50,6 +50,7 @@ #include "util.h" #include "DownloadContext.h" #include "TimeA2.h" +#include "LogFactory.h" namespace aria2 {
一直在路由器用修改版的ArDNSPod作为我的动态域名解析工具,一次偶然操作发现原来Dnspod支持给同一个域名设置多个IP,而我家的宽带是可以双拨的,马上设了个test二级域名实验了一下,稳了!既然有这需求,Dnspod也支持,应该也有现成轮子吧,于是看起搜寻模式,可是翻遍了GitHub愣是没找到...那就只能自己撸咯.于是就有了这个dns.sh脚本了.还是简单介绍下吧!需要在BASH下使用脚本使用了filedb (有修改)作为缓存,因此只有IP改变了才会调用DNSPod的接口使用json作为配置文件,那当然少不了json解析工具jq了不限制IP个数IP获取方法需要自行用shell实现
缘由自以为很明智地买了32G的手机,应该能用上3,4年.没想还不到两年,女王就天天抱怨手机木有空间.还好,不像我整个手机装的都是各种软件,她手机里占空间的主要还是照片和视频.那就弄个远程备份咯.可是放眼在线云盘,国内的说真的不敢用,要担心那天挂掉,或者那天数据泄露.免费用户还得忍受龟速下载.国外的虽然节操比国内的好些,但是订阅的价钱不低,还有墙的存在,始终放心不下.既然家里的城中村宽带在万能淘宝加持下变成了200M下行/30M上行,那就得用起来.开搞家里的路由器跑的梅林,早已经搭好了Optware-Ng的环境了,综合的各家评论,觉得搭个NextCloud云盘好了.下载源码包,配置nginx,开始安装,路由器是EA6500V2,armv7的CPU,256M的内存,挂载了一个1T的硬盘和远古时代的2G MiniSD卡,几分钟下来,终于是装好了,可是一登录,就来一个500错误!调试虽说有了心里准备,在路由上跑大型软件会有坑,可这登录就摸门钉,太...二话不说,看日志,可是这日志记录的太长了,被截断了.本来还想看看源码的,可面对着这庞大的项目却不知从何下手,算了,还是编译个xdebug慢慢追吧.既然抛出一个NotFoundException,那就在报错的地方,抛异常前打几个断点.试了几次,终于找到问题所在了.本来在获取一个文件时,应该返回该文件的元信息,例如创建时间,修改时间,文件大小之类的.可程序的到的却是一个个的null.这些元信息是原本是通过PHP的stat方法获取的.但是,如果PHP_INT_SIZE===4的话,却是通过系统命令stat获取的,而这梅林却没有这命令,stat命令不存在,PHP自然就返回null了,继而就抛出了NotFoundException.解决方法也很简单.查到stat命令是属于coreutils-stat这个包的,ipkg update && ipkg install coreutils-stat装上stat命令后果然正常了,创建个账号,试了下手机客户端,可以正常备份了.