Retrieve Web Contents without curl or wget

I came across a piece of interesting vulnerable script from a thread on V2EX. A bash function in it named __curl as a simple alternative for command curl or wget, works in scenarios where no such utilities available.

#!/bin/bash
function __curl() {
read proto server path <<<$(echo ${1//// })
DOC=/${path// //}
HOST=${server//:*}
PORT=${server//*:}
[[ x"${HOST}" == x"${PORT}" ]] && PORT=80

exec 3<>/dev/tcp/${HOST}/$PORT
echo -en "GET ${DOC} HTTP/1.0\r\nHost: ${HOST}\r\n\r\n" >&3
(while read line; do
[[ "$line" == $'\r' ]] && break
done && cat) <&3
exec 3>&-
}

The function involves certain less known features of Linux and Bash language.

First is communicating over TCP through files. Linux employs a design of “everything are files”. Certain devices are exposed as files which could be accessed under directory /dev. For example, one could manipulate a TCP socket connected to ${HOST}:${PORT} through device file /dev/tcp/${HOST}/${PORT}. The line exec 3<>$FILENAME opens file $FILENAME under read-write mode and binds it to descriptor 3. The next line composes HTTP message manually and writes out to &3, which is in fact sending a request to the URL. Reading from the same file should then retrieve the response content. The trick serves as a primitive workaround for retrieving web contents.

Another one is parameter substitution in Bash. The syntax ${var//PATTERN/REPL} globally replaces all occurrences of PATTERN in var into REPL. If REPL omitted, the matched substrings are deleted. For example, in this script, ${1//// } would replace all slashes / into white spaces in variable $1.

References

  1. Parameter Substitution

Visualizing Correlation

Say we have a matrix A of shape N x M , which can be viewed as a collection of N vectors of shape 1 x M . The code below gives us the correlation matrix of A :

A_corr = np.corrcoef(A)  # shape: (N, N)

To visualize it, just use plt.matshow(A_corr) .

If N is so large that the figure could not provide a clear insight, we might alternatively use histograms like this:

def corr_matrix_to_array(corr_mat):
N = corr_mat.shape[0]
return np.array([corr_mat[i][j] for i in range(1, N) for j in range(i + 1, N)])

plt.hist(corr_matrix_to_array(A_corr), bins=np.linspace(-1, 1, N_bins))

SS Configuration

SS Client

$ [sudo] pip3 install shadowsocks

/etc/ss.json :

{
"server": "<server ip>",
"server_port": "<server port>", // must be Number
"password": "<password>",
"local_address": "127.0.0.1",
"local_port": 1081,
"timeout": 300,
"method": "aes-256-cfb",
"fast_open": false
}
$ [sudo] sslocal -c /etc/ss.json -d start

proxychains

clone repository from https://github.com/rofl0r/proxychains-ng , make && sudo make install .

Append following lines to /etc/proxychains.conf :

[ProxyList]
# add proxy here ...
# meanwile
# defaults set to "tor"
socks5 127.0.0.1 1081

Usage: proxychains [command] .

Chrome Addons

SwitchyOmega .


使用 Aria2 在 Ubuntu 中下载百度云资源

可以实现满带宽下载。

配置 Aria2

Github 下载源码 ./configure && make -j8 && sudo make install

配置 Chrome 插件

clone https://github.com/acgotaku/BaiduExporter

$ cd ariac
$ cat > start.sh
> #!/bin/bash
> aria2c --conf=aria2.conf
> ^D
$ chmod +x start.sh
$ ./start.sh

安装 Chrome 插件

打开 chrome://extensionsLoad Unpacked 选择 chrome/release

完成后在百度云页面上会有 导出下载 按钮。


Ubuntu 重新映射键盘布局

键盘持续失灵,已经到了让我忍无可忍的地步了。

刚开始只是方向键失灵,好在可以用小键盘替代;后来右 Ctrl 和 Alt 也失灵了,好在可以用左边的替代;直到最近 Fn 键也失灵了,终于逼疯了我——因为这意味着 F1 ~ F12 都将不能使用。

我曾试图寻找方法将 CapsLock 键映射为 Fn 键映射,但失败了——Fn 键消息是由 BIOS 拦截的,无法被操作系统捕获。

但今天我找到了一个更好的替代方案:

  • 交换 Fn 和 Ctrl。这是唯一一种能让 Fn 键移位的方式,在所有的 BIOS 中都可以设置。
  • 将 CapsLock 映射为 Ctrl。反正 CapsLock 闲着也是闲着,不如用它代替坏了的键。

ubuntu 下需要执行:

setxkbmap -layout us -option ctrl:nocaps

参考: How do I turn Caps Lock into an extra Control key? - Ask Ubuntu


如何备份 apt-get 已安装的软件列表

apt-get 是 Ubuntu 下管理软件包的一个工具,实用简单,功能强大。平时若要安装或卸载软件包,只需轻敲一条指令即可。每一台 Ubuntu 上,都安装着数以千百计的软件包——或是内核模块,或是工作、娱乐所需的软件,在它们的支持下,工作着这个开放的操作系统。

但,如果有一天,系统需要被重装——或是无可救药了,抑或是购置了新的设备,问题来了:

如何将现有电脑上的软件包迁移至新的系统呢?

很简单。首先,将原有的软件列表导出:

sudo dpkg --get-selections  > app-backup-list.lst

最好是设置一个定时任务,每隔一段时间就保存一次列表,并且要保存到一个独立的分区。以免某天系统真的坏了。

接下来便是导入了:

sudo dpkg --set-selections < app-backup-list.lst
sudo apt-get -y update
sudo apt-get dselect-upgrade

至于软件源的备份,只需将 /etc/apt/sources.list 文件复制过去即可。


Ubuntu 网络每20秒断开重连一次的解决方案

从昨天到现在一直都有这个问题,刚刚突然就解决了,至今不明白原理,在此记录一下:

在网络设置中取消「需要 IPv6 完成这个链接」的选项


Ubuntu 杀死进程

ps -e #查看进程列表
sudo kill <pid>

Ubuntu 安装 Sublime 2

sudo add-apt-repository ppa:webupd8team/sublime-text-2
sudo apt-get update
sudo apt-get install sublime-text-2

Ubuntu加入自己的字体

假设字体文件夹为:~/Fonts。执行:

sudo mkdir -p /usr/share/fonts/myFonts
sudo cp ~/Fonts/*.ttf /usr/share/fonts/myFonts/
sudo chmod 644 /usr/share/fonts/myFonts/*.ttf
cd /usr/share/fonts/winFonts/
sudo mkfontscale #创建雅黑字体的fonts.scale文件,它用来控制字体旋转缩放
sudo mkfontdir #创建雅黑字体的fonts.dir文件,它用来控制字体粗斜体产生
sudo fc-cache -fv #建立字体缓存信息,也就是让系统认识雅黑