Read More

Git sparse-checkout and partial clones for Mega-Repos

Recently I was planning to make a small contribution to project DefinitelyTyped/DefinitelyTyped. It is a huge repository that hosts and provides the type declaration files for thousands of packages on npm, with a structure like the following

├── 11ty__eleventy-img
├── 1line-aa
├── 3box
├── ...and more of them...

Each sub-folder under the types/ directory corresponds to a specific npm package.

The package I am interested in is marked, whose type declaration files reside in types/marked/ directory. Towards this end, cloning the whole repository and doing a full checkout is considered worthless, since my disk space and network traffic would be eaten up by a bunch of irrelevant files.

Read More

Git-based Dependencies in Dart and Go

Both Dart and Go support decentralized package distribution. One is able to directly adopt an existing git repository as dependency, easing the effort of distributing packages.

Sometimes we might expect more fine-grained control on what to pull from a git repository. For example, to lock a package’s version, we would specify a particular tag, commit or branch name to pull from. Or if it’s a mono-repo, we would choose a sub-directory from the repository root. This post summarizes how to achieve these purposes in both languages.

Read More

Retrieve Contents over HTTP without curl or wget

I came across a piece of interesting vulnerable script from a post on V2EX 1) on V2EX. A bash function named __curl inside the file retrieves contents over HTTP as a simple alternative for command curl or wget, in scenarios where no such utilities available.

function __curl() {
read proto server path <<<$(echo ${1//// })
DOC=/${path// //}
[[ x"${HOST}" == x"${PORT}" ]] && PORT=80

exec 3<>/dev/tcp/${HOST}/$PORT
echo -en "GET ${DOC} HTTP/1.0\r\nHost: ${HOST}\r\n\r\n" >&3
(while read line; do
[[ "$line" == $'\r' ]] && break
done && cat) <&3
exec 3>&-

The function makes use of certain less known features of Linux and the Bash language.

The first one is communicating over TCP through files. Linux employs a design philosophy of “everything are files”. One could find some special device files in directory /dev, through which we can manipulate the underlying devices. Specifically, manipulating a TCP socket connecting ${HOST}:${PORT} could be achieved by accessing device file /dev/tcp/${HOST}/${PORT}. Since HTTP is a text-based protocol over TCP, working with it is no more difficult than reading / writing a text file. Line exec 3<>$FILENAME opens file $FILENAME under read-write mode and binds it to descriptor 3. The next line then manually composes an HTTP payload and writes out to &3, which is in fact requesting the URL http ://${HOST}:${PORT}. By reading the same file descriptor, we retrieve the response content from the service. The trick serves as a primitive workaround for retrieving contents from web.

Another one is parameter substitution in Bash. The expression ${var//PATTERN/REPL} substitutes all occurrences of PATTERN in var into REPL. If REPL omitted, the matched substrings will be deleted. For example, in this script, ${1//// } would replace all slashes / into white spaces in variable $1.


  1. Parameter Substitution
  1. [收到条阿里云的告警,看不懂是做什么用的,请教一下 - V2EX](

Visualizing Correlation

Say we have a matrix A of shape N x M , which can be viewed as a collection of N vectors of shape 1 x M . The code below gives us the correlation matrix of A :

A_corr = np.corrcoef(A)  # shape: (N, N)

To visualize it, just use plt.matshow(A_corr) .

If N is so large that the figure could not provide a clear insight, we might alternatively use histograms like this:

def corr_matrix_to_array(corr_mat):
N = corr_mat.shape[0]
return np.array([corr_mat[i][j] for i in range(1, N) for j in range(i + 1, N)])

plt.hist(corr_matrix_to_array(A_corr), bins=np.linspace(-1, 1, N_bins))

SS Configuration

SS Client

$ [sudo] pip3 install shadowsocks

/etc/ss.json :

"server": "<server ip>",
"server_port": "<server port>", // must be Number
"password": "<password>",
"local_address": "",
"local_port": 1081,
"timeout": 300,
"method": "aes-256-cfb",
"fast_open": false
$ [sudo] sslocal -c /etc/ss.json -d start


clone repository from , make && sudo make install .

Append following lines to /etc/proxychains.conf :

# add proxy here ...
# meanwile
# defaults set to "tor"
socks5 1081

Usage: proxychains [command] .

Chrome Addons

SwitchyOmega .

使用 Aria2 在 Ubuntu 中下载百度云资源


配置 Aria2

Github 下载源码 ./configure && make -j8 && sudo make install

配置 Chrome 插件


$ cd ariac
$ cat >
> #!/bin/bash
> aria2c --conf=aria2.conf
> ^D
$ chmod +x
$ ./

安装 Chrome 插件

打开 chrome://extensionsLoad Unpacked 选择 chrome/release

完成后在百度云页面上会有 导出下载 按钮。

Ubuntu 重新映射键盘布局


刚开始只是方向键失灵,好在可以用小键盘替代;后来右 Ctrl 和 Alt 也失灵了,好在可以用左边的替代;直到最近 Fn 键也失灵了,终于逼疯了我——因为这意味着 F1 ~ F12 都将不能使用。

我曾试图寻找方法将 CapsLock 键映射为 Fn 键映射,但失败了——Fn 键消息是由 BIOS 拦截的,无法被操作系统捕获。


  • 交换 Fn 和 Ctrl。这是唯一一种能让 Fn 键移位的方式,在所有的 BIOS 中都可以设置。
  • 将 CapsLock 映射为 Ctrl。反正 CapsLock 闲着也是闲着,不如用它代替坏了的键。

ubuntu 下需要执行:

setxkbmap -layout us -option ctrl:nocaps

参考: How do I turn Caps Lock into an extra Control key? - Ask Ubuntu

如何备份 apt-get 已安装的软件列表

apt-get 是 Ubuntu 下管理软件包的一个工具,实用简单,功能强大。平时若要安装或卸载软件包,只需轻敲一条指令即可。每一台 Ubuntu 上,都安装着数以千百计的软件包——或是内核模块,或是工作、娱乐所需的软件,在它们的支持下,工作着这个开放的操作系统。




sudo dpkg --get-selections  > app-backup-list.lst



sudo dpkg --set-selections < app-backup-list.lst
sudo apt-get -y update
sudo apt-get dselect-upgrade

至于软件源的备份,只需将 /etc/apt/sources.list 文件复制过去即可。