月度归档:2016年09月

golang hello world

安装golang

下载

wget -c https://storage.googleapis.com/golang/go1.7.1.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.7.1.linux-amd64.tar.gz

vim ~/.bash_profile

export PATH=$PATH:/usr/local/go/bin

测试

go version  

设置gopath环境变量

mkdir ~/gocode
vim ~/.bash_profile

新增以下内容

export GOPATH=~/gocode
export PATH=$PATH:$GOPATH/bin

导入环境变量

source ~/.bash_profile

hello world

mkdir $GOPATH/bin 
mkdir $GOPATH/pkg 
mkdir $GOPATH/github.com/user/hello -p
cd $GOPATH/github.com/user/hello

vim hello.go

package main

import "fmt"

func main() {
        fmt.Printf("Hello, world.\n")
}

go build # 构建(可执行程序在$GOPATH/github.com/user/hello) go install # 安装(可执行程序在$GOPATH/bin) hello # 运行

linux cp overwrite confirmation

This is probably caused by cp being already aliased to something like cp -i. Calling cp directly should work:

/bin/cp -rf /zzz/zzz/* /xxx/xxx

Another way to get around this is to use the yes command:

yes | cp -rf /zzz/zzz/* /xxx/xxx

waf比较(lua-nginx-module,modsecurity,naxsi)

ngx_lua_waf是一个基于lua-nginx-module(openresty)的web应用防火墙

https://github.com/loveshell/ngx_lua_waf

nginx配合modsecurity实现WAF功能

http://www.52os.net/articles/nginx-use-modsecurity-module-as-waf.html

NAXSI is an open-source, high performance, low rules maintenance WAF for NGINX

https://github.com/nbs-system/naxsi

中、小企业如何自建免费的云WAF

https://zhuanlan.zhihu.com/p/22068364

X-WAF是一款适用中、小企业的云WAF系统,让中、小企业也可以非常方便地拥有自己的免费云WAF。

https://waf.xsec.io/docs

基于openresty的Web应用安全防护系统(WAF)
http://git.oschina.net/miracleqi/OpenWAF

kjournald占I/O高,如何优化

nfsd 42687 kjournald 31201 rsync 21846 两分钟内的磁盘块读写数量排行

1、用dumpe2fs看一下Journal size是多大,把Journal size的值改大一些。

2、调整一下文件系统的jounal模式,默认为ordered ,改成writeback会提高一些效率。

Despite writing some data more than once, ext3 is often faster (higher throughput) than ext2 because ext3's journaling optimizes hard drive head motion. You can choose from three journaling modes to optimize speed, optionally choosing to trade off some data integrity.

  • One mode, data=writeback, limits the data integrity guarantees, allowing old data to show up in files after a crash, for a potential increase in speed under some circumstances. (This mode, which is the default journaling mode for most journaling file systems, essentially provides the more limited data integrity guarantees of the ext2 file system and merely avoids the long file system check at boot time.)

  • The second mode, data=ordered (the default mode), guarantees that the data is consistent with the file system; recently-written files will never show up with garbage contents after a crash.

  • The last mode, data=journal, requires a larger journal for reasonable speed in most cases and therefore takes longer to recover in case of unclean shutdown, but is sometimes faster for certain database operations.

The default mode is recommended for general-purpose computing needs. To change the mode, add the data=something option to the mount options for that file system in the /etc/fstab file, as documented in the mount man page (man mount).

查看linux服务器硬盘IO读写负载

用top命令查看

top – 16:15:05 up 6 days,  6:25,  2 users,  load average: 1.45, 1.77, 2.14
Tasks: 147 total,   1 running, 146 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.2% us,  0.2% sy,  0.0% ni, 86.9% id, 12.6% wa,  0.0% hi,  0.0% si
Mem:   4037872k total,  4003648k used,    34224k free,     5512k buffers
Swap:  7164948k total,   629192k used,  6535756k free,  3511184k cached

查看12.6% wa

IO等待所占用的CPU时间的百分比,高过30%时IO压力高

iostat -x 1 10

avg-cpu:  %user   %nice    %sys %iowait   %idle
0.00       0.00     0.25    33.46    66.29

Device:    rrqm/s  wrqm/s   r/s    w/s     rsec/s   wsec/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sda          0.00    0.00      0.00   0.00    0.00    0.00         0.00     0.00     0.00           0.00    0.00    0.00   0.00
sdb          0.00   1122  17.00  9.00  192.00 9216.00    96.00  4608.00   123.79   137.23 1033.43  13.17 100.10
sdc          0.00    0.00     0.00   0.00     0.00     0.00      0.00     0.00     0.00             0.00    0.00      0.00   0.00

查看%util 100.10 %idle 66.29

如果 %util 接近 100%,说明产生的I/O请求太多,I/O系统已经满负荷,该磁盘可能存在瓶颈。

idle小于70% IO压力就较大了,一般读取速度有较多的wait.

vmstat

同时可以结合vmstat 查看查看b参数(等待资源的进程数)

vmstat -1

再通过如下脚本查看高峰的进程io情况

#!/bin/sh
/etc/init.d/syslog stop
echo 1 > /proc/sys/vm/block_dump
sleep 60
dmesg | awk '/(READ|WRITE|dirtied)/ {process[$1]++} END {for (x in process) \
print process[x],x}' |sort -nr |awk '{print $2 " " $1}' | \
head -n 10
echo 0 > /proc/sys/vm/block_dump
/etc/init.d/syslog start

使用rvm安装多个ruby版本

Step 1: Upgrade Packages

# yum update

Step 2: Installing Recommended Packages

# yum install gcc-c++ patch readline readline-devel zlib zlib-devel
# yum install libyaml-devel libffi-devel openssl-devel make
# yum install bzip2 autoconf automake libtool bison iconv-devel

Step 3: Install RVM ( Ruby Version Manager )

# curl -L get.rvm.io | bash -s stable

Step 4: Setup RVM Environment

# source /etc/profile.d/rvm.sh

Step 5: Install Required Ruby Version

# rvm install 1.9.3

Step 6: Setup Default Ruby Version

# rvm use 1.9.3 --default 

Step 7: Check Current Ruby Version

# ruby --version

配置nginx使用letsencrypt免费https证书

https://letsencrypt.org/

生成证书

certbot certonly --webroot \
-w /data/web/c4ys/frontend/web/ -d c4ys.com -d www.c4ys.com \
-w /data/web/c4ys/mobile/web/ -d m.c4ys.com

修改nginx配置

listen       443 ssl;
ssl                  on;
ssl_certificate /etc/letsencrypt/live/www.c4ys.com/fullchain.pem;     
ssl_certificate_key /etc/letsencrypt/live/www.c4ys.com/privkey.pem;
ssl_session_cache    shared:SSL:1m;
ssl_session_timeout  5m;
ssl_ciphers  HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers  on;
error_page 497  https://$host$uri?$args;

自动更新证书

0 3 */10 * * certbot renew --quiet
0 3 */10 * * service nginx restart

inotify-tools + rsync 实现文件自动同步备份

公司图片服务器以前以nfs挂载在各台php-fpm服务器下,最近因为部分fpm服务器迁移到aliyun,远程挂载效率低下,为了本地不修改代码,又可以达到不低,所以想到了inotify + rsync。

inotify-tools介绍:https://github.com/rvoicilas/inotify-tools/wiki

#!/bin/sh

# get the current path
CURPATH=`pwd`

inotifywait -mr --timefmt '%d/%m/%y %H:%M' --format '%T %w %f' \
-e close_write /tmp/test | while read date time dir file; do

       FILECHANGE=${dir}${file}
       # convert absolute path to relative
       FILECHANGEREL=`echo "$FILECHANGE" | sed 's_'$CURPATH'/__'`

       rsync --progress --relative -vrae 'ssh -p 22'  $FILECHANGEREL usernam@example.com:/backup/root/dir && \
       echo "At ${time} on ${date}, file $FILECHANGE was backed up via rsync"
done