RSS feed [root] /unix /weblog



title search:


Wed Jan 18 22:23:58 HKT 2017


(google search) (amazon search)
download zip of files only

Sun Aug 06 17:21:16 HKT 2017 From /weblog/unix/script


10 awk tips -

8 Powerful Awk Built-in Variables – FS, OFS, RS, ORS, NR, NF, FILENAME, FNR -[..]iables-fs-ofs-rs-ors-nr-nf-filename-fnr/

Tutorial of text editing commands -[..]works/aix/library/au-unixtext/index.html

wc and nl tutorial -

tips for uniq, like show only items have duplications -

Tips of "cat", include display by column and by delimiter -

Using lynx to convert HTML to text -

(google search) (amazon search)

Sun Aug 06 17:20:16 HKT 2017 From /weblog/unix/script


Various ways to working with file -

Use truncate to pad the file size up -

Making a file unalterable with chattr -

(google search) (amazon search)

Sun Jul 30 17:10:51 HKT 2017 From /weblog/unix


Few config suggestions -[..]20-linux-server-performance-tips-part-2/

Linux kernel profiling with perf -

Intel VTune Amplifier profiler -

Oracle solaris studio -[..]torage/solarisstudio/overview/index.html <- "This page links to various Linux performance material I've created, including the tools maps on the right."

(google search) (amazon search)

Wed Jun 14 18:34:19 HKT 2017 From /weblog/unix/script


A bash debugger -

Really really good tips -[..]3/10/shell-scripting-best-practices.html

Style guide -[..]og/2012/11/14/defensive-bash-programming

Script checker -

How to write a shell -

(google search) (amazon search)

Wed Jun 14 11:13:47 HKT 2017 From /weblog/unix


How to get the system thread version - getconf GNU_LIBPTHREAD_VERSION[..]wse_thread/thread/81ff4b5d1ea3372d?hl=en

How to prevent module getting load, by edit /etc/modprobe.d/blacklist.conf :

Standard Linux error code -[..]11/03/linux-performance-monitoring-intro

Discussion about why Linux is way faster than Windows for huge number of small file processing -[..]s-to-go-as-fast-as-linux-for-compiling-c

"sync; sudo echo 3 > /proc/sys/vm/drop_caches" -[..]ow-to-clear-cached-memory-on-ubuntu.html

Funny linux commands -

25 years of linux -[..]acts-about-linux-c8d8ac30076d#.m6b9wnhni

basic-principle-of-Linux-privilege-control -[..]sic-principle-of-Linux-privilege-control

(google search) (amazon search)

Mon Feb 13 14:38:25 HKT 2017 From /weblog/unix


Web base vim as tutorials -

History of VIM -[..]f-productivity-vims-20th-anniversary.ars

Guide for editing text object, most already familiar with using 'w' for word editing, this blog discuss more about using 'p' for paragrahy and other related stuff. -[..]7/vim-text-objects-the-definitive-guide/

Seven habits of effective text editing , a lot of nice tip of using vim, like hlsearch , omni-completion

vimdiff tips -[..]cles/2008/04/30/vim-as-a-diff-merge-tool[..]9899-uid-36931-do-showone-type-blog.html

How to regex with line number -

cheat sheet of jumping around source code -[..]inux-HOWTO/C-editing-with-VIM-HOWTO.html

Nice analysis of good programming environment -[..]why-i-switched-from-textmate-to-vim.html

Creating macros in vim - <- introduction to Command-T and FuzzyFinder

Some VIM tips:

Essential Vim editor navigation commands
Vim search and replace – 12 powerful find and replace examples.
How To add bookmarks inside the Vim editor
How To record and play inside the Vim editor
Correct spelling mistakes automatically inside the Vim Editor
Automatic word completion using Ctrl-X
Enable thesaurus option in the Vim editor
Vim autocommand magic. Add custom header to your files automatically.
Convert Vim editor to a beautiful source code browser.
Use the Vim editor as a bash IDE, or C/C++ IDE, or Perl IDE.[..]com/2016/04/19/vim-commands-cheat-sheet/

ctrl+w, ctrl+w , vimdiff switch folder -

tutorial -[..]editor-in-unix-example-tutorial-and.html[..]s/3mfvdg/how_do_i_go_about_learning_vim/

Compilation plugin -

Highlight multiple searches at the same time, each with a different color. -

C/C++ IDE -[..]10/vim-plugin-for-navigating-c-with.html

Evil, putting vi in emacs -

Fixing syntax at solaris, "export TERM=xtermc", ":syntax enable" -

Graph your Vim undo tree in style. -

Introduction to Selecta and Command-T, tool for fuzzy mapping and open files easier -[..]d-the-vimrc-file-be-located-on-windows-7

Open Visual Studio config as XML -!topic/vim_use/HzVM9tjJP08

Snippert library ( e.g. function template ) -[..]3h7lca/which_snippet_library_do_you_use/[..]t_are_your_musthave_configs_and_plugins/[..]mments/2w22gf/edit_google_docs_with_vim/[..]nts/3ijsi2/original_post_a_vimrc_primer/[..]an_autocomplete_plugin_if_yes_which_one/

gUU to turn whole line upper case, and guu to turn whole line lower case -[..]_guu_makes_an_entire_line_lowercase_and/

can use :TOhtml to turn a text file to HTML -

set encoding=utf-8 -[..]how_to_reliably_switch_to_using_utf8_by/

If you like to test the regex for vi, can refer to this -[..]/blog/vim-creator-champions-charityware/

Another blog about using vim as IDE -

(google search) (amazon search)

Fri Jan 20 11:52:35 HKT 2017 From /weblog/unix


Some useful tools -

Tuning -[..]1/17/optimizing-linux-for-slow-computers

(google search) (amazon search)

Wed Nov 30 12:57:15 HKT 2016 From /weblog/unix


25 years old bug -

The problem of copying to other machine and screw up the permission -[..]nix-horror-stories-good-thing-about.html

(google search) (amazon search)

Thu Oct 06 23:23:20 HKT 2016 From /weblog/unix


A very nice sectioned list of unix command -

Backup a list of directories -

A quick reference -

Tuning cheat sheet -

Common and difference between systems -

BSD for linux user -

Actually a quick tutorial -

(google search) (amazon search)

Thu May 26 10:40:14 HKT 2016 From /weblog/unix


Advisory Locking and Mandatory Locking -

Basic info of inode -

Explanation of /proc filesystem -[..]anual/ref-guide/s1-proc-directories.html

Linux file system structure -

File system for archive files -

BeOS file system, other than discuss about BeOS FS, also discussing about evolution of design of file system -[..]rce/news/2010/06/the-beos-filesystem.ars[..]03/past-present-future-file-systems.ars/


The file size of command "df" return is compressed data, raw disk usage
If we want compress amount, we need to get the compressratio using /usr/sbin/zfs
We can get more system detail via perl package Sun::Solaris::Kstat , we can google "" as an example

Using ZFS as version control -

(google search) (amazon search)

Fri Apr 01 11:29:26 HKT 2016 From /weblog/unix/script


How to use eval, hash, readonly, shift, getopts, set, unset, let and shopt -

Sample bash completion

SSH_COMPLETE=( $(cut -f1 -d' ' ~/.ssh/known_hosts |\
tr ',' '\n' |\
sort -u |\
grep -e '[:alpha:]') )
complete -o default -W "${SSH_COMPLETE[*]}" ssh

Check history statistic - brianm@binky:~$ history | awk {'print $2'} | sort | uniq -c | sort -k1 -rn | head
164 svn
52 cd
42 ssh
32 sudo
22 git
16 ls
16 for
14 echo
13 man
10 curl

easy way to do "[-h|--help][-n|--name ]" -

(google search) (amazon search)

Fri Mar 18 12:15:20 HKT 2016 From /weblog/unix


TCP tuning -

UNIX network performance analysis -[..]works/aix/library/au-networkperfanalysis

Netstat tips -

Introduction to a lot of difference commands -[..]work-monitoring-commands-in-linux-part-i

Commandline tools which can do most HTTP message -[..]11/scripting-web-applications-with-curl/

Tools to show contents from network -[..]/powerful-command-line-tools-developers/[..]nd-to-find-ip-address-from-hostname.html

/rmem_default: The default setting of the socket receive buffer in bytes.
/rmem_max: The maximum receive socket buffer size in bytes.
/wmem_default: The default setting (in bytes) of the socket send buffer.
/wmem_max: The maximum send socket buffer size in bytes.
Only for Linux -

Getting IP of the host: "wget -O - -q ; echo" from -[..]ou-want-to-check-you-current-public.html

Use tcpdump and tshark to monitor network activity -

(google search) (amazon search)

Wed Feb 03 22:00:00 HKT 2016 From /weblog/unix


Sum time in script -[..]-summing-the-total-time-from-a-log-file/

Special bash parameters -[..]ial-bash-parameters-in-script-linux.html

ls sort by file size -[..]ns-answers/6518-sorting-ls-filesize.html

How to use "~" -

use rlwrap to enable BASH like editing in SQLplus, and use screen to presist terminal sessions -[..]miracle-utilities-rlwrap-and-screen.html

Ctrl+r searches through history
Use the history command plus !# to run old commands
Shell expansion using {} , e.g. cp{,.old} == cp
zsh have good auto complete

mkdir -p creates nested directories
Use less instead of tail
lsof shows open files
ps will show process trees with the f flag

!! is the previous command in the shell history
Use vim -b to show nonprintable characters
** is a recursive wildcard in zsh
cd – will return to the previous folder
Use ctrl+z and kill %1 to kill a process that will not die
pwdx shows the working directory of a process
Use sh -x to debug shell scripts
sysctl replaces /proc on macs

ps -eo pcpu,pid,args | sort -n => check CPU%
grep -sq "" /etc/lsb-release && lsb_release -rd => run command if file exist

More about ps and free -

HTML tools for commandline -

Bash history -

Tips of using shell, like saving all history -

Standard Shell completion:

Variablename completion
Username completion
Executable completion
Filename and directory completion
Hostname completion

cp long_and_complex_file_name long_and_complex_file_name.backup
with this:
cp long_and_complex_file_name{,.backup}[..]/2013/12/unix-trick-brace-expansion.html

Use the history -

How to use xargs -

Nice presentation of man page -

Parallel shell -

(google search) (amazon search)

Wed Feb 03 21:57:47 HKT 2016 From /weblog/unix/script


An question about how to remove unused package from MacOSX turn out to be an example of "find" command:[..]find-command-in-unix-examples-basic.html

File Search tips in Linux -[..]iscussTopicParent=18272&ixDiscussGroup=3

Another search tip, example of find base on time and size, and how to compare with other files -[..]e-useful-unix-file-finding-commands.html

Like search by name, depth, size -[..]5-practical-linux-find-command-examples/

use 'find' to delete branch of file:

By time -[..]unix-linux-find-command-examples-part-2/

Use of empty -[..]010/03/find-empty-directories-and-files/

(google search) (amazon search)

Wed Feb 03 21:55:53 HKT 2016 From /weblog/unix/script


(google search) (amazon search)

Fri Jan 22 11:00:48 HKT 2016 From /weblog/unix


Using unhide to check hidden process -[..]unhide-the-opensource-forensic-tool.html

fuser, check which user using the file or directory -

htop, look much better than top -[..]ght-no-i-am-loving-htop-it-is-linux.html

Tutorial of both lsof and pstack -[..]ternals-with-lsof-and-strace/#strace_run

Tutorial of SAR - . In Solaris, you can check SAR from previous days using "sar -f /var/adm/sa/sa30 " ( last 30th )

If we like to see the detail from "ps" command:
solaris: /usr/ucb/ps -awwx , or, use pargs, but need sudo to that user -[..]ow-long-argument-of-running-process.html
linux: ps -aef --cols [XXXX]

prstat - use prstat instead of ps in solaris
How to check physical memory on solaris? /usr/sbin/prtconf | grep Memory

Other forensics tools -[..]om/computer-forensics-software-intro.php

Sun monitoring tools collections -[..]try/solaris_performance_monitoring_tools[..]sed-system-monitoring-commands-in-linux/

time, basic unix util -[..]ow-level-discovery-cores-cpus-hard-disk/

10 commands for system monitoring -[..]1/linux-performance-analysis-in-60s.html

(google search) (amazon search)

Wed Dec 30 15:09:53 HKT 2015 From /weblog/unix


Embedded Programming with the GNU Toolchain -

(google search) (amazon search)

Mon Nov 09 15:34:36 HKT 2015 From /weblog/unix


Look like a nice tools provide more information for performance issue -

Check CPU problem with prstat[..]stat-solaris-pinpoint-high-cpu-java.html[..]stat-linux-how-to-pinpoint-high-cpu.html[..]prstat-aix-how-to-pinpoint-high-cpu.html

If we like to trace process detail flow, system call and file opened -

-l Includes the id of the responsible lightweight process
(LWP) with each line of trace output. If -f is also
specified, both the process-id and the LWP-id are

If the process hang and we like to check why, can try pstack -[..]=pstack&sourceid=opera&ie=utf-8&oe=utf-8

Check where is the core dump to -[..]s/browse_thread/thread/9f46dfb277c72cc8#

Use gcore to generate coredump for running process -

If we like to check the which process id using which port, we can use lsof or "pfiles /proc/*"
1) save the output of "pfiles /proc/*" to a file
2) grep the file for that port, then find the PID, e.g., say here is the part of the output:

5859: java -Djava.awt.headless=true -DX_R_BRGD -Xms512M -Xmx1024M
Current rlimit: 1024 file descriptors
0: S_IFCHR mode:0666 dev:85,0 ino:980953 uid:0 gid:3 rdev:13,2

....... ( 125 FD after) ......

125: S_IFSOCK mode:0666 dev:313,0 ino:40404 uid:0 gid:0 size:0
sockname: AF_INET port: 18075
peername: AF_INET port: 41792

Then PID 5859 is the process working with the port 18075, however, it only work for process which is viewable according to your permission setting

Another probably easier way, and provide more information, is using
lsof -i :[port]

If we like to check the which process id using which file, we can use fuser -

Check host and port that any process accessing using lsof -[..]-are-accessing-internet-on-unix-machine/

How do I monitor or view the thread count of a certain process - ps -o pid,comm,user,thcount -p -[..]thread-count-of-a-certain-process-on-aix

Examples of using strace, use -e to trace function call is useful -

The nm commands provides information on the symbols being used in an object file or executable file. -

Use pmap -s -x to look at what's in the virtual address space of yr process and what portion thereof is resident in physical memory.

(google search) (amazon search)

Thu Aug 20 23:40:08 HKT 2015 From /weblog/unix


Unix timezone -[..]isdk400pubs/ReferenceGuide/timezone.html

(google search) (amazon search)

Sat May 02 13:23:20 HKT 2015 From /weblog/unix


Introduction of 10 useful commend: objdump, file, od, xxd, strings, nm, time, ps, fuser, strace, ctags

Install a Visual OS server at Linux , may be useful later -[..]-feisty-fawn-the-complete-newbies-guide/

Hello world of driver development -

Commands cheat sheet -

Tutorial of build various servers -

# !!- run the previous command.
# !ls - run the last command that started with ‘ls’.
# !ls:p - This will display the command instead of running it.
# !$ - run the last word of the previous command. (for substitutions)
# !$:p - Instead of running the last word of the previous command this will print it out.
# !* run the previous command without the first word. (for substitutions)
# !*:p - print, again

Tutorial of system logging under unix -[..]mmon-linux-log-files-name-and-usage.html

(google search) (amazon search)

Sat Jan 24 09:05:28 HKT 2015 From /weblog/unix


Use GIMP for serverside scripting:

Call GIMP from java -[..]anic?entry=calling_gimp_from_within_java

Linux Music workflow -[..]rom-mac-os-x-to-ubuntu-with-kim-cascone/

Write / Create Image from DVD disc -

A simple paint program -[..]inta-lightweight-paint-app-that-has.html

(google search) (amazon search)

Mon Jan 12 16:39:36 HKT 2015 From /weblog/unix

X 7.4 need Option "AllowEmptyInput" "off" in Section "ServerFlags". Otherwise mouse and keyboard will not working

And some driver program, like nvidia, doesn't aware of that, yet. So after recompilation, mouse and keyboard are not working.... and need to update the config manually.

Good reasons, showing how easy to manual GUI under unix desktop -

Allow you mstsc to unix -[..]-remote-desktop-protocol-rdp-server.html

(google search) (amazon search)

Thu Feb 06 17:54:50 HKT 2014 From /weblog/unix


Reference of linux proc file system -[..]-power-of-proc-file-system-in-linux.html

Power save tool -[..]er-saving-package-for-linux-systems.html

Disk usage report -[..]usage-and-availability-report-tools.html

USB info at /var/log/messages

Oct 20 03:56:14 carfield kernel: [ 3568.521099] usb 1-6: configuration #1 chosen from 1 choice
Oct 20 03:56:14 carfield kernel: [ 3568.557038] Initializing USB Mass Storage driver...
Oct 20 03:56:14 carfield kernel: [ 3568.557228] scsi4 : SCSI emulation for USB Mass Storage devices
Oct 20 03:56:14 carfield kernel: [ 3568.557407] usbcore: registered new interface driver usb-storage
Oct 20 03:56:14 carfield kernel: [ 3568.557429] USB Mass Storage support registered.
Oct 20 03:56:19 carfield kernel: [ 3573.557428] scsi 4:0:0:0: Direct-Access BUFFALO USB Flash Disk 4000 PQ: 0 ANSI: 0 CCS
Oct 20 03:56:19 carfield kernel: [ 3573.559904] sd 4:0:0:0: [sdc] 7864320 512-byte hardware sectors: (4.02 GB/3.75 GiB)
Oct 20 03:56:19 carfield kernel: [ 3573.560766] sd 4:0:0:0: [sdc] Write Protect is off
Oct 20 03:56:19 carfield kernel: [ 3573.562913] sd 4:0:0:0: [sdc] 7864320 512-byte hardware sectors: (4.02 GB/3.75 GiB)
Oct 20 03:56:19 carfield kernel: [ 3573.563765] sd 4:0:0:0: [sdc] Write Protect is off
Oct 20 03:56:19 carfield kernel: [ 3573.563772] sdc: sdc1

(google search) (amazon search)