Unix
2007-12-28
Folding Shell Style Comments with Vim
I often experience with people new to Unix or even people who have been using it for a while, that they tend to brush off vi/emacs editors quickly and then use simpler editors like pico, nano, gedit and the like.
They then never see why a lot of old time Unix system administrators and developers use these tools. vi for example was designed to edit files over a very slow connection (300bps). It's still very useful now, as lag still exists. It's very painful for me to see young developers use the mouse and scroll up, scroll down, when without lifting your hands from the keyboard you can jump to different functions, mark locations and jump back.
If you need to comment out some lines over a laggy ssh connection you can simple type :4,12s/^/#/ instead of type # and using cursor keys over the next 8 lines.
Speaking of comments, here's a common scenario. Sometimes configuration files are huge, because they're well commented with examples. You want to edit it, but sometimes you just want to see your modifications and only comments for the section you're looking at and not scroll through everything. You can hide the # comments with vim's folding feature.
So let's define a function to do this in our ~/.vimrc:
function! FoldShellComments()
let &foldexpr = 'getline(v:lnum)[0]=="#"'
g/.*/ if foldlevel(line(".")) > 0 | s/$/ !!!/ | endif
set foldmethod=expr
endfunction
We then want to call this up easily so we define a user command:
command! -nargs=0 FoldShellComments :call FoldShellComments()
With this whenever we want to fold commented lines, we simple type :FoldShellComments This will tab complete by the way, you just need to type :Fo<tab>
That huge squid.conf file now looks like this:

2007-12-25
Reducing complexity and resource usage for Zope front-end
Keeping things simple is important. Keeping things simple however does not necessarily mean things are dumbed down. There is a Unix mantra, which is to do one thing and to do it well. For example take bzip2, does it's job well which is to compress things, it doesn't do anything else. Yes it has a lot of options, but they're related to compressing data.
Sometimes when programs do too many things, they end up being bloated and complicated to set up. So when archiving http://www.apdip.net to a virtual server, it was good opportunity to simplify things, reduce resource usage, but not reduce functionality or performance.
So I've reduced the backend setup to a chain of:
squid -> pound -> zope
-> cherokee
- Squid here does what it does best, which is to cache requests
- Pound to load balance between application servers and httpd server
- Cherokee a lightweight httpd server
Except for Squid, the others do their job really well with simple small configuration files. With squid, by keeping it's role to strictly as a caching server, the configuration is also simplified and less prone to errors.
Proper articles later, but hopefully some tips here will help you on your way.
Squid
Start here: http://wiki.squid-cache.org/Squid_Faq/ReverseProxy
Set our Squid to listen to port 80 and also deal with named vhost requests
http_port 80 accel vhost
Set Squid to go to Pound to manage rediretions and load balance the backend services
cache_peer 127.0.0.1 parent 81 0 originserver default
We needed to deal logs for different vhosts, and that was not too difficult. First we set the ACLs
acl apdip dstdomain www.apdip.net http_access allow apdip acl stats dstdomain stats.apdip.net http_access allow stats
Then we split the logs by referring to the acls, so that each site has their own logs
access_log /var/log/httpd/www.apdip.net/access.log combined apdip access_log /var/log/httpd/stats.apdip.net/access.log combined stats
Pound
We then configure Pound to deal with the backend services. Just man pound, the man page is all you need to setup different priorities of servers, redirect requests to different servers, set time outs on backend pools etc. It's quite simple, because well.. that's what Pound is supposed to do, and do well.
Here it's listening on 81, and redirecting the right requests to the right servers.
User "www"
Group "www"
Client 300
ListenHTTP
Address 127.0.0.1
Port 81
End
Service
HeadRequire "Host:.*stats.apdip.net.*"
Backend
TimeOut 120
Address 127.0.0.1
Port 8081
End
End
Service
Backend
TimeOut 120
Address 127.0.0.1
Port 8080
End
End
Cherokee

Love it.. small and simple configuration files. vhosts, cgi etc. snap to set up. The configs are even set up debian style (sites-enabled), brilliant. It's also uses much less memory than Apache and since we don't use any of the modules, php support etc, it makes much more sense. I got introduced to this by Alvaro some time back, since he wrote it, I trust it's as good as he says it is. :)
I won't paste the config files here, you can check them out, but all I can say is that it was simple, just go do the documention on the website to see examples for common uses.
2007-12-21
apdip.net on gambit
Just moved apdip.net from servers in Bangkok to Inigo's server in Kuala Lumpur, remotely of course.
These kind of tasks are quite easy with standard unix tools such as ssh and tar. Assuming of course you're familiar with them (hint LPI Level 1 objectives are important). Setting it up was made even easier thought the use of FreeBSD jails, which allows me to host it on a virtual server which shares some of the host directories, but has it's own ip, own configuration and own third party (ports) software installed. This method shares kernel also, so it's very efficient. This allows me to make very efficient use of my resources. As an SME it's very important to keep costs low, and if servers are able to do more, then it's better for the bottom line.Unlike proprietary software, there are no silly things like registration keys, shared registry tied to physical server and other mechanisms that make it difficult to copy and move things. Every single software I need to duplicate it is available for free without any feature restrictions. Furthermore software versions are also not usually tied to specific OS releases. Meaning that I can run python 2.3 and zope 2.7 on FreeBSD 6.2 while the older server was on FreeBSD 5.3. I could replicate it on different Linux distributions too. All these kind of freedoms for the user, end up making things look amazingly easy to do compared to doing it on a platform such as Microsoft. Latest Plone 3.0 will run on FreeBSD 5,6 and upcoming 7, Fedora, Centos, Suse and even Windows. Contrast this to Sharepoint 2007 which requires Windows 2003. Running Windows 2000? Tough.. pay for more licenses. Already have a Linux server? Tough, pay up for more licensing fees.
Eventually apdip.net will be moved to IOSN servers. Either way over 7 years worth of ICT4D material will continue be available with the same urls.
2007-10-24
Auto reply the Unix way
Don't you just hate those auto-reply emails to mailing lists and such? Auto-replies are still important though, especially if you are on extended vacation leave and want people who are trying to contact you to be aware of it.
The Unix way to do this is through the vacation(1) command and formail or in my case procmail.
vacation(1) is smart in the sense that it only provides replies once, and keeps track of this in a .vacation.db file in your home directory. Your auto reply message is a plain text file also in your home directory as .vacation.msg
From: kaeru@inigo-tech.com (Khairil Yusof) Subject: At the beach suckers! Precedence: bulk I asked that there would be no salt in the margerita.. but but there is salt in it. I could burn this resort down you know.. sir.. sir..
The bulk header will be respected by mailing lists and the auto reply will not be sent.
For your procmail recipe, just add (check your paths) and username.
:0 c | /usr/bin/vacation username
The c will tell it to continue and go on to other filters after it so you will have a copy of the original email, or forward it etc.
When you're back, just remove the files and procmail entry.
2007-08-28
loading multiple files that match a string with vim
loading multiple files that match a string with vim using command line unix tools
Using the ever useful grep, with -l option, it outputs only the matching filenames:
grep -rl --exclude '*.svn*' CMFCore *
vim loads multiples file in one line separated by spaces. So I need to join the multiple lines into one, separated by space. Here we have the paste command. For each line, put a space as the separator:
paste -s -d " "
Finally I wanted vim to load this in GUI mode with tabs, so we put it together:
gvim -p $(grep -rl --exclude '*.svn*' CMFCore * | paste -s -d " " -)
- the part inside the $( ) is the output of the command.
- using pipes | the output of grep (the list of files) is read by paste
- the - in paste is telling it to input the stdout output of grep
I then quickly have my editor setup for work,

