Sunday, March 27, 2016

kswapd0 high cpu consumption

OS: Ubuntu 15.10
Linux Kernel: 4.2.0-25-generic

Running MongoDB gets the CPU consumption high, with the culprit being kswapd0 process


add this to /etc/rc.local:
echo madvise > /sys/kernel/mm/transparent_hugepage/enabled

As per this URL:

The next attempt:

echo 3 > /proc/sys/vm/drop_caches

One more attempt:

worked around by commenting out line 2 in /lib/udev/rules.d/40-vm-hotadd.rules.

Friday, April 27, 2012

Sending Google Docs Document as Email and Viewing in Gmail

Our system uses Google Docs to create and edit documents, which is used as email templates. When we need to send email, we fetch the Google Docs document as HTML (using gdata PHP/Python libraries). And this HTML contents is substituted with data from the database (aka merge fields -- I use smarty to accomplish) and this HTML contents is sent as html email.

All this is fine, but when the mail sent is viewed in Gmail (or probably Hotmail and other web-based email solutions), most HTML decorations are lost and the email is seen in Gmail mostly as text. However, when the same email is viewed using Thunderbird, the email retains all the HTML styles, decorations etc.

After some googling, the following was clear:

  1. Gmail (and others) do not like CSS in the header, but they are somewhat okay with inline styles.
  2. Some of the styles (like font-weight: bold, color within style tags ) etc are stripped out in Gmail loosing the HTML styles for their equivalents. There could be many such styles stripped out, I still don't know the logic of stripping styles like font-weight - but it happens.
  3. Instead of such font-weight styles, they seem to like bold tags (<b>...</b>) etc.
  4. Instead of color styles, they seem to like old font tags (<font color="#xxxxxx">....)
  5. Google Docs sets up styles for a link (anchor) as color:inherit and text-decoration:inherit, which should rather not be there and the default behavior is preferred.
This is the minimum that I needed to get off, there could be various other styles and tags, that need to handled to show in Gmail properly.

The following script in PHP is helpful to move styles inline and then replace some styles with specific tags. 

function move_css_inline($text)
    $html = $text;

Friday, January 20, 2012

Add Javascript to Existing PDF files (Python)

There are several tools on the net, that helps you to add javascript to new PDF files - but I could not find any tool to add Javascript to existing PDF files.

Here is a python script that accomplishes that. For this you need to install the following python package (which is a modified pyPDF - found at The modified pyPDF is found at Or you could simply unzip the modified pyPDF in your current directory.

Help text and Usage for this script:
$ python 
Usage: [options] in-pdf-file out-pdf-file

  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -j JAVASCRIPT, --javascript=JAVASCRIPT
                        javascript to embed (default embedded JavaScript is
                        app.alert messagebox)
                        javascript file to embed

  add-js-to-pdf, use it to add embedded JavaScript to a PDF document that will execute automatically when the document is opened
  Based on modified pyPDF and inspiration from


Here is the
from pyPdf import PdfFileWriter, PdfFileReader

import optparse

def Main():
    """add-js-to-pdf, use it to add embedded JavaScript to a PDF document that will execute automatically when the document is opened

    parser = optparse.OptionParser(usage='usage: %prog [options] in-pdf-file out-pdf-file', version='%prog 0.1')
    parser.add_option('-j', '--javascript', help='javascript to embed (default embedded JavaScript is app.alert messagebox)')
    parser.add_option('-f', '--javascriptfile', help='javascript file to embed')
    (options, args) = parser.parse_args()

    if len(args) != 2:
        print ''
        print '  add-js-to-pdf, use it to add embedded JavaScript to a PDF document that will execute automatically when the document is opened'
        print '  Based on modified pyPDF and inspiration from'
        print ''

    input1 = PdfFileReader(file(args[0], "rb"))
    output = PdfFileWriter()
    pages = input1.getNumPages()
    for p in range(pages):
    if options.javascript == None and options.javascriptfile == None:
            javascript = """app.alert({cMsg: 'Hello from PDF JavaScript', cTitle: 'Testing PDF JavaScript', nIcon: 3});"""
    elif options.javascript != None:
            javascript = options.javascript
            fileJavasScript = open(options.javascriptfile, 'rb')
            print "error opening file %s" % options.javascriptfile

            javascript =
            print "error reading file %s" % options.javascriptfile

    outputStream = file(args[1], "wb")

if __name__ == '__main__':

Thursday, January 5, 2012

Setup Ubuntu server for PHP-MongoDB on Amazon AWS

This is a record of all the steps that I did to setup a Ubuntu server on Amazon AWS, primarily for PHP-Mongo setup with Apache2. I may consider nginx and fastcgi sometime later, but for now, will go ahead with Apache2.


AMI used is alestic-git-server-ubuntu-11.10-oneiric-amd64-20111017 (ami-dd73bfb4)

Packages Installed

Base Packages:

sudo apt-get -y install language-pack-en-base
sudo apt-get -y install unzip
sudo apt-get -y install python-setuptools
sudo apt-get install build-essential

PHP and Apache2:

sudo apt-get install apache2
sudo apt-get install php5
sudo apt-get install libapache2-mod-php5 php5-curl php5-gd php5-imap
sudo apt-get install php-pear php5-dev

MongoDB Install:

Steps as per

sudo apt-key adv --keyserver --recv 7F0CEB10

Append the following line to /etc/apt/sources.list:
deb dist 10gen

sudo apt-get update
sudo apt-get install mongodb-10gen

Update (April 28th 2014): This has changed slightly. The updated steps below:

Steps as per:

sudo apt-key adv --keyserver hkp:// --recv 7F0CEB10
echo 'deb dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list

sudo apt-get update
sudo apt-get install mongodb-org

PHP and Python Drivers for MongoDB:

sudo pecl install mongo
Add "" in /etc/php5/apache2/php.ini and /etc/php5/cli/php.ini

Update (April 28th 2014): The above adding to php.ini is changed for Ubuntu 14.04

echo '' | sudo tee /etc/php5/mods-available/mongo.ini
sudo php5enmod mongo

sudo apt-get install python-dev
sudo easy_install pymongo

SunJava Install:

As per this link:

sudo apt-get install python-software-properties
sudo add-apt-repository ppa:ferramroberto/java
sudo apt-get update
sudo apt-get install sun-java6-jdk

Update (April 28th 2014): The above has changed for Ubuntu 14.04

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer

SVN and other Tools:

sudo apt-get install subversion
sudo apt-get install libapache2-svn
sudo apt-get install git
sudo apt-get install mpack              # For MIME Pack/unpack

Specific Packages/Modules for my App:

sudo a2enmod rewrite
sudo a2enmod actions
sudo pear install --alldeps Mail
sudo pear channel-discover
sudo pear install pearplex/PHPExcel

Install  including wkhtmltopdf, wkhtmltoimage and libwkhtmltox  to /usr/local/bin, /usr/local/lib and /usr/local/include

Also install on Ubuntu libxrender, needed for wkhtmltopdf

sudo apt-get install libxrender-dev

PHP and Python bindings for libwkhtmltox:
git clone git://   and follow README
git clone git://    and sudo python install


sudo apt-get -y install fontconfig xfs

sudo apt-get install libuuid1 uuid-dev uuid-runtime
sudo pecl install uuid

echo '' | sudo tee /etc/php5/mods-available/uuid.ini
sudo php5enmod uuid


sudo apt-get install memcached php5-memcached
sudo apt-get install mcrypt php5-mcrypt
sudo php5enmod mcrypt

sudo apt-get install sqlite3 php5-sqlite
sudo apt-get install mysql-server php5-mysql

sudo apt-get install imagemagick

# Install Font?
sudo mkdir /usr/share/fonts/truetype/customttf
sudo cp /home/live/ereceipts/server/fonts/MTCORSVA.TTF /usr/share/fonts/truetype/customttf
sudo fc-cache -f -v

Other Edits:

1. php.ini in /etc/php5/apache2/php.ini (and the cli version)

error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT & ~E_NOTICE
date.timezone = Asia/Kolkata
session.gc_maxlifetime = 86400

Monday, December 5, 2011

Batch Convert Multipage CR2 to JPG

I landed up with a bunch of CR2 files (Canon Digital Raw Images from camera), which I needed to convert to JPG. On Ubuntu, I was able to view the CR2 format file using the installed Image Viewer and Gimp, after installing ufraw package.

However, the "convert" tool of ImageMagick was not able to convert to jpeg in a proper way - see the attached example converted images. This was due to the fact that the CR2 image has multiple pages and the "convert" tool is unable to understand this properly. (CR2 images with single page was successfully converted to jpg).

Gimp was able to open the CR2 file, but I had to specifically mention "Page1" in the "Import from Tiff"  dialog box. However, since I have to convert several CR2 files, manually opening them and converting was not an option. And I was also unable to figure out how to specify "Page1" in the gimp scripts (.scm files) or the Python-fu (pdb.file_load_tiff) scripts.

Finally, I was able to use the "convert" tool itself to convert to JPEG, but simply renaming the CR2 file as a TIFF file and using the same convert command like
convert renamed_cr2.tiff image.jpg
And that's it! It showed some errors about reading Tiff file (as it is not really a TIFF file) - but was able to successfully convert it.

Thursday, November 17, 2011

Killgrep - Kill a process by part of its name

We know the command kill that can kill a process by its id, and we know killall that can stop the processes by its full name. What if we need to kill a process by some part of its name and/or arguments. A simple killgrep script can do and here is how it is done:


# killgrep: Usage: killgrep 
# will kill the process running like "python"

echo $1

if [ -z "$1" ]; then
 echo "killgrep: "
 exit 1

PROC=`ps auxwww | grep "$1" | grep -v grep | wc -l`
if [ $PROC = "0" ]; then
 echo "killgrep: None killed"
 exit 1

PROC=`ps auxwww | grep "$1" | grep -v grep | perl -e '$_=<>; @u = split; print $u[1]' -`
if [ ! -z $PROC ]; then
 echo "killgrep: Stopping process $PROC"
 kill $PROC

Saturday, November 12, 2011

How to add a custom launcher in Ubuntu Unity Panels? (11.04+)

Prior to Ubuntu 11.04 (Natty Narhwal), you had Gnome Panel and it was easy to add a new custom launcher or an application to your Panel. But with Natty, by default, Ubuntu has the Unity interface which is very different from the classic Gnome Panel.

Now, you can easily create a "shortcut" on the Unity sidebar by opening the list of applications and then dragging a particular app onto the sidebar. But there is no obvious way to create a custom launcher - an application you have installed manually or a script that you have written, and get it placed on the sidebar. In my case, it was Thunderbird 8 which I just downloaded and placed it in /usr/local/thunderbird and I need to launch that instead of the regular packaged thunderbird.

Here is a simple method of how you can accomplish that:

  1. Right click on the Desktop. Click on "Create Launcher"
  2. Provide the name and path of the Launcher application and click OK to create the Launcher.
  3. Then just drag that launcher from the desktop to the sidebar panel! That's it!