Friday, April 27, 2012

Sending Google Docs Document as Email and Viewing in Gmail

Our system uses Google Docs to create and edit documents, which is used as email templates. When we need to send email, we fetch the Google Docs document as HTML (using gdata PHP/Python libraries). And this HTML contents is substituted with data from the database (aka merge fields -- I use smarty to accomplish) and this HTML contents is sent as html email.

All this is fine, but when the mail sent is viewed in Gmail (or probably Hotmail and other web-based email solutions), most HTML decorations are lost and the email is seen in Gmail mostly as text. However, when the same email is viewed using Thunderbird, the email retains all the HTML styles, decorations etc.

After some googling, the following was clear:

  1. Gmail (and others) do not like CSS in the header, but they are somewhat okay with inline styles.
  2. Some of the styles (like font-weight: bold, color within style tags ) etc are stripped out in Gmail loosing the HTML styles for their equivalents. There could be many such styles stripped out, I still don't know the logic of stripping styles like font-weight - but it happens.
  3. Instead of such font-weight styles, they seem to like bold tags (<b>...</b>) etc.
  4. Instead of color styles, they seem to like old font tags (<font color="#xxxxxx">....)
  5. Google Docs sets up styles for a link (anchor) as color:inherit and text-decoration:inherit, which should rather not be there and the default behavior is preferred.
This is the minimum that I needed to get off, there could be various other styles and tags, that need to handled to show in Gmail properly.

The following script in PHP is helpful to move styles inline and then replace some styles with specific tags. 


function move_css_inline($text)
{
    $html = $text;
    if(!preg_match("/\

Friday, January 20, 2012

Add Javascript to Existing PDF files (Python)

There are several tools on the net, that helps you to add javascript to new PDF files - but I could not find any tool to add Javascript to existing PDF files.

Here is a python script that accomplishes that. For this you need to install the following python package (which is a modified pyPDF - found at http://pybrary.net/pyPdf/). The modified pyPDF is found at http://goo.gl/sJcMO. Or you could simply unzip the modified pyPDF in your current directory.


Help text and Usage for this script:
$ python addjs2pdf.py 
Usage: addjs2pdf.py [options] in-pdf-file out-pdf-file

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -j JAVASCRIPT, --javascript=JAVASCRIPT
                        javascript to embed (default embedded JavaScript is
                        app.alert messagebox)
  -f JAVASCRIPTFILE, --javascriptfile=JAVASCRIPTFILE
                        javascript file to embed

  add-js-to-pdf, use it to add embedded JavaScript to a PDF document that will execute automatically when the document is opened
  Based on modified pyPDF http://pybrary.net/pyPdf/ and inspiration from https://DidierStevens.com

$

Here is the addjstopdf.py:
from pyPdf import PdfFileWriter, PdfFileReader

import optparse

def Main():
    """add-js-to-pdf, use it to add embedded JavaScript to a PDF document that will execute automatically when the document is opened
    """

    parser = optparse.OptionParser(usage='usage: %prog [options] in-pdf-file out-pdf-file', version='%prog 0.1')
    parser.add_option('-j', '--javascript', help='javascript to embed (default embedded JavaScript is app.alert messagebox)')
    parser.add_option('-f', '--javascriptfile', help='javascript file to embed')
    (options, args) = parser.parse_args()

    if len(args) != 2:
        parser.print_help()
        print ''
        print '  add-js-to-pdf, use it to add embedded JavaScript to a PDF document that will execute automatically when the document is opened'
        print '  Based on modified pyPDF http://pybrary.net/pyPdf/ and inspiration from https://DidierStevens.com'
        print ''
        return

    input1 = PdfFileReader(file(args[0], "rb"))
    output = PdfFileWriter()
        
    pages = input1.getNumPages()
    for p in range(pages):
        output.addPage(input1.getPage(p))
    if options.javascript == None and options.javascriptfile == None:
            javascript = """app.alert({cMsg: 'Hello from PDF JavaScript', cTitle: 'Testing PDF JavaScript', nIcon: 3});"""
    elif options.javascript != None:
            javascript = options.javascript
    else:
        try:
            fileJavasScript = open(options.javascriptfile, 'rb')
        except:
            print "error opening file %s" % options.javascriptfile
            return

        try:
            javascript = fileJavasScript.read()
        except:
            print "error reading file %s" % options.javascriptfile
            return
        finally:
            fileJavasScript.close()

    output.addJS(javascript)
    outputStream = file(args[1], "wb")
    output.write(outputStream)
    outputStream.close()

if __name__ == '__main__':
    Main()

Thursday, January 5, 2012

Setup Ubuntu server for PHP-MongoDB on Amazon AWS

This is a record of all the steps that I did to setup a Ubuntu server on Amazon AWS, primarily for PHP-Mongo setup with Apache2. I may consider nginx and fastcgi sometime later, but for now, will go ahead with Apache2.

AMI

AMI used is alestic-git-server-ubuntu-11.10-oneiric-amd64-20111017 (ami-dd73bfb4)


Packages Installed

Base Packages:

sudo apt-get -y install language-pack-en-base
sudo apt-get -y install unzip
sudo apt-get -y install python-setuptools
sudo apt-get install build-essential

PHP and Apache2:

sudo apt-get install apache2
sudo apt-get install php5
sudo apt-get install libapache2-mod-php5 php5-curl php5-gd php5-imap
sudo apt-get install php-pear php5-dev






MongoDB Install:

Steps as per http://www.mongodb.org/display/DOCS/Ubuntu+and+Debian+packages

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

Append the following line to /etc/apt/sources.list:
deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen

sudo apt-get update
sudo apt-get install mongodb-10gen

PHP and Python Drivers for MongoDB:

sudo pecl install mongo
Add "extension=mongo.so" in /etc/php5/apache2/php.ini and /etc/php5/cli/php.ini


sudo apt-get install python-dev
sudo easy_install pymongo

SunJava Install:

As per this link:

sudo apt-get install python-software-properties
sudo add-apt-repository ppa:ferramroberto/java
sudo apt-get update
sudo apt-get install sun-java6-jdk

SVN and other Tools:

sudo apt-get install subversion
sudo apt-get install libapache2-svn
sudo apt-get install mpack              # For MIME Pack/unpack


Specific Packages/Modules for my App:


sudo a2enmod rewrite
sudo a2enmod actions
sudo pear install --alldeps Mail
sudo pear channel-discover pear.pearplex.net
sudo pear install pearplex/PHPExcel

wkhtmltopdf:
Install http://code.google.com/p/wkhtmltopdf/  including wkhtmltopdf, wkhtmltoimage and libwkhtmltox  to /usr/local/bin, /usr/local/lib and /usr/local/include

PHP and Python bindings for libwkhtmltox:
git clone git://github.com/mreiferson/php-wkhtmltox.git   and follow README
git clone git://github.com/mreiferson/py-wkhtmltox.git    and sudo python setup.py install


Misc:

sudo apt-get -y install fontconfig xfs
sudo pecl install uuid




Monday, December 5, 2011

Batch Convert Multipage CR2 to JPG

I landed up with a bunch of CR2 files (Canon Digital Raw Images from camera), which I needed to convert to JPG. On Ubuntu, I was able to view the CR2 format file using the installed Image Viewer and Gimp, after installing ufraw package.

However, the "convert" tool of ImageMagick was not able to convert to jpeg in a proper way - see the attached example converted images. This was due to the fact that the CR2 image has multiple pages and the "convert" tool is unable to understand this properly. (CR2 images with single page was successfully converted to jpg).

Gimp was able to open the CR2 file, but I had to specifically mention "Page1" in the "Import from Tiff"  dialog box. However, since I have to convert several CR2 files, manually opening them and converting was not an option. And I was also unable to figure out how to specify "Page1" in the gimp scripts (.scm files) or the Python-fu (pdb.file_load_tiff) scripts.

Finally, I was able to use the "convert" tool itself to convert to JPEG, but simply renaming the CR2 file as a TIFF file and using the same convert command like
convert renamed_cr2.tiff image.jpg
And that's it! It showed some errors about reading Tiff file (as it is not really a TIFF file) - but was able to successfully convert it.

Thursday, November 17, 2011

Killgrep - Kill a process by part of its name

We know the command kill that can kill a process by its id, and we know killall that can stop the processes by its full name. What if we need to kill a process by some part of its name and/or arguments. A simple killgrep script can do and here is how it is done:


#!/bin/sh

# killgrep: Usage: killgrep server.py 
# will kill the process running like "python server.py"

echo $1

if [ -z "$1" ]; then
 echo "killgrep: "
 exit 1
fi

PROC=`ps auxwww | grep "$1" | grep -v grep | wc -l`
if [ $PROC = "0" ]; then
 echo "killgrep: None killed"
 exit 1
fi

PROC=`ps auxwww | grep "$1" | grep -v grep | perl -e '$_=<>; @u = split; print $u[1]' -`
if [ ! -z $PROC ]; then
 echo "killgrep: Stopping process $PROC"
 kill $PROC
fi

Saturday, November 12, 2011

How to add a custom launcher in Ubuntu Unity Panels? (11.04+)

Prior to Ubuntu 11.04 (Natty Narhwal), you had Gnome Panel and it was easy to add a new custom launcher or an application to your Panel. But with Natty, by default, Ubuntu has the Unity interface which is very different from the classic Gnome Panel.

Now, you can easily create a "shortcut" on the Unity sidebar by opening the list of applications and then dragging a particular app onto the sidebar. But there is no obvious way to create a custom launcher - an application you have installed manually or a script that you have written, and get it placed on the sidebar. In my case, it was Thunderbird 8 which I just downloaded and placed it in /usr/local/thunderbird and I need to launch that instead of the regular packaged thunderbird.

Here is a simple method of how you can accomplish that:


  1. Right click on the Desktop. Click on "Create Launcher"
  2. Provide the name and path of the Launcher application and click OK to create the Launcher.
  3. Then just drag that launcher from the desktop to the sidebar panel! That's it!

Thursday, February 12, 2009

Windows XP Install CD won't boot

Recently, I corrupted my Windows partition by wrongly installing grub on the Windows partition - rather than on the MBR. Either this or due to some reason, my Windows partition got corrupted and was not booting.

In order to install Windows XP, I tried booting with WinXP Install CD. After going through few screens on booting from the Install CD, the install process would forever hang. I had no ideas, why this was behaving so. I even tried booting the Install CD from within my Linux using qemu, with the partition to be installed as the WinXP hard-drive partition. Same problem, Install CD would hang.

By chance, I discovered a trick to resolve this problem. Before booting from the XP Install CD, I went to BIOS setup and disabled the IDE hard disk. XP Install CD did not hang now and went ahead with the Install process. Since the later stage of XP Installer has its own drivers to discover the IDE hard disk, the hard disk was discovered and it installed XP on the hard drive. It looks like the earlier stages of XP install CD, tries to read the existing Windows partitions and got stuck, as the partition was unreadable. After the installation, I enabled the IDE hard drive in BIOS.

Ofcourse, having understood the problem, I realized later that I could have formatted the Windows partition from within my Linux - but if you don't have a Linux partition or a Linux install CD, disabling the IDE hard disk in the BIOS is probably a easy solution.

As a matter of coincidence, in the next few days, my cousin reported that his Windows registry is corrupted and Windows won't boot. He also tried reinstalling Windows XP, but faced the same problem! Once again, the disabling of hard disk in BIOS was the trick that helped him move forward!