Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts

Sunday, May 13, 2012

Strategy to handle worker threads in php and create daemon

If you are working on a project which requires fair amount of background work to be done, then you must have seen the requirement of keeping background threads separate. Unfortunately, there is no explicit way of threading in php (there are a number of way around to do this, but those to me look like hacks rather a supported feature). Such needs are normally seen in a project which requires some work to be done which is not needed immediately. For example, if a user uploads a picture and you want to crop that picture, or, you want to send some sort of notification email etc. This is a classical case of having a worker thread doing all these tasks in parallel. There are a number of open source libraries available which take care of such things e.g. https://github.com/pda/pheanstalk . But if you want to do this yourself, then here is a simple strategy.

  1. Create a queue in memcache or add the jobs in some table in DB
  2. Create a php script which takes a job out from memcache queue to from the table in DB and executes the job
  3. Run the php script as a background thread.

 First two points are fairly simple. So, lets talk about third point now. How would you run the php script as a background thread. There are two ways in linux to do this. One is to run the script as a cron. To know how to run a script as a cron job, refer to my post here => http://saadnawaz.blogspot.com/2012/05/create-cron-job-in-linux.html. Now, if you think that your job is something which can run after 1 minute, then simply write the php script and add entry in the crontab. Though, in most of cases, 1 minute is just a bit too much delay. So, the next option is creating a daemon. Daemon is a process which runs in the background. What you should be doing is running the worker php script as a daemon. Follow the following steps to do that:(this is just one way of creating a daemon, there are other ways as well, but this seems simplest to me)

1. Create a shell script which executes the php script file, which is gonna be something like:

while true
do
     php /pathtofile/filename.php
     sleep 5
done

What this shell script is doing is fairly straight forward. You have an infinite loop which runs a php script file, and after than sleeps for 5 seconds. So, basically you are achieving something like cron, but you are executing the script every 5 seconds (which cron can't do, since, the granularity is upto minutes).

Since, this is treated as an executable, therefore, make sure you have given appropriate rights on the script and php worker file.

2. Run the shell script as a daemon:

/completePathToFile/shell.sh &

The "&" at the end runs this as a daemon.

Now, keep in mind, it's always better to use an existing solution rather reinventing the wheel. But, if your problem is not big enough to use open source solutions, then the above strategy should get you started.

Create a cron job in linux

Cron is a system daemon used to execute a specific task at specific time intervals. In linux, there is a simple text file called crontab which contains a list of commands to be run at specific times. In order to add an entry in crontab, first you need to open the crontab file. In order to open the crontab file in edit mode, use the following command

crontab -e

Keep in mind you need to edit crontab file with administrator privileges. So, you might have to use sudo crontab -e.

You can add an entry into the crontab file using the following format:

minute (0-59) hour (0-23, 0 = midnight) day (1-31) month (1-12) weekday (0-6, 0 = Sunday) command 

Notice, there is a space between each section. There are total 5 sections, with each section separated by a space. You can use a * (asterisk) for a section if you want to run the command for every instance of that section. A sample crontab instruction is:

10 08 4 2 1 /somedirectorypath/desiredCommand

The above will execute /somedirectorypath/desiredCommand at 8:10am on February 4th plus every Monday in February. If you want to run a command every 5 minutes, use the following instruction:

*/5 * * * * /somedirectorypath/desiredCommand

The first section i.e. */5 is going to be true for a minute which is divisible by 5. Hence, the command will execute every 5th minute. If you want to run a command every minute, then simply have all sections as *.


There is one limitation to the cron job though, the lowest level of granularity is minute. So, you can't run a command after some specific seconds.

Thursday, April 28, 2011

Monitor and restart services in linux

Working on a project lately, I wanted to achieve a simple goal that if some particular service is not running, then start it automatically. Normally, such things are done through shell/perl scripts which are executed as cron job. There are some other solutions for this thing as well i.e. www.pingdom.com though this is a paid service. Looking further, I found a very nice open source service for this thing called MONIT.. Following link explains all the steps for installation and some usage examples as well

http://www.cyberciti.biz/tips/howto-monitor-and-restart-linux-unix-service.html

Thursday, February 10, 2011

Thrift installation on ubuntu (for usage with ruby)

(This tutorial is tested on thrift 0.5.0)

I was working on thrift (http://thrift.apache.org/), found a couple of issues during the installation and had to look at a couple of links to figure out the correct method.. so i thought to sum it up here.
  1. Download thrift (http://thrift.apache.org/)
  2. Extract the downloaded archive and go to the folder through terminal
  3. In order to install thrift, you'd need following dependencies: (as mentioned on the thrift wiki page)
    1.  g++ 3.3.5+
    2. boost 1.33.1+ (1.34.0 for building all tests)
    3. Runtime libraries for lex and yacc might be needed for the compiler
    4. The dependencies for each language are different which you can see at the link => http://wiki.apache.org/thrift/ThriftRequirements
    5. GNU build tools: autoconf 2.59+ (2.60+ recommended), automake 1.9+, libtool
    6. For C, you'll also might need to install "libglib2.0-dev"
  4. In order to install above dependencies, you can use the following command
    sudo apt-get install g++ libboost-dev libevent-dev automake pkg-config libtool flex bison
  5. You might need to bootstrap first, for that run ./bootstrap.sh (assuming you are in the directory of extracted thrift)
  6. Now, run the following commands (in sequence) to configure and install thrift
    1. ./configure
    2. make
    3. sudo make install
  7. Now, you are good to go with thrift. The final piece missing is the thrift gem. You can install that using command => sudo gem install thrift

Hope you find the post helpful. I will try to post example of thrift using ruby soon :)