Wednesday 12 December 2012

PHP FAQ

How to secure get parameters in php ?

Try using

sending end

$a="some crap";
$key = 'pass';// can use your own key here
$string = " ".$a." ";
$encrypteda = base64_encode(mcrypt_encrypt(MCRYPT_RIJNDAEL_256, md5($key), $string, MCRYPT_MODE_CBC, md5(md5($key))));



receiving end

$key = 'pass';
$rand=($_GET['parameter']);
$rand = str_replace(" ", "+", $rand);
$rand = rtrim(mcrypt_decrypt(MCRYPT_RIJNDAEL_256, md5($key), base64_decode($rand), MCRYPT_MODE_CBC, md5(md5($key))), "\0");
$rand = str_replace(" ", "", $rand);



--------------------------------------------------------------------------------------------------------------------


 How to download multiple files in zipped form ?
Want to download files gmail style ? , well you can do it by using simple php script
we shall be using the following approach

open a ziparchive

add files to the ziparchive (you can add multiple files here )

close it after adding

Now just pass a header information containing the name of zip archive

finally read the archive file


That would be -
a reference to archive (inspired by java i guess!)

$ziparch= new ZipArchive()

open an archive
$ziparch->open($somename);

Then add files by

A loop
{
$ziparch->addFile($filename);
 }

 Close once done
$ziparch->close();

followed by provision of header information indicating the ziparchive name

header("Content-type: application/zip");
    header("Content-Disposition: attachment; filename=$somename");
    header("Pragma: no-cache");
    header("Expires: 0");


finally

readfile("$somename");



--------------------------------------------------------------------------------------------------------------------


How to pass data between php pages ?
If you intend to pass data and yet find the popular post and get method to be incompatible use session .

all you got to do is start session at the beginning of the script

session_start()


Add data to session variables
(say in page 1)

$_SESSION['identifier']=$data;


Retrieve data by
(say in page 2)

$get_data=$_SESSION['identifier'];




--------------------------------------------------------------------------------------------------------------------



How to periodically execute php scripts ?
Without any user pinging the server, you can use cron service to schedule execution of scripts.Check out crontab in your favorite search engine.But there is a catch ,you have to incorporate this service onto the server and in some of the cases it is non trivial.
another method is to use the php's time() which will give the unix time stamp. well You can use just this.

Consider a time() instant   1358185093   ,get the ,say last 5 numbers   or  85093
now in your index.php or in your php script use this

if($last_5_digit == 50000)
{
 execute a periodic script.
}

NOTE : this works good for servers which get heavy traffic,since the users are constantly pinging the server (even when time()(last 5) == 5000)

The best ,easier method which i generally use is to add a time field in a table and observe the current time .Then delete all rows whose time field and the current time difference is beyond a threshold.



------------------------------------------------------------------------------------------------------------------------------

How to ping urls when your server is behind a proxy ?


use

$acontext = array(
    'http' => array(
        'proxy' => 'tcp://proxy_server:port',          // example tcp://10.10.10.10//3128
        'request_fulluri' => true,
    ),
);

$cxContext = stream_context_create($acontext);
 $geturlcontents = file_get_contents("$url", False, $cxContext);





Incidentally for getting web resource in python

use

html=urllib2.urlopen('http://python.org/').read()





but ensure in case of proxy server,to set your environment variable http_proxy

use            $ export http_proxy=http://proxy_server:port


$ echo $http_proxy            should echo something like http://proxy_server:port/

Thursday 6 December 2012

Document similarity using Hadoop

This article provides means to achieve document similarity which can be used for recommending similar webpages,news feeds,articles etc from the big web.
To begin with we shall be using hadoop for handling the colossal data.

The procedure involves considering each document and eliminating its stop words,(stop words are redundant words such as a,the,an,which etc..A list of stop words can be obtained from here) .Then we can use map reduce algorithm in hadoop distributed file system to identify the TFIDF associated with each word in a document

What is TFIDF ?
Well its composed of TF (Term frequency)and IDF(Inverse Document frequency)
The term frequency indicates the how many times a term(word ) occurs in a document.And the document frequency (DF in IDF, Also IDF=1/DF) indicates how many times the word occurs in the document.. However actually they give you the term count and document count for the word.To compute the frequency just divide the term count by total number of words in that document

                                     (number of times W occurs in document D)
OR TF for word W=  ------------------------------------------------------------------
                                      (Total number of words in document D)

and IDF would be


                                     (number of times W occurs in all documents)
                                DF =  ------------------------------------------------------------------
                                      (Total number of words in all documents)

So you can agree that higher the TF and lower the DF ,the word is important and can be used to analyze the characteristics of the document.Say a word w which occurs many times in a single document D but less in all other documents then this word can be used statistically to identify D.

So TFIDF would be  TF * log(1/DF)
Log is logarithm to base 10
_
A_________
|   w         |                
|           w |                 
|    w        |
|_______w|


B__________
|   w     |
|           |
|            |
|_______|


C________
|         w|
|           |
|           |
|_______|


D__________
|            |
|            |
|            |
|_______|
Consider the above example where document A has a lot of W words the same W is rarely present in other document then it has a lower DF, and higher TF
Thus the TFIDF for the word W in document A is high.

So what we do here is identify the tfidf of all the word|document in corpus .Set a threshold for the  TFIDF and consider those words whose tfidf is above the threshold.We can identify the cosine similarity between two documents from these TFIDF

Build a graph of the corpus wherin each node is a document and the edge between them are weighted based on cosine similarity .Higher the weight more similar are the nodes.

(Doc a)------<cosine similarity(a,b)>-----(Doc b) ..etc

Cosine-similarity between two documents a,b= sum ( TFIDF of a * TFIDF b) 

Use graph clustering technique to cluster similar documents.We shall implement
this paper for clustering.

The technique involves randomly move about the graph initially and intialise the nodes by random labels , then we shall set a threshold on the weight of the edge between nodes , or we remove weak links.And we shall recursively analyze for a given node what label does majority of its neighbor are pointing to.We shall then assign this label to the node, we go on about like this until the labels assigned to the nodes don't change.
Most importantly this method automatically identifies the cluster size unlike k-means where we choose k. OR in this case k is identified by the algorithm not us.

Get the top tfidf keywords for the documents (using Hadoop)

Use majorclust to cluster the documents

When this is done use a php script or so. To use this data to provide the document/webpage relevance.
say the user chooses/views certain document/webpage the relevance can be provided to similar context
Here each document's name is a url as in url1,url2 etc

Get the tfidf hadoop implementation here, Edit this particular code
1. Add more stop words use this , Look at WordFrequenceInDocument class and observe googleStopwords.add("word");.. use the list of stopwords in the given file and copy it here.
2.Write the output to a file and parse it to obtain the file format as shown in the pics earlier so the python script can be run as such

Majorclust for clustering docs are implemented using this python script,(got it from stackoverflow response and edited as per requirement) see here



To illustrate the entire process:
And the architecture for implementation would be:

Sunday 11 November 2012

How to make a web server

Any embedded device can be connected to the cloud and provide its own service,So after choosing you hardware such as a ARM kit or even a FPGA, The simplest method is to port an operating system on it.

Check link for porting linux on arm9
Check link for porting FreeRtos on arm7
check link for porting RT-Linux on powerpc (FPGA)


Once you have ported an OS , you got to enable ethernet facilities .

Then you can run a socket based program (check tutorial) , with this program running all you got to do is access your machine from any other PC connected to the ethernet by specifying the IP address.
That's it, by pinging the server it can execute any task as defined by your program,(Note socket just provides the means of connection,the program must also process data).

If i define port id as 80 and provide service of handling web pages or html contents , then i would have developed a HTTP server.

For a FTP server (check sourcecodes) you have to use a port id of 21.


Porting FreeRtos on ARM 7

Start by downloading a FreeRtos kernel in this case V5.3.0 , but feel free to use your favorite version from here which is completely free.And the ARM7 is KEIL MCB2100

We shall also be needing Flash Magic from here and Keil uVision Integrated Development Environment.

Start uVision IDE (I have used uVision 4).

Open project.
Select RTOS Demo from FreeRtos folder
Go to Project in uVision and select Device .
Then build the target
Here you may have to set the crystal frequency and other options (Check the data sheet for your ARM), in my case the frequency would be 12Mhz
In the output tab select create HEX file

Now Build Target in Project tab.This creates the hex file in your directory.
Start Flash Magic
     communication settings
        Select Device (mine would be LPC2129)
        COM port COM1(or others)
        Baud rate 115200(your choice)
        Interface none
        Oscillator 12 (Mhz)

Erase All Flash

Select Hex File

Set "Verify after programming"

Start it!

Thats it , FreeRtos is on ARM7 , you are greeted by blinking LED's
Now prior to generation of HEX file you can EDIT the FreeRtos source code , You can even define your own task in its main program; Thus you can execute just any application.

Monday 15 October 2012

Security in mobile computing

I present here a report on security in mobile computing after in depth analysis of several research papers.
A summary is presented as pdf for free download  here

In the given pdf consider the Trust based security , A detailed explanation is presented


"Jhon wants to access services such as printer but is not authorised so he request susan by sending request and credentials,susan sends a certificate and may impose constrains (say impose time restrictions on access to printer)on the request, this data will then be sent to security agent which will service request.
This can be used to secure smart spaces

Role is assigned to each user(software/individual),role is associated with security policies and authority
A hierarchical layer of security agents
Authorised user can delegate authority to other user by signed assertion,these signed assertions are validated by agents and the request is serviced
Delegation chain :If a entity delegate authority to a malicious module it can lose its delegation capability
"

Now for the concentric centric security

"The user application will have a corresponding mobile agent (MA)proxy ,The MA is software with data which can move from environment to other,thus it can save the session state and move between network,allowing the user to roam and seamlessly connect with different networks
Ubiquitous context based security middleware(Ubicosm) allows the user/service provider to specify the security capabilities and requirements as a metadata


Ubicosm will also appraise the resource availability and state in the new environment
And along with the metadata will provide a visible window comprising of only those resources which the MA can access,and which are validated. Also the metadata allows the MA to identify other MA’s of required security and other features and interact.
Changing security requirements are reflected as change in metadta with no change in codes

"

Sunday 14 October 2012

Porting Linux on ARM 9

We shall consider the TS-7300 arm9 from Technologic Systems.TS-7300 allows us to port using the SD-card.

Problems of porting linux on ARM7 lpc2100
please note any attempt to port linux onto boards like arm7 mcb2100 is intractable with generic methods as the linux incorporate memory management unit which needs to be supported and which is missing in the said board (mcb2100)
Additionally porting kernel image onto flash which typically has size of 256KB is again not possible for linux,Pretty bulky!

To begin porting on TS-7300

First we shall partition the SD card
1.Download SD image from here
2.Extract the SD image ,insert the SD card into the SD card reader(either you can use card reader available in your system or may have to acquire a card reader,generally they are available in standard Laptops)
3.Format the SD card,this will delete all partitions(backup already loaded data if required).For formating try Gparted tool,install by :
$>sudo apt-get install gparted
4.Identify the SD card in your system.
$>fdisk -l 
lists the partition in your machine ,The SD card is identified by /dev/sdxx xx being certain integer/character variables






5.Bootable SD card can then be developed by using
$>dd if=/path_to_SDimage/sdimage.dd of=/dev/sdxx
Note the dd command is a linux nuke, if you do not specify the correct disk at of= field then it may crash your system.Happened to me !.In case you do not heed the warning , not to worry check this
6.Minicom is something like hyperterminal in windows, allowing you to work with serial port, to install
$>sudo apt-get install minicom
7.Insert the SD card onto the TS-7300 machine and using a RS232 cable connect the TS-7300 with your system,A serial to USB adaptor may be required if your system does not have the DB connector for the Serial port.
8.Switch on TS-7300 and use the minicom to access the serial port
$> minicom -s
shows a configuration window

To provide appropriate serial port , you have to find which serial port is connected to the TS-7300 for that
$>dmesg | grep ttyUSB
lists all the USB terminals,in case you are using the serial to usb adaptor.However if you are directly using a single serial cable then
$>dmesg | grep ttyS*
will list the serial port , Identify your serial port and enter in serial device field of minicom.Note for example the above screen grab indicates /dev/ttyUSB0 as the USB port to which the serial cable is interface.
Also set the desired,data rate
9.Choosing Exit in minicom will establish the link between host machine and peripheral, In this case you may see the Board booting up !
Enter
$> uname -r
In the minicom terminal this indicate the version of kernel on the board,We have ported an OS !!

link is very useful for developers,The repository also provides various flavors of OS and tools.

Filesystem may be needed to develop device drivers,If the filesystem is not inherently provided by the SD image ,a filesystem is needed to be created after partitioning  the SD card.

The C program which provides the functionality needs to be crosscompiled and introduced into the userspace ie the filesystem , by copying the crosscompiled C code(Crosscompiling produces an executable) onto the SD card.

Acquire the cross compiler by using
$>wget http://code.google.com/p/princess-alist/downloads/arm-linux-gcc-4.3.2.tgz

Extract at usr/local/bin , and set the environment fields by
$> export$PATH=PATH:/usr/local/bin/4.1.1-920t/bin/
$>export ARCH=arm

$>export CROSS_COMPILE=arm-linux-

Cross compile by
$>arm-linux-gcc -o objectname filename.c

Transfer the objectname file to the SD card and after booting you can execute it by using ./objectname in ARM9

Check out a LED driver,link
 
This is sufficient to get you started on ARM!

We have just ARMed the Penguin !

PS:If you face any problems , specify the same in the comment field.


Saturday 13 October 2012

Clamscan Antivirus in Linux

Free antivirus is available in Linux , ClamAV seems to be pretty credible and is provided free of cost, The same can also be used to check for virus in windows partition: 1.If you have windows and Linux in separate partitioned disk in your machine then you can simply mount the disk which has windows installed in it and run the clamscan in the terminal as $> clamscan /pathtomounteddisk 2.If you have sole windows machine , again you can use live cd/usb to run clamscan Install clamscan by using Sudo apt-get install clamav The virus signature database can be obtained from Clamav To update the database use freshclam as $>freshclam ; To install this use $> sudo apt-get install clamav-freshclam If you face problem updating using freshclam (If you are behind a proxy) , you can manually download the virus database main.cvd from the clamav.net link mentioned above and paste it to /var/lib/clamav You can then use clamscan !

Sunday 5 August 2012

Adding repository in ubuntu

In order to download and install a particular software from a given url in ubuntu after recognizing it as a repository so tools such as synaptic package manager can automatically download and install the packages,you need to add the given url as a repository for that

Type sudo gedit /etc/apt/sources.list in terninal

paste the sources or url's to sources.list file and save

Then you may reload your synaptic package manager,you will then be in a position to install packages from the url

Wednesday 18 July 2012

Installing lamp

Installing lamp in ubuntu is pretty simple.

$> sudo apt-get install tasksel

where tasksel provides a user friendly interface allowing easier installation of lamp.
Run tasksel as a super user

$>sudo tasksel

A gui interface opens up,use the up/down arrow key to select the lamp server.press space to set it for installation and continue.

during installation provide the mysql password.

Once installation is complete,how to check lamp is working ?

Go to /home/var/www this is the directory from which you have to work as it is the default directory of lamp.Make a text document for a simple php script.

<?php
phpinfo();
?>

save it as index.php

Go to your browser and type localhost/index.php.


This then prints all the information of your server,in your browser ... Its a hello world program in php !!

If it does not do so your apache server may not be running.

To start it.
$ sudo /etc/init.d/apache2 start
To restart
$ sudo /etc/init.d/apache2 restart
To stop
$ sudo /etc/init.d/apache2 stop

You can also install phpmyadmin which provides a swanky gui to manage databases,

$>sudo apt-get install phpmyadmin

to access localhost/phpmyadmin needs to be entered in the browser,the user being "root" and password is what you have set.


Sunday 15 July 2012

steps to make zimage, uimage

Here we attempt to create zimage and uimage associated with linux kernel required
to port linux on arm9

First download the kernel in our case,linux-2.6.24-rc8, from kernel.org.
Then extract the file.
Then type the following in terminal:
First make a directory

$ mkdir cs-e9302
$ cd cs-e9302
$ wget http://www.kernel.org/pub/linux/kernel/v2.6/testing/v2.6.24/linux-2.6.24-rc8.tar.bz2
$ wget http://dev.ivanov.eu/projects/cs-e9302/linux-2.6.24-rc8_ep93xx_mmc.patch.gz
$ tar -xjf linux-2.6.24-rc8.tar.bz2
$ cd linux-2.6.24-rc8
$ zcat ../linux-2.6.24-rc8_ep93xx_mmc.patch.gz | patch -p1
$ cp cs-e9302_kernel_config .config


$ cd linux-2.6.24-rc8
$ nano Makefile

in the makefile change:
ARCH? =arm
CROSS_COMPILE=arm-linux-gnueabi-


Then save the changes in the Makefile.
After that

$ cd linux-2.6.24-rc8
$ make oldconfig
$ make menuconfig

In the menuconfig file
Make sure your system type is "EP93xx-based" and "Support Cirrus Logic EDB9302" platform:
And for more functions check "Root file system on NFS" and SMB file system support".

after making these changes just write the make command:
$ make
we will get the kernel in the arch/arm/boot directory.




Now for creating the uImage:
We will need to install mkimage in the PC.
$ sudo apt-get install uboot-mkimage

After this to get the uImage:
$ mkimage -A arm -O linux -T kernel -C none -a 30008000 -e 30008000 -n \linux-2.6.24-rc8

This would create the uImage in the same directory as the zImage.(in  linux-2.6.24-rc8/arch/arm/boot)




Making a Facebook App

For making a facebook application you need access to server where your code operates,you can however carry on with app development by using your system as the server,localhost.

Facebook accepts only https or secure http so make sure your server can support the same.

To begin with,login to your facebook account and in a different tab open this ,this being a portal for facebook developers.You can later start working by using their tutorial

For now click the create new app tab at the top right corner.An ajax alert like window opens,
Provide an app name,and you have to then provide a unique name in name space which is then used to identify your app.Also in case you do not have hosting facilities,you can sign up for the free hosting at heroku.

Pass the subsequent security check to prove your not a bot.

Then if you encounter "You've been blocked from creating apps because we haven't been able to verify your account" , this means you have to verify yourself by going here the verification requires a government issued id with name, date of birth and photo .Once verified you will be good to start your apps.

Upon success you will be given app id and secret key.

Now in a text editor compose your app code,Then go to app on facebook tab,in the canvas url provide your code file name which you have created in the default localhost folder,The secure canvas requires you to just replace http with https.
Save the changes.

Then visit https://apps.facebook.com/your-app-namespace. viola its your app!!

To develop the app in localhost check out this

Arduino tutorial

we intend to dump small codes onto arduino in linux,

We will use roboduino which is similar to arduino

Start by Downloading the IDE for Arduino boards, namely Arduino 1.0.1 from
http://arduino.cc/en/Main/Software

Open the IDE

Connect the Arduino Board using the USB cable provided with the
board


Go to Tools Serial Port to make sure that the Board has been recog-
nised. Select the appropriate port from the list. Also select the right
board from Tools Board. Roboduino uses Arduino Duemilanove. The
board given to us was Arduino Duemilanove with ATMega328 proces-
sor (If you are using any other board with different processor then all you have to do is choose the appropriate one from the list).

Go to File Examples 1.Basics Blink . This will open the blink pro-
gram from the list of sample programs provided with the IDE

Compile the program either by going to Sketch Verify/Compile or by
pressing Ctrl+R . Correct any errors that may appear. Since this is a
preloaded example program, it should compile without errors.

Go to File Upload to upload the program to the board or press
Ctrl+U.


Arduino led blink code

int led = 13;
// the setup routine runs once when you press reset:
void setup() {
// initialize the digital pin as an output.
pinMode(led, OUTPUT);
}
// the loop routine runs over and over again forever:
void loop() {
digitalWrite(led, HIGH);
// turn the LED on (HIGH is the voltage level)
delay(1000);
// wait for a second
digitalWrite(led, LOW);
// turn the LED off by making the voltage LOW
delay(1000);
// wait for a second
}

Code transfered to arduino board.

Monday 2 July 2012

system crash recovery

I had mistakenly used the killer command dd on dev/sda.Which caused my grub to obliterate and the partition table as well completely disappeared ,hence i was unable to boot my system ,i had windows as well as ubuntu running on separate partition;fearing the worst i searched for ways to recover data on web.
I hence state here the complete method to recover data from your system in case you face the same problem.
To begin with you have to use a live CD or live USB ,the live indicates that the OS can be booted from them.This also ensures that you do not overwrite any data on your hard disk as you are using a bootable usb or cd.
Plugin the Live USB or insert the Live CD and restart your system ,you may have to press f12 or any equivalent key to provide booting priority.
We shall be using the Ubuntu OS .Once the OS boots up ,open a terminal and type:

$ sudo su 
 This makes you a super user.
then type:

$ sudo apt-get install testdisk

The testdisk checks for partitions on the harddisk and the detected partition can then be saved and once rebooted with the Live CD/USB the partitions are then visible and can be mounted which then can allow the user to access the data from the harddisk,itself.

upon downloading the testdisk ,start it by the command
$ testdisk

This starts the programme ,you can choose to create a new log file.
As the next step choose from the list your hard disk,observe the size of the media presented and determine your harddisk from its size.
After choosing your harddisk proceed ,you then have to select the partition table type,the software will choose the most probable partition table type,however you can be extra careful and determine the same from other sources.

Then select analyze which will start scanning your harddisk for any information on partition, After some time when it is done with scanning it provides a list of partition,you can recollect the partition of your system ,if it matches select write,which causes the partition to be saved.If however the provided partition does not match then select deeper search ,which forces the software to search for the partition information .

Once it is done reboot your system and then observe that your drives have been mounted and you can access the data!!

If the deleted data needs to be recovered then use photorec
download it by 
$ sudo apt-get install photorec
and use it by

$ photorec

You then have to provide the appropriate file type you are searching for and viola it finds all the deleted files of the indicated type for you in the chosen destination folder.

  Also there may be possibility of recovering the grub or bootloader it self,but for the provided steps please use extreme caution before using them.

If you have windows and linux partitions on the same machine,and if the bootloader or grub has been missing then you may recover by using the following steps .
use $ fdisk -l
To check the partition on your system,identify the dev/sdx which corresponds to the linux partition x may be 1,2,3, and so on.
Once you have identified ,you have to mount it onto the live cd/usb by using
$ sudo mount /dev/sdx /mnt

lets mount the boot as well

$ sudo mount /dev/sdx /mnt/boot

and then we can use bind to make a mirror image of the linux partition on the live cd/usb by using 
$ sudo mount --bind /dev /mnt/dev


now you can access all your data on the linux partition as the /mnt will replicate the root of your linux.


we will now switch root to /mnt
$ sudo chroot /mnt


we will install grub here by
$ sudo grub-install /dev/sda


Reboot the system and viola !! the purple screen greets us with our favorite OS's.

Wednesday 16 May 2012

Latex tutorial,sample code included

Latex is a really pretty tool used for creating all sort of document viewing format mainly pdf's .Yes there exists tools for converting MS word to pdf but its not as flexible as Latex ,By latex you have total control on just everything.

Latex can be used in linux OS ,for that you may have to install kile .
Kile provides gui tools so that you can just click and the corresponding codes are incorporated into the latex.

Generally another counterpart of kile is dia diagram editor ,using which you can just do any diagrams with built in symbols to assist you ,so again it has gui help for the user.

I will not dwell into deeper waters as the web is filled with ubiquitous help for latex and dia.
Latex can also be used for making ppt slides .

A sample latex format for presentation slides are provided here

A ieee latex format is provided here

If you encounter any errors when using this format,that may be because you may not have IEEEtran.cls in your working directory , download from here

Sunday 29 April 2012

Write your own shell in c

The main shell program which uses fork and exec to fork a process when the user provides a command :download
The shell itself can be forked ,thus multiple terminal can be created and program runs in each of the terminal :


#include<stdio.h>
int main()
{char c;
while(1)
{printf("want to login ? press enter\n");
scanf("%c",&c);
if(fork()==0)
{
system("xterm -e 'path-to-the-executable file of shell program' ");

}
}
}

save it as say main.c


Here create the executable for the shell program after downloading
> gcc filename -o executablename

then put the path to the executable in the above script then run it as

>gcc main.c -o main
>./main

The shell program involves the use of pushd,popd and dirs command which are used to change the directory "pushd directory-path " will take you to that directory and "popd " takes you back to the previous directory
dirs provides the list of directory which is in the stack ,which you have pushed

Also there exist option of altering the path variable wherin "path + path-name" will add the path to the shell and "path - path-name" will remove it from shell
and "path" gives the list of path.The provision of path allows the shell to search for executables at that path.Thus when the user provides a command at the terminal it exhaustively searches for any executable file at all the path list specified corresponding to the user terminal command.




Executing the shell: password and user name is stored in a file named passwords.txt ,note the usage of path and pushd popd options .
pushd , popd allows you to change the directory and go back to the older one similar to the implementation of a stack
The path allows you to specify the location of executable,note i have specified path as /bin which contain all the executables thus allowing ls command to work.
Also in the current directory i have stored my own c file generated executables such as ls1 which does the ls operation.

 Thus you can create your own c files and save the executables in the current or path specified directory and use the same in your shell program.

Happy shelling !!

<A tutorial on shell will be provided shortly>

Friday 27 April 2012

How to call C functions in python ,A tutorial of swig

Here we shall use a binary search tree c program and invoke the same from python

The c function file : download here , name it as bst.c
The swig interface file : download here , name it as bst.i
The make file would be : saved as mf


all:bst.o bst_wrap.o
    ld -shared bst.o bst_wrap.o -o _bst.so    
    python p.py   

bst_wrap.c:bst.i   
    swig -python bst.i

bst.o:bst.c
    gcc -c bst.c  -o bst.o  -I/usr/include/python2.6
   

bst_wrap.o:bst_wrap.c
    gcc -c bst_wrap.c -o bst_wrap.o -I/usr/include/python2.6
   
  
kindly note that the path to python directory and the version may be different for you .



The python p.py file :

import bst
bst.bst()





Execute the make file  as > make -f mf


But with the make file:











<A tutorial and explanation for all stuff's mentioned here will be provided asap>

Thursday 26 April 2012

Page Replacement in c ,A simulation

Least recently used: download code here
Optimal page replacement : download code here
First come first out: download code here
A shell script for providing graphs of the page replacement is also provided
in case you dont want to use it ,modify the codes above accordingly

The shell script being



#!/bin/sh

gnuplot -persist <<PLOT

plot 'a' using 1:2 title 'page replacement' with lines;set xlabel 'time'; set ylabel 'fault'

quit
PLOT




Save the above in a text file as plo.sh (or any other name for that you may need to modify the name as well in the c codes above)

<The explanation for the above codes will be provided shortly >

Scheduling using threads

Thread based programming for simulation of :

First come first serve : download code link here
Multi level queue : download code link here
Multi level Queue with feed back : download code link here
Threads with Priority : download code link here
Round Robin:download code here
Shortest Job first:download code here

<A tutorial on thread based programming and a brief summary of the codes will be provided shortly >

Friday 20 April 2012

VHDL to Spartan tutorial

We will be attempting (and succeeding) to make a simple vhdl execute in spartan 3 fpga board, I am using the one from digilient.But if your's is different don't fret this tutorial will work just fine.

So go ahead and click on the xilinx ise (the one used here is 10.1 again if your's is different it works just fine),

Provide a meaningful name (unlike what a lazy guy like me have done) then click next


Please make sure it is selected to your board version ,here being spartan 3 and if you care to simulate then select an appropriate simulator.


Select a new source to create a new instance of a vhdl file.






We are here creating a vhdl file hence select the same,but in case you need to create any other instance you have to do the same sequence of operation

 

Specify the input and output port identities ,note even if you skip this step ,you can still edit in the entity of the program the same port data


  Code your program and check the syntax by clicking synthesize - XST - > Check Syntax

You can simulate the same by choosing behavioural and simulate using the specified simulator ,ise comes inherently with a ise simulator .You can even procure another simulator by the name modelsim



With that done we have verified the program syntax and logic

 
We need the vhdl module to communicate with the external world once synthesized into the fpga and hence we need to map the programming ports onto the peripheral present ,we do so by observing the peripheral id which in some instance (The spartan 3 of digilent provides the peripheral id just below the peripheral) will be provided on the board itself.The peripheral may be switches or LED's or 7 segment and so on.
Also note that you are forced to use the peripherals on the board and you will not be blessed by additional peripherals so in case your program needs a 32 bit data to be fed and you intend to feed the data through switches ,Then you are forced to use 8 such slide switches provided on the board .A possible way out is to use the 8 slide switches 4 times thus you may hence need a enable signal and map it to one of the four push buttons,to indicate a 8 bit is entered to the board.



The xilinx pace above will allow you to provide the location of the ports mapped to the peripherals.
In case it does not work you may create a new source as a implementation constraint file or the UCF and manually edit it.It looks something like:

#PACE: Start of Constraints generated by PACE

#PACE: Start of PACE I/O Pin Assignments
NET "a"  LOC = "g12"  ;
NET "b"  LOC = "f12"  ;
NET "c"  LOC = "p13"  ;
#PACE: Start of PACE Area Constraints

#PACE: Start of PACE Prohibit Constraints

#PACE: End of Constraints generated by PACE

Where a and b are mapped to switches and c to a led


Save it.and observe the ucf  file,you can also check the ucf file in the directory where you have saved this program


With that done we (most optimistically ) go to the end of the tool chain and execute the last task (Doing so will cause all the subsequent tasks to be executed but hey,it's a leap of faith all the same).




Select Boundary scan and if any other dialog boxes do open just press OK,





Whoa we are pretty close ,the impact gui tool will ask for your bit file ,select the vhdl-program-name.bit file and press ok,It the opens up another window just press bypass this time.




We then select our device ,that would be the green one and right click it to program.If everything is correct you will be greeted with a program succeeded message

Any day i'll vouch for one spartan board instead of 300 spartans

Wednesday 18 April 2012

Porting guide to port on virtex5 fpga

Porting Rtlinux on virtex 5 fpga ,We will be using the linux environment for developing the bit or an ace file.The reason being is linux offers certain cross compiling tools which offers greater flexibility of usage.

We shall procure the rtlinux 3.1 (even 3.2 would be just fine) then we shall use the patch obtained from link ,(get a .bz2 file).This patch contains the real time constraints which we can incorporate with a linux kernel ,I suggest you to use a linux kernel 2.6 or greater as the linux kernels before 2.6 were non premptive and those after 2.6 are premptive,The premptiveness is needed for a real time environment.

I have used a 2.6.33.9

then enter to the directory where the linux kernel is stored by specifying cd directory-path-name

  cd linux-kernel-directory-path-name
     patch –p1 < /path-to-kernel-patch

(please note for beginners the path name eg would be something like '/home/proton/CA_Project/os/os3/fin' )

This creates a patch to the linux kernel with rt constraints,then we need a cross compiler

We can use  -crosstool-0.43
or crosstool-ng
or you can use ELDK ,I recommend the usage of ELDK as its usage is pretty simple unlike the others where you may have to alter some parameters in the shell scripts

Download the ELDK from here.,download the ppc-2008-04-01.iso

Installing ELDK:
change the directory to the one containing the EDLK image file

Now we got to mount this image by

>mount -o loop -t iso9660 ppc-2008-04-01.iso /folderpath

note the folder path indicates the path where the mounting is carried out,once mounted

>cd folderpath

and install by

>./install -d ~/directorypath
directory path indicate the destination where installation occurs.

After this for usage we may have to set the environment parametersfor which we go to the directory where the eldk is installed and run
 
>source eldk_init 4xx

This automatically sets up the environmemnt variables,which means certain parameters will be assigned certain values









mounting




installing






setting up environment variables (note that i have renamed the temp2 to crosscompilereldk ,just to make sense!)

executing
>ppc_4xx-gcc
shows that it is actually working,now you can specify
>ppc_4xx-gcc /pathtofile
path to file indicate the path to file for crosscrompiling wit respect to ppc-4xx

where xx corresponds to any number it may be 405,440 etc

now we have successfully installed ELDK by mounting it and also we can crosscompile any file to ppc architecture.

For porting we need to perform another operation,we need to make a device tree..oops why i didn't tell you before ?,gee i didn't want you to be scared by all these jargons eh ?.

But relax building a device tree is pretty simple (very easy ) it's easy because now we can use the edk gui to build it ,yes no more linux terminal for the time being.

To put it crudely ,device tree contains basic information of the hardware (on which OS is to be ported) .

Now i'll switch over to windows to use the edk ,however YOU CAN CARRY OUT THIS STEP IN LINUX EDK ITSELF

<Development of device tree procedure will be made available as soon as possible>


Now that the device tree is donet.We shall use this .dts file with the kernel .

enter the linux-xx file (file where the linux is stored go inside this linux kernel file which has been already patched) ,now copy the .dts file to /arch/powerpc/boot/dts  after copying you have to RENAME it to  virtex440-anyname.dts (for the virtex 2 pro its virtex405-anyname.dts)


you can also acquire the ram disk image for powerpc linux from here download the ramdisk_image.gz
and save it in /arch/powerpc/boot ,don't extract it ,save it as such.

This ramdisk contains data that allows the real root file system to be mounted so during booting this is first used and unmounted when the actual root file system takes over using the support from ramdisk
Its also called initial ramdisk.

We need to edit a wrapper file in /arch/powerpc/boot file name wrapper

check line 145

ksection=.kernel:vmlinux.strip
isection=.kernel:initrd
link_address='0x400000'

change the link_address='0x400000' to link_address='0x800000'

This is an effort to provide more memory to the kernel

Now we have to configure the kernel or sort of tell the kernel , Ok you are going to be ported to this hardware so make yourself ready.
check the /arch/powerpc/configs it contain 40x and 44x inside this folders are the config files related to various hardwares ,we shall be using the virtex5_defconfig in 44x ,execute
>make ARCH=powerpc 44x/virtex5_defconfig


now its configured ,we will now generate a .elf file by

before this we got to set the system parameters of cross compiler
 by using source eldk_init 4xx in the directory where we have developed the cross compiler so the binutils use the ppc_4xx as the cross compiler


>make ARCH=powerpc simpleImage.virtex440-anyname

virtex440-anyname is our .dts file,you can also equivalently use a 405

for the ramdisk incorporated .elf file

>make ARCH=powerpc simpleImage.intrd.virtex440-anyname

we have a kernel image with a ramdisk for supporting rootfile systems

check /arch/powerpc/boot there you can find the elf file as simpleImage.virtex405-anyname.elf  or  simpleImage.intrd.virtex405-anyname.elf



LAND ahoy!!!

to port we use the xmd

XMD% connect ppc hw -debugdevice deviceNr 3 cpunr 1
XMD% dow arch/powerpc/boot/simpleImage.virtex405-anyname.elf
XMD%run

Porting complete!!!!!!!!!

Thursday 12 April 2012

When to use signals and variables in VHDL

First let's see the purpose of a signal and a variable,The signals are used as a wire acting as a communication (conveying event information ) between design components ,so to put simply we use signals when we want to connect together blocks just like a wire between them.
The variables on the other hand are used for computation so you can roughly imagine a register whenever you use one ,And the most subtle distinction between the two is how they behave when used in a process.

Before seeing that let's understand why exactly do we need a process ?

A process is something like a wrapper that allows all the statements inside it to be executed sequentially that's one after the another.and those statements outside the process block execute concurrently.

Thus if we have  a = expression(b,c,d);

lets say we assign to a the result of the expression involving b,c and d now since all the parameters in vhdl will eventually mapped to hardware signals ,the expression will be evaluated whenever there is a change of signal status involving b,c and d or whenever either of b,c or d changes then the particular statement
a = expression(b,c,d);
gets executed so what if the program is something like

a1= expr(b,c);
a2 = expr (b,e);
a3 = expr (c,e);

so lets for example say signal b changes its state then statement a1 and a2 will get evaluated concurrently or simultaneously.

So we need a mechanism to prevent such concurrency depending on certain programming paradigms for which we use process whenever we need such a scenario.

The process expects a sensitivity list which is a list of signals specified to it and the process sleeps(sort of) until there is a change in signal status in its sensitivity list or the occurrence of an event which causes the signal to change its state will lead to waking up the process,so once the process is active then all the statements inside it will execute (yes!) sequentially.

But (Ahh) there is yet another subtlety and more often a wrong usage of variables and signals inside here causes error .


so for example if  v,v1 and so on are variables and s,s1 are signals
then inside a process if we use


lets say s ,s1 v and v1 are all type integer

and say s=0 and s1=0 v=0 and v1=0 initially
process(s,s1)
{
s1<= s+1; -- instruction executed but not assigned
s<=s1; -- s still has its old value
s<=s+1;
v:= v1 +1; -- instruction executed with immediate assignment
v1:=v+1;
}

So what are v ,v1 and s,s1 ? note that v,v1 are variables and s,s1 are signals


so what are the end values of each parameter ?
is s = 2 ?
whoa hold it right there,the statements are executed sequentially but the assignments to the signals happen when it exits the process but the variables inside the process is quite immune to this effect thus


lets see the values after the end of process block
s1=1
s=1
v=1
and v1 is 2

we see here that the variables are assigned immediately but the signal assignment only occurs once the process ends

And hence in all the signal assignment effectively takes the last expression which was assigned to it.


Wednesday 28 March 2012

A function for the main file with doxygen,c programming

This is the function fu.c (I am worse at naming stuff ,sorry)

 ...................................................................................................


/*!
*
 * 
\see a4.c
*/

/**
 * \brief   The function is intended to alter the arrays as per the rule depending on the user input in the file b1.txt,Temporary arrays are internally generated and which holds the intermediate data ,eventually the content of the temporary array is transfered to the main array.And similar process is carried out in the next iteration
 *
 * \details   Three iterations are performed and at each iteration the corresponding number of characters in the user chosen card is provided in the array "co" which is utilised,The iteration eventually causes the desired card to arrive at the slot as premised.
 *
 *
 *
 *
 * @param a
 *  The <a href="en.wikipedia.org/wiki/Pointer"> pointer</a>  to an <a href="en.wikipedia.org/wiki/Array"> array</a> is passed here,the array content being the card list ,As the approach illustrates representation of datum of cards as a two one dimensional array rather than a single two dimensional array.
 *   The array holds one component of a two component list which uniquely identifies the card
 *  
 *  
 * @param d
 *   It represent a pointer to an array which holds the other half of the card representation parameter
 * @param co
 *   It also represents an array which is used to hold the length of the spoken word by the user which being indicated in the file b1.txt
 *   Only optional parameters are explicitly stated as such. The description
 *   should clarify the default value if omitted.
 *
 * @return
 *   No return is required as the pointer data is
 *   <a href="http://publib.boulder.ibm.com/infocenter/lnxpcomp/v8v101/topic/com.ibm.xlcpp8l.doc/language/ref/cplr233.htm"> passed by reference</a>
 */

void fu(char *a,char *d,char *co)
{char tmp1[9],tmp2[9];
char b[9];
int i=0,j=0,k=0,l=0;
char t;
for(k=0;k<3;k++)
{printf("--------------------------------------%d iteration\n",k);
for(i=0;i<co[k];i++)
{tmp1[8-i]=a[i];
tmp2[8-i]=d[i];

}


for(l=0,j=i;j<9;j++,l++)
{tmp1[l]=a[j];
tmp2[l]=d[j];
}
for(i=0;i<9;i++)
{a[i]=tmp1[i];
d[i]=tmp2[i];
printf("%c(%c)\n",a[i],d[i]);}

}
return;

}


c program file with doxygen ,makefile provided

The main file i'll call it a4.c to the lack of a better name,(actually it was my fourth assignemnt,or to quote the bard "What's in a name ?")

Make a header file by adding

#include<stdio.h>

and save it as head.h    The motive here is to follow a good programming practice where we keep all the headers in a single file and all functions(some if not all,as the case is) in another file and incorporate them into the main file


Though i can no way compete against a simpler explanation for make files ,refer this link



 save it as mf



all: a4

a4: a4.o fu.o
    gcc a4.o -o a4

a4.o: a4.c head.h
    gcc -c -g a4.c

fu.o: fu.c
    gcc -c -g fu.c

clean:
    rm -rf *o a4



So in all we have a main file a4.c provided here,a function fu.c provided  here
and the makefile mf

to run execute " make -f mf "
and " ./a4 " in terminal


..............................................................................................................


#include"head.h"
#include"fu.c"

/*! \mainpage Welcome to the much awaited card game
 *
 * \section welcome Introducing a program for your gaming pleasure please do check the files
 *
 * \image html index.jpeg
 *

 * \section  install  To run you need the ubiquitous
* <a href="en.wikipedia.org/wiki/Linux"> linux</a> OS and the
 * <a href="en.wikipedia.org/wiki/GNU_Compiler_Collection">  gcc</a> <a href="en.wikipedia.org/wiki/Compiler"> compiler </a>
 *\section usage Follow the specified steps to get it going
 * \subsection step1 Step 1:Enter the list of cards in the a1.txt file one after the other ,The desired card is entered at third slot
* \subsection step2 Step 2:Enter the user desired card statement into the b1.txt parse the ending with a '$'
* \subsection step3 Step 3:Run the make file and execute the .exe thus generated and you are good to go   
 *
 *
 */

/*!
 *  \brief    program to implementation of mathematical card spell out trick in c
 *  \details   The program accepts the set of cards in a certain order from a file named a1.txt and also gets the user choice for the card.
it then based on the user choice of card realters the original order of cards read from file a1.txt to generate the list of card in order dictated by the program such that the card arrives at the fifth slot as the case demands
 *  \author    Vinayak
 *  \image html images.jpeg
 *  \version   1.0
 *  \date      2-2011
 *  \pre       First initialize the a1.txt and b1.txt files.
 *  \bug       Altering the file format may provide unknown results.
 *  \warning   Improper use can crash your application
 *  \copyright  Public License.
 *  \see fu.c
 * 
 *  
 */


void main()
{char a[9],c[20],d[9],tmp1[9],tmp2[9];
char b[9],co[3];
int i=0,j=0,k=0,l=0;
char t;
FILE *f;
f=fopen("a1.txt","rt");
if(f==NULL)
{printf("cant open file of cards");
exit(0);
}
//Read from the file the list of cards


while(t=fgetc(f))
{if(t==EOF)
break;
c[i]=t;i++;
}
i=0;
/*store the list of cards into two single dimension arrays if needed a two *dimension array can be used,as well.
*/
while(c[l]!='\0' && l<26 && k<=8 && j<=8)
{
if(c[l]!=10 && c[l+1]!=10)
{a[j]=c[l];
l++;
d[j]=c[l];
j++;}
l++;
}

printf("The cards before you are\n");
for(l=0;l<9;l++)
printf("%c(%c)\n",a[l],d[l]);

l=0;
fclose(f);

//It now reads user choice of cards from another file
f=fopen("b1.txt","rt");

if(f==NULL)
{printf("cant open file of your input");
exit(0);
}

i=0;
while(t=fgetc(f))
{if(t==EOF)
break;
c[i]=t;i++;
}
l=0;
printf("your lucky call is\n");
while(c[l]!='$')
{
printf("%c  ",c[l]);
l++;
}
/*Calculate the legth of each word demarcated by space ,an additional *delimiter '$' is used at the end for convienience.The calculated word *length is stored in an array
*/
i=0;k=0;j=0;
for(;k<3;k++)co[k]=0;
k=0;l=0;

while(c[i]!='$' )
{
if(c[i]==32 )
{

l++;
}
else co[l]+=1;
i++;

}

/*Perform three iterations each time using the length of respective word *the card array content is realtered as per the rule
*/

fu(a,d,co);

//The result is displayed for the users viewing pleasure
printf("\nThe final results for The bet\n");
for(i=0;i<9;i++)
{
printf("  %c(%c) \n ",a[i],d[i]);

}

//The end!!!!!!!
return;
}

Sunday 25 March 2012

A shell script for Xml to json , Json to Xml converter and parser for accessing record data

This script can also be used to convert RSS to JSON , or any XML to JSON

The shell script does the following things:

1. Display the records to the user.
2. User can query for value of any particular record.
3. Convert from one format to other.

The Designing strategy followed was
The designing algorithm for xml to json conversion and vice versa involves
the mapping from one format to other after identifying the tag nodes and
their elements the identification of which follows certain regular expression
statements ,which in this case is implemented by utlisation of a foreign sript
in the shell , a python script and a javascript are utilised ,they being procured
from a public open source domain and tailored to the needs of this problem.
Both of this scripts invariably use the regular expression for parsing the xml
and json for conversion as the tags their attributes and the elements are
converted from the xml to their respective place holders in the json.However
the attribute in the xml is treated as one more of its element ,though in the
data transfer by utilisation of the xml this does not alter the originality or
the information of the xml content.Thus the conversion process treats the
attributes just as same as the elements.
A linux tool is used for xml parsing this being the xmllint which allows the
utilisation of xpath wherin the user has to provide the path data for accessing
the record data. Another tool of importance is rhino which allows the js file
to run in the linux environment.
The shell script as such utilises certain temporary files to store the xml
and json data and the partially parsed values also grep functions are used to
eliminate certain redundant data to appreciate the pretty printing for xml
and json files. A separate json parser is not implemented as the motivation
was to use the xmllint itself thus the conversion of the json to xml before its
parsing is carried out

Apart from xmllint another intersting tool was the rhino from mozilla guys ,it allows you to run a javascript in the linux terminal
You just got to install rhino by using "sudo apt-get install rhino" then specify to run the javascript

rhino "path to .js file"  "any argument if any "

To use this shell script you need to install python and xmllint ,use the sudo apt-get to do so.

Xml file parser



Json file parser


Xml to json converter

note here you can actually save the display in a file by using the >filename command in the shellscript where required




Json to xml converter


The shell script code

The python script code

The javascript,code