Skip to main content


Showing posts from September, 2009

PyTorch Code for Simple Neural Networks for MNIST Dataset

Representation of File System in the Kernel

Representation of data on a floppy and hard disk may consistently differ, but representing them in the kernel is almost same. VFS mounts the file system through the function register_filesystem(struct file_system_type *fs) , Example register_filesystem(&ext2_fs_type); Mounting Before a file can be accessed, the file system containing the file should be mounted. Done by the call using mount or function mount_root() Every mounted file system is represented by a super_block structure. The function read_super()of the VFS is used to initialize the superblock Managing the file system Will issue some flags during the mounting like MS_RDONLY //File system is read only MS_REMOUNT //flags have been changed   Superblock Operations Super block provides functions needed to access the file system and its processing. Some of the operation includes read_inode();// must execute and Responsible for filling the su

Linux File System

Linux supports many number of file systems, that’s why Linux is accepted quickly as one of the best Operating systems. Linux has the unified interface called as Virtual File system (VFS) which serves as the interface defined in between the OS kernel and the different file systems as illustrated below. The Virtual file System Supplies the applications with the system calls for file management Maintains the internal structures Passes tasks on to the appropriate actual file system Performance of default actions Basic Principles Two main factors taken into consideration while designing any file system Speed of access of data and Facility for Random access is made possible by means of the block oriented devices which are divided into specific number of equal sized blocks. When using the blocks, linux uses a buffer cache to speed up the process in random access. In Linux/Unix, the data is stored i

Sockets for Inter process communication

Socket programming interfaces provides communication via a network as well as locally on a single computer. Example is INET daemon which waits for incoming network service requests and then call the appropriate service program using the socket file descriptor as standard input and output. Implementation of Unix domain sockets Represented by a kernel data structure socket Socket specific functions like Socket(), setsockout() The functions are implemented with a single system call socketcall which calls all the necessary functions by reference to the first parameter. The file operation read(), write(), poll(), ioctl(), lseek(), close() are called directly Operations of unix domain sockets long sys_socket(int family, int type, int protocol); // creates a socket file descriptor long sys_connect(int fd, struct sockaddr * uservaddr, int addrlen); //bind the socket to the unix domain address with specified length long sys_listen(int fd, int bac

Debugging using ptrace

Ptrace is a system call provided in unix for one process to take control of another process to debug errors or bugs in the system The process under control can be run step by step and its memory can be read and modified. int sys_ptrace(long request, long pid, long addr, long data); the function processes various request defined in the parameter request and pid indicates the process id of the process to be controlled. Using the request PTRACE_TRACEME, a process can specify that its parent process controls iut via ptrace().

System V IPC

Three forms of System V IPC Semaphores Message Queues Shared Memory System V IPC is different from POSIX API, but both are available in the linux kernel. GNU C library in kernel version 2.2 includes the interfaces for shared memory and the semaphore according to POSIX. Access Rights and Numbers struct kern_ipc_perm { key_t key;  //key uid_t uid; //Owner gid_t gid;  //Owner uid_t cuid;  //Creator gid_t cgid; //Creator mode_t mode; //Access Mode unsigned long seq; //counter used to calculate the identifier }; the user and group id needs 32bit for the Intel 32 bit architecture, so the kernel supported both the IPC_OLD and IPC_64. Semaphores (System V) Array of semaphores can be setup using the system calls It is always possible to modify a number of semaphores. They can be incremented or decremented in steps greater than 1. Semaphores are created using the following structure str

Pipes and Named Pipes (FIFO)

Pipes are the classical method of interprocess communication. For example # ls –l | more The symbol | indicates a pipe and the shell in the above example runs the processes ls and more which are linked with a pipe. ls writes data to the pipe and more reads it. Named pipes otherwise called as the FIFO (First in First Out) is the other variant of pipe. They can be created like this # mkfifo pathname Example # mkfifo hello # ls –l hello prw-r- -r- - 1 temp users 0 Aug 28 10.45 hello | there are many similarities between the pipes and the FIFO, but the inode specification for the both are more or less the same. following is the inode specification for the pipe struct pipe_inode_info { wait_queue_head_t wait; //wait queue char * base; //address of the FIFO buffer unsigned int readers; //no of processes reading at this moment unsigned int writers; // no of processes writing at this moment unsigned int waiting_readers; /

Communication Via Files (IPC)

This is one of the oldest way of data exchange in which the information is sent as a file. File needs two type of locking Mandatory Locking Read wirte is blocked during the entire locking period Advisory locking Allows Read and write even after the lock has been set processes accessing the file for Read and Write has to lock it and release it again Locking Entire Files There are two methods for locking the entire file First Method is by the use of several system calls link – a system call create – another system call combination of flags like O_CREAT and O_EXEC with open (system call) lock a fle with open() function using O_CREAT | O_WRONLY | O_TRUNK, but this option fails under the superuser mode Second Method fcntl – A system call to lock the entire file. It is also useful to lock the file areas. In Linux 2.0 it is called flock() which is not advisable to use. Locking file areas int sys_fcntl(unsigned int fd, unsigned int cmd,

Inter process Communication in Linux Kernel (IPC)

When two processes trying to access a resource is called race condition. Main challenge is to avoid the race condition, the following method is usually adopted to avoid the race condition use of shared memory can prevent race condition Use of threads. Threads use the same address space when switching it to other threads(so same page directories and tables), whereas processes uses different address which leads to exchange of page directories and page tables. This is made with the help of POSIX Threads. Forms of IPC Resource Sharing (system V Shared memory ) Synchronization (Mutex, Semaphores) Connection oriented data exchange (Virtual Sockets, Pipes, named Pipes(FIFO)) Connectionless data exchange (Signals) Linux implements all forms of IPC Synchronization in the Kernel When multiple processors are used, how each processor is synchronized in the kernel is what described in this section. Processor can acquire a spinlock whenever it wanted to access

Micriµm Unveils New Book: µC/OS-III, The Real-Time Kernel

After the introduction of Micriµm's µC/OS-III earlier this year, the biggest question on many engineers' minds was, "Will there be a book?"  Undoubtedly, these engineers were hoping for a follow-up to the highly popular MicroC/OS-II, The Real-Time Kernel .  One of the all-time best sellers in the embedded field, this text has long been recognized not just as the definitive guide to µC/OS-III's vaunted predecessor (µC/OS-II), but as a first-rate introduction to kernels in general.  The thousands of embedded systems developers who use µC/OS-II embrace the book as a welcome departure from the confusing users' manuals that accompany most software vendors' kernels. With the official release yesterday of µC/OS-III, The Real-Time Kernel , the embedded community no longer has to wonder whether there will be a similar book for Micriµm's newest software component.  Developers seeking to better understand µC/OS-III can now turn to a thorough, well-written te

How to Recompile Network Simulator 2

Assumptions: I have installed my ns-allinone-2.33 under /home/pradeep/ns-allinone-2.33 Path is been set in the .bash_profile (which are located at /home/pradeep/) in fedora and .bashrc in ubuntu There are two ways in which the NS2 can be recompiled 1. You modify the existing algorithm (use the same header and the C++ source files) 2. Adding new files (adding new header and C++ source files) In first method, Step 1: open the shell prompt and go to the ns-allinone-2.33/ns-2.33/ folder(there are 3 makefiles,, Makefile,, for adding any new file should be done at the file) Step 2: Once the existing algorithm is changed (after modifying the existing header and the C++ source file), go to step 1 Step 3: execute the command, make and press enter, If you have no errors in your modified code, then the compilation will be success, else you may get errors, which needs to be corrected Step 4: Test your tcl script for the new algorithm

url and email validation using Perl Regular expression

#!c:/perl/bin/perl -w print “Content-Type: text/html \n\n”; $email=”tspradeepkumar\”; print $email; if($email =~ /^(\w|\-|\_|\.)+\@((\w|\-|\_)+\.)+[a-zA-Z]{2,}$/i) { print “\n $email is a valid id”; } else { print “\n Provide a valid email id”; } $url=” “; print “\n $url \n”; if($url =~ /^http:\/\/+(www)+\.+[a-zA-Z0-9\-]+\.+[a-zA-Z]{2,6}$/i) { print “$url is a valid url”; } else { print “\n Provide a valid url “; }

Hit Counter using PHP files

The same file (counter.txt) is used for both reading as well as writing. So each time the file should be closed before opening for the next time as read or write mode. <?php $filename=”counter.txt”; $f=fopen($filename,”r”) or die(”Read Failed”); $str=fread($f,filesize($filename)); echo “The page has been visited for $str times since 2nd June 2009″; fclose($f); $f=fopen($filename, “w”) or die(”Write Failed”); $str=$str+1; fwrite($f,$str); fclose($f); ?>

Setting root password in MySQL and Phpmyadmin

If you are using PhpMyAdmin as part of WAMP(Windows, Apache, MySQL and PHP), then you need to take care of the root password and hence follow the steps. If you change, only the MySQL password or PHPMyAdmin Password, then the PHPMyAdmin screen does not show the databases, files, etc (instead it will show access denied). So Both MySQL and PhpMyAdmin password should be changed and both password should be equal. Go to mysql command window (by default the MySQL has no password) Execute the command set password=password(”abc123″) ; find the file under phpmyadmin folder (usually in WAMP it will be under C:\wamp\apps\phpmyadmin2.11.6) and find the following line in the file $cfg['Servers'][$i]['password'] = ‘ ‘; // MySQL password Replace the line to $ cfg['Servers'][$i]['password'] = ‘abc123′; (as the password the mysql is also “abc123″. now restart the server and now open http://localhost/phpmyadmin/

File Upload to Server in PHP

For uploading Files to the server, we need to run two files, For example let us say upload.html (to select the file) and upload.php (to upload)…. let us define each of the two files //upload.html <form method=post action=upload.php enctype=multipart/form-data> Select a File: <input type=file name=myfile> <input type=submit value=upload> </form> //upload.php <?php echo $_FILES['myfile']['size']; echo $_FILES['myfile']['type']; echo $_FILES['myfile']['tmp_name']; echo $_FILES['myfile']['name']; @copy(echo $_FILES['myfile']['tmp_name'], “./upload/”.$_FILES['myfile']['name']) or die(”Couldn’t Upload); ?> in the above php script, instead of copy() function, move_uploaded_file() function can also be used. Before running this script, a folder called as upload should be created in the root path, for example, if /var/www/ is the doc

Connect to MySQL Database using PHP

The following script tries to connect the database and if it is failed it will display relevant message. The server is localhost and username/password combination is root/abc123. it is necessary to create a database ‘dummy’ before executing the script. Click here to set password for mysql and phpmyadmin <?php $con=mysql_connect(”localhost”,”root”,”abc123″) or die(”Couldnt Connect to database”); $a=mysql_select_db(”dummy”,$con); if($a) echo “Database Selection Success”; else echo “Database Selection Failed”; ?>

Sorting Functions in PHP

There are different sorting functions used to sort the arrays and other variables. Some of the their functions are listed below: sort() – used to sort an array in normal order rsort() – used to sort an array in normal reverse order asort() – used to sort array and maintain the index association(key value pair maintains even after sorting) arsort() – same as asort() but in reverse order ksort() – used to sort an array by its keys…. Example Program <?php $fruits[0]=”Orange”; $fruits[1]=”Mango”; $fruits[2]=”Banana”; print_r($fruits); sort($fruits); print_r($fruits); ?> The above script displays the fruits array in normal order and also in sorted order…. simply replace the sort function with other sort functions and may see the results. for key value, change the fruits array like the following script $fruits=array(10=> “Orange”, 100=> “Mango”, 1000=>”Banana”); and use the functions ksort, asort and arsort () and see that the key value pair

User Validation/Authentication in PHP

If a user visits a website, where he is asked for validating his username and password and will be validated with the MySQL Database. the following scripts demonstrate this. Step 1: create a database called dummy by executing “create database dummy” in mysql command window or shell window Step 2: Create a table called usertable with two fields ‘user’ and ‘pass’ and populate with some values. create table usertable (user varchar(20), pass varchar(30)); insert into usertable values (’pradeep’,'pradeep’); Step 3: write the following html script and name it as user.html <form method=post action=user.php> Enter the Username: <input type=text name=username> <BR> Enter the password: <input type=password name=password> <BR> <input type=submit> </form> Step 4: Write the following PHP script and name it as user.php <?php $user=$_POST['username']; //get the username from the previous page (html page) $pass=$_POST['

User Creation using PHP/MySQL

Username/password Creation in PHP and MySQL. a Simple form is designed to get the username and two passwords the objectives are 1. Both the username and password should contain atleast 6 characters 2. both the passwords should be same 3. upon submitting to the database, the existence of username is checked with the database, if the username already exists, then the user is allowed to select some other, else user is created. //html file is given below, file name is <h2>User Registration Form</h2> <pre> <form action=”userregis.php” method=”post”> Enter the username: <input name=”username” type=”text” /> Enter the password: <input name=”password1″ type=”password” /> Confirm the password: <input name=”password2″ type=”password” /> <input type=”submit” value=”create” /> </form> </pre> //userregis.php <?php include(”dbconnect.php”); $u=$_POST['username']; $p1=$_POST['password1'];

Sending Email in PHP

How to send email using PHP. The following section describes that with coding….. Before using Mail in PHP, note that, PHP mail is intended for only a small scale use and not for large scale Email systems. So for large scale, you may use other mal softwares. Email can be sent as ordinary plain message as an HTML Message with attachments We will see, one by one Mail in PHP there is a function mail() which helps in sending mails through PHP Scripts. the syntax is mail(To address, Subject, Message, headers) To address – to which address you want to send mail Subject of the mail - Message – body of the mail and headers – are the http headers sent along with the mail to inform the email clients Example Headers are “ ” //to inform the from address “ ” //Reply to a different email id “ ” //Blind Carbon Copy “” //Carbon Copy” X-Mailer: PHP5 //inform the ema

How to Set Cookies in PHP

To Set cookies using PHP, setcookie() function is used. Cookie is a piece of information that stored in the client’s machine (usually browser) by the server. setcookie() contains 6 parameters Name of the cookie- usage- “FirstCookie” The value of the cookie. This value is stored on the clients computer; do not store sensitive information. Assuming the name is ‘Iamthecookie’, this value is retrieved through $_COOKIE[' Iamthecookie' ] Expiration time of a cookie- usage – time()+60 //means that current time + 60 seconds, The path on the server in which the cookie will be available on. If set to ‘/’ , the cookie will be available within the entire domain . If set to ‘/osp/’ , the cookie will only be available within the /osp/ directory and all sub-directories such as /osp/bar/ of domain . The default value is the current directory that the cookie is being set in. The domain that the cookie is available. To make the cookie available on all subdomains of e

Input Validators in PHP

Source : <?php /* * FileName:validation.php * Created on Jan 16, 2007 * Author:Antony */ function validateUname($uname) { if (ereg(’^[a-zA-Z0-9_]{3,50}$’, $uname)) return true; else return false; } function validatePassword($pwd) { if (strlen($pwd) >= 6) return true; else return false; } function validateEmail($email) { if (eregi(’^[a-zA-Z0-9._-]+@[a-zA-Z0-9-]+\.[a-zA-Z.]{2,5}$’, $email)) return true; else return false; } function validateDate($date) { if (ereg(’(^[0-9]{1,2})-[0-9]{1,2}-([0-9]{4}$)’, $date)) return true; else return false; } function validateMobile($mobile) { if (ereg(’(^[+0-9]{1,3})*([0-9]{10,11}$)’, $mobile)) return true; else return false; } function validateUrl($url) { if (ereg(’^[http://]+[www]*\.[0-9a-zA-Z_.]+\.[a-zA-Z.]{1,5}$’, $url)) return true; else return false; } function validatePin($pin) { if (ereg(’^[0-9]

Subscribe/Unsubscribe to a Mailing List

The following PHP scripts shows how to maintain a mailing list. the users are asked to subscribe/unsubscribe to a mailing list and the entire mails are stored in the database. The user has to type his email id and his/her name and select the radio button subscribe/unsubscribe. Since one email ids are unique, the email id field is a primary key. Database Queries: create database dummy; create table mail(id varchar(100) primary key, name varchar(100)) HTML Form: <FORM method =post action=sub.php> <CENTER> <FONT FACE = “ALGERIAN” SIZE = 6><BR><BR><B><u>SUBSCRIBER / UNSUBSCRIBER </U></B></FONT><BR><BR><BR><BR> <FONT SIZE = 4><B>Enter E-Mail ID:</B></FONT><INPUT TYPE = text NAME = eid><BR><BR> <FONT SIZE = 4><B>Name:</B></FONT><INPUT TYPE = text NAME = ename><BR><BR> <INPUT TYPE = “radio” name = “radi

Page Directory, Page table and Memory Mapping Functions

Page Directories and Page Tables   Page directories and the page tables for the user and the kernel segment is managed by the kernel itself. The short identifier for the page directory is pgd, pmd (page middle directory) and the short identifier for the page table is pte. The data type for the page directory and the page table is respectively pgd_t, pmd_t and pte_t.  Functions  relevant to Page directory pgd_val(), pmd_val() - These functions allow to access the real value of the directory entry (either 32 bit or 64bit). pgd_alloc(), pmd_alloc() - provides the memory page for the respective page directory.   pgd_free(), pmd_free() - The directories are freed. pgd_clear, pmd_clear() - deletes the entry in the page directory. pgd_present, pmd_present() - checks whether the directoryentry refers to the page middle directory or page table, returns apositive result if there is an entry present, else returns a negativevalue. pgd_none(), pmd_none() - just the reverse of pgd_present(), pmd_pre

Conversion of Linear Address to Physical Address

The Conversion from the linear address to physical address is done by a separate processor or a MMU In architecture Independent model , page conversion is a three level process which takes four steps 1st part is used as an index in the page directory which refers to the page middle directory 2nd part serves as an index to the page middle directory, there it refers to the page table 3rd Part used as an index to the page table, there it refers to the physical memory 4th Part of the address gives the offset within the selected page of memory . The x86 model supports a two level conversion of linear address to physical address. Highmem support helps upto physical memory size of 4GB being the address space of 32bit address, but Intel added four additional address pins to the Pentium Pro and created the physical address extension (PAE). Since the two level conversion supported only 32 bit addresses, the PAE has the control of

Memory Management (Linux Kernel)

  The data related to the processes are to be stored in the memory somehow by the operating system. In the olden days, primary memory is a scarce resource. and the application demands increases, so store the data of a process, it is necessary to use other forms of memory, either we can use Physical address space virtual address space Mapping the virtual addresses onto the physical addresses is the responsibility of the operating system’s memory management system . Memory protection prevents two processes exchanging data by changing the areas of memory used by both. The program code of a process can be executed in different threads at the same time. In principle, these threads share a common virtual address space. Architecture Independent Memory Model Access time is the primary factor used in Physical (main) memory. To minimize the access times , the cache memory versions like Level1 Cache (L1 cache) and level2 cache ( L2 Cache ) were used. Cache mem

System Calls

System calls provides a layer between hardware and the user space. Provides abstracted hardware interface for user space system calls ensure security and stability single common layer between user space and rest of the system System calls usually accessed via function calls. for example call to the printf () function   //through a function call Access to the C library functions access the kernel and execute the write() function to write on to the screen System calls returns a long variable for success, negative value indicates error Each system call is assigned a syscall number defined in sys_call_table in entry.S system calls are faster than many other operating systems Example System Calls //Example1 asmlinkage long sys_getpid) { return current->tgid; } Pause: Interrupts the execution of the program until the process is reactivated by a signal. asmlinkage int sys_pause(void) { current-> stat

Page Directories and Page Tables in Linux Kernel

Page directories and the page tables for the user and the kernel segment is managed by the kernel itself. The short identifier for the page directory is pgd, pmd (page middle directory) and the short identifier for the page table is pte. The data type for the page directory and the page table is respectively pgd_t, pmd_t and pte_t.  Functions  relevant to Page directory pgd_val(), pmd_val() - These functions allow to access the real value of the directory entry (either 32 bit or 64bit). pgd_alloc(), pmd_alloc() - provides the memory page for the respective page directory.   pgd_free(), pmd_free() - The directories are freed. pgd_clear, pmd_clear() - deletes the entry in the page directory. pgd_present, pmd_present() - checks whether the directory entry refers to the page middle directory or page table, returns a positive result if there is an entry present, else returns a negative value. pgd_none(), pmd_none() - just the reverse of pgd_present(), pmd_present() set_pgd(), set_pmd() - th

Main Algorithms of a kernel…

Timer Interrupts There is one hardware timer that generates interrupts every 10ms and all the software timer synchronizes with it. Usually the timer stored in the variable jiffies. unsigned long volatile jiffies; The variable jiffies is modified by the timer interrupt every 10ms and hence it is declared as volatile. volatile struct timeval xtime: This is the actual time which again modified by the timer interrupt Other functions of timer interrupt like do_timer(); updates the jiffies timer_bh(); updates the timer and processing of the timer related functions update_process_time(); collects data for the scheduler and decides whether it has to be scheduled.   The Scheduler schedule () is function declared in kernel/sched.c The actions of the scheduler is given below, once the schedule() function is called, Upcoming software interrupts are processed (so interrupts are given higher priority over the other entities in the