How to handle sessions in Google app engine

While developing a GAE app engine recently, I came across the need to handle session variables. While session was a thing that I use to take for granted in ASP.NET (Session[“xyz”]) or php ($_SESSION[‘xyz’]), storing and fetching session variables isn’t something that straightforward with webapp2, the web framework widely used for building GAE apps with python.

However, I found a pretty easy way to implement sessions in my existing app (which was almost half-way done!) by creating a simple BaseHandler class that is session-powered, and then deriving all my url handlers with this new class instead of the usual webapp2.RequestHandler. Once you do that, you can straightaway use the python dictionary self.session[‘my_variable’] (that pretty much resembles $_SESSION[‘my_var’] in php). Here is the implementation for BaseHandler:

class BaseHandler(webapp2.RequestHandler):

    def dispatch(self):

        # Get a session store for this request.

        self.session_store = sessions.get_store(request=self.request)

        try:

            # Dispatch the request.

            webapp2.RequestHandler.dispatch(self)

        finally:

            # Save all sessions.

            self.session_store.save_sessions(self.response)

    @webapp2.cached_property

    def session(self):

        # Returns a session using the default cookie key.

        sess = self.session_store.get_session()

        #add some default values:

        if not sess.get(‘theme’):

            sess[‘theme’]=’cosmo’#’slate’

        return sess

Here, you may initialize any particular session variables if you want and set default values for them (as I’ve done for the cosmo theme above).

Hopefully, this should save you some development time if you want to implement sessions in existing app engine code. As an example, here is how I make use of a session variable to pass on a user’s preferred theme to the template engine, so that the webpage can be displayed accordingly:

class MainPage(BaseHandler):

    def get(self):

        template = JINJA_ENVIRONMENT.get_template(‘stereo.html’)

        if self.request.get(‘theme’):

            theme=self.request.get(‘theme’)

            self.session[‘theme’]=theme

        else:

            theme=self.session[‘theme’]

A simple pythonic script to log internet bandwidth usage on Linux

We often want to log internet bandwidth on our laptops or PCs in order to track our usage over a period of time. This also comes handy to cross-check our usage with that of our ISP’s invoice. Now how to do it on linux? While the Windows folk have all kinds of utilities for this kind of thing, we lunix folk have to invent it ourselves.

Some time ago, I wrote this small python script that writes your bandwidth stats to a log file each time you disconnect your internet connection. This, I’ve tested on ubuntu 12.04 and debian wheezy:

Source file: Tata.py

#!/usr/bin/python

# author: Prahlad Yeri

import subprocess, os, datetime, sys

def execute(command):

    try:

        p=subprocess.Popen(command, shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)

        p.wait()

        result=p.communicate()

        #print str(result)

        if len(result[0])>0:

            return result[0].strip()

        else:

            return result[1].strip()

    except Exception as e:

        print ‘error occured:’ + errorstring

        return errorstring

#main

mypath = ‘/etc/NetworkManager/dispatcher.d/’

iface = sys.argv[1].strip()

status = sys.argv[2].strip()

log=open(mypath + ‘tata.log’,’a’)

#log.write(iface + ‘:’ + status + ‘\n’)

#log.close()

for part in sys.argv:

        log.write(‘argv:’ + part + ‘\n’)

if (iface == ‘ppp0’ or iface == ‘ttyUSB0’) and status == ‘down’:

        s = execute(‘iptables -L -v’)

        rules = s.splitlines()

        rx,tx = 0,0

        rxdrop, txdrop = 0,0

        inrule = False

        outrule = False

        for rule in rules:

                log.write(‘rule:’ + rule.strip() + ‘\n’)

                words = rule.strip().split()

                if rule.strip().startswith(“Chain OUTPUT”):

                        log.write(‘switch to outrule’)

                        outrule = True

                        inrule = False

                elif rule.strip().startswith(“Chain INPUT”):

                        log.write(‘switch to inrule’)

                        inrule = True

                        outrule = False

                elif rule.strip().startswith(“Chain FORWARD”):

                        log.write(‘switch to no rule’)

                        inrule=False

                        outrule=False

                elif len(words) >=3:

                        if words[6].strip() == “ppp0”:

                                #log.write(‘bytes=’ + words[1]

                                if inrule:

                                        if words[2].strip() == ‘DROP’:

                                                rxdrop += int(words[1].replace(‘K’,’000′).replace(‘M’,’000000′))

                                        else:

                                                rx += int(words[1].replace(‘K’,’000′).replace(‘M’,’000000′))

                                elif outrule:

                                        if words[2].strip() == ‘DROP’:

                                                txdrop += int(words[1].replace(‘K’,’000′).replace(‘M’,’000000′))

                                        else:

                                                tx += int(words[1].replace(‘K’,’000′).replace(‘M’,’000000′))

        #print ‘rx=’ + str(rx) + ‘ tx=’ + str(tx)

        #print ‘rxdrop=’ + str(rxdrop) + ‘ txdrop=’ + str(txdrop)

        execute(“iptables –zero”)

        header=”

        lines=[]

        if not os.path.exists(mypath + ‘usage.csv’):

                #create header

                header = ‘date,rx,tx,rx-dropped,tx-dropped,rx-tot,tx-tot\n’

        f = open(mypath + ‘usage.csv’,’a’)

        if len(header) > 0:

                lines.append(header)

        lines.append(str(datetime.datetime.now()).split(‘.’)[0] + ‘,’ + str(rx) + ‘,’ + str(tx) + ‘,’ + str(rxdrop) + ‘,’ + str(txdrop) + ‘,’ + str(rx+rxdrop) + ‘,’ + str(tx+txdrop) + ‘\n’)

        #rx = execute(‘cat /sys/class/net/ppp0/statistics/rx_bytes’)

        #tx = execute(‘cat /sys/class/net/ppp0/statistics/tx_bytes’)

        #strdate=str(datetime.datetime.now())

        #line = strdate + ‘,’ + rx + ‘,’ + tx + ‘\n’

        f.writelines(lines)

        f.close()

        log.close()

        sys.exit(0)

In the above script, replace ppp0 with the interface-name for the name of interface through which you access internet. For mobile broadbands it is typically ppp0, but you can issue the “ifconfig” command to list them all and make sure.

You have to place this script at /etc/NetworkManager/dispatcher.d on debian/ubuntu systems so that it runs each time you stop using the modem. In case of other systems, or in case you don’t use NetworkManager, refer to your distro documentation to check the folder/script which is executed when an interface is down.

Android app can now save people’s lives!

PressureNET

Don’t believe it? I too didn’t until I read about this PressureNET open-source android app recently. This is a crowd-sourced app and its working is simple: Just gauge these two parameters:

1) Atmospheric pressure using the pressure sensors (available in latest android phones like Galaxy S4/Nexus), and

2) GPS coordinates using the GPS sensor.

and send the data to the PressureNET web-server through Internet. Now, with enough number of people running this app on their devices, weather scientists across the nation have access to substantial data regarding pressure-belts forming in various parts of the country. They can then input this data to their weather modeling systems and come-up with solid predictions! (not just vague guesses like now) regarding for example, when and exactly where the next hurricane is going to hit. According to Cliff Mass, an atmospheric Scientist at the University of Washington, this data can help predict an occurrence of a hurricane up to six hours earlier. There is currently, no technique to do it now.

Whilst the pressure-sensor is being used for this purpose, the intentions of Google was something totally different when putting this feature into the android OS. It was to fetch the altitude information. You see, the GPS tells you only the Geo-coordinates (Lat/Lon) on a plane surface. But the atmospheric-pressure changes as a person travels uphill or downwards, and thus it can let us know exactly what altitude a person is on!

 

 

 

References:

http://en.wikipedia.org/wiki/PressureNET

http://www.wired.co.uk/news/archive/2013-01/08/android-weather-app

A Python finds out which is the best performing Linux distro!

With more and more linux distributions competing among themselves for the userspace, we are currently living in the best of times as far as choice in open-source software is concerned. However, more the number of choices, more is the head-scratching to decide what to choose amongst them. Like many, I am also one of the victims of “distro-hopping”. No sooner a major distribution declares a new version, I’m itching to try one out in my virtual box or as a LiveCD version. Few days ago, this benchmarking idea occurred to me.

I’m a software developer and my requirements were clear. I wanted a distro that is good at performance. Since my desktop is cluttered with “heavy-duty” programs like Eclipse-ADT, Java, IDLE, etc., performance was my primary concern. I mostly write application software and android apps, so things like disk I/O, memory and CPU utilization matter.

With these things in my mind, I was seeking a simple way to do a quick benchmark of various distros to find out what suits my needs. Thats when this idea came to me – Almost all distros come preloaded with a general-purpose scripting language called python which could be summoned for the job. I took “time taken in milliseconds” to do 5 basic tasks to decide on the performance. These tasks were:

1. Zip-test: A test to compress a large file to a zip file, and extract in back in python code.
2. Random-test: Generate 5 million random numbers and round them to zero decimals.
3. I/O test: Write a unicode string ‘spam and eggs’ 5 million times to a text file, and read them back.
4. CSV test: Generate a CSV spreadsheet with 5 million rows in it, and read them back.
5. Bandwidth test: Download the 800K akonadi RPM file from IIT Kanpur’s website.

(The last one was just for the sake of completeness. Internet bandwidth depends more on other parameters such as your ISP limits, time of the day, latency, etc. rather than the disto you are using).

The contestants in the arena were:

1. openSUSE 12.2
2. Debian 7 (GNOME version)
3. Fedora 18
4. Ubuntu 12.04 LTS
5. Xubuntu 12.04

My previous experience with Ubuntu made be biased towards it, but the tests showed me how wrong I was. Also, from what I’d read in most blogs on the Internet, KDE was one of the most bloated distros ever. However, in my tests KDE on openSUSE topped the race in most parameters, though the credit should go to openSUSE for optimizing the KDE. The second spot saw Fedora and Debian fighting amongst them, while Ubuntu (that I was presently using) fared the worst! Here are the detailed results:

Test	zt1 (compress)	zt2 (archive)	random	io	csv	bandwidth
debian1	7622	2753	9142	5732	12784	118978
debian2	7724	2752	9161	5287	12112	92386
fedora18_1	8605	4168	6287	5972	12750	0
fedora18_2	7762	4164	6419	6160	12818	0
openSUSE1	9001	2313	5915	5904	12715	115999
openSUSE2	7253	2245	6035	5935	11492	71257
precise1	9649	5012	8807	6846	13552	78560
precise2	10555	4434	8890	7229	13455	48286
xubuntu1	13305	4827	8954	5839	14676	58538
xubuntu2	10826	4760	8934	7908	12802	52768

*Above figures represent time-taken in milliseconds

The bandwidth test could not be performed on fedora as the Live CD did not have the wget package installed, but I did not need that to get a general idea of things.

My Conclusion: openSUSE wins the race, though Debian-7 and Fedora-18 are also fine distros for performance.

Here is the entire python benchmarking script that you may customize as per your requirement (change initializing variables in the script and run by issuing “python benchmark.py” from the shell):

 

import sys,time,os
import zipfile,random,csv
import subprocess
from zipfile import ZipFile,ZIP_DEFLATED
from random import random

##Variables
#This should be initially present before running the program:
test_name=’debian1′
file_to_extract = “VirtualBox.exe” #”dotnetfx35.exe”
##

def timer(test):
t1=time.time()#.clock()
elapsed=0
#
if test == “ziptest1”:
ziptest1()
elif test == “ziptest2”:
ziptest2()
elif test == “randomtest”:
randomtest()
elif test==”iotest”:
iotest()
elif test==”csvtest”:
csvtest()
elif test==”bwtest”:
bwtest()

elapsed=time.time()-t1
timetaken=round(elapsed*1000,0)
print test,timetaken,”msecs.”
return timetaken

def ziptest1():
myzip=zipfile.ZipFile(‘t.zip’,’w’,ZIP_DEFLATED) #ZIP_STORED
myzip.write(file_to_extract)
myzip.close()
return

def ziptest2():
os.remove(file_to_extract)
myzip=zipfile.ZipFile(‘t.zip’,’r’)
myzip.extractall()
myzip.close()
return

def randomtest():
for i in range(1,5000000):
r = round(random()*100,0);
#print r
return

def iotest():
file=open(‘myfile1k.txt’,’w’)
for i in range(1,5000000):
file.write(u’spam and eggs’)
file.close()
#
file=open(‘myfile1k.txt’,’r’)
s=file.readline()
while (s!=”):
#print s
s=file.readline()
file.close()
return

def csvtest():
file=open(‘myfile.csv’,’w+b’)
#
writer= csv.writer(file,delimiter=’,’,quotechar=’|’)#,quoting=csv.QUOTE_MINIMAL)
for i in range(1,5000000):
writer.writerow([‘spam’,’eggs’,’spam1′])
#writer.close()
file=open(‘myfile.csv’,’r+b’)
reader=csv.reader(file,delimiter=’,’,quotechar=’|’)
for row in reader:
s= ‘,’.join(row)
#reader.close()
#
file.close()
return

def bwtest():
subprocess.call([“wget”,”http://mirror.cse.iitk.ac.in/fedora/releases/18/Fedora/i386/os/Packages/a/akonadi-1.8.1-1.fc18.i686.rpm”,”-O akonadi.rpm”])
#subprocess.call([“wget”,”http://www.google.com”,”-O”,”akonadi.rpm”])
return

if __name__ == “__main__”:
#print time.strftime(“%d-%m-%y %H:%M”, time.gmtime())
#print time.strftime(“%d-%m-%y %H:%M”, time.localtime())
zt1=timer(“ziptest1”)
zt2=timer(“ziptest2”)
random=timer(“randomtest”)
io=timer(“iotest”)
cst=timer(“csvtest”)
bw=timer(“bwtest”)

#Store results to csv file.
file=open(‘results.csv’,’ab’)
writer= csv.writer(file,delimiter=’,’,quotechar=’|’)
#writer.writerow([‘test’,’zt1′,’zt2′,’random’,’io’,’cst’,’bw’])
writer.writerow([test_name,zt1,zt2,random,io,cst,bw])
file.close()
print ‘\n\nWritten results.csv’
#
exit(0)

How to turn your Linux machine into a wifi Access Point

Update on 08-jul-2014: To easily create a wifi AP on ubuntu and other distros, use the hotspotd daemon – Opensource, available on github.

To install:

wget https://github.com/prahladyeri/hotspotd/raw/master/dist/hotspotd-0.1.4.tar.gz
tar xvf hotspotd-0.1.tar.gz
cd hotspotd-0.1/
sudo python setup.py install

---

Few weeks ago I stumbled upon the challenge of creating a wifi access-point on my ubuntu 12.04 linux machine. Whilst I knew about ubuntu’s built-in wifi hotspot feature that works in adhoc mode, it was pretty much useless to connect to my new Android smartphone since it did not support the ad-hoc wifi mode. Most phones these days only support the infrastructure mode (a.k.a access-point mode), and in fact, they won’t even detect devices running on ad-hoc mode. After doing some research, I gathered this simple (though a bit lengthy) set of steps to turn your linux machine into a wifi access-point:

Pre-requsites:

1. Ability of your wireless card to support infrastructure mode. To find out:

(i) Find your kernel driver module in use by issuing the below command:
lspci -k | grep -A 3 -i “network”
(In my case, the driver in use was ath9k)
(ii) Now, use the below command to find out your wifi capabilities (replace ath9k by your kernel driver):
modinfo ath9k | grep ‘depend’
(iii) If the above output includes “mac80211” then it means your wifi card will support the AP mode.

2. Hostapd software: Hostapd is the standard linux daemon that will be used to create your access-point.

3. Dhcp software: Even after hostapd creates the AP and your device detects it, you will still need a  dhcp server to assign a dynamic ip-address to your AP client. (unless you are assigning static address to each device)

4. Iptables: In order to share internet on your AP clients through wifi (a.k.a reverse-tethering), you will have to setup a NAT (Network Address Translation), so that your linux machine, acting as a middleman transfers the internet packets to and from your AP client and the internet modem card (typically ppp0).

EDIT: As of 06-Jul-2013, I’ve developed a python program with GTK+ front end called ‘pyforward’ which automates the below procedure for you. You can find it here: https://sourceforge.net/projects/pyforward/

Procedure:

1. Install the hostapd package. On ubuntu:
sudo apt-get install hostapd

2. Install Dhcp server. On ubuntu:
sudo apt-get install dhcp3-server

3. Make sure that packet forwarding is turned on. This means that your computer is able to forward request of connected clients to other devices, which in my case happened to be from wlan0 to ppp0. (forwarding is different than sending and receiving packets). To enable packet forwarding, issue the following linux command:

sysctl -w net.ipv4.ip_forward=1

To make the change permanent, make sure that the below line is uncommented in your /etc/sysctl.conf file. If not, change it and restart your machine:

        net.ipv4.ip_forward=1

4. The next step is to set up your dhcp. First, decide the subnet and ip-address range in which your virtual AP will sit and your clients will be automatically assigned using dhcp. In my case, I used the subnet 192.168.5.0. My virtual AP is assigned 192.168.5.1 and each connecting wifi device gets an IP in range of 192.168.5.3-45. In order to set the rule, add this block to your /etc/dhcp/dhcpd.conf:

subnet
192.168.5.0 netmask 255.255.255.0 {
interface “wlan0”;
# — default gateway
option routers
192.168.5.1;
# — Netmask
option subnet-mask
255.255.255.0;
# — Broadcast Address
option broadcast-address
192.168.5.255;
# — Domain name servers, tells the clients which DNS servers to use.
#option domain-name-servers
#10.0.0.1, 8.8.8.8, 8.8.4.4;
option time-offset
0;
#range 10.0.0.3 10.0.0.13;
range 192.168.5.3 192.168.5.45;
default-lease-time 1209600;
max-lease-time 1814400;
}

5. Now that packet forwarding and dhcp are set up, we have to create a NAT (Network Address Translation) table using iptables. Please note that if you are using any other controlling software on top of iptables such as ufw firewall or firestarter, then you have to manage the NAT yourself. In that case, there is no need to follow this step:

Create a file called iptables_start in your home folder using gedit or nano and add the below contents to it:

#!/bin/bash
            #First, delete all existing rules
iptables –flush
iptables –delete-chain
iptables –table nat –delete-chain
iptables –table nat -F
iptables –table nat -X

#Allow incoming – already established connections:
iptables -A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT

#Allow outgoing on tcp80, tcp443, udp53
iptables -A OUTPUT -p tcp –dport 80 -j ACCEPT
iptables -A OUTPUT -p tcp –dport 443 -j ACCEPT
iptables -A OUTPUT -p udp –dport 53 -j ACCEPT

#NAT Forwarding for wifi access point
iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE
iptables -A FORWARD -i ppp0 -o wlan0 -j ACCEPT -m state –state RELATED,ESTABLISHED
iptables -A FORWARD -i wlan0 -o ppp0 -j ACCEPT

#Block all incoming & outgoing traffic after that
iptables -A INPUT -j DROP
iptables -A OUTPUT -j DROP

Please note that some of the above rules are customized according to my particular requirement. For eg: I’m allowing outgoing traffic on only tcp80, tcp443 & udp53 ports. Your requirements may be different. Only the NAT forwarding part is what you should be concerned about. Also note that here ppp0 refers to the interface that connects to the internet using modem, and wlan0 is your wifi AP interface that connects to client devices that need internet access.

Your particular interfaces might have different names. You may query all interfaces by using ifconfig command to be sure of them.

Once you create the above file, make it executable using a command like:

sudo chmod +x /home/user_xyz/iptables_start

Now run it by issuing “sudo ./iptables_start” in your home folder. Each time you have to make any changes to your firewall, you may edit and run this file. To test your new iptables rules, issue the command “sudo iptables –list”. It will list your newly added rules.

Once you run this however, the iptables rules are set, but they are not stored permanently. To start these rules each time your computer starts, do the following:

1. Save existing rules to a file using “sudo iptables-save > /home/user_xyz/iptables.rules”.
2. Copy the iptables.rules file to your /etc folder.
3. Now, in order for your computer to load rules from iptables.rules when your network interfaces turn up, create a file called /etc/network/if-pre-up.d/iptablesload and add below script to it:
#!/bin/sh
iptables-restore < /etc/iptables.rules
exit 0

4. Make the above file executable by running “sudo chmod +x /etc/network/if-pre-up.d/iptablesload”

Test these settings by restarting your computer and issuing “sudo iptables –list”. This will list your current firewall rules.

6. Now that all hard work is done, you are ready to start your virtual AP. First, create a configuration file for the hostapd called hostapd.conf. It can be located in either /etc or your home folder:

#sets the wifi interface to use, is wlan0 in most cases
interface=wlan0
#driver to use, nl80211 works in most cases
driver=nl80211
#sets the ssid of the virtual wifi access point
ssid=YourAPName
#sets the mode of wifi, depends upon the devices you will be using. It can be a,b,g,n. Setting to g ensures backward compatiblity.
hw_mode=g
#sets the channel for your wifi
channel=6
#macaddr_acl sets options for mac address filtering. 0 means “accept unless in deny list”
macaddr_acl=0
#setting ignore_broadcast_ssid to 1 will disable the broadcasting of ssid
ignore_broadcast_ssid=0
#Sets authentication algorithm
#1 – only open system authentication
#2 – both open system authentication and shared key authentication
auth_algs=1
#####Sets WPA and WPA2 authentication#####
#wpa option sets which wpa implementation to use
#1 – wpa only
#2 – wpa2 only
#3 – both
wpa=3
#sets wpa passphrase required by the clients to authenticate themselves on the network
wpa_passphrase=your_passphrase
#sets wpa key management
wpa_key_mgmt=WPA-PSK
#sets encryption used by WPA
wpa_pairwise=TKIP
#sets encryption used by WPA2
rsn_pairwise=CCMP
#################################
#####Sets WEP authentication#####
#WEP is not recommended as it can be easily broken into
#wep_default_key=0
#wep_key0=qwert    #5,13, or 16 characters
#optionally you may also define wep_key2, wep_key3, and wep_key4
#################################
#For No encryption, you don’t need to set any options

In above script, replace YourAPName with ssid of your AP. This will be detected when you run a scan on your device. Similarly, replace your_passphrase with the actual password you wish to set up.

7. The last and final step is to create an AP script and run it. Create a file called AccessPoint in your home folder:

#!/bin/bash
ifconfig wlan0 up 192.168.5.1 netmask 255.255.255.0
sleep 5
###########Start DHCP, comment out / add relevant section##########
#Doesn’t try to run dhcpd when already running
if [ “$(ps -e | grep dhcpd)” == “” ]; then
dhcpd wlan0 &
fi
###########
#start hostapd
sleep 2
hostapd ~/hostapd.conf 1>/dev/null
killall dhcpd

8. Make above file by issuing “sudo chmod +x ~/AccessPoint”. Now execute this script in the terminal by “./AccessPoint” and keep it running. If all goes well, your devices should now be able to scan and connect to your new virtual AP.

References:

If for some reasons, all doesn’t go well, then here are some links that can help you:

Hostapd:

Hostapd : The Linux Way to create Virtual Wifi Access Point

http://ubuntuforums.org/showthread.php?t=151781
Iptables basics:
https://help.ubuntu.com/community/IptablesHowTo

NAT and port forwarding:
http://ubuntulinux.co.in/blog/ubuntu/nat-configuration-with-iptables-in-ubuntu/
http://www.howtoforge.com/nat-gateway-iptables-port-forwarding-dns-and-dhcp-setup-ubuntu-8.10-server

(If you have some specific issue, you can always get back to me).

Some free tools to help you develop better .NET Solutions!

1. DebugView: DebugView is a free tool that lets you view the debug output written by applications on the windows operating system. You might be aware that many Try/Catch blocks in .NET applications use the Debug.WriteLine statement to make note that an error or exception occurred. DebugView displays an output log of all such diagnostics written by your application thus making the task of tracking errors much easier. In fact, on live or production systems, it is not always possible to run your application in debug mode. In such cases, DebugView comes very handy as it lets you see the exceptions written by your .NET application without disrupting any user activity.

http://www.sysinternals.com/Utilities/DebugView.html

2. FxCop: For those who wish to follow the industry best-practices while coding .NET applications, this free tool from Microsoft is a great boon. FxCop helps you with code-review by analyzing your .NET assembly and highlighting several programming and design issues such as:

• Not following proper naming-conventions for constants and variables.
• Improper commenting of code.
• Inappropriate use of public variables.
• Using obsolete or irregular keywords like goto.
• Performance issues (use of excessive locals, declaring unused parameters, etc.)
• Security issues (not specifying marshalling for pinvoke string arguments, etc.)
• Design issues (Not strong-naming an assembly, hiding base-class methods, etc.)

To see a complete list of FxCop rules with an in-depth discussion and example, see msdn:

http://msdn.microsoft.com/en-us/library/bb429379(v=vs.80).aspx

3. StyleCop: StyleCop is another opensource tool for static code analysis. It analyzes the C# source-code unlike FxCop which analyzes the assembly. It thus applies a different set of rules from FxCop. These rules fall in seven basic categories:

• Documentation
• Layout
• Maintainability
• Naming
• Ordering
• Readability
• Spacing

4. CLR Profiler:CLR Profiler is an excellent tool to help you identify performance issues in your .NET application. Many a times it so happens that you notice your application is performing poorly or consuming a lot of memory, but could not put your finger on the exact function or class in code that is causing this. This tool does exactly that! It creates a call graph analysis when your .NET application is run, along with the behaviour of the Grabage-Collector. CLR Profiler is a free download from Microsoft:

Download CLR-Profiler from Microsoft

5. NANT: NANT is a free and opensource tool for automating the software build process.  NANT is specifically targetted at building .NET solutions. It automatically fetches the dependency tree and builds them in order for your .NET solution. NANT is free software:

http://nant.sourceforge.net/

6. Subversion/TortoiseSVN: Subversion is an excellent opensource collaboration software used by several large firms to manage inhouse development in large teams. Subversion is the server component that provides a central repository to store your .NET code, while there are several Subversion clients (TortoiseSVN being the most popular) that integrate with your windows-shell and provide SCM features and tools to:

• Check-out, browsing and check-in of code in the SVN repository.
• View change log and changes made by other users to the repository.
• Compare or merge your local code with the base version in repository.
• Resolve conflicts (if any) with the base version by gradually accepting/rejecting the changes.

Both Subversion and TortoiseSVN are 100% free and opensource:

http://subversion.tigris.org/

http://tortoisesvn.net/downloads.html

7. Redgate Reflector: Not many people know that the EXE or DLL compiled from your Visual-Studio IDE is prone to reverse-engineering if you ship them as it is. That is because, a .NET assembly (EXE/DLL) is not directly compiled to the low-level machine code, but rather, into CIL (Common Intermediary Language) to achieve architecture independence and reduce compile time. One such industry-standard tool to reverse engineer a .NET assembly is the Redgate Reflector. Reflector takes the .NET assembly as input and regenerates the entire source-code and solution in the language of your choice (VB/C#/F#/etc). Though Reflector is a commercial product, you can use the free demo version to decompile .NET assemblies (but with some limitations):

http://www.reflector.net/

8. AnkhSVN/SharpSVN: AnkhSVN is yet another opensource tool which is managed by Collabnet, the creators of  SVN Server itself. It seamlessly fits into the Visual Studio IDE and allows you to check-out and commit code to your SVN repositories. Also, the api used by SVN called SharpSVN is also an opensource .NET 2.0 library. You can add its reference to your .NET project if you need to automate code check-outs and commits from your application itself:

http://sharpsvn.open.collab.net/

9. Windows SDK Tools: You can find a bunch of usefull stuff already installed in your Windows SDK directory itself! These are typically installed in the below folder:

C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\

sn.exe is the strong-name utility that you can use to strong-name your .NET assemblies.

gacutil.exe is used to install/uninstall an assembly from the .NET GAC (Global Assembly Cache).

wsdl.exe automatically generates .NET code for xml web-service clients to connect to a web-service.

svcutil.exe pregenerates C# serialization code that is required for types that can be serialized using XML Serializer.

ildasm.exe is a disassembler that generates CIL code from .NET assemblies.

References:

http://en.wikipedia.org/wiki/.NET_Reflector

http://en.wikipedia.org/wiki/StyleCop

http://en.wikipedia.org/wiki/CLR_Profiler

http://en.wikipedia.org/wiki/NAnt

http://en.wikipedia.org/wiki/AnkhSVN

How to create an FTP client in C#

Creating an FTP client in a CLR compliant languge such as C# is truly a breeze compared to native languages like C or C++. The .NET Framework class library hands you the useful classes of System.Net on a platter which you can use to write your FTP functionality without worrying about things like memory allocation, protocol handling, etc. The .NET API does that on your behalf!

Before writing the code, it is useful to sit down and give a thought to your program design. Do you want to create a library to reuse in a larger project, or just a Windows app that acts as an FTP client? If you are a student doing some test project, I bet its the latter you will prefer! However, object-oriented programming and the concept of object-reuse dictate that you create a library first and then proceed with building your GUI, though it takes a little bit of extra effort.

In this tutorial I’ll briefly explain the functionality of libftp, an FTP library I have created in C# which I can seamlessly integrate into any application I develop or distribute it as a class-library/DLL. I also built a proof-of-concept Windows Forms application along with, to ensure the new functionality is working well. The library and application are available as open-source project on codeplex. You can browse through the code, understand it and straight away plug-it in your .NET application 100% free of cost.

The workings of this library is simplicity itself. There are just two classes – ftper class that acts as an interface to this library. It exposes public methods like connect(), disconnect(), browse(), addFilesToDownloadQueue(), etc. Here are some implementation details:

public bool isProcessing()
{
    return _threadRunning; //check if a thread is running for up/download
}

public List<ftpinfo> connect(string host, string username, string password)
{
    return ftpobject.connect(host,username,password);
}

public void disconnect()
{
    if (_threadRunning)
    {
        _threadRunning=false;
    }

    int timeout=60; //seconds
    DateTime start=DateTime.Now;
    while(queue.Count==0) //wait till running up/download threads complete.
    {
        if (DateTime.Now.Subtract(start).Seconds>timeout)
            break;
    }
}

public List<ftpinfo> browse(string path)
{
    return ftpobject.browse(path);
}

The second class, ftp is the core class which does the actual ftp stuff by connecting to the remote-host and transferring files. It is the underlying class for ftpobject in the above code.  It also exposes some events like uploadComplete and downloadComplete that our client can subscribe to, so that he/she can get notified of these events.

internal ftp() //hide constructor from the outside world.
{
context = SynchronizationContext.Current;
}

//tests connection and browse to home-directory
 public List<ftpinfo> connect(string host,string username,string password)
 {
 this._username=username;
 this._password=password;

//FtpWebRequest.Create(host); //test connect;
 context = SynchronizationContext.Current;

return browse(host);
 }

//public bool exists(string remotefilename)
 //{
 //    return true;
 //}

/// <summary>
 /// Fetch all files/folders in this directory and return the ftpinfo array.
 /// </summary>
 public List<ftpinfo> browse(string path) //eg: "ftp.xyz.org", "ftp.xyz.org/ftproot/etc"
 {
 FtpWebRequest request=(FtpWebRequest)FtpWebRequest.Create(path);
 request.Method=WebRequestMethods.Ftp.ListDirectoryDetails;
 List<ftpinfo> files=new List<ftpinfo>();

//request.Proxy = System.Net.WebProxy.GetDefaultProxy();
 //request.Proxy.Credentials = CredentialCache.DefaultNetworkCredentials;
 request.Credentials = new NetworkCredential(_username, _password);
 Stream rs=(Stream)request.GetResponse().GetResponseStream();

OnStatusChange("CONNECTED: " + path, 0, 0);

StreamReader sr = new StreamReader(rs);
 string strList = sr.ReadToEnd();
 string[] lines=null;

if (strList.Contains("\r\n"))
 {
 lines=strList.Split(new string[] {"\r\n"},StringSplitOptions.None);
 }
 else if (strList.Contains("\n"))
 {
 lines=strList.Split(new string[] {"\n"},StringSplitOptions.None);
 }

//now decode this string array

if (lines==null || lines.Length == 0)
 return null;

foreach(string line in lines)
 {
 if (line.Length==0)
 continue;
 //parse line
 Match m= GetMatchingRegex(line);
 if (m==null) {
 //failed
 throw new ApplicationException("Unable to parse line: " + line);
 }

ftpinfo item=new ftpinfo();
 item.filename = m.Groups["name"].Value.Trim('\r');
 item.path = path;
 item.size = Convert.ToInt64(m.Groups["size"].Value);
 item.permission = m.Groups["permission"].Value;
 string _dir = m.Groups["dir"].Value;
 if(_dir.Length>0  && _dir != "-")
 {
 item.fileType = directoryEntryTypes.directory;
 }
 else
 {
 item.fileType = directoryEntryTypes.file;
 }

try
 {
 item.fileDateTime = DateTime.Parse(m.Groups["timestamp"].Value);
 }
 catch
 {
 item.fileDateTime = DateTime.MinValue; //null;
 }

files.Add(item);
 }

return files;
 }

The reason I’ve kept the constructor method ftp() internal is that we don’t want our client creating these objects as they form the core ftp functionality that only our library is supposed to use. By using the keyword “internal”, C# compiler understands that we don’t want this consturctor to be visible outside our assembly.

List of development tools for Linux platform – IDEs, Compilers, etc.

One of my most recent endeavours was to set up my old linux machine for development. I chose the Debian Squeeze(6.1.5) distro because of its reputation for stability and also its minimalistic approach towards installing packages, and both these features were quite welcome. Having said that, the tools listed here can be downloaded and installed on any distro, in case they are not already included in the installation CD/DVD. Based on my development experience with these tools, this list is frequently updated with new information:

1. GCC (GNU Compiler Collection): The GCC toolchain is considered a sin-qua-non of any linux developer’s toolbox. In fact, the linux kernel itself relies on several libraries provided as part of GCC. The debian squeeze 6.1.5 includes GCC 4.4 which is pretty stable. I’ve compiled several programs in C/C++ without any issues. You can choose to install specific packages such as gcc4.4 for C, or g++4.4 for the C++ language.

Only issue is that you should not install the OpenJDK package (gcj) of the GNU collection. The reason being that it will clash with the Sun/Oracle Java version which is very much preferred if you are into Android development or use other Java features such as Swing or AWT.

GCC is one of your core tools. Whether you use Eclipse or Netbeans to code your C/C++ programs, whether you use the QTcreator or Glade for designing interfaces, GCC is one good toolchain that most IDEs rely on to build your application.

2. Java/Netbeans: You can download all the java editions for linux including Java ME/EE, documentation & samples and also the Netbeans package from here. I specifically chose the combined Netbeans+JDK7.5 to avoid getting Netbeans separately.

3. MonoDevelop: One of the most important tools that allows me to leverage my Microsoft VB.NET/C# skills on linux is the MonoDevelop IDE. Except for Microsoft proprietary classes such as System.Windows.Forms, all your .NET code is 100% portable to linux through the Mono platform. The only support that MonoDevelop lacks as of now is the ability to design ASP.NET web pages. I believe this limitation is going to be overcome pretty soon.

4. Eclipse: Eclipse was essential for me as I wanted to develop Android apps too which is easier using the Eclipse IDE. The debian squeeze CD comes with version 3.5.2 of eclipse package, whereas I wanted to try the latest Juno version, so I got it from the eclipse site. However, the eclipse SDK for linux is a tar package and not an installer package. This means you have to extract this tar separately in a folder and use eclipse from there. If you do so, ensure that the ..JDK/bin is set in your $PATH variable, so that you can invoke java from the command line. If not, then add it by editing the .profile file in your home directory (/home/xyz, etc..).

5. Android Suite: The android developer site contains complete information about how to download and install the android sdk, along with the system images for the particular android platforms for which you wish to develop. Just keep in mind that android needs the Sun/Oracle java sdk and not the OpenJDK provided by GNU to write android programs.

6. Glade: Glade is an excellent RAD tool to rapidly develop professional-looking user interfaces for your applications. Glade is based on the Gtk+ toolkit that forms the core of the GNOME desktop. Glade basically generates an xml file (similar to the XAML generated in visual-studio) that can be used in many languages such as  C, C++ and Java. However, since the Gtk+ framework is built in C, it is advisable to build your GNOME applications in C for performance benefits. Glade can also be integrated with Anjuta, a full fledged IDE to develop GNOME applications that includes full toolchain integration and debugging support.

7. QTcreator: Based on the extensive QT framework, QTcreator is to KDE desktop what Glade is to GNOME. However, don’t feel crippled because of this as everything is “mix-and-match-able” in the linux world. QT apps can run on GNOME, as do Gtk+ apps run on KDE desktop. Since QT framework is written in C++, the language is well suited to write QT apps. Qtcreator can also be integrated with KDevelop, the equivalent of Anjuta IDE on KDE.