Sunday, September 18, 2011

The story of "How Postman got featured in The Mumbai Mirror"


Its been around 4 months since Postman was available on the Android Market. With around 962 downloads and around 30 (5 star) ratings it was doing "Well".But who wants their product to do just well? "Awesome" ,"Kickass" ,"The most downloaded application" now that's what I want for Postman.

So here is the story of "How Postman got featured in The Mumbai Mirror"

It was Monday morning around 3.30 AM when I tweeted about Postman to one of Mumbai Mirror's reporters "Kunal Pradhan". Within minutes he replied back to my tweet indicating to get in touch with "Jaison Lewis ".I then spent around 2 minutes trying to search for his twitter handle and redirected my tweet to him. After this I called it a night.

On Tuesday around 9.30 AM Jasion replies back to my tweet stating he was "Looking into my android app".I was happy but I had no clue what was in store for me the next day.

It was a normal Wednesday (14th September 2011).I got up around 9.30 AM and as usual I scanned The Times Of India and Mumbai Mirror for top stories.When I reach page 28 of The Mumbai Mirror (Tech Section) I was ecstatic ,excited ,and for a moment couldn't believe my eyes. To cut to the point Postman was featured in the Mobile app section.














It was a proud day in my life and a big milestone for Postman.

Till next time ...

Tuesday, July 12, 2011

Browser internals when it comes to "Caching"

So as mentioned in one of my previous articles My take on Optimizing HTTP Caching for your web application there are two response headers that any browser depends upon for caching a file i.e Last-Modified time and (max-age or Expires)

So lets deep dive into this.In the demo below I have used the following:
When you browse the homepage notice that for any static file (in this case I have picked up a css file) there is always a Last-Modified and an Expires.











There are multiple ways you could browse a website .I have broken them down below:

1) The General way  : 
This is the mostly used way.I say this because in general a user will keep clicking links and browse the website. Here for any GET requests for a static file 
  • The browser will first check if the file is present in its cache. 
  • If the file is present , the browser will check if file is not stale. It does this by verifying that the max age or expires value for that file has not been reached . 
  • If the file in the cache is stale, it will make a request to the server using a conditional GET.
This is how the headers look when the browser makes a conditional GET










Notice the browser sends a If-Modified-Since header in the request .The value of which is the Last Modified timestamp it got when it last received the file from the server. The server on receiving the conditional GET request will either reply with the new file content or a 304 "Not Modified".


2) Forcing the browser to make a conditional GET
This can be done by hitting F5 on the page.Notice that when this is done the browser forces a conditional GET.








3) Forcing the browser to get the file from the server
This can be done by hitting a CTRL+F5 (Windows) or COMMAND+SHIFT+R (MAC). When this is done notice that the browser sends a request without a If-Modified-Since header.Hence this request will hit the server and not get served from the browser's cache or the proxy cache if any.











Till next time ...

Sunday, June 12, 2011

Sick of PuTTY ?? Try ZOC

Are you looking for an alternative to PuTTY? Then try ZOC http://www.emtec.com/zoc/index.html

ZOC is an awesome SSH/telnet client and terminal emulator for Windows and Mac OS X. Ive been using it for over a year now and its been brilliant in every sense.

Till next time...

Wednesday, June 1, 2011

Keep it simple SILLY !!!!

The first version of Postman did well. 218 downloads in just 2 months and literally everyone around me knew about Postman. It was a HIT. But after talking to some of the users I did learn something. That something  made me change the entire way Postman works and probably my perspective of building any android application that targets an audience such as "The Entire World".

Postman v1.0 was based on user defined filters.Letting the user define custom filters,I thought would be awesome and powerful but in fact I did notice that most of the users liked the application to do the magic of filtering out spam rather than he or she defining custom filters.Also more importantly I encountered a different section of users who got confused with so many options and set wrong filters and for them the application did not work as expected.

So after giving it allot of thought I realized "lesser the configuration ,would in turn lead to lesser confusion and probably a scope for a wider user base" .

I then decided to revamp Postman and make it intelligent enough to detect the SPAM SMS itself .Now for the end user it would just be a one step process. i.e Install Postman and let it do its magic of filtering out Spam SMS.

Thus the phrase - Keep it simple SILLY !!!! 


Till next time... 

Friday, April 1, 2011

The internal working of Postman

From the diagram below its quite clear how Postman works:




  • Postman intercepts all messages that come to the phone.
  • When a message arrives it gets scanned to see if it matches any one of the multiple filters set by the user.
  • Every filter has a set of actions associated with it.
    • When a match occurs, an action is added to the Postman Action Queue
    • If no match occurs, postman just forwards the message to the phone's default messaging application.

Postman has everything from 
  • android services 
  • activities 
  • broadcast receivers
  • multi-threading 
  • single action queue
  • customized views
  • notifications etc etc.....

The underlying data storage of Postman is a SQLite database.Ive attached the diagram of it below:




The entire code base of  Postman is up at  https://github.com/reinwald/Postman .Ive tried my best to make it as easily pluggable as possible. If you guys want to add custom actions when filter matches occur its as simple as following some contracts defined ,then adding the custom action object to the action queue and letting postman handle the rest.


Till next time ...

Postman beta launches today

So after 2 weeks of working 18 hours a day, the beta of my android application "Postman" http://itsmrpostman.appspot.com  launched today.

Postman in short is a personal message assistant for your Android phone. I shall let some of the images do the talking. It works on any android phone running a version greater than 1.6. 




A big thank you to :
  • Sandeep Shetty : The creator of http://owe.to , for putting the seed of thought in my head to create an message filter application
  • Vikrama Dhiman : For his awesome suggestions and feedback.
  • Nadine Pereira : The one who designed Postman's logo and "The best graphic designer ive worked with".
  • My Dad : Who let me use his Android phone to do some USB debugging and testing (Since I don't have an android phone ;) )
Hope you guys like the idea of Postman and try it out soon.

Till next time ...

Saturday, March 26, 2011

One word for the USB debugging feature on the android phone .... "Awesome"

Even though practically everything is possible from the emulator, I wanted to check out the USB debugging functionality in my android phone.It was super simple to setup

  • Enable the USB Debugging option under Applications->Development.
  • Before plugging in your phone you need to install the USB adb drivers ( if your running on windows).These drivers are vendor specific so you can download the appropriate one from here. http://developer.android.com/sdk/oem-usb.html
  • Plug in the phone and hit run->cmd-> adb "deviceid" connect and viola your in..... 

Some points to note
  • In order to debug the attribute android:debuggable="true" in the application tag of the AndroidManifest.xml is mandatory.
  • Access to the SQLite database on your phone directly from adb (like the way it works when using the emulator) is not allowed .If you do try it you would be greeted with a Security error. One workaround to read the database, would be to transfer it to the memory card and then read it locally.

And yeah its as easy as that.So simple but yet so powerful !!

Till next time....

Friday, March 11, 2011

Say hi to your android "Postman"

How often did you tell yourself  "Damn I wish I could block that SMS ?" or "Why does Android NOT come with an option to create Sms's Filters and move them to specific folders."

So say hello to "Postman".  Postman is an application that you would want to install and use as a replacement, instead of the standard boring messaging client that ships with your android phone.


Some of the main features that would make you want to use Postman are :
  • Store messages in appropriate folders
  • Create multiple filters based on the senders numbers or a particular message body and move them to specified folders.
  • Completely block an Sms from even reaching your phone
  • Back and Restore Sms's on any android phone

And for you geeks out there the application will be completely open source https://github.com/reinwald so you could tweak it to your fancy.

An initial very rough mock-up of how the main interface would look



I am just in the initial phase of developing Postman and would be done with a beta version that would hit the android market in a few months.

Do you like the idea of Postman ? and have any suggestions that you would like Postman to do for you when you receive an SMS?  Do let me know by, reaching me at reinwald@gmail.com or post a comment here or even follow my blog .Looking forward to hearing from you...

Till next time...

Friday, February 25, 2011

Searching for a Git Client for your windows machine?

Have you moved to GIT ?

Are you searching for a free GIT client for your windows machine?

If you are like me who use SVN,CVS for version control for various projects and were comfortable with using the respective clients in windows and are searching for the same for GIT then say hi to "TortoiseGit "http://code.google.com/p/tortoisegit/

With a few steps you can install it and it sweetly integrates with windows. Screen shots below


Try it out !!

Till next time...

Wednesday, February 23, 2011

My take on “Optimizing HTTP Caching” for your web application.

One quick way to a snappy and fast loading web page is to add appropriate HTTP headers to your response to ensure that resources are cached, by a browser or proxy. Browsers or proxies can refer to the locally cached copy instead of having to download it again on subsequent visits to the web page.

Let’s quickly dive into the various headers you’ll need to set with the optimal values. The values will defer from app to app depending on how long you may want the browsers or proxies to cache the content.


Expires  or max-age ? What do you use ?

  • Expires
    • The Expires HTTP header was the basic means of controlling caches back in the good old days of HTTP 1.0. It tells all caches how long the data in its local storage is fresh for. After that time, caches will always check back with the origin server to see if a document is changed. Expires headers are supported by practically every cache.
    • Expires header was very useful ,but it had some limitations .One being ,since there’s a date involved, the clocks on the Web server and the cache must be synchronized.
    • Example : "Expires: Sat, 01 Jan 2000 00:00:00 GMT"
  • max-age=[seconds]
    • Introduced in the HTTP 1.1 spec, this is a replacement for Expires but the advantage here is that this directive is relative to the time of the request, rather than an absolute date.
Expires or Cache-Control max-age when set ensures that no request is ever sent out from the browser and the file will be picked up from the local cache (provided other Cache control headers like no-cache ,no-store, must-revalidate, proxy-revalidate are not set) .



What if we cannot predict the lifetime of the page content and want the browser to check with the server on a regular basis if the cached copy is still FRESH. ??

Don’t worry we can do this to with headers like Last-Modified and Etag.
  • Last-Modified
    • This indicates the date and time at which the origin server believes the file was last modified
    • This is a "weak" caching header on which the browser applies a heuristic to determine whether to fetch the item from cache or not. (The heuristics are different among different browsers.)
  • ETag
    • This can be any value that uniquely identifies a resource (file versions or content hashes are typical)
So why do we need to set this?
  • These headers allow the browser to efficiently update its cached resources by issuing conditional GET requests when the user explicitly reloads the page.
  • If the resource has not changed the browser will NOT reply back with the entire file content in the response but just with a header  “304 Not Modified“ indicating to the browser that it can serve the file from its cache since its still fresh.
  • If you rely on ETags ,you may face a problem if you have multiple servers behind a load balancer (configured in a non-sticky environment) since the ETag generation may result in a different key.

What are some other Cache control headers that I can set ??
  • public or private
    • Setting the Cache-Control header to private disables proxy caching altogether for these resources. 
    • If your application relies less on proxy caches for user locality, this might be an appropriate setting.
  • must-revalidate
    • This directive specifies that the cache MUST revalidate with the server that the content is still fresh before serving the file from local cache.

How can I speed up my web application ??
  • Specify one of the two headers Expires or Cache-Control max-age for all cacheable resources. 
  • Specify one of two headers Last-Modified or ETag for all cacheable resources.
  • Set values appropriate to your web application
  • Use a tool like YSlow or Google’s Page Speed to check the end result.



Till next time …

Tuesday, February 15, 2011

Benchmarking Tomcat 6.0.32 and Resin 3.1.3 Pro

So we are in the process of migration from Resin 3.1.3 Pro to Tomcat 6.0.32. So the questions haunting my mind were “What if Resin is actually better than Tomcat”? Will Tomcat handle load? Will it serve static content faster than Resin? . To answer all these questions I configured JMeter and ran a quick test.

What did the test consist of ?
Made a call to the login page of my application which included (6 static calls,1 call to a jsp and 2 to servlets).Configured Jmeter to create 350 concurrent clients. Hence each client made the 9 calls which resulted in a total of 3150 Requests in all.

What was the Raw Result?
While performing the test even though the network traffic on my laptop got real slow, I tried accessing the page served by both resin and tomcat from another pc and it was very responsive. Also monitored the CPU & memory usage on the server and didn't notice any spikes .Tomcat hardly used 350 MB of memory during the load. Check the screen shots below.





What’s my conclusion?
The results are very very close .Resin's throughput (49.9 requests/min) was a bit higher than that of Tomcat (47.7 requests/min). I am assuming it’s because most of the files were static .One could further tweak Tomcat's thread creation to handle more requests and may also think of running apache over tomcat (tried setting it up and its quite simple to configure).But in short if your replacing Resin with Tomcat ,I am sure Tomcat would perform just as well as Resin.

Till next time...

Monday, February 14, 2011

How the hell did that value come in the cookie ??? I swear I never set it !!!!

Cookies have two version i.e Version 0 and Version 1. Version 0 adheres to the original Netscape spec while Version 1 adheres to RFC 2109.According to specs, in Version 0, cookie values "SHOULD NOT" contain white space, brackets, parentheses, equals signs, commas, double quotes, slashes, question marks, at signs, colons, and semicolons. Version 1 cookies on the other hand allow such values but the entire value will be quoted. Eg: "testing:value".

Version 1 cookies are not fully supported by all user agents. By default (if not explicitly specified)cookies are created using Version 0 to ensure the best interoperability. One can explicitly set the version of the cookie by specifying Version="0/1"

I carried out some tests using IE 8, Firefox 3.6, Chrome 9 and JMeter as user-agents and Tomcat 6.0.32 as the application server. The test simply created a cookie storing a url as a value i.e URL=http://reinwaldwarapen.com. Tomcat docs mention that if a value contains illegal characters it would set the version of the cookie to 1 and quote the value. Using fiddler to check the raw request /responses, I confirmed it did just that. When the first request goes out ,it went without a cookie and when the response came back it returned with the cookie set with version 1 (the value was quoted). Now the user-agents especially JMeter and on occasions the browsers mentioned above ,when sending the next request ,it sent the cookie but with a version set to 0.Tomcat on receiving the request ,then treated the version of cookie as 0 and truncated everything after : (colon). Hence when the value of the cookie was read it gave just http instead of http://reinwaldwarapen.com.

Next time you are using cookies keep these points in mind or you might just go crazy thinking "How the hell did that value come in the cookie ???". Or a better option would be to URL or Base 64 encode/decode values in cases where the value contains special characters.


Till next time ...

Wednesday, January 26, 2011

Turned on the Jasper Validation in IntelliJ Idea and facing weird / illogical errors?

So we are in the process of migrating our web application from Resin to Tomcat. The first major problem was that Resin does not use Jasper for compiling the JSP files. So it’s obvious since we were running on Resin ,we had the “Jasper Validation On Build” option in the Idea IDE turned off.

Since Tomcat uses Jasper for its compilation we needed to turn this option on. After doing that what you think happened?? Approx 400 compile issues!!

So we fixed most of the issues and we came down to just 10 errors which were all caused because of a package that could not be resolved. Thinking this was some setting problem we dug through this for nearly 2 days and we nearly pulled our hair out .I say nearly pulled our hair out because Idea IDE was acting weird and completely illogical .We changed the name of the package and it suddenly detected the new package and all the errors disappeared .What was even worse the other packages which was on the same level were getting detected .If we had say “com.rein.test” and “com.rein.newtest” ,the latter was found . “com.rein.test” was physically present but to Idea it just could not find it. Ok let me stop and tell you how we knew it was an IntelliJ Idea issue.

We used Jasper libraries in Tomcat to do the validation and compilation via an ant script and voila it worked .Build Successful !! is what we got. I’ve added the ant script below so that you to don’t go through the same “What the hell is wrong here?” state

// Begin Ant Script

<project name=”JSP Compilation”>
<property name=”tomcathome.dir” location=” “/>
<property name=”webapps.dir” location=” “/>
<target name=”jsp_jasper_compilation”>
<taskdef resource=”org/apache/catalina/ant/catalina.tasks”>
<classpath>
<fileset file=”${ tomcathome.dir}/bin/tomcat-juli.jar”/>
<fileset file=”${ tomcathome.dir}/lib/jasper.jar”/>
<fileset file=”${ tomcathome.dir}/lib/jasper-el.jar”/>
<fileset file=”${ tomcathome.dir}/lib/el-api.jar”/>
<fileset file=”${ tomcathome.dir}/lib/jsp-api.jar”/>
<fileset file=”${ tomcathome.dir}/lib/servlet-api.jar”/>
<fileset file=”${ tomcathome.dir}/lib/catalina-ant.jar”/>
</classpath>
</taskdef>
<jasper uriroot=”${ webapps.dir }”
outputDir=”${webapps.dir }/WEB-INF/compiledJSP” failonerror=”true”/>
</target>
</project>

// End Ant Script

I am not saying don’t use IntelliJ idea to validate your Jsps. But if you ever face weird /illogical errors, don’t break your head but instead try the ant script.
Till next time…

Monday, January 17, 2011

Storing and Sharing Sessions among standalone Tomcat instances

Are you migrating from Resin to Tomcat?

Were you using a database like mysql to store sessions?

Did you configure Resin to <always-load> and <always-save> sessions after each request and trying to figure out if there is a way in Tomcat to configure the same?

Are you trying to figure out how to configure a backup database for High Availability (HA) and Failover?

Well if any one (or all) are applicable to you ...then this post will be very helpful

First things first you need to know and which is clearly mentioned in the docs Persistent Manager (the guy who does all the magic of storing sessions based on the configuration in the server.xml) in Tomcat has not been thoroughly tested, and should be considered experimental!. Some more hard facts which are NOT mentioned in the docs are and which you may have thought were possible:

# Are you trying to configure the Persistent Manager to <always-load> and <always-save> sessions after each request? Well then try no more cause there simply no way "yet". You may read some articles mentioning this is possible by setting maxIdleSwap='0' but it won’t achieve the desired result. The reason being PersistentManagerBase backups all sessions in batches (processMaxIdleSwaps(),processMaxIdleBackups() are called by the background thread). You might try to set maxIdleBackup and maxIdleSwap to 0. But even then sessions are only stored by the background thread (running every 10seconds by default IIRC) and not directly at the end of the request.

# Are you trying to figure out how to configure a backup database for High Availability (HA) and Failover? Well then stop trying. There is no way to configure the Persistent Manager to use a backup database which will be used in case the primary one kicks the bucket.

So whether you are going the sticky sessions or non-sticky sessions way, the Tomcat implementation of a Session Store (Persistent Manager) does not provide a reliable failover / HA solution for standalone instances of Tomcat.

Is there another solution? Well Yes and No.

Yes, if you’re going the Sticky Session approach. http://code.google.com/p/memcached-session-manager/#How_does_it_work? .I also had a conversation with the developer of the memcached-session-manager and he confirmed that it’s very stable and thoroughly tested. They are running www.tchibo.de (and tchibo.ch, tchibo.at etc.) with memcached-session-manager with sticky sessions in production (Tchibo is a German chain of coffee shops and caf├ęs).

No, if you’re going the Non-Sticky approach .There is the memcached-session-manager version for non-sticky sessions https://github.com/magro/memcached-session-manager/tree/nonsticky-sessions. But according Martin Grotzke it’s not for production and still has issues like concurrent session modification,SPOF etc etc.. which still needs to be fixed.

Till next time....

Sticky Sessions Or Non-Sticky Sessions ? Keep this in mind

I had an interesting discussion with Martin Grotzke (the guy creator of the memcached-manager for Tomcat) and during that discussion he raised some good points when configuring standalone instances of tomcat/resin.

In the case of non-sticky sessions, a session is never loaded from the memory of the app server. The session will always have to be loaded from store (in our case a database) for each request. Since the sessions are non-sticky, requests can go to any Resin/Tomcat instance. Concurrent requests can go out from the client’s browser (multiple windows or tabbed-browsing).In such a case some kind of “session locking” needs to be done since multiple requests may access/modify the same session object. There is no clear documentation in Resin if they have implemented any session locking but Tomcat does not have any such session locking when configured as a standalone instance. Period!! .

In the case of sticky-sessions, a request will always go to the same Resin/Tomcat instance ( which the session was first created on).But we still need to handle concurrent requests accessing the same session object. One way this would be achieved is by ensuring we do all the session value modification (setAttributes) in a synchronized block. But we all know it would affect performance when handling multiple concurrent requests.

Till next time…..