crawler4j example | java web crawler

Why to write the whole thing from the scratch if the same thing is already available ? Better to build on top! Isn’t it ?

Framework says : Concentrate on your objective rather then supporting things needed to accomplish the objective. Framework provide the facility and tools and expect the programmer to build the things on top of it.

crawler4j is a framework to that provide multi-threaded crawler with some extra features like Logging, customization’s, crawling password protected sites etc.

Here is a small example of starting with crawler4j.
Build env :Maven

Continue reading

html input hidden field poor usage can lead to major Web vulnerability

E-commarce, Online Premuium Service providers and many others uses payment medium to get the payment using CC,Net Banking etc. Payment Gateways are the Integration layer between the web application and the Banking service layer. These gateways are mostly the loosely coupled,third party source code provided by the authorized Payment Gateway Providers. Ex of Payment gateways are paypal,authorize,securepay.

Once i found a serious issue on all the bharatmatrimony.com applications. The payment medium was suffering from CSRF that was leading to access to their premium services for free. I contacted the organization regarding the same and reported the issue upfront. Although I wasn’t expecting any thing much in return but being a very good startup , they should have given something for the the issue that may have lead to a serious loss to them. Those greedy , smart executives pulled out all the Bug information from me and called me cya in the end. ūüėõ

On the similar line , the demo issue is explained below.

Continue reading

Web service Security{REST/SOAP}

  • Basic Authentication: Sending Base64-encoded combination of username:password to the Webservice¬† server. ex: base64(vaibs:polo) will be something like¬† ‚ÄúNJjks2njL8‚ÄĚ . On server side the same info will be decoded to check for the authentication with database/LDAP/Any other Auth medium. More secure ways to authenticate is¬†Digest(MD).¬†Client send md5 hashe username:password combination to the WS server. WS server

Continue reading

New and enhanced feature in JDK family [covered jdk 1.5 to 1.8]

What’s new in JDK 1.5 over 1.4 !

1. Generics {Compile}
2. Annotations {suppress ,override, deprecated….}
3. Enumerations
4. Variable arguments{void test(String…)}
5. Changes in concurrency utilities.Now includes high-level concurrency APIs.{ java.util.concurrent}
6. Autoboxing and Unboxing
7. Static imports {less keystrokes/time and the same outcomes.}
8. forEach loop {Beautify the existing for loop usage while iterating over collections}

Continue reading