In this post I will be discussing about creating an application that would reside and run on Google's App engine and can be used for developing web applications that will store images in different formats and render them on demand.
For developing this web application, you would need
- Eclipse platform
- JDK ( preferably 1.7)
- Google's App Engine ( I am using v 1.8.1)
- Google Web Toolkit (GWT v 2.5.1)
- Maven
- Google GAE plugins for Eclipse ( I am using plugin v 3.6, 4.2)
Since this blog is for Advanced Java-J2EE users, hence I will not be going into details of how to setup Google's App Engine. However resources listed below will guide you to setup GAE on Eclipse. Reading these and setting up the environment is the prerequisite for creating the application mentioned in this blog.
- You can checkout that your environment is correctly setup by creating this "Hello World" web application. This can be done very quickly.
http://googcloudlabs.appspot.com/codelabexercise0.html
- Deploy the application on GAE environment
http://googcloudlabs.appspot.com/appendix2.html
Once done with these initial tasks, you are ready to explore some advanced concepts provided by GAE. For the application we will be building for storing and retrieving images, we will be using the DataStore APIs from Datastore defined by GAE.
As mentioned on Google's website, App Engine Datastore is a schemaless object datastore providing robust, scalable storage for your web application, with the following features:
- No planned downtime
- Atomic transactions
- High availability of reads and writes
- Strong consistency for reads and ancestor queries
- Eventual consistency for all other queries
These features make this a highly viable option to download, process and host tons of images rather than maintain a data center by our self to achieve the same result.
Creating Web application on Eclipse
1. Right click and select "New->Web Application Project"
2. Now enter the Project Name, Package information and click Finish
You will notice that some Greeting service and related client/server classes and packages are generated. You can go ahead and delete those or keep them.
3. Now create servlet ImageStoreServlet (extends HttpServlet ). This will be used to download images using provided URLs and store them in google's datastore using Datastore APIs
Code for the servlet is as follows.
import java.io.IOException;
import java.net.URL;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import javax.jdo.PersistenceManager;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import com.gaetest.image.jdo.Image;
import com.gaetest.pmf.PMF;
import com.google.appengine.api.urlfetch.HTTPHeader;
import com.google.appengine.api.urlfetch.HTTPResponse;
import com.google.appengine.api.urlfetch.URLFetchService;
import com.google.appengine.api.urlfetch.URLFetchServiceFactory;
public class ImageStoreServlet extends HttpServlet {
private static String url1 = "http://testsite.com/photos/1.jpg" ;
private static String url2 = "http://testsite.com/photos/2.jpg" ;
private static String url3 = "http://testsite.com/photos/3.jpg" ;
private static String url4 = "http://testsite.com/photos/4.jpg" ;
@Override
public void doGet(HttpServletRequest req, HttpServletResponse resp)throws IOException
{
//create list
List<String> imageList = new ArrayList<String>();
imageList.add(url1);
imageList.add(url2);
imageList.add(url3);
imageList.add(url4);
URLFetchService fetchService = URLFetchServiceFactory.getURLFetchService();
HTTPResponse fetchResponse;
String fetchResponseContentType = null;
PersistenceManager pm = PMF.get().getPersistenceManager();
try{
int ii = 0;
for (Iterator<String> iterator = imageList.iterator(); iterator.hasNext();)
{
String imageObj = (String) iterator.next();
// Fetch the image at the location given by the url query string parameter
fetchResponse = fetchService.fetch(new URL(imageObj));
for (HTTPHeader header : fetchResponse.getHeaders()) {
if (header.getName().equalsIgnoreCase("content-type")) {
fetchResponseContentType = header.getValue();
break;
}
}//for
if (fetchResponseContentType != null) {
// Create a new Movie instance
Image imageJDO = new Image();
imageJDO.setImageName("image"+ii++);
imageJDO.setImageType(fetchResponseContentType);
imageJDO.setImage(fetchResponse.getContent());
// Store the image in App Engine's datastore
pm.makePersistent(imageJDO);
}//if
}//outer for
}finally{
pm.close();
}
}
}
In the above code, following observations need to be made:-
- It is a simple HTTP servlet
- We are using GAE URLFetchService to fetch the URL contents.
- We are using JDO to persist the image and additional information into datastore. These and more datastore APIs can be found at https://developers.google.com/appengine/docs/java/datastore/queries
- Following is the code of com.gaetest.pmf.PMF, used to return the PersistenceManager.
import javax.jdo.JDOHelper;
import javax.jdo.PersistenceManagerFactory;
public final class PMF {
private static final PersistenceManagerFactory pmfInstance =
JDOHelper.getPersistenceManagerFactory("transactions-optional");
private PMF() {}
public static PersistenceManagerFactory get() {
return pmfInstance;
}
}
4. Now create GetImageServlet (extends HttpServlet ) to dynamically on demand render the image. This again uses the datastore APIs to retrieve the requested data.
import java.io.IOException;
import java.util.List;
import javax.jdo.PersistenceManager;
import javax.jdo.Query;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import com.gaetest.image.jdo.Image;
import com.gaetest.pmf.PMF;
public class GetImageServlet extends HttpServlet {
@Override
public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException
{
String imageName = req.getParameter("imageName");
Image imageJDO = getImage(imageName);
if (imageJDO != null && imageJDO.getImageType() != null && imageJDO.getImage() != null) {
// Set the appropriate Content-Type header and write the raw bytes
// to the response's output stream
resp.setContentType(imageJDO.getImageType());
resp.getOutputStream().write(imageJDO.getImage());
} else {
// If no image is found with the given title, redirect the user to
// a static image
resp.sendRedirect("/static/noimage.jpg");
}
}
private Image getImage(String imageName)
{
PersistenceManager pm = PMF.get().getPersistenceManager();
Query query = pm.newQuery(Image.class, "imageName == imageNameParam");
query.declareParameters("String imageNameParam");
query.setRange(0, 1);
try {
List<Image> results = (List<Image>) query.execute(imageName);
if (results.iterator().hasNext()) {
// If the results list is non-empty, return the first (and only) result
return results.get(0);
}
} finally {
query.closeAll();
pm.close();
}
return null;
}
}
Some of the observations about this code are as follows:-
- based on the query parameter "imageName" the servlet uses JDO to retrieve image from the datastore.
5. Add following entry in your web.xml for the servlets created.
<!-- image servlets and mappings -->
<servlet>
<servlet-name>ImageStoreServlet</servlet-name>
<servlet-class>com.gaetest.image.servlet.ImageStoreServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>ImageStoreServlet</servlet-name>
<url-pattern>/addImages</url-pattern>
</servlet-mapping>
<servlet>
<servlet-name>GetImageServlet</servlet-name>
<servlet-class>com.gaetest.image.servlet.GetImageServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>GetImageServlet</servlet-name>
<url-pattern>/image</url-pattern>
</servlet-mapping>
And you are done...
Now you can deploy the application on GAE using the information given above. Once deployed you can access the images using following URL
http://<yourapplicationid>.appspot.com/image?imageName=image2
Before accessing the above images, you will need to add the images. Simply run following URL,
http://<yourapplicationid>.appspot.com/addImages
This will download and store the images defined in the servlet to your datastore.
Thanks and enjoy your new application on GAE...
In a similar way, you could write application to upload and download files or other data types using datastore. Remember datastore APIs are different from blobstore APIs offered by Google.
We will be exploring blobstore APIs someother day..
