# HARD-SOFT Something More

## Trying To Explorer Computer World ..!!

+919730604834
Lunch: 11am - 2pm
Dinner: M-Th 9pm - 10pm,
Fri-Sat: 9.30pm - 11.30p m

 barnet on GPU Acceleration In Multilayer… POOJA on Steganography Matlab Code Pragnesh Patel on Steganography Matlab Code joh on Steganography Matlab Code angel on Steganography Matlab Code

• ## Blog Stats

• 14,325 hits

• ## Follow Blog via Email

Join 2,098 other followers

# How to Install Apache Tomcat

## Get Started with Java Servlet Programming

This tutorial can be completed in a 3-hour session.

This installation and configuration guide is applicable to Tomcat 7/8, and possibly the earlier versions.

Take note that Tomcat 8 requires JDK 1.7. It will NOT work with JDK 1.6.

### 1.  Introduction

#### 1.1  Web Application (Webapp)

web application (or webapp), unlike standalone application, runs over the Internet. Examples of webapps are google, amazon, ebay, facebook and twitter.

A webapp is typically a 3-tier (or multi-tierclient-server database application run over the Internet as illustrated in the diagram below. It comprises five components:

1. HTTP Server: E.g., Apache HTTP Server, Apache Tomcat Server, Microsoft Internet Information Server (IIS), nginx, Google Web Server (GWS), and others.
2. HTTP Client (or Web Browser): E.g., Internet Explorer (MSIE), FireFox, Chrome, Safari, and others.
3. Database: E.g., Open-source MySQL, Apache Derby, mSQL, SQLite, PostgreSQL, OpenOffice’s Base; Commercial Oracle, IBM DB2, SAP SyBase, MS SQL Server, MS Access; and others.
4. Client-Side Programs: could be written in HTML Form, JavaScript, VBScript, Flash, and others.
5. Server-Side Programs: could be written in Java Servlet/JSP, ASP, PHP, Perl, Python, CGI, and others.

The typical use-case is:

1. A user, via a web browser (HTTP client), issues a URL request to an HTTP server to start a webapp.
2. A client-side program (such as an HTML form) is loaded into client’s browser.
3. The user fills up the query criteria in the form.
4. The client-side program sends the query parameters to a server-side program.
5. The server-side program receives the query parameters, queries the database and returns the query result to the client.
6. The client-side program displays the query result on the browser.
7. The process repeats.

#### 1.2  Hypertext Transfer Protocol (HTTP)

• HTTP is an application layer protocol runs over TCP/IP. The IP provides support for routing and addressing (via an unique IP address for machines on the Internet); while TCP supports multiplexing via 64K ports from port number 0 to 65535. The default port number assigned to HTTP is TCP port 80.
• HTTP is an asynchronous request-response application-layer protocol. A client sends a request message to the server. The server then returns a response message to the client.
• HTTP is a pull protocol, a client pulls a page from the server (instead of server pushes pages to the clients).
• The syntax of the message is defined in the HTTP specification.

#### 1.3  Apache Tomcat HTTP Server

Apache Tomcat is a Java-capable HTTP server, which could execute special Java programs known as Java Servlet and Java Server Pages (JSP). It is the official Reference Implementation (RI) for Java Servlets and JavaServer Pages (JSP) technologies. Tomcat is an open-source project, under the “Apache Software Foundation” (which also provides the most use, open-source, industrial-strength Apache HTTP Server). The mother site for Tomcat is http://tomcat.apache.org. Alternatively, you can find tomcat via the Apache mother site @ http://www.apache.org.

Tomcat was originally written by James Duncan Davison (then working in Sun), in 1998, based on an earlier Sun’s server called Java Web Server (JWS). It began at version 3.0 after JSWDK 2.1 it replaced. Sun subsequently made Tomcat open-source and gave it to Apache.

The various Tomcat releases are:

1. Tomcat 3.x (1999): RI for Servlet 2.2 and JSP 1.1.
2. Tomcat 4.x (2001): RI for Servlet 2.3 and JSP 1.2.
3. Tomcat 5.x (2002): RI for Servlet 2.4 and JSP 2.0.
4. Tomcat 6.x (2006): RI for Servlet 2.5 and JSP 2.1.
5. Tomcat 7.x (2010): RI for Servlet 3.0, JSP 2.2 and EL 2.2.
6. Tomcat 8.x (2010): RI for Servlet 3.1, JSP 2.3, EL 3.0 and Java WebSocket 1.0.

Tomcat is an HTTP application runs over TCP/IP. In other words, the Tomcat server runs on a specific TCP port in a specific IP address. The default TCP port number for HTTP protocol is 80, which is used for the production HTTP server. For test HTTP server, you can choose any unused port number between 1024 and 65535; while port numbers 1-1023 are reserved.

### 2.  How to Install Tomcat 7 and Get Started with Java Servlet Programming

For Windows

1. Goto http://tomcat.apache.org ⇒ Downloads ⇒ Tomcat 8.0 ⇒ “8.0.{xx}” (where {xx} is the latest upgrade number) ⇒ Binary Distributions ⇒ Core ⇒ “zip” package (e.g., “apache-tomcat-8.0.{xx}.zip“, about 8 MB).
2. UNZIP into a directory of your choice. DO NOT unzip onto the Desktop (because its path is hard to locate). I suggest using “d:\myproject“. Tomcat will be unzipped into directory “d:\myproject\apache-tomcat-8.0.{xx}“. For ease of use, we shall shorten and rename this directory to “d:\myproject\tomcat“. Take note of Your Tomcat Installed Directory. Hereafter, I shall refer to the Tomcat installed directory as <TOMCAT_HOME> (or<CATALINA_HOME> - “Catalina” is the codename for Tomcat 5 and above).

(Advanced) A better approach is to keep the original directory name, such as apache-tomcat-8.0.{xx}, but create a symlink called tomcat via command “mklink /D tomcat apache-tomcat-8.0.{xx}“. Symlink is available in Windows Vista/7/8 only.

For Mac

1. Goto http://tomcat.apache.org ⇒ Download ⇒ Tomcat 8.0 ⇒ “8.0.{xx}” (where {xx} denotes the latest upgrade number) ⇒ Binary distribution ⇒ Core ⇒ “tar.gz” package (e.g., “apache-tomcat-8.0.{xx}.tar.gz“, about 8 MB).
2. To install Tomcat:
1. Goto~/Downloads“, double-click the downloaded tarball (e.g., “apache-tomcat-8.0.{xx}.tar.gz“) to expand it into a folder (e.g., “apache-tomcat-8.0.{xx}“).
2. Move the extracted folder (e.g., “apache-tomcat-8.0.{xx}“) to “/Applications“.
3. Rename the folder to “tomcat”, for ease of use. Take note of Your Tomcat Installed Directory. Hereafter, I shall refer to the Tomcat installed directory as <TOMCAT_HOME> (or <CATALINA_HOME> - “Catalina” is the codename for Tomcat 5 and above).

For Ubuntu

Read “How to Install Tomcat 7 on Ubuntu“. You need to switch between these two articles.

For academic learning, I recommend “zip” (or “tar.gz“) version, as you could simply delete the entire directory when Tomcat is no longer needed (without running any un-installer). You are free to move or rename the Tomcat’s installed directory. You can install (unzip) multiple copies of Tomcat in the same machine. For production, it is easier to use the installer to properly configure the Tomcat.

##### Tomcat’s Directories

Take a quick look at the Tomcat installed directory. It contains the following sub-directories:

• bin: contains the binaries; and startup script (startup.bat for Windows and startup.sh for Unix and Mac), shutdown script (shutdown.bat for Windows and shutdown.sh for Unix and Mac), and other binaries and scripts.
• conf: contains the system-wide configuration files, such as server.xmlweb.xmlcontext.xml, and tomcat-users.xml.
• lib: contains the Tomcat’s system-wide JAR files, accessible by all webapps. You could also place external JAR file (such as MySQL JDBC Driver) here.
• logs: contains Tomcat’s log files. You may need to check for error messages here.
• webapps: contains the webapps to be deployed. You can also place the WAR (Webapp Archive) file for deployment here.
• work: Tomcat’s working directory used by JSP, for JSP-to-Servlet conversion.
• temp: Temporary files.

#### 2.2  STEP 2: Create an Environment Variable JAVA_HOME

For Windows

You need to create an environment variable called “JAVA_HOME” and set it to your JDK installed directory.

1. First, take note of your JDK installed directory. The default is “c:\Program Files\Java\jdk1.7.0_{xx}“, where {xx} is the latest upgrade number. It is important to verify your JDK installed directory, via the “Computer”, before you proceed further.
2. Start a CMD shell, and issue the command “set JAVA_HOME” to check if variable JAVA_HOME has been set:
> set JAVA_HOME
Environment variable JAVA_HOME not defined

If JAVA_HOME is set, check if it is set to your JDK installed directory correctly. Otherwise, goto next step.

3. To set the environment variable JAVA_HOME in Windows 2000/XP/Vista/7/8: Push “Start” button ⇒ Control Panel ⇒ System ⇒ (Vista/7/8) Advanced system settings ⇒ Switch to “Advanced” tab ⇒ Environment Variables ⇒ System Variables ⇒ “New” (or “Edit” for modification) ⇒ In “Variable Name”, enter “JAVA_HOME” ⇒ In “Variable Value”, enter your JDK installed directory (e.g., “c:\Program Files\Java\jdk1.7.0_{xx}“).
4. To verify, RE-START a CMD shell (need to refresh the environment) and issue:
> set JAVA_HOME
JAVA_HOME=c:\Program Files\Java\jdk1.7.0_{xx}   <== Verify that this is YOUR JDK installed directory

For Mac

Skip this step. No need to do anything.

#### 2.3  STEP 3: Configure Tomcat Server

The Tomcat configuration files are located in the “conf” sub-directory of your Tomcat installed directory, e.g. “d:\myproject\tomcat\conf” (for Windows) or “/Applications/tomcat” (for Mac). There are 4 configuration XML files:

1. server.xml
2. web.xml
3. context.xml
4. tomcat-users.xml

Make a BACKUP of the configuration files before you proceed.

##### Step 3(a) “conf\server.xml” - Set the TCP Port Number

Use a programming text editor (e.g., NotePad++, TextPad for Windows; or gEdit, jEdit, TextEdit for Mac) to open the configuration file “server.xml“, under the “conf” sub-directory of Tomcat installed directory.

The default TCP port number configured in Tomcat is 8080, you may choose any number between 1024 and 65535, which is not used by an existing application. We shall choose 9999 in this article. (For production server, you should use port 80, which is pre-assigned to HTTP server as the default port number.)

Locate the following lines, and change port="8080" to port="9999".

<!-- A "Connector" represents an endpoint by which requests are received
and responses are returned. Documentation at :
Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)
Java AJP  Connector: /docs/config/ajp.html
APR (HTTP/AJP) Connector: /docs/apr.html
Define a non-SSL HTTP/1.1 Connector on port 8080
-->
<Connector port="9999" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
##### Step 3(b) “conf\web.xml” - Enabling Directory Listing

Again, use a programming text editor to open the configuration file “web.xml“, under the “conf” sub-directory of Tomcat installed directory.

We shall enable directory listing by changing “listings” from “false” to “true” for the “default” servlet. This is handy for test system, but not for production system for security reasons.

Locate the following lines and change from “false” to “true“.

<!-- The default servlet for all web applications, that serves static     -->
<!-- resources.  It processes all requests that are not mapped to other   -->
<!-- servlets with servlet mappings.                                      -->
<servlet>
<servlet-name>default</servlet-name>
<servlet-class>org.apache.catalina.servlets.DefaultServlet</servlet-class>
<init-param>
<param-name>debug</param-name>
<param-value>0</param-value>
</init-param>
<init-param>
<param-name>listings</param-name>
<param-value>true</param-value>
</init-param>
</servlet>
##### Step 3(c) “conf\context.xml” - Enabling Automatic Reload

We shall add the attribute reloadable="true" to the <Context> element to enable automatic reload after code changes. Again, this is handy for test system but not for production, due to the overhead of detecting changes.

Locate the <Context> start element, and change it to <Context reloadable="true">.

<Context reloadable="true">
......
</Context>
##### Step 3(d) (Optional) “conf\tomcat-users.xml”

Enable the Tomcat’s manager by adding the highlighted lines, inside the <tomcat-users> elements:

<tomcat-users>
<role rolename="manager-gui"/>
</tomcat-users>

This enables the manager GUI app for managing Tomcat server.

#### 2.4  STEP 4: Start Tomcat Server

The Tomcat’s executable programs and scripts are kept in the “bin” sub-directory of the Tomcat installed directory, e.g., “d:\myproject\tomcat\bin” (for Windows) or “/Applications/tomcat/bin” (for Mac).

##### Step 4(a) Start Server

For Windows

Launch a CMD shell. Set the current directory to “<TOMCAT_HOME>\bin“, and run “startup.bat” as follows:

// Change the current directory to Tomcat's "bin"
// Assume that Tomcat is installed in "d:\myproject\tomcat"
> d:                           // Change the current drive
d:\> cd \myproject\tomcat\bin  // Change Directory to YOUR Tomcat's "bin" directory

// Start Tomcat Server
D:\myproject\tomcat\bin> startup

For Mac

I assume that Tomcat is installed in “/Applications/tomcat“. To start the Tomcat server, open a new “Terminal” and issue:

// Change current directory to Tomcat's binary directory
$cd /Applications/tomcat/bin // Start tomcat server$ ./catalina.sh run

new Tomcat console window appears. Study the messages on the console. Look out for the Tomcat’s port number (double check that Tomcat is running on port 9999). Future error messages will be send to this console. System.out.println()issued by your Java servlets will also be sent to this console.

......
......
xxx xx, xxxx x:xx:xx xx org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-9999"]
xxx xx, xxxx x:xx:xx xx org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8009"]
xxx xx, xxxx x:xx:xx xx org.apache.catalina.startup.Catalina start
INFO: Server startup in 2477 ms

(Skip Unless …) Cannot Start Tomcat: Read “How to Debug“.

##### Step 4(b) Start a Client to Access the Server

Start a browser (as HTTP client). Issue URL “http://localhost:9999” to access the Tomcat server’s welcome page. The hostname “localhost” (with IP address of 127.0.0.1) is meant for local loop-back testing inside the same machine. For users on the other machines over the net, they have to use the server’s IP address or DNS domain name or hostname in the format of “http://serverHostnameOrIPAddress:9999“.

Try issuing URL http://localhost:9999/examples to view the servlet and JSP examples. Try running some of the servlet examples.

(Optional) Try issuing URL http://localhost:9999/manager/html to run the Tomcat Web Manager. Enter the username and password configured earlier in tomcat-users.xml.

##### Step 4(c) Shutdown Server

For Windows

You can shutdown the tomcat server by either:

1. Press ctrl-c on the Tomcat console; or
2. Run “<TOMCAT_HOME>\bin\shutdown.bat” script:
// Change the current directory to Tomcat's "bin"
> d:                           // Change the current drive
d:\> cd \myproject\tomcat\bin  // Change Directory to YOUR Tomcat's "bin" directory

// Shutdown the server
d:\myproject\tomcat\bin> shutdown

For Mac

To shutdown the Tomcat server:

1. Press control-c (NOT command-c); or
2. Run the “<TOMCAT_HOME>/bin/shutdown.sh” script. Open a new “Terminal” and issue:
// Change current directory to Tomcat's bin directory
$cd /Applications/tomcat/bin // Shutdown the server$ ./shutdown.sh

WARNING: You MUST properly shutdown the Tomcat. DO NOT kill the cat by pushing the window’s “CLOSE” button.

#### 2.5  STEP 5: Develop and Deploy a WebApp

##### Step 5(a) Create the Directory Structure for your WebApp

First of all, choose a name for your webapp. Let’s call it “hello“. Goto Tomcat’s “webapps” sub-directory. Create the following directory structure for you webapp “hello” (as illustrated):

1. Under Tomcat’s “webapps“, create your webapp root directory “hello” (i.e., “<TOMCAT_HOME>\webapps\hello“).
2. Under “hello“, create a sub-directory “WEB-INF” (case sensitive, a “dash” not an underscore) (i.e., “<TOMCAT_HOME>\webapps\hello\WEB-INF“).
3. Under “WEB-INF“, create a sub-sub-directory “classes” (case sensitive, plural) (i.e., “<TOMCAT_HOME>\webapps\hello\WEB-INF\classes“).

You need to keep your web resources (e.g., HTMLs, CSSs, images, scripts, servlets, JSPs) in the proper directories:

• hello“: The is called the context root (or document base directory) of your webapp. You should keep all your HTML files and resources visible to the web users (e.g., HTMLs, CSSs, images, scripts, JSPs) under thiscontext root.
• hello\WEB-INF“: This directory, although under the context root, is not visible to the web users. This is where you keep your application’s web descriptor file “web.xml“.
• hello\WEB-INF\classes“: This is where you keep all the Java classes such as servlet class-files.

You should RE-START your Tomcat server to pick up the hello webapp. Check the Tomcat’s console to confirm that “hello” application has been properly depolyed:

......
INFO: Deploying web application directory D:\myproject\tomcat\webapps\hello
......

You can issue the following URL to access the web application “hello“:

http://localhost:9999/hello

You should see the directory listing of the directory “<TOMCAT_HOME>\webapps\hello“, which shall be empty (provided you have enabled directory listing in web.xml earlier).

##### Step 5(b) Write a Welcome Page

Create the following HTML page and save as “HelloHome.html” in your application’s root directory “hello“.

 1 2 3 4 5 6  My Home Page

My Name is so and so. This is my HOME.



http://localhost:9999/hello/HelloHome.html

Alternatively, you can issue an URL to your web application root “hello“:

http://localhost:9999/hello

The server will return the directory listing of your base directory. You can then click on “HelloHome.html“.

Rename “HelloHome.html” to “index.html“, and issue a directory request again:

http://localhost:9999/hello

Now, the server will redirect the directory request to “index.html“, if the root directory contains an “index.html“, instead of serving the directory listing.

http://YourPeerHostnameOrIPAddress:9999/hello

http://YourPeerHostnameOrIPAddress:9999/hello/index.html

with a valid “YourPeerHostnameOrIPAddress“, provided that your peer has started his tomcat server and his firewall does not block your access. You can use command such as “ipconfig“, “winipcfg“, “ping” to find the IP address.
(Skip Unless…) The likely errors are “Unable to Connect”, “Internet Explorer cannot display the web page“, and “404 File Not Found”. Read “How to Debug” section.

#### 2.6  STEP 6: Write a “Hello-world” Java Servlet

servlet is Java program that runs inside a Java-capable HTTP Server, such as Apache Tomcat. A web user invokes a servlet by issuing an appropriate URL from a web browser (HTTP client).

Before you proceed, I shall assume that you are familiar with Java Programming and have installed the followings:

1. JDK (Read “How to install JDK and Get Started“).
2. A programming text editor, such as TextPad or Notepad++ (Read “Programming Text Editor“); or a Java IDE such as Eclipse or NetBeans (Read “How to Install Eclipse” or “How to Install NetBeans“).
##### Step 6(a) Install Servlet API Library

Before we can write our first servlet, we need to install the Servlet API. Servlet API is not part of JDK or Java SE (but belongs to Java EE). Tomcat provides a copy of Servlets API.

For Windows

COPY the Tomcat’s Servlet API jar-file located at “<TOMCAT_HOME>\lib\servlet-api.jar“, (e.g., “d:\myproject\tomcat\lib\servlet-api.jar“) into JDK’s extension directory at “<JAVA_HOME>\jre\lib\ext“, (e.g., “c:\Program Files\Java\jdk1.7.0\jre\lib\ext“).

For Mac

COPY the Servlet API jar-file (“servlet-api.jar“) from “/Applications/tomcat/lib” to the JDK’s extension directory at “/Library/Java/Extension“.

(For Advanced Users Only) Alternatively, you could include the Servlet API jar-file in the CLASSPATH: or the JDK’s extension directory: or in the javac|java‘s command-line option -cp <classpaths>.

##### Step 6(b) Write a “Hello-world” Java Servlet

A Java servlet is a Java program that runs inside a HTTP server. A web user invokes a servlet by issuing a URL from a browser (or HTTP client).

In this example, we are going to write a Java servlet called HelloServlet, which says “Hello, world!”. We will then write a configuration such that web users can invoke this servlet by issuing URL http://hostname:port/hello/sayhello from their browser, as illustrated:

Write the following source codes called “HelloServlet.java” and save it under your application “classes” directory (i.e., “<TOMCAT_HOME>\webapps\hello\WEB-INF\classes\HelloServlet.java“). This servlet says “Hello”, echos some request information, and prints a random number upon each request.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 // To save as "\webapps\hello\WEB-INF\classes\HelloServlet.java" import java.io.*; import javax.servlet.*; import javax.servlet.http.*; public class HelloServlet extends HttpServlet { @Override public void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException { // Set the response MIME type of the response message response.setContentType("text/html"); // Allocate a output writer to write the response message into the network socket PrintWriter out = response.getWriter(); // Write the response message, in an HTML page try { out.println(""); out.println("Hello, World"); out.println(""); out.println("

Hello, world!

"); // says Hello // Echo client's request information out.println("

Request URI: " + request.getRequestURI() + "

"); out.println("

Protocol: " + request.getProtocol() + "

"); out.println("

PathInfo: " + request.getPathInfo() + "

"); out.println("

"); // Generate a random number upon each request out.println("

A Random Number: " + Math.random() + "

"); out.println(""); } finally { out.close(); // Always close the output writer } } }

Compile the source “HelloServlet.java” into “HelloServlet.class“:

> cd [Path-to-the-source-file]
> javac HelloServlet.java

(Skip Unless…) Read “Common Errors in Compiling Java Servlet“.

##### Step 6(c) Configure Servlet’s Request URL in “webapps\hello\WEB-INF\web.xml”

A web user invokes a servlet, which is kept in the web server, by issuing a request URL from the browser. We need to configure this request URL for our HelloServlet.

Create the following configuration file called “web.xml“, and save it under “webapps\hello\WEB-INF” (i.e., “<TOMCAT_HOME>\webapps\hello\WEB-INF\web.xml“).

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21  HelloWorld HelloServlet HelloWorld /sayhello 

In the above configuration, a servlet having a class file “HelloServlet.class” is mapped to request URL “/sayhello” (via an arbitrary servlet-name ”HelloWorld“), under this web application “hello“. In other words, the complete request URL for this servlet is “http://hostname:port/hello/sayhello“.

This configuration file, saved under your webapp “hello“, is applicable only to this particular webapp “hello“.

RESTART your Tomcat server to refresh the “web.xml” file.

IMPORTANT: For EACH servlet, you need to write a pair of <servlet> and <servlet-mapping> elements with a common but arbitrary <servlet-name>. Take note that all the <servlet> elements MUST be grouped together and placed IN FRONT of the <servlet-mapping> elements.

##### Step 6(d) Invoke the Servlet

To run this servlet, start a browser, and issue the request URL configured earlier:

http://localhost:9999/hello/sayhello

Refresh the browser, you shall see a new random number upon each refresh. In other word, the doGet() method of the servlet runs once per request.

Try “View Source” to look at the output received by the web users. Take note that the web users receive only the output of the servlet (generated via the out.println() statements). They have no access to the servlet programs (which may contain confidential information).

<html>
<body>
<h1>Hello, world!</h1>
<p>Request URI: /hello/sayhello</p>
<p>Protocol: HTTP/1.1</p>
<p>PathInfo: null</p>
<p>A Random Number: <strong>0.3523682325749493</strong></p>
</body>
</html>

(Skip Unless…) The likely errors are “404 File Not Found” and “500 Internal Server Error”. Read “How to debug” Section.

#### 2.7  STEP 7: Write a Database Servlet

This section assumes that you are familiar with “Java database programming” and “MySQL database server”. Otherwise, read “Java Database Program” and “How to Install MySQL 5 and Get Started“.

##### Step 7(a) Setup a Database on MySQL

Start your MySQL server. Take note of the server’s port number. I shall assume that the MySQL server is running on port 8888 (whereas the Tomcat is running on port 9999).

// For Windows
> d:
> cd \myproject\mysql\bin
> mysqld --console

// For Mac
$cd /usr/local/mysql/bin$ sudo ./mysqld_safe --console

Start a MySQL client. I shall assume that there is a user called “myuser” with password “xxxx“.

// For Windows
> d:
> cd \myproject\mysql\bin
> mysql -u myuser -p

// For Mac
$cd /usr/local/mysql/bin$ ./mysql -u myuser -p

Run the following SQL statements to create a database called “ebookshop“, with a table called “books” with 5 columns: idtitleauthorpriceqty.

create database if not exists ebookshop;

use ebookshop;

drop table if exists books;
create table books (
id     int,
title  varchar(50),
author varchar(50),
price  float,
qty    int,
primary key (id));

insert into books values (1001, 'Java for dummies', 'Tan Ah Teck', 11.11, 11);
insert into books values (1002, 'More Java for dummies', 'Tan Ah Teck', 22.22, 22);
insert into books values (1003, 'More Java for more dummies', 'Mohammad Ali', 33.33, 33);
insert into books values (1004, 'A Cup of Java', 'Kumar', 55.55, 55);
insert into books values (1005, 'A Teaspoon of Java', 'Kevin Jones', 66.66, 66);select * from books;
##### Step 7(b) Install MySQL JDBC Driver

You need to download MySQL JDBC driver if you have not done so. Read “Installing the MySQL JDBC Driver“.

(For Advanced Users Only) You could also place the MySQL driver jar-file “mysql-connector-java-5.1.{xx}-bin.jar” in Tomcat’s “lib” directory.

##### Step 7(c) Write a Client-side HTML Form

Let’s write an HTML script to create a query form with 3 checkboxes and a submit button, as illustrated below.  Save the HTML file as “querybook.html” in your application root directory “<TOMCAT_HOME>\webapps\hello”.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15  Yet Another Bookshop

Yet Another Bookshop

Choose an author: Ah Teck Ali Kumar 

You can browse the HTML page by issuing the following URL:

http://localhost:9999/hello/querybook.html

Check a box (e.g., “Tan Ah Teck”) and click the “Search” button.  An HTTP GET request will be issued to the URL specified in the <form>‘s “action” attribute.  Observe the URL of the HTTP GET request:

http://localhost:9999/hello/query?author=Tan+Ah+Teck

The request consists of two part: a URL corresponding to the “action” attribute of the <form> tag, and the “name=value” pair extracted from the <input> tag, separated by a '?'. Take note that blanks are replaced by '+' (or %20), because blanks are not allowed in the URL.

If you check two boxes (e.g., “Tan Ah Teck” and “Mohammad Ali”), you will get this URL, which has two “name=value” pairs separated by an '&'.

http://localhost:9999/hello/query?author=Tan+Ah+Teck&author=Mohammad+Ali

You are expected to get an error “404 File Not Found”, as you have yet to write the server-side program.

##### Step 7(d) Write the Server-side Database Query Servlet

The next step is to write a Java servlet, which responses to the client’s request by querying the database and returns the query results.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 // To save as "\webapps\hello\WEB-INF\classes\QueryServlet.java". import java.io.*; import java.sql.*; import javax.servlet.*; import javax.servlet.http.*; public class QueryServlet extends HttpServlet { // JDK 6 and above only // The doGet() runs once per HTTP GET request to this servlet. @Override public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { // Set the MIME type for the response message response.setContentType("text/html"); // Get a output writer to write the response message into the network socket PrintWriter out = response.getWriter(); Connection conn = null; Statement stmt = null; try { // Step 1: Allocate a database Connection object conn = DriverManager.getConnection( "jdbc:mysql://localhost:8888/ebookshop", "myuser", "xxxx"); // <== Check! // database-URL(hostname, port, default database), username, password // Step 2: Allocate a Statement object within the Connection stmt = conn.createStatement(); // Step 3: Execute a SQL SELECT query String sqlStr = "select * from books where author = " + "'" + request.getParameter("author") + "'" + " and qty > 0 order by price desc"; // Print an HTML page as the output of the query out.println("Query Response"); out.println("

"); out.println("

You query is: " + sqlStr + "

"); // Echo for debugging ResultSet rset = stmt.executeQuery(sqlStr); // Send the query to the server // Step 4: Process the query result set int count = 0; while(rset.next()) { // Print a paragraph

...

for each record out.println("

" + rset.getString("author") + ", " + rset.getString("title") + ", $" + rset.getDouble("price") + " "); count++; } out.println(" ==== " + count + " records found ===== "); out.println(""); } catch (SQLException ex) { ex.printStackTrace(); } finally { out.close(); // Close the output writer try { // Step 5: Close the resources if (stmt != null) stmt.close(); if (conn != null) conn.close(); } catch (SQLException ex) { ex.printStackTrace(); } } } } Compile the source “QueryServlet.java” into “QueryServlet.class“. ##### Step 7(e) Configure the Request URL for the Servlet Open the configuration file “web.xml” of your application “hello” that you have created earlier for the HelloServlet, i.e., “<TOMCAT_HOME>\webapps\hello\WEB-INF\web.xml“. Add the lines that are shown in red at the LOCATIONS INDICATED.  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31  HelloWorld HelloServlet UserQuery QueryServlet HelloWorld /sayhello UserQuery /query  The above lines configure the following URL to invoke QueryServlet: http://localhost:9999/hello/query ##### Step 7(f) Invoke the Servlet from the Client-Side Form Issue the following URL to browse the HMTL form “querybook.html” that you have created earlier: http://localhost:9999/hello/querybook.html Select an author (e.g., “Tan Ah Teck”) and click the submit button, which activates the following URL coded in the <form>‘s “action” attribute, together with the name=value pair: http://localhost:9999/hello/query?author=Tan+Ah+Teck This URL “/query” triggers QueryServlet. The QueryServlet retrieves the name=value pair of “author=Tan+Ah+Teck“. Inside the QueryServlet, the method request.getParameter("author") returns “Tan Ah Teck“, which is inserted into the SQL SELECT command to query the database. The processed query result is then written to the client as an HTML document. (Skip Unless…) The likely errors are “404 File Not Found” and “500 Internal Server Error”. Read “How to debug” Section. #### 2.8 (Advanced) Deploying Servlet using @WebServlet (Servlet 3.0 on Tomcat 7) Servlet 3.0, which is supported by Tomcat 7, introduces the @WebServlet annotation, which greatly simplifies the deployment of servlets. You no longer need to write the deployment descriptor in “web.xml“. Instead, you can use the@WebServlet annotation to specify the url mapping. For example, let us write a new servlet called AnotherHelloServlet.java, by modifying the HelloServlet.java written earlier, with url mapping of “sayhi“.  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 // To save as "\webapps\hello\WEB-INF\classes\AnotherHelloServlet.java" import java.io.*; import javax.servlet.*; import javax.servlet.http.*; import javax.servlet.annotation.*; @WebServlet("/sayhi") public class AnotherHelloServlet extends HttpServlet { @Override public void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException { // Set the response MIME type response.setContentType("text/html;charset=UTF-8"); // Allocate a output writer to write the response message into the network socket PrintWriter out = response.getWriter(); // Write the response message, in an HTML page try { out.println(""); out.println("Hello, World"); out.println(""); out.println(" Hello world, again! "); // says Hello // Echo client's request information out.println(" Request URI: " + request.getRequestURI() + " "); out.println(" Protocol: " + request.getProtocol() + " "); out.println(" PathInfo: " + request.getPathInfo() + " "); out.println(" Remote Address: " + request.getRemoteAddr() + " "); // Generate a random number upon each request out.println(" A Random Number: " + Math.random() + " "); out.println(""); } finally { out.close(); // Always close the output writer } } } In Line 7, the annotation @WebServlet("/sayhi") is used to declare the URL mapping for this servlet, i.e., http://localhost:9999/hello/sayhi. There is no need to provide any more configuration in “web.xml“! ### 3. How to Debug? “Everything that can possibly go wrong will go wrong.” The most important thing to do is to find the ERROR MESSAGE!!! ##### Always… 1. Refresh your browser using Cntl-F5 (instead of refresh button or simply F5) to get a fresh copy, instead of from the cache. 2. You may re-start your Tomcat server. You may also re-start your browser to clear the cache. 3. Check your spelling! Always assume that all programs are case-sensitive. Don’t type, copy and paste if possible! 4. and MOST IMPORTANTLY – Find the ERROR MESSAGE!!! 1. Check the Error Messages on Tomcat’s Console. Most of the error messages have a few screens of lines. You need to scroll up slowly from the last line to look for the FIRST LINE of the error messages. 2. Check the Tomcat’s log files, located at “<TOMCAT_HOME>\logs“. The “catalina.yyyy-mm-dd.log” shows the Tomcat’s startup messages. Also check the “localhost.yyyy-mm-dd.log“. 5. If things were running fine until the lightning strikes, ask yourself “What have I changed?” ##### Cannot Start Tomcat – Tomcat’s Console Flashes and Disappears 1. Try running the script “configtest.bat” (for Windows) or “./configtest.sh” (for Mac/Linux) to check your configuration files. 2. Check the Tomcat’s log files for error messages. The log files are located at “<TOMCAT_HOME>\logs“. The “catalina.{yyyy-mm-dd}.log” shows the Tomcat’s startup messages. Also check the “localhost.{yyyy-mm-dd}.log“. 3. If the error messages indicate that another Tomcat instance is running (java.net.BindException: Address already in use: JVM_Bind), kill the Tomcat process (see below); or try running the “shutdown” script at Tomcat’s bin (For Windows, simply double-click the “shutdown.bat” or issue “shutdown” from CMD. For Mac, issue “./shutdown.sh” from Terminal.) 4. If the error messages indicate that another application is running on the Tomcat’s port numbers, then you need to change the Tomcat’s port number in server.xml. You can issue command “netstat -an” to check the status of all the ports. 5. Start the tomcat in the debugging mode by running “catalina debug” (or ./catalina.sh debug) and type “run” in the “jdb” prompt. Look for the error messages. ##### Locating/Killing Tomcat’s Process • In windows, start “Task Manager”, Tomcat run as a “process” named “java.exe“. You may need to kill the process. • In Mac, start “Activity Monitor”. Select “All Processes” and look for “java.exe“. • In Linux/Mac, you may issue “ps aux | grep tomcat” to locate the Tomcat process. Note down the process ID (pid). You can kill the Tomcat process via “kill -9 pid“. ##### (Firefox) Unable to Connect (IE) Internet Explorer cannot display the webpage (Chrome) Oops! Google Chrome could not connect to … (Safari) Safari can’t connect to the server Cause: You are simply not connecting to your Tomcat. Solution: 1. Check if your Tomcat server has been started? 2. Check the hostname and port number, separated by a colon ':', of your URL (http://localhost:9999/...). ##### Error 404 File Not Found Cause: You have connected to your Tomcat. But Tomcat server cannot find the HTML file or Servlet that your requested. Solution: 1. Check your spelling! The path is case-sensitive! 2. For HTML file with URL http://localhost:9999/xxxx/filename.html: 1. Open Tomcat’s “webapps” directory, check if sub-directory “xxxx” exists. It is case-sensitive. 2. Open the “xxxx” directory, check if “filename.html” exists. 3. For Servlet with URL http://localhost:9999/xxxx/servletURL: 1. Check the Tomcat’s console for error message. Your application cannot be deployed if you make a mistake in editing “web.xml“, which triggered many error messages. 2. Check the Tomcat console to make sure that your application has been deployed. 3. Open Tomcat’s “webapps” directory, check if sub-directory “xxxx” exists. 4. Open the “xxxx” directory, check if sub-sub-directory “WEB-INF” (uppercase with a dash) exists. 5. Open the “WEB-INF“, check if sub-sub-sub directory “classes” (lowercase, plural) exists. 6. Open the configuration file “WEB-INF\web.xml“: 1. Check that servletURL is defined in a <servlet-mapping> tag. Take note of the name in <servlet-name> tag. 2. Based on the name noted, look for the matching <servlet-class> tag. Take note of the ServletClassname. 3. Open “WEB-INF\classes“, check if “ServletClassname.class” that you noted exists (Note: It is “.class“, and NOT “.java“. You need to compile the “.java” to get the “.class“.) ##### Error 500 Internal Server Error Error 500 should have triggered many error message in the Tomcat’s console. Go to the Tomcat’s console, find the error message. The error message spans tens of lines. You need to scroll up slowly to look for the first line of the error message. The error message should tell you the cuase of this error, e.g. SQL syntax error, wrong user/password, etc. For database servlet, you may check the error messages at “Common Errors in JDBC Programming“. • For “No suitable driver found” (Windows) or NullPointerException (Mac and Linux): Read Step 7(b) again, again, and again. ##### More Errors Try searching “Common Error Messages“. REFERENCES & RESOURCES 1. Apache Tomcat mother site @ http://tomcat.apache.org. 2. Apache Tomcat Documentation @ “<TOMCAT_HOME>\webapps\docs“. 3. How to install MySQL and Get Started“. 4. Introduction to Java Database (JDBC) Programming“. 5. Jason Brittain, Ian F. Darwin, “Tomcat The Definitive Guide“, 2nd eds, OReilly, 2007. ## Tasksel Posted by Hemprasad Y. Badgujar on November 30, 2013 # Tasksel Tasksel is a Debian/Ubuntu tool that installs multiple related packages as a co-ordinated “task” onto your system. This function is similar to that of meta-packages, and, in fact, most of the tasks available from tasksel are also available as meta-packages from the Ubuntu package managers (such as Synaptic Package Manager or KPackageKit). ## Installation Because the package managers now have most of the tasks as meta-packages, tasksel is not installed by default on editions of Ubuntu (such as Desktop editions) that have package managers. (Tasksel is still installed by default on server editions). If tasksel is not installed, install it from the command-line: sudo apt-get install tasksel ## Usage To run tasksel from the command line, type: sudo tasksel the tasksel menu will be shown: Already-installed tasks will have an asterisk beside their name. Select a task by scrolling down and pressing space. This will put an asterisk beside the selected task and mark it for installation. Removing an asterisk marks the task for removal. Once “ok” is selected the task installations and/or removals will take place using apt-get. ## Command line arguments You can also directly specify which task to install. For instance, to add the Apache-MySQL-PHP stack to an existing system: sudo tasksel install lamp-server To see available (unistalledu) and installed (i) tasks: tasksel --list-task For complete options, see the tasksel manpage: man tasksel ## Usage (alternative) Tasks can also be installed with apt-get using the syntax: sudo apt-get install the_name_of_the_task_you_want^ Examples: sudo apt-get install ubuntu-desktop^ sudo apt-get install kubuntu-desktop^ sudo apt-get install lamp-server^ Note: The caret (^) was generally required at the end of the command in older versions of tasksel. It is not required in newer versions of tasksel. Note – Calling tasks from apt-get is not documented anywhere. If you find a page that documents this, please update this page with a link. ## Tasks List Tasks are defined in .desc files found in /usr/share/tasksel. Debian derivatives can easily add tasks. The default list available in Ubuntu can be viewed with this command-line: grep Task /usr/share/tasksel/ubuntu-tasks.desc ## Task descriptions Here is a list of the tasks in Lucid Lynx 10.04 LTS (from tasksel –list): • eucalyptus-walrus Cloud computing: Walrus storage service • eucalyptus-simple-cluster Cloud computing: all-in-one cluster • eucalyptus-cluster Cloud computing: cluster controller • eucalyptus-node Cloud computing: node controller • eucalyptus-storage Cloud computing: storage controller • eucalyptus-cloud Cloud computing: top-level cloud controller • dns-server DNS server (Bind9) • edubuntu-server Edubuntu server (for classrooms) • lamp-server LAMP server (Linux, Apache2, MySQL, PHP) • mail-server Mail server • openssh-server OpenSSH server • postgresql-server PostgreSQL database • print-server Print server • samba-server Samba file server • tomcat-server Tomcat Java server • uec Ubuntu Enterprise Cloud (instance) • virt-host Virtual Machine host • ubuntustudio-graphics 2D/3D creation and editing suite • ubuntustudio-audio Audio creation and editing suite • edubuntu-desktop-kde Edubuntu KDE desktop • edubuntu-desktop-gnome Edubuntu desktop • kubuntu-desktop Kubuntu desktop • kubuntu-netbook Kubuntu netbook • ubuntustudio-audio-plugins LADSPA and DSSI audio plugins • ubuntustudio-font-meta Large selection of font packages • mythbuntu-desktop Mythbuntu additional roles • mythbuntu-frontend Mythbuntu frontend • mythbuntu-backend-master Mythbuntu master backend • mythbuntu-backend-slave Mythbuntu slave backend • ubuntu-netbook Ubuntu Netbook • ubuntu-desktop Ubuntu desktop • ubuntustudio-video Video creation and editing suite • xubuntu-desktop Xubuntu desktop • edubuntu-dvd-live Edubuntu live DVD • kubuntu-netbook-live Kubuntu Netbook Remix live CD • kubuntu-live Kubuntu live CD • kubuntu-dvd-live Kubuntu live DVD • netbook-live Ubuntu Netbook live environment • ubuntu-live Ubuntu live CD (extra packages installed on the Ubuntu LiveCD) • ubuntu-dvd-live Ubuntu live DVD • xubuntu-live Xubuntu live CD Posted in Linux OS, Operating Systems | Leave a Comment » ## To install the default LAMP stack in Ubuntu 10.04 and above Posted by Hemprasad Y. Badgujar on November 30, 2013 # ApacheMySQLPHP ### Hint: Server Guide To find the Ubuntu Server Guide related to your specific version, please go to: https://help.ubuntu.com/, select your Ubuntu version and then click on Ubuntu Server Guide. For the latest LTS version (12.04 LTS) of Ubuntu Server, please go tohttps://help.ubuntu.com/12.04/serverguide/index.html Parent page: Programming Applications This is to help people setup and install a LAMP (Linux-Apache-MySQL-PHP) server in Ubuntu, including Apache 2, PHP 5 and MySQL 4.1 or 5.0. ## To install the default LAMP stack in Ubuntu 10.04 and above First install tasksel $ sudo apt-get install tasksel

and then the LAMP stack:

$sudo tasksel install lamp-server See Tasksel - be warned, only use tasksel to install tasks, not to remove them – see https://launchpad.net/bugs/574287 DO NOT UNCHECK ANY PACKAGES IN THE MENU WHICH APPEARS You can leave your system in an unusable state. ## Starting over: How to remove the LAMP stack To remove the LAMP stack remove the following packages: • Note: This assumes you have no other programs that require any of these packages. You might wish to simulate this removal first, and only remove the packages that don’t cause removal of something desired. apache2 apache2-mpm-prefork apache2-utils apache2.2-common libapache2-mod-php5 libapr1 libaprutil1 libdbd-mysql-perl libdbi-perl libnet-daemon-perl libplrpc-perl libpq5 mysql-client-5.5 mysql-common mysql-server mysql-server-5.5 php5-common php5-mysql To also remove the debconf data, use the purge option when removing. To get rid of any configurations you may have made to apache, manually remove the /etc/apache2 directory once the packages have been removed. You may also want to purge these packages: mysql-client-core-5.5 mysql-server-core-5.5 ## Installing Apache 2 To only install the apache2 webserver, use any method to install: apache2 It requires a restart for it to work: $ sudo /etc/init.d/apache2 restart

or

$sudo service apache2 restart ### Checking Apache 2 installation With your web browser, go to the URI http://localhost : if you read “It works!”, which is the content of the file /var/www/index.html , this proves Apache works. ### Troubleshooting Apache If you get this error: apache2: Could not determine the server’s fully qualified domain name, using 127.0.0.1 for ServerName then use a text editor such as “sudo nano” at the command line or “gksudo gedit” on the desktop to create a new file, $ sudo nano /etc/apache2/conf.d/fqdn

or

$gksu "gedit /etc/apache2/conf.d/fqdn" then add ServerName localhost to the file and save. This can all be done in a single command with the following: $ echo "ServerName localhost" | sudo tee /etc/apache2/conf.d/fqdn

### Virtual Hosts

Apache2 has the concept of sites, which are separate configuration files that Apache2 will read. These are available in/etc/apache2/sites-available. By default, there is one site available called default this is what you will see when you browse tohttp://localhost or http://127.0.0.1. You can have many different site configurations available, and activate only those that you need.

As an example, we want the default site to be /home/user/public_html/. To do this, we must create a new site and then enable it in Apache2.

To create a new site:

• Copy the default website as a starting point. sudo cp /etc/apache2/sites-available/default /etc/apache2/sites-available/mysite
• Edit the new configuration file in a text editor “sudo nano” on the command line or “gksudo gedit”, for example:gksudo gedit /etc/apache2/sites-available/mysite
• Change the DocumentRoot to point to the new location. For example, /home/user/public_html/
• Change the Directory directive, replace <Directory /var/www/> to <Directory /home/user/public_html/>
• You can also set separate logs for each site. To do this, change the ErrorLog and CustomLog directives. This is optional, but handy if you have many sites
• Save the file

Now, we must deactivate the old site, and activate our new one. Ubuntu provides two small utilities that take care of this: a2ensite (apache2enable site) and a2dissite (apache2disable site).

$sudo a2dissite default && sudo a2ensite mysite Finally, we restart Apache2: $ sudo /etc/init.d/apache2 restart

If you have not created /home/user/public_html/, you will receive an warning message

To test the new site, create a file in /home/user/public_html/:

$echo '<b>Hello! It is working!</b>' > /home/user/public_html/index.html Finally, browse to http://localhost/ ## Installing PHP 5 To only install PHP5. use any method to install the package libapache2-mod-php5 Enable this module by doing $ sudo a2enmod php5

which creates a symbolic link /etc/apache2/mods-enabled/php5 pointing to /etc/apache2/mods-availble/php5 .

Except if you use deprecated PHP code beginning only by “<?” instead of “<?php” (which is highly inadvisable), open, as root, the file /etc/php5/apache2/php.ini , look for the line “short_open_tag = On”, change it to “short_open_tag = Off” (not including the quotation marks) and add a line of comment (beginning by a semi-colon) giving the reason, the author and the date of this change. This way, if you later want some XML or XHTML file to be served as PHP, the “<?xml” tag will be ignored by PHP instead of being seen as a PHP code mistake.

Relaunch Apache 2 again:

$sudo service apache2 restart ### Checking PHP 5 installation In /var/www , create a text file called “test.php”, grant the world (or, at least, Ubuntu user “apache”) permission to read it, write in it the only line: “<?php phpinfo(); ?>” (without the quotation marks) then, with your web browser, go to the URI “http://localhost/test.php“: if you can see a description of PHP5 configuration, it proves PHP 5 works with Apache. ### Troubleshooting PHP 5 Does your browser ask if you want to download the php file instead of displaying it? If Apache is not actually parsing the php after you restarted it, install libapache2-mod-php5. It is installed when you install the php5 package, but may have been removed inadvertently by packages which need to run a different version of php. If sudo a2enmod php5 returns “$ This module does not exist!”, you should purge (not just remove) the libapache2-mod-php5package and reinstall it.

Be sure to clear your browser’s cache before testing your site again. To do this in Firefox 4: Edit → Preferences … Privacy → History: clear your recent history → Details : choose “Everything” in “Time range to clean” and check only “cache”, then click on “Clear now”.

Remember that, for Apache to be called, the URI in your web browser must begin with “http://“. If it begins with “file://“, then the file is read directly by the browser, without Apache, so you get (X)HTML and CSS, but no PHP. If you didn’t configure any host alias or virtual host, then a local URI begins with “http://localhost“, “http://127.0.0.1” or http://” followed by your IP number.

### php.ini development vs. production

After standard installation, php configuration file /etc/php5/apache2/php.ini is set so as “production settings” which means, among others, that no error messages are displayed. So if you e.g. make a syntax error in your php source file, apache server would return HTTP 500 error instead of displaying the php syntax error debug message.

If you want to debug your scripts, it might be better to use the “development” settings. Both development and production settings ini’s are located in /usr/share/php5/

/usr/share/php5/php.ini-development
/usr/share/php5/php.ini-production

so you can compare them and see the exact differences.

To make the “development” settings active, just backup your original php.ini

sudo mv /etc/php5/apache2/php.ini /etc/php5/apache2/php.ini.bak

sudo cp -s /usr/share/php5/php.ini-development /etc/php5/apache2/php.ini

or you may of course also edit the /etc/php5/apache2/php.ini directly on your own, if you wish.

### PHP in user directories

According to this blog, newer versions of Ubuntu do not have PHP enabled by default for user directories (your public_html folder). See the blog for instructions on how to change this back.

## Installing MYSQL with PHP 5

Use any method to install

mysql-server libapache2-mod-auth-mysql php5-mysql

## After installing PHP

You may need to increase the memory limit that PHP imposes on a script. Edit the /etc/php5/apache2/php.ini file and increase thememory_limit value.

## After installing MySQL

Before you can access the database from other computers in your network, you have to change its bind address. Note that this can be a security problem, because your database can be accessed by other computers than your own. Skip this step if the applications which require mysql are running on the same machine.

type:

$sudo nano /etc/mysql/my.cnf and change the line: bind-address = localhost to your own internal ip address e.g. 192.168.1.20 bind-address = 192.168.1.20 If your ip address is dynamic you can also comment out the bind-address line and it will default to your current ip. If you try to connect without changing the bind-address you will recieve a “Can not connect to mysql error 10061″. ### Set mysql root password Before accessing the database by console you need to type: $ mysql -u root

At the mysql console type:

$mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('yourpassword'); A successful mysql command will show: Query OK, 0 rows affected (0.00 sec) Mysql commands can span several lines. Do not forget to end your mysql command with a semicolon. Note: If you have already set a password for the mysql root, you will need to use: $ mysql -u root -p

$mysql> CREATE DATABASE database1; ### Create a mysql user For creating a new user with all privileges (use only for troubleshooting), at mysql prompt type: $ mysql> GRANT ALL PRIVILEGES ON *.* TO 'yourusername'@'localhost' IDENTIFIED BY 'yourpassword' WITH GRANT OPTION;

For creating a new user with fewer privileges (should work for most web applications) which can only use the database named “database1″, at mysql prompt type:

$mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES ON database1.* TO 'yourusername'@'localhost' IDENTIFIED BY 'yourpassword'; yourusername and yourpassword can be anything you like. database1 is the name of the database the user gets access to.localhost is the location which gets access to your database. You can change it to ‘%’ (or to hostnames or ip addresses) to allow connections from every location (or only from specific locations) to the database. Note, that this can be a security problem and should only be used for testing purposes! To exit the mysql prompt type: $ mysql> \q

Since the mysql root password is now set, if you need to use mysql again (as the mysql root), you will need to use:

and

$mysqladmin -u root -p create database1 mysqladmin is a command-line tool provided by the default LAMP install. ## Phpmyadmin and mysql-workbench All mysql tasks including setting the root password and creating databases can be done via a graphical interface usingphpmyadmin or mysql-workbench. To install one or both of them, first enable the universe repository Use any method to install phpmyadmin ### Troubleshooting Phpmyadmin & mysql-workbench If you get blowfish_secret error: Choose and set a phrase for cryptography in the file /etc/phpmyadmin/blowfish_secret.inc.php and copy the line (not the php tags) into the file /etc/phpmyadmin/config.inc.php or you will receive an error. If you get a 404 error upon visiting http://localhost/phpmyadmin: You will need to configure apache2.conf to work with Phpmyadmin. $ gksudo gedit /etc/apache2/apache2.conf

Include the following line at the bottom of the file, save and quit.

$Include /etc/phpmyadmin/apache.conf ### Alternative: install phpMyAdmin from source See the phpMyAdmin page for instructions on how to install phpmyadmin from source: ### Mysql-workbench Mysql-workbench runs locally, on the desktop. Use any method to install mysql-workbench ### For more information 2.9.3. Securing the Initial MySQL Accounts from the MySQL Reference Manual is worth reading. ## Edit Apache Configuration You may want your current user to be the PHP pages administrator. To do so, edit the Apache configuration file : $ gksudo "gedit /etc/apache2/envvars"

Search both the strings starting by “APACHE_RUN_USER” and “APACHE_RUN_GROUP”, and change the names to the current username and groupname you are using. Then you’ll need to restart Apache. (look at the next chapter concerning apache commands)

Configuration options relating specifically to user websites (accessed through localhost/~username) are in /etc/apache2/mods-enabled/userdir.conf.

## Installing suPHP

suPHP is a tool for executing PHP scripts with the permissions of their owners. It consists of an Apache module (mod_suphp) and a setuid root binary (suphp) that is called by the Apache module to change the uid of the process executing the PHP interpreter.

Note: suPHP enforces, security and helps avoid file permission problems under development environments with several users editing the site files, but it also demands more memory and CPU usage, which can degrade your server performance under certain circumstances.

To only install suPHP. use any method to install the package

libapache2-mod-suphp

Enable this module by doing

sudo a2enmod suphp

then use a text editor such as “sudo nano” at the command line or “gksudo gedit” on the desktop to edit this file

sudo nano /etc/apache2/mods-available/php5.conf

or

gksu "gedit /etc/apache2/mods-available/php5.conf"

make a new empty line at the top of the content, then add

<Directory /usr/share>

make a new empty line at the bottom of the content, then add

</Directory>

save changes

For security reasons we need to specify to suPHP what are the document paths allowed to execute scripts, use a text editor such as “sudo nano” at the command line or “gksudo gedit” on the desktop to edit this file

sudo nano /etc/suphp/suphp.conf

or

gksu "gedit /etc/suphp/suphp.conf

find the value “docroot” and specify the document path of your site files, for example:

docroot=/var/www/

that value restrict script execution only to files inside “/var/www/”

docroot=/var/www/:${HOME}/public_html that value restrict script execution only to files inside a custom home folder for each configured user inside “/var/www/:${HOME}/public_html”

for this tutorial we are going to use this value

docroot=/home/user/public_html/

which is the same Apache directory directive set before in this document

save changes

to restart Apache, type in your terminal

sudo /etc/init.d/apache2 restart

Now lets create a test script to see if suPHP is working correctly, in your terminal type

echo "<?php echo 'whoim = '.exec('/usr/bin/whoami');?>" | tee /home/user/public_html/whomi.php

that command creates a quick php test file to display the current user executing the script

open your browser and navigate to “localhost/whomi.php”, most likely the browser will show you a “500″ server error, this is because suPHP does not allow too permissive file and folder permissions and also does not allow mixed file and folder ownership, to correct this type in your terminal

sudo find /home/user/public_html/ -type f -exec chmod 644 {} \;
sudo find /home/user/public_html/ -type d -exec chmod 755 {} \;
sudo chown user:group -R /home/user/public_html/

those commands enforce a secure and correct file and folder permission and also set a correct user and group ownership for all of them

Now open your browser and navigate to “localhost/whomi.php”, if everything went fine you should see the name of the file owner executing the script and not “www-data” unless you specified so

## Run, Stop, Test, And Restart Apache

Use the following command to run Apache :

$sudo /usr/sbin/apache2ctl start To stop it, use : $ sudo /usr/sbin/apache2ctl stop

To test configuration changes, use :

$sudo /usr/sbin/apache2ctl configtest Finally, to restart it, run : $ sudo /usr/sbin/apache2ctl restart

Alternatively, you can use a graphical interface by installing Rapache or the simpler localhost-indicator.

## Using Apache

You can access apache by typing 127.0.0.1 or http://localhost (by default it will be listening on port 80) in your browser address bar.By default the directory for apache server pages is /var/www . It needs root access in order to put files in. A way to do it is just starting the file browser as root in a terminal:

$gksudo nautilus or if you want to make /var/www your own. (Use only for non-production web servers – this is not the most secure way to do things.) $ sudo chown -R $USER:$USER /var/www

## Status

To check the status of your PHP installation:

 $gksudo "gedit /var/www/testphp.php" and insert the following line  <?php phpinfo(); ?> View this page on a web browser at http://yourserveripaddress/testphp.php or http://localhost/testphp.php ## Securing Apache If you just want to run your Apache install as a development server and want to prevent it from listening for incoming connection attempts, this is easy to do. $ gksudo "gedit /etc/apache2/ports.conf"
\$ password:

Change ports.conf so that it contains:

Listen 127.0.0.1:80

Save this file, and restart Apache (see above). Now Apache will serve only to your home domain, http://127.0.0.1 or http://localhost.

There are 2 ways to password-protect a specific directory. The recommended way involves editing  /etc/apache2/apache2.conf .(To do this, you need root access). The other way involves editing a .htaccess file in the directory to be protected. (To do this, you need access to that directory).

### Password-Protect a Directory With .htaccess

Warning: On at least some versions of Ubuntu, .htaccess files will not work by default. SeeEnablingUseOfApacheHtaccessFiles for help on enabling them.

## thumbnails

If you direct your web browser to a directory (rather than a specific file), and there is no “index.html” file in that directory, Apache will generate an index file on-the-fly listing all the files and folders in that directory. Each folder has a little icon of a folder next to it.

To put a thumbnail of that specific image (rather than the generic “image icon”) next to each image file (.jpg, .png, etc.):

… todo: add instructions on how to do thumbnails here, perhaps using Apache::AutoIndex 0.08 or Apache::Album 0.95 …

## Known problems

### Skype incompatibility

Skype uses port 80 for incoming calls, and thus, may block Apache. The solution is to change the port in one of the applications. Usually, port 81 is free and works fine. To change the port number in Skype go to menu Tools > Options, then click on the Advanced tab, then in the box of the port for incoming calls write your preference.

# Pictures from a developer’s life

A friend sent me this link that was composed from other links from other blogs. The origin is here and another here

Sorry for the brute copy-past, but it’s much easier to read as a whole – you can check the links for more…

## 25 Apache Performance Tuning Tips

We all know and love Apache.  Its great, it allows us to run websites on the Internet with minimal configuration and administration.

However, this same ease of flexibility and lack of tuning, is typically what leads Apache to becoming a memory hog.  Utilizing these easy to understand tips, you can gain a significant performance boost from Apache.

### Apache Specifics

1. Remove unused modules – save memory by not loading modules that you do not need, including but not limited to mod_php, mod_ruby, mod_perl, etc.

2. Use mod_disk_cache NOT mod_mem_cache - mod_mem_cache will not share its cache amongst different apache processes, which results in high memory usage with little performance gain since on an active server, mod_mem_cache will rarely serve the same page twice in the same apache process.

3. Configure mod_disk_cache with a flat hierarchy – ensure that you are using CacheDirLength=2 and CacheDirLevels=1 to ensure htcacheclean will not take forever when cleaning up your cache directory.

4. Setup appropriate Expires, Etag, and Cache-Control Headers – to utilize your cache, you must tell it when a file expires, otherwise your client will not experience the caching benefits.

5. Put Cache on separate disk – place your cache on a separate physical disk for fastest access without slowing down other processes.

6. Use Piped Logging instead of direct logging – directly logging to a file has issues when you want to rotate the log file.  It must restart apache to use the next log file.  This will cause significant slowness for your users during the restart.  Particularly if you are using Passenger or some other app loader.

7. Log to a different disk than disk serving pages – put your logs on physically different disks than the files you are serving.

8. Utilize mod_gzip/mod_deflate – gzip your content before sending it off and then the client will ungzip upon receipt, this will minimize the size of file transfers, it generally will help all user experience.

9. Turn HostnameLookups Off – stop doing expensive DNS lookups.  You will rarely ever need them and when you do, you can look them up after the fact.

10. Avoid using hostname in configs – if you have HostnameLookups off, this will prevent you from having to wait for the DNS resolve of the hostnames in your configs, use IP addresses instead.

11. Use Persistent Connections - Set KeepAlive On and then set KeepAliveTimeout and KeepAliveRequests.  KeepAliveTimeout is how long apache will wait for the next request, and KeepAliveRequests is the max number of requests for a client prior to resetting the connection.  This will prevent the client from having to reconnect between each request.

12. Do Not set KeepAliveTimeout too high – if you have more requests than apache children, this setting can starve your pool of available clients.

13. Disable .htaccess - i.e. AllowOverride None  This will prevent apache from having to check for a .htaccess file on each request.

14. Allow symlinks – i.e. Options +FollowSymLinks -SymLinksIfOwnerMatch.  Otherwise, apache will make a separate call on each filename to ensure it is not a symlink.

15. Set ExtendedStatus Off - Although very useful, the ExtendedStatus will produce several system calls for each request to gather statistics.  Better to utilize for a set time period in order to benchmark, then turn back off.

16. Avoid Wildcards in DirectoryIndex – use a specific DirectoryIndex, i.e. index.html or index.php, not index

17. Increase Swappiness – particularly on single site hosts this will increase performance.  On linux systems increase /proc/sys/vm/swappiness to at least 60 if not greater.  This will try to load as many files as possible into the memory cache for faster access.

18. Increase Write Buffer Size – increase your write buffer size for tcp/ip buffers.  On linux systems increase /proc/sys/net/core/wmem_max and /proc/sys/net/core/wmem_default. If your pages fit within this buffer, apache will complete a process in one call to the tcp/ip buffer.

19. Increase Max Open Files – if you are handling high loads increase the number of allowed open files.  On linux, increase /proc/sys/fs/file-max and run ulimit -H -n 4096.

### Application Specifics

20. Setup Frontend proxy for images and stylesheets – allow your main web servers to process the application while images and stylesheets are served from frontend webservers

21. Use mod_passenger for rails – mod_passenger is able to share memory and resources amongst several processes, allowing for faster spawning of new application instances.  It will also monitor these processes and remove them when they are unnecessary.

22. Turn off safe_mode for php – it will utilize about 50-70% of your script time checking against these safe directives.  Instead configure open_base_dir properly and utilize plugins such as mod_itk.

23. Don’t use threaded mpm with mod_php – look at using mod_itk, mod_php tends to segfault with threaded mpm.

24. Flush buffers early for pre-render - it takes a relatively long time to create a web page on the backend, flush your buffer prior to page completion to send a partial page to the client, so it can start rendering.  A good place to do this is right after the HEAD section – so that the browser can start fetching other objects.

25. Use a Cache for frequently accessed data - memcached is a great for frequently used data and sessions.  It will speed up your apache render time as databases are slow.

In a previous article, I gave an introduction to programming with CUDA. Now I’d like to go into a little bit more depth about the CUDA thread execution model and the architecture of a CUDA enabled GPU. I assume that the reader has basic knowledge about CUDA and already knows how to setup a project that uses the CUDA runtime API. If you don’t know how to setup a project with CUDA, you can refer to my previous article:Introduction to CUDA.

# GPU Architecture

To understand the thread execution model for modern GPU’s, we must first make an analysis of the GPU compute architecture. In this article I will focus on the Fermi compute architecture found in modern GPU’s (GTX 580).

## Overview of the Fermi Architecture

A Fermi GPU consists of 512 CUDA cores. These 512 CUDA cores are split across 16 Streaming Multiprocessors (SM) each SM consisting of 32 CUDA cores. The GPU has 6 64-bit memory partitions supporting up to 6 GB of GDDR5 DRAM memory.

Fermi Arcitecture

Each streaming multiprocessor (SM) has 32 cuda cores. Each CUDA core consists of an integer arithmetic logic unit (ALU) and a floating point unit (FPU).

Fermi Streaming Multiprocessor (SM)

The SM has 16 load/store units allowing source and destination addresses to be calculated for sixteen threads per clock.

Each SM also has four Special Function Units (SFU) that execute transcendental instructions such as sin, cosine, reciprocal, and square root.

Now that we’ve seen the specific architecture of a Fermi GPU, let’s analyze the more general CUDA thread execution model.

Each kernel function is executed in a grid of threads. This grid is divided into blocks also known as thread blocks and each block is further divided into threads.

Cuda Execution Model

In the image above we see that this example grid is divided into nine thread blocks (3×3), each thread block consists of 9 threads (3×3) for a total of 81 threads for the kernel grid.

This image only shows 2-dimensional grid, but if the graphics device supports compute capability 2.0, then the grid of thread blocks can actually be partitioned into 1, 2 or 3 dimensions, otherwise if the device supports compute capability 1.x, then thread blocks can be partitioned into 1, or 2 dimensions (in this case, then the 3rd dimension should always be set to 1).

The thread block is partitioned into individual threads and for all compute capabilities, threads can be partitioned into 1, 2, or 3 dimensions. The maximum number of threads that can be assigned to a thread block is 512 for devices with compute capability 1.x and 1024 threads for devices that support compute capability 2.0.

 Technical Specifications 1 1.1 1.2 1.3 2.0 Maximum dimensionality of a grid of thread blocks 2 3 Maximum x-, y-, or z-dimension of a grid of thread blocks 65535 Maximum dimensionality of a thread block 3 Maximum x- or y-dimension of a block 512 1024 Maximum z-dimension of a block 64 Maximum number of threads per block 512 1024

The number of blocks within a gird can be determined within a kernel by using the built-in variable gridDim and the number of threads within a block can be determined by using the built-in variable blockDim.

A thread block is uniquely identified in a kernel function by using the built-in variableblockIdx and a thread within a block is uniquely identified in a kernel function by using the built-in variable threadIdx.

The built-in variables gridDimblockDimblockIdx, and threadIdx are each 3-component structs with members x, y, z.

With a 1-D kernel, the unique thread ID within a block is the same as the x component of the threadIdx variable.

$threadID=threadIdx.x$

and the unique block ID within a grid is the same as the x component of the blockIdx variable:

$blockID=blockIdx.x$

To determine the unique thread ID in a 2-D block, you would use the following formula:

$threadID=(threadIdx.y*blockDim.x)+threadIdx.x$

and to determine the unique block ID within a 2-D grid, you would use the following formula:

$blockID=(blockIdx.y*gridDim.x)+blockIdx.x$

I’ll leave it as an exercise for the reader to determine the formula to compute the unique thread ID and block ID in a 3D grid.

Let’s take a look at an example kernel that one might execute.

Let’s assume we want to implement a kernel function that adds two matrices and stores the result in a 3rd.

The general formula for matrix addition is:

$\begin{matrix}C && = && A+B \\ c_{i,j} && = && a_{i,j}+b_{i,j}\end{matrix}$

That is, the sum of matrix A and matrix B is the sum of the components of matrix A and matrix B.

Let’s first write the host version of this method that we would execute on the CPU.

 1 2 3 4 5 6 7 8 9 10 11 void MatrixAddHost( float* C, float* A, float* B, unsigned int matrixRank ) {     for( unsigned int j = 0; j < matrixRank; ++j )     {         for ( unsigned int i = 0; i < matrixRank; ++i )         {             unsigned int index = ( j * matrixRank ) + i;             C[index] = A[index] + B[index];         }     } }

This is a pretty standard method that loops through the rows and columns of a matrix and adds the components and stores the results in a 3rd. Now let’s see how we might execute this kernel on the GPU using CUDA.

First, we need to think of the problem domain. I this case, the domain is trivial: it is the components of a matrix. Since we are operating on 2-D arrays, it seems reasonable to split our domain into two dimensions; one for the rows, and another for the columns of the matrices.

We will assume that we are working on square matrices. This simplifies the problem but mathematically matrix addition only requires that the two matrices have the same number of rows and columns but does not have the requirement that the matrices must be square.

Since we know that a kernel is limited to 512 threads/block with compute capability 1.x and 1024 threads/block with compute capability 2.0, then we know we can split our job into square thread blocks each consisting of 16×16 threads (256 threads per block) with compute capability 1.x and 32×32 threads (1024 threads per block) with compute capability 2.0.

For simplicity, I will assume compute capability 1.x for the remainder of this tutorial.

If we limit the size of our matrix to no larger than 16×16, then we only need a single block to compute the matrix sum and our kernel execution configuration might look something like this:

 1 2 3 dim3 gridDim( 1, 1, 1 ); dim3 blockDim( matrixRank, matrixRank, 1 ); MatrixAddDevice<<>>( C, A, B, matrixRank );

In this simple case, the kernel grid consists of only a single block with matrixRank xmatrixRank threads.

However, if we want to sum matrices larger than 512 components, then we must split our problem domain into smaller groups that can be processed in multiple blocks.

Let’s assume that we want to limit our blocks to execute in 16×16 (256) threads. We can determine the number of blocks that will be required to operate on the entire array by dividing the size of the matrix dimension by the maximum number of threads per block and round-up to the nearest whole number:

$blocks=Ceiling\lceil{\frac{matrixRank}{16}}\rceil$

And we can determine the number of threads per block by dividing the size of the matrix dimension by the number of blocks and round-up to the nearest whole number:

$threads=Ceiling\lceil{\frac{matrixRank}{blocks}}\rceil$

So for example, for a 4×4 matrix, we would get

$\begin{matrix}blocks && = && \lceil\frac{4}{16}\rceil \\ blocks && = && \lceil{0.25}\rceil \\ blocks && = && 1\end{matrix}$

and the number of threads is computed as:

$\begin{matrix}threads && = && \lceil\frac{4}{1}\rceil \\ threads && = && 4\end{matrix}$

resulting in a 1×1 grid of 4×4 thread blocks for a total of 16 threads.

Another example a 512×512 matirx, we would get:

$\begin{matrix}blocks && = && \lceil\frac{512}{16}\rceil \\ blocks && = && \lceil{32}\rceil \\ blocks && = && 32\end{matrix}$

and the number of threads is computed as:

$\begin{matrix}threads && = && \lceil\frac{512}{32}\rceil \\ threads && = && 16\end{matrix}$

resulting in a 32×32 grid of 16×16 thread blocks for a total of 262,144 threads.

The host code to setup the kernel granularity might look like this:

 1 2 3 4 5 6 size_t blocks = ceilf( matrixRank / 16.0f ); dim3 gridDim( blocks, blocks, 1 ); size_t threads = ceilf( matrixRank / (float)blocks ); dim3 blockDim( threads, threads, 1 );   MatrixAddDevice<<< gridDim, blockDim >>>( C, A, B, matrixRank );
You may have noticed that if the size of the matrix does not fit nicely into equally divisible blocks, then we may get more threads than are needed to process the array. It is not possible to configure a gird of thread blocks with 1 block containing less threads than the others. The only way to solve this is to execute multiple kernels – one that handles all the equally divisible blocks, and a 2nd kernel invocation that handles the partial block. The other solution to this problem is simply to ignore any of the threads that are executed outside of our problem domain which is generally the easier (and more efficient) than invoking multiple kernels (this should be profiled to be proven).

## The Matrix Addition Kernel Function

On the device, one kernel function is created for every thread in the problem domain (the matrix elements). We can use the built-in variables gridDimblockDimblockIdx, and threadIdx, to identify the current matrix element that the current kernel is operating on.

If we assume we have a 9×9 matrix and we split the problem domain into 3×3 blocks each consisting of 3×3 threads as shown in the CUDA Grid below, then we could compute the ith column and the jth row of the matrix with the following formula:

$\begin{matrix}i && = && (blockDim.x*blockIdx.x)+threadIdx.x \\ j && = && (blockDim.y*blockIdx.y)+threadIdx.y\end{matrix}$

So for thread (0,0) of block (1,1) of our 9×9 matrix, we would get:

$\begin{matrix}i && = && (3*1)+0 \\ i && = && 3\end{matrix}$

for the column and:

$\begin{matrix}j && = && (3*1)+0 \\ j && = && 3\end{matrix}$

for the row.

The index into the 1-D buffer that store the matrix is then computed as:

$index=(rank*i)+j$

and substituting gives:

$\begin{matrix}index && = && (rank*3)+3 \\ index && = && (9*3)+3 \\ index && = && 30\end{matrix}$

Which is the correct element in the matrix. This solution assumes we are accessing the matrix in row-major order.

CUDA Grid Example

Let’s see how we might implement this in the kernel.

 1 2 3 4 5 6 7 8 9 10 11 __global__ void MatrixAddDevice( float* C, float* A, float* B, unsigned int matrixRank ) {     unsigned int column = ( blockDim.x * blockIdx.x ) + threadIdx.x;     unsigned int row    = ( blockDim.y * blockIdx.y ) + threadIdx.y;       unsigned int index = ( matrixRank * row ) + column;     if ( index < matrixRank * matrixRank ) // prevent reading/writing array out-of-bounds.     {         C[index] = A[index] + B[index];     } }

On line 3, and 4 we compute the column and row of the matrix we are operating on using the formulas shown earlier.

On line 6, the 1-d index in the matrix array is computed based on the size of a single dimension of the square matris.

We must be careful that we don’t try to read or write out of the bounds of the matrix. This might happen if the size of the matrix does not fit nicely into the size of the CUDA grid (in the case of matrices whose size is not evenly divisible by 16) To protect the read and write operation, on line 7 we must check that the computed index does not exceed the size of our array.

CUDA provides a synchronization barrier for all threads in a block through the__syncthreads() method. A practical example of thread synchronization will be shown in a later article about optimization a CUDA kernel, but for now it’s only important that you know this functionality exists.

Thread synchronization is only possible across all threads in a block but not across all threads running in the grid. By not allowing threads across blocks to be synchronized, CUDA enables multiple blocks to be executed on other streaming multiprocessors (SM) in any order. The queue of blocks can be distributed to any SM without having to wait for blocks from another SM to be complete. This allows the CUDA enabled applications to scale across platforms that have more SM at it’s disposal, executing more blocks concurrently than another platforms with less SM’s.

Thread synchronization follows strict synchronization rules. All threads in a block must hit the synchronization point or none of them must hit synchronization point.

Give the following code block:

 1 2 3 4 5 6 7 8 if ( threadID % 2 == 0 ) {     __syncthreads(); } else {     __syncthreads(); }

will cause the threads in a block to wait indefinitely for each other because the two occurrences of __syncthreads are considered separate synchronization points and all threads of the same block must hit the same synchronization point, or all of them must not hit it.

When a kernel is invoked, the CUDA runtime will distribute the blocks across the SM’s on the device. A maximum of 8 blocks (irrelevant of platform) will be assigned to each SM as long as there are enough resources (registers, shared memory, and threads) to execute all the blocks. In the case where there are not enough resources on the SM, then the CUDA runtime will automatically assign less blocks per SM until the resource usage is below the maximum per SM.

The total number of blocks that can be executed concurrently is dependent on the device. In the case of the Fermi architecture discussed earlier, a total of 16 SM’s can concurrently handle 8 blocks for a total of 128 blocks executing concurrently on the device.

Because the Fermi architecture support compute compatibility 2.0, we can create thread blocks consisting of at most 1024 threads, then the Fermi device can technically support 131,072 threads residing in the SM’s for execution. This does not mean that every clock tick the devices is executing 131,072 instruction simultaneously. In order to understand how the blocks are actually executed on the device, we must look one step further to see how the threads of a block are actually scheduled on the SM’s.

When a block is assigned to a SM, it is further divided into groups of 32 threads called a warp. Warp scheduling is different depending on the platform, but if we take a look at the Fermi architecture, we see that a single SM consists of 32 CUDA cores (or streaming processor) – two groups of 16 per SM.

Each SM in the Fermi architecture (see Fermi architecture image above) features two warp schedulers allowing two warps to be issued and executed concurrently. Fermi’s dual-warp scheduler selects two warps and issues one instruction from each warp to a group of sixteen cores, sixteen load/store units, or four special function units (SFU’s).

Most instructions can be dual-issued; two integer instructions, two floating point instructions, or a mix of integer, floating point, load, store, and SFU instructions can be issued concurrently.

Fermi – Dual Warp Scheduler

You might be wondering why it would be useful to schedule 8 blocks of a maximum of 1024 threads if the SM only has 32 SP’s? The answer is that each instruction of a kernel may require more than a few clock cycles to execute (for example, an instruction to read from global memory will require multiple clock cycles). Any instruction that requires multiple clock cycles to execute incurs latency. The latency of long-running instructions can be hidden by executing instructions from other warps while waiting for the result of the previous warp. This technique of filling the latency of expensive operations with work from other threads is often called latency hiding.

It is reasonable to imagine that your CUDA program contains flow-control statements like if-then-elseswitchwhile loops, or for loops. Whenever you introduce these flow-control statements in your code, you also introduce the possibility of thread divergence. It is important to be aware of the consequence of thread divergence and also to understand how you can minimize the negative impact of divergence.

Thread divergence occurs when some threads in a warp follow a different execution path than others. Let’s take the following code block as an example:

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 __global__ void TestDivergence( float* dst, float* src ) {     unsigned int index = ( blockDim.x * blockIdx.x ) + threadIdx.x;     float value = 0.0f;       if ( threadIdx.x % 2 == 0 )     {         // Threads executing PathA are active while threads         // executing PathB are inactive.         value = PathA( src );     }     else     {         // Threads executing PathB are active while threads          // executing PathA are inactive.         value = PathB( src );     }     // Threads converge here again and execute in parallel.     dst[index] = value; }

Then our flow control and thread divergence would look something like this:

As you can see from this example, the even numbered threads in each block will execute PathA while the odd numbered threads in the block will execute PathB. This is pretty much the worst-case scenario for simple divergence example.

Both PathA and PathB cannot be executed concurrently on all threads because their execution paths are different. Only the threads that execute the exact same execution path can run concurrently so the total running time of the warp is the sum of the execution time of both PathA and PathB.

In this example, the threads in the warp that execute PathA are activated if the condition is true and all the other threads are deactivated. Then, in another pass, all the threads that execute PathB are activated if the condition is false are activated and the other threads are deactivated. This means that to resolve this condition requires 2-passes to be executed for a single warp.

The overhead of having the warp execute both PathA and PathB can be eliminated if the programmer takes careful consideration when writing the kernel. If possible, all threads of a block (since warps can’t span thread blocks) should execute the same execution path. This way you guarantee that all threads in a warp will execute the same execution path and there will be no thread divergence within a block.

# Exercise

If a device supports compute capability 1.3 then it can have blocks with a maximum of 512 threads/block and 8 blocks/SM can be scheduled concurrently. Each SM can schedule groups of 32-thread units called warps. The maximum number of resident warps per SM in a device that supports compute capability 1.3 is 32 and the maximum number of resident threads per SM is 1024.

Q. What would be the ideal block granularity to compute the product of two 2-D matrices of size 1024 x 1024?

1. 4×4?
2. 8×8?
3. 16×16?
4. or 32×32?

A. To answer this question, let’s analyze each choice and give pros and cons for each one.

4×4: If we decide to split our domain into 4×4 thread blocks, then we have 16 threads per block. In order to fully occupy the SM that can support 1024 threads per SM, we would need 1024/16 = 64 blocks but the SM can only schedule 8 blocks/SM so each SM would be scheduled with 8 blocks each having 16 threads which is 128 threads/SM. When divided into warps, we only have 4 warps scheduled per SM out of a total of 32 which gives only 12.5% occupancy.

8×8: We have the same problem here as we did with the 4×4 thread block granularity except not as severe. With 8×8 thread blocks, we get 64 threads per block. For a SM that can support 1024 threads per SM, we would need 1024/64 = 16 blocks but since we are limited to 8 blocks maximum per SM, we can only execute 8×64 = 512 threads/SM. When split into warps of 32-threads each, we get 512/32 = 16 warps scheduled per SM from a possible total 32 warps. This give only 50% occupancy.

16×16: A 16×16 thread block gives 256 threads/block. With a maximum thread limit per SM of 1024, we get 1024/256 = 4 blocks/SM. This is within the 8 block limit so 4 blocks, each of 256 threads can be scheduled on one SM. With 4 blocks each with 256 threads, we get a total of 1024 threads. The threads are further split into warps of 32 threads each for a total of 32 warps. Since the device can support 32 warps/SM we have achieved 100% occupancy.

32×32: This option is not even an option since a 32×32 thread block produces a single block with 1024 threads. As stated earlier, we are limited to 512 threads per block with compute capability 1.3 so our kernel wouldn’t even run.

So the best choice for this problem domain would be to invoke a kernel with block size16×16.

# Conclusion

In this article, I discussed the architecture of a CUDA enabled GPU, in particular the Fermi architecture. I also showed how a kernel function is scheduled on the GPU and how the warp scheduler executes instructions from different warps in order to minimize the amount of noticeable latency between kernel instructions.

Posted on November 15, 2011 by

## UPDATED! AMD Announces FX-9590 and FX-9370: Return of the GHz Race

AMD Announces FX-9590 and FX-9370: Return of the GHz Race

Today at E3 AMD announced their latest CPUs, the FX-9590 and FX-9370. Similar to what we’re seeing with Richland vs. Trinity, AMD is incrementing the series number to 9000 while sticking with the existing Piledriver Vishera architecture. These chips are the result of tuning and binning on GlobalFoundries’ 32nm SOI process, though the latest jump from the existing FX-8350 is nonetheless quite impressive.

The FX-8350 had a base clock of 4.0GHz with a maximum Turbo Core clock of 4.2GHz; the FX-9590 in contrast has a maximum Turbo clock of 5GHz and the FX-9370 tops out at 4.7GHz. We’ve asked AMD for details on the base clocks for the new parts, but so far have not yet received a response; we’re also missing details on TDP, cache size, etc. but those will likely be the same as the FX-8350/8320 (at least for everything but TDP).

## CPU vs GPU performance

in the performance of GPUs and CPUs. This has to be quite a compromise since actual performance depends heavily on the suitability of the chip to a particular problem/algorithm among many other specifics. The simplest method is to plot theoretical peak performance over time; I chose to show it for single and double precision for NVIDIA GPUs and Intel CPUs.

In the past, I have used the graph in the CUDA C Programming Guide, but that is frequently out of date, I have no control of the formatting, I have to settle for a screenshot instead of vector output, and, until I did my own research, I wasn’t sure if it was biased.

Below is my current attempt (click to enlarge).

## Whom would you cast if you were to make movie on Mahabharata in Hollywood/Indian cinema?

Dronacharya (Sanskrit: द्रोणाचार्य, droṇācārya)
[ Liam Neeson ]
He was the royal guru to Kauravas and Pandavas. He was a master of advanced military arts, including the Devāstras. It was not possible to defeat Droṇācārya when he had bow and arrow in his hands!

Arjuna (Sanskrit: अर्जुन,  ɐrˈɟunɐ)
[ Christian Bale ]
Arjuna, considered the finest archer and a peerless warrior by many notable figures in the Mahabharata such as Bhishma, Drona, Krishna, Vidura, Sage Naradha and Dhiritharashtra, played a key role in ensuring the defeat of the Kauravas in theKurukshetra War. He is the only undefeated hero in the Mahabharata. Dronacharya(Liam Neeson)’s favorite student.

Christian Bale as Bruce Wayne in Batman: Begins

Vidura (Sanskrit: विदुर, Vidura)
[ Michael Caine ]
Vidura was most respected as an advisor by the Pandavas whom he forewarned on various occasions of Duryodhana’s plots to exterminate them, such asDuryodhana‘s plan to burn them alive in the Lakshagraha or “House of Lac“. He was known for his statecraft and wisdom. Vidura is held to be a paragon of truth, dutifulness, impartial judgement and steadfast dharma. He is considered the embodiment of the inner consciousness of the Mahabharata.

Michael Caine as Alfred in Batman

Krishna (Sanskrit: कृष्ण, Kṛṣṇa)
[ Morgan Freeman ]
Based on his name, Krishna is often depicted in murtis as black or blue-skinned. Krishna is central to many of the main stories of the epic. The eighteen chapters of the sixth book (Bhishma Parva) of the epic that constitute the Bhagavad Gitacontain the advice of Krishna to the warrior-hero Arjuna, on the battlefield. Krishna had a profound effect on the Mahabharata war and its consequences. He considered the Kurukshetra war as a last resort by voluntarily making himself as a messenger in order to establish peace between the Pandavas and Kauravas. But, once these peace negotiations failed and was embarked into the war, then he became a clever strategist.

Morgan Freeman as God in Bruce Almighty

Bheeshma (Sanskrit: भीष्म)
[ Ian Mckellen ]
Bhishma is the one who witnessed the Mahābhārata completely from the beginning since the rule of Shantanu. In the great battle at Kurukshetra, Bhishma was the supreme commander of the Kaurava forces for ten days compared to Drona’s five, Karna’s two and Salya’s one-the last day. He fought reluctantly on the side of the Kauravas; nevertheless, he gave it his best effort. At one stageArjuna‘s disinclination to fight him, nearly made Krishna break his vow not to raise a weapon in the war. Not even Arjuna was able to defeat Bhishma because in addition to being a peerless warrior he was blessed with choosing the moment of his death.Bhishma was the strongest barrier that had protected the Kaurava camp from impending defeat.

Ian Mckellen as Gandalf, The White in Lord of the Rings

Indra (Sanskrit: इन्द्र)
[ Robert Downey Jr ]
Indra is the king of the gods and ruler of the heavens. He is the god of thunder and rain and a great warrior, a symbol of courage and strength. Indra wields a lightning thunderbolt known as vajra. He leads the Deva (the gods who form and maintainHeaven) and the elements, such as Agni (Fire), Varuna (Water) and Surya(Sun), and constantly wages war against the opponents of the gods, the demon-likeAsuras. As the god of war, he is also regarded as one of the Guardians of the directions, representing the east. Indra is also known to be a womanizer and is remarked as a trickster in many occasions.

Robert Downey Jr as Tony Stark in Iron Man

Shakuni (Sanskrit: शकुनि)
[ Ben Kingsley ]
He was the prince of Gandhara and one of the main villains in the Hindu . He was the brother of Gandhari and hence Duryodhana‘s maternal uncle. He is believed to be an extremely intelligent but devious man. Shakuni is often credited as the mastermind behind the Mahabharata war.

Ben Kingsley as Nizam in Price of Persia

Yudhisthira (Sanskrit: युधिष्ठिर, yudhiṣṭhira)
[ Tom Hanks ]
Yudhisthira’s true prowess was shown in his unflinching adherence to Satya(truth) and Dharma(righteousness to fulfill one’s moral duty), which were more precious to him than royal ambitions, material pursuits and family relations.

Tom Hanks as Forrest Gump in Forrest Gump

Bhima (Sanskrit: भीम, Bhīma)
[ Arnold Schwarzenegger ]
One of the central reasons behind the envy of Duryodhana towards the Pandavas was the inability of Kauravas to match Bhima’s strength. Eventually Bhima is responsible for slaying all hundred Kaurava brothers in the Kurukshetra War. Bhima slew Dushasana, the Kaurava most responsible for undressing Draupadi. On the final day of the battle, Bhima fought Duryodana and he struck both his thighs with his mace. Thus, he fulfilled the vow he took during Draupadi’s undressing.

Arnold Schwarzenegger as Terminator in Terminator 3

Duryodhana (Sanskrit: दुर्योधन)
[ Russel Crowe ]
Emperor of Bharatvarsha. Duryodhana’s body is said to be made out of lightning, and he is extremely strong, second only to Bhima. He is revered by his younger brothers, especially Dushasana. Learning martial skills from his gurus, Kripaand Drona, he proved to be extremely skilled with the mace. He then went to specialize inmace fighting under Balarama and went on to become his favourite pupil. With the mace he was equal to the much stronger Bhima. He is one of the heavily mis-understood characters of Mahabharat. When Karna is killed in the war, Duryodhana mourns his death intensely, even more than the death of his brothers. He accorded Karna, a place among the royals, by crowning him the King of Anga and standing by him when people pointed a finger at his lower-birth. A true Kshatriya(Warrior) – who choose to face Bhima in combat over all the other Pandavas, whom he could have easily defeated in war. In the end, as Duryodhana is about to die, he looks at Krishna malevolently. “I have been a good king,” he says. “I have conducted myself as a Kshatriya should and have come by death in battle. I’ll die and attain heaven, but you will live in grief and sorrow.” He fell back and flowers rained from the heavens to drop gently on his pain-wracked body.

Russel Crowe as Maximus in Gladiator

Karna (Sanskrit: कर्ण)
Karna was one of the greatest warriors whose martial exploits are recorded in the Mahābhārata, an admiration expressed by Krishnaand Bhishma within the body of this work. He was said to be born with a natural armor with power of Surya deva (Sun God). Karna was the closest friend of Duryodhana and fought on his behalf against the Pandavas (his brothers) in the famous Kurukshetra war. Karna fought against misfortune throughout his life and kept his word under all circumstances. Many admire him for his courage and generosity. Many believe that he was the greatest warrior of Mahabharata since he was only able to be defeated by Arjuna along with a combination of 3 curses, Indra’s efforts and Kunti’s request.

Brad Pitt as Achilles in Troy

[ Vidya Balan ]
Draupadi was a multifaceted personality: she could be fiery and angry when the situation called for it, but she still had a compassionate nature. She encouraged people to face life with the same inner strength that she did.

Vidya Blan as Silk in The Dirty Picture

Nakula (Sanskrit: नकुल)
[ Orlando Bloom ]
Nakula was known to be the most handsome person in the Kuru lineage. In his childhood, along with the other Pandava brothers, Nakula mastered his skills in archery under his father Pandu and a hermit named Suka at the Satasringa ashram. Especially, Nakula turned out to be an accomplished wielder of the sword. Expert in Herbology, Astronomy and Horse Keeping.

Orlando Bloom as Legolas in Lord of the Rings

[ Andrew Garfield ]
Like Nakula, Sahadeva’s core skill lay in the wielding of the sword. Sahadeva is said to be mild-mannered, bashful and virtuous. Expert in astronomy.

Andrew Garfield as Peter Parker in The Amazing Spiderman

[ Johnny Depp ]
Narada is portrayed as a travelling monk with the ability to visit distant worlds or planets (lokas in Sanskrit). He is often described as a mischievous character, though his intentions are always good at heart.

Johnny Depp as Jack Sparrow in Pirates of the Caribbean

Posted in Mixed | 1 Comment »

## GPU Acceleration In Multilayer Parallel User Authentication Schemes By Optimizing Hardware Use

P0212 GPU Acceleration In Multilayer Parallel User Authentication Schemes By Optimizing Hardware Use

Posted in Mixed | Tagged: , , , , | 1 Comment »

Ronzii's Blog

Karan Jitendra Thakkar

Everything I think. Everything I do. Right here.

VentureBeat

News About Tech, Money and Innovation

Chetan Solanki

Helpful to u, if u need it.....

ScreenCrush

Trying To Explorer Computer World ..!!

managedCUDA

Trying To Explorer Computer World ..!!

siddheshsathe

A great WordPress.com site

Ari's

This is My Space so Dont Mess With IT !!