Unix : Filenames,Relative Pathnames

Filenames
In UNIX, every file and directory has a name associated with it. This name is referred to as the file or
directory's filename.
In addition to their filenames, every file and directory is associated with the name of its parent directory.
When a filename is combined with the parent directory's name, the result is called a pathname. Two
examples of pathnames are
/home/ranga/docs/book/ch5.doc
/usr/local/bin/
As you can see, each of these pathnames consists of several "words" separated by the slash ( /) character.
In UNIX, the slash separates directories, whereas the individual words are the names of files or directories.
The sum of all the words and the / characters makes up the pathname.
The last set of characters in a pathname is the actual name of the file or directory being referred to: The rest
of the characters represent its parent directories. In the first example, the filename is ch5.doc.
The name of a file can be up to 255 characters long and can contain any ASCII character except /.
Generally, the characters used in pathnames are the alphanumeric characters ( a to z, A to Z, and 0 to 9)
along with periods ( .), hyphens ( -), and underscores ( _).
Other characters, especially the space, are usually avoided because many programs cannot deal with them
properly. For example, consider a file with the following name:
A Farewell To Arms
Most programs treat this a four separate files named A, Farewell, To, and Arms, instead of one file.
One thing to keep in mind about filenames is that two files in the same directory cannot have the same
name. Thus both of the following filenames
/home/ranga/docs/ch5.doc
/home/ranga/docs/ch5.doc
refer to the same file, but the following filenames
/home/ranga/docs/ch5.doc
/home/ranga/docs/books/ch5.doc
refer to different files because they are located in different directories. In addition, because UNIX is casesensitive,
you can have two files in the same directory whose names differ only by case. UNIX considers the
following
/home/ranga/docs/ch5.doc
/home/ranga/docs/CH5.doc
to be different files. This often confuses users coming from the Windows or DOS environments.

Relative Pathnames
A relative pathname enables you to access files and directories by specifying a path to that file or
directory within your current directory. When your current directory changes, the relative pathname to a file
can also change.
To find out what the current directory is, use the pwd (print working directory ) command, which
prints the name of the directory in which you are currently located. For example
$ pwd
/home/ranga/pub
tells me that I am located in the directory /home/ranga/pub.
When you're specifying a relative pathname, the slash character is not present at the beginning of the
pathname. This indicates that a relative pathname is being used instead of an absolute pathname. The
relative pathname is a list of the directories located between your current directory and the file or directory
you are representing.
If you are pointing to a directory in your pathname that is below your current one, you can access it by
specifying its name. For example, the directory name:
docs/
refers to the directory docs located in the current directory.
In order to access the current directory's parent directory or other directories at a higher level in the tree
than the current level, use the special name of two dots ( ..).
The UNIX file system uses two dots ( ..) to represent the directory above you in the tree, and a single dot (
.) to represent your current directory.
Look at an example that illustrates how relative pathnames are used. Assume that the current directory is
/home/ranga/work
Then the relative pathname

../docs/ch5.doc
represents the file
/home/ranga/docs/ch5.doc
whereas
./docs/ch5.doc
represents the file
/home/ranga/work/docs/ch5.doc
You can also refer to this file using the following relative path:
docs/ch5.doc
As mentioned previously, you do not have to append the ./ to the beginning of pathnames that refer to files
or directories located within the current directory or one of its subdirectories.

Linux : Working with Files

Listing Files

First, list the files and directories stored in the current directory. Use the following command:
$ ls
Here's a sample directory listing:
bin hosts lib res.03
ch07 hw1 pub test_results
ch07.bak hw2 res.01 users
docs hw3 res.02 work
This output indicates that several items are in the current directory, but this output does not tell us whether
these items are files or directories. To find out which of the items are files and which are directories, specify
the -F option to ls:

$ ls -F
Now the output for the directory is slightly different:
bin/ hosts lib/ res.03
ch07 hw1 pub/ test_results
ch07.bak hw2 res.01 users
docs/ hw3 res.02 work/
As you can see, some of the items now have a / at the end: each of these items is a directory. The other
items, such as hw1, have no character appended to them. This indicates that they are ordinary files.
When the -F option is specified to ls, it appends a character indicating the file type of each of the items it
lists. The exact character depends on your version of ls. For ordinary files, no character is appended. For
special files, a character such as !, @, or # is appended to the filename.
For more information on the exact characters your version of ls appends to the end of a filename when the -
F option is specified, please check the UNIX manual page for the ls command. You can do this as follows:
$ man ls
So far, you have seen ls list more than one file on a line. Although this is fine for humans reading the output,
it is hard to manipulate in a shell script. Shell scripts are geared toward dealing with lines of text, not the
individual words on a line.
"Filtering Text Using awk," it is hard to deal with the words on a line.
In a shell script it is much easier to manipulate the output when each file is listed on a separate line.
Fortunately ls supports the -1 option to do this. For example,
$ ls -1
produces the following listing:
bin
ch07
ch07.bak
docs
hosts
hw1
hw2
hw3
lib
pub
res.01
res.02
res.03
test_results
users
work


So far you have used ls to list visible files and directories, but ls can also list invisible or hidden
files and directories. An invisible file is one whose first character is the dot or period character ( .). UNIX
programs (including the shell) use most of these files to store configuration information. Some common
examples of hidden files include the files
l .profile, the Bourne shell ( sh) initialization script
l .kshrc, the Korn shell ( ksh) initialization script
l .cshrc, the C shell ( csh) initialization script
l .rhosts, the remote shell configuration file
All files that do not start with the . character are considered visible.
To list invisible files, specify the -a option to ls:
$ ls -a
The directory listing now looks like this:
. .profile docs lib test_results
.. .rhosts hosts pub users
.emacs bin hw1 res.01 work
.exrc ch07 hw2 res.02
.kshrc ch07.bak hw3 res.03
As you can see, this directory contains many invisible files.
Notice that in this output, the file type information is missing. To get the file type information, specify the -F
and the -a options as follows:
$ ls -a -F
The output changes to the following:
./ .profile docs/ lib/ test_results
../ .rhosts hosts pub/ users
.emacs bin/ hw1 res.01 work/
.exrc ch07 hw2 res.02
.kshrc ch07.bak hw3 res.03
With the file type information you see that there are two hidden directories (. and ..). These two directories
are special entries that are present in all directories. The first one, ., represents the current directory. The
second one, .., represents the parent directory. We discuss these concepts in greater detail.


Option Grouping

In the previous example, the command that you used specified the options to ls separately. These options
can also be grouped together. For example, the commands
$ ls -aF
$ ls -Fa
are the same as the command
$ ls -a -F
As you can see, the order of the options does not matter to ls. As an example of option grouping, consider
the equivalent following commands:
ls -1 -a -F
ls -1aF
ls -a1F
ls -Fa1
Any combination of the options -1, -a, and -F produces identical output:
./
../
.emacs
.exrc
.kshrc
.profile
.rhosts
bin/
ch07
ch07.bak
docs/
hosts
hw1
hw2
hw3
lib/
pub/
res.01
res.02
res.03
test_results
users
work/
Sams

Shell Script Basic bash

//b1
#!/bin/bash

family=tiangtae
name=narathip
echo "I am ${name} ${family}"


//b2
#!/bin/bash

age1=18
age2=25

echo "I am ${age1} years old"
echo "He is ${age2} years old"

//b3
#!/bin/bash

age1=18
age2=`expr $age1 + 7`

echo "I am ${age1} years old"
echo "He is ${age2} years old"


//b4
#!/bin/bash

echo $@
echo $#
echo $0
echo $1


//b5
#!/bin/bash

hour=`date +%H`

if [ "$hour" -lt 12 ]
then
echo "AM:"
else
echo "PM:"
fi


//b6
#!/bin/bash

hour=`date +%H`

if [ "$hour" -lt 12 ]
then
echo "AM:"
else
echo "PM:"
fi

# =
# !=
# -eq
# -ne
# -gt
# -ge
# -lt
# -le


//b7
#!/bin/bash

echo "Put the number: "
read num

if [ "$num" -lt 100 ]
then
echo "$num is less than 100"
elif [ "$num" -lt 1000 ]
then
echo "$num is less than 1000 and more than 100"
else
echo "$num is more than 1000"
fi


//b8
#!/bin/bash

echo "Put your score: "
read num

if [ "$num" -ge 0 -a "$num" -lt 7 ]
then
echo "You have to Phayayamm much more!"
elif [ "$num" -ge 7 -a "$num" -lt 15 ]
then
echo "Thammada!"
elif [ "$num" -ge 15 -a "$num" -le 20 ]
then
echo "Great!"
else
echo "Your score must be 0-20."
fi


//b9
#!/bin/bash

echo "Write your teacher: "
read name

if [ "$name" = "Narathip" -o "$name" = "Thanyarak" ]
then
echo "You are class A"
elif [ "$name" = "Somchai" -o "$name" = "Somsri" -o "$name" = "Somyot" ]
then
echo "You are class B"
elif [ "$name" = "Tukky" -o "$name" = "Tsubasa" ]
then
echo "Your are Johnny Junior's fan"
else
echo "Unavailable!"
fi


//b10
#!/bin/bash

if [ "$1" = "" ]
then
echo "Write your teacher'name in command line."
elif [ "$1" = "Narathip" -o "$1" = "Thanyarak" ]
then
echo "You are class A"
elif [ "$1" = "Somchai" -o "$1" = "Somsri" -o "$1" = "Somyot" ]
then
echo "You are class B"
elif [ "$1" = "Tukky" -o "$1" = "Tsubasa" ]
then
echo "Your are Johnny Junior's fan"
else
echo "Unavailable!"
fi


//b11
#!/bin/bash

while [ "$1" != "" ]
do
echo $1
shift
done


//b12
#!/bin/bash

num=3
while [ "$num" -lt 10 ]
do
echo '$num'" is $num"
num=`expr $num + 1`
done


//b13
#!/bin/bash

num=$1
while [ "$num" -lt 10 ]
do
echo '$num'" is $num"
num=`expr $num + 1`
done


//b14
#!/bin/bash

num=$RANDOM

echo "Put your favorite number: "

while true
do
echo "Put --> 0-32767"
read x
if [ $x -gt $num ]
then
echo "less than your number!"
elif [ $x -lt $num ]
then
echo "more than your number"
else
echo "OK!"
break
fi
done


//b15
#!/bin/bash

i=1
while [ "$i" -le 100 ]
do
touch temp${i}.txt
i=`expr $i + 1`
done


//b16
#!/bin/bash

i=1
while [ "$i" -le 100 ]
do
if [ "$i" -lt 10 ]
then
num=00${i}
elif [ "$i" -lt 100 ]
then
num=0${i}
else
num=${i}
fi

mv temp${i}.txt temp${num}.txt
i=`expr $i + 1`
done


//b17
#!/bin/bash

while [ "$1" != "" ]
do
name=`echo $1 cut -f1 -d'.'`
echo $1, $name
shift
done


//b18
#!/bin/bash

while [ "$1" != "" ]
do
name=`echo $1 cut -f1 -d'.'`
fname=${name}_e.html
echo $1, $fname
shift
done


//b19
#!/bin/bash

while [ "$1" != "" ]
do
name=`echo $1 cut -f1 -d'.'`
fname=${name}_e.html
cat $1 > $fname
mv $fname ~/public_html
chmod 604 ~/public_html/$fname
shift
done


//b20
#!/bin/bash

i=1
while [ "$i" -le 100 ]
do
echo "***** ${i} *****"
if [ -e temp0${i}.txt ]
then
echo "temp0${i}.txt already exists !"
fi
i=`expr $i + 1`
done


//b21
#!/bin/bash

i=1
while [ "$i" -le 100 ]
do
echo "***** ${i} *****"
if [ ! -e temp${i}.txt ]
then
touch tmp${i}.txt
fi
i=`expr $i + 5`
done


//b22
#!/bin/bash

for file in tmp1.txt tmp4.txt tmp6.txt
do
echo "*** ${file} ***"
done


//b23
#!/bin/bash

for file in `ls tmp*.txt`
do
echo "*** ${file} ***"
done


//b24
#!/bin/bash

for file in $(ls tmp*.txt)
do
echo "*** ${file} ***"
done


//b25
#!/bin/bash

for file in `ls tmp*.txt`
do
i=`echo $file sed -e "s/tmp//" sed -e "s/.txt//"`

if [ "$i" -lt 10 ]
then
num=00${i}
elif [ "$i" -lt 100 ]
then
num=0${i}
else
num=${i}
fi

mv tmp${i}.txt tmp${num}.txt
done


//b26
#!/bin/bash

for file in `ls tmp*.txt`
do
i=`echo $file sed -e "s/[a-z.]//g"`

if [ "$i" -lt 10 ]
then
num=00${i}
elif [ "$i" -lt 100 ]
then
num=0${i}
else
num=${i}
fi

mv tmp${i}.txt tmp${num}.txt
done

The UNIX System Linux shellscript

The UNIX System
Logging In
The UNIX system consists of two components:
l Utilities
l The kernel
Utilities are programs you can run or execute. The programs who and date that you saw in the
previous are examples of utilities. Almost every program that you know is considered a utility.
Commands are slightly different than utilities. The term utility refers to the name of a program,
whereas the term command refers to the program and any arguments you specify to that program to change
its behavior. You might see the term command used instead of the term utility for simple commands, where
only the program name to execute is given.
The kernel is the heart of the UNIX system. It provides utilities with a means of accessing a machine's
hardware. It also handles the scheduling and execution of commands.
When a machine is turned off, both the kernel and the utilities are stored on the machine's hard disks. But
when the computer is booted, the kernel is loaded from disk into memory. The kernel remains in memory
until the machine is turned off.
Utilities, on the other hand, are stored on disk and loaded into memory only when they are executed. For
example, when you execute the command
$ who
the kernel loads the who command from the machine's hard disk, places it in memory, and executes it.
When the program finishes executing, it remains in the machine's memory for a short period of time before it
is removed. This enables frequently used commands to execute faster. Consider what happens when you
execute the date command three times:
$ date
Sun Dec 27 09:42:37 PST 1998
$ date
Sun Dec 27 09:42:38 PST 1998
$ date
Sun Dec 27 09:42:39 PST 1998
The first time the date command can be loaded from the machine's hard disk, but the second and third time
the date command usually remains in the machine's memory allowing it to execute faster.
The shell is a program similar to the who command. The main difference is that the shell is loaded into
memory when you log in.
Logging In
When you first connect to a UNIX system, you usually see a prompt such as the following:
login:
You need to enter your username at this prompt. After you enter your username, another prompt is
presented:
login: ranga
Password:
You need to enter your password at this prompt.
These two prompts are presented by a program called getty. These are its tasks:
1. Display the prompt login.
2. Wait for a user to type a username.
3. After a username has been entered, display the password prompt.
4. Wait for a user to enter a password.
5. Give the username and password entered by the user to the login command and exit.
After login receives your username and password, it looks through the file /etc/passwd for an entry
matching the information you provided. If it finds a match, login executes a shell and exits.
As an example, on my system the matching entry for my username, ranga, in file /etc/passwd is:
ranga:x:500:100:Sriranga Veeraraghavan:/home/ranga:/bin/bash
As you progress through the book, I will explain the information stored here.
Note - For those readers who are not familiar with UNIX files or filenames such as /etc/passwd, this Working with Directories.
I will discuss files briefly . A general idea from other operating systems of what files are is
enough to understand these examples.
If no match is found, the login program issues an error message and exits. At this point the getty
program takes over and displays a new login prompt.
The shell that login executes is specified in the file /etc/passwd. Usually this is one of the shells that I
covered .
I assume that the shell started by the login program is /bin/sh. Depending on the version of
UNIX you are running, this might or might not be the Bourne shell:
l On Solaris and FreeBSD, it is the Bourne shell.
l On HP-UX, it is the POSIX shell.
l On Linux, it is the Bourne Again shell.


The UNIX System Summary
Shell Initialization Questions
Getting Help Terms
Previous Section Next Section
Shell Initialization
Interactive Versus Noninteractive Shells Making a Shell Script Executable
Initialization File Contents
When the login program executes a shell, that shell is uninitialized. When a shell is uninitialized, important
parameters required by the shell to function correctly are not defined.
The shell undergoes a phase called initialization to set up these parameters. This is usually a two step
process that involves the shell reading the following files:
l /etc/profile
l profile
The process is as follows:
1. The shell checks to see whether the file /etc/profile exists.
2. If it exists, the shell reads it. Otherwise, this file is skipped. No error message is displayed.
3. The shell checks to see whether the file .profile exists in your home directory. Your home
directory is the directory that you start out in after you log in.
4. If it exists, the shell reads it; otherwise, the shell skips it. No error message is displayed.
As soon as both of these files have been read, the shell displays a prompt:
$
This is the prompt where you can enter commands in order to have them execute.
Note - The shell initialization process detailed here applies to all Bourne type shells, but some additional
files are used by bash and ksh.
You can obtain more information about this process for a particular shell using the man command.
Interactive Versus Noninteractive Shells
When the shell displays a prompt for you, it is running in interactive mode.
Interactive mode means that the shell expects to read input from you and execute the commands
that you specify. This mode is called interactive because the shell is interacting with a user. This is usually
the mode of the shell that most users are familiar with: you log in, execute some commands, and log out.
When you log out using the exit command, the shell exits.
The shell can be run in another mode, called noninteractive mode . In this mode, the shell does
not interact with you; instead it reads commands stored in a file and executes them. When it reaches the
end of the file, the shell exits.
How login Starts a Shell
When the login program starts a shell, it basically executes the following command:
/bin/sh
By issuing this command, it puts the shell into interactive mode. You can start a shell in interactive mode by
issuing the same command at the prompt:
$ /bin/sh
$
The first prompt $ is displayed by the shell that login started; the second one is displayed by the shell you
started. To exit from this shell, use the exit command:
$ exit
$
The prompt that is displayed now is from the original shell started by login. Typing exit at this prompt
logs you out.
How to Start the Shell Noninteractively
You can start the shell noninteractively as follows:
$ /bin/sh filename
Here filename is the name of a file that contains commands to execute. As an example, consider the
compound command:
$ date ; who
Put these commands into a file called logins. First open a file called logins in an editor and type the
command shown previously. Assuming that the file is located in the current directory, after the file is saved,
the command can run as
$ /bin/sh logins
This executes the compound command and displays its output.
This is the first example of a shell script . Basically, a shell script is a list of commands stored in a
file that the shell executes noninteractively.
Initialization File Contents
Usually the shell initialization files are quite short. They are designed to provide a complete working
environment with as little overhead as possible for both interactive and noninteractive shells.
The file /etc/profile is maintained by the system administrator of your UNIX machine and contains shell
initialization information required by all users on a system.
The file .profile is under your control. You can add as much shell customization information as you want
to this file. The minimum set of information that you need to configure includes
l The type of terminal you are using
l A list of directories in which to locate commands
l A list of directories in which to locate manual pages for commands
Setting the Terminal Type
Usually the type of terminal you are using is automatically configured by either the login or getty
programs. Sometimes, the autoconfiguration process guesses your terminal incorrectly. This can occur
when you are using a dial-up or modem connection.
If your terminal is set incorrectly, the output of commands might look strange, or you might not be able to
interact with the shell properly. To make sure that this is not the case, most users set their terminal to the
lowest common denominator as follows:
TERM=vt100
When I introduce the case statement, "Flow Control," you will see a more advanced method
of setting the terminal type that enables access to advanced terminal features.
Setting the PATH
When you type the command
$ date
the shell has to locate the command date before it can be executed. The PATH specifies the locations in
which the shell should look for commands. Usually it is set as follows:
PATH=/bin:/usr/bin
Each of the individual entries separated by the colon character, :, are directories. Directories are discussed.
If you request the shell to execute a command and it cannot find it in any of the directories given in the PATH
variable, a message similar to the following appears:
$ hello
hello: not found
Setting the MANPATH
In UNIX, online help has been available since the beginning. In the section "Getting Help" I will discuss how
to access it using the man command.
In order for you to access all the available help, you have to tell the shell where to look for the online help
pages. This information is specified using the MANPATH. A common setting is
MANPATH=/usr/man:/usr/share/man
Like the path, each of the individual entries separated by the colon character, :, are directories.
When you use the man command to request online help as follows, the man command searches every
directory given in the MANPATH for an online help page corresponding to the topic you requested.
$ man who
In this case it looks for the online help page corresponding to the who command. If this page is found, it is
displayed as discussed in the next section.
Making a Shell Script Executable
One of the most important tasks in writing shell scripts is making the shell script executable and making sure
that the correct shell is invoked on the script.
In a previous example, you created the logins script that executes the following compound command:
date ; who ;
If you wanted to run the script by typing its name, you need to do two things:
l Make it executable.
l Make sure that the right shell is used when the script is run.
To make this script executable, do the following:
chmod a+x ./logins
Here you are using the chmod command. For a complete discussion of how to use this command.
to the beginning of the script:
#!/bin/sh
Your script then has two lines:
#/bin/sh
date ; who ;
The magic line causes a new shell (in this case, /bin/sh) to be called to execute the script. Without the
magic line, the current shell is always used to evaluate the script, regardless of which shell the script was
written for. For example, without a magic line, csh and tcsh users might not be able to get a Bourne shell (
sh) script to run correctly.
The Magic of #!/bin/sh
The #!/bin/sh must be the first line of a shell script in order for sh to be used to run the script. If this
appears on any other line, it is treated as a comment and ignored by all shells.
Comments
The magic first line #!/bin/sh introduces the topic of comments. A comment is a statement
that is embedded in a shell script but should not be executed by the shell.
In shell scripts, comments start with the # character. Everything between the # and end of the line are
considered part of the comment and are ignored by the shell.
Adding comments to a script is quite simple: Open the script using an editor and add lines that start with the
# character. For example, to add the following line to the logins shell script:
# print out the date and who's logged on
I opened the file logins with my editor and inserted this line as the second line in the file. The shell script is
now as follows:
#!/bin/sh
# print out the date and who's logged on
date ; who ;
There is no change in the output of the script because comments are ignored. Also comments do not slow
down a script because the shell can easily skip them.
You can also add comments to lines that contain commands by adding the # character after the commands.
For example, you can add a comment to the line date ; who ; as follows:
date ; who ; # execute the date and who commands
When you are writing a shell script, make sure to use comments to explain what you are doing in case
someone else has to look at your shell script. You might find that this helps you figure out what your own
scripts are doing, months after you write them.

Linux Shell Basics

My father has a tool chest that holds all his woodworking tools, from screwdrivers and chisels to power
sanders and power drills. He has used these tools to build several desks, a shed, a bridge, and many toys.
By applying the same tools, he has been able to build all the different elements required for his projects.
Shell scripting is similar to a woodworking project. To build something out of wood, you need to
use the right tools. In UNIX, the tools you use are called utilities or commands. There are simple commands
like ls and cd, and there are power tools like awk, sed, and the shell.
One of the biggest problems in woodworking is using the wrong tool or technique while building a project.
Knowing which tool to use comes from experience. In this book, you will learn how to use the UNIX tools via
examples and exercises.
The simple tools are easy to learn. You probably already know how to use many of them. The power tools
take longer to learn, but when you get the hang of them, you'll be able to tackle any problem. This book
teaches you how to use both the simple tools and the power tools. The main focus is on the most powerful
tool in UNIX, the shell.
Before you can build things using the shell, you need to learn some basics. This chapter looks at the
following topics:
l Commands
l The shell
It's time to get started.
What Is a Command?
Simple Commands Compound Commands
Complex Commands Command Separators
In UNIX, a command is a program that you can run. In other operating systems, such as Mac OS
or Windows, you point to the program you want to run and click it. To run a command in UNIX, you type its
name and press Enter.
For example:

$ date [ENTER]
Wed Dec 9 08:49:13 PST 1998
$
Here, the date command has been entered. This command displays the current day, date, time, and year.
After the current date appears, notice that the $ character is displayed.
In this book, I use the $ character to indicate the prompt. Wherever you see a prompt, you can type the
name of a command and press Enter. This executes the command that you type. While a command
executes, the prompt is not displayed. When the command finishes executing, the prompt is displayed
again.
Caution - The $ character is a prompt for you to enter a command. It is not part of the command itself.
For example, to execute the date command, you type the word date at the prompt, $. Don't type $ date
. Depending on your version of UNIX, an error message might be displayed if you type $ date instead of
date at the prompt.
Now look at another example of running a command:
$ who
vathsa tty1 Dec 6 19:36
sveerara ttyp2 Dec 6 19:38
ranga ttyp0 Dec 9 09:23
$
Here, I entered the command who at the prompt. This command displays a list of all the people, or users,
who are currently using the UNIX machine.
The first column of the output lists the usernames of the people who are logged in. On my system, you can
see that there are three users, vathsa, sveerara, and ranga. The second column lists the terminals they
are logged in to, and the final column lists the time they logged in.
The output varies from system to system. Try it on your system to see who is logged in.
For those readers who are not familiar with the process of logging in to a UNIX system, the details are
discussed in Chapter 2, "Script Basics."
Simple Commands
The who and date commands are examples of simple commands. A simple command is one
that you can execute by just giving its name at the prompt:
$ command
Here, command is the name of the command you want to execute. Simple commands in UNIX can be small
commands like who and date, or they can be large commands like a Web browser or a spreadsheet
program.You can execute most commands in UNIX as simple commands.

Complex Commands
You can use the who command to gather information about yourself when you execute it as follows:
$ who am i
ranga pts/0 Dec 9 08:49
$
This tells me the following information:
l My username is ranga.
l I am logged in to the terminal pts/0.
l I logged in at 8:49 on Dec 9.
This command also introduces the concept of a complex command, which is a command that
consists of a command name and a list of arguments.
Arguments are command modifiers that change the behavior of a command. In this case, the
command name is who, and the arguments are am and i.
When the who command runs as a simple command, it displays information about everyone who
is logged in to a UNIX system. The output that is generated when a command runs as a simple command is
called the default behavior of that command.
The arguments am and i change the behavior of the who command to list information about you only. In
UNIX, most commands accept arguments that modify their behavior.
The formal syntax for a complex command is:
$ command argument1 argument2 argument3 ... argumentN
Here, command is the name of the command you want to execute, and argument1 through argumentN
are the arguments you want to give command.
Compound Commands
One of the most powerful features of UNIX is the capability to combine simple and complex commands
together to obtain compound commands.
A compound command consists of a list of simple and complex commands separated by the
semicolon character ( ;). An example of a complex command is
$ date ; who am i ;
Wed Dec 9 10:10:10 PST 1998

ranga pts/0 Dec 9 08:49
$
Here, the compound command consists of the simple command date and the complex command who am
i. As you can see from the output, the date command executes first, followed by the who am i
command. When you give a compound command, each of the individual commands that compose it
execute in order.
In this example, the complex command behaves as if you typed the commands in the following order:
$ date
Wed Dec 9 10:25:34 PST 1998
$ who am i
ranga pts/0 Dec 9 08:49
$
The main difference between executing commands in this fashion and using a complex command is that in
a complex command you do not get the prompt back between the two commands.
The formal syntax for a complex command is:
$ command1 ; command2 ; command3 ; ... ; commandN ;
Here, command1 through commandN are either simple or complex commands. The order of execution is
command1, followed by command2, followed by command3, and so on. When commandN finishes
executing, the prompt returns.
Command Separators
The semicolon character ( ;) is treated as a command separator, which indicates where one
command ends and another begins.
If you don't use it to separate each of the individual commands in a complex command, the computer will
not be able to tell where one command ends and the next command starts. If you execute the previous
example without the first semicolon
$ date who am i
an error message similar to the following will be produced:
date: bad conversion
Here, the date command thinks that it is being run as a complex command with the arguments who, am,
and i. The date command is confused by these arguments and displays an error message. When using
complex commands, remember to use the semicolon character.
You can also terminate individual simple and complex commands using the semicolon character. For
example, the commands

$ date
and$ date ;
produce the same output due to the order in which commands execute.
In the first case, the simple command date executes, and the prompt returns.
In the second case, the computer thinks that a complex command is executing. It begins by executing the
first command in the complex command. In this case, it is the date command. When this command finishes,
the computer tries to execute the next command. Because no other commands are left to execute, the
prompt returns.

The Linux Environment

Files and File Systems
Each Linux disk (or other long-term block storage device) contains a collection of files
organized according to a policy or set of rules called a file system. Each disk can be divided
into partitions (or “slices”), whereby every partition has its own file system. Linux is
not restricted to a single file system for all disks: the user can use disks created by other
operating systems as if they were native Linux disks.
The standard file system is the second extended file system, or ext2.This is the second
revision of the Minix file system with support for large disk sizes and filenames. ext2
permits partitions up to 4TB (terabytes), files up to 2GB (gigabytes), and 255-character
filenames. Newer distributions use ext3, a version of ext2 with special features for error
recovery.
Support for other file systems might be available depending on your distribution and
installation options.They might include Microsoft Windows NT, Apple HFS, or journaling
file systems.
The ext2 file system uses caching to increase performance. If an ext2 disk is not
properly shut down, files can be corrupted or lost. It is vitally important that a Linux
computer is shut down properly or is protected by some kind of uninterruptible power
supply.
To save space, ext2 files that contain large amounts of zeros (called sparse files) are not
actually stored on a disk. Certain shell commands provide options for creating and handling
sparse files.
Each file is identified by a name, and the allowed names are determined by the file
system. For practicality, the names seldom exceed 32 characters and usually consist of
lowercase characters, underscores, minus signs, and periods. Spaces and punctuation symbols,
for example, are permitted, but can cause problems in shell scripts that do not
expect them.
Filenames do not require a suffix to identify their contents, but they are often used to
avoid confusion about what data is contained in files. Some common suffix codes
include:
Directories 9
n .sh—A Bash shell script
n .txt—A generic text file
n .log—A log file
n .html—A HTML Web page
n .tgz (or .tar.gz)—Compressed file archive
Commands usually have no suffix.
Directories
Shell scripts, text files, and executable commands and other normal files are collectively
referred to as regular files.They contain data that can be read or instructions that can be
executed.There are also files that are not regular, such as directories or named pipes; they
contain unique data or have special behaviors when they are accessed.
Files are organized into directories, or listings of files. Like all other files in Linux, a
directory is also treated as a file. Each directory can, in turn, contain subdirectories, creating
hierarchical listings.
Directories are organized into a single monolithic tree.The top-most directory is
called the root directory. Unlike some other operating systems that have separately labeled
disks, Linux treats any disk drives as subdirectories within the main directory structure.
From a user’s point of view, it’s impossible to tell which disk a particular directory
belongs to: Everything appears as if it belongs to a single disk.
A pathname is a string that identifies the location of a particular file, the sequences of
directories to move through to find it.The root directory is denoted by a forward slash
(/) character, and so /payroll.dat represents a file named payroll.dat, located in the
top-most directory. Using more directory names and forward slashes can specify additional
directories.
When users log in, they are placed in a personal directory called their home directory.
By convention, Linux home directories are located under the directory /home.The pathname
/home/jgulbis/payroll.dat indicates a file named payroll.dat in the home
directory of the user jgulbis.The home directory is represented by a tilde (~) in Bash.
The current directory (or working directory) is denoted by a period (.).When a pathname
doesn’t start with a leading slash, Bash assumes it’s a path relative to the current directory.
./payroll.dat and payroll.dat both refer to a file named payroll.txt in the current
directory.When running programs, this might not be true.This exception is discussed in
the next chapter.
The parent directory is represented by a double period (..).The double period can be
used at any place in a path to move towards the root of the directory tree, effectively
canceling the previously mentioned directory in a path. However, it makes the most
sense to use the double period as the first directory in a path. If your current directory is
/home/jgulbis, the pathname ../kburtch/payroll.dat is the same as the pathname
/home/kburtch/payroll.dat.The double period represents the parent directory of
/home/jgulbis, the /home directory.

Pathnames without a beginning slash are called relative paths because they specify the
location of a file in comparison to the current directory. Relative paths are useful for
representing files in your current directory or subdirectories of your current directory.
Pathnames with a beginning slash are called absolute paths. Absolute paths describe the
location of a file in relationship to the root directory. No matter where your current
directory is, absolute paths always identify the file precisely. Absolute paths are useful
when locating common files that are always stored in the same place.
There are no set rules governing where files are located, and their placement is chosen
by your Linux distribution. Early variations of Unix stored standard programs in
/bin, home directories in /usr, and programs specific to a computer in /usr/bin.As
the number and type of programs grew, the number and function of the common directories
changed.
Most Linux distributions include the following directories:
n /dev—Contains device drivers
n /bin and /usr/bin—Contains standard Linux commands
n /lib and /usr/lib—Contains standard Linux libraries
n /var—Contains configuration and log files
n /etc—Contains default configuration files
n /usr/local/bin—Contains commands not a part of the distribution, added by
your administrator
n /opt—Contains commercial software
n /tmp—Stores temporary files
n /sbin and /usr/sbin—Contains system administration commands (/sbin stands
for “safe” bin)
Inodes and Links
Normally, each file is listed in a single directory. Linux can create additional listings for a
single file.These shortcuts are called links and can refer to any kind of file.
Links come in two varieties. A hard link is a reference to another file in the current
directory or a different directory.Whenever some action is performed to the hard link, it
is actually done to the file the hard link refers to. Hard links are accessed quickly because
they do not have to be dereferenced, but Linux limits where a hard link can be placed. As
long as a file is being referred to by at least one directory entry, it won’t be deleted. For
example, if a file has one hard link, both the link and the original file have to be deleted
to remove the file.
The second and more common link is the symbolic link.This link is a file that contains
the pathname of another file. Unlike hard links, symbolic links have no restrictions on
where they can be used.They are slower and some commands affect the link file itself
instead of the file the link points to. Symbolic links are not “hard” because they have to
Device Files 11
be dereferenced:When Linux opens the symbolic link file, it reads the correct pathname
and opens that file instead.When the file being referred to is deleted, the symbolic link
file becomes a dangling link to a non-existent file.
Using links means that two different pathnames can indicate the same file.To identify
a particular file, Linux assigns a number to each file.This number is called the inode (or
“index node”) and is unique to any storage device. If two pathnames refer to a file with
the same inode, one of the paths is a hard link.
In the ext2 file system, there is a limited number of inodes, which in turn places an
upper limit to the number of files that can be stored on a disk.The number of inodes
compared to the amount of disk space is called the inode density. The density is specified
when a disk or partition is initialized. Most Linux distributions use an inode density of
4K, or one node per every 4096 bytes of disk space.
Pipe and Socket Files
Pipe files are a special kind of file shared between two programs.The file acts as a buffer
for sharing information. One program writes to the pipe file and the other reads from
the pipe.When the pipe file reaches a certain size, Linux halts the writing program until
the reading program can “catch up.”
A similar kind of file is called a Unix domain socket file.A socket file acts like a pipe
but works using network sockets. However, this kind of file is not easily used in shell
scripts and it won’t be covered in this book.
Device Files
The last common kind of nonregular file is a device file. Keeping with the file-oriented
design of Linux, devices are represented by files. Device files allow direct communication
to a particular device attached to a computer.There are actually two kinds of device
files, but shell programmers are mainly interested in the type called character device files.
All devices files are located in the /dev directory. Even though many files are listed in
/dev, not all of these devices might actually be present. Rather, /dev contains a list of
devices that can be attached to your computer because the Linux kernel was configured
to recognize them if they were attached.
Most of these files are not accessible to regular users, but there are a few that are open
to general use. One important device file available to all users is /dev/null.This file represents
an imaginary “black hole” device attached to your computer that consumes anything
sent to it.This is useful for discarding unwanted responses from a shell command.
/dev/null can also be read, but the file is always empty.
Another device file is /dev/zero.This file contains an endless stream of zeros, and
can be used to create new files that are filled entirely with zeros.
There are a variety of other devices that might appear in /dev, depending on your
distribution and computer hardware. Common device files include:

n /dev/tty—The terminal window (or console) your program is running under
n /dev/dsp—The interface that plays AU sound files on your sound card
n /dev/fd0—The first floppy drive
n /dev/hda1—The first IDE drive partition
n /dev/sda1—The first SCSI drive partition
The name tty, for historical reasons, is a short form of “teletypewriter,” a printer and
keyboard connected to a computer by a cable.
With this overview of the Linux philosophy, you are ready to begin using Linux
through the Bash shell.

Using JOGL

JOGL is very easy to set up and use. JOGL does not currently have an install system because it is simply two files: the jogl.jar containing all the JOGL Java binary, and an additional native code file that is machine specific and links to that architecture’s OpenGL implementation. On Windows, that file is simply jogl.dll. These two files can be put in the JVM directories for the system to use anywhere the Java classpath would find them, or the classpath could be appended to a new location for them, whichever the developer chooses.
In beginning a new class to make use of JOGL, we add the main JOGL import: import net.java.games.jogl.*;
Any application that intends to display a JOGL render will need to make a AWT or Swing frame. In addition, the example will catch a key press for stepping through the different render states, so it will also need AWT events imported. Therefore, we also import:import java.awt.*;
import java.awt.event.*;
Last, we are going to use vertex arrays and vertex buffer objects, and for that we will need two more imports, Java’s NIO and a JOGL utility class that makes the special offset ByteBuffer used by vertex buffer objects.import java.nio.*;
import net.java.games.jogl.util.*;
To keep this as simple as possible, we are going to make one class called Sample, which will implement the GLEventListener interface, expecting it to respond to the display() events generated by the main GLCanvas. The interface GLEventListener declares events that client code can use to manage OpenGL rendering onto a GLDrawable. When any of these methods is called, the drawable has made its associated OpenGL context current, so it is valid to make OpenGL calls within them.
Let’s take a look at main():public static void main(String[] args) {
Frame frame = new Frame("Spinning Cube Four JOGL Ways!");
GLCanvas canvas =
GLDrawableFactory.getFactory().createGLCanvas(new
GLCapabilities());
canvas.addGLEventListener(new Sample());
System.err.println("CANVAS GL IS: " +
canvas.getGL().getClass().getName());
System.err.println("CANVAS GLU IS: " +
canvas.getGLU().getClass().getName());

frame.add(canvas);
frame.setSize(500,500); // hard coded width/height
animator = new Animator(canvas);
frame.addWindowListener(new WindowAdapter() {
public void windowClosing(WindowEvent e) {
animator.stop();
System.exit(0);
}
});
frame.show();
animator.start();
canvas.requestFocus();
}
The GLDrawableFactory.getFactory().createGLCanvas() is the simplest of the series of create() methods for GLCanvases or GLJPanels. We use the “new GLCapabilities()” for the display settings, which simply means to use the defaults.
Next, we add this Sample class to the canvas’s GLEventListener list so Sample can receive the callbacks:
void display(GLDrawable drawable)
Called by the drawable to initiate OpenGL rendering by
the client.
void displayChanged(GLDrawable drawable, boolean modeChanged,
void boolean deviceChanged)
Called by the drawable when the display mode or the
display device associated with the GLDrawable has
changed.
void init(GLDrawable drawable)
Called by the drawable immediately after the OpenGL
context is initialized for the first time.
void reshape(GLDrawable drawable, int x, int y, int width, int
height)
Called by the drawable during the first repaint after the
component has been resized.
The rest of the code is the typical frame initialization, window-setting size, adding an anonymous WindowListener for quitting, and finally showing the frame.
One other interesting piece is the Animator class. Animator is a JOGL utility class that sets up a looping thread that continually triggers a repaint on the canvas. An Animator can be attached to a GLDrawable to drive its display() method in a loop. For efficiency, it sets up the rendering thread for the drawable to be its own internal thread, so it cannot be combined with manual repaints of the surface.
It is a simple class with the full interface being the only the constructor—the start() and stop() methods:Animator(GLDrawable drawable)
Creates a new Animator for a particular drawable.
void start()
Starts this animator.
void stop()
Stops this animator, blocking until the animation
thread has finished.
More full-featured applications may wish to create their own Animator-type class with more functionality, but this one will do fine for this example.
When the Animator start method is called, it creates an internal Thread instance and sets the GLDrawables rendering thread reference to it. It then goes into a loop that will repeatedly call display() on its GLDrawable, which is, in this case, our Sample class.
When GLDrawable.setRenderingThread() is called, JOGL will carry out a series of initializations triggering the canvas to call GLEventListener.init() on our Sample class. Here we will do a bit of setup of our own and see some typical OpenGL calls pass in the Sample object as the KeyListener to the GLDrawable. In addition, doing so will build up our geometry to be used later in display() and check for availability of vertex buffer objects in this OpenGL implementation. This gives an example of how to use gl.isExtensionAvailable() to check which ARB and vendor-specific extensions are supported on the current system. Extension functions can also be checked using the method isFunctionAvailable(java.lang.String).
This example tries to use vertex buffer objects, but if they are not supported on the current system, it will skip both building the vertex buffer objects and rendering them in display():
/** Called by the drawable immediately after the OpenGL context
* is initialized for the first time. Can be used to perform
* one-time OpenGL initialization such as setup of lights and display lists.
* */
public void init(GLDrawable gLDrawable) {
final GL gl = gLDrawable.getGL();
gl.glShadeModel(GL.GL_SMOOTH); // Enable Smooth Shading
gl.glClearColor(0.0f, 0.0f, 0.0f, 0.5f); // Black Background
gl.glClearDepth(1.0f); // Depth Buffer Setup
gl.glEnable(GL.GL_DEPTH_TEST); // Enables Depth Testing
gl.glDepthFunc(GL.GL_LEQUAL); // The Type Of Depth Testing
gl.glHint(GL.GL_PERSPECTIVE_CORRECTION_HINT, GL.GL_NICEST);
// Really Nice Perspective Calculations
gLDrawable.addKeyListener(this);
// Use Debug pipeline
// comment out to use standard pipeline
gLDrawable.setGL(new DebugGL(gLDrawable.getGL()));

supportedVBO =
gl.isExtensionAvailable("GL_ARB_vertex_buffer_object");
if ( supportedVBO )
System.out.println("Vertex Buffer Objects Supported");
else
{
System.out.println("Vertex Buffer Objects NOT Supported - part of this example will not execute");
}
buildCubeDisplayList(gl);
buildCubeVertexArrays(gl);
}
The last two methods are called to build a display list and vertex arrays and also bind OpenGL buffers for vertex buffer objects, if they are supported.
This simple display method is where the rendering actually gets done every frame. In this example, we do the typical operations of clearing the buffers, translating and rotating the objects for viewing, and finally drawing the cube. This example has the four main render methods set up—batched vertex calls, a display list of those calls, an indexed vertex array of the same shape, and a vertex buffer object of those same vertex arrays. All are set up in other later methods for cleanness, except for the display list render, which is simply one glCallList() command anyway.
public void display(GLDrawable gLDrawable) {
GL gl = gLDrawable.getGL();
gl.glClear(GL.GL_COLOR_BUFFER_BIT GL.GL_DEPTH_BUFFER_BIT);
gl.glLoadIdentity();
gl.glTranslatef(0.0f, 0.0f, -6.0f);
gl.glRotatef(rotX, 1.0f, 0.0f, 0.0f);
gl.glRotatef(rotY, 0.0f, 1.0f, 0.0f);
rotX += 1.0f * scale;
rotY += 0.5f * scale;
//for(int i = 0;i<10; i++) // uncomment to slow down
{
switch( renderType )
{
case rtVertex:
drawCubeVertex(gl);
break;
case rtDisplayList:
gl.glCallList(gear1);
break;
case rtVertexArray:
drawCubeVertexArrays(gl);
break;
case rtVertexBufferObject:
if ( supportedVBO )
drawCubeVertexBufferObjects(gl);
break;
}
}
}
The rest of this Sample is the actual render methods, the object build methods, and the KeyListener methods, of which keyPressed() catches the spacebar for toggling through the different render modes and Escape for exiting.
And there you have it—Java bindings for OpenGL rendering an amazing spinning colored cube.

JOGL and OpenGL Differences

There are key differences between using JOGL and OpenGL, due partly from the fact that JOGL maps a function-based C API to object-oriented Java, and partly from the fact that Java is secure, (here we mean in terms of illegal memory access) and OpenGL is not. These differences instantly make the JOGL interface slightly different everywhere, and very different in a few critical places. In addition, not every single API difference is documented here (depending on how you look at it, there are hundreds!), and inevitably there will be new differences in future JOGL versions. This book is meant to help you understand the major differences and learn to use them in Java.

OpenGL’s Static Constants and Functions
The C version of OpenGL makes heavy use of static constants throughout the OpenGL API, particularly for function parameters. It is an accepted way to create some level of type-safety for the C language interface it uses. However, in Java there is no such thing as true, stand-alone (classless) static variables or constants. All Java constructs are either a field or method of a class. Java objects may have static class variables but not any global classless static variables such as those used in C and OpenGL.
OpenGL also uses static functions. This is the nature of pure C code. Similar to the static constants, Java has no true classless methods that can be mapped directly to the OpenGL C functions.

This is a typical problem when mapping a functional C API to a object-oriented language such as Java. Fortunately, the solution is simple and effective in most cases. A single Java class is created that has all the C APIs static variables and functions, which are mapped to identically or similarly named static (or instance) variables and methods. The developer then either accesses the variables/functions through the classes’ static variables/functions directly, or, depending on the design, a runtime instance is created and used as the accessing object.
JOGL uses this second method of a runtime class instantiation for various reasons. That is, a special GL object is created, and most JOGL operations are done using that single instance. To an experienced OpenGL programmer this process may appear strange at first, and to anyone using existing OpenGL resources, any sample C-based OpenGL code will not port unmodified but must be converted to this class instance-accessing method.
Fortunately, for a Java programmer the conversion is quite simple and straightforward. An example code segment follows, first in C, then in Java with JOGL:// Original C OpenGL source
glBegin( GL_QUADS );
glColor3f( 1.0, 0.0, 0.0 );
glVertex3f( 10.0, 0.0, 0.0 );
glColor3f( 0.0, 1.0, 0.0 );
glVertex3f( 0.0, 10.0, 0.0 );
glColor3f( 0.0, 0.0,1.0 );
glVertex3f( -10.0, 0.0, 0.0 );
glColor3f( 1.0, 1.0, 1.0 );
glVertex3f( 0.0, -10.0, 0.0 );
glEnd();
// Ported to JOGL
// using local gl instance reference
// and GL class reference
// and explicit f for floats
gl.glBegin( GL.GL_QUADS );
gl.glColor3f( 1.0f, 0.0f, 0.0f );
gl.glVertex3f( 10.0f, 0.0f, 0.0f );
gl.glColor3f( 0.0f, 1.0f, 0.0f );
gl.glVertex3f( 0.0f, 10.0f, 0.0f );
gl.glColor3f( 0.0f, 0.0f,1.0f );
gl.glVertex3f( -10.0f, 0.0f, 0.0f );
gl.glColor3f( 1.0f, 1.0f, 1.0f );
gl.glVertex3f( 0.0f, -10.0f, 0.0f );
gl.glEnd();

OpenGL’s Use of C Pointers to Arrays
Several functions in OpenGL make use of pointers to arrays in C. This is done as a mechanism to return a series of OpenGL names, or handles, for what it calls server-side data objects. For example, multiple texture names can be generated at once in the C API by calling:void glGenTextures(GLsizei n,GLuint *textures)

where n specifies the number of texture names to be generated and *textures specifies a pointer to an array in which the generated texture names are stored. The GLuint *textures is a C pointer to GLuint that should be at the beginning of an array of GLuint type that is the length of n. Because arrays are a formal type in Java, JOGL simply uses Java array objects

instead.public void glGenTextures(int n,int[] textures)

After calling this method, the Java int[] array argument “textures” will contain the texture names OpenGL has generated (just like it would for Gluint *texture in C) and can be accessed in the usual Java way.

This method shows another difference as well. In C, OpenGL has all sorts of additional primitive data types usage beyond standard C types to help type-safe OpenGL.
JOGL has mapped the OpenGL types to native Java types in the JNI layer where applicable, so standard Java types are supported directly.
The C literal suffix is used in the function names in OpenGL, and the same suffixes are used in JOGL for compatibility and listed here for reference. The way the functions are named generally .

Creating JOGL Textures
To reveal a more profound JOGL difference, we will further examine setting up textures in JOGL.
Java has existing standard images classes, including image loaders, and it would be nice to be able to use those to create the textures for OpenGL. It’s not automatic, but it’s not terribly difficult, either. It also exposes another significant difference in JOGL—its use of ByteBuffer objects.

ByteBuffers
JOGL needs fast and efficient ways to process and reference chucks of “raw” memory that would contain texture data or geometry data to which the OpenGL layer can have direct access, just as it does in C. The Java solution is to use NIO’s ByteBuffers wherever JOGL needs that kind of direct access. So, many functions that expect C pointers in OpenGL alternatively access ByteBuffers in the Java bindings and under-the-covers on the JNI-side OpenGL, which can have direct access to this data without any wasteful copying.
Let’s look at how this affects the texture functions by following the process of creating a texture from a standard Java BufferImage.

Getting a texture into an OpenGL environment is at least a three-step process on any system. Because of all the existing standard Java image APIs, it’s probably one of the easiest environments.

Step 1—Load a image from the file system: Image loading is a well-supported functionality in Java, so this is probably the most familiar step for the typical Java developer. Several supported mechanisms are available for loading images, and this can always be done though direct file access as well, if needed.

Step 2—Format the image data appropriately: The JOGL texture methods accept texture data as ByteBuffers where the buffer data must be encoded to a format that the OpenGL standard accepts. Unfortunately, this is almost never going to be the same format as the data format returned from a loaded image, unless the image loader was specifically designed for JOGL, which none of the existing standard image loaders are. Therefore, some data conversion will be required to get from the loaded image data to the OpenGL required data format.

Step 3—Create OpenGL texture binding for the new texture and set the texture data and parameters: After the ByteBuffer is packed with the image data correctly formatted, we can get a texture bind ID from OpenGL and pass the texture ByteBuffer along with any OpenGL texture parameters that we want to set with it.

Of the methods to load images in Java that the example uses, is ImageIO.read(), which returns BufferedImages. BufferedImages are the preferred type because they support a pathway to get the image data to a byte[] array with which we can load up a ByteBuffer for OpenGL. One problem that we wouldn’t catch until actually viewing in OpenGL is that the image would be upside down in most cases. This happens in many systems; it is not a Java-specific issue.
What happens is that the convention for assigning texture coordinates is with positive U (X in the image) to the right and positive V (Y in the image) up. But most image formats consider the positive y-direction as down, as it is in screen space. It is simple to flip the image with the AffineTransformOp class right after loading but before the OpenGL formatting, if needed. Given the correct string filename for an image file, here’s how it can be done:

static public BufferedImage loadImage(String resourceName)
throws IOException
{
BufferedImage bufferedImage = null;
try
{
bufferedImage = ImageIO.read(new File(resourceName));
}
catch (Exception e)
{
// file not found or bad format maybe
// do something good
}
if (bufferedImage != null)
{
// Flip Image
//
AffineTransform tx = AffineTransform.getScaleInstance(1, -
1);
tx.translate(0, -bufferedImage.getHeight(null));
AffineTransformOp op = new AffineTransformOp(tx,
AffineTransformOp.TYPE_NEAREST_NEIGHBOR);
bufferedImage = op.filter(bufferedImage, null);
}
return bufferedImage;
}
At this point we have a BufferedImage in memory. Now it must be formatted for OpenGL.
This process could be a bit messy in practice, because BufferedImages can themselves be many different data formats. The trick is to use the Java image APIs and let them handle the conversion for you. Using ComponentColorModel and the Raster class, we can set up a utility routine that can convert any BufferedImage to an OpenGL-acceptable data format. First, we will make two reusable ComponentColorModels, one for regular RGB color images and another for RGBA (color-transparent) images. Then we will use the ComponentColorModels to build a Raster from which we can get bytes.
One last detail is that textures need to be sized to powers of 2 due to most graphics hardware requirements. It is simple to check the image’s width and height, and upsize the image height and/or width to the next power of 2. This action is typically considered wasteful in terms of graphics memory because the image will be larger but not anymore detailed. However, it will allow loading of non-power-of-2-size images without failing out, which is especially useful when testing.
ByteBuffer AllocateDirect and Native Order
There are two additional important points about using ByteBuffers in OpenGL. The ByteBuffers need to be created with ByteBuffer.allocateDirect() so that the ByteBuffers are C accessible without copying, and the byte order must be set to native order with byteBuffer.order(ByteOrder.nativeOrder()) to work correctly. Failing to do this is a common error when first starting out with JOGL.
Putting this all together, we have a versatile image conversion routine.
glAlphaColorModel = new ComponentColorModel(ColorSpace.getInstance(ColorSpace.CS_sRGB),
new int[]{8, 8, 8, 8}, true, false, ComponentColorModel.TRANSLUCENT, DataBuffer.TYPE_BYTE);
glColorModel = new ComponentColorModel(ColorSpace.getInstance(ColorSpace.CS_sRGB),
new int[]{8, 8, 8, 0}, false, false, ComponentColorModel.OPAQUE, DataBuffer.TYPE_BYTE);
public static ByteBuffer convertImageData(BufferedImage bufferedImage) throws TextureFormatException
{
ByteBuffer imageBuffer = null;
try
{
WritableRaster raster;
BufferedImage texImage;
int texWidth = 2;
int texHeight = 2;
while (texWidth #### bufferedImage.getWidth())
{
texWidth *= 2;
}
while (texHeight #### bufferedImage.getHeight())
{
texHeight *= 2;
}
if (bufferedImage.getColorModel().hasAlpha())
{
raster = Raster.createInterleavedRaster(DataBuffer.TYPE_BYTE, texWidth,
texHeight, 4, null);
texImage = new BufferedImage(glAlphaColorModel, raster, false, new Hashtable());
}
else
{
raster = Raster.createInterleavedRaster(DataBuffer.TYPE_BYTE, texWidth,
texHeight, 3, null);
texImage = new BufferedImage(glColorModel, ####dis:raster, false, new Hashtable());
}
texImage.getGraphics().drawImage(bufferedImage, 0, 0, null);
byte[] data = ((DataBufferByte) texImage.getRaster().getDataBuffer()).getData();
imageBuffer = ByteBuffer.allocateDirect(data.length);
imageBuffer.order(ByteOrder.nativeOrder());
imageBuffer.put(data, 0, data.length);
}
catch (Exception e)
{
throw new TextureFormatException("Unable to convert data
for texture " + e);
}
return imageBuffer;
}

At last we have the ByteBuffer correctly formatted for OpenGL. All that is left is to get a bind ID and hand over the data to OpenGL. Everything we’ve done until now could have been done offline—that is, before OpenGL is configured, initialized, and rendered—and is recommended when possible. Loading and converting images in the middle of an OpenGL render cycle will inevitably decrease runtime performance.

To get a bind ID as well as to pass over the texture data to OpenGL, we will need a valid GL reference. Also, it must be done within the thread that is assigned to the OpenGL context. Often that means the texture binding will be done in the init() method of GLEventListener, but if it happens later on during execution, it will most likely be inside the display() method. In any case we must have a live and current GL object. The JOGL bind ID calls are straight, standard OpenGL texture commands with the exception of the int[] handle for the bind and ByteBuffer syntax. Putting it all together would look something like the following code:
static public int createTexture(String name,
String resourceName,
int target,
int dstPixelFormat,
int minFilter,
int magFilter,
boolean wrap,
boolean mipmapped) throws
IOException, TextureFormatException
{
// create the texture ID for this texture
//
int[] tmp = new int[1];
gl.glGenTextures(1, tmp);
int textureID = tmp[0];
// bind this texture
//
gl.glBindTexture(GL.GL_TEXTURE_2D, textureID);
// load the buffered image for this resource - save a copy so
we can draw into it later
//
BufferedImage bufferedImage = loadImage(resourceName);
int srcPixelFormat;
if (bufferedImage.getColorModel().hasAlpha())
{
srcPixelFormat = GL.GL_RGBA;
}
else
{
srcPixelFormat = GL.GL_RGB;
}
// convert that image into a byte buffer of texture data
//
ByteBuffer textureBuffer = convertImageData(bufferedImage);
// set up the texture wrapping mode depending on whether
// this texture is specified for wrapping or not
//
int wrapMode = wrap ? GL.GL_REPEAT : GL.GL_CLAMP;
if (target == GL.GL_TEXTURE_2D)
{
gl.glTexParameteri(target, GL.GL_TEXTURE_WRAP_S,
wrapMode);
gl.glTexParameteri(target, GL.GL_TEXTURE_WRAP_T,
wrapMode);
gl.glTexParameteri(target, GL.GL_TEXTURE_MIN_FILTER,
minFilter);
gl.glTexParameteri(target, GL.GL_TEXTURE_MAG_FILTER,
magFilter);
}
// create either a series of mipmaps or a single texture image
// based on what’s loaded
//
if (mipmapped)
{
glu.gluBuild2DMipmaps(target,
dstPixelFormat,
bufferedImage.getWidth(),
bufferedImage.getHeight(),
srcPixelFormat,
GL.GL_UNSIGNED_BYTE,
textureBuffer);
}
else
{
gl.glTexImage2D(target,
0,
dstPixelFormat,
bufferedImage.getWidth(),
bufferedImage.getHeight(),
0,
srcPixelFormat,
GL.GL_UNSIGNED_BYTE,
textureBuffer);
}
return textureID;
}
After this, the OpenGL textures and IDs are properly set up and ready to use.
Vertex Arrays and ByteBuffers
ByteBuffers are also the mechanism used to set up vertex arrays in JOGL. Whereas OpenGL takes C float-array pointers for vertex arrays using the following function:void glVertexPointer(GLint size,
GLenum type,
GLsizei stride,
const GLvoid *pointer)
Similar to texture ByteBuffers, the JOGL method is:public void glVertexPointer(int size,
int type,
int stride,
Buffer ptr)
The easiest way to use vertex arrays is to make a regular Java float[] array containing the appropriate vertex, color, normal, or texture coordinate data and pass the array into a ByteBuffer in the array put() call. A simple triangle vertex array for a unit square follows:
float[] verts = new float[]
{
-1.0f, 1.0f, 0.0f,
-1.0f, -1.0f, 0.0f,
1.0f, -1.0f, 0.0f,
-1.0f, 1.0f, 0.0f,
1.0f, -1.0f, 0.0f,
1.0f, 1.0f, 0.0f;
};
FloatBuffer vertexArray = ByteBuffer.allocateDirect(verts.length*4).order
(ByteOrder.nativeOrder()).asFloatBuffer();
vertexArray.put( verts );
Getting it to render is nearly identical to standard OpenGL:gl.glVertexPointer(3, GL.GL_FLOAT, 0, vertexArray);
gl.glEnableClientState(GL.GL_VERTEX_ARRAY);
gl.glDrawArrays(GL.GL_TRIANGLES, 0, vertexArray.capacity());
Multithreading
Multithreading is another issue that creates some difficulties for developers first working with JOGL. Usually this step occurs in basic GUI test applications that aren’t designed around the single-threaded nature of OpenGL. Most errors happen when the user sets up some GUI components such as a button, wants to catch that button’s press-action event, and then wants the event action to do some direct OpenGL state setting.
For example, the developer might wish to have a GUI button enable or disable texturing in the OpenGL renderer. The developer makes the GL instance object available to the anonymous event Listener object, either by a static reference or through an accessor method, and then proceeds to call that GL object’s glEnable/Disable() on whatever state they want to affect. Unfortunately, this is not going to work because the GUI event thread now performing the GL call is not the assigned rendering thread. This is a violation of the threaded access to OpenGL and at best will result in no change or possibly a runtime exception, and at worst an application or system crash.

Two popular solutions are available for this problem. One is to have the desired modifiable states in OpenGL declared as Java class variables that are used to modify the OpenGL render states using the assigned render thread when display() is called.
The second is to set up a messaging system, where messages or requests for OpenGL changes are made and queued up until the next display() is called when the rendering thread reads though the messages performing the requested OpenGL operations. This is a typical design pattern in Swing and other GUI apps.

Either design is acceptable, but no matter which way multithreading is managed, it is likely that this issue will need to be addressed in all but the simplest of JOGL applications.

JOGL-Specific Features

JOGL is the shortened name for the Java bindings to OpenGL developed and maintained by Sun Microsystems’ Java Game Technology Group and distributed on the Web at their main Java game development site. Whereas JOGL serves to provide clean, object-oriented access to standard OpenGL features, it also adds additional features beyond standard OpenGL. The JOGL team took the best ideas from several other previous Java OpenGL wrapper projects, such as GL4Java, while keeping it fast, clean, and easy to use.

JOGL is intended to be a community-development process. Developers and users can get involved through existing forums and mailing lists the java.net Web site on game development. Everyone is encouraged to post, and the participants on those sites are eager to hear about impressions and suggestions for its development.

JOGL is integrated with Java’s Abstract Window Toolkit and Swing widget sets. JOGL provides access to the latest OpenGL 1.4 with vendor extensions. JOGL also provides popular features introduced by previous Java bindings, including a composable pipeline model that can provide simpler debugging for JOGL applications than the analogous C OpenGL program. The GL class provides a minimal and easy-to-use API designed to handle many of the issues associated with multithreaded OpenGL applications. JOGL uses the most recent Java Platform features and, therefore, supports only J2SE 1.4 and later. Several complex and leading-edge OpenGL demonstrations have been successfully ported from C/C++ to JOGL. Finally, the JOGL binding is written almost completely in the Java programming language. The native code is autogenerated during the build process by a tool called GlueGen, which is also publicly available.
Integration with AWT

Java’s Abstract Window Toolkit, or AWT, is the set of standard Java APIs for GUI building and is part of the larger group of APIs called the Java Foundation Classes (JFC), included with the standard Java distributions. The vast majority of Java applications that use a graphic interface use the display classes from the JFC. Because JOGL is also a display technology, it was built to be integrated with AWT and compatible with AWT components.
The main display class in JOGL is an interface called GLDrawable. JOGL provides two implementations of this interface, GLCanvas and GLJPanel.
GLCanvas is a heavyweight AWT component, and GLJPanel is a lightweight Swing component, both of which provide OpenGL rendering support. The class inheritance structure for GLCanvas and GLJPanel follows, showing their relationship to AWT:net.java.games.jogl
Class GLCanvas

java.lang.Object

+—java.awt.Component

+—java.awt.Canvas

+—net.java.games.jogl.GLCanvas
net.java.games.jogl
Class GLJPanel
java.lang.Object

+—java.awt.Component

+—java.awt.Container

+—javax.swing.JComponent

+—javax.swing.JPanel

+—net.java.games.jogl.GLJPanel
Because GLCanvas and GLJPanel inherit from AWT and Swing components, respectively, they can be added to standard AWT and Swing applications and used directly there. This ability makes for nice integration options with existing as well as new AWT/Swing applications, gaming or otherwise.
Neither GLCanvas nor GLJPanel can be instantiated directly; the class GLDrawableFactory is used to create them. The GLDrawableFactory creates the canvas or panel based on default or user-defined GLCapabilities, which configure things such as the display area’s color depth, Z-depth, and other frame bufferer settings. See the API docs for more information.
Supports OpenGL 1.4
JOGL is completely up-to-date and supports the latest OpenGL 1.4 release. This is important for JOGL to be considered a leading-edge technology, and it will need to follow closely future OpenGL releases. OpenGL is a fairly large (and growing) API, and although designed to be as simple as possible, it is still fairly complex by most standards. Keeping the Java wrappers current would be a tedious task if it were done by hand as it was done in the past with other Java OpenGL bindings. One way this process is improved with JOGL is the use of a new tool, GlueGen.
GlueGen
GlueGen is an automatic tool for creating JNI wrappers from regular C code to become part of Java APIs.
JOGL is completely automatically generated from the OpenGL C header files. When a new GL version or function is released, GlueGen can reprocess the C source, and the new JOGL will instantly have access to the new functions.
GlueGen parses C header files using a full GNU C grammar and builds a memory mapping of all typedefs, enums, structs, pointers, function pointers, and other C structures. It then builds a parallel mapping of Java methods and structures to the C equivalents and writes all the Java code and JNI code necessary to link the two. In fact, it’s powerful enough to allow you to call C functions that use C structs as parameters or return values and builds Java-side accessor classes that wrap NIO buffers. This gives you fast, completely safe access to native data.
It cleanly handles all the platform-dependent complexities involved with determining whether or not a GL function is supported at runtime. This process takes into account the GL versions of the server and client and the extensions that are exposed on the client and server. At this writing, GlueGen is also used to build Java bindings for NVidia’s Cg language for JOGL. This is currently experimental, but it shows the power of GlueGen and the forward thinking of the JOGL team.

Vendor Extensions Exposed
All public OpenGL 1.4 vendor extensions are exposed directly in the GL interface. This does not mean that they are all implemented on any given system and will most likely need to be queried with the isExtensionAvailable() and isFunctionAvailable() GL methods, similar to regular C OpenGL. On many platforms, most functionality is still only OpenGL 1.1, so, in theory, any routines first exposed in OpenGL 1.2, 1.3, and 1.4 as well as vendor extensions should all be queried. Still, should an application call an unavailable or unsupported OpenGL function on the existing hardware, JOGL will gracefully throw a runtime GLException.
Designed for NIO
JOGL uses NIO’s ByteBuffers for many data access and manipulation operations. Rendering APIs such as OpenGL access much data and need it as fast as possible. Copying large arrays of floats between Java and the underlining system memory is prohibitively slow for a cutting-edge rendering system. Therefore, JOGL uses direct ByteBuffers for data wrappers where the Java and JNI OpenGL-side need to share data.

Composable Pipelines and DebugGL
JOGL has what is called composable pipelines. Composable pipelines are alternate render objects that can provide debugging and error-checking options in OpenGL. The DebugGL object can be swapped for the standard GL object at will, and DebugGL will do automatic error-checking “under the hood” to catch GL error conditions in application commands. This tool is valuable for debugging, as will be shown in the easy use of this in the main example.
Open Source

Finally, although not really an API feature, the designers choose to author the JOGL implementation as open source. As with any open-source project, one of the greatest benefits is that the developer always has the option to dig into the binding and make changes if needed. This flexibility is extremely helpful when a particular feature is not yet available in the current release build, or the implementation is buggy, or the interface disagrees with another possible design. During the course of writing this book, we built our own custom build of JOGL to allow access to features that were planned for future JOGL releases but not yet implemented in the public releases. This was of great help, allowing us to make progress on our projects independently of the current public release. When the following public release supported those features that we had added in our custom builds, we easily switched back to the standard public

OpenGL Overview

If you are familiar with OpenGL and have programmed with in the past, this section may serve as a brief review. The more experienced OpenGL programmer may choose to skip this section and continue on to the JOGL section.OpenGL is a widely used industry-standard real-time 3D graphics API. OpenGL is currently up to version 2.0.

OpenGL is no small system. It is large, complex, and often perplexing because errors (actual programming errors or just incorrect visual effects) can be hard to identify due to OpenGL’s state-based rendering. This section in no way covers everything a developer can create using OpenGL. Dozens of books are dedicated to that subject. At the same time, though, it is possible to use OpenGL (or JOGL) through a render-engine layer that simplifies the task of controlling, manipulating, and rendering a 3D scene, which is typical for 3D games as well. A solid understanding of OpenGL will benefit any developer using a 3D engine, whether or not they are called on to write actual OpenGL code.

High-Level or Low-Level API?
Depending on which group of developers you ask, OpenGL may be considered a low-level or high-level API. Because it allows the developer to work with geometric primitives such as triangles instead of pixels, it can be considered high-level. However, in the world of 3D game development, it is usually considered low-level because it is hardware supported, and you cannot program any lower while still using the hardware to its fullest capabilities. This is due to how video drivers use OpenGL as their main access layer. In theory, a developer could write his own device driver and use that to access the hardware, but it would have to be reverse-engineered because the specific hardware communication is almost always proprietary.
OpenGL provides access only to graphics-rendering operations. There are no facilities for obtaining user input from such devices as keyboards and mice, because it is expected that any system (in particular, a Windows system) under which OpenGL runs, must already provide such facilities.

One of OpenGL’s major strengths from the perceptive of a Java developer is that OpenGL is by far the most cross-platform 3D hardware-accelerated API available. Combining OpenGL with Java gives game developers an excellent cross-platform 3D game-technology foundation.

OpenGL Render Features

OpenGL has numerous features, many of which are not used in typical OpenGL-based games (such as anti-aliased lines) because most games simply don’t need them. It also has even more features that are not used regularly in games for other reasons, such as performance costs. A short subset of hardware-accelerated OpenGL features that do end up in games follows, many of which will be used in our example render engine:
Polygon/triangle, line, and pixel rendering
Smooth (Gourand) and Flat (Lambert) shading (with diffuse, ambient, and specular lighting effects)
Z buffering and double buffering
Stencil buffer
Colored directional and point light sources
Texture mapping
Texture filtering (MIPmapping and magmapping)
Texture environment controls, such as automatic texture coordinate generation
Backface culling
Matrix transformations
Perspective and parallel projections
Vertex programmability (on the latest hardware)
Fragments shaders, coming soon as a standard feature, currently supported through extensions
In addition, through the OpenGL extension mechanism, the latest hardware-accelerated features are readily accessible, even if the features are not part of the OpenGL standard yet. This is an excellent benefit for building and testing ahead of the curve for 3D game engines.
Immediate Mode Rendering

The basic model for OpenGL command interpretation is immediate mode, in which a command is executed as soon as the hardware receives it; vertex processing, for example, may begin even before specification of the primitive of which it is a part has been completed. Immediate mode execution is best suited to applications in which primitives and modes are constantly altered.
An example of immediate mode commands follow:

void display(void)
{
// draw diamond with radius of 10
glBegin( GL_QUADS );
glVertex3f( 10.0, 0.0, 0.0 );
glVertex3f( 0.0, 10.0, 0.0 );
glVertex3f( -10.0, 0.0, 0.0 );
glVertex3f( 0.0, -10.0, 0.0 );
glEnd();
// flush GL buffers
glFlush();
}

In immediate mode, a set of glBegin() and glEnd() calls establishing the primitive type contain the appropriate calls for that primitive to be rendered. This is sometimes called a batch.
Whereas immediate mode provides flexibility, it can be inefficient for large and unchanging parameters because each execution of the batch is respecifying the render object’s parameters redundantly. In the past, to accommodate such situations, OpenGL provided a feature known as display lists. A display list encapsulated a sequence of OpenGL commands and stored them on the hardware. The display list was given a numeric ID by the application when it was specified, and the application needed only to reference the display list ID to cause the hardware to effectively execute the list. Another similar option is vertex arrays and vertex buffer objects; we will examine those in a later section.
State Control

OpenGL is a state machine, meaning OpenGL’s render pipeline is defined by certain states at any given time, and those states control how the final image is rendered. Geometric primitives are also specified in a standard way. The current render state determines their shape, material properties, and orientation.

For example, setting the current drawing color could be executed by simply calling glColor3f (float red, float green, float blue) with the color represented as floats.
Calling glColor3f (1.0, 0.0, 0.0) would set the current color to bright red. Once set all geometric primitives rendered are affected by that red state until the states are changed.
Many states can be changed at any time, even per vertex. For example, the current color can potentially be set to a different color for each vertex in a batch, as shown in the following code:

glBegin( GL_QUADS );
// set this vertex’s color
glColor3f( 1.0, 0.0, 0.0 );
// set this vertex’s coordinate
glVertex3f( x1, y1, z1 );
// same as above, but for each new vertex
glColor3f( 0.0, 1.0, 0.0 );
glVertex3f( x2, y2, z2 );
glColor3f( 0.0, 0.0,1.0 );
glVertex3f( x3, y3, z3 );
glColor3f( 1.0, 1.0, 1.0 );
glVertex3f( x4, y4, z4 );
glEnd();

Light and lighting effects are also states in OpenGL. Like many other states, they must first be enabled to be used. In addition, other render states must be properly set in coordination with the lighting states to get the desired lighting effects. For example, object material states and vertex normals are needed for correct lighting, which can also be changed per vertex.

glEnable(GL_LIGHTING);
glEnable(GL_LIGHT0);
glLightfv(GL_LIGHT0, GL_DIFFUSE, color);
Triangle style is another controllable state in OpenGL. For example, you can control whether to draw the front or back of polygons (known as face culling) as well as whether they should be completely filled in or only in outline. glPolygonStyle(GL_FRONT, GL_FILL);
glFrontFace(GL_CCW); (How is front defined)
glCullFace(GL_BACK);
glEnable(GL_CULL_FACE);

Additional useful miscellaneous functions include setting the background color and clearing the screen and Z-buffer.
// Clears Color buffer and resets Z-buffer
glClear(GL_COLOR_BUFFER_BIT GL_DEPTH_BUFFER_BIT);
// Sets the clear color often thought of as the background color
glClearColor(....);
Transform Control
3D matrix transformations are key to manipulating 3D graphics. Therefore, OpenGL supports matrix transformations in the form of a render state. The current matrix state is applied to each vertex (and existing normals) during the geometry specification. On the modern graphics cards these transformations are accelerated by the graphics hardware.
There are several commands in OpenGL for matrix state manipulation, including load and multiple for complete, user-supplied matrices, and utility commands such as glLoadIdentity(), glRotate3f(d, x, y, z), glTranslate(tx, ty, tz), and glScale3f(sx, sy, sz). Also, OpenGL currently specifies three matrix modes—GL_MODELVIEW, GL_PROJECTION, and GL_TEXTURE—with the current mode set with the glMatrixMode command. OpenGL also implements for each of the matrix modes a matrix stack, which the user can then utilize with glPushMatrix and glPopMatrix. The stack depth is limited, depending on the mode—at least 32 for GL_MODELVIEW and at least 2 for GL_PROJECTION and GL_TEXTURE.
The glPushMatrix function duplicates the current matrix and then pushes that matrix into the stack. The glPopMatrix function pops the top matrix off of the stack and replaces the current matrix with that matrix. Each stack contains the identity matrix at initialization.
A typical routine setting up a basic camera matrix state (given the camera matrices) might look like the following code sample:// set up camera view
glMatrixMode(GL_PROJECTION);
glLoadMatrixd(camera.eyeFrustumMatrix);
glMatrixMode(GL_MODELVIEW);
glLoadMatrixd(camera.eyeViewMatrix);
The camera or view matrix is usually thought of as the product of the Projection matrix and the ModelView matrix, where the ModelView matrix represents the global camera/model transformation and the Projection matrix contains the view from which the related Projection matrix is to be applied.

Java New IO (java.nio)

Java New IO (java.nio)
The java.nio package was introduced with JDK 1.4 to provide more powerful IO and accommodate for some of the shortcomings of the existing java.io package and other IO-related packages. It is important to note that NIO does not replace the standard IO package but serves as a new approach to some IO problems. The NIO package was designed to provide the following:
Character sets
Scalable network IO
High-performance file IO
As explained in the Character Stream section, character streams use charsets, which are comprehensive mechanisms for converting between bytes and characters. Asynchronous or nonblocking network IO has been a long-standing problem for Java. Prior to NIO, an application could run into severe limitations if it needed to establish and maintain thousands of connections. The typical approach had been to create a thread for every connection. This approach is not a problem for a few dozen connections, but resource consumption becomes an issue when dealing with hundreds of threads, which is typical for server-side code. High-performance file IO is also important, and by providing a new approach to file IO, it has become possible to make many other performance improvements in other areas. java.nio introduced Buffers, which are one of the best additions ever made to Java, and in fact, games are one of the biggest beneficiaries of them. A significant benefit of Buffers is that they allow for efficient sharing of data between Java and native code. Because data such as textures, geometry, sound, and files is passed between Java and native code and is in turn passed to the corresponding device, Buffers can result in a noticeable performance gain. Before getting to Buffers, let’s look at channels, which are NIO’s version of streams.
Channels
Channels are used to represent a connection to an entity such as a file, network socket, or even a program component. They are in many ways comparable to streams and can be arguably referred to as more platform-dependent versions of streams. Because they have closer ties to the underlying platform, in conjunction with NIO buffers, they have the potential for achieving very efficient IO. Different types of channels such as DatagramChannel, FileChannel, and SocketChannel exist. Each of them provides a new way of dealing with IO. A DatagramChannel is a selectable channel for datagram-oriented sockets. A FileChannel is a channel for reading, writing, mapping, and manipulating a file. A SocketChannel is a select-able channel for stream-oriented connecting sockets and supports nonblocking connections.
A Channel object can be obtained by calling the getChannel methods of classes such as:java.io.FileInputStream
java.io.FileOutputStream
java.io.RandomAccessFile
java.net.Socket
java.net.ServerSocket
java.net.DatagramSocket
java.net.MulticastSocket
For example, a FileChannel can simply be obtained by using the following code:File inFile = new File(inputFilename);
FileInputStream fis = new FileInputStream(inFile);
FileChannel ifc = fis.getChannel();
A FileChannel is tightly bound to the object from which it is obtained. For example, reading a byte from the FileChannel updates the current position of the FileInputStream. A FileChannel can be obtained from a FileInputStream, FileOutputStream, or a RandomAccessFile. Note that if a FileChannel is obtained from a FileInputStream, you can only read from it; you cannot write to it. Similarly, if a channel is obtained from a FileOutputStream, it can only be written to and not read from. File channels can be used to duplicate a file in much the same way that file stream objects can be used to duplicate a file. One distinction is that file streams are typically read from and written to using an array of bytes, whereas channels are read from and written to using ByteBuffer objects A FileChannel can also be used to map a file into memory, which we will look at in an upcoming section. // using ByteBuffer
void test(String inputFilename, String outputFilename){
File inFile = new File(inputFilename);
File outFile = new File(outputFilename);
FileInputStream fis = new FileInputStream(inFile);
FileOutputStream fos = new FileOutputStream(outFile);
FileChannel ifc = fis.getChannel();
FileChannel ofc = fos.getChannel();

ByteBuffer bytes = ByteBuffer.allocate(bufferSize);
while (true){
bytes.clear();
int count = ifc.read(bytes);

if (count <= 0) break; bytes.flip(); ofc.write(bytes); } } Buffers
Buffers can result in substantial performance gains for tasks that involve getting a rather large chunk of data to the operating system or the hardware. This gain is not specific to file IO.

As you can a different buffer is available for every primitive type. A buffer is in many ways similar to an array. It represents a contiguous chunk of data and has a capacity that is fixed during its life. One of the fundamental differences between an array and a buffer is that a buffer can be direct, whereas arrays are always nondirect. A buffer can be thought of as an object that contains a reference to an array. A nondirect buffer’s array is a typical Java array that resides in the Java heap, but a direct buffer’s array resides in system memory.

It is important to understand the difference between the Java heap and the system memory. Here we define the Java heap as the memory maintained by the garbage collector and system memory as all the memory on the host machine minus the Java heap. Typical native APIs including the OS API take the memory address of (or a pointer to) the data that needs to be manipulated. Because the data in the Java heap is managed by the garbage collector, direct access to data in the Java heap cannot be safely granted to native APIs without serious implications. This means that Java applications or the VM have an extra problem to deal with.
Note that the garbage collector may decide to move objects around in the heap when performing housekeeping. If some data in the Java heap must be passed to, say, the hard drive or the video card, the data in the heap is not guaranteed to stay where it is until the transfer is completed. One workaround would be to disable the garbage collector when any data in the heap must be passed to native APIs, such as the operating system. However, this action can result in serious problems. For example, if the garbage collector is disabled and a data transfer to a device is time consuming, the VM may run out of memory, even if there is a substantial amount of dead objects that could have been collected to free up some space. The other option would be to pin or lock the object so that the garbage collector knows not to move it. This technique would require less efficient and more complicated collection algorithms and has other side effects. Because of this, the VM does not allow direct access to the data in the Java heap.
Before the introduction of direct byte buffers, the solution to dealing with this problem was to copy the data from the Java heap to the system memory before it could be passed to a native function. Direct byte buffers allow the data to reside in the system memory so that when it needs to be passed to a native function, it is already in the system memory. This is the main idea behind why direct byte buffers can improve the performance of an application.
The overhead of copying the data may not be a bottleneck for small chunks of data that are not time critical and are rarely passed to native functions. However, the copy overhead can prove to be a bottleneck in many situations in different applications, especially games. Games rely heavily on the manipulation of data that is passed to the OS or the hardware. Geometry that needs to be manipulated by Java code is one example of where direct buffers can result in considerable performance gains.

A ByteBuffer is for storing any binary data. The CharBuffer, DoubleBuffer, FloatBuffer, IntBuffer, and ShortBuffer classes, however, are for dealing with data of a specific type. There are different ways to allocate the array that stores the content of a buffer. The following code segment results in a creation of a nondirect buffer:ByteBuffer byteBuffer = ByteBuffer.allocate(bufferSize);
IntBuffer intBuffer = IntBuffer.allocate(bufferSize);
In addition, a buffer can simply wrap an existing array of appropriate type using its wrap method. Because Java arrays reside in the Java heap, the following buffer is implicitly a nondirect buffer:int myArray[] = new int[8];
IntBuffer intBuffer = IntBuffer.wrap(myArray);
ByteBuffer objects provide the allocateDirect method that can be used to explicitly ask the array to reside in the system memory as opposed to the Java heap. The following call is used to create a direct byte buffer:
ByteBuffer byteBuffer = ByteBuffer.allocateDirect(bufferSize);
Direct buffers can also be created by wrapping any region of the system memory through the NewDirectByteBuffer function, which is provided by JNI. In addition, a MappedbyteBuffer, which is a form of direct byte buffer, can be used to access memory in the OS buffer .

A byte buffer also provides methods that create other buffers to view its content as other primitive types. A view buffer is a buffer whose array is the same as the array of the byte buffer that was used to create it. A view buffer is not indexed in bytes but in corresponding primitive type. The only way to create a direct, say, float buffer would be to first create a direct byte buffer and then call its asFloatBuffer method to create a view. Because a byte buffer provides an interface for reading and writing primitive types, when dealing with multibyte data, the order of the bytes matters. The byte order can be either big-endian or little-endian. In the big-endian byte order, the most significant byte is stored at the lowest storage address (that is, the first address). In a little-endian byte order, the least significant byte is stored first .

Note that if you put an integer that occupies four bytes as little-endian and get it back as big-endian, the result will be entirely different. This is not a concern specific to Java but has to be dealt with in any language, especially when different applications share data through means such as a file or network packets.

The default byte order is big-endian, but it can be set by calling order (ByteOrder newOrder). Note that different hardware uses different byte order. For example, Intel hardware uses little-endian byte order. The method ByteOrder.nativeOrder can be used to obtain the byte order of the underlying hardware.
A Buffer is allocated once and is typically reused. A Buffer contains member variables such as capacity, limit, and position. It also contains methods such as clear, rewind, flip, and reset that manipulate these members. Once allocated, the capacity of a buffer cannot be changed. When the clear method of a buffer is called, its position is 0 and its limit is set to its capacity.

The position indicates the next index of the array where data can be written to or read from. As data is written to the buffer, the position is updated. The position of a buffer can be retrieved and set using the position() and position(int newPosition) accessors. Subclasses of the Buffer class can choose to use position to provide relative put and get methods. The limit of a buffer is used to keep track of the segment of the buffer that contains valid data. For example, when a buffer is filled only half way, it is important to know the index up to where valid data has been filled. As with position, the limit can be retrieved and set using the limit() and limit(int newPosition) accessors. If a relative put operation tries to write past the limit, a BufferOverflowException is thrown and if the get operation tries to read past the limit, a BufferUnderflowException is thrown. The remaining method simply returns the difference between position and limit.
The flip method is one of the most important. Once all the data of interest has been written to a buffer, the limit of the buffer must be set to the end of the new or relevant data in the buffer so that an object that reads the data knows when it has reached the end of relevant data. It is important to note that reading until the capacity is reached is not correct because the buffer might not be full. The flip method sets the limit to position and position to 0. By doing so, a reading object can start reading from position and stop when position has reached the limit.

As the data in a buffer is read, the position is updated accordingly. If you want to reread the same data, you can invoke the reset method. Buffers also allow an index to be marked using the mark method. If you care to set the position back to a marked index, the reset method can be used. The rewind method simply sets the position to 0. In general, it is correct to say that the mark can be greater or equal to 0, the position can be greater or equal to the limit, and the limit can be greater or equal to the capacity. 0 <= mark <= position <= limit <= capacity
MappedByteBuffers

MappedByteBuffers are a form of direct byte buffer that were also introduced with JDK 1.4. They are rather complicated to understand compared to typical buffers, and if you want to use them to improve the performance of your game, you really must understand how they work. They can be rather mysterious if you do not understand what happens behind the scenes. They can bring efficiency if they are used properly but can also decrease performance, increase memory consumption, and reduce available disk space if they are not used properly. It is important to note that MappedByteBuffers are one of the most OS-dependent objects that have been added to Java. Some exact specifics of their behavior are not defined in the JDK documentation, and you are advised to check the OS-specific documentation. When necessary, this section assumes the behavior of MappedByteBuffers under the Windows operating system.
A MappedByteBuffer is a direct byte buffer whose content corresponds to a region of a file, and is managed by the operating system. In fact, the content of a MappedByteBuffer resides in the buffer or cache labeled as the OS buffer . In other words, a MappedbyteBuffer can be used to wrap a region of the OS buffer. This means that an application can manipulate the data in the OS buffer without having to use an intermediate buffer, or a buffer managed by the application. By being able to access the OS buffer, an application does not have to endure the cost of copying data from the OS buffer to the application buffer. However, this is not the main advantage. The main advantage is that because the OS is managing the data, it can do certain things that would not be possible if the application managed the data. The operating system can decide to update some of the data in the buffer so that it corresponds with the data on the disk, or to update some of the data on the disk to correspond to the data in the buffer that has been manipulated by the application. Because the OS is aware of the state of the system, it may decide to unload parts of the file data from the OS buffer if it is low on memory. It can also avoid reloading the content of the file until it knows for a fact that an application needs to use a specific portion of a file. In addition, because the data is maintained at the OS level, if multiple applications need to access a certain file, they can simply gain access to the same region of the OS buffer. In fact, this technique is sometimes used to share data between multiple applications. The shared part of the OS buffer can be looked at as a form of shared memory backed by a file. All of this can happen without the application having to know about what happens behind the scenes. In fact, once the buffer is retrieved, it is used as a typical direct byte buffer.
A region of a file that resides in the OS buffer. The region is visible to two different processes. Each process, represented by a block, stands for the memory local to the process. Each view in a process indicates a different MappedByteBuffer object. The diagram shows three different MappedByteBuffers of which one belongs to Process 1 and two belong to Process 2. The memory addresses wrapped by each byte buffer is actually local to its corresponding process. That is, as far as the process is concerned, it thinks that it is accessing local data. The OS, however, knows that the memory region mapped by each view really refers to the memory pages in the OS buffer. Therefore, even though each process thinks that it is using some chunk of memory to represent the file region, the data is strictly resident in the OS buffer.
Figure 5.10: Memory mapped file region.
Process 1 has a view that can see the entire memory mapped region. Process 2 has two different views that see parts of the region visible to Process 1. As you can see, the content of the byte buffers are overlapping. Therefore, if Process 2 modifies the content of the buffer denoted by View 2, both View 1 of Process 2 and View 1 of Process 1 would be able to access the updated data. When the content of Region 1 is modified through any of the byte buffers, the OS does not immediately update the content of the file. It updates the file only when it finds it necessary.
Region 1 is only part of the actual file on the disk. Therefore, to modify a region of the file, the entire file does not have to be loaded. Furthermore, the amount of memory occupied in the OS buffer is not as much as the size of the region. If the three views are created but none of them is accessed, no memory is committed in the OS buffer. However, when the first byte of the byte buffer of Process 1 is accessed, one page of memory is committed in the OS buffer and updated to reflect the data of the corresponding section of the file. If the second byte of the same byte buffer is accessed, no IO will occur and no additional memory is committed. If the first byte of the byte buffer corresponding to View 1 of Process 2 is accessed, the data is available and already up to date. In fact, 4K of consecutive bytes can be read without any need for the disk to be accessed. If the last byte of the second byte buffer of Process 2 is accessed, another page is committed, which would mean that only two pages have actually been allocated.
Now that you see when the OS actually reads data from the disk, let’s see when it decides to unload them. There is no direct way to force the OS to unload the committed pages because that would intervene with the management strategy of the OS. Additionally, the memory can be uncommitted only if no other views of the region exist. To decrease the time it takes before the OS uncommits the pages, you can unmap the buffers and close any streams or channels that refer to the file. A mapped byte buffer is unmapped when its finalizer is called, which is some time after the object is no longer reachable.
The OS may decide to release a page if it needs to make room for other purposes. For example, if the game starts to perform some memory-intensive computations, the OS may decide to unload some of the pages. Whether the pages are written back to the disk depends on whether the pages have been modified. If the pages have not been modified, the memory is simply freed. In fact, when a buffer is mapped, a parameter is used to specify its mode. A MappedByteBuffer can be either read-only, read-write, or private. If a buffer is read-only, it slightly helps the unloading process. When a buffer that is in private mode is written to, the OS copies any of the necessary pages and modifies only the copies. The advantage of this approach over making a local copy of the file content and storing it locally to the process is that the OS will still manage the pages. If the OS needs to make more room, a modified copy is saved to the system page file instead of the original file.
It is important to note that the use of a MappedByteBuffer can affect the amount of memory used as well as the amount of space used on the disk. If, for example, many sections of a rather large file are loaded into the OS buffer, the virtual memory manager may consider saving out other memory pages to the system page file, which can in turn increase the amount of space used on the drive.
The data maintained in the OS buffer is not saved to the file immediately, so it is crucial to have a way of forcing an update. This is exactly what the force method of MappedByteBuffer does. Note that forcing the changes to be written to the file is different from trying to make the OS uncommit or free some of the pages that it has allocated to store sections of the file. As mentioned already, the latter is not possible. The MappedByteBuffer object also has a load method that forces the entire mapped region to be loaded into the OS buffer. The load method simply walks the direct buffer at 4K (or page size) intervals and touches each page to achieve the desired result. Keep in mind, however, that loading an entire mapped region causes the application to use a significant amount of memory if the mapped region is large.
A MappedByteBuffer can be retrieved from a file channel. As mentioned earlier, a file channel can be retrieved from a FileInputStream, FileOutputStream, or RandomAccessFile. The following example uses a MappedByteBuffer to copy a file:File inFile = new File(inputFilename);
File outFile = new File(outputFilename);
FileInputStream fis = new FileInputStream(inFile);
RandomAccessFile fos = new RandomAccessFile(outFile, "rw");
FileChannel ifc = fis.getChannel();
FileChannel ofc = fos.getChannel();

long sourceSize = ifc.size();

MappedByteBuffer bytes1 = ifc.map(
FileChannel.MapMode.READ_ONLY, 0, sourceSize);
bytes1.load();
MappedByteBuffer bytes2 = ofc.map(
FileChannel.MapMode.READ_WRITE, 0, sourceSize);
bytes2.load();
// finally, copy the data
bytes2.put(bytes1);
How expensive do you think executing the put method is? Note that because the mapped regions have been loaded already through the load method, the put method results only in copying of data from one part of the OS buffer to another. This is a memory copy and the disk is not even accessed, so the put method alone is extremely fast. However, if the buffer’s load method is not called before the put method is invoked, the put method would end up being very expensive because the entire source file would have to first be loaded into the OS buffer. Also, note that when the put method is done, the destination file has not been literally updated yet. To have the data written to the file, you can call the force method of the buffer. Note that closing the channel does not affect the mapped buffer. Once the file is mapped into memory, it has nothing to do with the channel from which it was created. As a side note, the behavior of the map method of Channel is unspecified and OS dependent when the requested region is not completely contained within the corresponding channel’s file. The example just shown assumes that the source and destination files of the given size exist already and the intention is to overwrite the content of the destination file.