diff --git a/.gitignore b/.gitignore index 0d2a5781e2e..dca64778300 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ build/build.number +<<<<<<< HEAD build.number bin cloudstack-proprietary @@ -15,3 +16,23 @@ dist cloud-*.tar.bz2 *.pyc build.number +======= +bin/ +cloudstack-proprietary/ +premium/ +.lock-wscript +artifacts/ +.waf-* +waf-* +target/ +override/ +.metadata +dist/ +*~ +*.bak +cloud-*.tar.bz2 +*.log +*.pyc +build.number +cloud.log.*.* +>>>>>>> master diff --git a/HACKING b/HACKING deleted file mode 100644 index b6a16c3ef5e..00000000000 --- a/HACKING +++ /dev/null @@ -1,652 +0,0 @@ ---------------------------------------------------------------------- -THE QUICK GUIDE TO CLOUDSTACK DEVELOPMENT ---------------------------------------------------------------------- - - -=== Overview of the development lifecycle === - -To hack on a CloudStack component, you will generally: - -1. Configure the source code: - ./waf configure --prefix=/home/youruser/cloudstack - (see below, "./waf configure") - -2. Build and install the CloudStack - ./waf install - (see below, "./waf install") - -3. Set the CloudStack component up - (see below, "Running the CloudStack components from source") - -4. Run the CloudStack component - (see below, "Running the CloudStack components from source") - -5. Modify the source code - -6. Build and install the CloudStack again - ./waf install --preserve-config - (see below, "./waf install") - -7. GOTO 4 - - -=== What is this waf thing in my development lifecycle? === - -waf is a self-contained, advanced build system written by Thomas Nagy, -in the spirit of SCons or the GNU autotools suite. - -* To run waf on Linux / Mac: ./waf [...commands...] -* To run waf on Windows: waf.bat [...commands...] - -./waf --help should be your first discovery point to find out both the -configure-time options and the different processes that you can run -using waf. - - -=== What do the different waf commands above do? === - -1. ./waf configure --prefix=/some/path - - You run this command *once*, in preparation to building, or every - time you need to change a configure-time variable. - - This runs configure() in wscript, which takes care of setting the - variables and options that waf will use for compilation and - installation, including the installation directory (PREFIX). - - For convenience reasons, if you forget to run configure, waf - will proceed with some default configuration options. By - default, PREFIX is /usr/local, but you can set it e.g. to - /home/youruser/cloudstack if you plan to do a non-root - install. Be ware that you can later install the stack as a - regular user, but most components need to *run* as root. - - ./waf showconfig displays the values of the configure-time options - -2. ./waf - - You run this command to trigger compilation of the modified files. - - This runs the contents of wscript_build, which takes care of - discovering and describing what needs to be built, which - build products / sources need to be installed, and where. - -3. ./waf install - - You run this command when you want to install the CloudStack. - - If you are going to install for production, you should run this - process as root. If, conversely, you only want to install the - stack as your own user and in a directory that you have write - permission, it's fine to run waf install as your own user. - - This runs the contents of wscript_build, with an option variable - Options.is_install = True. When this variable is set, waf will - install the files described in wscript_build. For convenience - reasons, when you run install, any files that need to be recompiled - will also be recompiled prior to installation. - - -------------------- - - WARNING: each time you do ./waf install, the configuration files - in the installation directory are *overwritten*. - - There are, however, two ways to get around this: - - a) ./waf install has an option --preserve-config. If you pass - this option when installing, configuration files are never - overwritten. - - This option is useful when you have modified source files and - you need to deploy them on a system that already has the - CloudStack installed and configured, but you do *not* want to - overwrite the existing configuration of the CloudStack. - - If, however, you have reconfigured and rebuilt the source - since the last time you did ./waf install, then you are - advised to replace the configuration files and set the - components up again, because some configuration files - in the source use identifiers that may have changed during - the last ./waf configure. So, if this is your case, check - out the next way: - - b) Every configuration file can be overridden in the source - without touching the original. - - - Look for said config file X (or X.in) in the source, then - - create an override/ folder in the folder that contains X, then - - place a file named X (or X.in) inside override/, then - - put the desired contents inside X (or X.in) - - Now, every time you run ./waf install, the file that will be - installed is path/to/override/X.in, instead of /path/to/X.in. - - This option is useful if you are developing the CloudStack - and constantly reinstalling it. It guarantees that every - time you install the CloudStack, the installation will have - the correct configuration and will be ready to run. - - -=== Running the CloudStack components from source (for debugging / coding) === - -It is not technically possible to run the CloudStack components from -the source. That, however, is fine -- each component can be run -independently from the install directory: - -- Management Server - - 1) Execute ./waf install as your current user (or as root if the - installation path is only writable by root). - - WARNING: if any CloudStack configuration files have been - already configured / altered, they will be *overwritten* by this - process. Append --preserve-config to ./waf install to prevent this - from happening. Or resort to the override method discussed - above (search for "override" in this document). - - 2) If you haven't done so yet, set up the management server database: - - - either run ./waf deploydb_kvm, or - - run $BINDIR/cloud-setup-databases - - 3) Execute ./waf run as your current user (or as root if the - installation path is only writable by root). Alternatively, - you can use ./waf debug and this will run with debugging enabled. - - -- Agent (Linux-only): - - 1) Execute ./waf install as your current user (or as root if the - installation path is only writable by root). - - WARNING: if any CloudStack configuration files have been - already configured / altered, they will be *overwritten* by this - process. Append --preserve-config to ./waf install to prevent this - from happening. Or resort to the override method discussed - above (search for "override" in this document). - - 2) If you haven't done so yet, set the Agent up: - - - run $BINDIR/cloud-setup-agent - - 3) Execute ./waf run_agent as root - - this will launch sudo and require your root password unless you have - set sudo up not to ask for it - - -- Console Proxy (Linux-only): - - 1) Execute ./waf install as your current user (or as root if the - installation path is only writable by root). - - WARNING: if any CloudStack configuration files have been - already configured / altered, they will be *overwritten* by this - process. Append --preserve-config to ./waf install to prevent this - from happening. Or resort to the override method discussed - above (search for "override" in this document). - - 2) If you haven't done so yet, set the Console Proxy up: - - - run $BINDIR/cloud-setup-console-proxy - - 3) Execute ./waf run_console_proxy - - this will launch sudo and require your root password unless you have - set sudo up not to ask for it - - ---------------------------------------------------------------------- -BUILD SYSTEM TIPS ---------------------------------------------------------------------- - - -=== Integrating compilation and execution of each component into Eclipse === - -To run the Management Server from Eclipse, set up an External Tool of the -Program variety. Put the path to the waf binary in the Location of the -window, and the source directory as Working Directory. Then specify -"install --preserve-config run" as arguments (without the quotes). You can -now use the Run button in Eclipse to execute the Management Server directly -from Eclipse. You can replace run with debug if you want to run the -Management Server with the Debugging Proxy turned on. - -To run the Agent or Console Proxy from Eclipse, set up an External Tool of -the Program variety just like in the Management Server case. In there, -however, specify "install --preserve-config run_agent" or -"install --preserve-config run_console_proxy" as arguments instead. -Remember that you need to set sudo up to not ask you for a password and not -require a TTY, otherwise sudo -- implicitly called by waf run_agent or -waf run_console_proxy -- will refuse to work. - - -=== Building targets selectively === - -You can find out the targets of the build system: - -./waf list_targets - -If you want to run a specific task generator, - -./waf build --targets=patchsubst - -should run just that one (and whatever targets are required to build that -one, of course). - - -=== Common targets === - -* ./waf configure: you must always run configure once, and provide it with - the target installation paths for when you run install later - o --help: will show you all the configure options - o --no-dep-check: will skip dependency checks for java packages - needed to compile (saves 20 seconds when redoing the configure) - o --with-db-user, --with-db-pw, --with-db-host: informs the build - system of the MySQL configuration needed to set up the management - server upon install, and to do deploydb - -* ./waf build: will compile any source files (and, on some projects, will - also perform any variable substitutions on any .in files such as the - MANIFEST files). Build outputs will be in /artifacts/default. - -* ./waf install: will compile if not compiled yet, then execute an install - of the built targets. I had to write a significantly large amount of code - (that is, couple tens of lines of code) to make install work. - -* ./waf run: will run the management server in the foreground - -* ./waf debug: will run the management server in the foreground, and open - port 8787 to connect with the debugger (see the Run / debug options of - waf --help to change that port) - -* ./waf deploydb: deploys the database using the MySQL configuration supplied - with the configuration options when you did ./waf configure. RUN WAF BUILD - FIRST AT LEAST ONCE. - -* ./waf dist: create a source tarball. These tarballs will be distributed - independently on our Web site, and will form the source release of the - Cloud Stack. It is a self-contained release that can be ./waf built and - ./waf installed everywhere. - -* ./waf clean: remove known build products - -* ./waf distclean: remove the artifacts/ directory altogether - -* ./waf uninstall: uninstall all installed files - -* ./waf rpm: build RPM packages - o if the build fails because the system lacks dependencies from our - other modules, waf will attempt to install RPMs from the repos, - then try the build - o it will place the built packages in artifacts/rpmbuild/ - -* ./waf deb: build Debian packages - o if the build fails because the system lacks dependencies from our - other modules, waf will attempt to install DEBs from the repos, - then try the build - o it will place the built packages in artifacts/debbuild/ - -* ./waf uninstallrpms: removes all Cloud.com RPMs from a system (but not - logfiles or modified config files) - -* ./waf viewrpmdeps: displays RPM dependencies declared in the RPM specfile - -* ./waf installrpmdeps: runs Yum to install the packages required to build - the CloudStack - -* ./waf uninstalldebs: removes all Cloud.com DEBs from a system (AND logfiles - AND modified config files) -* ./waf viewdebdeps: displays DEB dependencies declared in the project - debian/control file - -* ./waf installdebdeps: runs aptitude to install the packages required to - build our software - - -=== Overriding certain source files === - -Earlier in this document we explored overriding configuration files. -Overrides are not limited to configuration files. - -If you want to provide your own server-setup.xml or SQL files in client/setup: - - * create a directory override inside the client/setup folder - * place your file that should override a file in client/setup there - -There's also override support in client/tomcatconf and agent/conf. - - -=== Environment substitutions === - -Any file named "something.in" has its tokens (@SOMETOKEN@) automatically -substituted for the corresponding build environment variable. The build -environment variables are generally constructed at configure time and -controllable by the --command-line-parameters to waf configure, and should -be available as a list of variables inside the file -artifacts/c4che/build.default.py. - - -=== The prerelease mechanism === - -The prerelease mechanism (--prerelease=BRANCHNAME) allows developers and -builders to build packages with pre-release Release tags. The Release tags -are constructed in such a way that both the build number and the branch name -is included, so developers can push these packages to repositories and upgrade -them using yum or aptitude without having to delete packages manually and -install packages manually every time a new build is done. Any package built -with the prerelease mechanism gets a standard X.Y.Z version number -- and, -due to the way that the prerelease Release tags are concocted, always upgrades -any older prerelease package already present on any system. The prerelease -mechanism must never be used to create packages that are intended to be -released as stable software to the general public. - -Relevant documentation: - - http://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Version - http://fedoraproject.org/wiki/PackageNamingGuidelines#Pre-Release_packages - -Everything comes together on the build server in the following way: - - -=== SCCS info === - -When building a source distribution (waf dist), or RPM/DEB distributions -(waf deb / waf rpm), waf will automatically detect the relevant source code -control information if the git command is present on the machine where waf -is run, and it will write the information to a file called sccs-info inside -the source tarball / install it into /usr/share/doc/cloud*/sccs-info when -installing the packages. - -If this source code conrol information cannot be calculated, then the old -sccs-info file is preserved across dist runs if it exists, and if it did -not exist before, the fact that the source could not be properly tracked -down to a repository is noted in the file. - - -=== Debugging the build system === - -Almost all targets have names. waf build -vvvvv --zones=task will give you -the task names that you can use in --targets. - - ---------------------------------------------------------------------- -UNDERSTANDING THE BUILD SYSTEM ---------------------------------------------------------------------- - - -=== Documentation for the build system === - -The first and foremost reference material: - -- http://freehackers.org/~tnagy/wafbook/index.html - -Examples - -- http://code.google.com/p/waf/wiki/CodeSnippets -- http://code.google.com/p/waf/w/list - -FAQ - -- http://code.google.com/p/waf/wiki/FAQ - - -=== Why waf === - -The CloudStack uses waf to build itself. waf is a relative newcomer -to the build system world; it borrows concepts from SCons and -other later-generation build systems: - -- waf is very flexible and rich; unlike other build systems, it covers - the entire life cycle, from compilation to installation to - uninstallation. it also supports dist (create source tarball), - distcheck (check that the source tarball compiles and installs), - autoconf-like checks for dependencies at compilation time, - and more. - -- waf is self-contained. A single file, distributed with the project, - enables everything to be built, with only a dependency on Python, - which is freely available and shipped in all Linux computers. - -- waf also supports building projects written in multiple languages - (in the case of the CloudStack, we build from C, Java and Python). - -- since waf is written in Python, the entire library of the Python - language is available to use in the build process. - - -=== Hacking on the build system: what are these wscript files? === - -1. wscript: contains most commands you can run from within waf -2. wscript_configure: contains the process that discovers the software - on the system and configures the build to fit that -2. wscript_build: contains a manifest of *what* is built and installed - -Refer to the waf book for general information on waf: - http://freehackers.org/~tnagy/wafbook/index.html - - -=== What happens when waf runs === - -When you run waf, this happens behind the scenes: - -- When you run waf for the first time, it unpacks itself to a hidden - directory .waf-1.X.Y.MD5SUM, including the main program and all - the Python libraries it provides and needs. - -- Immediately after unpacking itself, waf reads the wscript file - at the root of the source directory. After parsing this file and - loading the functions defined here, it reads wscript_build and - generates a function build() based on it. - -- After loading the build scripts as explained above, waf calls - the functions you specified in the command line. - -So, for example, ./waf configure build install will: - -* call configure() from wscript, -* call build() loaded from the contents of wscript_build, -* call build() once more but with Options.is_install = True. - -As part of build(), waf invokes ant to build the Java portion of our -stack. - - -=== How and why we use ant within waf === - -By now, you have probably noticed that we do, indeed, ship ant -build files in the CloudStack. During the build process, waf calls -ant directly to build the Java portions of our stack, and it uses -the resulting JAR files to perform the installation. - -The reason we do this rather than use the native waf capabilities -for building Java projects is simple: by using ant, we can leverage -the support built-in for ant in Eclipse and many other IDEs. Another -reason to do this is because Java developers are familiar with ant, -so adding a new JAR file or modifying what gets built into the -existing JAR files is facilitated for Java developers. - -If you add to the ant build files a new ant target that uses the -compile-java macro, waf will automatically pick it up, along with its -depends= and JAR name attributes. In general, all you need to do is -add the produced JAR name to the packaging manifests (cloud.spec and -debian/{name-of-package}.install). - - ---------------------------------------------------------------------- -FOR ANT USERS ---------------------------------------------------------------------- - - -If you are using Ant directly instead of using waf, these instructions apply to you: - -in this document, the example instructions are based on local source repository rooted at c:\root. You are free to locate it to anywhere you'd like to. -3.1 Setup developer build type - - 1) Go to c:\cloud\java\build directory - - 2) Copy file build-cloud.properties.template to file build-cloud.properties, then modify some of the parameters to match your local setup. The template properties file should have content as - - debug=true - debuglevel=lines,vars,source - tomcat.home=$TOMCAT_HOME --> change to your local Tomcat root directory such as c:/apache-tomcat-6.0.18 - debug.jvmarg=-Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n - deprecation=off - build.type=developer - target.compat.version=1.5 - source.compat.version=1.5 - branding.name=default - - 3) Make sure the following Environment variables and Path are set: - -set enviroment variables: -CATALINA_HOME: -JAVA_HOME: -CLOUD_HOME: -MYSQL_HOME: - -update the path to include - -MYSQL_HOME\bin - - 4) Clone a full directory tree of C:\cloud\java\build\deploy\production to C:\cloud\java\build\deploy\developer - - You can use Windows Explorer to copy the directory tree over. Please note, during your daily development process, whenever you see updates in C:\cloud\java\build\deploy\production, be sure to sync it into C:\cloud\java\build\deploy\developer. -3.2 Common build instructions - -After you have setup the build type, you are ready to perform build and run Management Server alone locally. - -cd java -python waf configure build install - -More at Build system. - -Will install the management server and its requisites to the appropriate place (your Tomcat instance on Windows, /usr/local on Linux). It will also install the agent to /usr/local/cloud/agent (this will change in the future). -4. Database and Server deployment - -After a successful management server build (database deployment scripts use some of the artifacts from build process), you can use database deployment script to deploy and initialize the database. You can find the deployment scripts in C:/cloud/java/build/deploy/db. deploy-db.sh is used to create, populate your DB instance. Please take a look at content of deploy-db.sh for more details - -Before you run the scripts, you should edit C:/cloud/java/build/deploy/developer/db/server-setup-dev.xml to allocate Public and Private IP ranges for your development setup. Ensure that the ranges you pick are unallocated to others. - -Customized VM templates to be populated are in C:/cloud/java/build/deploy/developer/db/templates-dev.sql Edit this file to customize the templates to your needs. - -Deploy the DB by running - -./deploy-db.sh ../developer/db/server-setup-dev.xml ../developer/db/templates-dev.xml -4.1. Management Server Deployment - -ant build-server - -Build Management Server - -ant deploy-server - -Deploy Management Server software to Tomcat environment - -ant debug - -Start Management Server in debug mode. The JVM debug options can be found in cloud-build.properties - -ant run - -Start Management Server in normal mode. - -5. Agent deployment - -After a successful build process, you should be able to find build artifacts at distribution directory, in this example case, for developer build type, the artifacts locate at c:\cloud\java\dist\developer, particularly, if you have run - -ant package-agent build command, you should see the agent software be packaged in a single file named agent.zip under c:\cloud\java\dist\developer, together with the agent deployment script deploy-agent.sh. -5.1 Agent Type - -Agent software can be deployed and configured to serve with different roles at run time. In current implementation, there are 3 types of agent configuration, respectively called as Computing Server, Routing Server and Storage Server. - - * When agent software is configured to run as Computing server, it is responsible to host user VMs. Agent software should be running in Xen Dom0 system on computer server machine. - - * When agent software is configured to run as Routing Server, it is responsible to host routing VMs for user virtual network and console proxy system VMs. Routing server serves as the bridge to outside network, the machine that agent software is running should have at least two network interfaces, one towards outside network, one participates the internal VMOps management network. Like computer server, agent software on routing server should also be running in Xen Dom0 system. - - * When agent software is configured to run as Storage server, it is responsible to provide storage service for all VMs. The storage service is based on ZFS running on a Solaris system, agent software on storage server is therefore running under Solaris (actually a Solaris VM), Dom0 systems on computing server and routing server can access the storage service through iScsi initiator. The storage volume will be eventually mounted on Dom0 system and make available to DomU VMs through our agent software. - -5.2 Resource sharing - -All developers can share the same set of agent server machines for development, to make this possible, the concept of instance appears in various places - - * VM names. VM names are structual names, it contains a instance section that can identify VMs from different VMOps cloud instances. VMOps cloud instance name is configured in server configuration parameter AgentManager/instance.name - * iScsi initiator mount point. For Computing servers and Routing servers, the mount point can distinguish the mounted DomU VM images from different agent deployments. The mount location can be specified in agent.properties file with a name-value pair named mount.parent - * iScsi target allocation point. For storage servers, this allocation point can distinguish the storage allocation from different storage agent deployments. The allocation point can be specified in agent.properties file with a name-value pair named parent - -5.4 Deploy agent software - -Before running the deployment scripts, first copy the build artifacts agent.zip and deploy-agent.sh to your personal development directory on agent server machines. By our current convention, you can create your personal development directory that usually locates at /root/your name. In following example, the agent package and deployment scripts are copied to test0.lab.vmops.com and the deployment script file has been marked as executible. - - On build machine, - - scp agent.zip root@test0:/root/your name - - scp deploy-agent.sh root@test0:/root/your name - - On agent server machine - -chmod +x deploy-agent.sh -5.4.1 Deploy agent on computing server - -deploy-agent.sh -d /root//agent -h -t computing -m expert -5.4.2 Deploy agent on routing server - -deploy-agent.sh -d /root//agent -h -t routing -m expert -5.4.3 Deploy agent on storage server - -deploy-agent.sh -d /root//agent -h -t storage -m expert -5.5 Configure agent - -After you have deployed the agent software, you should configure the agent by editing the agent.properties file under /root//agent/conf directory on each of the Routing, Computing and Storage servers. Add/Edit following properties. The rest are defaults that get populated by the agent at runtime. - workers=3 - host= - port=8250 - pod= - zone= - instance= - developer=true - -Following is a sample agent.properties file for Routing server - - workers=3 - id=1 - port=8250 - pod=RC - storage=comstar - zone=RC - type=routing - private.network.nic=xenbr0 - instance=RC - public.network.nic=xenbr1 - developer=true - host=192.168.1.138 -5.5 Running agent - -Edit /root//agent/conf/log4j-cloud.xml to update the location of logs to somewhere under /root/ - -Once you have deployed and configured the agent software, you are ready to launch it. Under the agent root directory (in our example, /root//agent. there is a scrip file named run.sh, you can use it to launch the agent. - -Launch agent in detached background process - -nohup ./run.sh & - -Launch agent in interactive mode - -./run.sh - -Launch agent in debug mode, for example, following command makes JVM listen at TCP port 8787 - -./run.sh -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n - -If agent is launched in debug mode, you may use Eclipse IDE to remotely debug it, please note, when you are sharing agent server machine with others, choose a TCP port that is not in use by someone else. - -Please also note that, run.sh also searches for /etc/cloud directory for agent.properties, make sure it uses the correct agent.properties file! -5.5. Stopping the Agents - -the pid of the agent process is in /var/run/agent..pid - -To Stop the agent: - -kill - - \ No newline at end of file diff --git a/INSTALL b/INSTALL deleted file mode 100644 index bcf10e20b23..00000000000 --- a/INSTALL +++ /dev/null @@ -1,155 +0,0 @@ ---------------------------------------------------------------------- -TABLE OF CONTENTS ---------------------------------------------------------------------- - - -1. Really quick start: building and installing a production stack -2. Post-install: setting the CloudStack components up -3. Installation paths: where the stack is installed on your system -4. Uninstalling the CloudStack from your system - - ---------------------------------------------------------------------- -REALLY QUICK START: BUILDING AND INSTALLING A PRODUCTION STACK ---------------------------------------------------------------------- - - -You have two options. Choose one: - -a) Building distribution packages from the source and installing them -b) Building from the source and installing directly from there - - -=== I want to build and install distribution packages === - -This is the recommended way to run your CloudStack cloud. The -advantages are that dependencies are taken care of automatically -for you, and you can verify the integrity of the installed files -using your system's package manager. - -1. As root, install the build dependencies. - - a) Fedora / CentOS: ./waf installrpmdeps - - b) Ubuntu: ./waf installdebdeps - -2. As a non-root user, build the CloudStack packages. - - a) Fedora / CentOS: ./waf rpm - - b) Ubuntu: ./waf deb - -3. As root, install the CloudStack packages. - You can choose which components to install on your system. - - a) Fedora / CentOS: the installable RPMs are in artifacts/rpmbuild - install as root: rpm -ivh artifacts/rpmbuild/RPMS/{x86_64,noarch,i386}/*.rpm - - b) Ubuntu: the installable DEBs are in artifacts/debbuild - install as root: dpkg -i artifacts/debbuild/*.deb - -4. Configure and start the components you intend to run. - Consult the Installation Guide to find out how to - configure each component, and "Installation paths" for information - on where programs, initscripts and config files are installed. - - -=== I want to build and install directly from the source === - -This is the recommended way to run your CloudStack cloud if you -intend to modify the source, if you intend to port the CloudStack to -another distribution, or if you intend to run the CloudStack on a -distribution for which packages are not built. - -1. As root, install the build dependencies. - See below for a list. - -2. As non-root, configure the build. - See below to discover configuration options. - - ./waf configure - -3. As non-root, build the CloudStack. - To learn more, see "Quick guide to developing, building and - installing from source" below. - - ./waf build - -4. As root, install the runtime dependencies. - See below for a list. - -5. As root, Install the CloudStack - - ./waf install - -6. Configure and start the components you intend to run. - Consult the Installation Guide to find out how to - configure each component, and "Installation paths" for information - on where to find programs, initscripts and config files mentioned - in the Installation Guide (paths may vary). - - -=== Dependencies of the CloudStack === - -- Build dependencies: - - 1. FIXME DEPENDENCIES LIST THEM HERE - -- Runtime dependencies: - - 2. FIXME DEPENDENCIES LIST THEM HERE - - ---------------------------------------------------------------------- -INSTALLATION PATHS: WHERE THE STACK IS INSTALLED ON YOUR SYSTEM ---------------------------------------------------------------------- - - -The CloudStack build system installs files on a variety of paths, each -one of which is selectable when building from source. - -- $PREFIX: - the default prefix where the entire stack is installed - defaults to /usr/local on source builds - defaults to /usr on package builds - -- $SYSCONFDIR/cloud: - - the prefix for CloudStack configuration files - defaults to $PREFIX/etc/cloud on source builds - defaults to /etc/cloud on package builds - -- $SYSCONFDIR/init.d: - the prefix for CloudStack initscripts - defaults to $PREFIX/etc/init.d on source builds - defaults to /etc/init.d on package builds - -- $BINDIR: - the CloudStack installs programs there - defaults to $PREFIX/bin on source builds - defaults to /usr/bin on package builds - -- $LIBEXECDIR: - the CloudStack installs service runners there - defaults to $PREFIX/libexec on source builds - defaults to /usr/libexec on package builds (/usr/bin on Ubuntu) - - ---------------------------------------------------------------------- -UNINSTALLING THE CLOUDSTACK FROM YOUR SYSTEM ---------------------------------------------------------------------- - - -- If you installed the CloudStack using packages, use your operating - system package manager to remove the CloudStack packages. - - a) Fedora / CentOS: the installable RPMs are in artifacts/rpmbuild - as root: rpm -qa | grep ^cloud- | xargs rpm -e - - b) Ubuntu: the installable DEBs are in artifacts/debbuild - aptitude purge '~ncloud' - -- If you installed from a source tree: - - ./waf uninstall - diff --git a/README b/README deleted file mode 100644 index b0478ff475f..00000000000 --- a/README +++ /dev/null @@ -1,52 +0,0 @@ -Hello, and thanks for downloading the Cloud.com CloudStack™! The -Cloud.com CloudStack™ is Open Source Software that allows -organizations to build Infrastructure as a Service (Iaas) clouds. -Working with server, storage, and networking equipment of your -choice, the CloudStack provides a turn-key software stack that -dramatically simplifies the process of deploying and managing a -cloud. - - ---------------------------------------------------------------------- -HOW TO INSTALL THE CLOUDSTACK ---------------------------------------------------------------------- - - -Please refer to the document INSTALL distributed with the source. - - ---------------------------------------------------------------------- -HOW TO HACK ON THE CLOUDSTACK ---------------------------------------------------------------------- - - -Please refer to the document HACKING distributed with the source. - - ---------------------------------------------------------------------- -BE PART OF THE CLOUD.COM COMMUNITY! ---------------------------------------------------------------------- - - -We are more than happy to have you ask us questions, hack our source -code, and receive your contributions. - -* Our forums are available at http://cloud.com/community . -* If you would like to modify / extend / hack on the CloudStack source, - refer to the file HACKING for more information. -* If you find bugs, please log on to http://bugs.cloud.com/ and file - a report. -* If you have patches to send us get in touch with us at info@cloud.com - or file them as attachments in our bug tracker above. - - ---------------------------------------------------------------------- -Cloud.com's contact information is: - -20400 Stevens Creek Blvd -Suite 390 -Cupertino, CA 95014 -Tel: +1 (888) 384-0962 - -This software is OSI certified Open Source Software. OSI Certified is a -certification mark of the Open Source Initiative. diff --git a/README.html b/README.html index 2ece7a070e7..8212176103e 100644 --- a/README.html +++ b/README.html @@ -512,6 +512,13 @@ Also see [[AdvancedOptions]]
+
+
|''Type:''|file|
+|''URL:''|http://tiddlyvault.tiddlyspot.com/#%5B%5BDisableWikiLinksPlugin%20(TiddlyTools)%5D%5D|
+|''Workspace:''|(default)|
+
+This tiddler was automatically created to record the details of this server
+
---------------------------------------------------------------------
 FOR ANT USERS
@@ -702,21 +709,18 @@ Once this command is done, the packages will be built in the directory {{{artifa
 # As a non-root user, run the command {{{./waf deb}}} in the source directory.
 Once this command is done, the packages will be built in the directory {{{artifacts/debbuild}}}.
-
-
!Obtain the source for the CloudStack
+

+
You need to do the following steps on each machine that will run a CloudStack component.
+!Obtain the source for the CloudStack
 If you aren't reading this from a local copy of the source code, see [[Obtaining the source]].
-!Prepare your development environment
-See [[Preparing your development environment]].
-!Configure the build on the builder machine
+!Prepare your environment
+See [[Preparing your environment]].
+!Configure the build
 As non-root, run the command {{{./waf configure}}}.  See [[waf configure]] to discover configuration options for that command.
-!Build the CloudStack on the builder machine
+!Build the CloudStack
 As non-root, run the command {{{./waf build}}}.  See [[waf build]] for an explanation.
-!Install the CloudStack on the target systems
-On each machine where you intend to run a CloudStack component:
-# upload the entire source code tree after compilation, //ensuring that the source ends up in the same path as the machine in which you compiled it//,
-## {{{rsync}}} is [[usually very handy|Using rsync to quickly transport the source tree to another machine]] for this
-# in that newly uploaded directory of the target machine, run the command {{{./waf install}}} //as root//.
-Consult [[waf install]] for information on installation.

+!Install the CloudStack
+Run the command {{{./waf install}}} //as root//.  Consult [[waf install]] for information on installation.
!Changing the [[configuration|waf configure]] process
@@ -737,11 +741,91 @@ See the files in the {{{debian/}}} folder.
The Cloud.com CloudStack is an open source software product that enables the deployment, management, and configuration of multi-tier and multi-tenant infrastructure cloud services by enterprises and service providers.
-
-
Not done yet!
+
+
Prior to building the CloudStack, you need to install the following software packages in your system.
+# Sun Java 1.6
+## You must install the Java Development Kit with {{{javac}}}, not just the Java Runtime Environment
+## The commands {{{java}}} and {{{javac}}} must be found in your {{{PATH}}}
+# Apache Tomcat
+## If you are using the official Apache binary distribution, set the environment variable {{{TOMCAT_HOME}}} to point to the Apache Tomcat directory
+# MySQL
+## At the very minimum, you need to have the client and libraries installed
+## If your development machine is also going to be the database server, you need to have the server installed and running as well
+# Python 2.6
+## Ensure that the {{{python}}} command is in your {{{PATH}}}
+## Do ''not'' install Cygwin Python!
+# The MySQLdb module for Python 2.6
+## If you use Windows, you can find a [[pre-built package here|http://soemin.googlecode.com/files/MySQL-python-1.2.3c1.win32-py2.6.exe]]
+# The Bourne-again shell (also known as bash)
+# GNU coreutils
+''Note for Windows users'': Some of the packages in the above list are only available on Windows through Cygwin.  If that is your case, install them using Cygwin and remember to include the Cygwin {{{bin/}}} directory in your PATH.  Under no circumstances install Cygwin Python!  Use the Python for Windows official installer instead.
+!Additional dependencies for Linux development environments
+# GCC (only needed on Linux)
+# glibc-devel / glibc-dev
+# The Java packages (usually available in your distribution):
+## commons-collections
+## commons-dbcp
+## commons-logging
+## commons-logging-api
+## commons-pool
+## commons-httpclient
+## ws-commons-util
+# useradd
+# userdel
-
-
Not done yet!
+
+
The following software / programs must be correctly installed in the machines where you will run a CloudStack component.  This list is by no means complete yet, but it will be soon.
+
+''Note for Windows users'':  Some of the packages in the lists below are only available on Windows through Cygwin.  If that is your case, install them using Cygwin and remember to include the Cygwin {{{bin/}}} directory in your PATH.  Under no circumstances install Cygwin Python!  Use the Python for Windows official installer instead.
+!Run-time dependencies common to all components of the CloudStack
+# bash
+# coreutils
+# Sun Java 1.6
+## You must install the Java Development Kit with {{{javac}}}, not just the Java Runtime Environment
+## The commands {{{java}}} and {{{javac}}} must be found in your {{{PATH}}}
+# Python 2.6
+## Ensure that the {{{python}}} command is in your {{{PATH}}}
+## Do ''not'' install Cygwin Python!
+# The Java packages (usually available in your distribution):
+## commons-collections
+## commons-dbcp
+## commons-logging
+## commons-logging-api
+## commons-pool
+## commons-httpclient
+## ws-commons-util
+!Management Server-specific dependencies
+# Apache Tomcat
+## If you are using the official Apache binary distribution, set the environment variable {{{TOMCAT_HOME}}} to point to the Apache Tomcat directory
+# MySQL
+## At the very minimum, you need to have the client and libraries installed
+## If you will be running the Management Server in the same machine that will run the database server, you need to have the server installed and running as well
+# The MySQLdb module for Python 2.6
+## If you use Windows, you can find a [[pre-built package here|http://soemin.googlecode.com/files/MySQL-python-1.2.3c1.win32-py2.6.exe]] 
+# openssh-clients (provides the ssh-keygen command)
+# mkisofs (provides the genisoimage command)
+
+
+
To support incremental migration from one version to another without having to redeploy the database, the CloudStack supports an incremental schema migration mechanism for the database.
+!!!How does it work?
+When the database is deployed for the first time with [[waf deploydb]] or the command {{{cloud-setup-databases}}}, a row is written to the {{{configuration}}} table, named {{{schema.level}}} and containing the current schema level.  This schema level row comes from the file {{{setup/db/schema-level.sql}}} in the source (refer to the [[Installation paths]] topic to find out where this file is installed in a running system).
+
+This value is used by the database migrator {{{cloud-migrate-databases}}} (source {{{setup/bindir/cloud-migrate-databases.in}}}) to determine the starting schema level.  The database migrator has a series of classes -- each class represents a step in the migration process and is usually tied to the execution of a SQL file stored in {{{setup/db}}}.  To migrate the database, the database migrator:
+# walks the list of steps it knows about,
+# generates a list of steps sorted by the order they should be executed in,
+# executes each step in order
+# at the end of each step, records the new schema level to the database table {{{configuration}}}
+For more information, refer to the database migrator source -- it is documented.
+!!!What impact does this have on me as a developer?
+Whenever you need to evolve the schema of the database:
+# write a migration SQL script and store it in {{{setup/db}}},
+# include your schema changes in the appropriate SQL file {{{create-*.sql}}} too (as the database is expected to be at its latest evolved schema level right after deploying a fresh database)
+# write a class in {{{setup/bindir/cloud-migrate-databases.in}}}, describing the migration step; in detail:
+## the schema level your migration step expects the database to be in,
+## the schema level your migration step will leave your database in (presumably the latest schema level, which you will have to choose!),
+## and the name / description of the step
+# bump the schema level in {{{setup/db/schema-level.sql}}} to the latest schema level
+Otherwise, ''end-user migration will fail catastrophically''.
[[Welcome]]
@@ -749,13 +833,115 @@ See the files in the {{{debian/}}} folder. 
#[[Source layout guide]]
+
+
/***
+|Name|DisableWikiLinksPlugin|
+|Source|http://www.TiddlyTools.com/#DisableWikiLinksPlugin|
+|Version|1.6.0|
+|Author|Eric Shulman|
+|License|http://www.TiddlyTools.com/#LegalStatements|
+|~CoreVersion|2.1|
+|Type|plugin|
+|Description|selectively disable TiddlyWiki's automatic ~WikiWord linking behavior|
+This plugin allows you to disable TiddlyWiki's automatic ~WikiWord linking behavior, so that WikiWords embedded in tiddler content will be rendered as regular text, instead of being automatically converted to tiddler links.  To create a tiddler link when automatic linking is disabled, you must enclose the link text within {{{[[...]]}}}.
+!!!!!Usage
+<<<
+You can block automatic WikiWord linking behavior for any specific tiddler by ''tagging it with<<tag excludeWikiWords>>'' (see configuration below) or, check a plugin option to disable automatic WikiWord links to non-existing tiddler titles, while still linking WikiWords that correspond to existing tiddlers titles or shadow tiddler titles.  You can also block specific selected WikiWords from being automatically linked by listing them in [[DisableWikiLinksList]] (see configuration below), separated by whitespace.  This tiddler is optional and, when present, causes the listed words to always be excluded, even if automatic linking of other WikiWords is being permitted.  
+
+Note: WikiWords contained in default ''shadow'' tiddlers will be automatically linked unless you select an additional checkbox option lets you disable these automatic links as well, though this is not recommended, since it can make it more difficult to access some TiddlyWiki standard default content (such as AdvancedOptions or SideBarTabs)
+<<<
+!!!!!Configuration
+<<<
+<<option chkDisableWikiLinks>> Disable ALL automatic WikiWord tiddler links
+<<option chkAllowLinksFromShadowTiddlers>> ... except for WikiWords //contained in// shadow tiddlers
+<<option chkDisableNonExistingWikiLinks>> Disable automatic WikiWord links for non-existing tiddlers
+Disable automatic WikiWord links for words listed in: <<option txtDisableWikiLinksList>>
+Disable automatic WikiWord links for tiddlers tagged with: <<option txtDisableWikiLinksTag>>
+<<<
+!!!!!Revisions
+<<<
+2008.07.22 [1.6.0] hijack tiddler changed() method to filter disabled wiki words from internal links[] array (so they won't appear in the missing tiddlers list)
+2007.06.09 [1.5.0] added configurable txtDisableWikiLinksTag (default value: "excludeWikiWords") to allows selective disabling of automatic WikiWord links for any tiddler tagged with that value.
+2006.12.31 [1.4.0] in formatter, test for chkDisableNonExistingWikiLinks
+2006.12.09 [1.3.0] in formatter, test for excluded wiki words specified in DisableWikiLinksList
+2006.12.09 [1.2.2] fix logic in autoLinkWikiWords() (was allowing links TO shadow tiddlers, even when chkDisableWikiLinks is TRUE).  
+2006.12.09 [1.2.1] revised logic for handling links in shadow content
+2006.12.08 [1.2.0] added hijack of Tiddler.prototype.autoLinkWikiWords so regular (non-bracketed) WikiWords won't be added to the missing list
+2006.05.24 [1.1.0] added option to NOT bypass automatic wikiword links when displaying default shadow content (default is to auto-link shadow content)
+2006.02.05 [1.0.1] wrapped wikifier hijack in init function to eliminate globals and avoid FireFox 1.5.0.1 crash bug when referencing globals
+2005.12.09 [1.0.0] initial release
+<<<
+!!!!!Code
+***/
+//{{{
+version.extensions.DisableWikiLinksPlugin= {major: 1, minor: 6, revision: 0, date: new Date(2008,7,22)};
+
+if (config.options.chkDisableNonExistingWikiLinks==undefined) config.options.chkDisableNonExistingWikiLinks= false;
+if (config.options.chkDisableWikiLinks==undefined) config.options.chkDisableWikiLinks=false;
+if (config.options.txtDisableWikiLinksList==undefined) config.options.txtDisableWikiLinksList="DisableWikiLinksList";
+if (config.options.chkAllowLinksFromShadowTiddlers==undefined) config.options.chkAllowLinksFromShadowTiddlers=true;
+if (config.options.txtDisableWikiLinksTag==undefined) config.options.txtDisableWikiLinksTag="excludeWikiWords";
+
+// find the formatter for wikiLink and replace handler with 'pass-thru' rendering
+initDisableWikiLinksFormatter();
+function initDisableWikiLinksFormatter() {
+	for (var i=0; i<config.formatters.length && config.formatters[i].name!="wikiLink"; i++);
+	config.formatters[i].coreHandler=config.formatters[i].handler;
+	config.formatters[i].handler=function(w) {
+		// supress any leading "~" (if present)
+		var skip=(w.matchText.substr(0,1)==config.textPrimitives.unWikiLink)?1:0;
+		var title=w.matchText.substr(skip);
+		var exists=store.tiddlerExists(title);
+		var inShadow=w.tiddler && store.isShadowTiddler(w.tiddler.title);
+		// check for excluded Tiddler
+		if (w.tiddler && w.tiddler.isTagged(config.options.txtDisableWikiLinksTag))
+			{ w.outputText(w.output,w.matchStart+skip,w.nextMatch); return; }
+		// check for specific excluded wiki words
+		var t=store.getTiddlerText(config.options.txtDisableWikiLinksList);
+		if (t && t.length && t.indexOf(w.matchText)!=-1)
+			{ w.outputText(w.output,w.matchStart+skip,w.nextMatch); return; }
+		// if not disabling links from shadows (default setting)
+		if (config.options.chkAllowLinksFromShadowTiddlers && inShadow)
+			return this.coreHandler(w);
+		// check for non-existing non-shadow tiddler
+		if (config.options.chkDisableNonExistingWikiLinks && !exists)
+			{ w.outputText(w.output,w.matchStart+skip,w.nextMatch); return; }
+		// if not enabled, just do standard WikiWord link formatting
+		if (!config.options.chkDisableWikiLinks)
+			return this.coreHandler(w);
+		// just return text without linking
+		w.outputText(w.output,w.matchStart+skip,w.nextMatch)
+	}
+}
+
+Tiddler.prototype.coreAutoLinkWikiWords = Tiddler.prototype.autoLinkWikiWords;
+Tiddler.prototype.autoLinkWikiWords = function()
+{
+	// if all automatic links are not disabled, just return results from core function
+	if (!config.options.chkDisableWikiLinks)
+		return this.coreAutoLinkWikiWords.apply(this,arguments);
+	return false;
+}
+
+Tiddler.prototype.disableWikiLinks_changed = Tiddler.prototype.changed;
+Tiddler.prototype.changed = function()
+{
+	this.disableWikiLinks_changed.apply(this,arguments);
+	// remove excluded wiki words from links array
+	var t=store.getTiddlerText(config.options.txtDisableWikiLinksList,"").readBracketedList();
+	if (t.length) for (var i=0; i<t.length; i++)
+		if (this.links.contains(t[i]))
+			this.links.splice(this.links.indexOf(t[i]),1);
+};
+//}}}
+
Not done yet!
-
+
Start here if you want to learn the essentials to extend, modify and enhance the CloudStack.  This assumes that you've already familiarized yourself with CloudStack concepts, installation and configuration using the [[Getting started|Welcome]] instructions.
 * [[Obtain the source|Obtaining the source]]
-* [[Prepare your environment|Preparing your development environment]]
+* [[Prepare your environment|Preparing your environment]]
 * [[Get acquainted with the development lifecycle|Your development lifecycle]]
 * [[Familiarize yourself with our development conventions|Development conventions]]
 Extra developer information:
@@ -764,6 +950,7 @@ Extra developer information:
 * [[How to integrate with Eclipse]]
 * [[Starting over]]
 * [[Making a source release|waf dist]]
+* [[How to write database migration scripts|Database migration infrastructure]]
@@ -785,13 +972,13 @@ Any ant target added to the ant project files will automatically be detected -- The reason we do this rather than use the native waf capabilities for building Java projects is simple: by using ant, we can leverage the support built-in for ant in [[Eclipse|How to integrate with Eclipse]] and many other """IDEs""". Another reason to do this is because Java developers are familiar with ant, so adding a new JAR file or modifying what gets built into the existing JAR files is facilitated for Java developers.
-
+
The CloudStack build system installs files on a variety of paths, each
 one of which is selectable when building from source.
 * {{{$PREFIX}}}:
 ** the default prefix where the entire stack is installed
-** defaults to /usr/local on source builds
-** defaults to /usr on package builds
+** defaults to {{{/usr/local}}} on source builds as root, {{{$HOME/cloudstack}}} on source builds as a regular user, {{{C:\CloudStack}}} on Windows builds
+** defaults to {{{/usr}}} on package builds
 * {{{$SYSCONFDIR/cloud}}}:
 ** the prefix for CloudStack configuration files
 ** defaults to $PREFIX/etc/cloud on source builds
@@ -901,16 +1088,17 @@ This will create a folder called {{{cloudstack-oss}}} in your current folder.
 !Browsing the source code online
 You can browse the CloudStack source code through [[our CGit Web interface|http://git.cloud.com/cloudstack-oss]].
-
-
!Install the build dependencies on the machine where you will compile the CloudStack
-!!Fedora / CentOS
-The command [[waf installrpmdeps]] issued from the source tree gets it done.
-!!Ubuntu
-The command [[waf installdebdeps]] issues from the source tree gets it done.
-!!Other distributions
-See [[CloudStack build dependencies]]
-!Install the run-time dependencies on the machines where you will run the CloudStack
-See [[CloudStack run-time dependencies]].
+
+
!Install the build dependencies
+* If you want to compile the CloudStack on Linux:
+** Fedora / CentOS: The command [[waf installrpmdeps]] issued from the source tree gets it done.
+** Ubuntu: The command [[waf installdebdeps]] issues from the source tree gets it done.
+** Other distributions: Manually install the packages listed in [[CloudStack build dependencies]].
+* If you want to compile the CloudStack on Windows or Mac:
+** Manually install the packages listed in [[CloudStack build dependencies]].
+** Note that you won't be able to deploy this compiled CloudStack onto Linux machines -- you will be limited to running the Management Server.
+!Install the run-time dependencies
+In addition to the build dependencies, a number of software packages need to be installed on the machine to be able to run certain components of the CloudStack.  These packages are not strictly required to //build// the stack, but they are required to run at least one part of it.  See the topic [[CloudStack run-time dependencies]] for the list of packages.
Every time you run {{{./waf install}}} to deploy changed code, waf will install configuration files once again.  This can be a nuisance if you are developing the stack.
@@ -1149,9 +1337,9 @@ Cloud.com's contact information is:
 !Legal information
 //Unless otherwise specified// by Cloud.com, Inc., or in the sources themselves, [[this software is OSI certified Open Source Software distributed under the GNU General Public License, version 3|License statement]].  OSI Certified is a certification mark of the Open Source Initiative.  The software powering this documentation is """BSD-licensed""" and obtained from [[TiddlyWiki.com|http://tiddlywiki.com/]].
-
-
This is the typical lifecycle that you would follow when hacking on a CloudStack component, assuming that your [[development environment has been set up|Preparing your development environment]]:
-# [[Configure|waf configure]] the source code<br>{{{./waf configure --prefix=/home/youruser/cloudstack}}}
+

+
This is the typical lifecycle that you would follow when hacking on a CloudStack component, assuming that your [[development environment has been set up|Preparing your environment]]:
+# [[Configure|waf configure]] the source code<br>{{{./waf configure}}}
 # [[Build|waf build]] and [[install|waf install]] the CloudStack
 ## {{{./waf install}}}
 ## [[How to perform these tasks from Eclipse|How to integrate with Eclipse]]
@@ -1229,7 +1417,7 @@ Makes an inventory of all build products in {{{artifacts/default}}}, and removes
 
 Contrast to [[waf distclean]].

 

-

+

 
{{{
 ./waf configure --prefix=/directory/that/you/have/write/permission/to
 }}}
@@ -1238,7 +1426,7 @@ This runs the file {{{wscript_configure}}}, which takes care of setting the  var
 !When / why should I run this?
 You run this command //once//, in preparation to building the stack, or every time you need to change a configure-time variable.  Once you find an acceptable set of configure-time variables, you should not need to run {{{configure}}} again.
 !What happens if I don't run it?
-For convenience reasons, if you forget to configure the source, waf will autoconfigure itself and select some sensible default configuration options.  By default, {{{PREFIX}}} is {{{/usr/local}}}, but you can set it e.g. to  {{{/home/youruser/cloudstack}}} if you plan to do a non-root install.  Be ware that you can later install the stack as a regular user, but most components need to //run// as root.
+For convenience reasons, if you forget to configure the source, waf will autoconfigure itself and select some sensible default configuration options.  By default, {{{PREFIX}}} is {{{/usr/local}}} if you configure as root (do this if you plan to do a non-root install), or {{{/home/youruser/cloudstack}}} if you configure as your regular user name.  Be ware that you can later install the stack as a regular user, but most components need to //run// as root.
 !What variables / options exist for configure?
 In general: refer to the output of {{{./waf configure --help}}}.
 
diff --git a/agent/bindir/cloud-setup-agent.in b/agent/bindir/cloud-setup-agent.in
index c9aeea6d146..26bf1a0f0ac 100755
--- a/agent/bindir/cloud-setup-agent.in
+++ b/agent/bindir/cloud-setup-agent.in
@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
-import sys, os, subprocess, errno, re, traceback
+import sys, os, subprocess, errno, re, traceback, getopt
 
 # ---- This snippet of code adds the sources path and the waf configured PYTHONDIR to the Python path ----
 # ---- We do this so cloud_utils can be looked up in the following order:
@@ -37,13 +37,44 @@ backupdir = "@SHAREDSTATEDIR@/@AGENTPATH@/etcbackup"
 
 #=================== the magic happens here ====================
 
-stderr("Welcome to the Cloud Agent setup")
-stderr("")
 
 try:
+# parse cmd line	
+	opts, args = getopt.getopt(sys.argv[1:], "a", ["host=", "zone=", "pod=", "cluster=", "no-kvm", "guid="])
+	host=None
+	zone=None
+	pod=None
+  	cluster=None
+	guid=None
+    	autoMode=False
+	do_check_kvm = True	
+	for opt, arg in opts:
+		if opt == "--host":
+			if arg != "":
+				host = arg
+		elif opt == "--zone":
+			if arg != "":
+				zone = arg
+		elif opt == "--pod":
+			if arg != "":
+				pod = arg
+		elif opt == "--cluster":
+			if arg != "":
+				cluster = arg
+		elif opt == "--guid":
+			if arg != "":
+				guid = arg
+		elif opt == "--no-kvm":
+			do_check_kvm = False
+        	elif opt == "-a":
+            		autoMode=True
+		
+	if autoMode:
+		cloud_utils.setLogFile("/var/log/cloud/setupAgent.log")
 	
+	stderr("Welcome to the Cloud Agent setup")
+	stderr("")
 	# pre-flight checks for things that the administrator must fix
-	do_check_kvm = not ( "--no-kvm" in sys.argv[1:] )
 	try:
 		for f,n in cloud_utils.preflight_checks(
 				do_check_kvm=do_check_kvm
@@ -59,6 +90,8 @@ try:
 	
 	try:
 		tasks = cloud_utils.config_tasks(brname)
+        	for t in tasks:
+            		t.setAutoMode(autoMode)
 		if all( [ t.done() for t in tasks ] ):
 			
 			stderr("All configuration tasks have been performed already")
@@ -83,7 +116,7 @@ try:
 		stderr(str(e))
 		bail(cloud_utils.E_SETUPFAILED,"Cloud Agent setup failed")
 	
-	setup_agent_config(configfile)
+	setup_agent_config(configfile, host, zone, pod, cluster, guid)
 	stderr("Enabling and starting the Cloud Agent")
 	stop_service(servicename)
 	enable_service(servicename)
diff --git a/agent/src/com/cloud/agent/resource/DummyResource.java b/agent/src/com/cloud/agent/resource/DummyResource.java
index c269c64284d..9ff60e1220d 100755
--- a/agent/src/com/cloud/agent/resource/DummyResource.java
+++ b/agent/src/com/cloud/agent/resource/DummyResource.java
@@ -32,6 +32,7 @@ import com.cloud.resource.ServerResource;
 
 @Local(value={ServerResource.class})
 public class DummyResource implements ServerResource {
+	private boolean _isRemoteAgent = false;
     String _name;
     Host.Type _type;
     boolean _negative;
@@ -101,4 +102,12 @@ public class DummyResource implements ServerResource {
     public void setAgentControl(IAgentControl agentControl) {
     	_agentControl = agentControl;
     }
+    
+    public boolean IsRemoteAgent() {
+    	return _isRemoteAgent;
+    }
+    
+    public void setRemoteAgent(boolean remote) {
+    	_isRemoteAgent = remote;
+    }
 }
diff --git a/agent/src/com/cloud/agent/resource/computing/LibvirtComputingResource.java b/agent/src/com/cloud/agent/resource/computing/LibvirtComputingResource.java
index 915b9ee2c26..7f9d79c65bc 100644
--- a/agent/src/com/cloud/agent/resource/computing/LibvirtComputingResource.java
+++ b/agent/src/com/cloud/agent/resource/computing/LibvirtComputingResource.java
@@ -134,9 +134,9 @@ import com.cloud.agent.api.storage.CreateAnswer;
 import com.cloud.agent.api.storage.CreateCommand;
 import com.cloud.agent.api.storage.CreatePrivateTemplateAnswer;
 import com.cloud.agent.api.storage.CreatePrivateTemplateCommand;
+import com.cloud.agent.api.storage.DestroyCommand;
 import com.cloud.agent.api.storage.DownloadAnswer;
 import com.cloud.agent.api.storage.PrimaryStorageDownloadCommand;
-import com.cloud.agent.api.to.DiskCharacteristicsTO;
 import com.cloud.agent.api.to.StoragePoolTO;
 import com.cloud.agent.api.to.VolumeTO;
 import com.cloud.agent.resource.computing.LibvirtStoragePoolDef.poolType;
@@ -160,13 +160,14 @@ import com.cloud.hypervisor.Hypervisor;
 import com.cloud.network.NetworkEnums.RouterPrivateIpStrategy;
 import com.cloud.resource.ServerResource;
 import com.cloud.resource.ServerResourceBase;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.StorageResourceType;
 import com.cloud.storage.StorageLayer;
 import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.Volume;
 import com.cloud.storage.VolumeVO;
 import com.cloud.storage.Storage.ImageFormat;
 import com.cloud.storage.Storage.StoragePoolType;
-import com.cloud.storage.Volume.StorageResourceType;
 import com.cloud.storage.Volume.VolumeType;
 import com.cloud.storage.template.Processor;
 import com.cloud.storage.template.QCOW2Processor;
@@ -182,6 +183,7 @@ import com.cloud.utils.net.NetUtils;
 import com.cloud.utils.script.OutputInterpreter;
 import com.cloud.utils.script.Script;
 import com.cloud.vm.ConsoleProxyVO;
+import com.cloud.vm.DiskProfile;
 import com.cloud.vm.DomainRouter;
 import com.cloud.vm.State;
 import com.cloud.vm.VirtualMachineName;
@@ -220,6 +222,7 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
     private String _host;
     private String _dcId;
     private String _pod;
+    private String _clusterId;
     private long _hvVersion;
     private final String _SSHKEYSPATH = "/root/.ssh";
     private final String _SSHPRVKEYPATH = _SSHKEYSPATH + File.separator + "id_rsa.cloud";
@@ -559,7 +562,9 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
         if (_pod == null) {
         	_pod = "default";
         }
-		
+        
+        _clusterId = (String) params.get("cluster");
+		    
         _createvnetPath = Script.findScript(networkScriptsDir, "createvnet.sh");
         if(_createvnetPath == null) {
             throw new ConfigurationException("Unable to find createvnet.sh");
@@ -1114,6 +1119,8 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
                 return execute((MaintainCommand) cmd);
             } else if (cmd instanceof CreateCommand) {
                 return execute((CreateCommand) cmd);
+            } else if (cmd instanceof DestroyCommand) {
+                return execute((DestroyCommand) cmd);
             } else if (cmd instanceof PrimaryStorageDownloadCommand) {
                 return execute((PrimaryStorageDownloadCommand) cmd);
             } else if (cmd instanceof CreatePrivateTemplateCommand) {
@@ -1161,13 +1168,13 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
 		}
 	}
 	
-	protected StorageResourceType getStorageResourceType() {
-		return StorageResourceType.STORAGE_POOL;
+	protected Storage.StorageResourceType getStorageResourceType() {
+		return Storage.StorageResourceType.STORAGE_POOL;
 	}
 
     protected Answer execute(CreateCommand cmd) {
     	  StoragePoolTO pool = cmd.getPool();
-          DiskCharacteristicsTO dskch = cmd.getDiskCharacteristics();
+          DiskProfile dskch = cmd.getDiskCharacteristics();
           StorageVol tmplVol = null;
           StoragePool primaryPool = null;
           StorageVol vol = null;
@@ -1187,10 +1194,9 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
         			  s_logger.debug(result);
         			  return new CreateAnswer(cmd, result);
         		  }
+        		  
+        		  vol = createVolume(primaryPool, tmplVol);
 
-        		  LibvirtStorageVolumeDef volDef = new LibvirtStorageVolumeDef(UUID.randomUUID().toString(), tmplVol.getInfo().capacity, volFormat.QCOW2, tmplVol.getPath(), volFormat.QCOW2);
-        		  s_logger.debug(volDef.toString());
-        		  vol = primaryPool.storageVolCreateXML(volDef.toString(), 0);
         		  if (vol == null) {
         			  return new Answer(cmd, false, " Can't create storage volume on storage pool");
         		  }
@@ -1224,24 +1230,68 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
         	  }
           }
     }
+    
+    public Answer execute(DestroyCommand cmd) {
+    	 VolumeTO vol = cmd.getVolume();
+    	
+    	 try {
+    		 StorageVol volume = getVolume(vol.getPath());
+        	 if (volume == null) {
+        		 s_logger.debug("Failed to find the volume: " + vol.getPath());
+        		 return new Answer(cmd, true, "Success");
+        	 }
+    		 volume.delete(0);
+    		 volume.free();
+    	 } catch (LibvirtException e) {
+    		 s_logger.debug("Failed to delete volume: " + e.toString());
+    		 return new Answer(cmd, false, e.toString());
+    	 }
+    	 return new Answer(cmd, true, "Success");
+    }
+    
     protected ManageSnapshotAnswer execute(final ManageSnapshotCommand cmd) {
     	String snapshotName = cmd.getSnapshotName();
     	String VolPath = cmd.getVolumePath();
+    	String snapshotPath = cmd.getSnapshotPath();
+    	String vmName = cmd.getVmName();
     	try {
-    		StorageVol vol = getVolume(VolPath);
-    		if (vol == null) {
-    			return new ManageSnapshotAnswer(cmd, false, null);
+    		DomainInfo.DomainState state = null;
+    		Domain vm = null;
+    		if (vmName != null) {
+    			try {
+    				vm = getDomain(cmd.getVmName());
+    				state = vm.getInfo().state;
+    			} catch (LibvirtException e) {
+
+    			}
     		}
-    		Domain vm = getDomain(cmd.getVmName());
-    		String vmUuid = vm.getUUIDString();
-    		Object[] args = new Object[] {snapshotName, vmUuid};
-    		String snapshot = SnapshotXML.format(args);
-    		s_logger.debug(snapshot);
-    		if (cmd.getCommandSwitch().equalsIgnoreCase(ManageSnapshotCommand.CREATE_SNAPSHOT)) {
-    			vm.snapshotCreateXML(snapshot);
+    		
+    		if (state == DomainInfo.DomainState.VIR_DOMAIN_RUNNING) {
+    			String vmUuid = vm.getUUIDString();
+    			Object[] args = new Object[] {snapshotName, vmUuid};
+    			String snapshot = SnapshotXML.format(args);
+    			s_logger.debug(snapshot);
+    			if (cmd.getCommandSwitch().equalsIgnoreCase(ManageSnapshotCommand.CREATE_SNAPSHOT)) {
+    				vm.snapshotCreateXML(snapshot);
+    			} else {
+    				DomainSnapshot snap = vm.snapshotLookupByName(snapshotName);
+    				snap.delete(0);
+    			}
     		} else {
-    			DomainSnapshot snap = vm.snapshotLookupByName(snapshotName);
-    			snap.delete(0);
+    			/*VM is not running, create a snapshot by ourself*/
+    			final Script command = new Script(_manageSnapshotPath, _timeout, s_logger);
+    			if (cmd.getCommandSwitch().equalsIgnoreCase(ManageSnapshotCommand.CREATE_SNAPSHOT)) {
+    				command.add("-c", VolPath);
+    			} else {
+    				command.add("-d", snapshotPath);
+    			}
+    			
+    			command.add("-n", snapshotName);
+    			String result = command.execute();
+    			if (result != null) {
+    				s_logger.debug("Failed to manage snapshot: " + result);
+    	    		return new ManageSnapshotAnswer(cmd, false, "Failed to manage snapshot: " + result);
+    			}
     		}
     	} catch (LibvirtException e) {
     		s_logger.debug("Failed to manage snapshot: " + e.toString());
@@ -1258,28 +1308,52 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
          String snapshotName = cmd.getSnapshotName();
          String snapshotPath = cmd.getSnapshotUuid();
          String snapshotDestPath = null;
+         String vmName = cmd.getVmName();
 
          try {
 			StoragePool secondaryStoragePool = getNfsSPbyURI(_conn, new URI(secondaryStoragePoolURL));
 			String ssPmountPath = _mountPoint + File.separator + secondaryStoragePool.getUUIDString();
-			snapshotDestPath = ssPmountPath + File.separator + dcId + File.separator + "snapshots" + File.separator + accountId + File.separator + volumeId; 
-			final Script command = new Script(_manageSnapshotPath, _timeout, s_logger);
+			snapshotDestPath = ssPmountPath + File.separator + "snapshots" + File.separator +  dcId + File.separator + accountId + File.separator + volumeId; 
+			Script command = new Script(_manageSnapshotPath, _timeout, s_logger);
 			command.add("-b", snapshotPath);
 			command.add("-n", snapshotName);
 			command.add("-p", snapshotDestPath);
+			command.add("-t", snapshotName);
 			String result = command.execute();
 			if (result != null) {
 				s_logger.debug("Failed to backup snaptshot: " + result);
 				return new BackupSnapshotAnswer(cmd, false, result, null);
 			}
 			/*Delete the snapshot on primary*/
-			Domain vm = getDomain(cmd.getVmName());
-    		String vmUuid = vm.getUUIDString();
-    		Object[] args = new Object[] {snapshotName, vmUuid};
-    		String snapshot = SnapshotXML.format(args);
-    		s_logger.debug(snapshot);
-    		DomainSnapshot snap = vm.snapshotLookupByName(snapshotName);
-    		snap.delete(0);
+			
+			DomainInfo.DomainState state = null;
+			Domain vm = null;
+			if (vmName != null) {
+				try {
+					vm = getDomain(cmd.getVmName());
+					state = vm.getInfo().state;
+				} catch (LibvirtException e) {
+					
+				}
+			}
+			
+			if (state == DomainInfo.DomainState.VIR_DOMAIN_RUNNING) {
+				String vmUuid = vm.getUUIDString();
+				Object[] args = new Object[] {snapshotName, vmUuid};
+				String snapshot = SnapshotXML.format(args);
+				s_logger.debug(snapshot);
+				DomainSnapshot snap = vm.snapshotLookupByName(snapshotName);
+				snap.delete(0);
+			} else {
+				command = new Script(_manageSnapshotPath, _timeout, s_logger);   			
+    			command.add("-d", snapshotPath);  			
+    			command.add("-n", snapshotName);
+    			result = command.execute();
+    			if (result != null) {
+    				s_logger.debug("Failed to backup snapshot: " + result);
+    	    		return new BackupSnapshotAnswer(cmd, false, "Failed to backup snapshot: " + result, null);
+    			}
+			}
 		} catch (LibvirtException e) {
 			return new BackupSnapshotAnswer(cmd, false, e.toString(), null);
 		} catch (URISyntaxException e) {
@@ -1295,7 +1369,7 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
     	try {
     		StoragePool secondaryStoragePool = getNfsSPbyURI(_conn, new URI(cmd.getSecondaryStoragePoolURL()));
 			String ssPmountPath = _mountPoint + File.separator + secondaryStoragePool.getUUIDString();
-			String snapshotDestPath = ssPmountPath + File.separator + dcId + File.separator + "snapshots" + File.separator + accountId + File.separator + volumeId;
+			String snapshotDestPath = ssPmountPath + File.separator + "snapshots"  + File.separator + dcId + File.separator + accountId + File.separator + volumeId;
 			
 			final Script command = new Script(_manageSnapshotPath, _timeout, s_logger);
 			command.add("-d", snapshotDestPath);
@@ -1317,11 +1391,12 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
     	try {
     		StoragePool secondaryStoragePool = getNfsSPbyURI(_conn, new URI(cmd.getSecondaryStoragePoolURL()));
 			String ssPmountPath = _mountPoint + File.separator + secondaryStoragePool.getUUIDString();
-			String snapshotDestPath = ssPmountPath + File.separator + dcId + File.separator + "snapshots" + File.separator + accountId + File.separator + volumeId;
+			String snapshotDestPath = ssPmountPath + File.separator + "snapshots" + File.separator +  dcId + File.separator + accountId + File.separator + volumeId;
 			
 			final Script command = new Script(_manageSnapshotPath, _timeout, s_logger);
 			command.add("-d", snapshotDestPath);
 			command.add("-n", cmd.getSnapshotName());
+			command.add("-f");
 			command.execute();
     	} catch (LibvirtException e) {
     		return new Answer(cmd, false, e.toString());
@@ -1355,7 +1430,9 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
     	 try {
     		 secondaryPool = getNfsSPbyURI(_conn, new URI(cmd.getSecondaryStoragePoolURL()));
     		 /*TODO: assuming all the storage pools mounted under _mountPoint, the mount point should be got from pool.dumpxml*/
-    		 String templatePath = _mountPoint + File.separator + secondaryPool.getUUIDString() + File.separator + templateInstallFolder;	 	    
+    		 String templatePath = _mountPoint + File.separator + secondaryPool.getUUIDString() + File.separator + templateInstallFolder;	 
+    		 _storage.mkdirs(templatePath);
+    		 
     		 String tmplPath = templateInstallFolder + File.separator + tmplFileName;
     		 Script command = new Script(_createTmplPath, _timeout, s_logger);
     		 command.add("-t", templatePath);
@@ -1402,38 +1479,55 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
     }
     protected CreatePrivateTemplateAnswer execute(CreatePrivateTemplateCommand cmd) {
     	 String secondaryStorageURL = cmd.getSecondaryStorageURL();
-         String snapshotUUID = cmd.getSnapshotPath();
 
          StoragePool secondaryStorage = null;
-         StoragePool privateTemplStorage = null;
-         StorageVol privateTemplateVol = null;
-         StorageVol snapshotVol = null;
          try {
         	 String templateFolder = cmd.getAccountId() + File.separator + cmd.getTemplateId() + File.separator;
         	 String templateInstallFolder = "/template/tmpl/" + templateFolder;
-        	  
+
         	 secondaryStorage = getNfsSPbyURI(_conn, new URI(secondaryStorageURL));
         	 /*TODO: assuming all the storage pools mounted under _mountPoint, the mount point should be got from pool.dumpxml*/
-        	 String mountPath = _mountPoint + File.separator + secondaryStorage.getUUIDString() + templateInstallFolder;
-        	 File mpfile = new File(mountPath);
-        	 if (!mpfile.exists()) {
-        		 mpfile.mkdir();
+        	 String tmpltPath = _mountPoint + File.separator + secondaryStorage.getUUIDString() + templateInstallFolder;
+        	 _storage.mkdirs(tmpltPath);
+
+        	 Script command = new Script(_createTmplPath, _timeout, s_logger);
+        	 command.add("-f", cmd.getSnapshotPath());
+        	 command.add("-c", cmd.getSnapshotName());
+        	 command.add("-t", tmpltPath);
+        	 command.add("-n", cmd.getUniqueName() + ".qcow2");
+        	 command.add("-s");
+        	 String result = command.execute();
+        	 
+        	 if (result != null) {
+        		 s_logger.debug("failed to create template: " + result);
+        		 return new CreatePrivateTemplateAnswer(cmd,
+            			 false,
+            			 result,
+            			 null,
+            			 0,
+            			 null,
+            			 null);
         	 }
-        	 
-        	 // Create a SR for the secondary storage installation folder
-        	 privateTemplStorage = getNfsSPbyURI(_conn, new URI(secondaryStorageURL + templateInstallFolder));
-        	 snapshotVol = getVolume(snapshotUUID);
-        	 
-        	 LibvirtStorageVolumeDef vol = new LibvirtStorageVolumeDef(UUID.randomUUID().toString(), snapshotVol.getInfo().capacity, volFormat.QCOW2, null, null);
-        	 s_logger.debug(vol.toString());
-        	 privateTemplateVol = copyVolume(privateTemplStorage, vol, snapshotVol);
+
+        	 Map params = new HashMap();
+        	 params.put(StorageLayer.InstanceConfigKey, _storage);
+        	 Processor qcow2Processor = new QCOW2Processor();
+
+        	 qcow2Processor.configure("QCOW2 Processor", params);
+
+        	 FormatInfo info = qcow2Processor.process(tmpltPath, null, cmd.getUniqueName());
+
+        	 TemplateLocation loc = new TemplateLocation(_storage, tmpltPath);
+        	 loc.create(1, true, cmd.getUniqueName());
+        	 loc.addFormat(info);
+        	 loc.save();
 
         	 return new CreatePrivateTemplateAnswer(cmd,
         			 true,
         			 null,
-        			 templateInstallFolder + privateTemplateVol.getName(),
-        			 privateTemplateVol.getInfo().capacity/1024*1024, /*in Mega unit*/
-        			 privateTemplateVol.getName(),
+        			 templateInstallFolder + cmd.getUniqueName() + ".qcow2",
+        			 info.virtualSize,
+        			 cmd.getUniqueName(),
         			 ImageFormat.QCOW2);
          } catch (URISyntaxException e) {
         	 return new CreatePrivateTemplateAnswer(cmd,
@@ -1452,7 +1546,31 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
         			 0,
         			 null,
         			 null);
-         }
+         } catch (InternalErrorException e) {
+        	 return new CreatePrivateTemplateAnswer(cmd,
+        			 false,
+        			 e.toString(),
+        			 null,
+        			 0,
+        			 null,
+        			 null);
+		} catch (IOException e) {
+			return new CreatePrivateTemplateAnswer(cmd,
+       			 false,
+       			 e.toString(),
+       			 null,
+       			 0,
+       			 null,
+       			 null);
+		} catch (ConfigurationException e) {
+			return new CreatePrivateTemplateAnswer(cmd,
+	       			 false,
+	       			 e.toString(),
+	       			 null,
+	       			 0,
+	       			 null,
+	       			 null);
+		}
     }
     
     private StoragePool getNfsSPbyURI(Connect conn, URI uri) throws LibvirtException {
@@ -1469,10 +1587,7 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
           
           if (sp == null) {
         	  try {
-        		  File tpFile = new File(targetPath);
-        		  if (!tpFile.exists()) {
-        			  tpFile.mkdir();
-        		  }
+        		  _storage.mkdir(targetPath);
         		  LibvirtStoragePoolDef spd = new LibvirtStoragePoolDef(poolType.NFS, uuid, uuid,
         				  sourceHost, sourcePath, targetPath);
         		  s_logger.debug(spd.toString());
@@ -1582,10 +1697,7 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
     	String targetPath = _mountPoint + File.separator + pool.getUuid();
     	LibvirtStoragePoolDef spd = new LibvirtStoragePoolDef(poolType.NFS, pool.getUuid(), pool.getUuid(),
     														  pool.getHostAddress(), pool.getPath(), targetPath);
-    	File tpFile = new File(targetPath);
-		  if (!tpFile.exists()) {
-			  tpFile.mkdir();
-		  }
+    	_storage.mkdir(targetPath);
     	StoragePool sp = null;
     	try {
     		s_logger.debug(spd.toString());
@@ -2359,7 +2471,7 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
 			Iterator> itr = entrySet.iterator();
 			while (itr.hasNext()) {
 				Map.Entry entry = itr.next();
-				if (entry.getValue().equalsIgnoreCase(sourceFile)) {
+				if ((entry.getValue() != null) && (entry.getValue().equalsIgnoreCase(sourceFile))) {
 					diskDev = entry.getKey();
 					break;
 				}
@@ -2449,6 +2561,7 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
         fillNetworkInformation(cmd);
         cmd.getHostDetails().putAll(getVersionStrings());
         cmd.setPool(_pool);
+        cmd.setCluster(_clusterId);
 
         return new StartupCommand[]{cmd};
 	}
@@ -2940,9 +3053,9 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
     }
     
     private String getHypervisorPath() {
-    	File f =new File("/usr/bin/cloud-qemu-kvm");
+    	File f =new File("/usr/bin/cloud-qemu-system-x86_64");
     	if (f.exists()) {
-    		return "/usr/bin/cloud-qemu-kvm";
+    		return "/usr/bin/cloud-qemu-system-x86_64";
     	} else {
     		if (_conn == null)
     			return null;
@@ -3096,7 +3209,7 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
     		brName = setVnetBrName(vnetId);
     		String vnetDev = "vtap" + vnetId;
     		createVnet(vnetId, _pifs.first());
-    		vnetNic.defBridgeNet(brName, vnetDev, guestMac, interfaceDef.nicModel.VIRTIO);
+    		vnetNic.defBridgeNet(brName, null, guestMac, interfaceDef.nicModel.VIRTIO);
     	}
     	nics.add(vnetNic);    	
     	
@@ -3112,7 +3225,7 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
     		brName = setVnetBrName(vnetId);
     		String vnetDev = "vtap" + vnetId;
     		createVnet(vnetId, _pifs.second());
-    		pubNic.defBridgeNet(brName, vnetDev, pubMac, interfaceDef.nicModel.VIRTIO); 		
+    		pubNic.defBridgeNet(brName, null, pubMac, interfaceDef.nicModel.VIRTIO); 		
     	}
 		nics.add(pubNic);
 		return nics;
@@ -3164,7 +3277,7 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
          String datadiskPath = tmplVol.getKey();
 	
 		diskDef hda = new diskDef();
-		hda.defFileBasedDisk(rootkPath, "vda", diskDef.diskBus.IDE,  diskDef.diskFmtType.QCOW2);
+		hda.defFileBasedDisk(rootkPath, "hda", diskDef.diskBus.IDE,  diskDef.diskFmtType.QCOW2);
 		disks.add(hda);
 		
 		diskDef hdb = new diskDef();
@@ -3246,7 +3359,7 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
     	 
     	 File logPath = new File("/var/run/cloud");
     	 if (!logPath.exists()) {
-    		 logPath.mkdir();
+    		 logPath.mkdirs();
     	 }
     	 
     	 cleanup_rules_for_dead_vms();
@@ -3500,6 +3613,22 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
     	}
     }
     
+    private StorageVol createVolume(StoragePool destPool, StorageVol tmplVol) throws LibvirtException {
+    	if (isCentosHost()) {
+    		LibvirtStorageVolumeDef volDef = new LibvirtStorageVolumeDef(UUID.randomUUID().toString(), tmplVol.getInfo().capacity, volFormat.QCOW2, null, null);
+    		s_logger.debug(volDef.toString());
+    		StorageVol vol = destPool.storageVolCreateXML(volDef.toString(), 0);
+    		
+    		/*create qcow2 image based on the name*/
+    		Script.runSimpleBashScript("qemu-img create -f qcow2 -b  " + tmplVol.getPath() + " " + vol.getPath() );
+    		return vol;
+    	} else {
+    		LibvirtStorageVolumeDef volDef = new LibvirtStorageVolumeDef(UUID.randomUUID().toString(), tmplVol.getInfo().capacity, volFormat.QCOW2, tmplVol.getPath(), volFormat.QCOW2);
+    		s_logger.debug(volDef.toString());
+    		return destPool.storageVolCreateXML(volDef.toString(), 0);
+    	}
+    }
+    
     private StorageVol getVolume(StoragePool pool, String volKey) {
     	StorageVol vol = null;
     	try {
diff --git a/agent/src/com/cloud/agent/resource/computing/LibvirtDomainXMLParser.java b/agent/src/com/cloud/agent/resource/computing/LibvirtDomainXMLParser.java
index 779962af0a4..b5271c8e62c 100644
--- a/agent/src/com/cloud/agent/resource/computing/LibvirtDomainXMLParser.java
+++ b/agent/src/com/cloud/agent/resource/computing/LibvirtDomainXMLParser.java
@@ -94,6 +94,8 @@ public class LibvirtDomainXMLParser extends LibvirtXMLParser {
 				} else if (qName.equalsIgnoreCase("disk")) {
 					diskMaps.put(diskDev, diskFile);
 					_disk = false;
+					diskFile = null;
+					diskDev = null;
 				} else if (qName.equalsIgnoreCase("description")) {
 					_desc = false;
 				}
diff --git a/agent/src/com/cloud/agent/resource/storage/IscsiMountPreparer.java b/agent/src/com/cloud/agent/resource/storage/IscsiMountPreparer.java
index 97198e77c7c..2def1977824 100644
--- a/agent/src/com/cloud/agent/resource/storage/IscsiMountPreparer.java
+++ b/agent/src/com/cloud/agent/resource/storage/IscsiMountPreparer.java
@@ -28,8 +28,8 @@ import org.apache.log4j.Logger;
 import com.cloud.resource.DiskPreparer;
 import com.cloud.storage.Volume;
 import com.cloud.storage.VolumeVO;
-import com.cloud.storage.VirtualMachineTemplate.BootloaderType;
 import com.cloud.storage.Volume.VolumeType;
+import com.cloud.template.VirtualMachineTemplate.BootloaderType;
 import com.cloud.utils.NumbersUtil;
 import com.cloud.utils.script.Script;
 
diff --git a/agent/wscript_build b/agent/wscript_build
new file mode 100644
index 00000000000..583f4ac8963
--- /dev/null
+++ b/agent/wscript_build
@@ -0,0 +1,7 @@
+import Options
+
+bld.install_files("${AGENTLIBDIR}",
+	bld.path.ant_glob("storagepatch/**",src=True,bld=False,dir=False,flat=True),
+	cwd=bld.path,relative_trick=True)
+if not Options.options.PRESERVECONFIG:
+	bld.install_files_filtered("${AGENTSYSCONFDIR}","conf/*")
diff --git a/api/src/com/cloud/acl/SecurityChecker.java b/api/src/com/cloud/acl/SecurityChecker.java
new file mode 100644
index 00000000000..b878ca0b005
--- /dev/null
+++ b/api/src/com/cloud/acl/SecurityChecker.java
@@ -0,0 +1,66 @@
+/**
+ * 
+ */
+package com.cloud.acl;
+
+import java.security.acl.NotOwnerException;
+
+import com.cloud.domain.PartOf;
+import com.cloud.exception.PermissionDeniedException;
+import com.cloud.user.Account;
+import com.cloud.user.OwnedBy;
+import com.cloud.user.User;
+import com.cloud.utils.component.Adapter;
+
+/**
+ * SecurityChecker checks the ownership and access control to objects within
+ * the management stack for users and accounts. 
+ */
+public interface SecurityChecker extends Adapter {
+    /**
+     * Checks if the account owns the object.
+     * 
+     * @param account account to check against.
+     * @param object object that the account is trying to access.
+     * @return true if access allowed.  false if this adapter cannot authenticate ownership.
+     * @throws PermissionDeniedException if this adapter is suppose to authenticate ownership and the check failed.
+     */
+    boolean checkOwnership(Account account, OwnedBy object) throws NotOwnerException;
+    
+    /**
+     * Checks if the user belongs to an account that owns the object.
+     * 
+     * @param user user to check against.
+     * @param object object that the account is trying to access.
+     * @return true if access allowed.  false if this adapter cannot authenticate ownership.
+     * @throws PermissionDeniedException if this adapter is suppose to authenticate ownership and the check failed.
+     */
+    boolean checkOwnership(User user, OwnedBy object) throws NotOwnerException;
+    
+    /**
+     * Checks if the account can access the object.
+     * 
+     * @param account account to check against.
+     * @param object object that the account is trying to access.
+     * @return true if access allowed.  false if this adapter cannot provide permission.
+     * @throws PermissionDeniedException if this adapter is suppose to authenticate ownership and the check failed.
+     */
+    boolean checkAccess(Account account, PartOf object) throws PermissionDeniedException;
+
+    /**
+     * Checks if the user belongs to an account that can access the object.
+     * 
+     * @param user user to check against.
+     * @param object object that the account is trying to access.
+     * @return true if access allowed.  false if this adapter cannot authenticate ownership.
+     * @throws PermissionDeniedException if this adapter is suppose to authenticate ownership and the check failed.
+     */
+    boolean checkAccess(User user, PartOf object) throws PermissionDeniedException;
+
+// We should be able to use this method to check against commands.  For example, we can
+// annotate the command with access annotations and this method can use it to extract
+// OwnedBy and PartOf interfaces on the object and use it to verify against a user.
+// I leave this empty for now so Kris and the API team can see if it is useful.
+//    boolean checkAuthorization(User user, Command cmd) throws PermissionDeniedException;
+
+}
diff --git a/api/src/com/cloud/dc/DataCenter.java b/api/src/com/cloud/dc/DataCenter.java
new file mode 100644
index 00000000000..83902e42e3f
--- /dev/null
+++ b/api/src/com/cloud/dc/DataCenter.java
@@ -0,0 +1,13 @@
+/**
+ * 
+ */
+package com.cloud.dc;
+
+import com.cloud.org.Grouping;
+
+/**
+ *
+ */
+public interface DataCenter extends Grouping {
+    long getId();
+}
diff --git a/api/src/com/cloud/dc/Pod.java b/api/src/com/cloud/dc/Pod.java
new file mode 100644
index 00000000000..47e61f14e79
--- /dev/null
+++ b/api/src/com/cloud/dc/Pod.java
@@ -0,0 +1,27 @@
+/**
+ * 
+ */
+package com.cloud.dc;
+
+import com.cloud.org.Grouping;
+
+/**
+ * Represents one pod in the cloud stack.
+ *
+ */
+public interface Pod extends Grouping {
+    /**
+     * @return unique id mapped to the pod.
+     */
+    long getId();
+    
+    String getCidrAddress();
+    
+    int getCidrSize();
+
+    String getGateway();
+    
+    long getDataCenterId();
+    
+    //String getUniqueName();
+}
diff --git a/api/src/com/cloud/deploy/DataCenterDeployment.java b/api/src/com/cloud/deploy/DataCenterDeployment.java
index ba8b227c179..e882102c959 100644
--- a/api/src/com/cloud/deploy/DataCenterDeployment.java
+++ b/api/src/com/cloud/deploy/DataCenterDeployment.java
@@ -17,13 +17,21 @@
  */
 package com.cloud.deploy;
 
-public class DataCenterDeployment implements DeploymentStrategy {
+public class DataCenterDeployment implements DeploymentPlan {
     long _dcId;
-    public DataCenterDeployment(long dataCenterId) {
+    int _count;
+    
+    public DataCenterDeployment(long dataCenterId, int count) {
         _dcId = dataCenterId;
     }
     
+    @Override
     public long getDataCenterId() {
         return _dcId;
     }
+    
+    @Override
+    public int getCount() {
+        return _count;
+    }
 }
diff --git a/api/src/com/cloud/deploy/DeployDestination.java b/api/src/com/cloud/deploy/DeployDestination.java
new file mode 100644
index 00000000000..c3f91880327
--- /dev/null
+++ b/api/src/com/cloud/deploy/DeployDestination.java
@@ -0,0 +1,84 @@
+/**
+ * 
+ */
+package com.cloud.deploy;
+
+import java.util.Map;
+
+import com.cloud.dc.DataCenter;
+import com.cloud.dc.Pod;
+import com.cloud.host.Host;
+import com.cloud.org.Cluster;
+import com.cloud.storage.StoragePool;
+import com.cloud.storage.Volume;
+import com.cloud.utils.NumbersUtil;
+
+public class DeployDestination {
+    DataCenter _dc;
+    Pod _pod;
+    Cluster _cluster;
+    Host _host;
+    Map _storage;
+
+    public DataCenter getDataCenter() {
+        return _dc;
+    }
+    
+    public Pod getPod() {
+        return _pod;
+    }
+    
+    public Cluster getCluster() {
+        return _cluster;
+    }
+    
+    public Host getHost() {
+        return _host;
+    }
+    
+    public Map getStorageForDisks() {
+        return _storage;
+    }
+
+    public DeployDestination(DataCenter dc, Pod pod, Cluster cluster, Host host) {
+        _dc = dc;
+        _pod = pod;
+        _cluster = cluster;
+        _host = host;
+    }
+    
+    public DeployDestination() {
+    }
+    
+    @Override
+    public int hashCode() {
+        return NumbersUtil.hash(_host.getId());
+    }
+    
+    @Override
+    public boolean equals(Object obj) {
+        DeployDestination that = (DeployDestination)obj;
+        if (this._dc == null || that._dc == null) {
+            return false;
+        }
+        if (this._dc.getId() != that._dc.getId()) {
+            return false;
+        }
+        if (this._pod == null || that._pod == null) {
+            return false;
+        }
+        if (this._pod.getId() != that._pod.getId()) {
+            return false;
+        }
+        if (this._cluster == null || that._cluster == null) {
+            return false;
+        }
+        if (this._cluster.getId() != that._cluster.getId()) {
+            return false;
+        }
+        if (this._host == null || that._host == null) {
+            return false;
+        }
+        return this._host.getId() == that._host.getId();
+    }
+}
diff --git a/api/src/com/cloud/deploy/DeploymentStrategy.java b/api/src/com/cloud/deploy/DeploymentPlan.java
similarity index 89%
rename from api/src/com/cloud/deploy/DeploymentStrategy.java
rename to api/src/com/cloud/deploy/DeploymentPlan.java
index 57bdeca6188..a36d7b35154 100644
--- a/api/src/com/cloud/deploy/DeploymentStrategy.java
+++ b/api/src/com/cloud/deploy/DeploymentPlan.java
@@ -21,6 +21,7 @@ package com.cloud.deploy;
  * Describes how a VM should be deployed.
  *
  */
-public interface DeploymentStrategy {
-
+public interface DeploymentPlan {
+    public long getDataCenterId();
+    public int getCount();
 }
diff --git a/api/src/com/cloud/deploy/DeploymentPlanner.java b/api/src/com/cloud/deploy/DeploymentPlanner.java
new file mode 100644
index 00000000000..d60f466314f
--- /dev/null
+++ b/api/src/com/cloud/deploy/DeploymentPlanner.java
@@ -0,0 +1,13 @@
+/**
+ * 
+ */
+package com.cloud.deploy;
+
+import java.util.Set;
+
+import com.cloud.utils.component.Adapter;
+import com.cloud.vm.VirtualMachineProfile;
+
+public interface DeploymentPlanner extends Adapter {
+    DeployDestination plan(VirtualMachineProfile vm, DeploymentPlan plan, Set avoid);
+}
diff --git a/api/src/com/cloud/domain/Domain.java b/api/src/com/cloud/domain/Domain.java
new file mode 100644
index 00000000000..5cbfb39b4dc
--- /dev/null
+++ b/api/src/com/cloud/domain/Domain.java
@@ -0,0 +1,37 @@
+/**
+ * 
+ */
+package com.cloud.domain;
+
+import java.util.Date;
+
+import com.cloud.user.OwnedBy;
+
+/**
+ * Domain defines the Domain object.
+ */
+public interface Domain extends OwnedBy {
+    public static final long ROOT_DOMAIN = 1L;
+    
+    long getId();
+
+    Long getParent();
+    
+    void setParent(Long parent);
+
+    String getName();
+    
+    void setName(String name);
+    
+    Date getRemoved();
+    
+    String getPath();
+    
+    void setPath(String path);
+    
+    int getLevel();
+    
+    int getChildCount();
+    
+    long getNextChildSeq();
+}
diff --git a/api/src/com/cloud/domain/PartOf.java b/api/src/com/cloud/domain/PartOf.java
new file mode 100644
index 00000000000..98b8b2ad6dd
--- /dev/null
+++ b/api/src/com/cloud/domain/PartOf.java
@@ -0,0 +1,15 @@
+/**
+ * 
+ */
+package com.cloud.domain;
+
+/**
+ * PartOf must be implemented by all objects that belongs 
+ * in a domain.
+ */
+public interface PartOf {
+    /**
+     * @return domain id that the object belongs to.
+     */
+    long getDomainId();
+}
diff --git a/api/src/com/cloud/exception/ConflictingNetworkSettingsException.java b/api/src/com/cloud/exception/ConflictingNetworkSettingsException.java
new file mode 100644
index 00000000000..5d3f2556b35
--- /dev/null
+++ b/api/src/com/cloud/exception/ConflictingNetworkSettingsException.java
@@ -0,0 +1,15 @@
+/**
+ * 
+ */
+package com.cloud.exception;
+
+import com.cloud.utils.SerialVersionUID;
+
+public class ConflictingNetworkSettingsException extends Exception {
+
+    private static final long serialVersionUID = SerialVersionUID.ConflictingNetworkSettingException;
+    
+    public ConflictingNetworkSettingsException() {
+        super();
+    }
+}
diff --git a/core/src/com/cloud/host/Host.java b/api/src/com/cloud/host/Host.java
similarity index 98%
rename from core/src/com/cloud/host/Host.java
rename to api/src/com/cloud/host/Host.java
index a8ec5301d32..f90307ed6a6 100755
--- a/core/src/com/cloud/host/Host.java
+++ b/api/src/com/cloud/host/Host.java
@@ -19,7 +19,9 @@ package com.cloud.host;
 
 import java.util.Date;
 
+import com.cloud.host.Status;
 import com.cloud.hypervisor.Hypervisor;
+import com.cloud.hypervisor.Hypervisor.Type;
 
 
 /**
diff --git a/core/src/com/cloud/host/Status.java b/api/src/com/cloud/host/Status.java
similarity index 100%
rename from core/src/com/cloud/host/Status.java
rename to api/src/com/cloud/host/Status.java
diff --git a/api/src/com/cloud/hypervisor/Hypervisor.java b/api/src/com/cloud/hypervisor/Hypervisor.java
index 17178bd2154..c938bba26e4 100644
--- a/api/src/com/cloud/hypervisor/Hypervisor.java
+++ b/api/src/com/cloud/hypervisor/Hypervisor.java
@@ -23,7 +23,9 @@ public class Hypervisor {
     	None, //for storage hosts
     	Xen,
     	XenServer,
-    	KVM;
+    	KVM,
+    	VmWare,
+    	VirtualBox,
+    	Parralels;
     }
-
 }
diff --git a/api/src/com/cloud/network/Network.java b/api/src/com/cloud/network/Network.java
index c5c35b0d13b..cf393f10fb8 100644
--- a/api/src/com/cloud/network/Network.java
+++ b/api/src/com/cloud/network/Network.java
@@ -17,6 +17,11 @@
  */
 package com.cloud.network;
 
+import java.net.URI;
+import java.net.URISyntaxException;
+
+import com.cloud.utils.exception.CloudRuntimeException;
+
 /**
  * Network includes all of the enums used within networking.
  *
@@ -34,17 +39,50 @@ public class Network {
     
     public enum AddressFormat {
         Ip4,
-        Ip6
+        Ip6,
+        Mixed
     }
 
     /**
      * Different types of broadcast domains. 
      */
     public enum BroadcastDomainType {
-        Native,
-        Vlan,
-        Vswitch,
-        Vnet;
+        Native(null, null),
+        Vlan("vlan", Integer.class),
+        Vswitch("vs", String.class),
+        LinkLocal(null, null),
+        Vnet("vnet", Long.class),
+        UnDecided(null, null);
+        
+        private String scheme;
+        private Class type;
+        
+        private BroadcastDomainType(String scheme, Class type) {
+            this.scheme = scheme;
+            this.type = type;
+        }
+        
+        /**
+         * @return scheme to be used in broadcast uri.  Null indicates that this type does not have broadcast tags.
+         */
+        public String scheme() {
+            return scheme;
+        }
+        
+        /**
+         * @return type of the value in the broadcast uri. Null indicates that this type does not have broadcast tags.
+         */
+        public Class type() {
+            return type;
+        }
+        
+        public  URI toUri(T value) {
+            try {
+                return new URI(scheme + "://" + value);
+            } catch (URISyntaxException e) {
+                throw new CloudRuntimeException("Unable to convert to broadcast URI: " + value);
+            }
+        }
     };
     
     /**
@@ -54,8 +92,57 @@ public class Network {
         Public,
         Guest,
         Storage,
-        LinkLocal,
+        Control,
         Vpn,
         Management
     };
+    
+    public enum IsolationType {
+        None(null, null),
+        Ec2("ec2", String.class),
+        Vlan("vlan", Integer.class),
+        Vswitch("vs", String.class),
+        Undecided(null, null),
+        Vnet("vnet", Long.class);
+        
+        private final String scheme;
+        private final Class type;
+        
+        private IsolationType(String scheme, Class type) {
+            this.scheme = scheme;
+            this.type = type;
+        }
+        
+        public String scheme() {
+            return scheme;
+        }
+        
+        public Class type() {
+            return type;
+        }
+        
+        public  URI toUri(T value) {
+            try {
+                return new URI(scheme + "://" + value.toString());
+            } catch (URISyntaxException e) {
+                throw new CloudRuntimeException("Unable to convert to isolation type URI: " + value);
+            }
+        }
+    }
+    
+    public enum BroadcastScheme {
+        Vlan("vlan"),
+        VSwitch("vswitch");
+        
+        private String scheme;
+        
+        private BroadcastScheme(String scheme) {
+            this.scheme = scheme;
+        }
+        
+        @Override
+        public String toString() {
+            return scheme;
+        }
+    }
 }
diff --git a/api/src/com/cloud/network/NetworkConfiguration.java b/api/src/com/cloud/network/NetworkConfiguration.java
new file mode 100644
index 00000000000..9ccf426a389
--- /dev/null
+++ b/api/src/com/cloud/network/NetworkConfiguration.java
@@ -0,0 +1,75 @@
+/**
+ * 
+ */
+package com.cloud.network;
+
+import java.util.List;
+import java.util.Set;
+
+import com.cloud.network.Network.BroadcastDomainType;
+import com.cloud.network.Network.Mode;
+import com.cloud.network.Network.TrafficType;
+import com.cloud.utils.fsm.FiniteState;
+import com.cloud.utils.fsm.StateMachine;
+
+/**
+ * A NetworkProfile defines the specifics of a network
+ * owned by an account. 
+ */
+public interface NetworkConfiguration {
+    enum Event {
+        ImplementNetwork,
+        DestroyNetwork;
+    }
+    
+    enum State implements FiniteState {
+        Allocated,  // Indicates the network configuration is in allocated but not setup.
+        Setup,      // Indicates the network configuration is setup.
+        Implemented,      // Indicates the network configuration is in use.
+        Destroying;
+
+        @Override
+        public StateMachine getStateMachine() {
+            return s_fsm;
+        }
+
+        @Override
+        public State getNextState(Event event) {
+            return s_fsm.getNextState(this, event);
+        }
+
+        @Override
+        public List getFromStates(Event event) {
+            return s_fsm.getFromStates(this, event);
+        }
+
+        @Override
+        public Set getPossibleEvents() {
+            return s_fsm.getPossibleEvents(this);
+        }
+        
+        private static StateMachine s_fsm = new StateMachine();
+    }
+    
+    
+    /**
+     * @return id of the network profile.  Null means the network profile is not from the database.
+     */
+    Long getId();
+
+    Mode getMode();
+
+    BroadcastDomainType getBroadcastDomainType();
+
+    TrafficType getTrafficType();
+
+    String getGateway();
+
+    String getCidr();
+
+    long getDataCenterId();
+    
+    long getNetworkOfferingId();
+    
+    State getState();
+}
diff --git a/api/src/com/cloud/network/configuration/NetworkGuru.java b/api/src/com/cloud/network/configuration/NetworkGuru.java
new file mode 100644
index 00000000000..2cb0a5a1764
--- /dev/null
+++ b/api/src/com/cloud/network/configuration/NetworkGuru.java
@@ -0,0 +1,37 @@
+/**
+ * 
+ */
+package com.cloud.network.configuration;
+
+import com.cloud.deploy.DeployDestination;
+import com.cloud.deploy.DeploymentPlan;
+import com.cloud.exception.InsufficientAddressCapacityException;
+import com.cloud.exception.InsufficientVirtualNetworkCapcityException;
+import com.cloud.network.NetworkConfiguration;
+import com.cloud.offering.NetworkOffering;
+import com.cloud.user.Account;
+import com.cloud.utils.component.Adapter;
+import com.cloud.vm.NicProfile;
+import com.cloud.vm.VirtualMachineProfile;
+
+/**
+ * NetworkGuru takes a network offering requested and figures
+ * out what is the correct network configuration that are needed to add
+ * to the account in order to support this network. 
+ *
+ */
+public interface NetworkGuru extends Adapter {
+    NetworkConfiguration design(NetworkOffering offering, DeploymentPlan plan, NetworkConfiguration userSpecified, Account owner);
+    
+    NetworkConfiguration implement(NetworkConfiguration config, NetworkOffering offering, DeployDestination destination);
+    
+    NicProfile allocate(NetworkConfiguration config, NicProfile nic, VirtualMachineProfile vm) throws InsufficientVirtualNetworkCapcityException, InsufficientAddressCapacityException;
+    
+//    NicProfile create(NicProfile nic, NetworkConfiguration config, VirtualMachineProfile vm, DeployDestination dest) throws InsufficientVirtualNetworkCapcityException, InsufficientAddressCapacityException;
+    
+    String reserve(NicProfile nic, NetworkConfiguration config, VirtualMachineProfile vm, DeployDestination dest) throws InsufficientVirtualNetworkCapcityException, InsufficientAddressCapacityException;
+    
+    boolean release(String uniqueId);
+    
+    void destroy(NetworkConfiguration config, NetworkOffering offering);
+}
diff --git a/api/src/com/cloud/network/element/NetworkElement.java b/api/src/com/cloud/network/element/NetworkElement.java
new file mode 100644
index 00000000000..4296fe36010
--- /dev/null
+++ b/api/src/com/cloud/network/element/NetworkElement.java
@@ -0,0 +1,36 @@
+/**
+ * 
+ */
+package com.cloud.network.element;
+
+import com.cloud.network.NetworkConfiguration;
+import com.cloud.offering.NetworkOffering;
+import com.cloud.utils.component.Adapter;
+import com.cloud.vm.NicProfile;
+import com.cloud.vm.VirtualMachineProfile;
+
+/**
+ * Represents one network element that exists in a network.
+ */
+public interface NetworkElement extends Adapter {
+    /**
+     * Implement the network configuration as specified. 
+     * @param config fully specified network configuration.
+     * @param offering network offering that originated the network configuration.
+     * @return true if network configuration is now usable; false if not.
+     */
+    boolean implement(NetworkConfiguration config, NetworkOffering offering);
+    
+    /**
+     * Prepare the nic profile to be used within the network.
+     * @param config
+     * @param nic
+     * @param offering
+     * @return
+     */
+    boolean prepare(NetworkConfiguration config, NicProfile nic, VirtualMachineProfile vm, NetworkOffering offering);
+    
+    boolean release(NetworkConfiguration config, NicProfile nic, VirtualMachineProfile vm, NetworkOffering offering);
+    
+    boolean shutdown(NetworkConfiguration config, NetworkOffering offering);
+}
diff --git a/api/src/com/cloud/offering/NetworkOffering.java b/api/src/com/cloud/offering/NetworkOffering.java
index 831a626c0f8..3d098f0069e 100644
--- a/api/src/com/cloud/offering/NetworkOffering.java
+++ b/api/src/com/cloud/offering/NetworkOffering.java
@@ -17,18 +17,20 @@
  */
 package com.cloud.offering;
 
+import com.cloud.network.Network.TrafficType;
 
 /**
  * Describes network offering
  *
  */
 public interface NetworkOffering {
-    public enum GuestIpType {
-        Virtualized,
-        DirectSingle,
-        DirectDual
-    }
     
+    public enum GuestIpType {
+    	Virtualized,
+    	DirectSingle,
+    	DirectDual
+    }
+
     long getId();
 
     /**
@@ -60,4 +62,6 @@ public interface NetworkOffering {
      * @return concurrent connections to be supported.
      */
     Integer getConcurrentConnections();
+    
+    TrafficType getTrafficType();
 }
diff --git a/api/src/com/cloud/offering/ServiceOffering.java b/api/src/com/cloud/offering/ServiceOffering.java
index 4cdaaeb4dce..87ff831d32e 100755
--- a/api/src/com/cloud/offering/ServiceOffering.java
+++ b/api/src/com/cloud/offering/ServiceOffering.java
@@ -22,13 +22,7 @@ package com.cloud.offering;
  * offered.
  */
 public interface ServiceOffering {
-	public enum GuestIpType {
-		Virtualized,
-		DirectSingle,
-		DirectDual
-	}
-	
-    /**
+	/**
      * @return user readable description
      */
     String getName();
@@ -66,7 +60,7 @@ public interface ServiceOffering {
     /**
      * @return the type of IP address to allocate as the primary ip address to a guest
      */
-    GuestIpType getGuestIpType();
+    NetworkOffering.GuestIpType getGuestIpType();
     
     /**
      * @return whether or not the service offering requires local storage
diff --git a/api/src/com/cloud/org/Cluster.java b/api/src/com/cloud/org/Cluster.java
new file mode 100644
index 00000000000..d5da8e2de01
--- /dev/null
+++ b/api/src/com/cloud/org/Cluster.java
@@ -0,0 +1,9 @@
+/**
+ * 
+ */
+package com.cloud.org;
+
+public interface Cluster extends Grouping {
+    long getId();
+
+}
diff --git a/api/src/com/cloud/org/Grouping.java b/api/src/com/cloud/org/Grouping.java
new file mode 100644
index 00000000000..f8516a66682
--- /dev/null
+++ b/api/src/com/cloud/org/Grouping.java
@@ -0,0 +1,8 @@
+/**
+ * 
+ */
+package com.cloud.org;
+
+public interface Grouping {
+
+}
diff --git a/api/src/com/cloud/org/RunningIn.java b/api/src/com/cloud/org/RunningIn.java
new file mode 100644
index 00000000000..f8b93e3afa5
--- /dev/null
+++ b/api/src/com/cloud/org/RunningIn.java
@@ -0,0 +1,11 @@
+/**
+ * 
+ */
+package com.cloud.org;
+
+public interface RunningIn {
+    long getDataCenterId();
+    long getPodId();
+    Long getClusterId();
+    Long getHostId();
+}
diff --git a/api/src/com/cloud/resource/Concierge.java b/api/src/com/cloud/resource/Concierge.java
new file mode 100644
index 00000000000..04b1538ef5b
--- /dev/null
+++ b/api/src/com/cloud/resource/Concierge.java
@@ -0,0 +1,10 @@
+/**
+ * 
+ */
+package com.cloud.resource;
+
+import com.cloud.utils.component.Adapter;
+
+public interface Concierge extends Adapter {
+
+}
diff --git a/api/src/com/cloud/resource/Resource.java b/api/src/com/cloud/resource/Resource.java
new file mode 100644
index 00000000000..310507061ae
--- /dev/null
+++ b/api/src/com/cloud/resource/Resource.java
@@ -0,0 +1,111 @@
+/**
+ * 
+ */
+package com.cloud.resource;
+
+import java.util.Date;
+import java.util.List;
+import java.util.Set;
+
+import com.cloud.utils.fsm.FiniteState;
+import com.cloud.utils.fsm.StateMachine;
+
+/**
+ * Indicates a resource in CloudStack.  
+ * Any resource that requires an reservation and release system
+ * must implement this interface.
+ *
+ */
+public interface Resource {
+    enum Event {
+        ReservationRequested,
+        ReleaseRequested,
+        CancelRequested,
+        OperationCompleted,
+        OperationFailed,
+    }
+    
+    enum State implements FiniteState {
+        Allocated, // Resource is allocated
+        Reserving, // Resource is being reserved right now.
+        Reserved,  // Resource is reserved
+        Releasing, // Resource is being released.
+        Ready;     // Resource is ready which means it does not need to go through reservation.
+
+        @Override
+        public StateMachine getStateMachine() {
+            
+            // TODO Auto-generated method stub
+            return null;
+        }
+
+        @Override
+        public State getNextState(Event event) {
+            return s_fsm.getNextState(this, event);
+        }
+
+        @Override
+        public List getFromStates(Event event) {
+            return s_fsm.getFromStates(this, event);
+        }
+
+        @Override
+        public Set getPossibleEvents() {
+            return s_fsm.getPossibleEvents(this);
+        }
+        
+        final static private StateMachine s_fsm = new StateMachine();
+        static {
+            s_fsm.addTransition(State.Allocated, Event.ReservationRequested, State.Reserving);
+            s_fsm.addTransition(State.Reserving, Event.CancelRequested, State.Allocated);
+            s_fsm.addTransition(State.Reserving, Event.OperationCompleted, State.Reserved);
+            s_fsm.addTransition(State.Reserving, Event.OperationFailed, State.Allocated);
+        }
+    }
+    
+    enum ReservationStrategy {
+        UserSpecified,
+        Create,
+        Start
+    }
+    
+    /**
+     * @return id in the CloudStack database
+     */
+    long getId();
+    
+    /**
+     * @return reservation id returned by the allocation source.  This can be the
+     * String version of the database id if the allocation source does not need it's 
+     * own implementation of the reservation id.  This is passed back to the 
+     * allocation source to release the resource.
+     */
+    String getReservationId();
+    
+    /**
+     * @return unique name for the allocation source.
+     */
+    String getReserver();
+    
+    /**
+     * @return the time a reservation request was made to the allocation source.
+     */
+    Date getUpdateTime();
+    
+    /**
+     * @return the expected reservation interval.  -1 indicates 
+     */
+    int getExpectedReservationInterval();
+    
+    /**
+     * @return the expected release interval.
+     */
+    int getExpectedReleaseInterval();
+    
+    /**
+     * @return the reservation state of the resource.
+     */
+    State getState();
+    
+    ReservationStrategy getReservationStrategy();
+}
diff --git a/api/src/com/cloud/storage/Storage.java b/api/src/com/cloud/storage/Storage.java
index 08eccfeb8ff..81d94ad0331 100644
--- a/api/src/com/cloud/storage/Storage.java
+++ b/api/src/com/cloud/storage/Storage.java
@@ -18,20 +18,30 @@
 package com.cloud.storage;
 
 public class Storage {
-    public enum ImageFormat {
+    public static enum ImageFormat {
         QCOW2(true, true, false),
         RAW(false, false, false),
         VHD(true, true, true),
-        ISO(false, false, false);
+        ISO(false, false, false),
+        VMDK(true, true, true, "vmw.tar");
         
         private final boolean thinProvisioned;
         private final boolean supportSparse;
         private final boolean supportSnapshot;
+        private final String fileExtension;
         
         private ImageFormat(boolean thinProvisioned, boolean supportSparse, boolean supportSnapshot) {
             this.thinProvisioned = thinProvisioned;
             this.supportSparse = supportSparse;
             this.supportSnapshot = supportSnapshot;
+            fileExtension = null;
+        }
+        
+        private ImageFormat(boolean thinProvisioned, boolean supportSparse, boolean supportSnapshot, String fileExtension) {
+            this.thinProvisioned = thinProvisioned;
+            this.supportSparse = supportSparse;
+            this.supportSnapshot = supportSnapshot;
+            this.fileExtension = fileExtension;
         }
         
         public boolean isThinProvisioned() {
@@ -47,11 +57,14 @@ public class Storage {
         }
         
         public String getFileExtension() {
-            return toString().toLowerCase();
+        	if(fileExtension == null)
+        		return toString().toLowerCase();
+        	
+        	return fileExtension;
         }
     }
     
-    public enum FileSystem {
+    public static enum FileSystem {
         Unknown,
         ext3,
         ntfs,
@@ -66,7 +79,7 @@ public class Storage {
         hfsp
     }
     
-    public enum StoragePoolType {
+    public static enum StoragePoolType {
         Filesystem(false), //local directory
         NetworkFilesystem(true), //NFS or CIFS
         IscsiLUN(true), //shared LUN, with a clusterfs overlay
@@ -84,4 +97,6 @@ public class Storage {
             return shared;
         }
     }
+
+    public static enum StorageResourceType {STORAGE_POOL, STORAGE_HOST, SECONDARY_STORAGE}
 }
diff --git a/core/src/com/cloud/storage/StoragePool.java b/api/src/com/cloud/storage/StoragePool.java
similarity index 100%
rename from core/src/com/cloud/storage/StoragePool.java
rename to api/src/com/cloud/storage/StoragePool.java
diff --git a/core/src/com/cloud/storage/Volume.java b/api/src/com/cloud/storage/Volume.java
similarity index 66%
rename from core/src/com/cloud/storage/Volume.java
rename to api/src/com/cloud/storage/Volume.java
index 45d22ee4018..75421e1880b 100755
--- a/core/src/com/cloud/storage/Volume.java
+++ b/api/src/com/cloud/storage/Volume.java
@@ -17,30 +17,38 @@
  */
 package com.cloud.storage;
 
-import com.cloud.async.AsyncInstanceCreateStatus;
+import java.util.Date;
 
-public interface Volume {
-	enum VolumeType {UNKNOWN, ROOT, SWAP, DATADISK};
+import com.cloud.domain.PartOf;
+import com.cloud.template.BasedOn;
+import com.cloud.user.OwnedBy;
+
+
+public interface Volume extends PartOf, OwnedBy, BasedOn {
+	enum VolumeType {UNKNOWN, ROOT, SWAP, DATADISK, ISO};
 	
 	enum MirrorState {NOT_MIRRORED, ACTIVE, DEFUNCT};
 	
-	enum StorageResourceType {STORAGE_POOL, STORAGE_HOST, SECONDARY_STORAGE};
-
-    /**
+	enum State {
+	    Allocated,
+	    Creating,
+	    Created,
+	    Corrupted,
+	    ToBeDestroyed,
+	    Expunging,
+	    Destroyed
+	}
+	
+	enum SourceType {
+		Snapshot,DiskOffering,Template,Blank
+	}
+	
+	long getId();
+	/**
      * @return the volume name
      */
     String getName();
     
-    /**
-     * @return owner's account id
-     */
-    long getAccountId();
-
-    /**
-     * @return id of the owning account's domain
-     */
-    long getDomainId();
-    
     /**
      * @return total size of the partition
      */
@@ -69,12 +77,21 @@ public interface Volume {
     
     VolumeType getVolumeType();
     
-    StorageResourceType getStorageResourceType();
+    Storage.StorageResourceType getStorageResourceType();
     
 	Long getPoolId();
 	
-    public AsyncInstanceCreateStatus getStatus();
+	State getState();
 	
-	public void setStatus(AsyncInstanceCreateStatus status);
+	SourceType getSourceType();
 	
+	void setSourceType(SourceType sourceType);
+
+	void setSourceId(Long sourceId);
+
+	Long getSourceId();
+
+	Date getAttached();
+
+	void setAttached(Date attached);
 }
diff --git a/api/src/com/cloud/template/BasedOn.java b/api/src/com/cloud/template/BasedOn.java
new file mode 100644
index 00000000000..4424cbdb520
--- /dev/null
+++ b/api/src/com/cloud/template/BasedOn.java
@@ -0,0 +1,16 @@
+/**
+ * 
+ */
+package com.cloud.template;
+
+/**
+ * BasedOn is implemented by all objects that are based on a certain template.
+ */
+public interface BasedOn {
+    
+    /**
+     * @return the template id that the volume is based on.
+     */
+    Long getTemplateId();
+
+}
diff --git a/core/src/com/cloud/storage/VirtualMachineTemplate.java b/api/src/com/cloud/template/VirtualMachineTemplate.java
similarity index 86%
rename from core/src/com/cloud/storage/VirtualMachineTemplate.java
rename to api/src/com/cloud/template/VirtualMachineTemplate.java
index 7a56a2c9477..0cdffe856d8 100755
--- a/core/src/com/cloud/storage/VirtualMachineTemplate.java
+++ b/api/src/com/cloud/template/VirtualMachineTemplate.java
@@ -15,14 +15,13 @@
  * along with this program.  If not, see .
  * 
  */
-package com.cloud.storage;
+package com.cloud.template;
 
-import com.cloud.async.AsyncInstanceCreateStatus;
 import com.cloud.storage.Storage.FileSystem;
 
-public interface VirtualMachineTemplate {
+public interface VirtualMachineTemplate  {
     
-    public static enum BootloaderType { PyGrub, HVM, External };
+    public static enum BootloaderType { PyGrub, HVM, External, CD };
     
     /**
      * @return id.
diff --git a/core/src/com/cloud/user/Account.java b/api/src/com/cloud/user/Account.java
similarity index 82%
rename from core/src/com/cloud/user/Account.java
rename to api/src/com/cloud/user/Account.java
index bb419b1007a..aebd92ab134 100644
--- a/core/src/com/cloud/user/Account.java
+++ b/api/src/com/cloud/user/Account.java
@@ -20,7 +20,22 @@ package com.cloud.user;
 
 import java.util.Date;
 
-public interface Account {
+import com.cloud.domain.PartOf;
+
+public interface Account extends PartOf {
+    public enum Type {
+        Normal,
+        Admin,
+        DomainAdmin,
+        CustomerCare
+    }
+    
+    public enum State {
+        Disabled,
+        Enabled,
+        Locked
+    }
+    
     public static final short ACCOUNT_TYPE_NORMAL = 0;
     public static final short ACCOUNT_TYPE_ADMIN = 1;
     public static final short ACCOUNT_TYPE_DOMAIN_ADMIN = 2;
@@ -32,14 +47,12 @@ public interface Account {
 
     public static final long ACCOUNT_ID_SYSTEM = 1; 
 
-    public Long getId();
+    public long getId();
     public String getAccountName();
     public void setAccountName(String accountId);
     public short getType();
-    public void setType(short type);
     public String getState();
     public void setState(String state);
-    public Long getDomainId();
-    public void setDomainId(Long domainId);
+    public long getDomainId();
     public Date getRemoved();
 }
diff --git a/api/src/com/cloud/user/OwnedBy.java b/api/src/com/cloud/user/OwnedBy.java
new file mode 100644
index 00000000000..78edbc33bd2
--- /dev/null
+++ b/api/src/com/cloud/user/OwnedBy.java
@@ -0,0 +1,14 @@
+/**
+ * 
+ */
+package com.cloud.user;
+
+/**
+ * OwnedBy must be inheritted by all objects that can be owned by an account.
+ */
+public interface OwnedBy {
+    /**
+     * @return account id that owns this object.
+     */
+    long getAccountId();
+}
diff --git a/core/src/com/cloud/user/User.java b/api/src/com/cloud/user/User.java
similarity index 96%
rename from core/src/com/cloud/user/User.java
rename to api/src/com/cloud/user/User.java
index f5e5adc34c3..2859e24697d 100644
--- a/core/src/com/cloud/user/User.java
+++ b/api/src/com/cloud/user/User.java
@@ -18,9 +18,9 @@
 
 package com.cloud.user;
 
-import java.util.Date;
+import java.util.Date;
 
-public interface User {
+public interface User extends OwnedBy {
 	public static final long UID_SYSTEM = 1; 
 	
 	public Long getId();
@@ -45,8 +45,6 @@ public interface User {
 	
 	public void setLastname(String lastname);
 
-	public long getAccountId();
-
 	public void setAccountId(long accountId);
 
     public String getEmail();
diff --git a/api/src/com/cloud/uservm/UserVm.java b/api/src/com/cloud/uservm/UserVm.java
index c4a9d2c923e..0709baec361 100755
--- a/api/src/com/cloud/uservm/UserVm.java
+++ b/api/src/com/cloud/uservm/UserVm.java
@@ -17,12 +17,14 @@
  */
 package com.cloud.uservm;
 
+import com.cloud.domain.PartOf;
+import com.cloud.user.OwnedBy;
 import com.cloud.vm.VirtualMachine;
 
 /**
  * This represents one running virtual machine instance.
  */
-public interface UserVm extends VirtualMachine {
+public interface UserVm extends VirtualMachine, OwnedBy, PartOf {
     
     /**
      * @return service offering id
@@ -39,11 +41,6 @@ public interface UserVm extends VirtualMachine {
      */
     String getVnet();
     
-    /**
-     * @return the account this vm instance belongs to.
-     */
-    long getAccountId();
-
     /**
      * @return the domain this vm instance belongs to.
      */
@@ -63,8 +60,6 @@ public interface UserVm extends VirtualMachine {
     
     String getDisplayName();
     
-    String getGroup();
-    
     String getUserData();
     
     void setUserData(String userData);
diff --git a/api/src/com/cloud/vm/DiskProfile.java b/api/src/com/cloud/vm/DiskProfile.java
new file mode 100644
index 00000000000..cad858b9dad
--- /dev/null
+++ b/api/src/com/cloud/vm/DiskProfile.java
@@ -0,0 +1,131 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.vm;
+
+import com.cloud.offering.DiskOffering;
+import com.cloud.storage.Volume;
+
+/**
+ * DiskCharacteristics describes a disk and what functionality is required from it.
+ * This object is generated by the management server and passed to the allocators
+ * and resources to allocate and create disks.  There object is immutable once
+ * it has been created.
+ */
+public class DiskProfile {
+    private long size;
+    private String[] tags;
+    private Volume.VolumeType type;
+    private String name;
+    private boolean useLocalStorage;
+    private boolean recreatable;
+    private long diskOfferingId;
+    private Long templateId;
+    private long volumeId;
+    private Volume vol;
+    private DiskOffering offering;
+    
+    protected DiskProfile() {
+    }
+    
+    public DiskProfile(long volumeId, Volume.VolumeType type, String name, long diskOfferingId, long size, String[] tags, boolean useLocalStorage, boolean recreatable, Long templateId) {
+        this.type = type;
+        this.name = name;
+        this.size = size;
+        this.tags = tags;
+        this.useLocalStorage = useLocalStorage;
+        this.recreatable = recreatable;
+        this.diskOfferingId = diskOfferingId;
+        this.templateId = templateId;
+        this.volumeId = volumeId;
+    }
+    
+    public DiskProfile(Volume vol, DiskOffering offering) {
+        this(vol.getId(), vol.getVolumeType(), vol.getName(), offering.getId(), vol.getSize(), offering.getTagsArray(), offering.getUseLocalStorage(), offering.getUseLocalStorage(), vol.getSize());
+        this.vol = vol;
+        this.offering = offering;
+    }
+    
+    /**
+     * @return size of the disk requested in bytes.
+     */
+    public long getSize() {
+        return size;
+    }
+    
+    /**
+     * @return id of the volume backing up this disk characteristics
+     */
+    public long getVolumeId() {
+        return volumeId;
+    }
+    
+    /**
+     * @return Unique name for the disk.
+     */
+    public String getName() {
+        return name;
+    }
+
+    /**
+     * @return tags for the disk.  This can be used to match it to different storage pools.
+     */
+    public String[] getTags() {
+        return tags;
+    }
+
+    /**
+     * @return type of volume.  
+     */
+    public Volume.VolumeType getType() {
+        return type;
+    }
+    
+    /**
+     * @return Does this volume require local storage?
+     */
+    public boolean useLocalStorage() {
+        return useLocalStorage;
+    }
+    
+    /**
+     * @return Is this volume recreatable?  A volume is recreatable if the disk's content can be
+     * reconstructed from the template.
+     */
+    public boolean isRecreatable() {
+        return recreatable;
+    }
+    
+    /**
+     * @return template id the disk is based on.  Can be null if it is not based on any templates.
+     */
+    public Long getTemplateId() {
+        return templateId;
+    }
+    
+    /**
+     * @return disk offering id that the disk is based on.
+     */
+    public long getDiskOfferingId() {
+        return diskOfferingId;
+    }
+    
+    @Override
+    public String toString() {
+        return new StringBuilder("DskChr[").append(type).append("|").append(size).append("|").append("]").toString();
+    }
+}
diff --git a/api/src/com/cloud/vm/Nic.java b/api/src/com/cloud/vm/Nic.java
index b477170ed11..621a7c1e43f 100644
--- a/api/src/com/cloud/vm/Nic.java
+++ b/api/src/com/cloud/vm/Nic.java
@@ -17,17 +17,14 @@
  */
 package com.cloud.vm;
 
+import com.cloud.network.Network.Mode;
+import com.cloud.resource.Resource;
+
 
 /**
  * Nic represents one nic on the VM.
  */
-public interface Nic {
-    enum State {
-        AcquireIp,
-        IpAcquired,
-    }
-
-    State getState();
+public interface Nic extends Resource {
 
     String getIp4Address();
     
@@ -36,15 +33,14 @@ public interface Nic {
     /**
      * @return network profile id that this 
      */
-    long getNetworkProfileId();
-    
-    /**
-     * @return the unique id to reference this nic.
-     */
-    long getId();
+    long getNetworkConfigurationId();
     
     /**
      * @return the vm instance id that this nic belongs to.
      */
     long getInstanceId();
+    
+    int getDeviceId();
+    
+    Mode getMode();
 }
diff --git a/api/src/com/cloud/vm/NicProfile.java b/api/src/com/cloud/vm/NicProfile.java
new file mode 100644
index 00000000000..621472723ea
--- /dev/null
+++ b/api/src/com/cloud/vm/NicProfile.java
@@ -0,0 +1,186 @@
+/**
+ * 
+ */
+package com.cloud.vm;
+
+import java.net.URI;
+
+import com.cloud.network.Network.AddressFormat;
+import com.cloud.network.Network.BroadcastDomainType;
+import com.cloud.network.Network.Mode;
+import com.cloud.network.Network.TrafficType;
+import com.cloud.network.NetworkConfiguration;
+import com.cloud.resource.Resource;
+import com.cloud.resource.Resource.ReservationStrategy;
+
+public class NicProfile {
+    long id;
+    BroadcastDomainType broadcastType;
+    Mode mode;
+    long vmId;
+    String gateway;
+    AddressFormat format;
+    TrafficType trafficType;
+    String ip4Address;
+    String ip6Address;
+    String macAddress;
+    URI isolationUri;
+    String netmask;
+    URI broadcastUri;
+    ReservationStrategy strategy;
+    String reservationId;
+    
+    public String getNetmask() {
+        return netmask;
+    }
+    
+    public void setNetmask(String netmask) {
+        this.netmask = netmask;
+    }
+    
+    public void setBroadcastUri(URI broadcastUri) {
+        this.broadcastUri = broadcastUri;
+    }
+    
+    public URI getBroadCastUri() {
+        return broadcastUri;
+    }
+    
+    public void setIsolationUri(URI isolationUri) {
+        this.isolationUri = isolationUri;
+    }
+    
+    public URI getIsolationUri() {
+        return isolationUri;
+    }
+    
+    public BroadcastDomainType getType() {
+        return broadcastType;
+    }
+
+    public void setBroadcastType(BroadcastDomainType broadcastType) {
+        this.broadcastType = broadcastType;
+    }
+
+    public void setMode(Mode mode) {
+        this.mode = mode;
+    }
+
+    public void setVmId(long vmId) {
+        this.vmId = vmId;
+    }
+
+    public void setGateway(String gateway) {
+        this.gateway = gateway;
+    }
+
+    public void setFormat(AddressFormat format) {
+        this.format = format;
+    }
+
+    public void setTrafficType(TrafficType trafficType) {
+        this.trafficType = trafficType;
+    }
+
+    public void setIp6Address(String ip6Address) {
+        this.ip6Address = ip6Address;
+    }
+
+    public Mode getMode() {
+        return mode;
+    }
+    
+    public long getNetworkId() {
+        return id;
+    }
+    
+    public long getVirtualMachineId() {
+        return vmId;
+    }
+    
+    public long getId() {
+        return id;
+    }
+
+    public BroadcastDomainType getBroadcastType() {
+        return broadcastType;
+    }
+
+    public void setMacAddress(String macAddress) {
+        this.macAddress = macAddress;
+    }
+
+    public long getVmId() {
+        return vmId;
+    }
+
+    public String getGateway() {
+        return gateway;
+    }
+
+    public AddressFormat getFormat() {
+        return format;
+    }
+
+    public TrafficType getTrafficType() {
+        return trafficType;
+    }
+
+    public String getIp4Address() {
+        return ip4Address;
+    }
+
+    public String getIp6Address() {
+        return ip6Address;
+    }
+    
+    public String getMacAddress() {
+        return macAddress;
+    }
+    
+    public void setIp4Address(String ip4Address) {
+        this.ip4Address = ip4Address;
+    }
+
+    public NicProfile(Nic nic, NetworkConfiguration network) {
+        this.id = nic.getId();
+        this.gateway = network.getGateway();
+        this.mode = network.getMode();
+        this.format = null;
+        this.broadcastType = network.getBroadcastDomainType();
+        this.trafficType = network.getTrafficType();
+        this.ip4Address = nic.getIp4Address();
+        this.ip6Address = null;
+        this.macAddress = nic.getMacAddress();
+        this.reservationId = nic.getReservationId();
+        this.strategy = nic.getReservationStrategy();
+    }
+
+    public NicProfile(long id, BroadcastDomainType type, Mode mode, long vmId) {
+        this.id = id;
+        this.broadcastType = type;
+        this.mode = mode;
+        this.vmId = vmId;
+    }
+    
+    public NicProfile(Resource.ReservationStrategy strategy, String ip4Address, String macAddress, String gateway, String netmask) {
+        this.format = AddressFormat.Ip4;
+        this.ip4Address = ip4Address;
+        this.macAddress = macAddress;
+        this.gateway = gateway;
+        this.netmask = netmask;
+        this.strategy = strategy;
+    }
+
+    public ReservationStrategy getReservationStrategy() {
+        return strategy;
+    }
+    
+    public String getReservationId() {
+        return reservationId;
+    }
+    
+    public void setReservationId(String reservationId) {
+        this.reservationId = reservationId;
+    }
+}
diff --git a/api/src/com/cloud/vm/RunningOn.java b/api/src/com/cloud/vm/RunningOn.java
new file mode 100644
index 00000000000..f491825dad1
--- /dev/null
+++ b/api/src/com/cloud/vm/RunningOn.java
@@ -0,0 +1,14 @@
+/**
+ * 
+ */
+package com.cloud.vm;
+
+/**
+ * RunningOn must be implemented by objects that runs on hosts.
+ *
+ */
+public interface RunningOn {
+    
+    Long getHostId();
+
+}
diff --git a/api/src/com/cloud/vm/State.java b/api/src/com/cloud/vm/State.java
index 81692497d9f..53bd798818f 100644
--- a/api/src/com/cloud/vm/State.java
+++ b/api/src/com/cloud/vm/State.java
@@ -19,10 +19,13 @@
 package com.cloud.vm;
 
 import java.util.List;
+import java.util.Set;
 
+import com.cloud.utils.fsm.FiniteState;
 import com.cloud.utils.fsm.StateMachine;
+import com.cloud.vm.VirtualMachine.Event;
 
-public enum State {
+public enum State implements FiniteState {
     Creating(true),
     Starting(true),
     Running(false),
@@ -44,22 +47,24 @@ public enum State {
     	return _transitional;
     }
     
-    public static String[] toStrings(State... states) {
-        String[] strs = new String[states.length];
-        for (int i = 0; i < states.length; i++) {
-            strs[i] = states[i].toString();
-        }
-        
-        return strs;
-    }
-    
+    @Override
     public State getNextState(VirtualMachine.Event e) {
         return s_fsm.getNextState(this, e);
     }
 
-    public State[] getFromStates(VirtualMachine.Event e) {
-        List from = s_fsm.getFromStates(this, e);
-        return from.toArray(new State[from.size()]);
+    @Override
+    public List getFromStates(VirtualMachine.Event e) {
+        return s_fsm.getFromStates(this, e);
+    }
+    
+    @Override
+    public Set getPossibleEvents() {
+        return s_fsm.getPossibleEvents(this);
+    }
+    
+    @Override
+    public StateMachine getStateMachine() {
+        return s_fsm;
     }
     
     protected static final StateMachine s_fsm = new StateMachine();
diff --git a/api/src/com/cloud/vm/VirtualMachine.java b/api/src/com/cloud/vm/VirtualMachine.java
index b8a7646a8d9..1984f83eac6 100755
--- a/api/src/com/cloud/vm/VirtualMachine.java
+++ b/api/src/com/cloud/vm/VirtualMachine.java
@@ -19,11 +19,14 @@ package com.cloud.vm;
 
 import java.util.Date;
 
+import com.cloud.domain.PartOf;
+import com.cloud.user.OwnedBy;
+
 /**
  * VirtualMachine describes the properties held by a virtual machine 
  *
  */
-public interface VirtualMachine {
+public interface VirtualMachine extends RunningOn, OwnedBy, PartOf {
     public enum Event {
     	CreateRequested,
     	StartRequested,
@@ -100,11 +103,6 @@ public interface VirtualMachine {
      */
     public long getDataCenterId();
     
-    /**
-     * @return id of the host it is running on.  If not running, returns null.
-     */
-    public Long getHostId();
-    
     /**
      * @return id of the host it was assigned last time.
      */
diff --git a/api/src/com/cloud/vm/VirtualMachineProfile.java b/api/src/com/cloud/vm/VirtualMachineProfile.java
new file mode 100644
index 00000000000..6d97f5d21c1
--- /dev/null
+++ b/api/src/com/cloud/vm/VirtualMachineProfile.java
@@ -0,0 +1,115 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.  
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.vm;
+
+import java.util.List;
+import java.util.Map;
+
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.offering.ServiceOffering;
+
+public class VirtualMachineProfile {
+    VirtualMachine _vm;
+    int _cpus;
+    int _speed; // in mhz
+    long _ram; // in bytes
+    Hypervisor.Type _hypervisorType;
+    VirtualMachine.Type _type;
+    Map _params;
+    Long _templateId;
+    List _disks;
+    List _nics;
+    
+    public VirtualMachineProfile(VirtualMachine.Type type) {
+        this._type = type;
+    }
+    
+    public long getId() {
+        return _vm.getId();
+    }
+    
+    public VirtualMachine.Type getType() {
+        return _type;
+    }
+    
+    public Long getTemplateId() {
+        return _templateId;
+    }
+    
+    public int getCpus() {
+        return _cpus;
+    }
+    
+    public int getSpeed() {
+        return _speed;
+    }
+    
+    public long getRam() {
+        return _ram;
+    }
+    
+    public void setNics(List profiles) {
+        this._nics = profiles;
+    }
+    
+    public List getNics() {
+        return _nics;
+    }
+    
+    public void setDisks(List profiles) {
+        this._disks = profiles;
+    }
+    
+    public List getDisks() {
+        return _disks;
+    }
+    
+    public Hypervisor.Type getHypervisorType() {
+        return _hypervisorType;
+    }
+    
+    public VirtualMachine getVm() {
+        return _vm;
+    }
+    
+    public VirtualMachineProfile(long id, int core, int speed, long ram, Long templateId, Hypervisor.Type type, Map params) {
+        this._cpus = core;
+        this._speed = speed;
+        this._ram = ram;
+        this._hypervisorType = type;
+        this._params = params;
+        this._templateId = templateId;
+    }
+    
+    public VirtualMachineProfile(VirtualMachine vm, ServiceOffering offering) {
+        this._cpus = offering.getCpu();
+        this._speed = offering.getSpeed();
+        this._ram = offering.getRamSize();
+        this._templateId = vm.getTemplateId();
+        this._type = vm.getType();
+        this._vm = vm;
+    }
+    
+    protected VirtualMachineProfile() {
+    }
+    
+    @Override
+    public String toString() {
+        return "VM-" + _type + "-" + _vm.getId();
+    }
+}
\ No newline at end of file
diff --git a/api/src/com/cloud/vm/VirtualMachineProfiler.java b/api/src/com/cloud/vm/VirtualMachineProfiler.java
new file mode 100644
index 00000000000..1156346bb22
--- /dev/null
+++ b/api/src/com/cloud/vm/VirtualMachineProfiler.java
@@ -0,0 +1,13 @@
+/**
+ * 
+ */
+package com.cloud.vm;
+
+import com.cloud.offering.ServiceOffering;
+import com.cloud.template.VirtualMachineTemplate;
+import com.cloud.utils.component.Adapter;
+
+public interface VirtualMachineProfiler extends Adapter {
+    
+    VirtualMachineProfile convert(ServiceOffering offering, VirtualMachineTemplate template);
+}
diff --git a/api/src/com/cloud/vm/VmCharacteristics.java b/api/src/com/cloud/vm/VmCharacteristics.java
deleted file mode 100644
index 6df26121c30..00000000000
--- a/api/src/com/cloud/vm/VmCharacteristics.java
+++ /dev/null
@@ -1,68 +0,0 @@
-/**
- *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
- * 
- * This software is licensed under the GNU General Public License v3 or later.  
- * 
- * It is free software: you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation, either version 3 of the License, or any later version.
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- * 
- * You should have received a copy of the GNU General Public License
- * along with this program.  If not, see .
- * 
- */
-package com.cloud.vm;
-
-import java.util.Map;
-
-import com.cloud.hypervisor.Hypervisor;
-
-public class VmCharacteristics {
-    int core;
-    int speed; // in mhz
-    long ram; // in bytes
-    Hypervisor.Type hypervisorType;
-    VirtualMachine.Type type;
-    
-    Map params;
-    
-    public VmCharacteristics(VirtualMachine.Type type) {
-        this.type = type;
-    }
-    
-    public VirtualMachine.Type getType() {
-        return type;
-    }
-    
-    
-    public VmCharacteristics() {
-    }
-    
-    public int getCores() {
-        return core;
-    }
-    
-    public int getSpeed() {
-        return speed;
-    }
-    
-    public long getRam() {
-        return ram;
-    }
-    
-    public Hypervisor.Type getHypervisorType() {
-        return hypervisorType;
-    }
-    
-    public VmCharacteristics(int core, int speed, long ram, Hypervisor.Type type, Map params) {
-        this.core = core;
-        this.speed = speed;
-        this.ram = ram;
-        this.hypervisorType = type;
-        this.params = params;
-    }
-}
\ No newline at end of file
diff --git a/build/.gitignore b/build/.gitignore
new file mode 100644
index 00000000000..ca06b0abecc
--- /dev/null
+++ b/build/.gitignore
@@ -0,0 +1 @@
+/override
diff --git a/build/build-cloud.xml b/build/build-cloud.xml
index 848af06df1e..1f9764271e3 100755
--- a/build/build-cloud.xml
+++ b/build/build-cloud.xml
@@ -140,7 +140,7 @@
   
 
   
-    
+    
     
       
     
diff --git a/build/build-common.xml b/build/build-common.xml
index 2131093314e..8038a62c78d 100755
--- a/build/build-common.xml
+++ b/build/build-common.xml
@@ -48,7 +48,7 @@
       
         
         
-        
+        
         
         
       
diff --git a/build/developer.xml b/build/developer.xml
index 9bb2d33e6e3..33d77a6c85a 100755
--- a/build/developer.xml
+++ b/build/developer.xml
@@ -27,6 +27,18 @@
   
   
   
+
+  
+	   
+    
+	  
+	    
+	    
+	  
+	  
+    
+    
+  
   
   
     
@@ -86,23 +98,11 @@
   
     
   
-  
+    
   
-    
+    
   
 
-
    
     
       
@@ -114,7 +114,6 @@
         
       
     
-    
     
     
     
@@ -169,11 +168,18 @@
       
     
 
+    
+      
+    
 
-    
+    
       
     
 
+    
+      
+    
+  	
     
     
       
diff --git a/client/tomcatconf/commands.properties.in b/client/tomcatconf/commands.properties.in
old mode 100644
new mode 100755
index fb62d18615e..c8ae73f15d1
--- a/client/tomcatconf/commands.properties.in
+++ b/client/tomcatconf/commands.properties.in
@@ -61,6 +61,7 @@ deleteTemplate=com.cloud.api.commands.DeleteTemplateCmd;15
 listTemplates=com.cloud.api.commands.ListTemplatesCmd;15
 updateTemplatePermissions=com.cloud.api.commands.UpdateTemplatePermissionsCmd;15
 listTemplatePermissions=com.cloud.api.commands.ListTemplatePermissionsCmd;15
+extractTemplate=com.cloud.api.commands.ExtractTemplateCmd;15
 
 #### iso commands
 attachIso=com.cloud.api.commands.AttachIsoCmd;15
@@ -72,6 +73,7 @@ deleteIso=com.cloud.api.commands.DeleteIsoCmd;15
 copyIso=com.cloud.api.commands.CopyIsoCmd;15
 updateIsoPermissions=com.cloud.api.commands.UpdateIsoPermissionsCmd;15
 listIsoPermissions=com.cloud.api.commands.ListIsoPermissionsCmd;15
+extractIso=com.cloud.api.commands.ExtractIsoCmd;15
 
 #### guest OS commands
 listOsTypes=com.cloud.api.commands.ListGuestOsCmd;15
@@ -139,6 +141,7 @@ listSystemVms=com.cloud.api.commands.ListSystemVMsCmd;1
 updateConfiguration=com.cloud.api.commands.UpdateCfgCmd;1
 listConfigurations=com.cloud.api.commands.ListCfgsByCmd;1
 addConfig=com.cloud.api.commands.AddConfigCmd;15
+listCapabilities=com.cloud.api.commands.ListCapabilitiesCmd;15
 
 #### pod commands
 createPod=com.cloud.api.commands.CreatePodCmd;1
@@ -177,6 +180,7 @@ detachVolume=com.cloud.api.commands.DetachVolumeCmd;15
 createVolume=com.cloud.api.commands.CreateVolumeCmd;15
 deleteVolume=com.cloud.api.commands.DeleteVolumeCmd;15
 listVolumes=com.cloud.api.commands.ListVolumesCmd;15
+extractVolume=com.cloud.api.commands.ExtractVolumeCmd;15
 
 #### registration command:  FIXME -- this really should be something in management server that
 ####                                 generates a new key for the user and they just have to
@@ -207,4 +211,11 @@ listNetworkGroups=com.cloud.api.commands.ListNetworkGroupsCmd;11
 
 registerPreallocatedLun=com.cloud.server.api.commands.RegisterPreallocatedLunCmd;1
 deletePreallocatedLun=com.cloud.server.api.commands.DeletePreallocatedLunCmd;1
-listPreallocatedLuns=com.cloud.api.commands.ListPreallocatedLunsCmd;1
\ No newline at end of file
+listPreallocatedLuns=com.cloud.api.commands.ListPreallocatedLunsCmd;1
+
+
+#### vm group commands
+createInstanceGroup=com.cloud.api.commands.CreateVMGroupCmd;15
+deleteInstanceGroup=com.cloud.api.commands.DeleteVMGroupCmd;15
+updateInstanceGroup=com.cloud.api.commands.UpdateVMGroupCmd;15
+listInstanceGroups=com.cloud.api.commands.ListVMGroupsCmd;15
diff --git a/client/tomcatconf/components.xml.in b/client/tomcatconf/components.xml.in
index 293f580c184..9e8733eb886 100755
--- a/client/tomcatconf/components.xml.in
+++ b/client/tomcatconf/components.xml.in
@@ -88,6 +88,7 @@
 	
         
         
+        
         
         
         
@@ -110,7 +111,12 @@
         
         
         
-        
+        
+        
+        
+        
+        
+
         
             
             
@@ -146,7 +152,8 @@
 
         
             
-            
+            
+            
         
 
         
@@ -166,6 +173,8 @@
         
         
         
+        
+        
         
         
         
@@ -209,8 +218,9 @@
         
 		
 		
-		
-        
+		
+        
+        
         
         
 		
diff --git a/client/wscript_build b/client/wscript_build
new file mode 100644
index 00000000000..3c153257032
--- /dev/null
+++ b/client/wscript_build
@@ -0,0 +1,11 @@
+import Options
+
+start_path = bld.path.find_dir("WEB-INF")
+bld.install_files('${MSENVIRON}/webapps/client/WEB-INF',
+	start_path.ant_glob("**",src=True,bld=False,dir=False,flat=True),
+	cwd=start_path,relative_trick=True)
+
+if not Options.options.PRESERVECONFIG:
+	bld.install_files_filtered("${MSCONF}","tomcatconf/*")
+	bld.install_files("${MSCONF}",'tomcatconf/db.properties',chmod=0640)
+	bld.setownership("${MSCONF}/db.properties","root",bld.env.MSUSER)
diff --git a/cloud.spec b/cloud.spec
index 7f8ee77a47a..3d03ea82383 100644
--- a/cloud.spec
+++ b/cloud.spec
@@ -34,6 +34,8 @@ BuildRequires: commons-httpclient
 BuildRequires: jpackage-utils
 BuildRequires: gcc
 BuildRequires: glibc-devel
+BuildRequires: /usr/bin/mkisofs
+BuildRequires: MySQL-python
 
 %global _premium %(tar jtvmf %{SOURCE0} '*/cloudstack-proprietary/' --occurrence=1 2>/dev/null | wc -l)
 
@@ -44,6 +46,7 @@ intelligent cloud implementation.
 %package utils
 Summary:   Cloud.com utility library
 Requires: java >= 1.6.0
+Requires: python
 Group:     System Environment/Libraries
 Obsoletes: vmops-utils < %{version}-%{release}
 %description utils
@@ -180,12 +183,11 @@ Summary:   Cloud.com setup tools
 Obsoletes: vmops-setup < %{version}-%{release}
 Requires: java >= 1.6.0
 Requires: python
-Requires: mysql
+Requires: MySQL-python
 Requires: %{name}-utils = %{version}-%{release}
 Requires: %{name}-server = %{version}-%{release}
 Requires: %{name}-deps = %{version}-%{release}
 Requires: %{name}-python = %{version}-%{release}
-Requires: MySQL-python
 Group:     System Environment/Libraries
 %description setup
 The Cloud.com setup tools let you set up your Management Server and Usage Server.
@@ -231,6 +233,9 @@ Requires: %{name}-daemonize
 Requires: /sbin/service
 Requires: /sbin/chkconfig
 Requires: kvm
+%if 0%{?fedora} >= 12
+Requires: cloud-qemu-system-x86
+%endif
 Requires: libcgroup
 Requires: /usr/bin/uuidgen
 Requires: augeas >= 0.7.1
@@ -368,7 +373,6 @@ if [ "$1" == "1" ] ; then
     /sbin/chkconfig --add %{name}-management > /dev/null 2>&1 || true
     /sbin/chkconfig --level 345 %{name}-management on > /dev/null 2>&1 || true
 fi
-test -f %{_sharedstatedir}/%{name}/management/.ssh/id_rsa || su - %{name} -c 'yes "" 2>/dev/null | ssh-keygen -t rsa -q -N ""' < /dev/null
 
 
 
@@ -447,82 +451,43 @@ fi
 %defattr(0644,root,root,0755)
 %{_javadir}/%{name}-utils.jar
 %{_javadir}/%{name}-api.jar
+%attr(755,root,root) %{_bindir}/cloud-sccs
+%attr(755,root,root) %{_bindir}/cloud-gitrevs
 %doc %{_docdir}/%{name}-%{version}/sccs-info
 %doc %{_docdir}/%{name}-%{version}/version-info
 %doc %{_docdir}/%{name}-%{version}/configure-info
-%doc README
-%doc INSTALL
-%doc HACKING
 %doc README.html
 %doc debian/copyright
 
 %files client-ui
 %defattr(0644,root,root,0755)
 %{_datadir}/%{name}/management/webapps/client/*
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
 
 %files server
 %defattr(0644,root,root,0755)
 %{_javadir}/%{name}-server.jar
 %{_sysconfdir}/%{name}/server/*
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
-
-%if %{_premium}
 
 %files agent-scripts
 %defattr(-,root,root,-)
 %{_libdir}/%{name}/agent/scripts/*
-%{_libdir}/%{name}/agent/vms/systemvm.zip
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
-
-%else
-
-%files agent-scripts
-%defattr(-,root,root,-)
-%{_libdir}/%{name}/agent/scripts/installer/*
-%{_libdir}/%{name}/agent/scripts/network/domr/*.sh
-%{_libdir}/%{name}/agent/scripts/storage/*.sh
-%{_libdir}/%{name}/agent/scripts/storage/zfs/*
-%{_libdir}/%{name}/agent/scripts/storage/qcow2/*
-%{_libdir}/%{name}/agent/scripts/storage/secondary/*
-%{_libdir}/%{name}/agent/scripts/util/*
-%{_libdir}/%{name}/agent/scripts/vm/*.sh
-%{_libdir}/%{name}/agent/scripts/vm/storage/nfs/*
-%{_libdir}/%{name}/agent/scripts/vm/storage/iscsi/*
-%{_libdir}/%{name}/agent/scripts/vm/network/*
-%{_libdir}/%{name}/agent/scripts/vm/hypervisor/*.sh
-%{_libdir}/%{name}/agent/scripts/vm/hypervisor/kvm/*
-%{_libdir}/%{name}/agent/scripts/vm/hypervisor/xen/*
-%{_libdir}/%{name}/agent/vms/systemvm.zip
-%{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/*
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
-
+# maintain the following list in sync with files agent-scripts
+%if %{_premium}
+%exclude %{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/check_heartbeat.sh
+%exclude %{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/find_bond.sh
+%exclude %{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/launch_hb.sh
+%exclude %{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/setup_heartbeat_sr.sh
+%exclude %{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/vmopspremium
+%exclude %{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/xenheartbeat.sh
+%exclude %{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/xenserver56/patch-premium
+%exclude %{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/xs_cleanup.sh
 %endif
+%{_libdir}/%{name}/agent/vms/systemvm.zip
+%{_libdir}/%{name}/agent/vms/systemvm.iso
 
 %files daemonize
 %defattr(-,root,root,-)
 %attr(755,root,root) %{_bindir}/%{name}-daemonize
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
 
 %files deps
 %defattr(0644,root,root,0755)
@@ -543,39 +508,20 @@ fi
 %{_javadir}/%{name}-xenserver-5.5.0-1.jar
 %{_javadir}/%{name}-xmlrpc-common-3.*.jar
 %{_javadir}/%{name}-xmlrpc-client-3.*.jar
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
 
 %files core
 %defattr(0644,root,root,0755)
 %{_javadir}/%{name}-core.jar
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc debian/copyright
 
 %files vnet
 %defattr(0644,root,root,0755)
 %attr(0755,root,root) %{_sbindir}/%{name}-vnetd
 %attr(0755,root,root) %{_sbindir}/%{name}-vn
 %attr(0755,root,root) %{_initrddir}/%{name}-vnetd
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
 
 %files python
 %defattr(0644,root,root,0755)
 %{_prefix}/lib*/python*/site-packages/%{name}*
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
 
 %files setup
 %attr(0755,root,root) %{_bindir}/%{name}-setup-databases
@@ -585,19 +531,18 @@ fi
 %{_datadir}/%{name}/setup/create-index-fk.sql
 %{_datadir}/%{name}/setup/create-schema.sql
 %{_datadir}/%{name}/setup/server-setup.sql
-%{_datadir}/%{name}/setup/templates.kvm.sql
-%{_datadir}/%{name}/setup/templates.xenserver.sql
+%{_datadir}/%{name}/setup/templates.*.sql
 %{_datadir}/%{name}/setup/deploy-db-dev.sh
 %{_datadir}/%{name}/setup/server-setup.xml
 %{_datadir}/%{name}/setup/data-20to21.sql
 %{_datadir}/%{name}/setup/index-20to21.sql
+%{_datadir}/%{name}/setup/index-212to213.sql
 %{_datadir}/%{name}/setup/postprocess-20to21.sql
 %{_datadir}/%{name}/setup/schema-20to21.sql
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
+%{_datadir}/%{name}/setup/schema-level.sql
+%{_datadir}/%{name}/setup/schema-21to22.sql
+%{_datadir}/%{name}/setup/data-21to22.sql
+%{_datadir}/%{name}/setup/index-21to22.sql
 
 %files client
 %defattr(0644,root,root,0755)
@@ -637,19 +582,10 @@ fi
 %dir %attr(770,root,%{name}) %{_localstatedir}/cache/%{name}/management/temp
 %dir %attr(770,root,%{name}) %{_localstatedir}/log/%{name}/management
 %dir %attr(770,root,%{name}) %{_localstatedir}/log/%{name}/agent
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
 
 %files agent-libs
 %defattr(0644,root,root,0755)
 %{_javadir}/%{name}-agent.jar
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc debian/copyright
 
 %files agent
 %defattr(0644,root,root,0755)
@@ -665,11 +601,6 @@ fi
 %{_libdir}/%{name}/agent/images
 %attr(0755,root,root) %{_bindir}/%{name}-setup-agent
 %dir %attr(770,root,root) %{_localstatedir}/log/%{name}/agent
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
 
 %files console-proxy
 %defattr(0644,root,root,0755)
@@ -682,11 +613,6 @@ fi
 %{_libdir}/%{name}/console-proxy/*
 %attr(0755,root,root) %{_bindir}/%{name}-setup-console-proxy
 %dir %attr(770,root,root) %{_localstatedir}/log/%{name}/console-proxy
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
 
 %if %{_premium}
 
@@ -697,20 +623,10 @@ fi
 %{_sharedstatedir}/%{name}/test/*
 %{_libdir}/%{name}/test/*
 %{_sysconfdir}/%{name}/test/*
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
 
 %files premium-deps
 %defattr(0644,root,root,0755)
 %{_javadir}/%{name}-premium/*.jar
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
 
 %files premium
 %defattr(0644,root,root,0755)
@@ -718,15 +634,18 @@ fi
 %{_javadir}/%{name}-server-extras.jar
 %{_sysconfdir}/%{name}/management/commands-ext.properties
 %{_sysconfdir}/%{name}/management/components-premium.xml
-%{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/*
-%{_libdir}/%{name}/agent/vms/systemvm-premium.zip
+%{_libdir}/%{name}/agent/vms/systemvm-premium.iso
 %{_datadir}/%{name}/setup/create-database-premium.sql
 %{_datadir}/%{name}/setup/create-schema-premium.sql
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
+# maintain the following list in sync with files agent-scripts
+%{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/check_heartbeat.sh
+%{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/find_bond.sh
+%{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/launch_hb.sh
+%{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/setup_heartbeat_sr.sh
+%{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/vmopspremium
+%{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/xenheartbeat.sh
+%{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/xenserver56/patch-premium
+%{_libdir}/%{name}/agent/scripts/vm/hypervisor/xenserver/xs_cleanup.sh
 
 %files usage
 %defattr(0644,root,root,0755)
@@ -737,11 +656,6 @@ fi
 %{_sysconfdir}/%{name}/usage/usage-components.xml
 %config(noreplace) %{_sysconfdir}/%{name}/usage/log4j-%{name}_usage.xml
 %config(noreplace) %attr(640,root,%{name}) %{_sysconfdir}/%{name}/usage/db.properties
-%doc README
-%doc INSTALL
-%doc HACKING
-%doc README.html
-%doc debian/copyright
 
 %endif
 
diff --git a/console-proxy/bindir/cloud-setup-console-proxy.in b/console-proxy/bindir/cloud-setup-console-proxy.in
index 9079c229ebb..9bf55bc2f09 100755
--- a/console-proxy/bindir/cloud-setup-console-proxy.in
+++ b/console-proxy/bindir/cloud-setup-console-proxy.in
@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 
-import sys, os, subprocess, errno, re
+import sys, os, subprocess, errno, re, getopt
 
 # ---- This snippet of code adds the sources path and the waf configured PYTHONDIR to the Python path ----
 # ---- We do this so cloud_utils can be looked up in the following order:
@@ -14,6 +14,7 @@ for pythonpath in (
 		if os.path.isdir(pythonpath): sys.path.insert(0,pythonpath)
 # ---- End snippet of code ----
 import cloud_utils
+from cloud_utils import stderr
 
 E_GENERIC= 1
 E_NOKVM = 2
@@ -27,13 +28,6 @@ E_CPRECONFIGFAILED = 9
 E_CPFAILEDTOSTART = 10
 E_NOFQDN = 11
 
-
-def stderr(msgfmt,*args):
-	msgfmt += "\n"
-	if args: sys.stderr.write(msgfmt%args)
-	else: sys.stderr.write(msgfmt)
-	sys.stderr.flush()
-
 def bail(errno=E_GENERIC,message=None,*args):
 	if message: stderr(message,*args)
 	stderr("Cloud Console Proxy setup aborted")
@@ -132,9 +126,30 @@ CentOS = os.path.exists("/etc/centos-release") or ( os.path.exists("/etc/redhat-
 #--------------- procedure starts here ------------
 
 def main():
-	
+	# parse cmd line	
+	opts, args = getopt.getopt(sys.argv[1:], "a", ["host=", "zone=", "pod="])
+	host=None
+	zone=None
+	pod=None
+    	autoMode=False
+	do_check_kvm = True	
+	for opt, arg in opts:
+		if opt == "--host":
+			if arg != "":
+				host = arg
+		elif opt == "--zone":
+			if arg != "":
+				zone = arg
+		elif opt == "--pod":
+		        if arg != "":
+				pod = arg
+        	elif opt == "-a":
+            		autoMode=True
 	servicename = "@PACKAGE@-console-proxy"
 	
+	if autoMode:
+		cloud_utils.setLogFile("/var/log/cloud/setupConsoleProxy.log")
+
 	stderr("Welcome to the Cloud Console Proxy setup")
 	stderr("")
 
@@ -176,7 +191,7 @@ def main():
 			print e.stdout+e.stderr
 			bail(E_FWRECONFIGFAILED,"Firewall could not be enabled")
 
-	cloud_utils.setup_consoleproxy_config("@CPSYSCONFDIR@/agent.properties")
+	cloud_utils.setup_consoleproxy_config("@CPSYSCONFDIR@/agent.properties", host, zone, pod)
 	stderr("Enabling and starting the Cloud Console Proxy")
 	cloud_utils.enable_service(servicename)
 	stderr("Cloud Console Proxy restarted")
diff --git a/console-proxy/scripts/_run.sh b/console-proxy/scripts/_run.sh
new file mode 100755
index 00000000000..e90005d2ebb
--- /dev/null
+++ b/console-proxy/scripts/_run.sh
@@ -0,0 +1,51 @@
+#!/usr/bin/env bash
+#run.sh runs the console proxy.
+
+# make sure we delete the old files from the original template 
+rm console-proxy.jar
+rm console-common.jar
+rm conf/cloud.properties
+
+set -x
+
+CP=./:./conf
+for file in *.jar
+do
+  CP=${CP}:$file
+done
+keyvalues=
+if [ -f /mnt/cmdline ]
+then
+    CMDLINE=$(cat /mnt/cmdline)
+else
+    CMDLINE=$(cat /proc/cmdline)
+fi
+#CMDLINE="graphical utf8 eth0ip=0.0.0.0 eth0mask=255.255.255.0 eth1ip=192.168.140.40 eth1mask=255.255.255.0 eth2ip=172.24.0.50 eth2mask=255.255.0.0 gateway=172.24.0.1 dns1=72.52.126.11 template=domP dns2=72.52.126.12 host=192.168.1.142 port=8250 mgmtcidr=192.168.1.0/24 localgw=192.168.140.1 zone=5 pod=5"
+for i in $CMDLINE
+  do
+     KEY=$(echo $i | cut -s -d= -f1)
+     VALUE=$(echo $i | cut -s -d= -f2)
+     [ "$KEY" == "" ] && continue
+     case $KEY in
+        *)
+          keyvalues="${keyvalues} $KEY=$VALUE"
+     esac
+  done
+   
+tot_mem_k=$(cat /proc/meminfo | grep MemTotal | awk '{print $2}')
+let "tot_mem_m=tot_mem_k>>10"
+let "eightypcnt=$tot_mem_m*8/10"
+let "maxmem=$tot_mem_m-80"
+
+if [ $maxmem -gt $eightypcnt ]
+then
+  maxmem=$eightypcnt
+fi
+
+EXTRA=
+if [ -f certs/realhostip.keystore ]
+then
+  EXTRA="-Djavax.net.ssl.trustStore=$(dirname $0)/certs/realhostip.keystore -Djavax.net.ssl.trustStorePassword=vmops.com"
+fi
+
+java -mx${maxmem}m ${EXTRA} -cp $CP com.cloud.agent.AgentShell $keyvalues $@
diff --git a/console-proxy/scripts/config_auth.sh b/console-proxy/scripts/config_auth.sh
index 893920d2be2..503c90f1d0a 100755
--- a/console-proxy/scripts/config_auth.sh
+++ b/console-proxy/scripts/config_auth.sh
@@ -2,7 +2,12 @@
 
 BASE_DIR="/var/www/html/copy/template/"
 HTACCESS="$BASE_DIR/.htaccess"
+
 PASSWDFILE="/etc/httpd/.htpasswd"
+if [ -d /etc/apache2 ]
+then
+  PASSWDFILE="/etc/apache2/.htpasswd"
+fi
 
 config_htaccess() {
   mkdir -p $BASE_DIR
diff --git a/console-proxy/scripts/config_ssl.sh b/console-proxy/scripts/config_ssl.sh
index a3be8d32dff..ef59852d69b 100755
--- a/console-proxy/scripts/config_ssl.sh
+++ b/console-proxy/scripts/config_ssl.sh
@@ -15,6 +15,17 @@ config_httpd_conf() {
   echo "" >> /etc/httpd/conf/httpd.conf
 }
 
+config_apache2_conf() {
+  local ip=$1
+  local srvr=$2
+  cp -f /etc/apache2/sites-available/default.orig /etc/apache2/sites-available/default
+  cp -f /etc/apache2/sites-available/default-ssl.orig /etc/apache2/sites-available/default-ssl
+  sed -i -e "s/VirtualHost.*:80$/VirtualHost $ip:80/" /etc/httpd/conf/httpd.conf
+  sed -i  's/_default_/$ip/' /etc/apache2/sites-available/default-ssl
+  sed -i  's/ssl-cert-snakeoil.key/realhostip.key/' /etc/apache2/sites-available/default-ssl
+  sed -i  's/ssl-cert-snakeoil.pem/realhostip.crt/' /etc/apache2/sites-available/default-ssl
+}
+
 copy_certs() {
   local certdir=$(dirname $0)/certs
   local mydir=$(dirname $0)
@@ -25,16 +36,37 @@ copy_certs() {
   return 1
 }
 
+copy_certs_apache2() {
+  local certdir=$(dirname $0)/certs
+  local mydir=$(dirname $0)
+  if [ -d $certdir ] && [ -f $certdir/realhostip.key ] &&  [ -f $certdir/realhostip.crt ] ; then
+      cp $certdir/realhostip.key /etc/ssl/private/   &&  cp $certdir/realhostip.crt /etc/ssl/certs/
+      return $?
+  fi
+  return 1
+}
+
 if [ $# -ne 2 ] ; then
 	echo $"Usage: `basename $0` ipaddr servername "
 	exit 0
 fi
 
-copy_certs
+if [ -d /etc/apache2 ]
+then
+  copy_certs_apache2
+else
+  copy_certs
+fi
+
 if [ $? -ne 0 ]
 then
   echo "Failed to copy certificates"
   exit 2
 fi
 
-config_httpd_conf $1 $2
+if [ -d /etc/apache2 ]
+then
+  config_apache2_conf $1 $2
+else
+  config_httpd_conf $1 $2
+fi
diff --git a/console-proxy/scripts/run.sh b/console-proxy/scripts/run.sh
index e90005d2ebb..4ea7608a8f5 100755
--- a/console-proxy/scripts/run.sh
+++ b/console-proxy/scripts/run.sh
@@ -1,51 +1,14 @@
-#!/usr/bin/env bash
-#run.sh runs the console proxy.
+#!/bin/bash
+#_run.sh runs the agent client.
 
-# make sure we delete the old files from the original template 
-rm console-proxy.jar
-rm console-common.jar
-rm conf/cloud.properties
-
-set -x
-
-CP=./:./conf
-for file in *.jar
+# set -x
+ 
+while true
 do
-  CP=${CP}:$file
+  ./_run.sh "$@"
+  ex=$?
+  if [ $ex -eq 0 ] || [ $ex -eq 1 ] || [ $ex -eq 66 ] || [ $ex -gt 128 ]; then
+    exit $ex
+  fi
+  sleep 20
 done
-keyvalues=
-if [ -f /mnt/cmdline ]
-then
-    CMDLINE=$(cat /mnt/cmdline)
-else
-    CMDLINE=$(cat /proc/cmdline)
-fi
-#CMDLINE="graphical utf8 eth0ip=0.0.0.0 eth0mask=255.255.255.0 eth1ip=192.168.140.40 eth1mask=255.255.255.0 eth2ip=172.24.0.50 eth2mask=255.255.0.0 gateway=172.24.0.1 dns1=72.52.126.11 template=domP dns2=72.52.126.12 host=192.168.1.142 port=8250 mgmtcidr=192.168.1.0/24 localgw=192.168.140.1 zone=5 pod=5"
-for i in $CMDLINE
-  do
-     KEY=$(echo $i | cut -s -d= -f1)
-     VALUE=$(echo $i | cut -s -d= -f2)
-     [ "$KEY" == "" ] && continue
-     case $KEY in
-        *)
-          keyvalues="${keyvalues} $KEY=$VALUE"
-     esac
-  done
-   
-tot_mem_k=$(cat /proc/meminfo | grep MemTotal | awk '{print $2}')
-let "tot_mem_m=tot_mem_k>>10"
-let "eightypcnt=$tot_mem_m*8/10"
-let "maxmem=$tot_mem_m-80"
-
-if [ $maxmem -gt $eightypcnt ]
-then
-  maxmem=$eightypcnt
-fi
-
-EXTRA=
-if [ -f certs/realhostip.keystore ]
-then
-  EXTRA="-Djavax.net.ssl.trustStore=$(dirname $0)/certs/realhostip.keystore -Djavax.net.ssl.trustStorePassword=vmops.com"
-fi
-
-java -mx${maxmem}m ${EXTRA} -cp $CP com.cloud.agent.AgentShell $keyvalues $@
diff --git a/console-proxy/wscript_build b/console-proxy/wscript_build
new file mode 100644
index 00000000000..d5221157768
--- /dev/null
+++ b/console-proxy/wscript_build
@@ -0,0 +1,10 @@
+import Options
+
+# binary unsubstitutable files:
+bld.install_files("${CPLIBDIR}",bld.path.ant_glob("images/**",src=True,bld=False,dir=False,flat=True),cwd=bld.path,relative_trick=True)
+
+# text substitutable files (substitute with tokens from the environment bld.env):
+bld.substitute('css/** js/** ui/** scripts/**',install_to="${CPLIBDIR}")
+
+# config files (do not replace them if preserve config option is true)
+if not Options.options.PRESERVECONFIG: bld.install_files_filtered("${CPSYSCONFDIR}","conf.dom0/*")
diff --git a/core/.classpath b/core/.classpath
index c50df24555a..d21c3e31c11 100644
--- a/core/.classpath
+++ b/core/.classpath
@@ -1,38 +1,44 @@
-
-
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-	
-
+
+
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+
diff --git a/core/src/com/cloud/agent/api/AbstractStartCommand.java b/core/src/com/cloud/agent/api/AbstractStartCommand.java
index b2a943ad5f0..e9f31204d15 100644
--- a/core/src/com/cloud/agent/api/AbstractStartCommand.java
+++ b/core/src/com/cloud/agent/api/AbstractStartCommand.java
@@ -21,7 +21,7 @@ package com.cloud.agent.api;
 import java.util.List;
 
 import com.cloud.storage.VolumeVO;
-import com.cloud.storage.VirtualMachineTemplate.BootloaderType;
+import com.cloud.template.VirtualMachineTemplate.BootloaderType;
 
 public abstract class AbstractStartCommand extends Command {
 	
diff --git a/core/src/com/cloud/agent/api/Start2Answer.java b/core/src/com/cloud/agent/api/Start2Answer.java
new file mode 100644
index 00000000000..7b6ec6eb740
--- /dev/null
+++ b/core/src/com/cloud/agent/api/Start2Answer.java
@@ -0,0 +1,21 @@
+/**
+ * 
+ */
+package com.cloud.agent.api;
+
+public class Start2Answer extends Answer {
+    protected Start2Answer() {
+    }
+    
+    public Start2Answer(Start2Command cmd, String msg) {
+        super(cmd, false, msg);
+    }
+    
+    public Start2Answer(Start2Command cmd, Exception e) {
+        super(cmd, false, e.getMessage());
+    }
+    
+    public Start2Answer(Start2Command cmd) {
+        super(cmd, true, null);
+    }
+}
diff --git a/core/src/com/cloud/agent/api/Start2Command.java b/core/src/com/cloud/agent/api/Start2Command.java
new file mode 100644
index 00000000000..4414b888d44
--- /dev/null
+++ b/core/src/com/cloud/agent/api/Start2Command.java
@@ -0,0 +1,62 @@
+/**
+ * 
+ */
+package com.cloud.agent.api;
+
+import com.cloud.agent.api.to.VirtualMachineTO;
+
+public class Start2Command extends Command {
+    VirtualMachineTO vm;
+    
+    public VirtualMachineTO getVirtualMachine() {
+        return vm;
+    }
+    
+    /*
+    long id;
+    String guestIpAddress;
+    String gateway;
+    int ramSize;
+    String imagePath;
+    String guestNetworkId;
+    String guestMacAddress;
+    String vncPassword;
+    String externalVlan;
+    String externalMacAddress;
+    int utilization;
+    int cpuWeight;
+    int cpu;
+    int networkRateMbps;
+    int networkRateMulticastMbps;
+    String hostName;
+    String arch;
+    String isoPath;
+    boolean bootFromISO;
+    String guestOSDescription;
+    
+    ---->console proxy
+    private ConsoleProxyVO proxy;
+    private int proxyCmdPort;
+    private String vncPort;
+    private String urlPort;
+    private String mgmt_host;
+    private int mgmt_port;
+    private boolean sslEnabled;
+
+    ----->abstract
+    protected String vmName;
+    protected String storageHosts[] = new String[2];
+    protected List volumes;
+    protected boolean mirroredVols = false;
+    protected BootloaderType bootloader = BootloaderType.PyGrub;
+    
+     */
+    
+    @Override
+    public boolean executeInSequence() {
+        return true;
+    }
+    
+    public Start2Command() {
+    }
+}
diff --git a/core/src/com/cloud/agent/api/StartConsoleProxyCommand.java b/core/src/com/cloud/agent/api/StartConsoleProxyCommand.java
index 9b6faa4b893..c1a0c31bb6e 100644
--- a/core/src/com/cloud/agent/api/StartConsoleProxyCommand.java
+++ b/core/src/com/cloud/agent/api/StartConsoleProxyCommand.java
@@ -25,7 +25,9 @@ import com.cloud.vm.ConsoleProxyVO;
 
 public class StartConsoleProxyCommand extends AbstractStartCommand {
 
-    private ConsoleProxyVO proxy;
+    private ConsoleProxyVO proxy;
+    int networkRateMbps;
+    int networkRateMulticastMbps;
     private int proxyCmdPort;
     private String vncPort;
     private String urlPort;
@@ -36,9 +38,12 @@ public class StartConsoleProxyCommand extends AbstractStartCommand {
 	protected StartConsoleProxyCommand() {
 	}
 	
-    public StartConsoleProxyCommand(int proxyCmdPort, ConsoleProxyVO proxy, String vmName, String storageHost, 
+    public StartConsoleProxyCommand(int networkRateMbps, int networkRateMulticastMbps, int proxyCmdPort, 
+            ConsoleProxyVO proxy, String vmName, String storageHost, 
     		List vols, String vncPort, String urlPort, String mgmtHost, int mgmtPort, boolean sslEnabled) {
-    	super(vmName, storageHost, vols);
+    	super(vmName, storageHost, vols);
+    	this.networkRateMbps = networkRateMbps;
+    	this.networkRateMulticastMbps = networkRateMulticastMbps;
     	this.proxyCmdPort = proxyCmdPort;
     	this.proxy = proxy;
     	this.vncPort = vncPort;
@@ -57,7 +62,15 @@ public class StartConsoleProxyCommand extends AbstractStartCommand {
 		return proxy;
 	}
 	
-	public int getProxyCmdPort() {
+	public int getNetworkRateMbps() {
+        return networkRateMbps;
+    }
+
+    public int getNetworkRateMulticastMbps() {
+        return networkRateMulticastMbps;
+    }
+
+    public int getProxyCmdPort() {
 		return proxyCmdPort;
 	}
 	
diff --git a/core/src/com/cloud/agent/api/StartRouterCommand.java b/core/src/com/cloud/agent/api/StartRouterCommand.java
index ec8ca7a808a..913745273ca 100755
--- a/core/src/com/cloud/agent/api/StartRouterCommand.java
+++ b/core/src/com/cloud/agent/api/StartRouterCommand.java
@@ -28,6 +28,8 @@ import com.cloud.vm.DomainRouter.Role;
 public class StartRouterCommand extends AbstractStartCommand {
 
     DomainRouterVO router;
+    int networkRateMbps;
+    int networkRateMulticastMbps;
 
     protected StartRouterCommand() {
     	super();
@@ -38,16 +40,27 @@ public class StartRouterCommand extends AbstractStartCommand {
         return true;
     }
     
-    public StartRouterCommand(DomainRouterVO router, String routerName, String[] storageIps, List vols, boolean mirroredVols) {
+    public StartRouterCommand(DomainRouterVO router, int networkRateMbps, int networkRateMulticastMbps, 
+            String routerName, String[] storageIps, List vols, boolean mirroredVols) {
         super(routerName, storageIps, vols, mirroredVols);
         this.router = router;
+        this.networkRateMbps = networkRateMbps;
+        this.networkRateMulticastMbps = networkRateMulticastMbps;
 	}
 
 	public DomainRouter getRouter() {
         return router;
     }
 	
-	public String getBootArgs() {
+	public int getNetworkRateMbps() {
+        return networkRateMbps;
+    }
+
+    public int getNetworkRateMulticastMbps() {
+        return networkRateMulticastMbps;
+    }
+
+    public String getBootArgs() {
 		String eth2Ip = router.getPublicIpAddress()==null?"0.0.0.0":router.getPublicIpAddress();
 		String basic = " eth0ip=" + router.getGuestIpAddress() + " eth0mask=" + router.getGuestNetmask() + " eth1ip="
         + router.getPrivateIpAddress() + " eth1mask=" + router.getPrivateNetmask() + " gateway=" + router.getGateway()
diff --git a/core/src/com/cloud/agent/api/StartSecStorageVmCommand.java b/core/src/com/cloud/agent/api/StartSecStorageVmCommand.java
index c41070eb028..19c0f30882e 100644
--- a/core/src/com/cloud/agent/api/StartSecStorageVmCommand.java
+++ b/core/src/com/cloud/agent/api/StartSecStorageVmCommand.java
@@ -27,7 +27,9 @@ import com.cloud.vm.SecondaryStorageVmVO;
 
 public class StartSecStorageVmCommand extends AbstractStartCommand {
 
-    private SecondaryStorageVmVO secStorageVm;
+    private SecondaryStorageVmVO secStorageVm;
+    int networkRateMbps;
+    int networkRateMulticastMbps;
     private int proxyCmdPort;
     private String mgmt_host;
     private int mgmt_port;
@@ -36,9 +38,12 @@ public class StartSecStorageVmCommand extends AbstractStartCommand {
 	protected StartSecStorageVmCommand() {
 	}
 	
-    public StartSecStorageVmCommand(int proxyCmdPort, SecondaryStorageVmVO secStorageVm, String vmName, String storageHost,
+    public StartSecStorageVmCommand(int networkRateMbps, int networkRateMulticastMbps, int proxyCmdPort, 
+            SecondaryStorageVmVO secStorageVm, String vmName, String storageHost,
     		List vols, String mgmtHost, int mgmtPort, boolean sslCopy) {
-    	super(vmName, storageHost, vols);
+    	super(vmName, storageHost, vols);
+        this.networkRateMbps = networkRateMbps;
+        this.networkRateMulticastMbps = networkRateMulticastMbps;
     	this.proxyCmdPort = proxyCmdPort;
     	this.secStorageVm = secStorageVm;
 
@@ -56,7 +61,15 @@ public class StartSecStorageVmCommand extends AbstractStartCommand {
 		return secStorageVm;
 	}
 	
-	public int getProxyCmdPort() {
+	public int getNetworkRateMbps() {
+        return networkRateMbps;
+    }
+
+    public int getNetworkRateMulticastMbps() {
+        return networkRateMulticastMbps;
+    }
+
+    public int getProxyCmdPort() {
 		return proxyCmdPort;
 	}
 	
diff --git a/core/src/com/cloud/agent/api/StartupStorageCommand.java b/core/src/com/cloud/agent/api/StartupStorageCommand.java
index a698f58468d..1122e82f0d3 100755
--- a/core/src/com/cloud/agent/api/StartupStorageCommand.java
+++ b/core/src/com/cloud/agent/api/StartupStorageCommand.java
@@ -20,7 +20,7 @@ package com.cloud.agent.api;
 import java.util.HashMap;
 import java.util.Map;
 
-import com.cloud.storage.Volume;
+import com.cloud.storage.Storage;
 import com.cloud.storage.Storage.StoragePoolType;
 import com.cloud.storage.template.TemplateInfo;
 
@@ -31,7 +31,7 @@ public class StartupStorageCommand extends StartupCommand {
     Map templateInfo;
     long totalSize;
     StoragePoolInfo poolInfo;
-    Volume.StorageResourceType resourceType;
+    Storage.StorageResourceType resourceType;
     StoragePoolType fsType;
     Map hostDetails = new HashMap();
     String nfsShare;
@@ -91,11 +91,11 @@ public class StartupStorageCommand extends StartupCommand {
 		this.poolInfo = poolInfo;
 	}
 
-	public Volume.StorageResourceType getResourceType() {
+	public Storage.StorageResourceType getResourceType() {
 		return resourceType;
 	}
 
-	public void setResourceType(Volume.StorageResourceType resourceType) {
+	public void setResourceType(Storage.StorageResourceType resourceType) {
 		this.resourceType = resourceType;
 	}
 
diff --git a/core/src/com/cloud/agent/api/routing/IPAssocCommand.java b/core/src/com/cloud/agent/api/routing/IPAssocCommand.java
index ced18f26737..42122d16716 100644
--- a/core/src/com/cloud/agent/api/routing/IPAssocCommand.java
+++ b/core/src/com/cloud/agent/api/routing/IPAssocCommand.java
@@ -29,16 +29,18 @@ public class IPAssocCommand extends RoutingCommand {
 	private String publicIp;
 	private boolean sourceNat;
 	private boolean add;
+	private boolean oneToOneNat;
 	private boolean firstIP;
 	private String vlanId;
 	private String vlanGateway;
 	private String vlanNetmask;
 	private String vifMacAddress;
+	private String guestIp;
 
 	protected IPAssocCommand() {
 	}
 	
-	public IPAssocCommand(String routerName, String privateIpAddress, String ipAddress, boolean add, boolean firstIP, boolean sourceNat, String vlanId, String vlanGateway, String vlanNetmask, String vifMacAddress) {
+	public IPAssocCommand(String routerName, String privateIpAddress, String ipAddress, boolean add, boolean firstIP, boolean sourceNat, String vlanId, String vlanGateway, String vlanNetmask, String vifMacAddress, String guestIp) {
 		this.setRouterName(routerName);
 		this.routerIp = privateIpAddress;
 		this.publicIp = ipAddress;
@@ -49,8 +51,13 @@ public class IPAssocCommand extends RoutingCommand {
 		this.vlanGateway = vlanGateway;
 		this.vlanNetmask = vlanNetmask;
 		this.vifMacAddress = vifMacAddress;
+		this.guestIp = guestIp;
 	}
 
+	public String getGuestIp(){
+		return guestIp;
+	}
+	
 	public String getRouterIp() {
 		return routerIp;
 	}
@@ -63,6 +70,10 @@ public class IPAssocCommand extends RoutingCommand {
 		return add;
 	}
 	
+	public boolean isOneToOneNat(){
+		return this.oneToOneNat;
+	}
+	
 	public boolean isFirstIP() {
 		return firstIP;
 	}
diff --git a/core/src/com/cloud/agent/api/storage/AbstractUploadCommand.java b/core/src/com/cloud/agent/api/storage/AbstractUploadCommand.java
new file mode 100644
index 00000000000..234ab6bbfcc
--- /dev/null
+++ b/core/src/com/cloud/agent/api/storage/AbstractUploadCommand.java
@@ -0,0 +1,52 @@
+package com.cloud.agent.api.storage;
+
+import com.cloud.storage.Storage.ImageFormat;
+
+public class AbstractUploadCommand  extends StorageCommand{
+
+
+    private String url;
+    private ImageFormat format;
+    private long accountId;
+    private String name;
+    
+    protected AbstractUploadCommand() {
+    }
+    
+    protected AbstractUploadCommand(String name, String url, ImageFormat format, long accountId) {
+        this.url = url;
+        this.format = format;
+        this.accountId = accountId;
+        this.name = name;
+    }
+    
+    protected AbstractUploadCommand(AbstractUploadCommand that) {
+        this(that.name, that.url, that.format, that.accountId);
+    }
+    
+    public String getUrl() {
+        return url;
+    }
+    
+    public String getName() {
+        return name;
+    }
+    
+    public ImageFormat getFormat() {
+        return format;
+    }
+    
+    public long getAccountId() {
+        return accountId;
+    }
+    
+    @Override
+    public boolean executeInSequence() {
+        return true;
+    }
+
+	public void setUrl(String url) {
+		this.url = url;
+	}
+
+}
diff --git a/core/src/com/cloud/agent/api/storage/CreateCommand.java b/core/src/com/cloud/agent/api/storage/CreateCommand.java
index 2c0ade671a9..d53e378aec7 100644
--- a/core/src/com/cloud/agent/api/storage/CreateCommand.java
+++ b/core/src/com/cloud/agent/api/storage/CreateCommand.java
@@ -18,18 +18,19 @@
 package com.cloud.agent.api.storage;
 
 import com.cloud.agent.api.Command;
-import com.cloud.agent.api.to.DiskCharacteristicsTO;
 import com.cloud.agent.api.to.StoragePoolTO;
 import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.VolumeVO;
+import com.cloud.vm.DiskProfile;
 import com.cloud.vm.VMInstanceVO;
 
 public class CreateCommand extends Command {
     private long volId;
     private StoragePoolTO pool;
-    private DiskCharacteristicsTO diskCharacteristics;
+    private DiskProfile diskCharacteristics;
     private String templateUrl;
     private long size;
+    private String instanceName;
     
     protected CreateCommand() {
         super();
@@ -44,7 +45,7 @@ public class CreateCommand extends Command {
      * @param templateUrl
      * @param pool
      */
-    public CreateCommand(VolumeVO vol, VMInstanceVO vm, DiskCharacteristicsTO diskCharacteristics, String templateUrl, StoragePoolVO pool) {
+    public CreateCommand(VolumeVO vol, VMInstanceVO vm, DiskProfile diskCharacteristics, String templateUrl, StoragePoolVO pool) {
         this(vol, vm, diskCharacteristics, pool, 0);
         this.templateUrl = templateUrl;
     }
@@ -57,12 +58,13 @@ public class CreateCommand extends Command {
      * @param diskCharacteristics
      * @param pool
      */
-    public CreateCommand(VolumeVO vol, VMInstanceVO vm, DiskCharacteristicsTO diskCharacteristics, StoragePoolVO pool, long size) {
+    public CreateCommand(VolumeVO vol, VMInstanceVO vm, DiskProfile diskCharacteristics, StoragePoolVO pool, long size) {
         this.volId = vol.getId();
         this.diskCharacteristics = diskCharacteristics;        
         this.pool = new StoragePoolTO(pool);
         this.templateUrl = null;
         this.size = size;
+        //this.instanceName = vm.getInstanceName();
     }
     
     @Override
@@ -78,7 +80,7 @@ public class CreateCommand extends Command {
         return pool;
     }
     
-    public DiskCharacteristicsTO getDiskCharacteristics() {
+    public DiskProfile getDiskCharacteristics() {
         return diskCharacteristics;
     }
     
@@ -89,4 +91,8 @@ public class CreateCommand extends Command {
     public long getSize(){
     	return this.size;
     }
+    
+    public String getInstanceName() {
+    	return instanceName;
+    }
 }
diff --git a/core/src/com/cloud/agent/api/storage/PrimaryStorageDownloadCommand.java b/core/src/com/cloud/agent/api/storage/PrimaryStorageDownloadCommand.java
index a9010d5782e..f11f17f1bb5 100644
--- a/core/src/com/cloud/agent/api/storage/PrimaryStorageDownloadCommand.java
+++ b/core/src/com/cloud/agent/api/storage/PrimaryStorageDownloadCommand.java
@@ -28,6 +28,16 @@ public class PrimaryStorageDownloadCommand extends AbstractDownloadCommand {
 	String localPath;
 	String poolUuid;
 	long poolId;
+	
+	//
+	// Temporary hacking to make vmware work quickly, expose NFS raw information to allow
+	// agent do quick copy over NFS.
+	//
+	// provide storage URL (it contains all information to help agent resource to mount the
+	// storage if needed, example of such URL may be as following
+	// 		nfs://192.168.10.231/export/home/kelven/vmware-test/secondary
+	String secondaryStorageUrl;
+	String primaryStorageUrl;
 
 	protected PrimaryStorageDownloadCommand() {
 	}
@@ -54,6 +64,22 @@ public class PrimaryStorageDownloadCommand extends AbstractDownloadCommand {
     	return localPath;
     }
     
+    public void setSecondaryStorageUrl(String url) {
+    	secondaryStorageUrl = url;
+    }
+    
+    public String getSecondaryStorageUrl() {
+    	return secondaryStorageUrl;
+    }
+    
+    public void setPrimaryStorageUrl(String url) {
+    	primaryStorageUrl = url;
+    }
+    
+    public String getPrimaryStorageUrl() {
+    	return primaryStorageUrl;
+    }
+    
     @Override
     public boolean executeInSequence() {
         return true;
diff --git a/core/src/com/cloud/agent/api/storage/UploadAnswer.java b/core/src/com/cloud/agent/api/storage/UploadAnswer.java
new file mode 100755
index 00000000000..b878bf7ca6b
--- /dev/null
+++ b/core/src/com/cloud/agent/api/storage/UploadAnswer.java
@@ -0,0 +1,105 @@
+package com.cloud.agent.api.storage;
+
+import java.io.File;
+
+import com.cloud.agent.api.Answer;
+import com.cloud.agent.api.Command;
+import com.cloud.storage.UploadVO;
+
+public class UploadAnswer extends Answer {
+
+	
+	private String jobId;
+	private int uploadPct;
+	private String errorString;
+	private UploadVO.Status uploadStatus;
+	private String uploadPath;
+	private String installPath;
+	public Long templateSize = 0L;
+	
+	public int getUploadPct() {
+		return uploadPct;
+	}
+	public String getErrorString() {
+		return errorString;
+	}
+	
+	public String getUploadStatusString() {
+		return uploadStatus.toString();
+	}
+	
+	public UploadVO.Status getUploadStatus() {
+		return uploadStatus;
+	}
+	
+	public String getUploadPath() {
+		return uploadPath;
+	}
+	protected UploadAnswer() {
+		
+	}
+		
+	public void setErrorString(String errorString) {
+        this.errorString = errorString;
+    }
+    public String getJobId() {
+		return jobId;
+	}
+	public void setJobId(String jobId) {
+		this.jobId = jobId;
+	}
+	
+	public UploadAnswer(String jobId, int uploadPct, String errorString,
+			UploadVO.Status uploadStatus, String fileSystemPath, String installPath, long templateSize) {
+		super();
+		this.jobId = jobId;
+		this.uploadPct = uploadPct;
+		this.errorString = errorString;
+		this.uploadStatus = uploadStatus;
+		this.uploadPath = fileSystemPath;
+		this.installPath = fixPath(installPath);
+		this.templateSize = templateSize;
+	}
+
+   public UploadAnswer(String jobId, int uploadPct, Command command,
+		   UploadVO.Status uploadStatus, String fileSystemPath, String installPath) {
+	    super(command);
+        this.jobId = jobId;
+        this.uploadPct = uploadPct;
+        this.uploadStatus = uploadStatus;
+        this.uploadPath = fileSystemPath;
+        this.installPath = installPath;
+    }
+		
+	private static String fixPath(String path){
+		if (path == null)
+			return path;
+		if (path.startsWith(File.separator)) {
+			path=path.substring(File.separator.length());
+		}
+		if (path.endsWith(File.separator)) {
+			path=path.substring(0, path.length()-File.separator.length());
+		}
+		return path;
+	}
+	
+	public void setUploadStatus(UploadVO.Status uploadStatus) {
+		this.uploadStatus = uploadStatus;
+	}
+	
+	public String getInstallPath() {
+		return installPath;
+	}
+	public void setInstallPath(String installPath) {
+		this.installPath = fixPath(installPath);
+	}
+
+	public void setTemplateSize(long templateSize) {
+		this.templateSize = templateSize;
+	}
+	
+	public Long getTemplateSize() {
+		return templateSize;
+	}
+	
+}
diff --git a/core/src/com/cloud/agent/api/storage/UploadCommand.java b/core/src/com/cloud/agent/api/storage/UploadCommand.java
new file mode 100644
index 00000000000..a3f0695694b
--- /dev/null
+++ b/core/src/com/cloud/agent/api/storage/UploadCommand.java
@@ -0,0 +1,126 @@
+package com.cloud.agent.api.storage;
+
+import com.cloud.storage.VMTemplateHostVO;
+import com.cloud.storage.Upload.Type;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.agent.api.storage.AbstractUploadCommand;
+import com.cloud.agent.api.storage.DownloadCommand.PasswordAuth;
+
+
+public class UploadCommand extends AbstractUploadCommand {
+
+	private VMTemplateVO template;
+	private String url;
+	private String installPath;	
+	private boolean hvm;
+	private String description;
+	private String checksum;
+	private PasswordAuth auth;
+	private long templateSizeInBytes;
+	private long id;
+	private Type type;
+
+	public UploadCommand(VMTemplateVO template, String url, VMTemplateHostVO vmTemplateHost) {
+		
+		this.template = template;
+		this.url = url;
+		this.installPath = vmTemplateHost.getInstallPath();
+		this.checksum = template.getChecksum();
+		this.id = template.getId();
+		this.templateSizeInBytes = vmTemplateHost.getSize();
+		
+	}
+	
+	public UploadCommand(String url, long id, long sizeInBytes, String installPath, Type type){
+		this.template = null;
+		this.url = url;
+		this.installPath = installPath;
+		this.id = id;
+		this.type = type;
+		this.templateSizeInBytes = sizeInBytes;
+	}
+	
+	protected UploadCommand() {
+	}
+	
+	public UploadCommand(UploadCommand that) {
+		this.template = that.template;
+		this.url = that.url;
+		this.installPath = that.installPath;
+		this.checksum = that.getChecksum();
+		this.id = that.id;
+	}
+
+	public String getDescription() {
+		return description;
+	}
+
+
+	public VMTemplateVO getTemplate() {
+		return template;
+	}
+
+	public void setTemplate(VMTemplateVO template) {
+		this.template = template;
+	}
+
+	public String getUrl() {
+		return url;
+	}
+
+	public void setUrl(String url) {
+		this.url = url;
+	}
+
+	public boolean isHvm() {
+		return hvm;
+	}
+
+	public void setHvm(boolean hvm) {
+		this.hvm = hvm;
+	}
+
+	public PasswordAuth getAuth() {
+		return auth;
+	}
+
+	public void setAuth(PasswordAuth auth) {
+		this.auth = auth;
+	}
+
+	public Long getTemplateSizeInBytes() {
+		return templateSizeInBytes;
+	}
+
+	public void setTemplateSizeInBytes(Long templateSizeInBytes) {
+		this.templateSizeInBytes = templateSizeInBytes;
+	}
+
+	public long getId() {
+		return id;
+	}
+
+	public void setId(long id) {
+		this.id = id;
+	}
+
+	public void setInstallPath(String installPath) {
+		this.installPath = installPath;
+	}
+
+	public void setDescription(String description) {
+		this.description = description;
+	}
+
+	public void setChecksum(String checksum) {
+		this.checksum = checksum;
+	}
+
+	public String getInstallPath() {
+		return installPath;
+	}
+
+	public String getChecksum() {
+		return checksum;
+	}
+}
diff --git a/core/src/com/cloud/agent/api/storage/UploadProgressCommand.java b/core/src/com/cloud/agent/api/storage/UploadProgressCommand.java
new file mode 100644
index 00000000000..9da9f2f1cb2
--- /dev/null
+++ b/core/src/com/cloud/agent/api/storage/UploadProgressCommand.java
@@ -0,0 +1,32 @@
+package com.cloud.agent.api.storage;
+
+public class UploadProgressCommand extends UploadCommand {
+
+	public static enum RequestType {GET_STATUS, ABORT, RESTART, PURGE, GET_OR_RESTART}
+	private String jobId;
+	private RequestType request;
+
+	protected UploadProgressCommand() {
+		super();
+	}
+	
+	public UploadProgressCommand(UploadCommand cmd, String jobId, RequestType req) {
+	    super(cmd);
+
+		this.jobId = jobId;
+		this.setRequest(req);
+	}
+
+	public String getJobId() {
+		return jobId;
+	}
+
+	public void setRequest(RequestType request) {
+		this.request = request;
+	}
+
+	public RequestType getRequest() {
+		return request;
+	}
+	
+}
\ No newline at end of file
diff --git a/core/src/com/cloud/agent/api/to/DiskCharacteristicsTO.java b/core/src/com/cloud/agent/api/to/DiskCharacteristicsTO.java
deleted file mode 100644
index ec4bf6f8482..00000000000
--- a/core/src/com/cloud/agent/api/to/DiskCharacteristicsTO.java
+++ /dev/null
@@ -1,79 +0,0 @@
-/**
- *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
- * 
- * This software is licensed under the GNU General Public License v3 or later.
- * 
- * It is free software: you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation, either version 3 of the License, or any later version.
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- * 
- * You should have received a copy of the GNU General Public License
- * along with this program.  If not, see .
- * 
- */
-package com.cloud.agent.api.to;
-
-import com.cloud.storage.DiskOfferingVO;
-import com.cloud.storage.Volume;
-
-public class DiskCharacteristicsTO {
-    private long size;
-    private String[] tags;
-    private Volume.VolumeType type;
-    private String name;
-    private boolean useLocalStorage;
-    private boolean recreatable;
-    
-    protected DiskCharacteristicsTO() {
-    }
-    
-    public DiskCharacteristicsTO(Volume.VolumeType type, String name, long size, String[] tags, boolean useLocalStorage, boolean recreatable) {
-        this.type = type;
-        this.name = name;
-        this.size = size;
-        this.tags = tags;
-        this.useLocalStorage = useLocalStorage;
-        this.recreatable = recreatable;
-    }
-    
-    public DiskCharacteristicsTO(Volume.VolumeType type, String name, DiskOfferingVO offering, long size) {
-        this(type, name, size, offering.getTagsArray(), offering.getUseLocalStorage(), offering.isRecreatable());
-    }
-    
-    public DiskCharacteristicsTO(Volume.VolumeType type, String name, DiskOfferingVO offering) {
-        this(type, name, offering.getDiskSizeInBytes(), offering.getTagsArray(), offering.getUseLocalStorage(), offering.isRecreatable());
-    }
-    
-    public long getSize() {
-        return size;
-    }
-    
-    public String getName() {
-        return name;
-    }
-
-    public String[] getTags() {
-        return tags;
-    }
-
-    public Volume.VolumeType getType() {
-        return type;
-    }
-    
-    public boolean useLocalStorage() {
-        return useLocalStorage;
-    }
-    
-    public boolean isRecreatable() {
-        return recreatable;
-    }
-    
-    @Override
-    public String toString() {
-        return new StringBuilder("DskChr[").append(type).append("|").append(size).append("|").append("]").toString();
-    }
-}
diff --git a/core/src/com/cloud/agent/api/to/NetworkTO.java b/core/src/com/cloud/agent/api/to/NetworkTO.java
index 50cb1cd034c..d6e428831f3 100644
--- a/core/src/com/cloud/agent/api/to/NetworkTO.java
+++ b/core/src/com/cloud/agent/api/to/NetworkTO.java
@@ -17,21 +17,79 @@
  */
 package com.cloud.agent.api.to;
 
+import com.cloud.network.Network.BroadcastDomainType;
+import com.cloud.network.Network.TrafficType;
+
 /**
  * Transfer object to transfer network settings.
  */
 public class NetworkTO {
+    private String uuid;
     private String ip;
     private String netmask;
     private String gateway;
     private String mac;
     private String dns1;
     private String dns2;
-    private String vlan;
+    private Long vlan;
+    private BroadcastDomainType broadcastType;
+    private TrafficType type;
     
-    protected NetworkTO() {
+    public NetworkTO() {
     }
     
+    public String getUuid() {
+        return uuid;
+    }
+
+    public void setUuid(String uuid) {
+        this.uuid = uuid;
+    }
+
+    public Long getVlan() {
+        return vlan;
+    }
+
+    public void setVlan(Long vlan) {
+        this.vlan = vlan;
+    }
+
+    public BroadcastDomainType getBroadcastType() {
+        return broadcastType;
+    }
+
+    public void setBroadcastType(BroadcastDomainType broadcastType) {
+        this.broadcastType = broadcastType;
+    }
+
+    public void setIp(String ip) {
+        this.ip = ip;
+    }
+
+    public void setNetmask(String netmask) {
+        this.netmask = netmask;
+    }
+
+    public void setGateway(String gateway) {
+        this.gateway = gateway;
+    }
+
+    public void setMac(String mac) {
+        this.mac = mac;
+    }
+
+    public void setDns1(String dns1) {
+        this.dns1 = dns1;
+    }
+
+    public void setDns2(String dns2) {
+        this.dns2 = dns2;
+    }
+
+    public void setType(TrafficType type) {
+        this.type = type;
+    }
+
     /**
      * This constructor is usually for hosts where the other information are not important.
      * 
@@ -55,7 +113,7 @@ public class NetworkTO {
      * @param dns1
      * @param dns2
      */
-    public NetworkTO(String ip, String vlan, String netmask, String mac, String gateway, String dns1, String dns2) {
+    public NetworkTO(String ip, Long vlan, String netmask, String mac, String gateway, String dns1, String dns2) {
         this.ip = ip;
         this.netmask = netmask;
         this.mac = mac;
@@ -88,4 +146,8 @@ public class NetworkTO {
     public String getDns2() {
         return dns2;
     }
+    
+    public TrafficType getType() {
+        return type;
+    }
 }
diff --git a/core/src/com/cloud/agent/api/to/NicTO.java b/core/src/com/cloud/agent/api/to/NicTO.java
new file mode 100644
index 00000000000..42e4f66bd94
--- /dev/null
+++ b/core/src/com/cloud/agent/api/to/NicTO.java
@@ -0,0 +1,36 @@
+/**
+ * 
+ */
+package com.cloud.agent.api.to;
+
+public class NicTO extends NetworkTO {
+    int deviceId;
+    Integer controlPort;
+    Integer networkRateMbps;
+    Integer networkRateMulticastMbps;
+
+    public NicTO() {
+        super();
+        controlPort = null;
+    }
+    
+    public void setDeviceId(int deviceId) {
+        this.deviceId = deviceId;
+    }
+    
+    public int getDeviceId() {
+        return deviceId;
+    }
+    
+    public Integer getControlPort() {
+        return controlPort;
+    }
+    
+    public Integer getNetworkRateMbps() {
+        return networkRateMbps;
+    }
+    
+    public Integer getNetworkRateMulticastMbps() {
+        return networkRateMulticastMbps;
+    }
+}
diff --git a/core/src/com/cloud/agent/api/to/VirtualMachineTO.java b/core/src/com/cloud/agent/api/to/VirtualMachineTO.java
new file mode 100644
index 00000000000..b37c2fd2426
--- /dev/null
+++ b/core/src/com/cloud/agent/api/to/VirtualMachineTO.java
@@ -0,0 +1,173 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.agent.api.to;
+
+import java.util.Map;
+
+import com.cloud.template.VirtualMachineTemplate.BootloaderType;
+import com.cloud.vm.VirtualMachine.Type;
+
+public class VirtualMachineTO {
+    private long id;
+    private String name;
+    private BootloaderType bootloader;
+    Type type;
+    int cpus;
+    Integer weight;
+    Integer utilization;
+    long minRam;
+    long maxRam;
+    String hostName;
+    String arch;
+    String os;
+    String bootArgs;
+    String[] bootupScripts;
+    boolean rebootOnCrash;
+    
+    VolumeTO[] disks;
+    NicTO[] nics;
+    
+    public VirtualMachineTO() {
+    }
+    
+    public long getId() {
+        return id;
+    }
+
+    public void setId(long id) {
+        this.id = id;
+    }
+
+    public String getName() {
+        return name;
+    }
+
+    public void setName(String name) {
+        this.name = name;
+    }
+    
+    public Type getType() {
+        return type;
+    }
+
+    public BootloaderType getBootloader() {
+        return bootloader;
+    }
+
+    public void setBootloader(BootloaderType bootloader) {
+        this.bootloader = bootloader;
+    }
+
+    public int getCpus() {
+        return cpus;
+    }
+
+    public void setCpus(int cpus) {
+        this.cpus = cpus;
+    }
+    
+    public Integer getWeight() {
+        return weight;
+    }
+    
+    public void setWeight(Integer weight) {
+        this.weight = weight;
+    }
+    
+    public Integer getUtilization() {
+        return utilization;
+    }
+    
+    public void setUtiliziation(Integer utilization) {
+        this.utilization = utilization;
+    }
+
+    public long getMinRam() {
+        return minRam;
+    }
+
+    public void setRam(long minRam, long maxRam) {
+        this.minRam = minRam;
+        this.maxRam = maxRam;
+    }
+
+    public long getMaxRam() {
+        return maxRam;
+    }
+
+    public String getHostName() {
+        return hostName;
+    }
+
+    public void setHostName(String hostName) {
+        this.hostName = hostName;
+    }
+
+    public String getArch() {
+        return arch;
+    }
+
+    public void setArch(String arch) {
+        this.arch = arch;
+    }
+
+    public String getOs() {
+        return os;
+    }
+
+    public void setOs(String os) {
+        this.os = os;
+    }
+
+    public String getBootArgs() {
+        return bootArgs;
+    }
+
+    public void setBootArgs(Map bootParams) {
+        StringBuilder buf   = new StringBuilder();
+        for (Map.Entry entry : bootParams.entrySet()) {
+            buf.append(" ").append(entry.getKey()).append("=").append(entry.getValue());
+        }
+        bootArgs = buf.toString();
+    }
+
+    public String[] getBootupScripts() {
+        return bootupScripts;
+    }
+
+    public void setBootupScripts(String[] bootupScripts) {
+        this.bootupScripts = bootupScripts;
+    }
+
+    public VolumeTO[] getDisks() {
+        return disks;
+    }
+
+    public void setDisks(VolumeTO[] disks) {
+        this.disks = disks;
+    }
+
+    public NicTO[] getNetworks() {
+        return nics;
+    }
+
+    public void setNics(NicTO[] nics) {
+        this.nics = nics;
+    }
+
+}
diff --git a/core/src/com/cloud/agent/api/to/VmTO.java b/core/src/com/cloud/agent/api/to/VmTO.java
deleted file mode 100644
index 7df5041def4..00000000000
--- a/core/src/com/cloud/agent/api/to/VmTO.java
+++ /dev/null
@@ -1,39 +0,0 @@
-/**
- *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
- * 
- * This software is licensed under the GNU General Public License v3 or later.
- * 
- * It is free software: you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation, either version 3 of the License, or any later version.
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- * 
- * You should have received a copy of the GNU General Public License
- * along with this program.  If not, see .
- * 
- */
-package com.cloud.agent.api.to;
-
-import com.cloud.vm.VMInstanceVO;
-
-public class VmTO {
-    private long id;
-    private String name;
-    NetworkTO[] networks;
-    
-    public VmTO() {
-    }
-    
-    // FIXME: Preferrably NetworkTO is constructed inside the TO objects.
-    //        But we're press for time so I'm going to let the conversion
-    //        happen outside.
-    public VmTO(VMInstanceVO instance, NetworkTO[] networks) {
-        id = instance.getId();
-        name = instance.getName();
-        this.networks = networks;
-    }
-
-}
diff --git a/core/src/com/cloud/agent/api/to/VolumeTO.java b/core/src/com/cloud/agent/api/to/VolumeTO.java
index e0afde2cb2e..15853b13098 100644
--- a/core/src/com/cloud/agent/api/to/VolumeTO.java
+++ b/core/src/com/cloud/agent/api/to/VolumeTO.java
@@ -17,12 +17,12 @@
  */
 package com.cloud.agent.api.to;
 
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.StoragePoolType;
 import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.VMTemplateStoragePoolVO;
 import com.cloud.storage.Volume;
 import com.cloud.storage.VolumeVO;
-import com.cloud.storage.Storage.StoragePoolType;
-import com.cloud.storage.Volume.StorageResourceType;
 
 
 public class VolumeTO {
@@ -35,11 +35,12 @@ public class VolumeTO {
     private String path;
     private long size;
     private Volume.VolumeType type;
-    private Volume.StorageResourceType resourceType;
+    private Storage.StorageResourceType resourceType;
     private StoragePoolType storagePoolType;
     private long poolId;
+    private int deviceId;
     
-    public VolumeTO(long id, Volume.VolumeType type, Volume.StorageResourceType resourceType, StoragePoolType poolType, String name, String mountPoint, String path, long size) {
+    public VolumeTO(long id, Volume.VolumeType type, Storage.StorageResourceType resourceType, StoragePoolType poolType, String name, String mountPoint, String path, long size) {
         this.id = id;
         this.name= name;
         this.path = path;
@@ -65,12 +66,16 @@ public class VolumeTO {
     	this.id = templatePoolRef.getId();
     	this.path = templatePoolRef.getInstallPath();
     	this.size = templatePoolRef.getTemplateSize();
-    	this.resourceType = StorageResourceType.STORAGE_POOL;
+    	this.resourceType = Storage.StorageResourceType.STORAGE_POOL;
     	this.storagePoolType = pool.getPoolType();
     	this.mountPoint = pool.getPath();
     }
+    
+    public int getDeviceId() {
+        return deviceId;
+    }
 
-    public Volume.StorageResourceType getResourceType() {
+    public Storage.StorageResourceType getResourceType() {
         return resourceType;
     }
     
diff --git a/core/src/com/cloud/alert/dao/AlertDaoImpl.java b/core/src/com/cloud/alert/dao/AlertDaoImpl.java
index c79c6c48b9c..bee71b47a73 100644
--- a/core/src/com/cloud/alert/dao/AlertDaoImpl.java
+++ b/core/src/com/cloud/alert/dao/AlertDaoImpl.java
@@ -40,7 +40,7 @@ public class AlertDaoImpl extends GenericDaoBase implements Alert
             sc.addAnd("podId", SearchCriteria.Op.EQ, podId);
         }
 
-        List alerts = listActiveBy(sc, searchFilter);
+        List alerts = listBy(sc, searchFilter);
         if ((alerts != null) && !alerts.isEmpty()) {
             return alerts.get(0);
         }
diff --git a/core/src/com/cloud/async/dao/AsyncJobDaoImpl.java b/core/src/com/cloud/async/dao/AsyncJobDaoImpl.java
index 24cacaa8cc3..13564f43961 100644
--- a/core/src/com/cloud/async/dao/AsyncJobDaoImpl.java
+++ b/core/src/com/cloud/async/dao/AsyncJobDaoImpl.java
@@ -61,7 +61,7 @@ public class AsyncJobDaoImpl extends GenericDaoBase implements
         sc.setParameters("instanceId", instanceId);
         sc.setParameters("status", AsyncJobResult.STATUS_IN_PROGRESS);
         
-        List l = listBy(sc);
+        List l = listIncludingRemovedBy(sc);
         if(l != null && l.size() > 0) {
         	if(l.size() > 1) {
         		s_logger.warn("Instance " + instanceType + "-" + instanceId + " has multiple pending async-job");
@@ -76,6 +76,6 @@ public class AsyncJobDaoImpl extends GenericDaoBase implements
 		SearchCriteria sc = expiringAsyncJobSearch.create();
 		sc.setParameters("created", cutTime);
 		Filter filter = new Filter(AsyncJobVO.class, "created", true, 0L, (long)limit);
-		return listBy(sc, filter);
+		return listIncludingRemovedBy(sc, filter);
 	}
 }
diff --git a/core/src/com/cloud/async/dao/SyncQueueDaoImpl.java b/core/src/com/cloud/async/dao/SyncQueueDaoImpl.java
index d009866e9e0..26026906c5f 100644
--- a/core/src/com/cloud/async/dao/SyncQueueDaoImpl.java
+++ b/core/src/com/cloud/async/dao/SyncQueueDaoImpl.java
@@ -66,7 +66,7 @@ public class SyncQueueDaoImpl extends GenericDaoBase implemen
     	SearchCriteria sc = TypeIdSearch.create();
     	sc.setParameters("syncObjType", syncObjType);
     	sc.setParameters("syncObjId", syncObjId);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
 	}
 	
 	protected SyncQueueDaoImpl() {
diff --git a/core/src/com/cloud/async/dao/SyncQueueItemDaoImpl.java b/core/src/com/cloud/async/dao/SyncQueueItemDaoImpl.java
index a0b4df70074..a8a3202be2b 100644
--- a/core/src/com/cloud/async/dao/SyncQueueItemDaoImpl.java
+++ b/core/src/com/cloud/async/dao/SyncQueueItemDaoImpl.java
@@ -53,7 +53,7 @@ public class SyncQueueItemDaoImpl extends GenericDaoBase
     	sc.setParameters("queueId", queueId);
     	
     	Filter filter = new Filter(SyncQueueItemVO.class, "created", true, 0L, 1L);
-        List l = listActiveBy(sc, filter);
+        List l = listBy(sc, filter);
         if(l != null && l.size() > 0)
         	return l.get(0);
     	
@@ -105,6 +105,6 @@ public class SyncQueueItemDaoImpl extends GenericDaoBase
     	sc.setParameters("lastProcessMsid", msid);
     	
     	Filter filter = new Filter(SyncQueueItemVO.class, "created", true, 0L, 1L);
-        return listActiveBy(sc, filter);
+        return listBy(sc, filter);
 	}
 }
diff --git a/core/src/com/cloud/capacity/dao/CapacityDao.java b/core/src/com/cloud/capacity/dao/CapacityDao.java
index f86bd97056f..726544f4384 100644
--- a/core/src/com/cloud/capacity/dao/CapacityDao.java
+++ b/core/src/com/cloud/capacity/dao/CapacityDao.java
@@ -22,7 +22,6 @@ import com.cloud.capacity.CapacityVO;
 import com.cloud.utils.db.GenericDao;
 
 public interface CapacityDao extends GenericDao {
-    void setUsedStorage(Long hostId, long totalUsed);
     void clearNonStorageCapacities();
     void clearStorageCapacities();
 }
diff --git a/core/src/com/cloud/capacity/dao/CapacityDaoImpl.java b/core/src/com/cloud/capacity/dao/CapacityDaoImpl.java
index 7151bff7ce9..e6e8d4dba25 100644
--- a/core/src/com/cloud/capacity/dao/CapacityDaoImpl.java
+++ b/core/src/com/cloud/capacity/dao/CapacityDaoImpl.java
@@ -34,9 +34,8 @@ public class CapacityDaoImpl extends GenericDaoBase implements
 
     private static final String ADD_ALLOCATED_SQL = "UPDATE `cloud`.`op_host_capacity` SET used_capacity = used_capacity + ? WHERE host_id = ? AND capacity_type = ?";
     private static final String SUBTRACT_ALLOCATED_SQL = "UPDATE `cloud`.`op_host_capacity` SET used_capacity = used_capacity - ? WHERE host_id = ? AND capacity_type = ?";
-    private static final String SET_USED_STORAGE_SQL = "UPDATE `cloud`.`op_host_capacity` SET used_capacity = ? WHERE host_id = ? AND capacity_type = 2";
-    private static final String CLEAR_STORAGE_CAPACITIES = "DELETE FROM `cloud`.`op_host_capacity` WHERE capacity_type=2 OR capacity_type=6"; //clear storage and secondary_storage capacities
-    private static final String CLEAR_NON_STORAGE_CAPACITIES = "DELETE FROM `cloud`.`op_host_capacity` WHERE capacity_type<>2 AND capacity_type <>6"; //clear non-storage and non-secondary_storage capacities
+    private static final String CLEAR_STORAGE_CAPACITIES = "DELETE FROM `cloud`.`op_host_capacity` WHERE capacity_type=2 OR capacity_type=3 OR capacity_type=6"; //clear storage and secondary_storage capacities
+    private static final String CLEAR_NON_STORAGE_CAPACITIES = "DELETE FROM `cloud`.`op_host_capacity` WHERE capacity_type<>2 AND capacity_type<>3 AND capacity_type<>6"; //clear non-storage and non-secondary_storage capacities
 
     public void updateAllocated(Long hostId, long allocatedAmount, short capacityType, boolean add) {
         Transaction txn = Transaction.currentTxn();
@@ -61,22 +60,6 @@ public class CapacityDaoImpl extends GenericDaoBase implements
         }
     }
 
-    public void setUsedStorage(Long hostId, long totalUsed) {
-        Transaction txn = Transaction.currentTxn();
-        PreparedStatement pstmt = null;
-        try {
-            txn.start();
-            String sql = SET_USED_STORAGE_SQL;
-            pstmt = txn.prepareAutoCloseStatement(sql);
-            pstmt.setLong(1, totalUsed);
-            pstmt.setLong(2, hostId);
-            pstmt.executeUpdate(); // TODO:  Make sure exactly 1 row was updated?
-            txn.commit();
-        } catch (Exception e) {
-            txn.rollback();
-            s_logger.warn("Exception setting used storage for host: " + hostId, e);
-        }
-    }
 
     @Override
     public void clearNonStorageCapacities() {
diff --git a/core/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java b/core/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java
index 8fb19e21942..ebef69d9e8d 100644
--- a/core/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java
+++ b/core/src/com/cloud/cluster/dao/ManagementServerHostDaoImpl.java
@@ -44,7 +44,7 @@ public class ManagementServerHostDaoImpl extends GenericDaoBase sc = MsIdSearch.create();
         sc.setParameters("msid", msid);
 		
-		List l = listBy(sc);
+		List l = listIncludingRemovedBy(sc);
 		if(l != null && l.size() > 0)
 			return l.get(0);
 		 
@@ -101,7 +101,7 @@ public class ManagementServerHostDaoImpl extends GenericDaoBase sc = activeSearch.create();
 	    sc.setParameters("lastUpdateTime", cutTime);
 	    
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 	
 	public void increaseAlertCount(long id) {
diff --git a/core/src/com/cloud/configuration/dao/ConfigurationDaoImpl.java b/core/src/com/cloud/configuration/dao/ConfigurationDaoImpl.java
index 54930bcb1dc..84f95d76d73 100644
--- a/core/src/com/cloud/configuration/dao/ConfigurationDaoImpl.java
+++ b/core/src/com/cloud/configuration/dao/ConfigurationDaoImpl.java
@@ -61,7 +61,7 @@ public class ConfigurationDaoImpl extends GenericDaoBase sc = InstanceSearch.create();
             sc.setParameters("instance", "DEFAULT");
 
-            List configurations = listBy(sc);
+            List configurations = listIncludingRemovedBy(sc);
 
             for (ConfigurationVO config : configurations) {
             	if (config.getValue() != null)
@@ -71,7 +71,7 @@ public class ConfigurationDaoImpl extends GenericDaoBase sc = NameSearch.create();
         sc.setParameters("name", name);
-        List configurations = listBy(sc);
+        List configurations = listIncludingRemovedBy(sc);
         
         if (configurations.size() == 0) {
         	return null;
diff --git a/core/src/com/cloud/configuration/dao/ResourceCountDaoImpl.java b/core/src/com/cloud/configuration/dao/ResourceCountDaoImpl.java
index e7921449ff2..e829cf7e2ef 100644
--- a/core/src/com/cloud/configuration/dao/ResourceCountDaoImpl.java
+++ b/core/src/com/cloud/configuration/dao/ResourceCountDaoImpl.java
@@ -52,7 +52,7 @@ public class ResourceCountDaoImpl extends GenericDaoBase
 		sc.setParameters("accountId", accountId);
 		sc.setParameters("type", type);
 
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 
 	private ResourceCountVO findByDomainIdAndType(long domainId, ResourceType type) {
@@ -64,7 +64,7 @@ public class ResourceCountDaoImpl extends GenericDaoBase
 		sc.setParameters("domainId", domainId);
 		sc.setParameters("type", type);
 
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 
 	@Override
diff --git a/core/src/com/cloud/configuration/dao/ResourceLimitDaoImpl.java b/core/src/com/cloud/configuration/dao/ResourceLimitDaoImpl.java
index d6b4a002f22..f266fadfee6 100644
--- a/core/src/com/cloud/configuration/dao/ResourceLimitDaoImpl.java
+++ b/core/src/com/cloud/configuration/dao/ResourceLimitDaoImpl.java
@@ -48,7 +48,7 @@ public class ResourceLimitDaoImpl extends GenericDaoBase
 		sc.setParameters("domainId", domainId);
 		sc.setParameters("type", type);
 		
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 	
 	public List listByDomainId(Long domainId) {
@@ -58,7 +58,7 @@ public class ResourceLimitDaoImpl extends GenericDaoBase
 		SearchCriteria sc = IdTypeSearch.create();
 		sc.setParameters("domainId", domainId);
 		
-		return listBy(sc);
+		return listIncludingRemovedBy(sc);
 	}
 	
 	public ResourceLimitVO findByAccountIdAndType(Long accountId, ResourceCount.ResourceType type) {
@@ -69,7 +69,7 @@ public class ResourceLimitDaoImpl extends GenericDaoBase
 		sc.setParameters("accountId", accountId);
 		sc.setParameters("type", type);
 		
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 	
 	public List listByAccountId(Long accountId) {
@@ -79,7 +79,7 @@ public class ResourceLimitDaoImpl extends GenericDaoBase
 		SearchCriteria sc = IdTypeSearch.create();
 		sc.setParameters("accountId", accountId);
 		
-		return listBy(sc);
+		return listIncludingRemovedBy(sc);
 	}
 	
 	public boolean update(Long id, Long max) {
diff --git a/core/src/com/cloud/dc/AccountVlanMapVO.java b/core/src/com/cloud/dc/AccountVlanMapVO.java
index a5fbb105e1d..b4d6969c7ca 100644
--- a/core/src/com/cloud/dc/AccountVlanMapVO.java
+++ b/core/src/com/cloud/dc/AccountVlanMapVO.java
@@ -25,9 +25,11 @@ import javax.persistence.GenerationType;
 import javax.persistence.Id;
 import javax.persistence.Table;
 
+import com.cloud.user.OwnedBy;
+
 @Entity
 @Table(name="account_vlan_map")
-public class AccountVlanMapVO {
+public class AccountVlanMapVO implements OwnedBy {
 	    
 	@Id
 	@GeneratedValue(strategy=GenerationType.IDENTITY)
@@ -52,7 +54,8 @@ public class AccountVlanMapVO {
 	public Long getId() {
 		return id;
 	}
-	
+
+	@Override
 	public long getAccountId() {
 		return accountId;
 	}
diff --git a/core/src/com/cloud/dc/ClusterVO.java b/core/src/com/cloud/dc/ClusterVO.java
index acb10a023e2..24c4eaa5779 100644
--- a/core/src/com/cloud/dc/ClusterVO.java
+++ b/core/src/com/cloud/dc/ClusterVO.java
@@ -24,11 +24,12 @@ import javax.persistence.GenerationType;
 import javax.persistence.Id;
 import javax.persistence.Table;
 
+import com.cloud.org.Cluster;
 import com.cloud.utils.NumbersUtil;
 
 @Entity
 @Table(name="cluster")
-public class ClusterVO {
+public class ClusterVO implements Cluster {
 
     @Id
     @GeneratedValue(strategy = GenerationType.IDENTITY)
diff --git a/core/src/com/cloud/dc/DataCenterVO.java b/core/src/com/cloud/dc/DataCenterVO.java
index c61c335ec8c..2ddacb29ddb 100644
--- a/core/src/com/cloud/dc/DataCenterVO.java
+++ b/core/src/com/cloud/dc/DataCenterVO.java
@@ -28,12 +28,12 @@ import javax.persistence.TableGenerator;
 
 @Entity
 @Table(name="data_center")
-public class DataCenterVO {
+public class DataCenterVO implements DataCenter {
     
     @Id
     @GeneratedValue(strategy=GenerationType.IDENTITY)
     @Column(name="id")
-    private Long id = null;
+    private long id;
     
 	@Column(name="name")
 	private String name = null;
@@ -66,20 +66,20 @@ public class DataCenterVO {
     @TableGenerator(name="mac_address_sq", table="data_center", pkColumnName="id", valueColumnName="mac_address", allocationSize=1)
     private long macAddress = 1;
     
-    public DataCenterVO(Long id, String name, String description, String dns1, String dns2, String dns3, String dns4, String vnet, String guestCidr) {
+    public DataCenterVO(long id, String name, String description, String dns1, String dns2, String dns3, String dns4, String vnet, String guestCidr) {
+        this(name, description, dns1, dns2, dns3, dns4, vnet, guestCidr);
         this.id = id;
-		this.name = name;
-		this.description = description;
-		this.dns1 = dns1;
-		this.dns2 = dns2;
-		this.internalDns1 = dns3;
-		this.internalDns2 = dns4;
-		this.vnet = vnet;
-		this.guestNetworkCidr = guestCidr;
 	}
 
     public DataCenterVO(String name, String description, String dns1, String dns2, String dns3, String dns4, String vnet, String guestCidr) {
-        this(null, name, description, dns1, dns2, dns3, dns4, vnet, guestCidr);
+        this.name = name;
+        this.description = description;
+        this.dns1 = dns1;
+        this.dns2 = dns2;
+        this.internalDns1 = dns3;
+        this.internalDns2 = dns4;
+        this.vnet = vnet;
+        this.guestNetworkCidr = guestCidr;
     }
     
     public String getDescription() {
@@ -117,7 +117,8 @@ public class DataCenterVO {
 	protected DataCenterVO() {
     }
 
-	public Long getId() {
+	@Override
+    public long getId() {
 		return id;
 	}
 	
diff --git a/core/src/com/cloud/dc/HostPodVO.java b/core/src/com/cloud/dc/HostPodVO.java
index 25f4a227bdd..b0ca1f4dd30 100644
--- a/core/src/com/cloud/dc/HostPodVO.java
+++ b/core/src/com/cloud/dc/HostPodVO.java
@@ -29,7 +29,7 @@ import com.cloud.utils.NumbersUtil;
 
 @Entity
 @Table(name = "host_pod_ref")
-public class HostPodVO {
+public class HostPodVO implements Pod {
 	@Id
 	@GeneratedValue(strategy = GenerationType.IDENTITY)
 	long id;
@@ -47,12 +47,12 @@ public class HostPodVO {
 	private String cidrAddress;
 
 	@Column(name = "cidr_size")
-	private long cidrSize;
+	private int cidrSize;
 
 	@Column(name = "description")
 	private String description;
 
-	public HostPodVO(String name, long dcId, String gateway, String cidrAddress, long cidrSize, String description) {
+	public HostPodVO(String name, long dcId, String gateway, String cidrAddress, int cidrSize, String description) {
 		this.name = name;
 		this.dataCenterId = dcId;
 		this.gateway = gateway;
@@ -67,7 +67,8 @@ public class HostPodVO {
 	protected HostPodVO() {
 	}
 
-	public long getId() {
+	@Override
+    public long getId() {
 		return id;
 	}
 
@@ -87,7 +88,8 @@ public class HostPodVO {
 		this.name = name;
 	}
 
-	public String getCidrAddress() {
+	@Override
+    public String getCidrAddress() {
 		return cidrAddress;
 	}
 
@@ -95,15 +97,17 @@ public class HostPodVO {
 		this.cidrAddress = cidrAddress;
 	}
 
-	public long getCidrSize() {
+	@Override
+    public int getCidrSize() {
 		return cidrSize;
 	}
 
-	public void setCidrSize(long cidrSize) {
+	public void setCidrSize(int cidrSize) {
 		this.cidrSize = cidrSize;
 	}
 	
-	public String getGateway() {
+	@Override
+    public String getGateway() {
 		return gateway;
 	}
 	
diff --git a/core/src/com/cloud/dc/dao/AccountVlanMapDaoImpl.java b/core/src/com/cloud/dc/dao/AccountVlanMapDaoImpl.java
index a59d622090e..ed979d56afe 100644
--- a/core/src/com/cloud/dc/dao/AccountVlanMapDaoImpl.java
+++ b/core/src/com/cloud/dc/dao/AccountVlanMapDaoImpl.java
@@ -38,14 +38,14 @@ public class AccountVlanMapDaoImpl extends GenericDaoBase listAccountVlanMapsByAccount(long accountId) {
 		SearchCriteria sc = AccountSearch.create();
     	sc.setParameters("accountId", accountId);
-    	return listBy(sc);
+    	return listIncludingRemovedBy(sc);
 	}
 	
 	@Override
 	public List listAccountVlanMapsByVlan(long vlanDbId) {
 		SearchCriteria sc = VlanSearch.create();
     	sc.setParameters("vlanDbId", vlanDbId);
-    	return listBy(sc);
+    	return listIncludingRemovedBy(sc);
 	}
 	
 	@Override
@@ -53,7 +53,7 @@ public class AccountVlanMapDaoImpl extends GenericDaoBase sc = AccountVlanSearch.create();
 		sc.setParameters("accountId", accountId);
 		sc.setParameters("vlanDbId", vlanDbId);
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 	
     public AccountVlanMapDaoImpl() {
diff --git a/core/src/com/cloud/dc/dao/ClusterDaoImpl.java b/core/src/com/cloud/dc/dao/ClusterDaoImpl.java
index e1d8ffe4b05..ffb4d2b6fc5 100644
--- a/core/src/com/cloud/dc/dao/ClusterDaoImpl.java
+++ b/core/src/com/cloud/dc/dao/ClusterDaoImpl.java
@@ -45,7 +45,7 @@ public class ClusterDaoImpl extends GenericDaoBase implements C
         SearchCriteria sc = PodSearch.create();
         sc.setParameters("pod", podId);
         
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
@@ -54,6 +54,6 @@ public class ClusterDaoImpl extends GenericDaoBase implements C
         sc.setParameters("pod", podId);
         sc.setParameters("name", name);
         
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 }
diff --git a/core/src/com/cloud/dc/dao/DataCenterDao.java b/core/src/com/cloud/dc/dao/DataCenterDao.java
index 8075e3b34bc..b66ace75bb5 100644
--- a/core/src/com/cloud/dc/dao/DataCenterDao.java
+++ b/core/src/com/cloud/dc/dao/DataCenterDao.java
@@ -18,8 +18,8 @@
 
 package com.cloud.dc.dao;
 
-import java.util.List;
-
+import java.util.List;
+
 import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.DataCenterVnetVO;
 import com.cloud.utils.db.GenericDao;
@@ -39,7 +39,10 @@ public interface DataCenterDao extends GenericDao {
     
     void releaseVnet(String vnet, long dcId, long accountId);
     void releasePrivateIpAddress(String ipAddress, long dcId, Long instanceId);
-    void releaseLinkLocalPrivateIpAddress(String ipAddress, long dcId, Long instanceId);
+    void releasePrivateIpAddress(long nicId);
+    void releaseLinkLocalPrivateIpAddress(String ipAddress, long dcId, Long instanceId);
+    void releaseLinkLocalPrivateIpAddress(long nicId);
+    
     boolean deletePrivateIpAddressByPod(long podId);
     boolean deleteLinkLocalPrivateIpAddressByPod(long podId);
     
diff --git a/core/src/com/cloud/dc/dao/DataCenterDaoImpl.java b/core/src/com/cloud/dc/dao/DataCenterDaoImpl.java
index a7d4b088ce8..9de84194ae9 100644
--- a/core/src/com/cloud/dc/dao/DataCenterDaoImpl.java
+++ b/core/src/com/cloud/dc/dao/DataCenterDaoImpl.java
@@ -66,7 +66,7 @@ public class DataCenterDaoImpl extends GenericDaoBase implem
     public DataCenterVO findByName(String name) {
     	SearchCriteria sc = NameSearch.create();
     	sc.setParameters("name", name);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 
     @Override
@@ -84,6 +84,16 @@ public class DataCenterDaoImpl extends GenericDaoBase implem
         _ipAllocDao.releaseIpAddress(ipAddress, dcId, instanceId);
     }
     
+    @Override
+    public void releasePrivateIpAddress(long nicId) {
+        _ipAllocDao.releaseIpAddress(nicId);
+    }
+    
+    @Override
+    public void releaseLinkLocalPrivateIpAddress(long nicId) {
+        _LinkLocalIpAllocDao.releaseIpAddress(nicId);
+    }
+    
     @Override
     public void releaseLinkLocalPrivateIpAddress(String ipAddress, long dcId, Long instanceId) {
     	_LinkLocalIpAllocDao.releaseIpAddress(ipAddress, dcId, instanceId);
diff --git a/core/src/com/cloud/dc/dao/DataCenterIpAddressDaoImpl.java b/core/src/com/cloud/dc/dao/DataCenterIpAddressDaoImpl.java
index 06cb6726af8..5947d45fbeb 100755
--- a/core/src/com/cloud/dc/dao/DataCenterIpAddressDaoImpl.java
+++ b/core/src/com/cloud/dc/dao/DataCenterIpAddressDaoImpl.java
@@ -136,6 +136,16 @@ public class DataCenterIpAddressDaoImpl extends GenericDaoBase sc = IpDcSearch.create();
+        sc.setParameters("instance", nicId);
+        
+        DataCenterIpAddressVO vo = createForUpdate();
+        vo.setTakenAt(null);
+        vo.setInstanceId(null);
+        update(vo, sc);
+    }
+    
     protected DataCenterIpAddressDaoImpl() {
     	super();
         FreeIpSearch = createSearchBuilder();
@@ -173,7 +183,7 @@ public class DataCenterIpAddressDaoImpl extends GenericDaoBase sc = PodDcSearch.create();
 		sc.setParameters("podId", podId);
 		sc.setParameters("dataCenterId", dcId);
-		return listBy(sc);
+		return listIncludingRemovedBy(sc);
 	}
     
     public List listByPodIdDcIdIpAddress(long podId, long dcId, String ipAddress) {
@@ -181,7 +191,7 @@ public class DataCenterIpAddressDaoImpl extends GenericDaoBase sc = FreeIpSearch.create();
         sc.setParameters("dc", dcId);
         sc.setParameters("pod", podId);
+        sc.setParameters("ipAddr", NetUtils.getLinkLocalGateway());  /*explicitly removing the gateway*/
         
         Transaction txn = Transaction.currentTxn();
         try {
@@ -136,11 +137,23 @@ public class DataCenterLinkLocalIpAddressDaoImpl extends GenericDaoBase sc = IpDcSearch.create();
+        sc.setParameters("instance", nicId);
+
+        DataCenterLinkLocalIpAddressVO vo = createForUpdate();
+        
+        vo.setTakenAt(null);
+        vo.setInstanceId(null);
+        update(vo, sc);
+    }
+    
     protected DataCenterLinkLocalIpAddressDaoImpl() {
     	super();
         FreeIpSearch = createSearchBuilder();
         FreeIpSearch.and("dc", FreeIpSearch.entity().getDataCenterId(), SearchCriteria.Op.EQ);
         FreeIpSearch.and("pod", FreeIpSearch.entity().getPodId(), SearchCriteria.Op.EQ);
+        FreeIpSearch.and("ipAddr", FreeIpSearch.entity().getIpAddress(), SearchCriteria.Op.NEQ);
         FreeIpSearch.and("taken", FreeIpSearch.entity().getTakenAt(), SearchCriteria.Op.NULL);
         FreeIpSearch.done();
         
@@ -173,7 +186,7 @@ public class DataCenterLinkLocalIpAddressDaoImpl extends GenericDaoBase sc = PodDcSearch.create();
 		sc.setParameters("podId", podId);
 		sc.setParameters("dataCenterId", dcId);
-		return listBy(sc);
+		return listIncludingRemovedBy(sc);
 	}
     
     public List listByPodIdDcIdIpAddress(long podId, long dcId, String ipAddress) {
@@ -181,7 +194,7 @@ public class DataCenterLinkLocalIpAddressDaoImpl extends GenericDaoBase listAllocatedVnets(long dcId) {
     	SearchCriteria sc = DcSearchAllocated.create();
     	sc.setParameters("dc", dcId);
-    	return listActiveBy(sc);
+    	return listBy(sc);
     }
     
     public List findVnet(long dcId, String vnet) {
     	SearchCriteria sc = VnetDcSearch.create();;
     	sc.setParameters("dc", dcId);
     	sc.setParameters("vnet", vnet);
-    	return listActiveBy(sc);
+    	return listBy(sc);
     }
     
     public void add(long dcId, int start, int end) {
@@ -114,7 +114,7 @@ public class DataCenterVnetDaoImpl extends GenericDaoBase implements H
 		SearchCriteria sc = DataCenterIdSearch.create();
 		sc.setParameters("dcId", id);
 		
-	    return listActiveBy(sc);
+	    return listBy(sc);
 	}
 	
 	public HostPodVO findByName(String name, long dcId) {
@@ -68,7 +68,7 @@ public class HostPodDaoImpl extends GenericDaoBase implements H
 	    sc.setParameters("dc", dcId);
 	    sc.setParameters("name", name);
 	    
-	    return findOneActiveBy(sc);
+	    return findOneBy(sc);
 	}
 	
 	@Override
diff --git a/core/src/com/cloud/dc/dao/PodVlanDaoImpl.java b/core/src/com/cloud/dc/dao/PodVlanDaoImpl.java
index f29e7bd7070..cfe3375f884 100755
--- a/core/src/com/cloud/dc/dao/PodVlanDaoImpl.java
+++ b/core/src/com/cloud/dc/dao/PodVlanDaoImpl.java
@@ -42,7 +42,7 @@ public class PodVlanDaoImpl extends GenericDaoBase implements G
     public List listAllocatedVnets(long podId) {
     	SearchCriteria sc = PodSearchAllocated.create();
     	sc.setParameters("podId", podId);
-    	return listActiveBy(sc);
+    	return listBy(sc);
     }
     
     public void add(long podId, int start, int end) {
@@ -106,7 +106,7 @@ public class PodVlanDaoImpl extends GenericDaoBase implements G
         sc.setParameters("podId", podId);
         sc.setParameters("account", accountId);
 
-        PodVlanVO vo = findOneBy(sc);
+        PodVlanVO vo = findOneIncludingRemovedBy(sc);
         if (vo == null) {
             return;
         }
diff --git a/core/src/com/cloud/dc/dao/PodVlanMapDaoImpl.java b/core/src/com/cloud/dc/dao/PodVlanMapDaoImpl.java
index 617a084a64b..00410a4002b 100644
--- a/core/src/com/cloud/dc/dao/PodVlanMapDaoImpl.java
+++ b/core/src/com/cloud/dc/dao/PodVlanMapDaoImpl.java
@@ -38,14 +38,14 @@ public class PodVlanMapDaoImpl extends GenericDaoBase implem
 	public List listPodVlanMapsByPod(long podId) {
 		SearchCriteria sc = PodSearch.create();
     	sc.setParameters("podId", podId);
-    	return listBy(sc);
+    	return listIncludingRemovedBy(sc);
 	}
 	
 	@Override
 	public List listPodVlanMapsByVlan(long vlanDbId) {
 		SearchCriteria sc = VlanSearch.create();
     	sc.setParameters("vlanDbId", vlanDbId);
-    	return listBy(sc);
+    	return listIncludingRemovedBy(sc);
 	}
 	
 	@Override
@@ -53,7 +53,7 @@ public class PodVlanMapDaoImpl extends GenericDaoBase implem
 		SearchCriteria sc = PodVlanSearch.create();
 		sc.setParameters("podId", podId);
 		sc.setParameters("vlanDbId", vlanDbId);
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 	
     public PodVlanMapDaoImpl() {
diff --git a/core/src/com/cloud/dc/dao/VlanDao.java b/core/src/com/cloud/dc/dao/VlanDao.java
index b70df875ed9..f96ff50d859 100644
--- a/core/src/com/cloud/dc/dao/VlanDao.java
+++ b/core/src/com/cloud/dc/dao/VlanDao.java
@@ -48,4 +48,8 @@ public interface VlanDao extends GenericDao {
 	
 	boolean zoneHasDirectAttachUntaggedVlans(long zoneId);
 
+	List listZoneWideVlans(long zoneId, VlanType vlanType, String vlanId);
+
+	List searchForZoneWideVlans(long dcId, String vlanType,String vlanId);
+
 }
diff --git a/core/src/com/cloud/dc/dao/VlanDaoImpl.java b/core/src/com/cloud/dc/dao/VlanDaoImpl.java
index 76e78b7b6f7..9bd8f962c70 100644
--- a/core/src/com/cloud/dc/dao/VlanDaoImpl.java
+++ b/core/src/com/cloud/dc/dao/VlanDaoImpl.java
@@ -18,6 +18,9 @@
 
 package com.cloud.dc.dao;
 
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
 import java.util.ArrayList;
 import java.util.List;
 import java.util.Map;
@@ -33,19 +36,25 @@ import com.cloud.dc.Vlan.VlanType;
 import com.cloud.network.dao.IPAddressDao;
 import com.cloud.utils.Pair;
 import com.cloud.utils.component.ComponentLocator;
+import com.cloud.utils.db.DB;
 import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.JoinBuilder;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.exception.CloudRuntimeException;
 
 @Local(value={VlanDao.class})
 public class VlanDaoImpl extends GenericDaoBase implements VlanDao {
     
+	private final String FindZoneWideVlans = "SELECT * FROM vlan WHERE data_center_id=? and vlan_type=? and vlan_id!=? and id not in (select vlan_db_id from account_vlan_map)";
+	
 	protected SearchBuilder ZoneVlanIdSearch;
 	protected SearchBuilder ZoneSearch;
 	protected SearchBuilder ZoneTypeSearch;
 	protected SearchBuilder ZoneTypeAllPodsSearch;
 	protected SearchBuilder ZoneTypePodSearch;
-
+	protected SearchBuilder ZoneVlanSearch;
 
 	protected PodVlanMapDaoImpl _podVlanMapDao = new PodVlanMapDaoImpl();
 	protected AccountVlanMapDao _accountVlanMapDao = new AccountVlanMapDaoImpl();
@@ -56,14 +65,14 @@ public class VlanDaoImpl extends GenericDaoBase implements VlanDao
     	SearchCriteria sc = ZoneVlanIdSearch.create();
     	sc.setParameters("zoneId", zoneId);
     	sc.setParameters("vlanId", vlanId);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
     
     @Override
     public List findByZone(long zoneId) {
     	SearchCriteria sc = ZoneSearch.create();
     	sc.setParameters("zoneId", zoneId);
-    	return listBy(sc);
+    	return listIncludingRemovedBy(sc);
     }
 	
     public VlanDaoImpl() {
@@ -80,14 +89,24 @@ public class VlanDaoImpl extends GenericDaoBase implements VlanDao
         ZoneTypeSearch.and("zoneId", ZoneTypeSearch.entity().getDataCenterId(), SearchCriteria.Op.EQ);
         ZoneTypeSearch.and("vlanType", ZoneTypeSearch.entity().getVlanType(), SearchCriteria.Op.EQ);
         ZoneTypeSearch.done();
+        
     }
 
+    @Override
+    public List listZoneWideVlans(long zoneId, VlanType vlanType, String vlanId){
+    	SearchCriteria sc = ZoneVlanSearch.create();
+    	sc.setParameters("zoneId", zoneId);
+    	sc.setParameters("vlanId", vlanId);
+    	sc.setParameters("vlanType", vlanType);
+    	return listBy(sc);
+    }
+    
 	@Override
 	public List listByZoneAndType(long zoneId, VlanType vlanType) {
 		SearchCriteria sc = ZoneTypeSearch.create();
     	sc.setParameters("zoneId", zoneId);
     	sc.setParameters("vlanType", vlanType);
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -153,7 +172,7 @@ public class VlanDaoImpl extends GenericDaoBase implements VlanDao
         
         SearchBuilder PodVlanSearch = _podVlanMapDao.createSearchBuilder();
         PodVlanSearch.and("podId", PodVlanSearch.entity().getPodId(), SearchCriteria.Op.NNULL);
-        ZoneTypeAllPodsSearch.join("vlan", PodVlanSearch, PodVlanSearch.entity().getVlanDbId(), ZoneTypeAllPodsSearch.entity().getId());
+        ZoneTypeAllPodsSearch.join("vlan", PodVlanSearch, PodVlanSearch.entity().getVlanDbId(), ZoneTypeAllPodsSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         
         ZoneTypeAllPodsSearch.done();
         PodVlanSearch.done();
@@ -164,7 +183,7 @@ public class VlanDaoImpl extends GenericDaoBase implements VlanDao
         
         SearchBuilder PodVlanSearch2 = _podVlanMapDao.createSearchBuilder();
         PodVlanSearch2.and("podId", PodVlanSearch2.entity().getPodId(), SearchCriteria.Op.EQ);
-        ZoneTypePodSearch.join("vlan", PodVlanSearch2,  PodVlanSearch2.entity().getVlanDbId(), ZoneTypePodSearch.entity().getId());
+        ZoneTypePodSearch.join("vlan", PodVlanSearch2,  PodVlanSearch2.entity().getVlanDbId(), ZoneTypePodSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         PodVlanSearch2.done();
         ZoneTypePodSearch.done();
 
@@ -230,7 +249,7 @@ public class VlanDaoImpl extends GenericDaoBase implements VlanDao
     	sc.setParameters("zoneId", zoneId);
     	sc.setParameters("vlanType", VlanType.DirectAttached);
     	
-        return listBy(sc).size() > 0;
+        return listIncludingRemovedBy(sc).size() > 0;
 	}
 
 
@@ -242,7 +261,7 @@ public class VlanDaoImpl extends GenericDaoBase implements VlanDao
     	sc.setParameters("vlanType", VlanType.DirectAttached);
     	sc.setJoinParameters("vlan", "podId", podId);
     	
-    	VlanVO vlan = findOneBy(sc);
+    	VlanVO vlan = findOneIncludingRemovedBy(sc);
     	if (vlan == null) {
     		return null;
     	}
@@ -254,5 +273,33 @@ public class VlanDaoImpl extends GenericDaoBase implements VlanDao
 		return new Pair(ipAddress, vlan);
 
 	}
+	
+	@Override
+	@DB
+	public List searchForZoneWideVlans(long dcId, String vlanType, String vlanId){
+		
+	    StringBuilder sql = new StringBuilder(FindZoneWideVlans);
+
+	    Transaction txn = Transaction.currentTxn();
+		PreparedStatement pstmt = null;
+	    try {
+	        pstmt = txn.prepareAutoCloseStatement(sql.toString());
+	        pstmt.setLong(1, dcId);
+	        pstmt.setString(2, vlanType);
+	        pstmt.setString(3, vlanId);
+	        
+	        ResultSet rs = pstmt.executeQuery();
+	        List zoneWideVlans = new ArrayList();
+
+	        while (rs.next()) {
+	        	zoneWideVlans.add(toEntityBean(rs, false));
+	        }
+	        
+	        return zoneWideVlans;
+	    } catch (SQLException e) {
+	        throw new CloudRuntimeException("Unable to execute " + pstmt.toString(), e);
+	    }
+
+	}
     
 }
diff --git a/core/src/com/cloud/domain/DomainVO.java b/core/src/com/cloud/domain/DomainVO.java
index 90ee26b5976..9b97065723c 100644
--- a/core/src/com/cloud/domain/DomainVO.java
+++ b/core/src/com/cloud/domain/DomainVO.java
@@ -18,26 +18,24 @@
 
 package com.cloud.domain;
 
-import java.util.Date;
-
-import javax.persistence.Column;
-import javax.persistence.Entity;
-import javax.persistence.GeneratedValue;
-import javax.persistence.GenerationType;
-import javax.persistence.Id;
-import javax.persistence.Table;
-
+import java.util.Date;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+
 import com.cloud.utils.db.GenericDao;
 
 @Entity
 @Table(name="domain")
-public class DomainVO {
-    public static final long ROOT_DOMAIN = 1L;
-    
+public class DomainVO implements Domain {
     @Id
     @GeneratedValue(strategy=GenerationType.IDENTITY)
     @Column(name="id")
-    private Long id = null;
+    private long id;
 
     @Column(name="parent")
     private Long parent = null;
@@ -46,13 +44,13 @@ public class DomainVO {
     private String name = null;
 
     @Column(name="owner")
-    private Long owner = null;
+    private long accountId;
     
     @Column(name="path")
     private String path = null;
     
     @Column(name="level")
-    private Integer level = null;
+    private int level;
 
     @Column(name=GenericDao.REMOVED_COLUMN)
     private Date removed;
@@ -65,15 +63,15 @@ public class DomainVO {
 
     public DomainVO() {}
 
-    public DomainVO(String name, Long owner, Long parentId) {
+    public DomainVO(String name, long owner, Long parentId) {
     	this.parent = parentId;
         this.name = name;
-        this.owner = owner;
+        this.accountId = owner;
         this.path ="";
         this.level = 0;
     }
 
-    public Long getId() {
+    public long getId() {
         return id;
     }
 
@@ -100,8 +98,8 @@ public class DomainVO {
         this.name = name;
     }
 
-    public Long getOwner() {
-        return owner;
+    public long getAccountId() {
+        return accountId;
     }
     
     public Date getRemoved() {
@@ -116,11 +114,11 @@ public class DomainVO {
     	this.path = path;
     }
     
-    public Integer getLevel() {
+    public int getLevel() {
     	return level; 
     }
     
-    public void setLevel(Integer level) {
+    public void setLevel(int level) {
     	this.level = level;
     }
     
diff --git a/core/src/com/cloud/domain/dao/DomainDaoImpl.java b/core/src/com/cloud/domain/dao/DomainDaoImpl.java
index dc2633cdd6f..02284978839 100644
--- a/core/src/com/cloud/domain/dao/DomainDaoImpl.java
+++ b/core/src/com/cloud/domain/dao/DomainDaoImpl.java
@@ -188,7 +188,7 @@ public class DomainDaoImpl extends GenericDaoBase implements Dom
     public DomainVO findDomainByPath(String domainPath) {
         SearchCriteria sc = createSearchCriteria();
         sc.addAnd("path", SearchCriteria.Op.EQ, domainPath);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 
     @Override
@@ -205,13 +205,13 @@ public class DomainDaoImpl extends GenericDaoBase implements Dom
         SearchCriteria sc = DomainPairSearch.create();
         sc.setParameters("id", parentId, childId);
 
-        List domainPair = listActiveBy(sc);
+        List domainPair = listBy(sc);
 
         if ((domainPair != null) && (domainPair.size() == 2)) {
             DomainVO d1 = domainPair.get(0);
             DomainVO d2 = domainPair.get(1);
 
-            if (d1.getId().equals(parentId)) {
+            if (d1.getId() == parentId) {
                 result = d2.getPath().startsWith(d1.getPath());
             } else {
                 result = d1.getPath().startsWith(d2.getPath());
diff --git a/core/src/com/cloud/event/EventTypes.java b/core/src/com/cloud/event/EventTypes.java
old mode 100644
new mode 100755
index 4ba66ee4762..de97d8dd864
--- a/core/src/com/cloud/event/EventTypes.java
+++ b/core/src/com/cloud/event/EventTypes.java
@@ -74,17 +74,19 @@ public class EventTypes {
 	//Template Events
 	public static final String EVENT_TEMPLATE_CREATE = "TEMPLATE.CREATE";
 	public static final String EVENT_TEMPLATE_DELETE = "TEMPLATE.DELETE";
-	public static final String EVENT_TEMPLATE_UPDATE = "TEMPLATE.UPDATE";
-	public static final String EVENT_TEMPLATE_COPY = "TEMPLATE.COPY";
-	public static final String EVENT_TEMPLATE_DOWNLOAD_START = "TEMPLATE.DOWNLOAD.START";
-	public static final String EVENT_TEMPLATE_DOWNLOAD_SUCCESS = "TEMPLATE.DOWNLOAD.SUCCESS";
-	public static final String EVENT_TEMPLATE_DOWNLOAD_FAILED = "TEMPLATE.DOWNLOAD.FAILED";
+	public static final String EVENT_TEMPLATE_UPDATE = "TEMPLATE.UPDATE";
+    public static final String EVENT_TEMPLATE_DOWNLOAD_START = "TEMPLATE.DOWNLOAD.START";
+    public static final String EVENT_TEMPLATE_DOWNLOAD_SUCCESS = "TEMPLATE.DOWNLOAD.SUCCESS";
+    public static final String EVENT_TEMPLATE_DOWNLOAD_FAILED = "TEMPLATE.DOWNLOAD.FAILED";
+	public static final String EVENT_TEMPLATE_COPY = "TEMPLATE.COPY";	
+	public static final String EVENT_TEMPLATE_UPLOAD = "TEMPLATE.UPLOAD";	
 	
 	// Volume Events
 	public static final String EVENT_VOLUME_CREATE = "VOLUME.CREATE";
 	public static final String EVENT_VOLUME_DELETE = "VOLUME.DELETE";
 	public static final String EVENT_VOLUME_ATTACH = "VOLUME.ATTACH";
-	public static final String EVENT_VOLUME_DETACH = "VOLUME.DETACH";	
+	public static final String EVENT_VOLUME_DETACH = "VOLUME.DETACH";
+	public static final String EVENT_VOLUME_UPLOAD = "VOLUME.UPLOAD";	
 	
 	// Domains
 	public static final String EVENT_DOMAIN_CREATE = "DOMAIN.CREATE";
@@ -104,6 +106,7 @@ public class EventTypes {
     public static final String EVENT_ISO_COPY = "ISO.COPY";
     public static final String EVENT_ISO_ATTACH = "ISO.ATTACH";
     public static final String EVENT_ISO_DETACH = "ISO.DETACH";
+	public static final String EVENT_ISO_UPLOAD = "ISO.UPLOAD";
     
     //SSVM
     public static final String EVENT_SSVM_CREATE = "SSVM.CREATE";
@@ -118,6 +121,11 @@ public class EventTypes {
 	public static final String EVENT_SERVICE_OFFERING_EDIT = "SERVICE.OFFERING.EDIT";
 	public static final String EVENT_SERVICE_OFFERING_DELETE = "SERVICE.OFFERING.DELETE";
 	
+	// Disk Offerings
+    public static final String EVENT_DISK_OFFERING_CREATE = "DISK.OFFERING.CREATE";
+    public static final String EVENT_DISK_OFFERING_EDIT = "DISK.OFFERING.EDIT";
+    public static final String EVENT_DISK_OFFERING_DELETE = "DISK.OFFERING.DELETE";
+	
 	// Pods
 	public static final String EVENT_POD_CREATE = "POD.CREATE";
 	public static final String EVENT_POD_EDIT = "POD.EDIT";
diff --git a/core/src/com/cloud/event/dao/EventDaoImpl.java b/core/src/com/cloud/event/dao/EventDaoImpl.java
index 45b34d5c3e1..a50c37cbd4b 100644
--- a/core/src/com/cloud/event/dao/EventDaoImpl.java
+++ b/core/src/com/cloud/event/dao/EventDaoImpl.java
@@ -53,7 +53,7 @@ public class EventDaoImpl extends GenericDaoBase implements Event
 
 	@Override
 	public List searchAllEvents(SearchCriteria sc, Filter filter) {
-	    return listBy(sc, filter);
+	    return listIncludingRemovedBy(sc, filter);
 	}
 
     @Override
@@ -61,7 +61,7 @@ public class EventDaoImpl extends GenericDaoBase implements Event
         if (oldTime == null) return null;
         SearchCriteria sc = createSearchCriteria();
         sc.addAnd("createDate", SearchCriteria.Op.LT, oldTime);
-        return listBy(sc, null);
+        return listIncludingRemovedBy(sc, null);
         
     }
     
@@ -72,7 +72,7 @@ public class EventDaoImpl extends GenericDaoBase implements Event
         sc.setParameters("state", EventState.Completed);
         sc.setParameters("startId", 0);
         sc.setParameters("createDate", minTime, maxTime);
-        return listBy(sc, null);
+        return listIncludingRemovedBy(sc, null);
     }
     
     @Override
@@ -80,6 +80,6 @@ public class EventDaoImpl extends GenericDaoBase implements Event
         SearchCriteria sc = CompletedEventSearch.create();
         sc.setParameters("state", EventState.Completed);
         sc.setParameters("startId", startId);
-        return findOneBy(sc);
+        return findOneIncludingRemovedBy(sc);
     }
 }
diff --git a/core/src/com/cloud/ha/dao/HighAvailabilityDaoImpl.java b/core/src/com/cloud/ha/dao/HighAvailabilityDaoImpl.java
index 9d5e5c85773..8d9077c2704 100644
--- a/core/src/com/cloud/ha/dao/HighAvailabilityDaoImpl.java
+++ b/core/src/com/cloud/ha/dao/HighAvailabilityDaoImpl.java
@@ -120,7 +120,7 @@ public class HighAvailabilityDaoImpl extends GenericDaoBase implem
     public List findPreviousHA(final long instanceId) {
         final SearchCriteria sc = PreviousInstanceSearch.create();
         sc.setParameters("instance", instanceId);
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
     }
 
     @Override
@@ -128,7 +128,7 @@ public class HighAvailabilityDaoImpl extends GenericDaoBase implem
         final SearchCriteria sc = CleanupSearch.create();
         sc.setParameters("time", time);
         sc.setParameters("step", HighAvailabilityManager.Step.Done, HighAvailabilityManager.Step.Cancelled);
-        delete(sc);
+        expunge(sc);
     }
 
     @Override
@@ -152,7 +152,7 @@ public class HighAvailabilityDaoImpl extends GenericDaoBase implem
     	sc.setParameters("type", type);
     	sc.setParameters("step", Step.Done, Step.Cancelled, Step.Error);
     	
-    	return listActiveBy(sc);
+    	return listBy(sc);
     }
     
     
@@ -161,7 +161,7 @@ public class HighAvailabilityDaoImpl extends GenericDaoBase implem
     	SearchCriteria sc = PreviousWorkSearch.create();
     	sc.setParameters("instance", instanceId);
     	sc.setParameters("type", type);
-    	return delete(sc) > 0;
+    	return expunge(sc) > 0;
     }
     
     @Override
@@ -169,6 +169,6 @@ public class HighAvailabilityDaoImpl extends GenericDaoBase implem
     	SearchCriteria sc = PreviousWorkSearch.create();
     	sc.setParameters("instance", instanceId);
     	sc.setParameters("type", type);
-    	return listActiveBy(sc, null).size() > 0;
+    	return listBy(sc, null).size() > 0;
     }
 }
\ No newline at end of file
diff --git a/core/src/com/cloud/host/dao/DetailsDaoImpl.java b/core/src/com/cloud/host/dao/DetailsDaoImpl.java
index 6bd554b683d..f4c95e82d11 100644
--- a/core/src/com/cloud/host/dao/DetailsDaoImpl.java
+++ b/core/src/com/cloud/host/dao/DetailsDaoImpl.java
@@ -51,7 +51,7 @@ public class DetailsDaoImpl extends GenericDaoBase implements De
         sc.setParameters("hostId", hostId);
         sc.setParameters("name", name);
         
-        return findOneBy(sc);
+        return findOneIncludingRemovedBy(sc);
     }
 
     @Override
@@ -73,7 +73,7 @@ public class DetailsDaoImpl extends GenericDaoBase implements De
         txn.start();
         SearchCriteria sc = HostSearch.create();
         sc.setParameters("hostId", hostId);
-        delete(sc);
+        expunge(sc);
         
         for (Map.Entry detail : details.entrySet()) {
             DetailVO vo = new DetailVO(hostId, detail.getKey(), detail.getValue());
diff --git a/core/src/com/cloud/host/dao/HostDaoImpl.java b/core/src/com/cloud/host/dao/HostDaoImpl.java
index a58178f9c36..01e5c1e173d 100644
--- a/core/src/com/cloud/host/dao/HostDaoImpl.java
+++ b/core/src/com/cloud/host/dao/HostDaoImpl.java
@@ -206,7 +206,7 @@ public class HostDaoImpl extends GenericDaoBase implements HostDao
         sc.setParameters("status", (Object[])statuses);
         sc.setParameters("pod", podId);
         
-        List rs = searchAll(sc, null);
+        List rs = searchIncludingRemoved(sc, null);
         if (rs.size() == 0) {
             return 0;
         }
@@ -219,7 +219,7 @@ public class HostDaoImpl extends GenericDaoBase implements HostDao
     	SearchCriteria sc = TypeDcSearch.create();
     	sc.setParameters("type", Host.Type.SecondaryStorage);
     	sc.setParameters("dc", dcId);
-    	List storageHosts = listActiveBy(sc);
+    	List storageHosts = listBy(sc);
     	
     	if (storageHosts == null || storageHosts.size() != 1) {
     		return null;
@@ -232,7 +232,7 @@ public class HostDaoImpl extends GenericDaoBase implements HostDao
     public List listSecondaryStorageHosts() {
     	SearchCriteria sc = TypeSearch.create();
     	sc.setParameters("type", Host.Type.SecondaryStorage);
-    	List secondaryStorageHosts = listBy(sc);
+    	List secondaryStorageHosts = listIncludingRemovedBy(sc);
     	
     	return secondaryStorageHosts;
     }
@@ -281,7 +281,7 @@ public class HostDaoImpl extends GenericDaoBase implements HostDao
         sc.setParameters("dc", dcId);
         sc.setParameters("status", Status.Up.toString());
 
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
@@ -290,7 +290,7 @@ public class HostDaoImpl extends GenericDaoBase implements HostDao
         
         sc.setParameters("cluster", clusterId);
         
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
@@ -300,7 +300,7 @@ public class HostDaoImpl extends GenericDaoBase implements HostDao
         sc.setParameters("dc", dcId);
         sc.setParameters("status", Status.Up.toString());
 
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
@@ -309,7 +309,7 @@ public class HostDaoImpl extends GenericDaoBase implements HostDao
         sc.setParameters("dc", dcId);
         sc.setParameters("privateIpAddress", privateIpAddress);
         
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
     
     @Override
@@ -318,7 +318,7 @@ public class HostDaoImpl extends GenericDaoBase implements HostDao
         sc.setParameters("dc", dcId);
         sc.setParameters("storageIpAddress", privateIpAddress);
         
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
     
     @Override
@@ -416,7 +416,7 @@ public class HostDaoImpl extends GenericDaoBase implements HostDao
     @Override
     public HostVO findByGuid(String guid) {
         SearchCriteria sc = GuidSearch.create("guid", guid);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 
     @Override
@@ -425,13 +425,13 @@ public class HostDaoImpl extends GenericDaoBase implements HostDao
         sc.setParameters("ping", timeout);
         sc.setParameters("state", Status.Up.toString(), Status.Updating.toString(),
                 Status.Disconnected.toString(), Status.Down.toString());
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     public List findHostsLike(String hostName) {
     	SearchCriteria sc = NameLikeSearch.create();
         sc.setParameters("name", "%" + hostName + "%");
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -439,25 +439,25 @@ public class HostDaoImpl extends GenericDaoBase implements HostDao
         SearchCriteria sc = LastPingedSearch2.create();
         sc.setParameters("ping", timeout);
         sc.setParameters("type", type.toString());
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
     public List listByDataCenter(long dcId) {
         SearchCriteria sc = DcSearch.create("dc", dcId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     public List listByHostPod(long podId) {
         SearchCriteria sc = PodSearch.create("pod", podId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
     public List listByStatus(Status... status) {
     	SearchCriteria sc = StatusSearch.create();
     	sc.setParameters("status", (Object[])status);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -466,7 +466,7 @@ public class HostDaoImpl extends GenericDaoBase implements HostDao
         sc.setParameters("type", type.toString());
         sc.setParameters("dc", dcId);
 
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -478,7 +478,7 @@ public class HostDaoImpl extends GenericDaoBase implements HostDao
             sc.addAnd("routing_capbable", SearchCriteria.Op.EQ, Integer.valueOf(1));
         }
 
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
     }
 
     protected void saveDetails(HostVO host) {
diff --git a/core/src/com/cloud/hypervisor/kvm/resource/KvmDummyResourceBase.java b/core/src/com/cloud/hypervisor/kvm/resource/KvmDummyResourceBase.java
new file mode 100644
index 00000000000..fce1ce38c8d
--- /dev/null
+++ b/core/src/com/cloud/hypervisor/kvm/resource/KvmDummyResourceBase.java
@@ -0,0 +1,73 @@
+package com.cloud.hypervisor.kvm.resource;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import javax.naming.ConfigurationException;
+
+import com.cloud.agent.api.Answer;
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.PingCommand;
+import com.cloud.agent.api.StartupCommand;
+import com.cloud.agent.api.StartupRoutingCommand;
+import com.cloud.host.Host.Type;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.hypervisor.xen.resource.CitrixResourceBase;
+import com.cloud.resource.ServerResource;
+import com.cloud.resource.ServerResourceBase;
+import com.cloud.vm.State;
+
+public class KvmDummyResourceBase extends ServerResourceBase implements ServerResource {
+	private String _zoneId;
+	private String _podId;
+	private String _clusterId;
+	private String _guid;
+	private String _agentIp;
+	@Override
+	public Type getType() {
+		// TODO Auto-generated method stub
+		return null;
+	}
+
+	@Override
+	public StartupCommand[] initialize() {
+		StartupRoutingCommand cmd = new StartupRoutingCommand(0, 0, 0, 0, null, Hypervisor.Type.KVM, new HashMap(), new HashMap());
+		cmd.setDataCenter(_zoneId);
+		cmd.setPod(_podId);
+		cmd.setCluster(_clusterId);
+		cmd.setGuid(_guid);
+		cmd.setName(_agentIp);
+		cmd.setPrivateIpAddress(_agentIp);
+		cmd.setStorageIpAddress(_agentIp);
+		cmd.setVersion(KvmDummyResourceBase.class.getPackage().getImplementationVersion());
+		return new StartupCommand[] { cmd };
+	}
+
+	@Override
+	public PingCommand getCurrentStatus(long id) {
+		// TODO Auto-generated method stub
+		return null;
+	}
+
+	@Override
+	public Answer executeRequest(Command cmd) {
+		// TODO Auto-generated method stub
+		return null;
+	}
+
+	@Override
+	protected String getDefaultScriptsDir() {
+		// TODO Auto-generated method stub
+		return null;
+	}
+	
+	@Override
+	public boolean configure(final String name, final Map params) throws ConfigurationException {
+		_zoneId = (String)params.get("zone");
+		_podId = (String)params.get("pod");
+		_clusterId = (String)params.get("cluster");
+		_guid = (String)params.get("guid");
+		_agentIp = (String)params.get("agentIp");
+		return true;
+	}
+}
diff --git a/core/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java b/core/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
deleted file mode 100644
index 0df9c7df36c..00000000000
--- a/core/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
+++ /dev/null
@@ -1,122 +0,0 @@
-package com.cloud.hypervisor.vmware.resource;
-
-import java.util.Map;
-
-import javax.naming.ConfigurationException;
-
-import com.cloud.agent.IAgentControl;
-import com.cloud.agent.api.Answer;
-import com.cloud.agent.api.Command;
-import com.cloud.agent.api.PingCommand;
-import com.cloud.agent.api.StartupCommand;
-import com.cloud.agent.api.storage.CopyVolumeAnswer;
-import com.cloud.agent.api.storage.CopyVolumeCommand;
-import com.cloud.agent.api.storage.CreateAnswer;
-import com.cloud.agent.api.storage.CreateCommand;
-import com.cloud.agent.api.storage.DestroyCommand;
-import com.cloud.agent.api.storage.DownloadAnswer;
-import com.cloud.agent.api.storage.PrimaryStorageDownloadCommand;
-import com.cloud.agent.api.storage.ShareAnswer;
-import com.cloud.agent.api.storage.ShareCommand;
-import com.cloud.host.Host.Type;
-import com.cloud.resource.ServerResource;
-import com.cloud.storage.resource.StoragePoolResource;
-
-public class VmwareResource implements StoragePoolResource, ServerResource {
-
-	@Override
-	public DownloadAnswer execute(PrimaryStorageDownloadCommand cmd) {
-		// TODO Auto-generated method stub
-		return null;
-	}
-
-	@Override
-	public Answer execute(DestroyCommand cmd) {
-		// TODO Auto-generated method stub
-		return null;
-	}
-
-	@Override
-	public ShareAnswer execute(ShareCommand cmd) {
-		// TODO Auto-generated method stub
-		return null;
-	}
-
-	@Override
-	public CopyVolumeAnswer execute(CopyVolumeCommand cmd) {
-		// TODO Auto-generated method stub
-		return null;
-	}
-
-	@Override
-	public CreateAnswer execute(CreateCommand cmd) {
-		// TODO Auto-generated method stub
-		return null;
-	}
-
-	@Override
-	public void disconnected() {
-		// TODO Auto-generated method stub
-		
-	}
-
-	@Override
-	public Answer executeRequest(Command cmd) {
-		// TODO Auto-generated method stub
-		return null;
-	}
-
-	@Override
-	public IAgentControl getAgentControl() {
-		// TODO Auto-generated method stub
-		return null;
-	}
-
-	@Override
-	public PingCommand getCurrentStatus(long id) {
-		// TODO Auto-generated method stub
-		return null;
-	}
-
-	@Override
-	public Type getType() {
-		// TODO Auto-generated method stub
-		return null;
-	}
-
-	@Override
-	public StartupCommand[] initialize() {
-		// TODO Auto-generated method stub
-		return null;
-	}
-
-	@Override
-	public void setAgentControl(IAgentControl agentControl) {
-		// TODO Auto-generated method stub
-	}
-
-	@Override
-	public boolean configure(String name, Map params) throws ConfigurationException {
-		
-		// TODO Auto-generated method stub
-		return true;
-	}
-
-	@Override
-	public String getName() {
-		// TODO Auto-generated method stub
-		return null;
-	}
-
-	@Override
-	public boolean start() {
-		// TODO Auto-generated method stub
-		return false;
-	}
-
-	@Override
-	public boolean stop() {
-		// TODO Auto-generated method stub
-		return false;
-	}
-}
diff --git a/core/src/com/cloud/hypervisor/xen/resource/CitrixHelper.java b/core/src/com/cloud/hypervisor/xen/resource/CitrixHelper.java
new file mode 100644
index 00000000000..23de8d0a0af
--- /dev/null
+++ b/core/src/com/cloud/hypervisor/xen/resource/CitrixHelper.java
@@ -0,0 +1,163 @@
+/**
+ *  Copyright (C) 2010 Cloud.com.  All rights reserved.
+ *
+ * This software is licensed under the GNU General Public License v3 or later. 
+ *
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later
+version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ *
+ */
+package com.cloud.hypervisor.xen.resource;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+
+/**
+ * Reduce bloat inside CitrixResourceBase
+ *
+ */
+public class CitrixHelper {
+	private static final HashMap _guestOsMap = new HashMap(70);
+	private static final ArrayList _guestOsList = new ArrayList(70);
+
+
+    static {
+        _guestOsMap.put("CentOS 4.5 (32-bit)", "CentOS 4.5");
+        _guestOsMap.put("CentOS 4.6 (32-bit)", "CentOS 4.6");
+        _guestOsMap.put("CentOS 4.7 (32-bit)", "CentOS 4.7");
+        _guestOsMap.put("CentOS 4.8 (32-bit)", "CentOS 4.8");
+        _guestOsMap.put("CentOS 5.0 (32-bit)", "CentOS 5.0");
+        _guestOsMap.put("CentOS 5.0 (64-bit)", "CentOS 5.0 x64");
+        _guestOsMap.put("CentOS 5.1 (32-bit)", "CentOS 5.1");
+        _guestOsMap.put("CentOS 5.1 (64-bit)", "CentOS 5.1 x64");
+        _guestOsMap.put("CentOS 5.2 (32-bit)", "CentOS 5.2");
+        _guestOsMap.put("CentOS 5.2 (64-bit)", "CentOS 5.2 x64");
+        _guestOsMap.put("CentOS 5.3 (32-bit)", "CentOS 5.3");
+        _guestOsMap.put("CentOS 5.3 (64-bit)", "CentOS 5.3 x64");
+        _guestOsMap.put("CentOS 5.4 (32-bit)", "CentOS 5.4");
+        _guestOsMap.put("CentOS 5.4 (64-bit)", "CentOS 5.4 x64");
+        _guestOsMap.put("Debian Lenny 5.0 (32-bit)", "Debian Lenny 5.0 (32-bit)");
+        _guestOsMap.put("Oracle Enterprise Linux 5.0 (32-bit)", "Oracle Enterprise Linux 5.0");
+        _guestOsMap.put("Oracle Enterprise Linux 5.0 (64-bit)", "Oracle Enterprise Linux 5.0 x64");
+        _guestOsMap.put("Oracle Enterprise Linux 5.1 (32-bit)", "Oracle Enterprise Linux 5.1");
+        _guestOsMap.put("Oracle Enterprise Linux 5.1 (64-bit)", "Oracle Enterprise Linux 5.1 x64");
+        _guestOsMap.put("Oracle Enterprise Linux 5.2 (32-bit)", "Oracle Enterprise Linux 5.2");
+        _guestOsMap.put("Oracle Enterprise Linux 5.2 (64-bit)", "Oracle Enterprise Linux 5.2 x64");
+        _guestOsMap.put("Oracle Enterprise Linux 5.3 (32-bit)", "Oracle Enterprise Linux 5.3");
+        _guestOsMap.put("Oracle Enterprise Linux 5.3 (64-bit)", "Oracle Enterprise Linux 5.3 x64");
+        _guestOsMap.put("Oracle Enterprise Linux 5.4 (32-bit)", "Oracle Enterprise Linux 5.4");
+        _guestOsMap.put("Oracle Enterprise Linux 5.4 (64-bit)", "Oracle Enterprise Linux 5.4 x64");
+        _guestOsMap.put("Red Hat Enterprise Linux 4.5 (32-bit)", "Red Hat Enterprise Linux 4.5");
+        _guestOsMap.put("Red Hat Enterprise Linux 4.6 (32-bit)", "Red Hat Enterprise Linux 4.6");
+        _guestOsMap.put("Red Hat Enterprise Linux 4.7 (32-bit)", "Red Hat Enterprise Linux 4.7");
+        _guestOsMap.put("Red Hat Enterprise Linux 4.8 (32-bit)", "Red Hat Enterprise Linux 4.8");
+        _guestOsMap.put("Red Hat Enterprise Linux 5.0 (32-bit)", "Red Hat Enterprise Linux 5.0");
+        _guestOsMap.put("Red Hat Enterprise Linux 5.0 (64-bit)", "Red Hat Enterprise Linux 5.0 x64");
+        _guestOsMap.put("Red Hat Enterprise Linux 5.1 (32-bit)", "Red Hat Enterprise Linux 5.1");
+        _guestOsMap.put("Red Hat Enterprise Linux 5.1 (64-bit)", "Red Hat Enterprise Linux 5.1 x64");
+        _guestOsMap.put("Red Hat Enterprise Linux 5.2 (32-bit)", "Red Hat Enterprise Linux 5.2");
+        _guestOsMap.put("Red Hat Enterprise Linux 5.2 (64-bit)", "Red Hat Enterprise Linux 5.2 x64");
+        _guestOsMap.put("Red Hat Enterprise Linux 5.3 (32-bit)", "Red Hat Enterprise Linux 5.3");
+        _guestOsMap.put("Red Hat Enterprise Linux 5.3 (64-bit)", "Red Hat Enterprise Linux 5.3 x64");
+        _guestOsMap.put("Red Hat Enterprise Linux 5.4 (32-bit)", "Red Hat Enterprise Linux 5.4");
+        _guestOsMap.put("Red Hat Enterprise Linux 5.4 (64-bit)", "Red Hat Enterprise Linux 5.4 x64");
+        _guestOsMap.put("SUSE Linux Enterprise Server 9 SP4 (32-bit)", "SUSE Linux Enterprise Server 9 SP4");
+        _guestOsMap.put("SUSE Linux Enterprise Server 10 SP1 (32-bit)", "SUSE Linux Enterprise Server 10 SP1");
+        _guestOsMap.put("SUSE Linux Enterprise Server 10 SP1 (64-bit)", "SUSE Linux Enterprise Server 10 SP1 x64");
+        _guestOsMap.put("SUSE Linux Enterprise Server 10 SP2 (32-bit)", "SUSE Linux Enterprise Server 10 SP2");
+        _guestOsMap.put("SUSE Linux Enterprise Server 10 SP2 (64-bit)", "SUSE Linux Enterprise Server 10 SP2 x64");
+        _guestOsMap.put("SUSE Linux Enterprise Server 10 SP3 (64-bit)", "Other install media");
+        _guestOsMap.put("SUSE Linux Enterprise Server 11 (32-bit)", "SUSE Linux Enterprise Server 11");
+        _guestOsMap.put("SUSE Linux Enterprise Server 11 (64-bit)", "SUSE Linux Enterprise Server 11 x64");
+        _guestOsMap.put("Windows 7 (32-bit)", "Windows 7");
+        _guestOsMap.put("Windows 7 (64-bit)", "Windows 7 x64");
+        _guestOsMap.put("Windows Server 2003 (32-bit)", "Windows Server 2003");
+        _guestOsMap.put("Windows Server 2003 (64-bit)", "Windows Server 2003 x64");
+        _guestOsMap.put("Windows Server 2008 (32-bit)", "Windows Server 2008");
+        _guestOsMap.put("Windows Server 2008 (64-bit)", "Windows Server 2008 x64");
+        _guestOsMap.put("Windows Server 2008 R2 (64-bit)", "Windows Server 2008 R2 x64");
+        _guestOsMap.put("Windows 2000 SP4 (32-bit)", "Windows 2000 SP4");
+        _guestOsMap.put("Windows Vista (32-bit)", "Windows Vista");
+        _guestOsMap.put("Windows XP SP2 (32-bit)", "Windows XP SP2");
+        _guestOsMap.put("Windows XP SP3 (32-bit)", "Windows XP SP3");
+        _guestOsMap.put("Other install media", "Other install media");
+
+        //access by index
+        _guestOsList.add("CentOS 4.5");
+        _guestOsList.add("CentOS 4.6");
+        _guestOsList.add("CentOS 4.7");
+        _guestOsList.add("CentOS 4.8");
+        _guestOsList.add("CentOS 5.0");
+        _guestOsList.add("CentOS 5.0 x64");
+        _guestOsList.add("CentOS 5.1");
+        _guestOsList.add("CentOS 5.1 x64");
+        _guestOsList.add("CentOS 5.2");
+        _guestOsList.add("CentOS 5.2 x64");
+        _guestOsList.add("CentOS 5.3");
+        _guestOsList.add("CentOS 5.3 x64");
+        _guestOsList.add("CentOS 5.4");
+        _guestOsList.add("CentOS 5.4 x64");
+        _guestOsList.add("Debian Lenny 5.0 (32-bit)");
+        _guestOsList.add("Oracle Enterprise Linux 5.0");
+        _guestOsList.add("Oracle Enterprise Linux 5.0 x64");
+        _guestOsList.add("Oracle Enterprise Linux 5.1");
+        _guestOsList.add("Oracle Enterprise Linux 5.1 x64");
+        _guestOsList.add("Oracle Enterprise Linux 5.2");
+        _guestOsList.add("Oracle Enterprise Linux 5.2 x64");
+        _guestOsList.add("Oracle Enterprise Linux 5.3");
+        _guestOsList.add("Oracle Enterprise Linux 5.3 x64");
+        _guestOsList.add("Oracle Enterprise Linux 5.4");
+        _guestOsList.add("Oracle Enterprise Linux 5.4 x64");
+        _guestOsList.add("Red Hat Enterprise Linux 4.5");
+        _guestOsList.add("Red Hat Enterprise Linux 4.6");
+        _guestOsList.add("Red Hat Enterprise Linux 4.7");
+        _guestOsList.add("Red Hat Enterprise Linux 4.8");
+        _guestOsList.add("Red Hat Enterprise Linux 5.0");
+        _guestOsList.add("Red Hat Enterprise Linux 5.0 x64");
+        _guestOsList.add("Red Hat Enterprise Linux 5.1");
+        _guestOsList.add("Red Hat Enterprise Linux 5.1 x64");
+        _guestOsList.add("Red Hat Enterprise Linux 5.2");
+        _guestOsList.add("Red Hat Enterprise Linux 5.2 x64");
+        _guestOsList.add("Red Hat Enterprise Linux 5.3");
+        _guestOsList.add("Red Hat Enterprise Linux 5.3 x64");
+        _guestOsList.add("Red Hat Enterprise Linux 5.4");
+        _guestOsList.add("Red Hat Enterprise Linux 5.4 x64");
+        _guestOsList.add("SUSE Linux Enterprise Server 9 SP4");
+        _guestOsList.add("SUSE Linux Enterprise Server 10 SP1");
+        _guestOsList.add("SUSE Linux Enterprise Server 10 SP1 x64");
+        _guestOsList.add("SUSE Linux Enterprise Server 10 SP2");
+        _guestOsList.add("SUSE Linux Enterprise Server 10 SP2 x64");
+        _guestOsList.add("Other install media");
+        _guestOsList.add("SUSE Linux Enterprise Server 11");
+        _guestOsList.add("SUSE Linux Enterprise Server 11 x64");
+        _guestOsList.add("Windows 7");
+        _guestOsList.add("Windows 7 x64");
+        _guestOsList.add("Windows Server 2003");
+        _guestOsList.add("Windows Server 2003 x64");
+        _guestOsList.add("Windows Server 2008");
+        _guestOsList.add("Windows Server 2008 x64");
+        _guestOsList.add("Windows Server 2008 R2 x64");
+        _guestOsList.add("Windows 2000 SP4");
+        _guestOsList.add("Windows Vista");
+        _guestOsList.add("Windows XP SP2");
+        _guestOsList.add("Windows XP SP3");
+        _guestOsList.add("Other install media");
+    }
+    
+    public static String getGuestOsType(String stdType) {
+        return _guestOsMap.get(stdType);
+    }
+
+    public static String getGuestOsType(long guestOsId) {
+        return _guestOsList.get((int) (guestOsId-1));
+    }
+}
diff --git a/core/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java b/core/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java
index 1f5ca0f6827..b30a0996e92 100644
--- a/core/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java
+++ b/core/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java
@@ -1,5 +1,5 @@
 /**
-: *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
  * 
  * This software is licensed under the GNU General Public License v3 or later.
  * 
@@ -17,6 +17,7 @@
  */
 package com.cloud.hypervisor.xen.resource;
 
+
 import java.io.BufferedReader;
 import java.io.File;
 import java.io.FileInputStream;
@@ -104,6 +105,8 @@ import com.cloud.agent.api.RebootCommand;
 import com.cloud.agent.api.RebootRouterCommand;
 import com.cloud.agent.api.SetupAnswer;
 import com.cloud.agent.api.SetupCommand;
+import com.cloud.agent.api.Start2Answer;
+import com.cloud.agent.api.Start2Command;
 import com.cloud.agent.api.StartAnswer;
 import com.cloud.agent.api.StartCommand;
 import com.cloud.agent.api.StartConsoleProxyAnswer;
@@ -141,22 +144,26 @@ import com.cloud.agent.api.storage.DownloadAnswer;
 import com.cloud.agent.api.storage.PrimaryStorageDownloadCommand;
 import com.cloud.agent.api.storage.ShareAnswer;
 import com.cloud.agent.api.storage.ShareCommand;
-import com.cloud.agent.api.to.DiskCharacteristicsTO;
+import com.cloud.agent.api.to.NicTO;
 import com.cloud.agent.api.to.StoragePoolTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
 import com.cloud.agent.api.to.VolumeTO;
 import com.cloud.exception.InternalErrorException;
 import com.cloud.host.Host.Type;
 import com.cloud.hypervisor.Hypervisor;
+import com.cloud.network.Network.BroadcastDomainType;
+import com.cloud.network.Network.TrafficType;
 import com.cloud.resource.ServerResource;
+import com.cloud.storage.Storage;
 import com.cloud.storage.Storage.ImageFormat;
 import com.cloud.storage.Storage.StoragePoolType;
 import com.cloud.storage.StorageLayer;
 import com.cloud.storage.StoragePoolVO;
-import com.cloud.storage.Volume.StorageResourceType;
 import com.cloud.storage.Volume.VolumeType;
 import com.cloud.storage.VolumeVO;
 import com.cloud.storage.resource.StoragePoolResource;
 import com.cloud.storage.template.TemplateInfo;
+import com.cloud.template.VirtualMachineTemplate.BootloaderType;
 import com.cloud.utils.NumbersUtil;
 import com.cloud.utils.Pair;
 import com.cloud.utils.Ternary;
@@ -165,9 +172,11 @@ import com.cloud.utils.exception.CloudRuntimeException;
 import com.cloud.utils.net.NetUtils;
 import com.cloud.utils.script.Script;
 import com.cloud.vm.ConsoleProxyVO;
+import com.cloud.vm.DiskProfile;
 import com.cloud.vm.DomainRouter;
 import com.cloud.vm.SecondaryStorageVmVO;
 import com.cloud.vm.State;
+import com.cloud.vm.VirtualMachine;
 import com.cloud.vm.VirtualMachineName;
 import com.trilead.ssh2.SCPClient;
 import com.xensource.xenapi.APIVersion;
@@ -223,6 +232,8 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
     protected String _guestNetworkName;
     protected int _wait;
     protected IAgentControl _agentControl;
+    protected boolean _isRemoteAgent = false;
+    
 
     protected final XenServerHost _host = new XenServerHost();
 
@@ -259,69 +270,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         s_statesTable.put(Types.VmPowerState.UNKNOWN, State.Unknown);
         s_statesTable.put(Types.VmPowerState.UNRECOGNIZED, State.Unknown);
     }
-    private static HashMap _guestOsType = new HashMap(50);
-    static {
-        _guestOsType.put("CentOS 4.5 (32-bit)", "CentOS 4.5");
-        _guestOsType.put("CentOS 4.6 (32-bit)", "CentOS 4.6");
-        _guestOsType.put("CentOS 4.7 (32-bit)", "CentOS 4.7");
-        _guestOsType.put("CentOS 4.8 (32-bit)", "CentOS 4.8");
-        _guestOsType.put("CentOS 5.0 (32-bit)", "CentOS 5.0");
-        _guestOsType.put("CentOS 5.0 (64-bit)", "CentOS 5.0 x64");
-        _guestOsType.put("CentOS 5.1 (32-bit)", "CentOS 5.1");
-        _guestOsType.put("CentOS 5.1 (64-bit)", "CentOS 5.1 x64");
-        _guestOsType.put("CentOS 5.2 (32-bit)", "CentOS 5.2");
-        _guestOsType.put("CentOS 5.2 (64-bit)", "CentOS 5.2 x64");
-        _guestOsType.put("CentOS 5.3 (32-bit)", "CentOS 5.3");
-        _guestOsType.put("CentOS 5.3 (64-bit)", "CentOS 5.3 x64");
-        _guestOsType.put("CentOS 5.4 (32-bit)", "CentOS 5.4");
-        _guestOsType.put("CentOS 5.4 (64-bit)", "CentOS 5.4 x64");
-        _guestOsType.put("Debian Lenny 5.0 (32-bit)", "Debian Lenny 5.0");
-        _guestOsType.put("Oracle Enterprise Linux 5.0 (32-bit)", "Oracle Enterprise Linux 5.0");
-        _guestOsType.put("Oracle Enterprise Linux 5.0 (64-bit)", "Oracle Enterprise Linux 5.0 x64");
-        _guestOsType.put("Oracle Enterprise Linux 5.1 (32-bit)", "Oracle Enterprise Linux 5.1");
-        _guestOsType.put("Oracle Enterprise Linux 5.1 (64-bit)", "Oracle Enterprise Linux 5.1 x64");
-        _guestOsType.put("Oracle Enterprise Linux 5.2 (32-bit)", "Oracle Enterprise Linux 5.2");
-        _guestOsType.put("Oracle Enterprise Linux 5.2 (64-bit)", "Oracle Enterprise Linux 5.2 x64");
-        _guestOsType.put("Oracle Enterprise Linux 5.3 (32-bit)", "Oracle Enterprise Linux 5.3");
-        _guestOsType.put("Oracle Enterprise Linux 5.3 (64-bit)", "Oracle Enterprise Linux 5.3 x64");
-        _guestOsType.put("Oracle Enterprise Linux 5.4 (32-bit)", "Oracle Enterprise Linux 5.4");
-        _guestOsType.put("Oracle Enterprise Linux 5.4 (64-bit)", "Oracle Enterprise Linux 5.4 x64");
-        _guestOsType.put("Red Hat Enterprise Linux 4.5 (32-bit)", "Red Hat Enterprise Linux 4.5");
-        _guestOsType.put("Red Hat Enterprise Linux 4.6 (32-bit)", "Red Hat Enterprise Linux 4.6");
-        _guestOsType.put("Red Hat Enterprise Linux 4.7 (32-bit)", "Red Hat Enterprise Linux 4.7");
-        _guestOsType.put("Red Hat Enterprise Linux 4.8 (32-bit)", "Red Hat Enterprise Linux 4.8");
-        _guestOsType.put("Red Hat Enterprise Linux 5.0 (32-bit)", "Red Hat Enterprise Linux 5.0");
-        _guestOsType.put("Red Hat Enterprise Linux 5.0 (64-bit)", "Red Hat Enterprise Linux 5.0 x64");
-        _guestOsType.put("Red Hat Enterprise Linux 5.1 (32-bit)", "Red Hat Enterprise Linux 5.1");
-        _guestOsType.put("Red Hat Enterprise Linux 5.1 (64-bit)", "Red Hat Enterprise Linux 5.1 x64");
-        _guestOsType.put("Red Hat Enterprise Linux 5.2 (32-bit)", "Red Hat Enterprise Linux 5.2");
-        _guestOsType.put("Red Hat Enterprise Linux 5.2 (64-bit)", "Red Hat Enterprise Linux 5.2 x64");
-        _guestOsType.put("Red Hat Enterprise Linux 5.3 (32-bit)", "Red Hat Enterprise Linux 5.3");
-        _guestOsType.put("Red Hat Enterprise Linux 5.3 (64-bit)", "Red Hat Enterprise Linux 5.3 x64");
-        _guestOsType.put("Red Hat Enterprise Linux 5.4 (32-bit)", "Red Hat Enterprise Linux 5.4");
-        _guestOsType.put("Red Hat Enterprise Linux 5.4 (64-bit)", "Red Hat Enterprise Linux 5.4 x64");
-        _guestOsType.put("SUSE Linux Enterprise Server 9 SP4 (32-bit)", "SUSE Linux Enterprise Server 9 SP4");
-        _guestOsType.put("SUSE Linux Enterprise Server 10 SP1 (32-bit)", "SUSE Linux Enterprise Server 10 SP1");
-        _guestOsType.put("SUSE Linux Enterprise Server 10 SP1 (64-bit)", "SUSE Linux Enterprise Server 10 SP1 x64");
-        _guestOsType.put("SUSE Linux Enterprise Server 10 SP2 (32-bit)", "SUSE Linux Enterprise Server 10 SP2");
-        _guestOsType.put("SUSE Linux Enterprise Server 10 SP2 (64-bit)", "SUSE Linux Enterprise Server 10 SP2 x64");
-        _guestOsType.put("SUSE Linux Enterprise Server 10 SP3 (64-bit)", "Other install media");
-        _guestOsType.put("SUSE Linux Enterprise Server 11 (32-bit)", "SUSE Linux Enterprise Server 11");
-        _guestOsType.put("SUSE Linux Enterprise Server 11 (64-bit)", "SUSE Linux Enterprise Server 11 x64");
-        _guestOsType.put("Windows 7 (32-bit)", "Windows 7");
-        _guestOsType.put("Windows 7 (64-bit)", "Windows 7 x64");
-        _guestOsType.put("Windows Server 2003 (32-bit)", "Windows Server 2003");
-        _guestOsType.put("Windows Server 2003 (64-bit)", "Windows Server 2003 x64");
-        _guestOsType.put("Windows Server 2008 (32-bit)", "Windows Server 2008");
-        _guestOsType.put("Windows Server 2008 (64-bit)", "Windows Server 2008 x64");
-        _guestOsType.put("Windows Server 2008 R2 (64-bit)", "Windows Server 2008 R2 x64");
-        _guestOsType.put("Windows 2000 SP4 (32-bit)", "Windows 2000 SP4");
-        _guestOsType.put("Windows Vista (32-bit)", "Windows Vista");
-        _guestOsType.put("Windows XP SP2 (32-bit)", "Windows XP SP2");
-        _guestOsType.put("Windows XP SP3 (32-bit)", "Windows XP SP3");
-        _guestOsType.put("Other install media", "Other install media");
-      
-    }
+    
     
     protected boolean isRefNull(XenAPIObject object) {
         return (object == null || object.toWireString().equals("OpaqueRef:NULL"));
@@ -332,10 +281,14 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         s_logger.debug("Logging out of " + _host.uuid);
         if (_host.pool != null) {
             _connPool.disconnect(_host.uuid, _host.pool);
-            _host.pool = null;
         }
     }
 
+    protected VDI cloudVDIcopy(VDI vdi, SR sr) throws BadServerResponse, XenAPIException, XmlRpcException{
+        Connection conn = getConnection();
+        return vdi.copy(conn, sr);
+    }
+    
     protected void destroyStoppedVm() {
         Map vmentries = null;
         Connection conn = getConnection();
@@ -491,15 +444,26 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
     }
 
     protected boolean pingxenserver() {
-        String status;
-        status = callHostPlugin("vmops", "pingxenserver");
-
-        if (status == null || status.isEmpty()) {
+        Session slaveSession = null;
+        Connection slaveConn = null;
+        try {
+            URL slaveUrl = null;
+            slaveUrl = new URL("http://" + _host.ip);
+            slaveConn = new Connection(slaveUrl, 100);
+            slaveSession = Session.slaveLocalLoginWithPassword(slaveConn, _username, _password);
+            return true;
+        } catch (Exception e) {
             return false;
+        } finally {
+            if( slaveSession != null ){
+                try{
+                    Session.localLogout(slaveConn);
+                } catch (Exception e) {
+                    
+                }
+                slaveConn.dispose();
+            }
         }
-
-        return true;
-
     }
 
     protected String logX(XenAPIObject obj, String msg) {
@@ -681,35 +645,381 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             return execute((ModifySshKeysCommand) cmd);
         } else if (cmd instanceof PoolEjectCommand) {
             return execute((PoolEjectCommand) cmd);
+        } else if (cmd instanceof Start2Command) {
+            return execute((Start2Command)cmd);
         } else {
             return Answer.createUnsupportedCommandAnswer(cmd);
         }
     }
+    
+    Pair getNetworkForTraffic(Connection conn, TrafficType type) throws XenAPIException, XmlRpcException {
+        if (type == TrafficType.Guest) {
+            return new Pair(Network.getByUuid(conn, _host.guestNetwork), _host.guestPif);
+        } else if (type == TrafficType.Control) {
+            return new Pair(Network.getByUuid(conn, _host.linkLocalNetwork), null);
+        } else if (type == TrafficType.Management) {
+            return new Pair(Network.getByUuid(conn, _host.privateNetwork), _host.privatePif);
+        } else if (type == TrafficType.Public) {
+            return new Pair(Network.getByUuid(conn, _host.publicNetwork), _host.publicPif);
+        } else if (type == TrafficType.Storage) {
+            return new Pair(Network.getByUuid(conn, _host.storageNetwork1), _host.storagePif1);
+        } else if (type == TrafficType.Vpn) {
+            return new Pair(Network.getByUuid(conn, _host.publicNetwork), _host.publicPif);
+        }
+        
+        throw new CloudRuntimeException("Unsupported network type: " + type);
+    }
+    
+    protected VIF createVif(Connection conn, String vmName, VM vm, NicTO nic) throws XmlRpcException, XenAPIException {
+        VIF.Record vifr = new VIF.Record();
+        vifr.VM = vm;
+        vifr.device = Integer.toString(nic.getDeviceId());
+        vifr.MAC = nic.getMac();
+        
+        Pair network = getNetworkForTraffic(conn, nic.getType());
+        if (nic.getBroadcastType() == BroadcastDomainType.Vlan) {
+            vifr.network = enableVlanNetwork(conn, nic.getVlan(), network.first(), network.second());
+        } else {
+            vifr.network = network.first();
+        }
+        
+        if (nic.getNetworkRateMbps() != null) {
+            vifr.qosAlgorithmType = "ratelimit";
+            vifr.qosAlgorithmParams = new HashMap();
+            // convert mbs to kilobyte per second
+            vifr.qosAlgorithmParams.put("kbps", Integer.toString(nic.getNetworkRateMbps() * 128));
+        }
+        
+        VIF vif = VIF.create(conn, vifr);
+        if (s_logger.isDebugEnabled()) {
+            vifr = vif.getRecord(conn);
+            s_logger.debug("Created a vif " + vifr.uuid + " on " + nic.getDeviceId());
+        }
+        
+        return vif;
+    }
+    
+    protected VDI mount(Connection conn, String vmName, VolumeTO volume) throws XmlRpcException, XenAPIException {
+        if (volume.getType() == VolumeType.ISO) {
+            String isopath = volume.getPath();
+            int index = isopath.lastIndexOf("/");
+
+            String mountpoint = isopath.substring(0, index);
+            URI uri;
+            try {
+                uri = new URI(mountpoint);
+            } catch (URISyntaxException e) {
+                throw new CloudRuntimeException("Incorrect uri " + mountpoint, e);
+            }
+            SR isoSr = createIsoSRbyURI(uri, vmName, false);
+
+            String isoname = isopath.substring(index + 1);
+
+            VDI isoVdi = getVDIbyLocationandSR(isoname, isoSr);
+
+            if (isoVdi == null) {
+                throw new CloudRuntimeException("Unable to find ISO " + volume.getPath());
+            }
+            return isoVdi;
+        } else {
+            return VDI.getByUuid(conn, volume.getPath());
+        }
+    }
+    
+    protected VBD createVbd(Connection conn, String vmName, VM vm, VolumeTO volume, boolean patch) throws XmlRpcException, XenAPIException {
+        VolumeType type = volume.getType();
+        
+        VDI vdi = mount(conn, vmName, volume);
+        
+        if (patch) {
+            if (!patchSystemVm(vdi, vmName)) {
+                throw new CloudRuntimeException("Unable to patch system vm");
+            }
+        }
+        
+        VBD.Record vbdr = new VBD.Record();
+        vbdr.VM = vm;
+        vbdr.VDI = vdi;
+        if (type == VolumeType.ROOT) {
+            vbdr.bootable = true;
+        }
+        vbdr.userdevice = Long.toString(volume.getDeviceId());
+        if (volume.getType() == VolumeType.ISO) {
+            vbdr.mode = Types.VbdMode.RO;
+            vbdr.type = Types.VbdType.CD;
+        } else {
+            vbdr.mode = Types.VbdMode.RW;
+            vbdr.type = Types.VbdType.DISK;
+            
+        }
+        
+        VBD vbd = VBD.create(conn, vbdr);
+        
+        if (s_logger.isDebugEnabled()) {
+            s_logger.debug("VBD " + vbd.getUuid(conn) + " created for " + volume);
+        }
+        
+        return vbd;
+    }
+    
+    protected Pair createVmFromTemplate(Connection conn, VirtualMachineTO vmSpec, Host host) throws XenAPIException, XmlRpcException {
+        String guestOsTypeName = getGuestOsType(vmSpec.getOs());
+        Set templates = VM.getByNameLabel(conn, guestOsTypeName);
+        assert templates.size() == 1 : "Should only have 1 template but found " + templates.size();
+        VM template = templates.iterator().next();
+        
+        VM vm = template.createClone(conn, vmSpec.getName());
+        vm.setAffinity(conn, host);
+        
+        VM.Record vmr = vm.getRecord(conn);
+        if (s_logger.isDebugEnabled()) {
+            s_logger.debug("Created VM " + vmr.uuid + " for " + vmSpec.getName());
+        }
+        
+        for (Console console : vmr.consoles) {
+            console.destroy(conn);
+        }
+        
+        vm.setIsATemplate(conn, false);
+        vm.removeFromOtherConfig(conn, "disks");
+        vm.setNameLabel(conn, vmSpec.getName());
+        setMemory(conn, vm, vmSpec.getMinRam());
+        vm.setVCPUsAtStartup(conn, (long)vmSpec.getCpus());
+        vm.setVCPUsMax(conn, (long)vmSpec.getCpus());
+        vm.setVCPUsNumberLive(conn, (long)vmSpec.getCpus());
+        
+        Map vcpuParams = new HashMap();
+
+        if (vmSpec.getWeight() != null) {
+            vcpuParams.put("weight", Integer.toString(vmSpec.getWeight()));
+        }
+        if (vmSpec.getUtilization() != null) {
+            vcpuParams.put("cap", Integer.toString(vmSpec.getUtilization()));
+        }
+        if (vcpuParams.size() > 0) {
+            vm.setVCPUsParams(conn, vcpuParams);
+        }
+
+        vm.setActionsAfterCrash(conn, Types.OnCrashBehaviour.DESTROY);
+        vm.setActionsAfterShutdown(conn, Types.OnNormalExit.DESTROY);
+        
+        String bootArgs = vmSpec.getBootArgs();
+        if (bootArgs != null && bootArgs.length() > 0) {
+            String pvargs = vm.getPVArgs(conn);
+            pvargs = pvargs + vmSpec.getBootArgs();
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("PV args are " + pvargs);
+            }
+            vm.setPVArgs(conn, pvargs);
+        }
+        
+        if (!(guestOsTypeName.startsWith("Windows") || guestOsTypeName.startsWith("Citrix") || guestOsTypeName.startsWith("Other"))) {
+            if (vmSpec.getBootloader() == BootloaderType.CD) {
+                vm.setPVBootloader(conn, "eliloader");
+                vm.addToOtherConfig(conn, "install-repository", "cdrom");
+            } else if (vmSpec.getBootloader() == BootloaderType.PyGrub ){
+                vm.setPVBootloader(conn, "pygrub");
+            } else {
+                vm.destroy(conn);
+                throw new CloudRuntimeException("Unable to handle boot loader type: " + vmSpec.getBootloader());
+            }
+        }
+        
+        return new Pair(vm, vmr.uuid);
+    }
+    
+    protected String handleVmStartFailure(String vmName, VM vm, String message, Throwable th) {
+        String msg = "Unable to start " + vmName + " due to " + message;
+        s_logger.warn(msg, th);
+        
+        if (vm == null) {
+            return msg;
+        }
+        
+        Connection conn = getConnection();
+        try {
+            VM.Record vmr = vm.getRecord(conn);
+            if (vmr.powerState == VmPowerState.RUNNING) {
+                try {
+                    vm.hardShutdown(conn);
+                } catch (Exception e) {
+                    s_logger.warn("VM hardshutdown failed due to ", e);
+                }
+            }
+            if (vm.getPowerState(conn) == VmPowerState.HALTED) {
+                try {
+                    vm.destroy(conn);
+                } catch (Exception e) {
+                    s_logger.warn("VM destroy failed due to ", e);
+                }
+            }
+            for (VBD vbd : vmr.VBDs) {
+                try {
+                    vbd.unplug(conn);
+                    vbd.destroy(conn);
+                } catch (Exception e) {
+                    s_logger.warn("Unable to clean up VBD due to ", e);
+                }
+            }
+            for (VIF vif : vmr.VIFs) {
+                try {
+                    vif.unplug(conn);
+                    vif.destroy(conn);
+                } catch (Exception e) {
+                    s_logger.warn("Unable to cleanup VIF", e);
+                }
+            }
+        } catch (Exception e) {
+            s_logger.warn("VM getRecord failed due to ", e);
+        }
+        
+        return msg;
+    }
+    
+    protected Start2Answer execute(Start2Command cmd) {
+        VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+        String vmName = vmSpec.getName(); 
+        
+        Connection conn = getConnection();
+        State state = State.Stopped;
+        VM vm = null;
+        try {
+            Host host = Host.getByUuid(conn, _host.uuid);
+            synchronized (_vms) {
+                _vms.put(vmName, State.Starting);
+            }
+            
+            Pair v = createVmFromTemplate(conn, vmSpec, host);
+            vm = v.first();
+            String vmUuid = v.second();
+            
+            for (VolumeTO disk : vmSpec.getDisks()) {
+                createVbd(conn, vmName, vm, disk, disk.getType() == VolumeType.ROOT && vmSpec.getType() != VirtualMachine.Type.User);
+            }
+            
+            NicTO controlNic = null;
+            for (NicTO nic : vmSpec.getNetworks()) {
+                if (nic.getControlPort() != null) {
+                    controlNic = nic;
+                }
+                createVif(conn, vmName, vm, nic);
+            }
+            
+            /*
+             *             
+                VBD.Record vbdr = new VBD.Record();
+                Ternary mount = mounts.get(0);
+                vbdr.VM = vm;
+                vbdr.VDI = mount.second();
+                vbdr.bootable = !bootFromISO;
+                vbdr.userdevice = "0";
+                vbdr.mode = Types.VbdMode.RW;
+                vbdr.type = Types.VbdType.DISK;
+                VBD.create(conn, vbdr);
+
+                for (int i = 1; i < mounts.size(); i++) {
+                    mount = mounts.get(i);
+                    // vdi.setNameLabel(conn, cmd.getVmName() + "-DATA");
+                    vbdr.VM = vm;
+                    vbdr.VDI = mount.second();
+                    vbdr.bootable = false;
+                    vbdr.userdevice = Long.toString(mount.third().getDeviceId());
+                    vbdr.mode = Types.VbdMode.RW;
+                    vbdr.type = Types.VbdType.DISK;
+                    vbdr.unpluggable = true;
+                    VBD.create(conn, vbdr);
+
+                }
+
+                VBD.Record cdromVBDR = new VBD.Record();
+                cdromVBDR.VM = vm;
+                cdromVBDR.empty = true;
+                cdromVBDR.bootable = bootFromISO;
+                cdromVBDR.userdevice = "3";
+                cdromVBDR.mode = Types.VbdMode.RO;
+                cdromVBDR.type = Types.VbdType.CD;
+                VBD cdromVBD = VBD.create(conn, cdromVBDR);
+
+                String isopath = cmd.getISOPath();
+                if (isopath != null) {
+                    int index = isopath.lastIndexOf("/");
+
+                    String mountpoint = isopath.substring(0, index);
+                    URI uri = new URI(mountpoint);
+                    isosr = createIsoSRbyURI(uri, cmd.getVmName(), false);
+
+                    String isoname = isopath.substring(index + 1);
+
+                    VDI isovdi = getVDIbyLocationandSR(isoname, isosr);
+
+                    if (isovdi == null) {
+                        String msg = " can not find ISO " + cmd.getISOPath();
+                        s_logger.warn(msg);
+                        return new StartAnswer(cmd, msg);
+                    } else {
+                        cdromVBD.insert(conn, isovdi);
+                    }
+
+                }
+             */
+            
+            vm.startOn(conn, host, false, true);
+            
+            if (_canBridgeFirewall) {
+                String result = null;
+                if (vmSpec.getType() != VirtualMachine.Type.User) {
+                    result = callHostPlugin("vmops", "default_network_rules_systemvm", "vmName", vmName);
+                } else {
+                }
+                
+                if (result == null || result.isEmpty() || !Boolean.parseBoolean(result)) {
+                    s_logger.warn("Failed to program default network rules for " + vmName);
+                } else {
+                    s_logger.info("Programmed default network rules for " + vmName);
+                }
+            }
+
+            if (controlNic != null) {
+                String privateIp = controlNic.getIp();
+                int cmdPort = controlNic.getControlPort();
+                
+                if (s_logger.isDebugEnabled()) {
+                    s_logger.debug("Ping command port, " + privateIp + ":" + cmdPort);
+                }
+    
+                String result = connect(vmName, privateIp, cmdPort);
+                if (result != null) {
+                    throw new CloudRuntimeException("Can not ping System vm " + vmName + "due to:" + result);
+                } 
+                if (s_logger.isDebugEnabled()) {
+                    s_logger.debug("Ping command port succeeded for vm " + vmName);
+                }
+            }
+            
+            state = State.Running;
+            return new Start2Answer(cmd);
+        } catch (XmlRpcException e) {
+            String msg = handleVmStartFailure(vmName, vm, "", e);
+            return new Start2Answer(cmd, msg);
+        } catch (XenAPIException e) {
+            String msg = handleVmStartFailure(vmName, vm, "", e);
+            return new Start2Answer(cmd, msg);
+        } catch (Exception e) {
+            String msg = handleVmStartFailure(vmName, vm, "", e);
+            return new Start2Answer(cmd, msg);
+        } finally {
+            synchronized (_vms) {
+                if (state != State.Stopped) {
+                    _vms.put(vmName, state);
+                } else {
+                    _vms.remove(vmName);
+                }
+            }
+        }
+    }
 
     protected Answer execute(ModifySshKeysCommand cmd) {
-        String publickey = cmd.getPubKey();
-        String privatekey = cmd.getPrvKey();
-
-        com.trilead.ssh2.Connection sshConnection = new com.trilead.ssh2.Connection(_host.ip, 22);
-        try {
-            sshConnection.connect(null, 60000, 60000);
-            if (!sshConnection.authenticateWithPassword(_username, _password)) {
-                throw new Exception("Unable to authenticate");
-            }
-            SCPClient scp = new SCPClient(sshConnection);
-
-            scp.put(publickey.getBytes(), "id_rsa.pub", "/opt/xensource/bin", "0600");
-            scp.put(privatekey.getBytes(), "id_rsa", "/opt/xensource/bin", "0600");
-            scp.put(privatekey.getBytes(), "id_rsa.cloud", "/root/.ssh", "0600");
-            return new Answer(cmd);
-
-        } catch (Exception e) {
-            String msg = " scp ssh key failed due to " + e.toString() + " - " + e.getMessage();
-            s_logger.warn(msg);
-        } finally {
-            sshConnection.close();
-        }
-        return new Answer(cmd, false, "modifySshkeys failed");
+    	return new Answer(cmd);
     }
 
     private boolean doPingTest(final String computingHostIp) {
@@ -763,7 +1073,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             Map hostMap = Host.getAllRecords(conn);
             if (hostMap.size() == 1) {
                 s_logger.debug("There's no one to take over as master");
-                return new MaintainAnswer(cmd, "Only master in the pool");
+                return new MaintainAnswer(cmd,false, "Only master in the pool");
             }
             Host newMaster = null;
             Host.Record newMasterRecord = null;
@@ -810,7 +1120,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             bootArgs += " pod=" + _pod;
             bootArgs += " localgw=" + _localGateway;
             String result = startSystemVM(vmName, storage.getVlanId(), network, cmd.getVolumes(), bootArgs, storage.getGuestMacAddress(), storage.getGuestIpAddress(), storage
-                    .getPrivateMacAddress(), storage.getPublicMacAddress(), cmd.getProxyCmdPort(), storage.getRamSize());
+                    .getPrivateMacAddress(), storage.getPublicMacAddress(), cmd.getProxyCmdPort(), storage.getRamSize(), storage.getGuestOSId(), cmd.getNetworkRateMbps());
             if (result == null) {
                 return new StartSecStorageVmAnswer(cmd);
             }
@@ -979,7 +1289,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
     }
 
     protected void assignPublicIpAddress(final String vmName, final String privateIpAddress, final String publicIpAddress, final boolean add, final boolean firstIP,
-            final boolean sourceNat, final String vlanId, final String vlanGateway, final String vlanNetmask, final String vifMacAddress) throws InternalErrorException {
+            final boolean sourceNat, final String vlanId, final String vlanGateway, final String vlanNetmask, final String vifMacAddress, String guestIp) throws InternalErrorException {
 
         try {
             Connection conn = getConnection();
@@ -1008,25 +1318,8 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                 if (vifDeviceNum == null) {
                     throw new InternalErrorException("There were no more available slots for a new VIF on router: " + router.getNameLabel(conn));
                 }
-
-                VIF.Record vifr = new VIF.Record();
-                vifr.VM = router;
-                vifr.device = vifDeviceNum;
-                vifr.MAC = vifMacAddress;
-
-                if ("untagged".equalsIgnoreCase(vlanId)) {
-                    vifr.network = Network.getByUuid(conn, _host.publicNetwork);
-                } else {
-                    Network vlanNetwork = enableVlanNetwork(Long.valueOf(vlanId), _host.publicNetwork, _host.publicPif);
-
-                    if (vlanNetwork == null) {
-                        throw new InternalErrorException("Failed to enable VLAN network with tag: " + vlanId);
-                    }
-
-                    vifr.network = vlanNetwork;
-                }
-
-                correctVif = VIF.create(conn, vifr);
+                
+                correctVif = createVIF(conn, router, vifMacAddress, vlanId, 0, vifDeviceNum, true);              
                 correctVif.plug(conn);
                 // Add iptables rule for network usage
                 networkUsage(privateIpAddress, "addVif", "eth" + correctVif.getDevice(conn));
@@ -1036,7 +1329,8 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                 throw new InternalErrorException("Failed to find DomR VIF to associate/disassociate IP with.");
             }
 
-            String args;
+            String args = null;
+            
             if (add) {
                 args = "-A";
             } else {
@@ -1054,6 +1348,11 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             args += " -g ";
             args += vlanGateway;
 
+            if(guestIp!=null){
+            	args += " -G ";
+            	args += guestIp;
+            }
+            
             String result = callHostPlugin("vmops", "ipassoc", "args", args);
             if (result == null || result.isEmpty()) {
                 throw new InternalErrorException("Xen plugin \"ipassoc\" failed.");
@@ -1096,7 +1395,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
     protected Answer execute(final IPAssocCommand cmd) {
         try {
             assignPublicIpAddress(cmd.getRouterName(), cmd.getRouterIp(), cmd.getPublicIp(), cmd.isAdd(), cmd.isFirstIP(), cmd.isSourceNat(), cmd.getVlanId(),
-                    cmd.getVlanGateway(), cmd.getVlanNetmask(), cmd.getVifMacAddress());
+                    cmd.getVlanGateway(), cmd.getVlanNetmask(), cmd.getVifMacAddress(), cmd.getGuestIp());
         } catch (InternalErrorException e) {
             return new Answer(cmd, false, e.getMessage());
         }
@@ -1119,8 +1418,8 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         }
     }
 
-    protected StorageResourceType getStorageResourceType() {
-        return StorageResourceType.STORAGE_POOL;
+    protected Storage.StorageResourceType getStorageResourceType() {
+        return Storage.StorageResourceType.STORAGE_POOL;
     }
 
     protected CheckHealthAnswer execute(CheckHealthCommand cmd) {
@@ -1323,7 +1622,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
 
                 if (param.contains("cpu")) {
                     vmStatsAnswer.setNumCPUs(vmStatsAnswer.getNumCPUs() + 1);
-                    vmStatsAnswer.setCPUUtilization(vmStatsAnswer.getCPUUtilization() + getDataAverage(dataNode, col, numRows));
+                    vmStatsAnswer.setCPUUtilization((vmStatsAnswer.getCPUUtilization() + getDataAverage(dataNode, col, numRows))*100);
                 } else if (param.equals("vif_0_rx")) {
                     vmStatsAnswer.setNetworkReadKBs(getDataAverage(dataNode, col, numRows));
                 } else if (param.equals("vif_0_tx")) {
@@ -1376,8 +1675,15 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         try {
             doc = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(statsSource);
         } catch (Exception e) {
+        	s_logger.warn("Exception caught whilst processing the document via document factory:"+e);
+        	return null;
         }
 
+        if(doc==null){
+        	s_logger.warn("Null document found after tryinh to parse the stats source");
+        	return null;
+        }
+        
         NodeList firstLevelChildren = doc.getChildNodes();
         NodeList secondLevelChildren = (firstLevelChildren.item(0)).getChildNodes();
         Node metaNode = secondLevelChildren.item(0);
@@ -1420,7 +1726,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
 
         if(numRowsUsed == 0)
         {
-        	if((!Double.isInfinite(value))&&(!Double.isInfinite(value)))
+        	if((!Double.isInfinite(value))&&(!Double.isNaN(value)))
         	{
         		return value;
         	}
@@ -1432,7 +1738,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         }
         else
         {
-        	if((!Double.isInfinite(value/numRowsUsed))&&(!Double.isInfinite(value/numRowsUsed)))
+        	if((!Double.isInfinite(value/numRowsUsed))&&(!Double.isNaN(value/numRowsUsed)))
         	{
         		return (value/numRowsUsed);
         	}
@@ -1646,22 +1952,21 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             for (VIF vif : vifs) {
                 Network network = vif.getNetwork(conn);
                 Set pifs = network.getPIFs(conn);
-                Long vlan = null;
+                long vlan = -1;
                 PIF npif = null;
                 for (PIF pif : pifs) {
                     try {
                         vlan = pif.getVLAN(conn);
-                        if (vlan != null) {
+                        if (vlan != -1 ) {
                             VLAN vland = pif.getVLANMasterOf(conn);
                             npif = vland.getTaggedPIF(conn);
-                            break;
                         }
+                        break;
                     }catch (Exception e) {
-                        vlan = null;
                         continue;
                     }
                 }
-                if (vlan == null) {
+                if (npif == null)  {
                     continue;
                 }
                 network = npif.getNetwork(conn);
@@ -1675,7 +1980,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                 } else {
                     continue;
                 }
-                Network vlanNetwork = enableVlanNetwork(vlan, nwuuid, pifuuid);
+                Network vlanNetwork = enableVlanNetwork(vlan, pifuuid);
 
                 if (vlanNetwork == null) {
                     throw new InternalErrorException("Failed to enable VLAN network with tag: " + vlan);
@@ -1693,6 +1998,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         }
     }
 
+    @Override
     public DownloadAnswer execute(final PrimaryStorageDownloadCommand cmd) {
         SR tmpltsr = null;
         String tmplturl = cmd.getUrl();
@@ -1716,6 +2022,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
 
             /* Does the template exist in primary storage pool? If yes, no copy */
             VDI vmtmpltvdi = null;
+            VDI snapshotvdi = null;
 
             Set vdis = VDI.getByNameLabel(conn, "Template " + cmd.getName());
 
@@ -1747,20 +2054,22 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                     s_logger.warn(msg);
                     return new DownloadAnswer(null, 0, msg, com.cloud.storage.VMTemplateStorageResourceAssoc.Status.DOWNLOAD_ERROR, "", "", 0);
                 }
-                vmtmpltvdi = tmpltvdi.copy(conn, poolsr);
-
-                vmtmpltvdi.setNameLabel(conn, "Template " + cmd.getName());
+                vmtmpltvdi = cloudVDIcopy(tmpltvdi, poolsr);
+                snapshotvdi = vmtmpltvdi.snapshot(conn, new HashMap());
+                vmtmpltvdi.destroy(conn);
+                snapshotvdi.setNameLabel(conn, "Template " + cmd.getName());
                 // vmtmpltvdi.setNameDescription(conn, cmd.getDescription());
-                uuid = vmtmpltvdi.getUuid(conn);
+                uuid = snapshotvdi.getUuid(conn);
+                vmtmpltvdi = snapshotvdi;
 
             } else
                 uuid = vmtmpltvdi.getUuid(conn);
 
             // Determine the size of the template
-            long createdSize = vmtmpltvdi.getVirtualSize(conn);
+            long phySize = vmtmpltvdi.getPhysicalUtilisation(conn);
 
             DownloadAnswer answer = new DownloadAnswer(null, 100, cmd, com.cloud.storage.VMTemplateStorageResourceAssoc.Status.DOWNLOADED, uuid, uuid);
-            answer.setTemplateSize(createdSize);
+            answer.setTemplateSize(phySize);
 
             return answer;
 
@@ -2231,46 +2540,47 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         return vm;
     }
 
-    protected String getGuestOsType(String stdType) {
-        return _guestOsType.get(stdType);
-    }
 
-    public boolean joinPool(String address, String username, String password) {
-        Connection conn = getConnection();
+
+    public boolean joinPool(String masterIp, String username, String password) {
+        Connection slaveConn = null;
         Connection poolConn = null;
+        Session slaveSession = null;
+        URL slaveUrl = null;
+        
         try {
-            // set the _host.poolUuid to the old pool uuid in case it's not set.
-            _host.pool = getPoolUuid();
 
             // Connect and find out about the new connection to the new pool.
-            poolConn = _connPool.connect(address, username, password, _wait);
-            Map pools = Pool.getAllRecords(poolConn);
-            Pool.Record pr = pools.values().iterator().next();
-
-            // Now join it.
-            String masterAddr = pr.master.getAddress(poolConn);
-            Pool.join(conn, masterAddr, username, password);
-            if (s_logger.isDebugEnabled()) {
-                s_logger.debug("Joined the pool at " + masterAddr);
+            poolConn = _connPool.masterConnect(masterIp, username, password);
+            //check if this host is already in pool
+            Set hosts = Host.getAll(poolConn);
+            for( Host host : hosts ) {
+                if(host.getAddress(poolConn).equals(_host.ip)) {
+                    return true;
+                }
             }
-            disconnected(); // Logout of our own session.
+            
+            slaveUrl = new URL("http://" + _host.ip);
+            slaveConn = new Connection(slaveUrl, 100);
+            slaveSession = Session.slaveLocalLoginWithPassword(slaveConn, _username, _password);
+            
+            // Now join it.
+
+            Pool.join(slaveConn, masterIp, username, password);
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("Joined the pool at " + masterIp);
+            }
+            
             try {
                 // slave will restart xapi in 10 sec
                 Thread.sleep(10000);
             } catch (InterruptedException e) {
             }
 
-            // Set the pool uuid now to the newest pool.
-            _host.pool = pr.uuid;
-            URL url;
-            try {
-                url = new URL("http://" + _host.ip);
-            } catch (MalformedURLException e1) {
-                throw new CloudRuntimeException("Problem with url " + _host.ip);
-            }
-            Connection c = null;
+            // check if the master of this host is set correctly.
+            Connection c = new Connection(slaveUrl, 100);
             for (int i = 0; i < 15; i++) {
-                c = new Connection(url, _wait);
+
                 try {
                     Session.loginWithPassword(c, _username, _password, APIVersion.latest().toString());
                     s_logger.debug("Still waiting for the conversion to the master");
@@ -2296,18 +2606,33 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                 } catch (InterruptedException e) {
                 }
             }
+
             return true;
+        } catch (MalformedURLException e) {
+            throw new CloudRuntimeException("Problem with url " + _host.ip);
         } catch (XenAPIException e) {
-            String msg = "Unable to allow host " + _host.uuid + " to join pool " + address + " due to " + e.toString();
+            String msg = "Unable to allow host " + _host.uuid
+                    + " to join pool " + masterIp + " due to " + e.toString();
             s_logger.warn(msg, e);
             throw new RuntimeException(msg);
         } catch (XmlRpcException e) {
-            String msg = "Unable to allow host " + _host.uuid + " to join pool " + address + " due to " + e.getMessage();
+            String msg = "Unable to allow host " + _host.uuid
+                    + " to join pool " + masterIp + " due to " + e.getMessage();
             s_logger.warn(msg, e);
             throw new RuntimeException(msg);
         } finally {
             if (poolConn != null) {
-                XenServerConnectionPool.logout(poolConn);
+                try {
+                    Session.logout(poolConn);
+                } catch (Exception e) {
+                }
+                poolConn.dispose();
+            }
+            if(slaveSession != null) {
+                try {
+                    Session.localLogout(slaveConn);
+                } catch (Exception e) {
+                }
             }
         }
     }
@@ -2529,13 +2854,24 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         String msg = "Start VM failed";
         return new StartAnswer(cmd, msg);
     }
-
-    protected void createVIF(Connection conn, VM vm, String mac, String vlanTag, int rate, String devNum, boolean isPub) throws XenAPIException, XmlRpcException,
-            InternalErrorException {
+    
+    protected VIF createVIF(Connection conn, VM vm, String mac, int rate, String devNum, Network network) throws XenAPIException, XmlRpcException,
+    InternalErrorException {
         VIF.Record vifr = new VIF.Record();
         vifr.VM = vm;
         vifr.device = devNum;
         vifr.MAC = mac;
+        vifr.network = network;
+        if ( rate == 0 ) rate = 200;
+        vifr.qosAlgorithmType = "ratelimit";
+        vifr.qosAlgorithmParams = new HashMap();
+        // convert mbs to kilobyte per second 
+        vifr.qosAlgorithmParams.put("kbps", Integer.toString(rate * 128));
+        return VIF.create(conn, vifr);
+    }
+
+    protected VIF createVIF(Connection conn, VM vm, String mac, String vlanTag, int rate, String devNum, boolean isPub) throws XenAPIException, XmlRpcException,
+            InternalErrorException {
 
         String nwUuid = (isPub ? _host.publicNetwork : _host.guestNetwork);
         String pifUuid = (isPub ? _host.publicPif : _host.guestPif);
@@ -2543,21 +2879,13 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         if ("untagged".equalsIgnoreCase(vlanTag)) {
             vlanNetwork = Network.getByUuid(conn, nwUuid);
         } else {
-            vlanNetwork = enableVlanNetwork(Long.valueOf(vlanTag), nwUuid, pifUuid);
+            vlanNetwork = enableVlanNetwork(Long.valueOf(vlanTag), pifUuid);
         }
 
         if (vlanNetwork == null) {
             throw new InternalErrorException("Failed to enable VLAN network with tag: " + vlanTag);
         }
-
-        vifr.network = vlanNetwork;
-        if (rate != 0) {
-            vifr.qosAlgorithmType = "ratelimit";
-            vifr.qosAlgorithmParams = new HashMap();
-            vifr.qosAlgorithmParams.put("kbps", Integer.toString(rate * 1000));
-        }
-
-        VIF.create(conn, vifr);
+        return createVIF(conn, vm, mac, rate, devNum,  vlanNetwork);
     }
 
     protected StopAnswer execute(final StopCommand cmd) {
@@ -2767,7 +3095,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                 Connection conn = getConnection();
                 network = Network.getByUuid(conn, _host.guestNetwork);
             } else {
-                network = enableVlanNetwork(Long.parseLong(tag), _host.guestNetwork, _host.guestPif);
+                network = enableVlanNetwork(Long.parseLong(tag), _host.guestPif);
             }
 
             if (network == null) {
@@ -2777,7 +3105,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             String bootArgs = cmd.getBootArgs();
 
             String result = startSystemVM(vmName, router.getVlanId(), network, cmd.getVolumes(), bootArgs, router.getGuestMacAddress(), router.getPrivateIpAddress(), router
-                    .getPrivateMacAddress(), router.getPublicMacAddress(), 3922, router.getRamSize());
+                    .getPrivateMacAddress(), router.getPublicMacAddress(), 3922, router.getRamSize(), router.getGuestOSId(), cmd.getNetworkRateMbps());
             if (result == null) {
                 networkUsage(router.getPrivateIpAddress(), "create", null);
                 return new StartRouterAnswer(cmd);
@@ -2792,7 +3120,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
     }
 
     protected String startSystemVM(String vmName, String vlanId, Network nw0, List vols, String bootArgs, String guestMacAddr, String privateIp, String privateMacAddr,
-            String publicMacAddr, int cmdPort, long ramSize) {
+            String publicMacAddr, int cmdPort, long ramSize, long guestOsId, int networkRateMbps) {
 
     	setupLinkLocalNetwork();
         VM vm = null;
@@ -2810,21 +3138,12 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
 
             Ternary mount = mounts.get(0);
 
-            if (!patchSystemVm(mount.second(), vmName)) { // FIXME make this
-                // nonspecific
-                String msg = "patch system vm failed";
-                s_logger.warn(msg);
-                return msg;
-            }
-
-            Set templates = VM.getByNameLabel(conn, "CentOS 5.3");
+            Set templates = VM.getByNameLabel(conn, CitrixHelper.getGuestOsType(guestOsId));
             if (templates.size() == 0) {
-                templates = VM.getByNameLabel(conn, "CentOS 5.3 (64-bit)");
-                if (templates.size() == 0) {
-                    String msg = " can not find template CentOS 5.3 ";
-                    s_logger.warn(msg);
-                    return msg;
-                }
+            	String msg = " can not find systemvm template " + CitrixHelper.getGuestOsType(guestOsId) ;
+            	s_logger.warn(msg);
+            	return msg;
+
             }
 
             VM template = templates.iterator().next();
@@ -2855,46 +3174,48 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             vbdr.type = Types.VbdType.DISK;
             VBD.create(conn, vbdr);
 
+            
+            /* create CD-ROM VBD */
+            VBD.Record cdromVBDR = new VBD.Record();
+            cdromVBDR.VM = vm;
+            cdromVBDR.empty = true;
+            cdromVBDR.bootable = false;
+            cdromVBDR.userdevice = "3";
+            cdromVBDR.mode = Types.VbdMode.RO;
+            cdromVBDR.type = Types.VbdType.CD;
+            VBD cdromVBD = VBD.create(conn, cdromVBDR);
+            cdromVBD.insert(conn, VDI.getByUuid(conn, _host.systemvmisouuid));
 
             /* create VIF0 */
-            VIF.Record vifr = new VIF.Record();
-            vifr.VM = vm;
-            vifr.device = "0";
-            vifr.MAC = guestMacAddr;
+            Network network = null;
             if (VirtualMachineName.isValidConsoleProxyName(vmName) || VirtualMachineName.isValidSecStorageVmName(vmName, null)) {
-            	vifr.network = Network.getByUuid(conn, _host.linkLocalNetwork);
-            } else
-            	vifr.network = nw0;
-            VIF.create(conn, vifr);
+            	network = Network.getByUuid(conn, _host.linkLocalNetwork);
+            } else {
+            	network = nw0;
+            }
+            createVIF(conn, vm, guestMacAddr, networkRateMbps, "0", network);
+
             /* create VIF1 */
             /* For routing vm, set its network as link local bridge */
-            vifr.VM = vm;
-            vifr.device = "1";
-            vifr.MAC = privateMacAddr;
             if (VirtualMachineName.isValidRouterName(vmName) && privateIp.startsWith("169.254")) {
-                vifr.network = Network.getByUuid(conn, _host.linkLocalNetwork);
+                network = Network.getByUuid(conn, _host.linkLocalNetwork);
             } else {
-                vifr.network = Network.getByUuid(conn, _host.privateNetwork);
+                network = Network.getByUuid(conn, _host.privateNetwork);
             }
-            VIF.create(conn, vifr);
+            createVIF(conn, vm, privateMacAddr,  networkRateMbps, "1", network);
+
+            /* create VIF2 */            
             if( !publicMacAddr.equalsIgnoreCase("FE:FF:FF:FF:FF:FF") ) {
-                /* create VIF2 */
-                vifr.VM = vm;
-                vifr.device = "2";
-                vifr.MAC = publicMacAddr;
-                vifr.network = Network.getByUuid(conn, _host.publicNetwork);
+                network = null;
                 if ("untagged".equalsIgnoreCase(vlanId)) {
-                    vifr.network = Network.getByUuid(conn, _host.publicNetwork);
+                    network = Network.getByUuid(conn, _host.publicNetwork);
                 } else {
-                    Network vlanNetwork = enableVlanNetwork(Long.valueOf(vlanId), _host.publicNetwork, _host.publicPif);
-    
-                    if (vlanNetwork == null) {
+                    network = enableVlanNetwork(Long.valueOf(vlanId), _host.publicPif);
+                    if (network == null) {
                         throw new InternalErrorException("Failed to enable VLAN network with tag: " + vlanId);
-                    }
-    
-                    vifr.network = vlanNetwork;
+                    }   
                 }
-                VIF.create(conn, vifr);
+                createVIF(conn, vm, publicMacAddr, networkRateMbps, "2", network);
             }
             /* set up PV dom argument */
             String pvargs = vm.getPVArgs(conn);
@@ -2974,7 +3295,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             bootArgs += " localgw=" + _localGateway;
 
             String result = startSystemVM(vmName, proxy.getVlanId(), network, cmd.getVolumes(), bootArgs, proxy.getGuestMacAddress(), proxy.getGuestIpAddress(), proxy
-                    .getPrivateMacAddress(), proxy.getPublicMacAddress(), cmd.getProxyCmdPort(), proxy.getRamSize());
+                    .getPrivateMacAddress(), proxy.getPublicMacAddress(), cmd.getProxyCmdPort(), proxy.getRamSize(), proxy.getGuestOSId(), cmd.getNetworkRateMbps());
             if (result == null) {
                 return new StartConsoleProxyAnswer(cmd);
             }
@@ -3111,12 +3432,18 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             return false;
         return true;
     }
-
+    
     protected String callHostPlugin(String plugin, String cmd, String... params) {
+        //default time out is 300 s
+        return callHostPluginWithTimeOut(plugin, cmd, 300, params);
+    }
+
+    protected String callHostPluginWithTimeOut(String plugin, String cmd, int timeout, String... params) {
         Map args = new HashMap();
+
         try {
             Connection conn = getConnection();
-            Host host = Host.getByUuid(conn, _host.uuid);
+
             for (int i = 0; i < params.length; i += 2) {
                 args.put(params[i], params[i + 1]);
             }
@@ -3124,8 +3451,10 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             if (s_logger.isTraceEnabled()) {
                 s_logger.trace("callHostPlugin executing for command " + cmd + " with " + getArgsString(args));
             }
-
-            String result = host.callPlugin(conn, plugin, cmd, args);
+            if( _host.host == null ) {
+                _host.host = Host.getByUuid(conn, _host.uuid);
+            }
+            String result = _host.host.callPlugin(conn, plugin, cmd, args);
             if (s_logger.isTraceEnabled()) {
                 s_logger.trace("callHostPlugin Result: " + result);
             }
@@ -3290,7 +3619,46 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         return null;
     }
 
-    protected Network enableVlanNetwork(long tag, String networkUuid, String pifUuid) throws XenAPIException, XmlRpcException {
+    protected synchronized Network getNetworkByName(Connection conn, String name, boolean lookForPif) throws XenAPIException, XmlRpcException {
+        Network found = null;
+        Set networks = Network.getByNameLabel(conn, name);
+        if (networks.size() == 1) {
+            found = networks.iterator().next();
+        } else if (networks.size() > 1) {
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("Found more than one network with the name " + name);
+            }
+            for (Network network : networks) {
+                if (!lookForPif) {
+                    found = network;
+                    break;
+                }
+                
+                Network.Record netr = network.getRecord(conn);
+                s_logger.debug("Checking network " + netr.uuid);
+                if (netr.PIFs.size() == 0) {
+                    if (s_logger.isDebugEnabled()) {
+                        s_logger.debug("Network " + netr.uuid + " has no pifs so skipping that.");
+                    }
+                } else {
+                    for (PIF pif : netr.PIFs) {
+                        PIF.Record pifr = pif.getRecord(conn);
+                        if (_host.uuid.equals(pifr.host.getUuid(conn))) {
+                            if (s_logger.isDebugEnabled()) {
+                                s_logger.debug("Network " + netr.uuid + " has a pif " + pifr.uuid + " for our host ");
+                            }
+                            found = network;
+                            break;
+                        }
+                    }
+                }
+            }
+        } 
+
+        return found;
+    }
+    
+    protected Network enableVlanNetwork(long tag, String pifUuid) throws XenAPIException, XmlRpcException {
         // In XenServer, vlan is added by
         // 1. creating a network.
         // 2. creating a vlan associating network with the pif.
@@ -3322,9 +3690,14 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             if (vlanNetworkr.PIFs != null) {
                 for (PIF pif : vlanNetworkr.PIFs) {
                     PIF.Record pifr = pif.getRecord(conn);
-                    if (pifr.device.equals(nPifr.device) && pifr.host.equals(nPifr.host)) {
-                        pif.plug(conn);
-                        return vlanNetwork;
+                    if(pifr.host.equals(nPifr.host)) {
+                        if (pifr.device.equals(nPifr.device) ) {
+                            pif.plug(conn);
+                            return vlanNetwork;
+                        } else {
+                            throw new CloudRuntimeException("Creating VLAN " + tag + " on " + nPifr.device + " failed due to this VLAN is already created on " + pifr.device);                	
+                        }
+                        
                     }
                 }
             }
@@ -3342,6 +3715,60 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         return vlanNetwork;
     }
 
+    protected Network enableVlanNetwork(Connection conn, long tag, Network network, String pifUuid) throws XenAPIException, XmlRpcException {
+        // In XenServer, vlan is added by
+        // 1. creating a network.
+        // 2. creating a vlan associating network with the pif.
+        // We always create
+        // 1. a network with VLAN[vlan id in decimal]
+        // 2. a vlan associating the network created with the pif to private
+        // network.
+
+        Network vlanNetwork = null;
+        String name = "VLAN" + Long.toString(tag);
+
+        vlanNetwork = getNetworkByName(conn, name, true);
+        if (vlanNetwork == null) { // Can't find it, then create it.
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("Creating VLAN network for " + tag + " on host " + _host.ip);
+            }
+            Network.Record nwr = new Network.Record();
+            nwr.nameLabel = name;
+            nwr.bridge = name;
+            vlanNetwork = Network.create(conn, nwr);
+        }
+
+        PIF nPif = PIF.getByUuid(conn, pifUuid);
+        PIF.Record nPifr = nPif.getRecord(conn);
+
+        Network.Record vlanNetworkr = vlanNetwork.getRecord(conn);
+        if (vlanNetworkr.PIFs != null) {
+            for (PIF pif : vlanNetworkr.PIFs) {
+                PIF.Record pifr = pif.getRecord(conn);
+                if (pifr.device.equals(nPifr.device) && pifr.host.equals(nPifr.host)) {
+                    pif.plug(conn);
+                    return vlanNetwork;
+                }
+            }
+        }
+
+        if (s_logger.isDebugEnabled()) {
+            s_logger.debug("Creating VLAN " + tag + " on host " + _host.ip + " on device " + nPifr.device);
+        }
+        VLAN vlan = VLAN.create(conn, nPif, tag, vlanNetwork);
+        VLAN.Record vlanr = vlan.getRecord(conn);
+        if (s_logger.isDebugEnabled()) {
+            s_logger.debug("VLAN is created for " + tag + ".  The uuid is " + vlanr.uuid);
+        }
+        
+        PIF untaggedPif = vlanr.untaggedPIF;
+        if (!untaggedPif.getCurrentlyAttached(conn)) {
+            untaggedPif.plug(conn);
+        }
+
+        return vlanNetwork;
+    }
+    
     protected void disableVlanNetwork(Network network) throws InternalErrorException {
         try {
             Connection conn = getConnection();
@@ -3355,13 +3782,9 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         } catch (XenAPIException e) {
             String msg = "Unable to disable VLAN network due to " + e.toString();
             s_logger.warn(msg, e);
-            throw new InternalErrorException(msg);
-        } catch (XmlRpcException e) {
+        } catch (Exception e) {
             String msg = "Unable to disable VLAN network due to " + e.getMessage();
             s_logger.warn(msg, e);
-            throw new InternalErrorException(msg);
-        } catch (Exception e) {
-            throw new InternalErrorException(e.getMessage());
         }
     }
 
@@ -3423,7 +3846,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             StartupStorageCommand cmd = new StartupStorageCommand();
             cmd.setPoolInfo(pInfo);
             cmd.setGuid(_host.uuid);
-            cmd.setResourceType(StorageResourceType.STORAGE_POOL);
+            cmd.setResourceType(Storage.StorageResourceType.STORAGE_POOL);
             return cmd;
         } catch (XenAPIException e) {
             String msg = "build startupstoragecommand err in host:" + _host.uuid + e.toString();
@@ -3509,14 +3932,48 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             s_logger.debug("Can't get xs-tools.iso: " + e.toString());
         }
     }
+    
+    protected boolean can_bridge_firewall() {
+        return false;
+    }
 
     protected boolean getHostInfo() throws IllegalArgumentException{
         Connection conn = getConnection();
 
         try {
             Host myself = Host.getByUuid(conn, _host.uuid);
-            _host.pool = getPoolUuid();
+            
+            boolean findsystemvmiso = false;
+            Set srs = SR.getByNameLabel(conn, "XenServer Tools");
+            if( srs.size() != 1 ) {
+            	throw new CloudRuntimeException("There are " + srs.size() + " SRs with name XenServer Tools");
+            }
+            SR sr = srs.iterator().next();
+            sr.scan(conn);
 
+            SR.Record srr = sr.getRecord(conn);
+            _host.systemvmisouuid = null;
+            for( VDI vdi : srr.VDIs ) {
+            	VDI.Record vdir = vdi.getRecord(conn);
+           		if(vdir.nameLabel.contains("systemvm-premium")){
+           			_host.systemvmisouuid = vdir.uuid;
+           			break;
+            	}                   	
+            }
+            if(  _host.systemvmisouuid == null ) {
+                for( VDI vdi : srr.VDIs ) {
+                    VDI.Record vdir = vdi.getRecord(conn);
+                        if(vdir.nameLabel.contains("systemvm")){
+                            _host.systemvmisouuid = vdir.uuid;
+                            break;
+                     }
+                }
+            }
+
+            if(  _host.systemvmisouuid == null ) {
+            	throw new CloudRuntimeException("can not find systemvmiso");
+            } 
+            
             String name = "cloud-private";
             if (_privateNetworkName != null) {
                 name = _privateNetworkName;
@@ -3528,7 +3985,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                 return false;
             }
 
-            _canBridgeFirewall = Boolean.valueOf(callHostPlugin("vmops", "can_bridge_firewall", "host_uuid", _host.uuid));
+            _canBridgeFirewall = can_bridge_firewall();
 
             Nic privateNic = getLocalNetwork(conn, name);
             if (privateNic == null) {
@@ -3538,6 +3995,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                     s_logger.warn("Unable to determine the private network for host " + _host.ip);
                     return false;
                 }
+                _privateNetworkName = name;
                 privateNic = getLocalNetwork(conn, name);
                 if (privateNic == null) {
                     s_logger.warn("Unable to get private network " + name);
@@ -3556,6 +4014,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             	}
             } else {
             	guestNic = privateNic;
+            	_guestNetworkName = _privateNetworkName;
             }
             _host.guestNetwork = guestNic.nr.uuid;
             _host.guestPif = guestNic.pr.uuid;
@@ -3569,6 +4028,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                 }
             } else {
                 publicNic = guestNic;
+                _publicNetworkName = _guestNetworkName;
             }
             _host.publicPif = publicNic.pr.uuid;
             _host.publicNetwork = publicNic.nr.uuid;
@@ -3591,6 +4051,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             _host.storagePif2 = storageNic2.pr.uuid;
             
             s_logger.info("Private Network is " + _privateNetworkName + " for host " + _host.ip);
+            s_logger.info("Guest Network is " + _guestNetworkName + " for host " + _host.ip);
             s_logger.info("Public Network is " + _publicNetworkName + " for host " + _host.ip);
             s_logger.info("Storage Network 1 is " + _storageNetworkName1 + " for host " + _host.ip);
             s_logger.info("Storage Network 2 is " + _storageNetworkName2 + " for host " + _host.ip);
@@ -3737,9 +4198,6 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         cmd.setCluster(_cluster);
 
         StartupStorageCommand sscmd = initializeLocalSR();
-
-        _host.pool = getPoolUuid();
-
         if (sscmd != null) {
             /* report pv driver iso */
             getPVISO(sscmd);
@@ -3797,63 +4255,64 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                 }
 
                 SCPClient scp = new SCPClient(sshConnection);
-                File file = new File(_patchPath);
-
-                Properties props = new Properties();
-                props.load(new FileInputStream(file));
 
                 String path = _patchPath.substring(0, _patchPath.lastIndexOf(File.separator) + 1);
-                for (Map.Entry entry : props.entrySet()) {
-                    String k = (String) entry.getKey();
-                    String v = (String) entry.getValue();
-
-                    assert (k != null && k.length() > 0 && v != null && v.length() > 0) : "Problems with " + k + "=" + v;
-
-                    String[] tokens = v.split(",");
-                    String f = null;
-                    if (tokens.length == 3 && tokens[0].length() > 0) {
-                        if (tokens[0].startsWith("/")) {
-                            f = tokens[0];
-                        } else if (tokens[0].startsWith("~")) {
-                            String homedir = System.getenv("HOME");
-                            f = homedir + tokens[0].substring(1) + k;
-                        } else {
-                            f = path + tokens[0] + '/' + k;
-                        }
-                    } else {
-                        f = path + k;
-                    }
-                    String d = tokens[tokens.length - 1];
-                    f = f.replace('/', File.separatorChar);
-
-                    String p = "0755";
-                    if (tokens.length == 3) {
-                        p = tokens[1];
-                    } else if (tokens.length == 2) {
-                        p = tokens[0];
-                    }
-
-                    if (!new File(f).exists()) {
-                        s_logger.warn("We cannot locate " + f);
-                        continue;
-                    }
-                    if (s_logger.isDebugEnabled()) {
-                        s_logger.debug("Copying " + f + " to " + d + " on " + hr.address + " with permission " + p);
-                    }
-                    scp.put(f, d, p);
-
+                List files = getPatchFiles();
+                if( files == null || files.isEmpty() ) {
+                    throw new CloudRuntimeException("Can not find patch file");
                 }
+                for( File file :files) {
+	                Properties props = new Properties();
+	                props.load(new FileInputStream(file));
+	
+	                for (Map.Entry entry : props.entrySet()) {
+	                    String k = (String) entry.getKey();
+	                    String v = (String) entry.getValue();
+	
+	                    assert (k != null && k.length() > 0 && v != null && v.length() > 0) : "Problems with " + k + "=" + v;
+	
+	                    String[] tokens = v.split(",");
+	                    String f = null;
+	                    if (tokens.length == 3 && tokens[0].length() > 0) {
+	                        if (tokens[0].startsWith("/")) {
+	                            f = tokens[0];
+	                        } else if (tokens[0].startsWith("~")) {
+	                            String homedir = System.getenv("HOME");
+	                            f = homedir + tokens[0].substring(1) + k;
+	                        } else {
+	                            f = path + tokens[0] + '/' + k;
+	                        }
+	                    } else {
+	                        f = path + k;
+	                    }
+	                    String d = tokens[tokens.length - 1];
+	                    f = f.replace('/', File.separatorChar);
+	
+	                    String p = "0755";
+	                    if (tokens.length == 3) {
+	                        p = tokens[1];
+	                    } else if (tokens.length == 2) {
+	                        p = tokens[0];
+	                    }
+	
+	                    if (!new File(f).exists()) {
+	                        s_logger.warn("We cannot locate " + f);
+	                        continue;
+	                    }
+	                    if (s_logger.isDebugEnabled()) {
+	                        s_logger.debug("Copying " + f + " to " + d + " on " + hr.address + " with permission " + p);
+	                    }
+	                    scp.put(f, d, p);
+	
+	                }
+                }
+
             } catch (IOException e) {
                 throw new CloudRuntimeException("Unable to setup the server correctly", e);
             } finally {
                 sshConnection.close();
             }
-            try {
-                // wait 2 seconds before call plugin
-                Thread.sleep(2000);
-            } catch (final InterruptedException ex) {
 
-            }
             if (!setIptables()) {
                 s_logger.warn("set xenserver Iptable failed");
             }
@@ -3871,6 +4330,13 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         }
     }
 
+    protected List getPatchFiles() {
+        List files = new ArrayList();
+        File file = new File(_patchPath);
+        files.add(file);
+        return files;
+    }
+
     protected SR getSRByNameLabelandHost(String name) throws BadServerResponse, XenAPIException, XmlRpcException {
         Connection conn = getConnection();
         Set srs = SR.getByNameLabel(conn, name);
@@ -3928,9 +4394,8 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             SR.Record srr = sr.getRecord(conn);
             Set pbds = sr.getPBDs(conn);
             if (pbds.size() == 0) {
-                String msg = "There is no PBDs for this SR: " + _host.uuid;
+                String msg = "There is no PBDs for this SR: " + srr.nameLabel + " on host:" + _host.uuid;
                 s_logger.warn(msg);
-                removeSR(sr);
                 return false;
             }
             Set hosts = null;
@@ -3984,17 +4449,11 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
 
     protected Answer execute(ModifyStoragePoolCommand cmd) {
         StoragePoolVO pool = cmd.getPool();
+        StoragePoolTO poolTO = new StoragePoolTO(pool);
         try {
             Connection conn = getConnection();
 
-            SR sr = getStorageRepository(conn, pool);
-            if (!checkSR(sr)) {
-                String msg = "ModifyStoragePoolCommand checkSR failed! host:" + _host.uuid + " pool: " + pool.getName() + pool.getHostAddress() + pool.getPath();
-                s_logger.warn(msg);
-                return new Answer(cmd, false, msg);
-            }
-            sr.setNameLabel(conn, pool.getUuid());
-            sr.setNameDescription(conn, pool.getName());
+            SR sr = getStorageRepository(conn, poolTO);
             long capacity = sr.getPhysicalSize(conn);
             long available = capacity - sr.getPhysicalUtilisation(conn);
             if (capacity == -1) {
@@ -4019,14 +4478,10 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
 
     protected Answer execute(DeleteStoragePoolCommand cmd) {
         StoragePoolVO pool = cmd.getPool();
+        StoragePoolTO poolTO = new StoragePoolTO(pool);
         try {
             Connection conn = getConnection();
-            SR sr = getStorageRepository(conn, pool);
-            if (!checkSR(sr)) {
-                String msg = "DeleteStoragePoolCommand checkSR failed! host:" + _host.uuid + " pool: " + pool.getName() + pool.getHostAddress() + pool.getPath();
-                s_logger.warn(msg);
-                return new Answer(cmd, false, msg);
-            }
+            SR sr = getStorageRepository(conn, poolTO);
             sr.setNameLabel(conn, pool.getUuid());
             sr.setNameDescription(conn, pool.getName());
 
@@ -4045,7 +4500,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
     }
 
     public Connection getConnection() {
-        return _connPool.connect(_host.uuid, _host.ip, _username, _password, _wait);
+        return _connPool.connect(_host.uuid, _host.pool, _host.ip, _username, _password, _wait);
     }
 
     protected void fillHostInfo(StartupRoutingCommand cmd) {
@@ -4112,6 +4567,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                     break;
                 }
             }
+            
             // assume the memory Virtualization overhead is 1/64
             ram = (ram - dom0Ram) * 63/64;
             cmd.setMemory(ram);
@@ -4176,8 +4632,10 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         } catch (NumberFormatException e) {
             throw new ConfigurationException("Unable to get the zone " + params.get("zone"));
         }
+        _host.host = null;
         _name = _host.uuid;
         _host.ip = (String) params.get("url");
+        _host.pool = (String) params.get("pool");
         _username = (String) params.get("username");
         _password = (String) params.get("password");
         _pod = (String) params.get("pod");
@@ -4222,8 +4680,6 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             throw new ConfigurationException("Unable to get the uuid");
         }
 
-        params.put("domr.scripts.dir", "scripts/network/domr");
-
         String patchPath = getPatchPath();
 
         _patchPath = Script.findScript(patchPath, "patch");
@@ -4261,9 +4717,10 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         }
     }
 
+    @Override
     public CreateAnswer execute(CreateCommand cmd) {
         StoragePoolTO pool = cmd.getPool();
-        DiskCharacteristicsTO dskch = cmd.getDiskCharacteristics();
+        DiskProfile dskch = cmd.getDiskCharacteristics();
 
         VDI vdi = null;
         Connection conn = getConnection();
@@ -4293,7 +4750,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             vdir = vdi.getRecord(conn);
             s_logger.debug("Succesfully created VDI for " + cmd + ".  Uuid = " + vdir.uuid);
 
-            VolumeTO vol = new VolumeTO(cmd.getVolumeId(), dskch.getType(), StorageResourceType.STORAGE_POOL, pool.getType(), vdir.nameLabel, pool.getPath(), vdir.uuid,
+            VolumeTO vol = new VolumeTO(cmd.getVolumeId(), dskch.getType(), Storage.StorageResourceType.STORAGE_POOL, pool.getType(), vdir.nameLabel, pool.getPath(), vdir.uuid,
                     vdir.virtualSize);
             return new CreateAnswer(cmd, vol);
         } catch (Exception e) {
@@ -4434,119 +4891,10 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             s_logger.warn(msg, e);
             throw new CloudRuntimeException(msg, e);
         }
-
     }
 
-    protected SR getIscsiSR(Connection conn, StoragePoolVO pool) {
-
-        synchronized (pool.getUuid().intern()) {
-            Map deviceConfig = new HashMap();
-            try {
-                String target = pool.getHostAddress().trim();
-                String path = pool.getPath().trim();
-                if (path.endsWith("/")) {
-                    path = path.substring(0, path.length() - 1);
-                }
-
-                String tmp[] = path.split("/");
-                if (tmp.length != 3) {
-                    String msg = "Wrong iscsi path " + pool.getPath() + " it should be /targetIQN/LUN";
-                    s_logger.warn(msg);
-                    throw new CloudRuntimeException(msg);
-                }
-                String targetiqn = tmp[1].trim();
-                String lunid = tmp[2].trim();
-                String scsiid = "";
-
-                Set srs = SR.getByNameLabel(conn, pool.getUuid());
-                for (SR sr : srs) {
-                    if (!SRType.LVMOISCSI.equals(sr.getType(conn)))
-                        continue;
-
-                    Set pbds = sr.getPBDs(conn);
-                    if (pbds.isEmpty())
-                        continue;
-
-                    PBD pbd = pbds.iterator().next();
-
-                    Map dc = pbd.getDeviceConfig(conn);
-
-                    if (dc == null)
-                        continue;
-
-                    if (dc.get("target") == null)
-                        continue;
-
-                    if (dc.get("targetIQN") == null)
-                        continue;
-
-                    if (dc.get("lunid") == null)
-                        continue;
-
-                    if (target.equals(dc.get("target")) && targetiqn.equals(dc.get("targetIQN")) && lunid.equals(dc.get("lunid"))) {
-                        return sr;
-                    }
-
-                }
-                deviceConfig.put("target", target);
-                deviceConfig.put("targetIQN", targetiqn);
-
-                Host host = Host.getByUuid(conn, _host.uuid);
-                SR sr = null;
-                try {
-                    sr = SR.create(conn, host, deviceConfig, new Long(0), pool.getUuid(), pool.getName(), SRType.LVMOISCSI.toString(), "user", true, new HashMap());
-                } catch (XenAPIException e) {
-                    String errmsg = e.toString();
-                    if (errmsg.contains("SR_BACKEND_FAILURE_107")) {
-                        String lun[] = errmsg.split("");
-                        boolean found = false;
-                        for (int i = 1; i < lun.length; i++) {
-                            int blunindex = lun[i].indexOf("") + 7;
-                            int elunindex = lun[i].indexOf("");
-                            String ilun = lun[i].substring(blunindex, elunindex);
-                            ilun = ilun.trim();
-                            if (ilun.equals(lunid)) {
-                                int bscsiindex = lun[i].indexOf("") + 8;
-                                int escsiindex = lun[i].indexOf("");
-                                scsiid = lun[i].substring(bscsiindex, escsiindex);
-                                scsiid = scsiid.trim();
-                                found = true;
-                                break;
-                            }
-                        }
-                        if (!found) {
-                            String msg = "can not find LUN " + lunid + " in " + errmsg;
-                            s_logger.warn(msg);
-                            throw new CloudRuntimeException(msg);
-                        }
-                    } else {
-                        String msg = "Unable to create Iscsi SR  " + deviceConfig + " due to  " + e.toString();
-                        s_logger.warn(msg, e);
-                        throw new CloudRuntimeException(msg, e);
-                    }
-                }
-                deviceConfig.put("SCSIid", scsiid);
-                sr = SR.create(conn, host, deviceConfig, new Long(0), pool.getUuid(), pool.getName(), SRType.LVMOISCSI.toString(), "user", true, new HashMap());
-                if( !checkSR(sr) ) {
-                    throw new Exception("no attached PBD");
-                }           
-                sr.scan(conn);
-                return sr;
-
-            } catch (XenAPIException e) {
-                String msg = "Unable to create Iscsi SR  " + deviceConfig + " due to  " + e.toString();
-                s_logger.warn(msg, e);
-                throw new CloudRuntimeException(msg, e);
-            } catch (Exception e) {
-                String msg = "Unable to create Iscsi SR  " + deviceConfig + " due to  " + e.getMessage();
-                s_logger.warn(msg, e);
-                throw new CloudRuntimeException(msg, e);
-            }
-        }
-    }
-
-    protected SR getIscsiSR(Connection conn, StoragePoolTO pool) {
-
+    protected SR getIscsiSR(StoragePoolTO pool) {
+        Connection conn = getConnection();
         synchronized (pool.getUuid().intern()) {
             Map deviceConfig = new HashMap();
             try {
@@ -4595,6 +4943,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                         if (checkSR(sr)) {
                             return sr;
                         }
+                        throw new CloudRuntimeException("SR check failed for storage pool: " + pool.getUuid() + "on host:" + _host.uuid);
                     }
 
                 }
@@ -4654,13 +5003,12 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         }
     }
 
-    protected SR getNfsSR(StoragePoolVO pool) {
+    protected SR getNfsSR(StoragePoolTO pool) {
         Connection conn = getConnection();
 
         Map deviceConfig = new HashMap();
         try {
-
-            String server = pool.getHostAddress();
+            String server = pool.getHost();
             String serverpath = pool.getPath();
             serverpath = serverpath.replace("//", "/");
             Set srs = SR.getAll(conn);
@@ -4689,59 +5037,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                     if (checkSR(sr)) {
                         return sr;
                     }
-                }
-
-            }
-
-            deviceConfig.put("server", server);
-            deviceConfig.put("serverpath", serverpath);
-            Host host = Host.getByUuid(conn, _host.uuid);
-            SR sr = SR.create(conn, host, deviceConfig, new Long(0), pool.getUuid(), pool.getName(), SRType.NFS.toString(), "user", true, new HashMap());
-            sr.scan(conn);
-            return sr;
-
-        } catch (XenAPIException e) {
-            String msg = "Unable to create NFS SR  " + deviceConfig + " due to  " + e.toString();
-            s_logger.warn(msg, e);
-            throw new CloudRuntimeException(msg, e);
-        } catch (Exception e) {
-            String msg = "Unable to create NFS SR  " + deviceConfig + " due to  " + e.getMessage();
-            s_logger.warn(msg);
-            throw new CloudRuntimeException(msg, e);
-        }
-    }
-
-    protected SR getNfsSR(Connection conn, StoragePoolTO pool) {
-        Map deviceConfig = new HashMap();
-
-        String server = pool.getHost();
-        String serverpath = pool.getPath();
-        serverpath = serverpath.replace("//", "/");
-        try {
-            Set srs = SR.getAll(conn);
-            for (SR sr : srs) {
-                if (!SRType.NFS.equals(sr.getType(conn)))
-                    continue;
-
-                Set pbds = sr.getPBDs(conn);
-                if (pbds.isEmpty())
-                    continue;
-
-                PBD pbd = pbds.iterator().next();
-
-                Map dc = pbd.getDeviceConfig(conn);
-
-                if (dc == null)
-                    continue;
-
-                if (dc.get("server") == null)
-                    continue;
-
-                if (dc.get("serverpath") == null)
-                    continue;
-
-                if (server.equals(dc.get("server")) && serverpath.equals(dc.get("serverpath"))) {
-                    return sr;
+                    throw new CloudRuntimeException("SR check failed for storage pool: " + pool.getUuid() + "on host:" + _host.uuid);
                 }
 
             }
@@ -4760,6 +5056,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         }
     }
 
+    @Override
     public Answer execute(DestroyCommand cmd) {
         VolumeTO vol = cmd.getVolume();
 
@@ -4803,6 +5100,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         return new Answer(cmd, true, "Success");
     }
 
+    @Override
     public ShareAnswer execute(final ShareCommand cmd) {
         if (!cmd.isShare()) {
             SR sr = getISOSRbyVmName(cmd.getVmName());
@@ -4822,9 +5120,11 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         return new ShareAnswer(cmd, new HashMap());
     }
 
+    @Override
     public CopyVolumeAnswer execute(final CopyVolumeCommand cmd) {
         String volumeUUID = cmd.getVolumePath();
         StoragePoolVO pool = cmd.getPool();
+        StoragePoolTO poolTO = new StoragePoolTO(pool);
         String secondaryStorageURL = cmd.getSecondaryStorageURL();
 
         URI uri = null;
@@ -4858,7 +5158,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                 srcVolume = getVDIbyUuid(volumeUUID);
 
                 // Copy the volume to secondary storage
-                destVolume = srcVolume.copy(conn, secondaryStorage);
+                destVolume = cloudVDIcopy(srcVolume, secondaryStorage);
             } else {
                 // Mount the volume folder
                 secondaryStorage = createNfsSRbyURI(new URI(secondaryStorageURL + "/volumes/" + volumeFolder), false);
@@ -4877,8 +5177,8 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                 }
 
                 // Copy the volume to the primary storage pool
-                primaryStoragePool = getStorageRepository(conn, pool);
-                destVolume = srcVolume.copy(conn, primaryStoragePool);
+                primaryStoragePool = getStorageRepository(conn, poolTO);
+                destVolume = cloudVDIcopy(srcVolume, primaryStoragePool);
             }
 
             String srUUID;
@@ -5267,7 +5567,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
 
             // Look up the snapshot and copy it to secondary storage
             VDI snapshot = getVDIbyUuid(snapshotUUID);
-            privateTemplate = snapshot.copy(conn, secondaryStorage);
+            privateTemplate = cloudVDIcopy(snapshot, secondaryStorage);
 
             if (userSpecifiedName != null) {
                 privateTemplate.setNameLabel(conn, userSpecifiedName);
@@ -5513,7 +5813,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
                     if (vdi != null) {
                         s_logger.debug("Successfully created VDI on secondary storage SR " + temporarySROnSecondaryStorage.getNameLabel(conn) + " with uuid " + vhdUUID);
                         s_logger.debug("Copying VDI: " + vdi.getLocation(conn) + " from secondary to primary");
-                        VDI vdiOnPrimaryStorage = vdi.copy(conn, primaryStorageSR);
+                        VDI vdiOnPrimaryStorage = cloudVDIcopy(vdi, primaryStorageSR);
                         // vdi.copy introduces the vdi into the database. Don't
                         // need to do a scan on the primary
                         // storage.
@@ -5751,40 +6051,6 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             throw new CloudRuntimeException("Unable to get SR " + pool.getUuid() + " due to " + e.getMessage(), e);
         }
 
-        if (srs.size() > 1) {
-            throw new CloudRuntimeException("More than one storage repository was found for pool with uuid: " + pool.getUuid());
-        }
-
-        if (srs.size() == 1) {
-            SR sr = srs.iterator().next();
-            if (s_logger.isDebugEnabled()) {
-                s_logger.debug("SR retrieved for " + pool.getId() + " is mapped to " + sr.toString());
-            }
-
-            if (checkSR(sr)) {
-                return sr;
-            }
-        }
-
-        if (pool.getType() == StoragePoolType.NetworkFilesystem)
-            return getNfsSR(conn, pool);
-        else if (pool.getType() == StoragePoolType.IscsiLUN)
-            return getIscsiSR(conn, pool);
-        else
-            throw new CloudRuntimeException("The pool type: " + pool.getType().name() + " is not supported.");
-
-    }
-
-    protected SR getStorageRepository(Connection conn, StoragePoolVO pool) {
-        Set srs;
-        try {
-            srs = SR.getByNameLabel(conn, pool.getUuid());
-        } catch (XenAPIException e) {
-            throw new CloudRuntimeException("Unable to get SR " + pool.getUuid() + " due to " + e.toString(), e);
-        } catch (Exception e) {
-            throw new CloudRuntimeException("Unable to get SR " + pool.getUuid() + " due to " + e.getMessage(), e);
-        }
-
         if (srs.size() > 1) {
             throw new CloudRuntimeException("More than one storage repository was found for pool with uuid: " + pool.getUuid());
         } else if (srs.size() == 1) {
@@ -5796,15 +6062,17 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             if (checkSR(sr)) {
                 return sr;
             }
+            throw new CloudRuntimeException("SR check failed for storage pool: " + pool.getUuid() + "on host:" + _host.uuid);
+        } else {
+	
+	        if (pool.getType() == StoragePoolType.NetworkFilesystem)
+	            return getNfsSR(pool);
+	        else if (pool.getType() == StoragePoolType.IscsiLUN)
+	            return getIscsiSR(pool);
+	        else
+	            throw new CloudRuntimeException("The pool type: " + pool.getType().name() + " is not supported.");
         }
 
-        if (pool.getPoolType() == StoragePoolType.NetworkFilesystem)
-            return getNfsSR(pool);
-        else if (pool.getPoolType() == StoragePoolType.IscsiLUN)
-            return getIscsiSR(conn, pool);
-        else
-            throw new CloudRuntimeException("The pool type: " + pool.getPoolType().name() + " is not supported.");
-
     }
 
     protected Answer execute(final CheckConsoleProxyLoadCommand cmd) {
@@ -5877,7 +6145,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             checksum = "";
         }
 
-        String result = callHostPlugin("vmopsSnapshot", "post_create_private_template", "remoteTemplateMountPath", remoteTemplateMountPath, "templateDownloadFolder", templateDownloadFolder,
+        String result = callHostPluginWithTimeOut("vmopsSnapshot", "post_create_private_template", 110*60, "remoteTemplateMountPath", remoteTemplateMountPath, "templateDownloadFolder", templateDownloadFolder,
                 "templateInstallFolder", templateInstallFolder, "templateFilename", templateFilename, "templateName", templateName, "templateDescription", templateDescription,
                 "checksum", checksum, "virtualSize", String.valueOf(virtualSize), "templateId", String.valueOf(templateId));
 
@@ -5915,7 +6183,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
 
         // Each argument is put in a separate line for readability.
         // Using more lines does not harm the environment.
-        String results = callHostPlugin("vmopsSnapshot", "backupSnapshot", "primaryStorageSRUuid", primaryStorageSRUuid, "dcId", dcId.toString(), "accountId", accountId.toString(), "volumeId",
+        String results = callHostPluginWithTimeOut("vmopsSnapshot", "backupSnapshot", 110*60, "primaryStorageSRUuid", primaryStorageSRUuid, "dcId", dcId.toString(), "accountId", accountId.toString(), "volumeId",
                 volumeId.toString(), "secondaryStorageMountPath", secondaryStorageMountPath, "snapshotUuid", snapshotUuid, "prevSnapshotUuid", prevSnapshotUuid, "prevBackupUuid",
                 prevBackupUuid, "isFirstSnapshotOfRootVolume", isFirstSnapshotOfRootVolume.toString(), "isISCSI", isISCSI.toString());
 
@@ -6018,7 +6286,7 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
 
         String failureString = "Could not create volume from " + backedUpSnapshotUuid;
         templatePath = (templatePath == null) ? "" : templatePath;
-        String results = callHostPlugin("vmopsSnapshot", "createVolumeFromSnapshot", "dcId", dcId.toString(), "accountId", accountId.toString(), "volumeId", volumeId.toString(),
+        String results = callHostPluginWithTimeOut("vmopsSnapshot","createVolumeFromSnapshot", 110*60, "dcId", dcId.toString(), "accountId", accountId.toString(), "volumeId", volumeId.toString(),
                 "secondaryStorageMountPath", secondaryStorageMountPath, "backedUpSnapshotUuid", backedUpSnapshotUuid, "templatePath", templatePath, "templateDownloadFolder",
                 templateDownloadFolder, "isISCSI", isISCSI.toString());
 
@@ -6072,6 +6340,15 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
         _agentControl = agentControl;
     }
 
+    @Override
+    public boolean IsRemoteAgent() {
+    	return _isRemoteAgent;
+    }
+    
+    @Override
+    public void setRemoteAgent(boolean remote) {
+    	_isRemoteAgent = remote;
+    }
 
     
     protected Answer execute(PoolEjectCommand cmd) {
@@ -6122,8 +6399,10 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
     // the resource first connects to XenServer. These UUIDs do
     // not change over time.
     protected class XenServerHost {
+        public String systemvmisouuid;
         public String uuid;
         public String ip;
+        public Host host;
         public String publicNetwork;
         public String privateNetwork;
         public String linkLocalNetwork;
@@ -6161,4 +6440,8 @@ public abstract class CitrixResourceBase implements StoragePoolResource, ServerR
             return virtualSize;
         }
     }
+
+	protected String getGuestOsType(String stdType) {
+		return CitrixHelper.getGuestOsType(stdType);
+	}
 }
diff --git a/core/src/com/cloud/hypervisor/xen/resource/XenServerConnectionPool.java b/core/src/com/cloud/hypervisor/xen/resource/XenServerConnectionPool.java
index 84614d8f888..d4ef72350a8 100644
--- a/core/src/com/cloud/hypervisor/xen/resource/XenServerConnectionPool.java
+++ b/core/src/com/cloud/hypervisor/xen/resource/XenServerConnectionPool.java
@@ -22,7 +22,6 @@ import java.net.SocketException;
 import java.net.SocketTimeoutException;
 import java.net.URL;
 import java.util.HashMap;
-import java.util.Iterator;
 import java.util.Map;
 import java.util.Set;
 
@@ -39,105 +38,145 @@ import com.xensource.xenapi.Types;
 import com.xensource.xenapi.Types.XenAPIException;
 
 public class XenServerConnectionPool {
-    private static final Logger s_logger = Logger.getLogger(XenServerConnectionPool.class);
-    
-    protected HashMap _conns = new HashMap();
-    protected HashMap _infos = new HashMap();
-    
+    private static final Logger s_logger = Logger
+            .getLogger(XenServerConnectionPool.class);
+
+    protected HashMap _conns = new HashMap();
+    protected HashMap _infos = new HashMap();
+
     protected int _retries;
     protected int _interval;
-    
+
     protected XenServerConnectionPool() {
-        _retries = 10;
+        _retries = 3;
         _interval = 3;
     }
-    
-    public synchronized void switchMaster(String lipaddr, String poolUuid, Connection conn, Host host, String username, String password, int wait) throws XmlRpcException, XenAPIException {
-        String ipaddr = host.getAddress(conn);
+
+    void forceSleep(long sec) {
+        long firetime = System.currentTimeMillis() + (sec * 1000);
+        long msec = sec * 1000;
+        while (true) {
+            if (msec < 100)
+                break;
+            try {
+                Thread.sleep(msec);
+                return;
+            } catch (InterruptedException e) {
+                msec = firetime - System.currentTimeMillis();
+            }
+        }
+    }
+
+    public synchronized void switchMaster(String slaveIp, String poolUuid,
+            Connection conn, Host host, String username, String password,
+            int wait) throws XmlRpcException, XenAPIException {
+        String masterIp = host.getAddress(conn);
         PoolSyncDB(conn);
-        s_logger.debug("Designating the new master to " + ipaddr);
+        s_logger.debug("Designating the new master to " + masterIp);
         Pool.designateNewMaster(conn, host);
-        XenServerConnection slaveConn = null;
-        XenServerConnection masterConn = null;
+        Connection slaveConn = null;
+        Connection masterConn = null;
         int retry = 30;
         for (int i = 0; i < retry; i++) {
-            try {
-                Thread.sleep(5000);
-            } catch (InterruptedException e) {
-            }
+            forceSleep(5);
             try {
                 if (s_logger.isDebugEnabled()) {
-                    s_logger.debug("Logging on as the slave to " + lipaddr);
+                    s_logger.debug("Logging on as the slave to " + slaveIp);
                 }
                 slaveConn = null;
                 masterConn = null;
                 URL slaveUrl = null;
                 URL masterUrl = null;
                 Session slaveSession = null;
-                
-                slaveUrl = new URL("http://" + lipaddr);
-                slaveConn = new XenServerConnection(slaveUrl, username, password, _retries, _interval, 10);
-                slaveSession = Session.slaveLocalLoginWithPassword(slaveConn, username, password);
+
+                slaveUrl = new URL("http://" + slaveIp);
+                slaveConn = new Connection(slaveUrl, 100);
+                slaveSession = Session.slaveLocalLoginWithPassword(slaveConn,
+                        username, password);
 
                 if (s_logger.isDebugEnabled()) {
-                    s_logger.debug("Slave logon successful. session= " + slaveSession);
+                    s_logger.debug("Slave logon successful. session= "
+                            + slaveSession);
                 }
-                
+
                 Pool.Record pr = getPoolRecord(slaveConn);
                 Host master = pr.master;
                 String ma = master.getAddress(slaveConn);
-                if( !ma.trim().equals(ipaddr.trim()) ) {
+                if (!ma.trim().equals(masterIp.trim())) {
                     continue;
                 }
                 Session.localLogout(slaveConn);
-                
-                masterUrl = new URL("http://" + ipaddr);
-                masterConn = new XenServerConnection(masterUrl, username, password, _retries, _interval, wait);
-                Session.loginWithPassword(masterConn, username,
-                                          password,
-                                          APIVersion.latest().toString());
+                slaveConn = null;
+                s_logger.debug("Logging on as the master to " + masterIp);
+                masterUrl = new URL("http://" + masterIp);
+                masterConn = new Connection(masterUrl, 100);
+                Session.loginWithPassword(masterConn, username, password,
+                        APIVersion.latest().toString());
                 cleanup(poolUuid);
-                Pool.recoverSlaves(masterConn);
+                ensurePoolIntegrity(masterConn, masterIp, username, password,
+                        wait);
                 PoolSyncDB(masterConn);
                 return;
             } catch (Types.HostIsSlave e) {
-                s_logger.debug("HostIsSlaveException: Still waiting for the conversion to the master");
+                s_logger
+                        .debug("HostIsSlaveException: Still waiting for the conversion to the master");
             } catch (XmlRpcException e) {
-                s_logger.debug("XmlRpcException: Still waiting for the conversion to the master " + e.getMessage());
+                s_logger
+                        .debug("XmlRpcException: Still waiting for the conversion to the master "
+                                + e.getMessage());
             } catch (Exception e) {
-                s_logger.debug("Exception: Still waiting for the conversion to the master" + e.getMessage());
+                s_logger
+                        .debug("Exception: Still waiting for the conversion to the master"
+                                + e.getMessage());
             } finally {
-            	if (masterConn != null) {
-            		try {
-            		  Session.logout(masterConn);
-            		} catch (Exception e) {
-            			s_logger.debug("Unable to log out of session: " + e.getMessage());
-            		}
-            		masterConn.dispose();
-            		masterConn = null;
-            	}
+                if (masterConn != null) {
+                    try {
+                        Session.logout(masterConn);
+                    } catch (Exception e) {
+                        s_logger.debug("Unable to log out of session: "
+                                + e.getMessage());
+                    }
+                    masterConn.dispose();
+                    masterConn = null;
+                }
+                if (slaveConn != null) {
+                    try {
+                        Session.localLogout(slaveConn);
+                    } catch (Exception e) {
+                        s_logger.debug("Unable to log out of session: "
+                                + e.getMessage());
+                    }
+                    slaveConn.dispose();
+                    slaveConn = null;
+                }
+
             }
 
         }
-        
-        throw new CloudRuntimeException("Unable to logon to the new master after " + retry + " retries");
+
+        throw new CloudRuntimeException(
+                "Unable to logon to the new master after " + retry + " retries");
     }
-    
+
     protected synchronized void cleanup(String poolUuid) {
+    	if( poolUuid == null ) {
+    		return;
+        }
         ConnectionInfo info = _infos.remove(poolUuid);
         if (info == null) {
-            s_logger.debug("Unable to find any information for pool " + poolUuid);
             return;
         }
-        
+
+        s_logger.debug("Cleaning up session for pool " + poolUuid);
         for (Member member : info.refs.values()) {
+            s_logger.debug("remove connection for host " + member.uuid);
             _conns.remove(member.uuid);
         }
-        
+
         if (info.conn != null) {
             try {
-                s_logger.debug("Logging out of session " + info.conn.getSessionReference());
-                
+                s_logger.debug("Logging out of session "
+                        + info.conn.getSessionReference());
                 Session.logout(info.conn);
             } catch (XenAPIException e) {
                 s_logger.debug("Unable to logout of the session");
@@ -148,58 +187,54 @@ public class XenServerConnectionPool {
         }
         s_logger.debug("Session is cleaned up");
     }
-    
+
     protected synchronized void cleanup(String poolUuid, ConnectionInfo info) {
         ConnectionInfo info2 = _infos.get(poolUuid);
+        if( info2 == null ) {
+        	return;
+        }
+        s_logger.debug("Cleanup for Session " + info.conn.getSessionReference());
         if (info != info2) {
-            s_logger.debug("Session " + info.conn.getSessionReference() + " is already logged out.");
+            s_logger.debug("Session " + info.conn.getSessionReference()
+                    + " is already logged out.");
             return;
         }
-        
+
         cleanup(poolUuid);
     }
-    
+
     public synchronized void disconnect(String uuid, String poolUuid) {
         Connection conn = _conns.remove(uuid);
         if (conn == null) {
             return;
         }
-        
+
         if (s_logger.isDebugEnabled()) {
-            s_logger.debug("Logging out of " + conn.getSessionReference() + " for host " + uuid);
+            s_logger.debug("Logging out of " + conn.getSessionReference()
+                    + " for host " + uuid);
         }
-        
-        conn.dispose();
-        
+
         ConnectionInfo info = _infos.get(poolUuid);
         if (info == null) {
             return;
         }
-        
+
         if (s_logger.isDebugEnabled()) {
-            s_logger.debug("Connection for pool " + poolUuid + " found. session=" + info.conn.getSessionReference());
+            s_logger.debug("Connection for pool " + poolUuid
+                    + " found. session=" + info.conn.getSessionReference());
         }
+
+        Member member = info.refs.remove(uuid);
         
-        info.refs.remove(uuid);
-        if (info.refs.size() == 0) {
-         
-            if (s_logger.isDebugEnabled()) {
-                s_logger.debug("Logging out of the session " + info.conn.getSessionReference());
-            }
-            _infos.remove(poolUuid);
-            try {
-                Session.logout(info.conn);
-                info.conn.dispose();
-            } catch (Exception e) {
-                s_logger.debug("Logout has a problem " + e.getMessage());
-            }
-            info.conn = null;
+        if (info.refs.size() == 0 || ( member != null && member.ipAddr.equals(info.masterIp) )) {
+            cleanup(poolUuid);
         }
     }
-    
+
     public static void logout(Connection conn) {
         try {
-            s_logger.debug("Logging out of the session " + conn.getSessionReference());
+            s_logger.debug("Logging out of the session "
+                    + conn.getSessionReference());
             Session.logout(conn);
         } catch (Exception e) {
             s_logger.debug("Logout has problem " + e.getMessage());
@@ -207,319 +242,483 @@ public class XenServerConnectionPool {
             conn.dispose();
         }
     }
-    
-    public Connection connect(String urlString, String username, String password, int wait) {
-        return connect(null, urlString, username, password, wait);
+
+    public Connection masterConnect(String ip, String username, String password) {
+        Connection slaveConn = null;
+        Connection masterConn = null;
+        try{ 
+            URL slaveUrl = new URL("http://" + ip);
+            slaveConn = new Connection(slaveUrl, 100);
+            Session.slaveLocalLoginWithPassword(slaveConn, username, password);
+
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("Slave logon to " + ip);
+            }
+            String masterIp = null;
+            try {
+                Pool.Record pr = getPoolRecord(slaveConn);
+                Host master = pr.master;
+                masterIp = master.getAddress(slaveConn);
+
+                s_logger.debug("Logging on as the master to " + masterIp);
+                URL masterUrl = new URL("http://" + masterIp);
+                masterConn = new Connection(masterUrl, 100);
+                Session.loginWithPassword(masterConn, username, password,
+                        APIVersion.latest().toString());
+                return masterConn;
+            } catch (Exception e) {
+                s_logger.debug("Failed to log on as master to ");
+                if( masterConn != null ) {
+                    try {
+                        Session.logout(masterConn);
+                    } catch (Exception e1) {
+                    }
+                    masterConn.dispose();
+                    masterConn = null;
+                }
+            }
+        }catch ( Exception e){
+            s_logger.debug("Failed to slave local login to " + ip);
+        } finally {
+            if( slaveConn != null ) {
+                try {
+                    Session.localLogout(slaveConn);
+                } catch (Exception e1) {
+                }
+                slaveConn.dispose();
+                slaveConn = null;
+            }
+            
+        }
+        return null;
     }
     
+    public Connection slaveConnect(String ip, String username, String password) {
+        Connection conn = null;
+        try{ 
+            URL url = new URL("http://" + ip);
+            conn = new Connection(url, 100);
+            Session.slaveLocalLoginWithPassword(conn, username, password);
+            return conn;
+        }catch ( Exception e){
+            s_logger.debug("Failed to slave local login to " + ip);
+        } 
+        return null;
+    }
+
     protected ConnectionInfo getConnectionInfo(String poolUuid) {
-        synchronized(_infos) {
+        synchronized (_infos) {
             return _infos.get(poolUuid);
         }
     }
-    
+
     protected XenServerConnection getConnection(String hostUuid) {
-        synchronized(_conns) {
+        synchronized (_conns) {
             return _conns.get(hostUuid);
         }
     }
-    
+
     static void PoolSyncDB(Connection conn) {
-        try{
+        try {
             Set hosts = Host.getAll(conn);
-            for(Host host : hosts) {
+            for (Host host : hosts) {
                 try {
                     host.enable(conn);
                 } catch (Exception e) {
                 }
             }
         } catch (Exception e) {
-            s_logger.debug("Enbale host failed due to " + e.getMessage() + e.toString());
+            s_logger.debug("Enbale host failed due to " + e.getMessage()
+                    + e.toString());
         }
-        try{      
+        try {
             Pool.syncDatabase(conn);
         } catch (Exception e) {
-            s_logger.debug("Sync Database failed due to " + e.getMessage() + e.toString());
+            s_logger.debug("Sync Database failed due to " + e.getMessage()
+                    + e.toString());
         }
-
-
     }
-    
-    protected synchronized URL ensurePoolIntegrity(Connection conn, Host master, Pool.Record poolr, String ipAddress, String username, String password, int wait) throws XenAPIException, XmlRpcException {
-        if (!ipAddress.equals(master.getAddress(conn))) {
-            return null;  // Doesn't think it is the master anyways.
-        }
-        
-        String poolUuid = poolr.uuid;
-        ConnectionInfo info = _infos.get(poolUuid);
-        if (info != null) {
-            Connection poolConn = info.conn;
-            if (poolConn != null) {
-                Pool.recoverSlaves(poolConn);
-                PoolSyncDB(poolConn);
-                return info.masterUrl;
+
+    void PoolEmergencyTransitionToMaster(String slaveIp, String username, String password) {
+        Connection slaveConn = null;
+        Connection c = null;
+        try{
+            s_logger.debug("Trying to transition master to " + slaveIp);
+            URL slaveUrl = new URL("http://" + slaveIp);
+            slaveConn = new Connection(slaveUrl, 100);
+            Session.slaveLocalLoginWithPassword(slaveConn, username, password);
+            Pool.emergencyTransitionToMaster(slaveConn);
+            try {
+                Session.localLogout(slaveConn);
+                slaveConn = null;
+            } catch (Exception e) {
+            }
+            // restart xapi in 10 sec
+            forceSleep(10);
+            // check if the master of this host is set correctly.
+            c = new Connection(slaveUrl, 100);
+            for (int i = 0; i < 30; i++) {
+                try {
+                    Session.loginWithPassword(c, username, password, APIVersion.latest().toString());
+                    s_logger.debug("Succeeded to transition master to " + slaveIp);
+                    return;
+                } catch (Types.HostIsSlave e) {
+                    s_logger.debug("HostIsSlave: Still waiting for the conversion to the master " + slaveIp);
+                } catch (Exception e) {
+                    s_logger.debug("Exception: Still waiting for the conversion to the master");
+                }
+                forceSleep(2);
+            }
+            throw new RuntimeException("EmergencyTransitionToMaster failed after retry 30 times");
+        } catch (Exception e) {
+            throw new RuntimeException("EmergencyTransitionToMaster failed due to " + e.getMessage());
+        } finally {
+            if(slaveConn != null) {
+                try {
+                    Session.localLogout(slaveConn);
+                } catch (Exception e) {
+                }
+            }
+            if(c != null) {
+                try {
+                    Session.logout(c);
+                    c.dispose();
+                } catch (Exception e) {
+                }
             }
         }
+        
+    }
 
+    void PoolEmergencyResetMaster(String slaveIp, String masterIp,
+            String username, String password) {
+        Connection slaveConn = null;
+        try {
+            s_logger.debug("Trying to reset master of slave " + slaveIp
+                    + " to " + masterIp);
+            URL slaveUrl = new URL("http://" + slaveIp);
+            slaveConn = new Connection(slaveUrl, 10);
+            Session.slaveLocalLoginWithPassword(slaveConn, username, password);
+            Pool.emergencyResetMaster(slaveConn, masterIp);
+            if (slaveConn != null) {
+                try {
+                    Session.localLogout(slaveConn);
+                } catch (Exception e) {
+                }
+            }
+            forceSleep(10);
+            for (int i = 0; i < 30; i++) {
+                try {
+                    Session.slaveLocalLoginWithPassword(slaveConn, username, password);
+                    Pool.Record pr = getPoolRecord(slaveConn);
+                    String mIp = pr.master.getAddress(slaveConn);
+                    if (mIp.trim().equals(masterIp.trim())) {
+                        return;
+                    }
+                } catch (Exception e) {
+                }
+                if (slaveConn != null) {
+                    try {
+                        Session.localLogout(slaveConn);
+                    } catch (Exception e) {
+                    }
+                }
+                // wait 2 second
+                forceSleep(2);
+            }
+        } catch (Exception e) {
+
+        } finally {
+            if (slaveConn != null) {
+                try {
+                    Session.localLogout(slaveConn);
+                    slaveConn = null;
+                } catch (Exception e) {
+                }
+            }
+        }
+    }
+
+    protected synchronized void ensurePoolIntegrity(Connection conn,
+            String masterIp, String username, String password, int wait)
+            throws XenAPIException, XmlRpcException {
+        try {
+            // try recoverSlave first
+            Set slaves = Pool.recoverSlaves(conn);
+            // wait 10 second
+            forceSleep(10);
+            for(Host slave : slaves ) {
+                for (int i = 0; i < 30; i++) {
+                    Connection slaveConn = null;
+                    try {
+                        
+                        String slaveIp = slave.getAddress(conn);
+                        s_logger.debug("Logging on as the slave to " + slaveIp);
+                        URL slaveUrl = new URL("http://" + slaveIp);
+                        slaveConn = new Connection(slaveUrl, 10);
+                        Session.slaveLocalLoginWithPassword(slaveConn, username, password);
+                        Pool.Record pr = getPoolRecord(slaveConn);
+                        String mIp = pr.master.getAddress(slaveConn);
+                        if (mIp.trim().equals(masterIp.trim())) {
+                            break;
+                        }
+                    } catch (Exception e) {
+                        try {
+                            Session.localLogout(slaveConn);
+                        } catch (Exception e1) {
+                        }
+                        slaveConn.dispose();
+                    }
+                    // wait 2 second
+                    forceSleep(2);
+                }
+                
+            }
+        } catch (Exception e) {
+        }
+
+        // then try emergency reset master
         Set slaves = Host.getAll(conn);
-        HashMap count = new HashMap(slaves.size());
         for (Host slave : slaves) {
-            String slaveAddress = slave.getAddress(conn);
+            String slaveIp = slave.getAddress(conn);
             Connection slaveConn = null;
             try {
-                slaveConn = new Connection(new URL("http://" + slaveAddress), wait);
+                s_logger.debug("Logging on as the slave to " + slaveIp);
+                URL slaveUrl = new URL("http://" + slaveIp);
+                slaveConn = new Connection(slaveUrl, 10);
+                Session.slaveLocalLoginWithPassword(slaveConn, username,
+                        password);
+                Pool.Record slavePoolr = getPoolRecord(slaveConn);
+                String ip = slavePoolr.master.getAddress(slaveConn);
+                if (!masterIp.trim().equals(ip.trim())) {
+                    PoolEmergencyResetMaster(slaveIp, masterIp, username,
+                            password);
+                }
+            } catch (MalformedURLException e) {
+                throw new CloudRuntimeException("Bad URL" + slaveIp, e);
+            } catch (Exception e) {
+                s_logger.debug("Unable to login to slave " + slaveIp
+                        + " error " + e.getMessage());
+            } finally {
                 if (slaveConn != null) {
-                    Session slaveSession = Session.slaveLocalLoginWithPassword(slaveConn, username, password);
-                    Pool.Record slavePoolr = getPoolRecord(slaveConn);
-                    String possibleMaster = slavePoolr.master.getAddress(slaveConn);
-                    Integer c = count.get(possibleMaster);
-                    if (c == null) {
-                        c = 1;
-                    } else {
-                        c++;
-                    }
-                    count.put(possibleMaster, c);
                     try {
-                        slaveSession.logout(slaveConn);
+                        Session.localLogout(slaveConn);
                     } catch (Exception e) {
-                        s_logger.debug("client session logout: " + e.getMessage());
                     }
                     slaveConn.dispose();
                 }
-            } catch (MalformedURLException e) {
-                throw new CloudRuntimeException("Bad URL" + slaveAddress, e);
-            } catch (Exception e) {
-                s_logger.debug("Unable to login to slave " + slaveAddress + " error " + e.getMessage());
-            } finally {
-                if (slaveConn != null) {
-                    slaveConn.dispose();
-                }
             }
         }
-        
-        Iterator> it = count.entrySet().iterator();
-       
-        Map.Entry newMaster = it.next();
-        while (it.hasNext()) {
-            Map.Entry entry = it.next();
-            
-            if (newMaster.getValue() < entry.getValue()) {
-                newMaster = entry;
-            }
-        }
-        
-        String newMasterAddress = newMaster.getKey();
-        if (count.size() > 1 && !ipAddress.equals(newMasterAddress)) {
-            s_logger.debug("Asking the correct master to recover the slaves: " + newMasterAddress);
-            
-            URL newMasterUrl = null;
-            try {
-                newMasterUrl = new URL("http://" + newMasterAddress);
-            } catch (MalformedURLException e) {
-                throw new CloudRuntimeException("Unable to get url from " + newMasterAddress, e);
-            }
-            
-            Connection newMasterConn = new Connection(newMasterUrl, wait);
-            try {
-                Session.loginWithPassword(newMasterConn, username, password, APIVersion.latest().toString());
-                Pool.recoverSlaves(newMasterConn);
-                PoolSyncDB(newMasterConn);
-            } catch (Exception e) {
-                throw new CloudRuntimeException("Unable to login to the real master at " + newMaster.getKey());
-            } finally {
-                try {
-                    Session.logout(newMasterConn);
-                } catch (Exception e) {
-                    s_logger.debug("Unable to logout of the session: " + e.getMessage());
-                }
-                newMasterConn.dispose();
-            }
-            
-            return newMasterUrl;
-        }
-        
-        return null;
     }
-    
-    public synchronized Connection connect(String hostUuid, String ipAddress, String username, String password, int wait) {
+
+    public synchronized Connection connect(String hostUuid, String poolUuid, String ipAddress,
+            String username, String password, int wait) {
+
         XenServerConnection masterConn = null;
-        if (hostUuid != null) { // Let's see if it is an existing connection.
-            masterConn = _conns.get(hostUuid);
-            if (masterConn != null) {
-                return masterConn;
-            }
-        }
-        
-        XenServerConnection slaveConn = null;
+        Connection slaveConn = null;
         URL slaveUrl = null;
         URL masterUrl = null;
-        Session slaveSession = null;
-        Session masterSession = null;
-        
+        String masterIp = null;
+        ConnectionInfo info = null;
+        if(hostUuid == null || poolUuid == null || ipAddress == null || username == null || password == null ) {
+            String msg = "Connect some parameter are null hostUuid:" + hostUuid + " ,poolUuid:"
+                + poolUuid + " ,ipAddress:" + ipAddress;
+            s_logger.debug(msg);
+            throw new CloudRuntimeException(msg);
+        }
+        // Let's see if it is an existing connection.
+        masterConn = _conns.get(hostUuid);
+        if (masterConn != null){
+            try{
+                Host.getByUuid(masterConn, hostUuid);
+                return masterConn;
+            } catch (Exception e) { 
+                if (s_logger.isDebugEnabled()) {
+                    s_logger.debug("Master Session " + masterConn.getSessionReference() + " is broken due to " + e.getMessage());
+                }
+                cleanup(masterConn.get_poolUuid());
+                masterConn = null;
+            }
+        }
         if (s_logger.isDebugEnabled()) {
             s_logger.debug("Creating connection to " + ipAddress);
         }
-        
-        // Well, it's not already in the existing connection list.
-        // Let's login and see what this connection should be.
-        // You might think this is slow.  Why not cache the pool uuid
-        // you say?  Well, this doesn't happen very often.
+
         try {
             if (s_logger.isDebugEnabled()) {
                 s_logger.debug("Logging on as the slave to " + ipAddress);
             }
             slaveUrl = new URL("http://" + ipAddress);
-            slaveConn = new XenServerConnection(slaveUrl, username, password, _retries, _interval, 10);
-            slaveSession = Session.slaveLocalLoginWithPassword(slaveConn, username, password);
+            slaveConn = new Connection(slaveUrl, 100);
+            Session.slaveLocalLoginWithPassword(slaveConn,
+                    username, password);
 
             if (s_logger.isDebugEnabled()) {
-                s_logger.debug("Slave logon successful. session= " + slaveSession);
+                s_logger.debug("Slave logon successful to " + ipAddress);
             }
-            
-            try {
-                Pool.Record pr = getPoolRecord(slaveConn);
-                Host master = pr.master;
-                String ma = master.getAddress(slaveConn);
-                masterUrl = new URL("http://" + ma);
-                s_logger.debug("Logging on as the master to " + ma);
-                masterConn = new XenServerConnection(masterUrl, username, password, _retries, _interval, wait);
-                masterSession = Session.loginWithPassword(masterConn, username, password, APIVersion.latest().toString());
-         
-                URL realMasterUrl = ensurePoolIntegrity(masterConn, master, pr, ipAddress, username, password, wait);
-                if (realMasterUrl != null) {
-                    s_logger.debug("The real master url is at " + realMasterUrl);
-                    masterUrl = realMasterUrl;
-                    masterConn.dispose();
-                    masterConn = new XenServerConnection(masterUrl, username, password, _retries, _interval, wait);
-                    masterSession = Session.loginWithPassword(masterConn, username, password, APIVersion.latest().toString());
-                    
-                }
-            } catch (XmlRpcException e) {
-                Throwable c = e.getCause();
-                if (c == null || (!(c instanceof SocketException) && !(c instanceof SocketTimeoutException))) {
-                    s_logger.warn("Unable to connect to " + masterUrl, e);
-                    throw e;
-                }
 
-                if (s_logger.isDebugEnabled()) {
-                    s_logger.debug("Unable to connect to the " + masterUrl + ". Attempting switch to master");
-                }
-                Pool.emergencyTransitionToMaster(slaveConn);
-                Pool.recoverSlaves(slaveConn);
-                
-                s_logger.info("Successfully converted to the master: " + ipAddress);
-                
-                masterUrl = slaveUrl;
-                masterConn = new XenServerConnection(masterUrl, username, password, _retries, _interval, wait);
-                masterSession = Session.loginWithPassword(masterConn, username, password, APIVersion.latest().toString());
-            }
-                
-            if (slaveSession != null) {
-                s_logger.debug("Login to the master is successful.  Signing out of the slave connection: " + slaveSession);
-                try {
-                    Session.localLogout(slaveConn);
-                } catch (Exception e) {
-                    s_logger.debug("Unable to logout of slave session " + slaveSession);
-                }
-                slaveConn.dispose();
-            }
-            
-            if (s_logger.isDebugEnabled()) {
-                s_logger.debug("Successfully logged on to the master.  Session=" + masterConn.getSessionReference());
-            }
-            if (hostUuid == null) {
-                s_logger.debug("Returning now.  Client is responsible for logging out of " + masterConn.getSessionReference());
-                return masterConn;
-            }
-            
-            if (s_logger.isDebugEnabled()) {
-                s_logger.debug("Logon is successfully.  Let's see if we have other hosts logged onto the same master at " + masterUrl);
-            }
-            
-            Pool.Record poolr = getPoolRecord(masterConn);
-            String poolUuid = poolr.uuid;
-            
-            ConnectionInfo info = null;
             info = _infos.get(poolUuid);
-            if (info != null && info.conn != null) {
-                if (s_logger.isDebugEnabled()) {
-                    s_logger.debug("The pool already has a master connection.  Session=" + info.conn.getSessionReference());
-                }
+            boolean create_new_session = true;
+            if (info != null) {
                 try {
-                    s_logger.debug("Logging out of our own session: " + masterConn.getSessionReference());
-                    Session.logout(masterConn);
-                    masterConn.dispose();
+                    masterConn = info.conn;
+                    Host.getByUuid(masterConn, hostUuid);
+                    ensurePoolIntegrity(masterConn, info.masterIp, username,
+                            password, wait);
+                    masterIp = info.masterIp;
+                    create_new_session = false;
                 } catch (Exception e) {
-                    s_logger.debug("Caught Exception while logging on but pushing on...." + e.getMessage());
+                    if (s_logger.isDebugEnabled()) {
+                        s_logger.debug("Unable to connect to master " + info.masterIp);
+                    }
+                    
+                    cleanup(poolUuid);
+                    masterConn = null;
+                    masterIp = null;
                 }
-                masterConn = new XenServerConnection(info.conn);
             } else {
+            	try {
+                    Pool.Record pr = getPoolRecord(slaveConn);
+                    masterIp = pr.master.getAddress(slaveConn);
+                    masterUrl = new URL("http://" + masterIp);;
+                    masterConn = new XenServerConnection(masterUrl, username,
+                            password, _retries, _interval, wait);
+                    Session.loginWithPassword(masterConn, username, password,
+                            APIVersion.latest().toString());
+                    create_new_session = false;
+            		
+            	} catch (Exception e) {
+                    cleanup(poolUuid);
+                    masterConn = null;
+                    masterIp = null;
+            	}
+            }
+            if (create_new_session) {
+                try{ 
+                    cleanup(poolUuid);
+                    s_logger.info("Attempting switch master to " + ipAddress);
+
+                    PoolEmergencyTransitionToMaster(ipAddress, username, password);
+    
+                    s_logger.info("Successfully converted to master: " + ipAddress);
+    
+                    s_logger.info("Loginning on as master to " + ipAddress);
+                    masterUrl = slaveUrl;
+                    masterConn = new XenServerConnection(masterUrl, username,
+                            password, _retries, _interval, wait);
+                    Session.loginWithPassword(masterConn, username, password,
+                            APIVersion.latest().toString());
+                    s_logger.info("Logined on as master to " + ipAddress);
+                    masterIp = ipAddress;
+    
+                    ensurePoolIntegrity(masterConn, ipAddress, username, password,
+                            wait);
+                } catch (Exception e) {
+                    if (s_logger.isDebugEnabled()) {
+                        s_logger.debug("failed to switch master to Unable to " + ipAddress + " due to " + e.getMessage());
+                    }
+                    cleanup(poolUuid);
+                    masterConn = null;
+                    masterIp = null;
+                }
+            }
+
+            if( masterConn == null ) {
+                throw new CloudRuntimeException(" Can not create connection to pool " + poolUuid);
+            }
+            info = _infos.get(poolUuid);
+
+            if ( info == null ) {
                 if (s_logger.isDebugEnabled()) {
-                    s_logger.debug("This is the very first connection");
+                    s_logger.debug("Create info on master :" + masterIp);
                 }
                 info = new ConnectionInfo();
                 info.conn = masterConn;
-                info.masterUrl = masterUrl;
+                info.masterIp = masterIp;
                 info.refs = new HashMap();
-                _infos.put(poolUuid, info);
                 masterConn.setInfo(poolUuid, info);
+                _infos.put(poolUuid, info);
                 if (s_logger.isDebugEnabled()) {
-                    s_logger.debug("Pool " + poolUuid + " is matched with session " + info.conn.getSessionReference());
+                    s_logger.debug("Pool " + poolUuid
+                            + " is matched with session "
+                            + info.conn.getSessionReference());
                 }
             }
-            
-            s_logger.debug("Added a reference for host " + hostUuid + " to session " + masterConn.getSessionReference() + " in pool " +  poolUuid);
-            info.refs.put(hostUuid, new Member(slaveUrl, hostUuid, username, password));
+            masterConn = new XenServerConnection(info.conn);
+
+            s_logger.debug("Added a reference for host " + hostUuid
+                    + " to session " + masterConn.getSessionReference()
+                    + " in pool " + poolUuid);
+            info.refs.put(hostUuid, new Member(ipAddress, hostUuid, username,
+                    password));
             _conns.put(hostUuid, masterConn);
-            
-            s_logger.info("Connection made to " + ipAddress + " for host " + hostUuid + ".  Pool Uuid is " + poolUuid);
-            
+
+            s_logger.info("Connection made to " + ipAddress + " for host "
+                    + hostUuid + ".  Pool Uuid is " + poolUuid);
+
             return masterConn;
         } catch (XenAPIException e) {
-            s_logger.warn("Unable to make a connection to the server " + ipAddress, e);
-            throw new CloudRuntimeException("Unable to make a connection to the server " + ipAddress, e);
+            s_logger.warn("Unable to make a connection to the server "
+                    + ipAddress);
+            throw new CloudRuntimeException(
+                    "Unable to make a connection to the server " + ipAddress);
         } catch (XmlRpcException e) {
-            s_logger.warn("Unable to make a connection to the server " + ipAddress, e);
-            throw new CloudRuntimeException("Unable to make a connection to the server " + ipAddress, e);
+            s_logger.warn("Unable to make a connection to the server "
+                    + ipAddress, e);
+            throw new CloudRuntimeException(
+                    "Unable to make a connection to the server " + ipAddress);
         } catch (MalformedURLException e) {
-            throw new CloudRuntimeException("How can we get a malformed exception for this " + ipAddress, e);
+            throw new CloudRuntimeException(
+                    "How can we get a malformed exception for this "
+                            + ipAddress);
+        } finally {
+            if (slaveConn != null) {
+                try {
+                    Session.localLogout(slaveConn);
+                } catch (Exception e) {
+                }
+                slaveConn.dispose();
+                slaveConn = null;
+            }
         }
     }
-    
-    
-    
-    protected Pool.Record getPoolRecord(Connection conn) throws XmlRpcException, XenAPIException {
+
+    protected Pool.Record getPoolRecord(Connection conn)
+            throws XmlRpcException, XenAPIException {
         Map pools = Pool.getAllRecords(conn);
-        assert pools.size() == 1 : "Pool size is not one....hmmm....wth? " + pools.size();
-        
+        assert pools.size() == 1 : "Pool size is not one....hmmm....wth? "
+                + pools.size();
+
         return pools.values().iterator().next();
     }
 
     private static final XenServerConnectionPool s_instance = new XenServerConnectionPool();
+
     public static XenServerConnectionPool getInstance() {
         return s_instance;
     }
-    
+
     protected class ConnectionInfo {
-        public URL masterUrl;
+        public String masterIp;
         public XenServerConnection conn;
         public HashMap refs = new HashMap();
     }
-    
+
     protected class Member {
-        public URL url;
+        public String ipAddr;
         public String uuid;
         public String username;
         public String password;
-        
-        public Member(URL url, String uuid, String username, String password) {
-            this.url = url;
+
+
+        public Member(String ipAddr, String uuid, String username, String password) {
+            this.ipAddr = ipAddr;
             this.uuid = uuid;
             this.username = username;
             this.password = password;
         }
     }
-    
+
     public class XenServerConnection extends Connection {
         long _interval;
         int _retries;
@@ -528,16 +727,17 @@ public class XenServerConnectionPool {
         URL _url;
         ConnectionInfo _info;
         String _poolUuid;
-        
-        public XenServerConnection(URL url, String username, String password, int retries, int interval, int wait) {
+
+        public XenServerConnection(URL url, String username, String password,
+                int retries, int interval, int wait) {
             super(url, wait);
             _url = url;
             _retries = retries;
             _username = username;
             _password = password;
-            _interval = (long)interval * 1000;
+            _interval = (long) interval * 1000;
         }
-
+        
         public XenServerConnection(XenServerConnection that) {
             super(that._url, that.getSessionReference(), that._wait);
             this._url = that._url;
@@ -547,28 +747,40 @@ public class XenServerConnectionPool {
             this._interval = that._interval;
             this._info = that._info;
             this._poolUuid = that._poolUuid;
-            
+
         }
-        
+
+
         public void setInfo(String poolUuid, ConnectionInfo info) {
             this._poolUuid = poolUuid;
             this._info = info;
         }
-        
+
         public int getWaitTimeout() {
             return _wait;
         }
         
+        public String get_poolUuid() {
+            return _poolUuid;
+        }
+
         @Override
-        protected Map dispatch(String method_call, Object[] method_params) throws XmlRpcException, XenAPIException {
-            if (method_call.equals("session.login_with_password") || method_call.equals("session.slave_local_login_with_password") || method_call.equals("session.logout")) {
+        protected Map dispatch(String method_call, Object[] method_params)  throws XmlRpcException, XenAPIException {
+            if (method_call.equals("session.local_logout") 
+                    || method_call.equals("session.slave_local_login_with_password") 
+                    || method_call.equals("session.logout")) {
+                return super.dispatch(method_call, method_params);
+            }
+            
+            if (method_call.equals("session.login_with_password")) {
                 int retries = 0;
                 while (retries++ < _retries) {
                     try {
                         return super.dispatch(method_call, method_params);
                     } catch (XmlRpcException e) {
                         Throwable cause = e.getCause();
-                        if (cause == null || !(cause instanceof SocketException)) {
+                        if (cause == null
+                                || !(cause instanceof SocketException)) {
                             throw e;
                         }
                         if (retries >= _retries) {
@@ -578,60 +790,70 @@ public class XenServerConnectionPool {
                     }
                     try {
                         Thread.sleep(_interval);
-                    } catch(InterruptedException e) {
-                        s_logger.debug("Man....I was just getting comfortable there....who woke me up?");
+                    } catch (InterruptedException e) {
+                        s_logger
+                                .debug("Man....I was just getting comfortable there....who woke me up?");
                     }
                 }
-            }
-            
-            int retries = 0;
-            while (retries++ < _retries) {
-                try {
-                    return super.dispatch(method_call, method_params);
-                } catch (Types.SessionInvalid e) {
-                    if (retries >= _retries) {
+            } else {
+                int retries = 0;
+                while (retries++ < _retries) {
+                    try {
+                        return super.dispatch(method_call, method_params);
+                    } catch (Types.SessionInvalid e) {
+                        s_logger.debug("Session is invalid for method: " + method_call + " due to " + e.getMessage() + ".  Reconnecting...retry="
+                                + retries);
+                        if (retries >= _retries) {
+                            if (_poolUuid != null) {
+                                cleanup(_poolUuid, _info);
+                            }
+                            throw e;
+                        }
+                        Session.loginWithPassword(this, _username,
+                                _password, APIVersion.latest().toString());
+                        method_params[0] = getSessionReference();
+                    } catch (XmlRpcException e) {
+                        s_logger.debug("XmlRpcException for method: " + method_call + " due to " + e.getMessage() + ".  Reconnecting...retry="
+                                + retries);
+                        if (retries >= _retries) {
+                            if (_poolUuid != null) {
+                                cleanup(_poolUuid, _info);
+                            }
+                            throw e;
+                        }
+                        Throwable cause = e.getCause();
+                        if (cause == null || !(cause instanceof SocketException)) {
+                            if (_poolUuid != null) {
+                                cleanup(_poolUuid, _info);
+                            }
+                            throw e;
+                        }
+                    } catch (Types.HostIsSlave e) {
+                        s_logger.debug("HostIsSlave Exception for method: " + method_call + " due to " + e.getMessage() + ".  Reconnecting...retry="
+                                + retries);
                         if (_poolUuid != null) {
                             cleanup(_poolUuid, _info);
                         }
                         throw e;
                     }
-                    s_logger.debug("Session is invalid.  Reconnecting...retry=" + retries);
-                } catch (XmlRpcException e) {
-                    if (retries >= _retries) {
-                        if (_poolUuid != null) {
-                            cleanup(_poolUuid, _info);
-                        }
-                        throw e;
+    
+                    try {
+                        Thread.sleep(_interval);
+                    } catch (InterruptedException e) {
+                        s_logger.info("Who woke me from my slumber?");
                     }
-                    Throwable cause = e.getCause();
-                    if (cause == null || !(cause instanceof SocketException)) {
-                        if (_poolUuid != null) {
-                            cleanup(_poolUuid, _info);
-                        }
-                        throw e;
-                    }
-                    s_logger.debug("Connection couldn't be made. Reconnecting....retry=" + retries);
-                } catch (Types.HostIsSlave e) {
-                    if (_poolUuid != null) {
-                        cleanup(_poolUuid, _info);
-                    }
-                    throw e;
-                }
-                
-                try {
-                    Thread.sleep(_interval);
-                } catch (InterruptedException e) {
-                    s_logger.info("Who woke me from my slumber?");
-                }
+    
 
-                Session session = Session.loginWithPassword(this, _username, _password, APIVersion.latest().toString());
-                method_params[0] = getSessionReference();
+                }
+                assert false : "We should never get here";
+                if (_poolUuid != null) {
+                    cleanup(_poolUuid, _info);
+                }
             }
-            assert false : "We should never get here";
-            if (_poolUuid != null) {
-                cleanup(_poolUuid, _info);
-            }
-            throw new CloudRuntimeException("After " + _retries + " retries, we cannot contact the host ");
+            throw new CloudRuntimeException("After " + _retries
+                    + " retries, we cannot contact the host ");
         }
+
+
     }
 }
diff --git a/core/src/com/cloud/hypervisor/xen/resource/XenServerResource.java b/core/src/com/cloud/hypervisor/xen/resource/XenServerResource.java
new file mode 100644
index 00000000000..713b3e6c3ae
--- /dev/null
+++ b/core/src/com/cloud/hypervisor/xen/resource/XenServerResource.java
@@ -0,0 +1,55 @@
+/**
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.hypervisor.xen.resource;
+
+
+import javax.ejb.Local;
+import org.apache.log4j.Logger;
+import org.apache.xmlrpc.XmlRpcException;
+
+import com.cloud.hypervisor.xen.resource.CitrixResourceBase;
+import com.xensource.xenapi.VM;
+import com.cloud.resource.ServerResource;
+import com.xensource.xenapi.Connection;
+import com.xensource.xenapi.Types.XenAPIException;
+
+
+@Local(value=ServerResource.class)
+public class XenServerResource extends CitrixResourceBase {
+    private static final Logger s_logger = Logger.getLogger(XenServerResource.class);
+    
+   
+    public XenServerResource() {
+        super();
+    }
+    
+    @Override
+    protected String getGuestOsType(String stdType) {
+        return stdType;
+    }
+
+    @Override
+    protected void setMemory(Connection conn, VM vm, long memsize) throws XmlRpcException, XenAPIException {
+        vm.setMemoryLimits(conn, memsize, memsize, memsize, memsize);
+    }   
+ 
+    @Override
+    protected String getPatchPath() {
+        return "scripts/vm/hypervisor/xenserver/xenserver56";
+    }
+
+}
diff --git a/core/src/com/cloud/maid/dao/StackMaidDaoImpl.java b/core/src/com/cloud/maid/dao/StackMaidDaoImpl.java
index 646b9e99c58..e7d8d6936f6 100644
--- a/core/src/com/cloud/maid/dao/StackMaidDaoImpl.java
+++ b/core/src/com/cloud/maid/dao/StackMaidDaoImpl.java
@@ -58,9 +58,9 @@ public class StackMaidDaoImpl extends GenericDaoBase implemen
         sc.setParameters("threadId", Thread.currentThread().getId());
         
 		Filter filter = new Filter(StackMaidVO.class, "seq", false, 0L, (long)1);
-		List l = listBy(sc, filter);
+		List l = listIncludingRemovedBy(sc, filter);
 		if(l != null && l.size() > 0) {
-			delete(l.get(0).getId());
+			expunge(l.get(0).getId());
 			return l.get(0);
 		}
 		
@@ -72,7 +72,7 @@ public class StackMaidDaoImpl extends GenericDaoBase implemen
         SearchCriteria sc = clearSearch.create();
         sc.setParameters("msid", msid);
         
-        delete(sc);
+        expunge(sc);
 	}
     
     @DB
diff --git a/core/src/com/cloud/network/SecurityGroupVO.java b/core/src/com/cloud/network/SecurityGroupVO.java
index 8ea991e01a9..691963775da 100644
--- a/core/src/com/cloud/network/SecurityGroupVO.java
+++ b/core/src/com/cloud/network/SecurityGroupVO.java
@@ -18,24 +18,27 @@
 
 package com.cloud.network;
 
-import javax.persistence.Column;
-import javax.persistence.Entity;
-import javax.persistence.GeneratedValue;
-import javax.persistence.GenerationType;
-import javax.persistence.Id;
-import javax.persistence.PrimaryKeyJoinColumn;
-import javax.persistence.SecondaryTable;
-import javax.persistence.Table;
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.PrimaryKeyJoinColumn;
+import javax.persistence.SecondaryTable;
+import javax.persistence.Table;
+
+import com.cloud.domain.PartOf;
+import com.cloud.user.OwnedBy;
 
 @Entity
 @Table(name=("security_group"))
 @SecondaryTable(name="account",
         pkJoinColumns={@PrimaryKeyJoinColumn(name="account_id", referencedColumnName="id")})
-public class SecurityGroupVO {
+public class SecurityGroupVO implements PartOf, OwnedBy {
     @Id
     @GeneratedValue(strategy=GenerationType.IDENTITY)
     @Column(name="id")
-    private Long id;
+    private long id;
 
     @Column(name="name")
     private String name;
@@ -44,24 +47,24 @@ public class SecurityGroupVO {
     private String description;
 
     @Column(name="domain_id")
-    private Long domainId;
+    private long domainId;
 
     @Column(name="account_id")
-    private Long accountId;
+    private long accountId;
     
     @Column(name="account_name", table="account", insertable=false, updatable=false)
     private String accountName = null;
 
     public SecurityGroupVO() {}
 
-    public SecurityGroupVO(String name, String description, Long domainId, Long accountId) {
+    public SecurityGroupVO(String name, String description, long domainId, long accountId) {
         this.name = name;
         this.description = description;
         this.domainId = domainId;
         this.accountId = accountId;
     }
 
-    public Long getId() {
+    public long getId() {
         return id;
     }
 
@@ -73,11 +76,11 @@ public class SecurityGroupVO {
         return description;
     }
 
-    public Long getDomainId() {
+    public long getDomainId() {
         return domainId;
     }
 
-    public Long getAccountId() {
+    public long getAccountId() {
         return accountId;
     }
     
diff --git a/core/src/com/cloud/network/dao/FirewallRulesDaoImpl.java b/core/src/com/cloud/network/dao/FirewallRulesDaoImpl.java
index e714f351d22..55bce08b553 100644
--- a/core/src/com/cloud/network/dao/FirewallRulesDaoImpl.java
+++ b/core/src/com/cloud/network/dao/FirewallRulesDaoImpl.java
@@ -143,7 +143,7 @@ public class FirewallRulesDaoImpl extends GenericDaoBase i
         SearchCriteria sc = FWByIPAndForwardingSearch.create();
         sc.setParameters("publicIpAddress", publicIPAddress);
         sc.setParameters("forwarding", forwarding);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -199,14 +199,14 @@ public class FirewallRulesDaoImpl extends GenericDaoBase i
     public List listIPForwarding(String publicIPAddress) {
         SearchCriteria sc = FWByIPSearch.create();
         sc.setParameters("publicIpAddress", publicIPAddress);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
 	@Override
 	public List listIPForwardingForUpdate(String publicIPAddress) {
 		SearchCriteria sc = FWByIPSearch.create();
         sc.setParameters("publicIpAddress", publicIPAddress);
-        return listActiveBy(sc, null);
+        return listBy(sc, null);
 	}
 
 	@Override
@@ -234,7 +234,7 @@ public class FirewallRulesDaoImpl extends GenericDaoBase i
         sc.setParameters("publicIpAddress", publicIp);
         sc.setParameters("publicPort", publicPort);
         sc.setParameters("algorithm", algo);
-        return listActiveBy(sc, null);
+        return listBy(sc, null);
 	}
 
 	@Override
@@ -245,7 +245,7 @@ public class FirewallRulesDaoImpl extends GenericDaoBase i
         sc.setParameters("publicPort", port);
         sc.setParameters("forwarding", forwarding);
 
-        return listActiveBy(sc);
+        return listBy(sc);
 	}
 
 	@Override
@@ -270,7 +270,7 @@ public class FirewallRulesDaoImpl extends GenericDaoBase i
         sc.setParameters("publicIpAddress", publicIpAddress);
         sc.setParameters("groupId", securityGroupId);
         sc.setParameters("forwarding", false);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
 	@Override
@@ -278,7 +278,7 @@ public class FirewallRulesDaoImpl extends GenericDaoBase i
 	    SearchCriteria sc = FWByGroupId.create();
 	    sc.setParameters("groupId", securityGroupId);
         sc.setParameters("forwarding", Boolean.TRUE);
-	    return listActiveBy(sc);
+	    return listBy(sc);
 	}
 
     @Override
@@ -287,7 +287,7 @@ public class FirewallRulesDaoImpl extends GenericDaoBase i
         sc.setParameters("publicIpAddress", publicIPAddress);
         sc.setParameters("forwarding", forwarding);
         sc.addAnd("privateIpAddress", SearchCriteria.Op.EQ, privateIp);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -295,7 +295,7 @@ public class FirewallRulesDaoImpl extends GenericDaoBase i
         SearchCriteria sc = FWByGroupId.create();
         sc.setParameters("groupId", loadBalancerId);
         sc.setParameters("forwarding", Boolean.FALSE);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -304,7 +304,7 @@ public class FirewallRulesDaoImpl extends GenericDaoBase i
         sc.setParameters("groupId", groupId);
         sc.setParameters("privateIpAddress", privateIp);
         sc.setParameters("forwarding", forwarding);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
         
     }
 }
diff --git a/core/src/com/cloud/network/dao/IPAddressDaoImpl.java b/core/src/com/cloud/network/dao/IPAddressDaoImpl.java
index cbb58710833..de01b197ddd 100644
--- a/core/src/com/cloud/network/dao/IPAddressDaoImpl.java
+++ b/core/src/com/cloud/network/dao/IPAddressDaoImpl.java
@@ -142,8 +142,9 @@ public class IPAddressDaoImpl extends GenericDaoBase implem
 				txn.commit();
 				return ip.getAddress();
 			} else {
-				txn.rollback();
-				s_logger.error("Unable to find an available IP address with related vlan, vlanDbId: " + vlanDbId);
+				txn.rollback();
+				//we do not log this as an error now, as there can be multiple vlans across which we iterate
+				s_logger.warn("Unable to find an available IP address with related vlan, vlanDbId: " + vlanDbId);
 			}
 		} catch (Exception e) {
 			s_logger.warn("Unable to assign IP", e);
@@ -172,14 +173,14 @@ public class IPAddressDaoImpl extends GenericDaoBase implem
     public List listByAccount(long accountId) {
     	SearchCriteria sc = AccountSearch.create();
         sc.setParameters("accountId", accountId);
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
     }
 	
 	public List listByDcIdIpAddress(long dcId, String ipAddress) {
 		SearchCriteria sc = DcIpSearch.create();
 		sc.setParameters("dataCenterId", dcId);
 		sc.setParameters("ipAddress", ipAddress);
-		return listBy(sc);
+		return listIncludingRemovedBy(sc);
 	}
 	
 	@Override @DB
diff --git a/core/src/com/cloud/network/dao/LoadBalancerDaoImpl.java b/core/src/com/cloud/network/dao/LoadBalancerDaoImpl.java
index 25d5eb42b33..a9dcda58557 100644
--- a/core/src/com/cloud/network/dao/LoadBalancerDaoImpl.java
+++ b/core/src/com/cloud/network/dao/LoadBalancerDaoImpl.java
@@ -89,7 +89,7 @@ public class LoadBalancerDaoImpl extends GenericDaoBase im
     public List listByIpAddress(String ipAddress) {
         SearchCriteria sc = ListByIp.create();
         sc.setParameters("ipAddress", ipAddress);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -97,7 +97,7 @@ public class LoadBalancerDaoImpl extends GenericDaoBase im
         SearchCriteria sc = IpAndPublicPortSearch.create();
         sc.setParameters("ipAddress", ipAddress);
         sc.setParameters("publicPort", publicPort);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 
     @Override
@@ -105,6 +105,6 @@ public class LoadBalancerDaoImpl extends GenericDaoBase im
         SearchCriteria sc = AccountAndNameSearch.create();
         sc.setParameters("accountId", accountId);
         sc.setParameters("name", name);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 }
diff --git a/core/src/com/cloud/network/dao/LoadBalancerVMMapDaoImpl.java b/core/src/com/cloud/network/dao/LoadBalancerVMMapDaoImpl.java
index f82dbbb7c17..ca9c6915dc4 100644
--- a/core/src/com/cloud/network/dao/LoadBalancerVMMapDaoImpl.java
+++ b/core/src/com/cloud/network/dao/LoadBalancerVMMapDaoImpl.java
@@ -34,7 +34,7 @@ public class LoadBalancerVMMapDaoImpl extends GenericDaoBase sc = createSearchCriteria();
         sc.addAnd("loadBalancerId", SearchCriteria.Op.EQ, loadBalancerId);
 
-        delete(sc);
+        expunge(sc);
     }
 
     @Override
@@ -46,7 +46,7 @@ public class LoadBalancerVMMapDaoImpl extends GenericDaoBase sc = createSearchCriteria();
         sc.addAnd("instanceId", SearchCriteria.Op.EQ, instanceId);
 
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -62,7 +62,7 @@ public class LoadBalancerVMMapDaoImpl extends GenericDaoBase sc = createSearchCriteria();
         sc.addAnd("loadBalancerId", SearchCriteria.Op.EQ, loadBalancerId);
 
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -71,6 +71,6 @@ public class LoadBalancerVMMapDaoImpl extends GenericDaoBase listBySecurityGroupId(long securityGroupId) {
         SearchCriteria sc = SecurityGroupIdSearch.create();
         sc.setParameters("securityGroupId", securityGroupId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     public void deleteBySecurityGroup(long securityGroupId) {
         SearchCriteria sc = SecurityGroupIdSearch.create();
         sc.setParameters("securityGroupId", securityGroupId);
-        delete(sc);
+        expunge(sc);
     }
 }
diff --git a/core/src/com/cloud/network/dao/SecurityGroupDaoImpl.java b/core/src/com/cloud/network/dao/SecurityGroupDaoImpl.java
index 99e230f3978..3fbb233f57a 100644
--- a/core/src/com/cloud/network/dao/SecurityGroupDaoImpl.java
+++ b/core/src/com/cloud/network/dao/SecurityGroupDaoImpl.java
@@ -41,7 +41,7 @@ public class SecurityGroupDaoImpl extends GenericDaoBase
     public List listByAccountId(long accountId) {
         SearchCriteria sc = AccountIdSearch.create();
         sc.setParameters("accountId", accountId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -55,7 +55,7 @@ public class SecurityGroupDaoImpl extends GenericDaoBase
             sc.addAnd("accountId", SearchCriteria.Op.NULL);
         }
 
-        List securityGroups = listActiveBy(sc);
+        List securityGroups = listBy(sc);
         return ((securityGroups != null) && !securityGroups.isEmpty());
     }
 }
diff --git a/core/src/com/cloud/network/dao/SecurityGroupVMMapDaoImpl.java b/core/src/com/cloud/network/dao/SecurityGroupVMMapDaoImpl.java
index c8da6f4da18..05be43c5002 100644
--- a/core/src/com/cloud/network/dao/SecurityGroupVMMapDaoImpl.java
+++ b/core/src/com/cloud/network/dao/SecurityGroupVMMapDaoImpl.java
@@ -57,28 +57,28 @@ public class SecurityGroupVMMapDaoImpl extends GenericDaoBase sc = ListByIpAndVmId.create();
         sc.setParameters("ipAddress", ipAddress);
         sc.setParameters("instanceId", vmId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
     public List listBySecurityGroup(long securityGroupId) {
         SearchCriteria sc = ListBySecurityGroup.create();
         sc.setParameters("securityGroupId", securityGroupId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
     public List listByIp(String ipAddress) {
         SearchCriteria sc = ListByIp.create();
         sc.setParameters("ipAddress", ipAddress);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
     public List listByInstanceId(long vmId) {
         SearchCriteria sc = ListByVmId.create();
         sc.setParameters("instanceId", vmId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
 }
diff --git a/core/src/com/cloud/network/security/NetworkGroupVO.java b/core/src/com/cloud/network/security/NetworkGroupVO.java
index d20823bb035..7aa736d070d 100644
--- a/core/src/com/cloud/network/security/NetworkGroupVO.java
+++ b/core/src/com/cloud/network/security/NetworkGroupVO.java
@@ -18,22 +18,23 @@
 
 package com.cloud.network.security;
 
-import javax.persistence.Column;
-import javax.persistence.Entity;
-import javax.persistence.GeneratedValue;
-import javax.persistence.GenerationType;
-import javax.persistence.Id;
-import javax.persistence.PrimaryKeyJoinColumn;
-import javax.persistence.SecondaryTable;
-import javax.persistence.Table;
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+
+import com.cloud.domain.PartOf;
+import com.cloud.user.OwnedBy;
 
 @Entity
 @Table(name=("network_group"))
-public class NetworkGroupVO {
+public class NetworkGroupVO implements PartOf, OwnedBy {
     @Id
     @GeneratedValue(strategy=GenerationType.IDENTITY)
     @Column(name="id")
-    private Long id;
+    private long id;
 
     @Column(name="name")
     private String name;
@@ -42,17 +43,17 @@ public class NetworkGroupVO {
     private String description;
 
     @Column(name="domain_id")
-    private Long domainId;
+    private long domainId;
 
     @Column(name="account_id")
-    private Long accountId;
+    private long accountId;
     
     @Column(name="account_name")
     private String accountName = null;
 
     public NetworkGroupVO() {}
 
-    public NetworkGroupVO(String name, String description, Long domainId, Long accountId, String accountName) {
+    public NetworkGroupVO(String name, String description, long domainId, long accountId, String accountName) {
         this.name = name;
         this.description = description;
         this.domainId = domainId;
@@ -60,7 +61,7 @@ public class NetworkGroupVO {
         this.accountName = accountName;
     }
 
-    public Long getId() {
+    public long getId() {
         return id;
     }
 
@@ -72,11 +73,11 @@ public class NetworkGroupVO {
         return description;
     }
 
-    public Long getDomainId() {
+    public long getDomainId() {
         return domainId;
     }
 
-    public Long getAccountId() {
+    public long getAccountId() {
         return accountId;
     }
     
diff --git a/core/src/com/cloud/network/security/dao/IngressRuleDaoImpl.java b/core/src/com/cloud/network/security/dao/IngressRuleDaoImpl.java
index 825630c1e43..7d3835d3f9d 100644
--- a/core/src/com/cloud/network/security/dao/IngressRuleDaoImpl.java
+++ b/core/src/com/cloud/network/security/dao/IngressRuleDaoImpl.java
@@ -28,6 +28,7 @@ import com.cloud.network.security.IngressRuleVO;
 import com.cloud.network.security.NetworkGroupVO;
 import com.cloud.utils.component.Inject;
 import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.JoinBuilder;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
 
@@ -73,20 +74,20 @@ public class IngressRuleDaoImpl extends GenericDaoBase impl
     public List listByNetworkGroupId(long networkGroupId) {
         SearchCriteria sc = networkGroupIdSearch.create();
         sc.setParameters("networkGroupId", networkGroupId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     public int deleteByNetworkGroup(long networkGroupId) {
         SearchCriteria sc = networkGroupIdSearch.create();
         sc.setParameters("networkGroupId", networkGroupId);
-        return delete(sc);
+        return expunge(sc);
     }
 
 	@Override
 	public List listByAllowedNetworkGroupId(long networkGroupId) {
 		 SearchCriteria sc = allowedNetworkGroupIdSearch.create();
 		 sc.setParameters("allowedNetworkId", networkGroupId);
-		 return listActiveBy(sc);
+		 return listBy(sc);
 	}
 
 	@Override
@@ -98,7 +99,7 @@ public class IngressRuleDaoImpl extends GenericDaoBase impl
 		sc.setParameters("startPort", startPort);
 		sc.setParameters("endPort", endPort);
 		sc.setParameters("cidr", cidr);
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -109,7 +110,7 @@ public class IngressRuleDaoImpl extends GenericDaoBase impl
 		sc.setParameters("startPort", startPort);
 		sc.setParameters("endPort", endPort);
 		sc.setJoinParameters("groupName", "groupName", networkGroup);
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -121,7 +122,7 @@ public class IngressRuleDaoImpl extends GenericDaoBase impl
         protoPortsAndNetworkGroupNameSearch.and("endPort", protoPortsAndNetworkGroupNameSearch.entity().getEndPort(), SearchCriteria.Op.EQ);
         SearchBuilder ngSb = _networkGroupDao.createSearchBuilder();
         ngSb.and("groupName", ngSb.entity().getName(), SearchCriteria.Op.EQ);
-        protoPortsAndNetworkGroupNameSearch.join("groupName", ngSb, protoPortsAndNetworkGroupNameSearch.entity().getAllowedNetworkId(), ngSb.entity().getId());
+        protoPortsAndNetworkGroupNameSearch.join("groupName", ngSb, protoPortsAndNetworkGroupNameSearch.entity().getAllowedNetworkId(), ngSb.entity().getId(), JoinBuilder.JoinType.INNER);
         protoPortsAndNetworkGroupNameSearch.done();
 		return super.configure(name, params);
 	}
@@ -135,7 +136,7 @@ public class IngressRuleDaoImpl extends GenericDaoBase impl
 		sc.setParameters("endPort", endPort);
 		sc.setParameters("allowedNetworkId", allowedGroupId);
 		
-        return delete(sc);
+        return expunge(sc);
 		
 	}
 
@@ -148,7 +149,7 @@ public class IngressRuleDaoImpl extends GenericDaoBase impl
 		sc.setParameters("endPort", endPort);
 		sc.setParameters("cidr", cidr);
 		
-		return delete(sc);
+		return expunge(sc);
 	}
 
 	@Override
@@ -161,6 +162,6 @@ public class IngressRuleDaoImpl extends GenericDaoBase impl
 		sc.setParameters("endPort", endPort);
 		sc.setParameters("allowedNetworkId", allowedGroupId);
 		
-        return findOneBy(sc);
+        return findOneIncludingRemovedBy(sc);
 	}
 }
diff --git a/core/src/com/cloud/network/security/dao/NetworkGroupDaoImpl.java b/core/src/com/cloud/network/security/dao/NetworkGroupDaoImpl.java
index 0c95e178492..7a7197242c1 100644
--- a/core/src/com/cloud/network/security/dao/NetworkGroupDaoImpl.java
+++ b/core/src/com/cloud/network/security/dao/NetworkGroupDaoImpl.java
@@ -52,7 +52,7 @@ public class NetworkGroupDaoImpl extends GenericDaoBase im
     public List listByAccountId(long accountId) {
         SearchCriteria sc = AccountIdSearch.create();
         sc.setParameters("accountId", accountId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -66,7 +66,7 @@ public class NetworkGroupDaoImpl extends GenericDaoBase im
             sc.addAnd("accountId", SearchCriteria.Op.NULL);
         }
 
-        List securityGroups = listActiveBy(sc);
+        List securityGroups = listBy(sc);
         return ((securityGroups != null) && !securityGroups.isEmpty());
     }
 
@@ -76,7 +76,7 @@ public class NetworkGroupDaoImpl extends GenericDaoBase im
 		sc.setParameters("accountId", accountId);
 		sc.setParameters("groupName", name);
 
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -86,6 +86,6 @@ public class NetworkGroupDaoImpl extends GenericDaoBase im
 
 		sc.setParameters("groupNames", (Object [])names);
 
-		return listActiveBy(sc);
+		return listBy(sc);
 	}
 }
diff --git a/core/src/com/cloud/network/security/dao/NetworkGroupRulesDaoImpl.java b/core/src/com/cloud/network/security/dao/NetworkGroupRulesDaoImpl.java
index 20ee78c7d9a..87c8ca52664 100644
--- a/core/src/com/cloud/network/security/dao/NetworkGroupRulesDaoImpl.java
+++ b/core/src/com/cloud/network/security/dao/NetworkGroupRulesDaoImpl.java
@@ -4,7 +4,6 @@ import java.util.List;
 
 import javax.ejb.Local;
 
-import com.cloud.domain.dao.DomainDao;
 import com.cloud.network.security.NetworkGroupRulesVO;
 import com.cloud.utils.db.Filter;
 import com.cloud.utils.db.GenericDaoBase;
@@ -15,9 +14,6 @@ import com.cloud.utils.db.SearchCriteria;
 public class NetworkGroupRulesDaoImpl extends GenericDaoBase implements NetworkGroupRulesDao {
     private SearchBuilder AccountGroupNameSearch;
     private SearchBuilder AccountSearch;
-    private SearchBuilder DomainSearch;
-
-    private DomainDao _domainDao = null;
 
     protected NetworkGroupRulesDaoImpl() {
         AccountGroupNameSearch = createSearchBuilder();
@@ -33,7 +29,7 @@ public class NetworkGroupRulesDaoImpl extends GenericDaoBase listNetworkGroupRules() {
         Filter searchFilter = new Filter(NetworkGroupRulesVO.class, "id", true, null, null);
-        return listAllActive(searchFilter);
+        return listAll(searchFilter);
     }
 
     @Override
@@ -44,7 +40,7 @@ public class NetworkGroupRulesDaoImpl extends GenericDaoBase sc = AccountSearch.create();
         sc.setParameters("accountId", accountId);
 
-        return listActiveBy(sc, searchFilter);
+        return listBy(sc, searchFilter);
     }
 }
diff --git a/core/src/com/cloud/network/security/dao/NetworkGroupVMMapDaoImpl.java b/core/src/com/cloud/network/security/dao/NetworkGroupVMMapDaoImpl.java
index fd985e134c8..7b22635dc18 100644
--- a/core/src/com/cloud/network/security/dao/NetworkGroupVMMapDaoImpl.java
+++ b/core/src/com/cloud/network/security/dao/NetworkGroupVMMapDaoImpl.java
@@ -79,35 +79,35 @@ public class NetworkGroupVMMapDaoImpl extends GenericDaoBase sc = ListByIpAndVmId.create();
         sc.setParameters("ipAddress", ipAddress);
         sc.setParameters("instanceId", vmId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
     public List listByNetworkGroup(long networkGroupId) {
         SearchCriteria sc = ListByNetworkGroup.create();
         sc.setParameters("networkGroupId", networkGroupId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
     public List listByIp(String ipAddress) {
         SearchCriteria sc = ListByIp.create();
         sc.setParameters("ipAddress", ipAddress);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
     public List listByInstanceId(long vmId) {
         SearchCriteria sc = ListByVmId.create();
         sc.setParameters("instanceId", vmId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
     public int deleteVM(long instanceId) {
     	SearchCriteria sc = ListByVmId.create();
         sc.setParameters("instanceId", instanceId);
-        return super.delete(sc);
+        return super.expunge(sc);
     }
 
 	@Override
@@ -115,14 +115,14 @@ public class NetworkGroupVMMapDaoImpl extends GenericDaoBase sc = ListByNetworkGroupAndStates.create();
 		sc.setParameters("networkGroupId", networkGroupId);
 		sc.setParameters("states", (Object[])vmStates);
-		return listActiveBy(sc);
+		return listBy(sc);
 	}
 	
     @Override
     public List listVmIdsByNetworkGroup(long networkGroupId) {
         SearchCriteria sc = ListVmIdByNetworkGroup.create();
         sc.setParameters("networkGroupId", networkGroupId);
-        return searchAll(sc, null);
+        return searchIncludingRemoved(sc, null);
     }
 
 	@Override
@@ -130,7 +130,7 @@ public class NetworkGroupVMMapDaoImpl extends GenericDaoBase sc = ListByVmIdGroupId.create();
         sc.setParameters("networkGroupId", networkGroupId);
         sc.setParameters("instanceId", instanceId);
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 	
 }
diff --git a/core/src/com/cloud/network/security/dao/NetworkGroupWorkDaoImpl.java b/core/src/com/cloud/network/security/dao/NetworkGroupWorkDaoImpl.java
index 34f219d132a..75b2d2ec370 100644
--- a/core/src/com/cloud/network/security/dao/NetworkGroupWorkDaoImpl.java
+++ b/core/src/com/cloud/network/security/dao/NetworkGroupWorkDaoImpl.java
@@ -89,7 +89,7 @@ public class NetworkGroupWorkDaoImpl extends GenericDaoBase sc = taken?VmIdTakenSearch.create():VmIdUnTakenSearch.create();
         sc.setParameters("vmId", vmId);
-        return findOneBy(sc);
+        return findOneIncludingRemovedBy(sc);
     }
 
 	@Override
@@ -161,7 +161,7 @@ public class NetworkGroupWorkDaoImpl extends GenericDaoBase sc = VmIdStepSearch.create();
         sc.setParameters("vmId", vmId);
         sc.setParameters("step", step);
-        return findOneBy(sc);
+        return findOneIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -187,7 +187,7 @@ public class NetworkGroupWorkDaoImpl extends GenericDaoBase result = listBy(sc);
+		List result = listIncludingRemovedBy(sc);
 		
 		NetworkGroupWorkVO work = createForUpdate();
 		work.setStep(Step.Error);
diff --git a/core/src/com/cloud/network/security/dao/VmRulesetLogDaoImpl.java b/core/src/com/cloud/network/security/dao/VmRulesetLogDaoImpl.java
index 469012ed53b..cb536c2c421 100644
--- a/core/src/com/cloud/network/security/dao/VmRulesetLogDaoImpl.java
+++ b/core/src/com/cloud/network/security/dao/VmRulesetLogDaoImpl.java
@@ -42,7 +42,7 @@ public class VmRulesetLogDaoImpl extends GenericDaoBase im
     public VmRulesetLogVO findByVmId(long vmId) {
         SearchCriteria sc = VmIdSearch.create();
         sc.setParameters("vmId", vmId);
-        return findOneBy(sc);
+        return findOneIncludingRemovedBy(sc);
     }
 
     
diff --git a/core/src/com/cloud/offerings/NetworkOfferingVO.java b/core/src/com/cloud/offerings/NetworkOfferingVO.java
index 26ad13b8c13..6a0a4339501 100644
--- a/core/src/com/cloud/offerings/NetworkOfferingVO.java
+++ b/core/src/com/cloud/offerings/NetworkOfferingVO.java
@@ -17,6 +17,8 @@
  */
 package com.cloud.offerings;
 
+import java.util.Date;
+
 import javax.persistence.Column;
 import javax.persistence.Entity;
 import javax.persistence.EnumType;
@@ -26,11 +28,19 @@ import javax.persistence.GenerationType;
 import javax.persistence.Id;
 import javax.persistence.Table;
 
+import com.cloud.network.Network.TrafficType;
 import com.cloud.offering.NetworkOffering;
+import com.cloud.utils.db.GenericDao;
 
 @Entity
 @Table(name="network_offerings")
 public class NetworkOfferingVO implements NetworkOffering {
+    public final static String SystemVmPublicNetwork = "System-Vm-Public-Network";
+    public final static String SystemVmGuestNetwork = "System-Vm-Guest-Network";
+    public final static String SystemVmControlNetwork = "System-Vm-Control-Network";
+    public final static String SystemVmManagementNetwork = "System-Vm-Management-Network";
+    public final static String SystemVmStorageNetwork = "System-Vm-Storage-Network";
+    
     @Id
     @GeneratedValue(strategy=GenerationType.IDENTITY)
     @Column(name="id")
@@ -42,10 +52,10 @@ public class NetworkOfferingVO implements NetworkOffering {
     @Column(name="display_text")
     String displayText;
     
-    @Column(name="rate")
+    @Column(name="nw_rate")
     Integer rateMbps;
     
-    @Column(name="multicast_rate")
+    @Column(name="mc_rate")
     Integer multicastRateMbps;
     
     @Column(name="concurrent_connections")
@@ -54,6 +64,22 @@ public class NetworkOfferingVO implements NetworkOffering {
     @Column(name="type")
     @Enumerated(value=EnumType.STRING)
     GuestIpType guestIpType;
+    
+    @Column(name="traffic_type")
+    @Enumerated(value=EnumType.STRING)
+    TrafficType trafficType;
+    
+    @Column(name="system_only")
+    boolean systemOnly;
+    
+    @Column(name="tags")
+    String tags;
+    
+    @Column(name=GenericDao.REMOVED_COLUMN)
+    Date removed;
+    
+    @Column(name=GenericDao.CREATED_COLUMN)
+    Date created;
 
     @Override
     public String getDisplayText() {
@@ -69,6 +95,11 @@ public class NetworkOfferingVO implements NetworkOffering {
     public long getId() {
         return id;
     }
+    
+    @Override
+    public TrafficType getTrafficType() {
+        return trafficType;
+    }
 
     @Override
     public Integer getMulticastRateMbps() {
@@ -85,21 +116,58 @@ public class NetworkOfferingVO implements NetworkOffering {
         return rateMbps;
     }
     
+    public Date getCreated() {
+        return created;
+    }
+    
+    public boolean isSystemOnly() {
+        return systemOnly;
+    }
+    
+    public Date getRemoved() {
+        return removed;
+    }
+    
+    @Override
+    public Integer getConcurrentConnections() {
+        return concurrentConnections;
+    }
+    
+    public String getTags() {
+        return tags;
+    }
+    
+    public void setTags(String tags) {
+        this.tags = tags;
+    }
+    
     public NetworkOfferingVO() {
     }
     
-    public NetworkOfferingVO(String name, String displayText, GuestIpType type, Integer rateMbps, Integer multicastRateMbps, Integer concurrentConnections) {
+    public NetworkOfferingVO(String name, String displayText, TrafficType trafficType, GuestIpType type, boolean systemOnly, Integer rateMbps, Integer multicastRateMbps, Integer concurrentConnections) {
         this.name = name;
         this.displayText = displayText;
         this.guestIpType = type;
         this.rateMbps = rateMbps;
         this.multicastRateMbps = multicastRateMbps;
         this.concurrentConnections = concurrentConnections;
+        this.trafficType = trafficType;
+        this.systemOnly = systemOnly;
     }
-
+    
+    /**
+     * Network Offering for all system vms.
+     * @param name
+     * @param trafficType
+     * @param type
+     */
+    public NetworkOfferingVO(String name, TrafficType trafficType, GuestIpType type) {
+        this(name, "System Offering for " + name, trafficType, type, true, null, null, null);
+    }
+    
     @Override
-    public Integer getConcurrentConnections() {
-        return concurrentConnections;
+    public String toString() {
+        StringBuilder buf = new StringBuilder("[Network Offering [");
+        return buf.append(id).append("-").append(trafficType).append("-").append(name).append("-").append(guestIpType).append("]").toString();
     }
-
 }
diff --git a/core/src/com/cloud/resource/DiskPreparer.java b/core/src/com/cloud/resource/DiskPreparer.java
index c24813d66d9..05ffa78465f 100644
--- a/core/src/com/cloud/resource/DiskPreparer.java
+++ b/core/src/com/cloud/resource/DiskPreparer.java
@@ -18,7 +18,7 @@
 package com.cloud.resource;
 
 import com.cloud.storage.VolumeVO;
-import com.cloud.storage.VirtualMachineTemplate.BootloaderType;
+import com.cloud.template.VirtualMachineTemplate.BootloaderType;
 import com.cloud.utils.component.Adapter;
 
 /**
diff --git a/core/src/com/cloud/resource/ServerResource.java b/core/src/com/cloud/resource/ServerResource.java
index b056a478016..75a37155361 100755
--- a/core/src/com/cloud/resource/ServerResource.java
+++ b/core/src/com/cloud/resource/ServerResource.java
@@ -71,4 +71,8 @@ public interface ServerResource extends Manager {
     public IAgentControl getAgentControl();
     
     public void setAgentControl(IAgentControl agentControl);
+    
+    public boolean IsRemoteAgent();
+    
+    public void setRemoteAgent(boolean remote);
 }
diff --git a/core/src/com/cloud/resource/ServerResourceBase.java b/core/src/com/cloud/resource/ServerResourceBase.java
index c10da164984..a60849c0772 100755
--- a/core/src/com/cloud/resource/ServerResourceBase.java
+++ b/core/src/com/cloud/resource/ServerResourceBase.java
@@ -48,6 +48,7 @@ public abstract class ServerResourceBase implements ServerResource {
     protected NetworkInterface _storageNic;
     protected NetworkInterface _storageNic2;
     protected IAgentControl _agentControl;
+    protected boolean _isRemoteAgent = false;
 
     @Override
     public String getName() {
@@ -301,4 +302,12 @@ public abstract class ServerResourceBase implements ServerResource {
     public boolean stop() {
         return true;
     }
+    
+    public boolean IsRemoteAgent() {
+    	return _isRemoteAgent;
+    }
+    
+    public void setRemoteAgent(boolean remote) {
+    	_isRemoteAgent = remote;
+    }
 }
diff --git a/core/src/com/cloud/server/Criteria.java b/core/src/com/cloud/server/Criteria.java
index f2d7090341c..6d41b967be9 100644
--- a/core/src/com/cloud/server/Criteria.java
+++ b/core/src/com/cloud/server/Criteria.java
@@ -76,7 +76,8 @@ public class Criteria {
     public static final String TARGET_IQN = "targetiqn";
     public static final String SCOPE = "scope";
     public static final String NETWORKGROUP = "networkGroup";
-
+    public static final String GROUP = "group";
+    public static final String EMPTY_GROUP = "emptyGroup";
 
 	public Criteria(String orderBy, Boolean ascending, Long offset, Long limit) {
 		this.offset = offset;
diff --git a/core/src/com/cloud/service/ServiceOfferingVO.java b/core/src/com/cloud/service/ServiceOfferingVO.java
index 8466ce82914..d5d4b4d7ce6 100644
--- a/core/src/com/cloud/service/ServiceOfferingVO.java
+++ b/core/src/com/cloud/service/ServiceOfferingVO.java
@@ -27,6 +27,7 @@ import javax.persistence.PrimaryKeyJoinColumn;
 import javax.persistence.Table;
 import javax.persistence.Transient;
 
+import com.cloud.offering.NetworkOffering;
 import com.cloud.offering.ServiceOffering;
 import com.cloud.storage.DiskOfferingVO;
 
@@ -55,13 +56,13 @@ public class ServiceOfferingVO extends DiskOfferingVO implements ServiceOffering
     
     @Column(name="guest_ip_type")
     @Enumerated(EnumType.STRING)
-    private GuestIpType guestIpType;
+    private NetworkOffering.GuestIpType guestIpType;
     
     protected ServiceOfferingVO() {
         super();
     }
 
-    public ServiceOfferingVO(String name, int cpu, int ramSize, int speed, int rateMbps, int multicastRateMbps, boolean offerHA, String displayText, GuestIpType guestIpType, boolean useLocalStorage, boolean recreatable, String tags) {
+    public ServiceOfferingVO(String name, int cpu, int ramSize, int speed, int rateMbps, int multicastRateMbps, boolean offerHA, String displayText, NetworkOffering.GuestIpType guestIpType, boolean useLocalStorage, boolean recreatable, String tags) {
         super(name, displayText, false, tags, recreatable, useLocalStorage);
         this.cpu = cpu;
         this.ramSize = ramSize;
@@ -135,11 +136,11 @@ public class ServiceOfferingVO extends DiskOfferingVO implements ServiceOffering
 		return multicastRateMbps;
 	}
 
-	public void setGuestIpType(GuestIpType guestIpType) {
+	public void setGuestIpType(NetworkOffering.GuestIpType guestIpType) {
 		this.guestIpType = guestIpType;
 	}
 
-	public GuestIpType getGuestIpType() {
+	public NetworkOffering.GuestIpType getGuestIpType() {
 		return guestIpType;
 	}
 }
diff --git a/core/src/com/cloud/service/dao/ServiceOfferingDaoImpl.java b/core/src/com/cloud/service/dao/ServiceOfferingDaoImpl.java
index 33ef7cc064c..d01f9384048 100644
--- a/core/src/com/cloud/service/dao/ServiceOfferingDaoImpl.java
+++ b/core/src/com/cloud/service/dao/ServiceOfferingDaoImpl.java
@@ -49,7 +49,7 @@ public class ServiceOfferingDaoImpl extends GenericDaoBase sc = UniqueNameSearch.create();
         sc.setParameters("name", name);
-        List vos = searchAll(sc, null, null, false);
+        List vos = searchIncludingRemoved(sc, null, null, false);
         if (vos.size() == 0) {
             return null;
         }
diff --git a/core/src/com/cloud/storage/DiskOfferingVO.java b/core/src/com/cloud/storage/DiskOfferingVO.java
index 5b972e817a0..48f817a2f9f 100644
--- a/core/src/com/cloud/storage/DiskOfferingVO.java
+++ b/core/src/com/cloud/storage/DiskOfferingVO.java
@@ -86,8 +86,7 @@ public class DiskOfferingVO implements DiskOffering {
     
     @Column(name="use_local_storage")
     private boolean useLocalStorage;
-
-    
+    
     public DiskOfferingVO() {
     }
 
@@ -235,5 +234,9 @@ public class DiskOfferingVO implements DiskOffering {
         buf.delete(buf.length() - 1, buf.length());
         
         setTags(buf.toString());
-    }
+    }
+
+	public void setUseLocalStorage(boolean useLocalStorage) {
+		this.useLocalStorage = useLocalStorage;
+	}
 }
diff --git a/core/src/com/cloud/storage/FileSystemStorageResource.java b/core/src/com/cloud/storage/FileSystemStorageResource.java
index 4df2f409583..72f180935bb 100644
--- a/core/src/com/cloud/storage/FileSystemStorageResource.java
+++ b/core/src/com/cloud/storage/FileSystemStorageResource.java
@@ -41,7 +41,7 @@ import com.cloud.agent.api.storage.UpgradeDiskAnswer;
 import com.cloud.agent.api.storage.UpgradeDiskCommand;
 import com.cloud.agent.api.to.VolumeTO;
 import com.cloud.storage.Storage.StoragePoolType;
-import com.cloud.storage.Volume.StorageResourceType;
+import com.cloud.storage.Storage.StorageResourceType;
 import com.cloud.storage.template.TemplateInfo;
 import com.cloud.utils.exception.CloudRuntimeException;
 import com.cloud.utils.net.NfsUtils;
@@ -329,8 +329,8 @@ public abstract class FileSystemStorageResource extends StorageResource {
 	}
 
 	@Override
-	protected StorageResourceType getStorageResourceType() {
-		return StorageResourceType.STORAGE_POOL;
+	protected Storage.StorageResourceType getStorageResourceType() {
+		return Storage.StorageResourceType.STORAGE_POOL;
 	}
 
 	protected String mountNfs(String hostAddress, String hostPath, String localPath) {
diff --git a/core/src/com/cloud/storage/StoragePoolVO.java b/core/src/com/cloud/storage/StoragePoolVO.java
index 41cabb559b7..f3a3460157c 100644
--- a/core/src/com/cloud/storage/StoragePoolVO.java
+++ b/core/src/com/cloud/storage/StoragePoolVO.java
@@ -181,6 +181,7 @@ public class StoragePoolVO implements StoragePool {
         this.path = hostPath;
         this.port = port;
         this.podId = podId;
+        this.setStatus(Status.Up);
     }
     
     public StoragePoolVO(StoragePoolType type, String hostAddress, int port, String path) {
@@ -188,6 +189,7 @@ public class StoragePoolVO implements StoragePool {
         this.hostAddress = hostAddress;
         this.port = port;
         this.path = path;
+        this.setStatus(Status.Up);
     }
     
     public void setStatus(Status status)
diff --git a/core/src/com/cloud/storage/StorageResource.java b/core/src/com/cloud/storage/StorageResource.java
index 6ca2dd80cc7..d63883be1b7 100755
--- a/core/src/com/cloud/storage/StorageResource.java
+++ b/core/src/com/cloud/storage/StorageResource.java
@@ -53,12 +53,14 @@ import com.cloud.agent.api.storage.ShareAnswer;
 import com.cloud.agent.api.storage.ShareCommand;
 import com.cloud.agent.api.storage.UpgradeDiskAnswer;
 import com.cloud.agent.api.storage.UpgradeDiskCommand;
+import com.cloud.agent.api.storage.UploadCommand;
 import com.cloud.host.Host;
 import com.cloud.resource.ServerResource;
 import com.cloud.resource.ServerResourceBase;
 import com.cloud.storage.Storage.StoragePoolType;
 import com.cloud.storage.template.DownloadManager;
 import com.cloud.storage.template.TemplateInfo;
+import com.cloud.storage.template.UploadManager;
 import com.cloud.utils.NumbersUtil;
 import com.cloud.utils.exception.CloudRuntimeException;
 import com.cloud.utils.script.OutputInterpreter;
@@ -112,6 +114,7 @@ public abstract class StorageResource extends ServerResourceBase implements Serv
 	protected String _zfsScriptsDir;
 
 	protected DownloadManager _downloadManager;
+	protected UploadManager _uploadManager;
 
 	protected Map _volumeHourlySnapshotRequests = new HashMap();
     protected Map _volumeDailySnapshotRequests = new HashMap();
@@ -127,6 +130,8 @@ public abstract class StorageResource extends ServerResourceBase implements Serv
         	return execute((PrimaryStorageDownloadCommand)cmd);
         } else if (cmd instanceof DownloadCommand) {
             return execute((DownloadCommand)cmd);
+        }else if (cmd instanceof UploadCommand) {
+                return execute((UploadCommand)cmd);
         } else if (cmd instanceof GetStorageStatsCommand) {
             return execute((GetStorageStatsCommand)cmd);
         } else if (cmd instanceof UpgradeDiskCommand) {
@@ -159,6 +164,10 @@ public abstract class StorageResource extends ServerResourceBase implements Serv
     protected Answer execute(final PrimaryStorageDownloadCommand cmd) {
     	return Answer.createUnsupportedCommandAnswer(cmd);
     }
+
+	private Answer execute(UploadCommand cmd) {		
+		return _uploadManager.handleUploadCommand(cmd);
+	}
     
     protected Answer execute(final DownloadCommand cmd) {
     	return _downloadManager.handleDownloadCommand(cmd);
@@ -195,7 +204,7 @@ public abstract class StorageResource extends ServerResourceBase implements Serv
         String path = rootdiskFolder + File.separator + "rootdisk";
         long totalSize = getVolumeSize(path);
 
-        VolumeVO vol = new VolumeVO(null, null, -1, -1, -1, -1, new Long(-1), rootdiskFolder, path, totalSize, Volume.VolumeType.ROOT);
+        VolumeVO vol = new VolumeVO(null, -1, -1, -1, -1, new Long(-1), rootdiskFolder, path, totalSize, Volume.VolumeType.ROOT);
         vols.add(vol);
 
         // Get the datadisk volume
@@ -203,7 +212,7 @@ public abstract class StorageResource extends ServerResourceBase implements Serv
         	path = datadiskFolder + File.separator + datadiskName;
             totalSize = getVolumeSize(path);
 
-            vol = new VolumeVO(null, null, -1, -1, -1, -1, new Long(-1), datadiskFolder, path, totalSize, Volume.VolumeType.DATADISK);
+            vol = new VolumeVO(null, -1, -1, -1, -1, new Long(-1), datadiskFolder, path, totalSize, Volume.VolumeType.DATADISK);
             vols.add(vol);
         }
 
@@ -216,7 +225,7 @@ public abstract class StorageResource extends ServerResourceBase implements Serv
         String path = getVolumeName(imagePath, null);
         long totalSize = getVolumeSize(path);
 
-        VolumeVO vol = new VolumeVO(null, null, -1, -1, -1, -1, new Long(-1), null, path, totalSize, Volume.VolumeType.ROOT);
+        VolumeVO vol = new VolumeVO(null, -1, -1, -1, -1, new Long(-1), null, path, totalSize, Volume.VolumeType.ROOT);
 
         vols.add(vol);
 
@@ -225,7 +234,7 @@ public abstract class StorageResource extends ServerResourceBase implements Serv
             totalSize = getVolumeSize(path);
 
 
-            vol = new VolumeVO(null, null, -1, -1, -1, -1, new Long(-1), null, path, totalSize, Volume.VolumeType.DATADISK);
+            vol = new VolumeVO(null, -1, -1, -1, -1, new Long(-1), null, path, totalSize, Volume.VolumeType.DATADISK);
             vols.add(vol);
         }
 
@@ -304,7 +313,7 @@ public abstract class StorageResource extends ServerResourceBase implements Serv
     protected abstract Answer execute(DestroyCommand cmd) ;
     protected abstract UpgradeDiskAnswer execute(final UpgradeDiskCommand cmd);
 	protected abstract String  delete(String imagePath, String extra);
-    protected abstract Volume.StorageResourceType getStorageResourceType();
+    protected abstract Storage.StorageResourceType getStorageResourceType();
     protected abstract void configureFolders(String name,  Map params) throws ConfigurationException ;
 
 
diff --git a/core/src/com/cloud/storage/Upload.java b/core/src/com/cloud/storage/Upload.java
new file mode 100755
index 00000000000..b6b54f7ce90
--- /dev/null
+++ b/core/src/com/cloud/storage/Upload.java
@@ -0,0 +1,7 @@
+package com.cloud.storage;
+
+public interface Upload {
+
+	public static enum Status  {UNKNOWN, ABANDONED, UPLOADED, NOT_UPLOADED, UPLOAD_ERROR, UPLOAD_IN_PROGRESS, NOT_COPIED, COPY_IN_PROGRESS, COPY_ERROR, COPY_COMPLETE}
+	public static enum Type  {VOLUME, TEMPLATE, ISO}
+}
diff --git a/core/src/com/cloud/storage/UploadVO.java b/core/src/com/cloud/storage/UploadVO.java
new file mode 100755
index 00000000000..5f1e6adf3d6
--- /dev/null
+++ b/core/src/com/cloud/storage/UploadVO.java
@@ -0,0 +1,219 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
+
+package com.cloud.storage;
+
+import java.util.Date;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.EnumType;
+import javax.persistence.Enumerated;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+import javax.persistence.Temporal;
+import javax.persistence.TemporalType;
+
+import com.cloud.utils.db.GenericDaoBase;
+
+/**
+ * Table for storing all the upload information.
+ * @author nitin
+ *
+ */
+@Entity
+@Table(name="upload")
+public class UploadVO implements Upload {
+	@Id
+	@GeneratedValue(strategy=GenerationType.IDENTITY)
+	Long id;
+	
+	@Column(name="host_id")
+	private long hostId;
+	
+	@Column(name="type_id")
+	private long typeId;
+	
+	@Column(name=GenericDaoBase.CREATED_COLUMN)
+	private Date created = null;
+	
+	@Column(name="last_updated")
+	@Temporal(value=TemporalType.TIMESTAMP)
+	private Date lastUpdated = null;	
+	
+	@Column (name="upload_pct")
+	private int uploadPercent;	
+	
+	@Column (name="type")
+	@Enumerated(EnumType.STRING)
+	private Type type;
+	
+	@Column (name="upload_state")
+	@Enumerated(EnumType.STRING)
+	private Status uploadState;
+	
+	@Column (name="error_str")
+	private String errorString;
+
+	@Column (name="job_id")
+	private String jobId;
+	
+	@Column (name="url")
+	private String uploadUrl;
+
+	public long getHostId() {
+		return hostId;
+	}
+
+	public void setHostId(long hostId) {
+		this.hostId = hostId;
+	}
+
+	public Long getId() {
+		return id;
+	}
+
+	public Date getCreated() {
+		return created;
+	}
+
+	public Date getLastUpdated() {
+		return lastUpdated;
+	}
+	
+	public void setLastUpdated(Date date) {
+	    lastUpdated = date;
+	}
+
+	public UploadVO(long hostId, long templateId) {
+		super();
+		this.hostId = hostId;
+		this.typeId = templateId;
+	}
+
+	public UploadVO(long hostId, long typeId, Date lastUpdated,
+			Status uploadState, int uploadPercent, Type type,
+			String errorString, String jobId, String uploadUrl) {
+		super();
+		this.hostId = hostId;
+		this.typeId = typeId;
+		this.lastUpdated = lastUpdated;
+		this.uploadState = uploadState;
+		this.uploadPercent = uploadPercent;
+		this.type = type;
+		this.errorString = errorString;
+		this.jobId = jobId;
+		this.uploadUrl = uploadUrl;
+	}
+
+	protected UploadVO() {		
+	}
+
+	public UploadVO(Long uploadId) {
+		this.id = uploadId;
+	}
+
+	public void setErrorString(String errorString) {
+		this.errorString = errorString;
+	}
+
+	public String getErrorString() {
+		return errorString;
+	}
+
+	public void setJobId(String jobId) {
+		this.jobId = jobId;
+	}
+
+	public String getJobId() {
+		return jobId;
+	}
+
+	@Override
+	public boolean equals(Object obj) {
+		if (obj instanceof UploadVO) {
+			UploadVO other = (UploadVO)obj;
+		   if (id == null && other.getId() == null) {
+			   return (this.typeId==other.getTypeId() && this.hostId==other.getHostId() && this.type == other.getType());
+		   } else if (id == null && other.getId() != null) {
+			   return false;
+		   } else if (id != null && other.getId() == null) {
+			   return false;
+		   } else {
+			   return (this.id.equals(other.getId()));
+		   }
+		}
+		return false;
+	}
+
+	@Override
+	public int hashCode() {
+		return id.hashCode();
+	}
+
+	public int getUploadPercent() {
+		return uploadPercent;
+	}
+
+	public void setUploadPercent(int uploadPercent) {
+		this.uploadPercent = uploadPercent;
+	}
+
+	public Status getUploadState() {
+		return uploadState;
+	}
+
+	public void setUploadState(Status uploadState) {
+		this.uploadState = uploadState;
+	}
+
+	public long getTypeId() {
+		return typeId;
+	}
+
+	public void setTypeId(long typeId) {
+		this.typeId = typeId;
+	}
+
+	public Type getType() {
+		return type;
+	}
+
+	public void setType(Type type) {
+		this.type = type;
+	}
+
+	public String getUploadUrl() {
+		return uploadUrl;
+	}
+
+	public void setUploadUrl(String uploadUrl) {
+		this.uploadUrl = uploadUrl;
+	}
+
+	public void setId(Long id) {
+		this.id = id;
+	}
+
+	public void setCreated(Date created) {
+		this.created = created;
+	}
+
+}
diff --git a/core/src/com/cloud/storage/VMTemplateHostVO.java b/core/src/com/cloud/storage/VMTemplateHostVO.java
old mode 100644
new mode 100755
index c7d78fd7af0..9e6c1e7b2cb
--- a/core/src/com/cloud/storage/VMTemplateHostVO.java
+++ b/core/src/com/cloud/storage/VMTemplateHostVO.java
@@ -59,14 +59,14 @@ public class VMTemplateHostVO implements VMTemplateStorageResourceAssoc {
 	private Date lastUpdated = null;
 	
 	@Column (name="download_pct")
-	private int downloadPercent;
+	private int downloadPercent;
 	
 	@Column (name="size")
 	private long size;
 	
 	@Column (name="download_state")
 	@Enumerated(EnumType.STRING)
-	private Status downloadState;
+	private Status downloadState;
 	
 	@Column (name="local_path")
 	private String localDownloadPath;
@@ -76,7 +76,7 @@ public class VMTemplateHostVO implements VMTemplateStorageResourceAssoc {
 	
 	@Column (name="job_id")
 	private String jobId;
-	
+
 	@Column (name="pool_id")
 	private Long poolId;
 	
@@ -85,7 +85,7 @@ public class VMTemplateHostVO implements VMTemplateStorageResourceAssoc {
 	
 	@Column (name="url")
 	private String downloadUrl;
-	
+
 	@Column(name="is_copy")
 	private boolean isCopy = false;
     
@@ -262,5 +262,6 @@ public class VMTemplateHostVO implements VMTemplateStorageResourceAssoc {
 
 	public boolean isCopy() {
 		return isCopy;
-	}
+	}
+
 }
diff --git a/core/src/com/cloud/storage/VMTemplateStorageResourceAssoc.java b/core/src/com/cloud/storage/VMTemplateStorageResourceAssoc.java
index 699f2f8297a..d42c7f10cb7 100644
--- a/core/src/com/cloud/storage/VMTemplateStorageResourceAssoc.java
+++ b/core/src/com/cloud/storage/VMTemplateStorageResourceAssoc.java
@@ -24,7 +24,7 @@ import java.util.Date;
  *
  */
 public interface VMTemplateStorageResourceAssoc {
-	public static enum Status  {UNKNOWN, DOWNLOAD_ERROR, NOT_DOWNLOADED, DOWNLOAD_IN_PROGRESS, DOWNLOADED, ABANDONED}
+	public static enum Status  {UNKNOWN, DOWNLOAD_ERROR, NOT_DOWNLOADED, DOWNLOAD_IN_PROGRESS, DOWNLOADED, ABANDONED, UPLOADED, NOT_UPLOADED, UPLOAD_ERROR, UPLOAD_IN_PROGRESS}
 
 	public String getInstallPath();
 
diff --git a/core/src/com/cloud/storage/VMTemplateVO.java b/core/src/com/cloud/storage/VMTemplateVO.java
index 43213545f8f..e46a3bcc66e 100644
--- a/core/src/com/cloud/storage/VMTemplateVO.java
+++ b/core/src/com/cloud/storage/VMTemplateVO.java
@@ -37,6 +37,7 @@ import com.cloud.storage.Storage.ImageFormat;
 import com.cloud.utils.db.GenericDao;
 import com.google.gson.annotations.Expose;
 import com.cloud.storage.Storage;
+import com.cloud.template.VirtualMachineTemplate;
 
 @Entity
 @Table(name="vm_template")
diff --git a/core/src/com/cloud/storage/VolumeVO.java b/core/src/com/cloud/storage/VolumeVO.java
index c85d628b8da..d5d23dea53d 100755
--- a/core/src/com/cloud/storage/VolumeVO.java
+++ b/core/src/com/cloud/storage/VolumeVO.java
@@ -27,6 +27,7 @@ import javax.persistence.GeneratedValue;
 import javax.persistence.GenerationType;
 import javax.persistence.Id;
 import javax.persistence.Table;
+import javax.persistence.TableGenerator;
 import javax.persistence.Temporal;
 import javax.persistence.TemporalType;
 
@@ -39,9 +40,10 @@ import com.google.gson.annotations.Expose;
 @Table(name="volumes")
 public class VolumeVO implements Volume {
     @Id
-    @GeneratedValue(strategy=GenerationType.IDENTITY)
+    @TableGenerator(name="volume_sq", table="sequence", pkColumnName="name", valueColumnName="value", pkColumnValue="volume_seq", allocationSize=1)
+    @GeneratedValue(strategy=GenerationType.TABLE)
     @Column(name="id")
-    Long id;
+    long id;
     
     @Expose
     @Column(name="name")
@@ -88,6 +90,10 @@ public class VolumeVO implements Volume {
     @Column(name="created")
     Date created;
     
+    @Column(name="attached")
+    @Temporal(value=TemporalType.TIMESTAMP)
+    Date attached;
+    
     @Column(name="data_center_id")
     long dataCenterId;
     
@@ -96,7 +102,7 @@ public class VolumeVO implements Volume {
     String hostip;
 
     @Column(name="disk_offering_id")
-    Long diskOfferingId;
+    long diskOfferingId;
 
     @Expose
     @Column(name="mirror_vol")
@@ -129,7 +135,7 @@ public class VolumeVO implements Volume {
     @Expose
     @Column(name="resource_type")
     @Enumerated(EnumType.STRING)
-	StorageResourceType storageResourceType;
+	Storage.StorageResourceType storageResourceType;
     
     @Expose
     @Column(name="status", updatable = true, nullable=false)
@@ -143,6 +149,17 @@ public class VolumeVO implements Volume {
     @Column(name="recreatable")
     boolean recreatable;
     
+    @Column(name="state")
+    @Enumerated(value=EnumType.STRING)
+    private State state;
+    
+    @Column(name="source_type")
+    @Enumerated(value=EnumType.STRING)
+    Volume.SourceType sourceType;
+    
+    @Column(name="source_id")
+    Long sourceId;
+    
     /**
      * Constructor for data disk.
      * @param type
@@ -167,9 +184,24 @@ public class VolumeVO implements Volume {
         this.templateId = null;
         this.mirrorState = MirrorState.NOT_MIRRORED;
         this.mirrorVolume = null;
-        this.storageResourceType = StorageResourceType.STORAGE_POOL;
+        this.storageResourceType = Storage.StorageResourceType.STORAGE_POOL;
         this.poolType = null;
     }
+ 
+    // Real Constructor
+    public VolumeVO(VolumeType type, String name, long dcId, long domainId, long accountId, long diskOfferingId, long size) {
+        this.volumeType = type;
+        this.name = name;
+        this.dataCenterId = dcId;
+        this.accountId = accountId;
+        this.domainId = domainId;
+        this.size = size;
+        this.mirrorVolume = null;
+        this.mirrorState = MirrorState.NOT_MIRRORED;
+        this.diskOfferingId = diskOfferingId;
+        this.status = AsyncInstanceCreateStatus.Creating;
+        this.state = State.Allocated;
+    }
 
     /**
      * Constructor for volume based on a template.
@@ -190,8 +222,7 @@ public class VolumeVO implements Volume {
     }
     
     
-    public VolumeVO(Long id, String name, long dcId, long podId, long accountId, long domainId, Long instanceId, String folder, String path, long size, Volume.VolumeType vType) {
-        this.id = id;
+    public VolumeVO(String name, long dcId, long podId, long accountId, long domainId, Long instanceId, String folder, String path, long size, Volume.VolumeType vType) {
         this.name = name;
         this.accountId = accountId;
         this.domainId = domainId;
@@ -214,7 +245,7 @@ public class VolumeVO implements Volume {
 		return iscsiName;
 	}
 
-	public Long getId() {
+	public long getId() {
         return id;
 	}
 	
@@ -269,7 +300,8 @@ public class VolumeVO implements Volume {
         return size;
     }
     
-	public void setSize(long size) {
+	@Override
+    public void setSize(long size) {
 		this.size = size;
 	}
     
@@ -291,10 +323,6 @@ public class VolumeVO implements Volume {
 		return volumeType;
 	}
 	
-	public void setId(Long id) {
-		this.id = id;
-	}
-
 	public void setName(String name) {
 		this.name = name;
 	}
@@ -371,15 +399,16 @@ public class VolumeVO implements Volume {
 		this.mirrorState = mirrorState;
 	}
 
-	public Long getDiskOfferingId() {
+	public long getDiskOfferingId() {
 		return diskOfferingId;
 	}
 
-	public void setDiskOfferingId(Long diskOfferingId) {
+	public void setDiskOfferingId(long diskOfferingId) {
 		this.diskOfferingId = diskOfferingId;
 	}
 
-	public Long getTemplateId() {
+	@Override
+    public Long getTemplateId() {
 		return templateId;
 	}
 
@@ -404,15 +433,16 @@ public class VolumeVO implements Volume {
 	}
 
 	@Override
-	public StorageResourceType getStorageResourceType() {
+	public Storage.StorageResourceType getStorageResourceType() {
 		return storageResourceType;
 	}
 
-	public void setStorageResourceType(StorageResourceType storageResourceType2) {
+	public void setStorageResourceType(Storage.StorageResourceType storageResourceType2) {
 		this.storageResourceType = storageResourceType2;
 	}
 
-	public Long getPoolId() {
+	@Override
+    public Long getPoolId() {
 		return poolId;
 	}
 	
@@ -420,12 +450,10 @@ public class VolumeVO implements Volume {
 		this.poolId = poolId;
 	}
 	
-	@Override
     public AsyncInstanceCreateStatus getStatus() {
 		return status;
 	}
 	
-	@Override
 	public void setStatus(AsyncInstanceCreateStatus status) {
 		this.status = status;
 	}
@@ -433,6 +461,11 @@ public class VolumeVO implements Volume {
 	public Date getUpdated() {
         return updated;
     }
+	
+	@Override
+    public State getState() {
+	    return state;
+	}
 
     public void setUpdated(Date updated) {
         this.updated = updated;
@@ -490,4 +523,35 @@ public class VolumeVO implements Volume {
     public String toString() {
 	    return new StringBuilder("Vol[").append(id).append("|vm=").append(instanceId).append("|").append(volumeType).append("]").toString();
 	}
+
+	@Override
+	public SourceType getSourceType() {
+		return this.sourceType;
+	}
+
+	@Override
+	public void setSourceType(SourceType sourceType) {
+		this.sourceType = sourceType;
+	}
+	
+	@Override
+	public void setSourceId(Long sourceId){
+		this.sourceId = sourceId;
+	}
+	
+	@Override
+	public Long getSourceId(){
+		return this.sourceId;
+	}
+	
+	@Override
+	public Date getAttached(){
+		return this.attached; 
+	}
+	
+	@Override
+	public void setAttached(Date attached){
+		this.attached = attached;
+	}
+
 }
diff --git a/core/src/com/cloud/storage/dao/DiskOfferingDaoImpl.java b/core/src/com/cloud/storage/dao/DiskOfferingDaoImpl.java
index 76811aa0eec..ae6f587d5fa 100644
--- a/core/src/com/cloud/storage/dao/DiskOfferingDaoImpl.java
+++ b/core/src/com/cloud/storage/dao/DiskOfferingDaoImpl.java
@@ -58,26 +58,26 @@ public class DiskOfferingDaoImpl extends GenericDaoBase im
         SearchCriteria sc = DomainIdSearch.create();
         sc.setParameters("domainId", domainId);
         // FIXME:  this should not be exact match, but instead should find all available disk offerings from parent domains
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
     public List findPrivateDiskOffering() {
         SearchCriteria sc = PrivateDiskOfferingSearch.create();
         sc.setParameters("diskSize", 0);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
-    public List searchAll(SearchCriteria sc, final Filter filter, final Boolean lock, final boolean cache) {
+    public List searchIncludingRemoved(SearchCriteria sc, final Filter filter, final Boolean lock, final boolean cache) {
         sc.addAnd(_typeAttr, Op.EQ, Type.Disk);
-        return super.searchAll(sc, filter, lock, cache);
+        return super.searchIncludingRemoved(sc, filter, lock, cache);
     }
     
     @Override
-    public  List searchAll(SearchCriteria sc, final Filter filter) {
+    public  List searchIncludingRemoved(SearchCriteria sc, final Filter filter) {
         sc.addAnd(_typeAttr, Op.EQ, Type.Disk);
-        return super.searchAll(sc, filter);
+        return super.searchIncludingRemoved(sc, filter);
     }
     
     @Override
diff --git a/core/src/com/cloud/storage/dao/DiskTemplateDaoImpl.java b/core/src/com/cloud/storage/dao/DiskTemplateDaoImpl.java
index 4b2a4b96ab0..7e0f68e75cf 100755
--- a/core/src/com/cloud/storage/dao/DiskTemplateDaoImpl.java
+++ b/core/src/com/cloud/storage/dao/DiskTemplateDaoImpl.java
@@ -46,7 +46,7 @@ public class DiskTemplateDaoImpl extends GenericDaoBase im
     	sc.setParameters("type", type);
     	sc.setParameters("size", size);
     	
-        List vos = listActiveBy(sc);
+        List vos = listBy(sc);
         assert(vos.size() <= 1);   // Should only have one.  If more than one something is wrong.
         return vos.size() == 0 ? null : vos.get(0);
     }
diff --git a/core/src/com/cloud/storage/dao/LaunchPermissionDaoImpl.java b/core/src/com/cloud/storage/dao/LaunchPermissionDaoImpl.java
index f3b076619bb..990523bb6da 100644
--- a/core/src/com/cloud/storage/dao/LaunchPermissionDaoImpl.java
+++ b/core/src/com/cloud/storage/dao/LaunchPermissionDaoImpl.java
@@ -92,7 +92,7 @@ public class LaunchPermissionDaoImpl extends GenericDaoBase sc = TemplateIdSearch.create();
         sc.setParameters("templateId", templateId);
-        delete(sc);
+        expunge(sc);
     }
 
     @Override
@@ -100,7 +100,7 @@ public class LaunchPermissionDaoImpl extends GenericDaoBase sc = TemplateAndAccountSearch.create();
         sc.setParameters("templateId", templateId);
         sc.setParameters("accountId", accountId);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 
     @Override
@@ -153,6 +153,6 @@ public class LaunchPermissionDaoImpl extends GenericDaoBase findByTemplate(long templateId) {
         SearchCriteria sc = TemplateIdSearch.create();
         sc.setParameters("templateId", templateId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 }
diff --git a/core/src/com/cloud/storage/dao/SnapshotDaoImpl.java b/core/src/com/cloud/storage/dao/SnapshotDaoImpl.java
index 4ca3e303b7d..67c59e2ed85 100644
--- a/core/src/com/cloud/storage/dao/SnapshotDaoImpl.java
+++ b/core/src/com/cloud/storage/dao/SnapshotDaoImpl.java
@@ -40,7 +40,7 @@ public class SnapshotDaoImpl extends GenericDaoBase implements
     public SnapshotVO findNextSnapshot(long snapshotId) {
         SearchCriteria sc = ParentIdSearch.create();
         sc.setParameters("prevSnapshotId", snapshotId);
-        return findOneBy(sc);
+        return findOneIncludingRemovedBy(sc);
     }
 
     @Override
@@ -52,7 +52,7 @@ public class SnapshotDaoImpl extends GenericDaoBase implements
     public List listByVolumeId(Filter filter, long volumeId ) {
         SearchCriteria sc = VolumeIdSearch.create();
         sc.setParameters("volumeId", volumeId);
-        return listActiveBy(sc, filter);
+        return listBy(sc, filter);
     }
 
     protected SnapshotDaoImpl() {
@@ -76,7 +76,7 @@ public class SnapshotDaoImpl extends GenericDaoBase implements
 		SearchCriteria sc = lastSnapSearch.create();
 		sc.setParameters("volumeId", volumeId);
 		sc.setParameters("snapId", snapId);
-		List prevSnapshots = searchAll(sc, null);
+		List prevSnapshots = searchIncludingRemoved(sc, null);
 		if(prevSnapshots != null && prevSnapshots.size() > 0 && prevSnapshots.get(0) != null) {
 			return prevSnapshots.get(0);
 		}
diff --git a/core/src/com/cloud/storage/dao/SnapshotPolicyDaoImpl.java b/core/src/com/cloud/storage/dao/SnapshotPolicyDaoImpl.java
index 4a4091d52f3..56d839ddbd4 100644
--- a/core/src/com/cloud/storage/dao/SnapshotPolicyDaoImpl.java
+++ b/core/src/com/cloud/storage/dao/SnapshotPolicyDaoImpl.java
@@ -40,7 +40,7 @@ public class SnapshotPolicyDaoImpl extends GenericDaoBase sc = VolumeIdIntervalSearch.create();
         sc.setParameters("volumeId", volumeId);
         sc.setParameters("interval", interval);
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 	
 	@Override
@@ -53,7 +53,7 @@ public class SnapshotPolicyDaoImpl extends GenericDaoBase sc = VolumeIdSearch.create();
         sc.setParameters("volumeId", volumeId);
         sc.setParameters("active", true);
-        return listActiveBy(sc, filter);
+        return listBy(sc, filter);
     }
 	
     protected SnapshotPolicyDaoImpl() {
@@ -76,6 +76,6 @@ public class SnapshotPolicyDaoImpl extends GenericDaoBase listActivePolicies() {
         SearchCriteria sc = ActivePolicySearch.create();
         sc.setParameters("active", true);
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
     }	
 }
\ No newline at end of file
diff --git a/core/src/com/cloud/storage/dao/SnapshotPolicyRefDaoImpl.java b/core/src/com/cloud/storage/dao/SnapshotPolicyRefDaoImpl.java
index f1d50aafde5..73479ad727e 100644
--- a/core/src/com/cloud/storage/dao/SnapshotPolicyRefDaoImpl.java
+++ b/core/src/com/cloud/storage/dao/SnapshotPolicyRefDaoImpl.java
@@ -55,7 +55,7 @@ public class SnapshotPolicyRefDaoImpl extends GenericDaoBase sc = snapPolicy.create();
 	    sc.setParameters("snapshotId", snapshotId);
 	    sc.setParameters("policyId", policyId);
-	    return findOneBy(sc);
+	    return findOneIncludingRemovedBy(sc);
 	}
 	
 	@Override
@@ -63,14 +63,14 @@ public class SnapshotPolicyRefDaoImpl extends GenericDaoBase sc = snapPolicy.create();
 	    sc.setParameters("snapshotId", snapshotId);
 	    sc.setParameters("policyId", policyId);
-	    return delete(sc);
+	    return expunge(sc);
 	}
 
 	@Override
 	public List listBySnapshotId(long snapshotId) {
 	    SearchCriteria sc = snapSearch.create();
 	    sc.setParameters("snapshotId", snapshotId);
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 	
 	@Override
@@ -78,6 +78,6 @@ public class SnapshotPolicyRefDaoImpl extends GenericDaoBase sc = policySearch.create();
 	    sc.setParameters("policyId", policyId);
 	    sc.setParameters("volumeId", volumeId);
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 }
\ No newline at end of file
diff --git a/core/src/com/cloud/storage/dao/SnapshotScheduleDaoImpl.java b/core/src/com/cloud/storage/dao/SnapshotScheduleDaoImpl.java
index dc31230cd5b..d358976b46d 100644
--- a/core/src/com/cloud/storage/dao/SnapshotScheduleDaoImpl.java
+++ b/core/src/com/cloud/storage/dao/SnapshotScheduleDaoImpl.java
@@ -64,7 +64,7 @@ public class SnapshotScheduleDaoImpl extends GenericDaoBase snapshotSchedules = listActiveBy(sc);
+        List snapshotSchedules = listBy(sc);
         // This will return only one schedule because of a DB uniqueness constraint.
         assert (snapshotSchedules.size() <= 1);
         if (snapshotSchedules.isEmpty()) {
diff --git a/core/src/com/cloud/storage/dao/StoragePoolDao.java b/core/src/com/cloud/storage/dao/StoragePoolDao.java
index 104f282d1f2..06d10c88b8a 100644
--- a/core/src/com/cloud/storage/dao/StoragePoolDao.java
+++ b/core/src/com/cloud/storage/dao/StoragePoolDao.java
@@ -101,5 +101,9 @@ public interface StoragePoolDao extends GenericDao {
 	List searchForStoragePoolDetails(long poolId, String value);
 	
     long countBy(long podId, Status... statuses);
+
+	List findIfDuplicatePoolsExistByUUID(String uuid);
+
+	List listPoolsByStatus(Status status);
     
 }
diff --git a/core/src/com/cloud/storage/dao/StoragePoolDaoImpl.java b/core/src/com/cloud/storage/dao/StoragePoolDaoImpl.java
index 4add45713d9..0c68217f3e8 100644
--- a/core/src/com/cloud/storage/dao/StoragePoolDaoImpl.java
+++ b/core/src/com/cloud/storage/dao/StoragePoolDaoImpl.java
@@ -24,7 +24,6 @@ import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
-import java.util.Set;
 
 import javax.ejb.Local;
 import javax.naming.ConfigurationException;
@@ -55,8 +54,10 @@ public class StoragePoolDaoImpl extends GenericDaoBase  imp
     protected final SearchBuilder HostPathDcSearch;
     protected final SearchBuilder DcPodAnyClusterSearch;
     protected final SearchBuilder DeleteLvmSearch;
+    protected final SearchBuilder StatusSearch;
     protected final GenericSearchBuilder MaintenanceCountSearch;
     
+    
     protected final StoragePoolDetailsDao _detailsDao;
 	
     private final String DetailsSqlPrefix = "SELECT storage_pool.* from storage_pool LEFT JOIN storage_pool_details ON storage_pool.id = storage_pool_details.pool_id WHERE storage_pool.data_center_id = ? and (storage_pool.pod_id = ? or storage_pool.pod_id is null) and (";
@@ -104,6 +105,10 @@ public class StoragePoolDaoImpl extends GenericDaoBase  imp
         HostSearch.and("host", HostSearch.entity().getHostAddress(), SearchCriteria.Op.EQ);
         HostSearch.done();
         
+        StatusSearch = createSearchBuilder();
+        StatusSearch.and("status",StatusSearch.entity().getStatus(),SearchCriteria.Op.EQ);
+        StatusSearch.done();
+        
         HostPathDcPodSearch = createSearchBuilder();
         HostPathDcPodSearch.and("hostAddress", HostPathDcPodSearch.entity().getHostAddress(), SearchCriteria.Op.EQ);
         HostPathDcPodSearch.and("path", HostPathDcPodSearch.entity().getPath(), SearchCriteria.Op.EQ);
@@ -129,7 +134,7 @@ public class StoragePoolDaoImpl extends GenericDaoBase  imp
 	public List findPoolByName(String name) {
 		SearchCriteria sc = NameSearch.create();
         sc.setParameters("name", name);
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
 	}
 
 
@@ -137,7 +142,14 @@ public class StoragePoolDaoImpl extends GenericDaoBase  imp
 	public StoragePoolVO findPoolByUUID(String uuid) {
 		SearchCriteria sc = UUIDSearch.create();
         sc.setParameters("uuid", uuid);
-        return findOneBy(sc);
+        return findOneIncludingRemovedBy(sc);
+	}
+
+	@Override
+	public List findIfDuplicatePoolsExistByUUID(String uuid) {
+		SearchCriteria sc = UUIDSearch.create();
+        sc.setParameters("uuid", uuid);
+        return listBy(sc);
 	}
 
 
@@ -145,7 +157,7 @@ public class StoragePoolDaoImpl extends GenericDaoBase  imp
 	public List listByDataCenterId(long datacenterId) {
 		SearchCriteria sc = DatacenterSearch.create();
         sc.setParameters("datacenterId", datacenterId);
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
 	}
 
 
@@ -169,7 +181,14 @@ public class StoragePoolDaoImpl extends GenericDaoBase  imp
     public List listByStorageHost(String hostFqdnOrIp) {
         SearchCriteria sc = HostSearch.create();
         sc.setParameters("host", hostFqdnOrIp);
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
+    }
+    
+    @Override
+    public List listPoolsByStatus(Status status){
+    	SearchCriteria sc = StatusSearch.create();
+    	sc.setParameters("status", status);
+    	return listBy(sc);
     }
 
     @Override
@@ -180,7 +199,7 @@ public class StoragePoolDaoImpl extends GenericDaoBase  imp
         sc.setParameters("datacenterId", datacenterId);
         sc.setParameters("podId", podId);
         
-        return findOneBy(sc);
+        return findOneIncludingRemovedBy(sc);
     }
 
 	@Override
@@ -191,12 +210,12 @@ public class StoragePoolDaoImpl extends GenericDaoBase  imp
             sc.setParameters("podId", podId);
            
             sc.setParameters("cluster", clusterId);
-            return listActiveBy(sc);
+            return listBy(sc);
 	    } else {
 	        SearchCriteria sc = DcPodAnyClusterSearch.create();
 	        sc.setParameters("datacenterId", datacenterId);
 	        sc.setParameters("podId", podId);
-	        return listActiveBy(sc);
+	        return listBy(sc);
 	    }
 	}
 
@@ -206,7 +225,7 @@ public class StoragePoolDaoImpl extends GenericDaoBase  imp
         sc.setParameters("hostAddress", host);
         sc.setParameters("path", path);
         
-        return listActiveBy(sc);
+        return listIncludingRemovedBy(sc);
 	}
 	
 	public StoragePoolVO listById(Integer id)
@@ -214,7 +233,7 @@ public class StoragePoolDaoImpl extends GenericDaoBase  imp
         SearchCriteria sc = HostSearch.create();
         sc.setParameters("id", id);
         
-        return findOneBy(sc);
+        return findOneIncludingRemovedBy(sc);
 	}
 	
 	@Override @DB
@@ -250,7 +269,6 @@ public class StoragePoolDaoImpl extends GenericDaoBase  imp
 	    if (clusterId != null) {
 	        sql.append("storage_pool.cluster_id = ? OR storage_pool.cluster_id IS NULL) AND (");
 	    }
-	    Set> entries = details.entrySet();
 	    for (Map.Entry detail : details.entrySet()) {
 	        sql.append("((storage_pool_details.name='").append(detail.getKey()).append("') AND (storage_pool_details.value='").append(detail.getValue()).append("')) OR ");
 	    }
@@ -363,7 +381,7 @@ public class StoragePoolDaoImpl extends GenericDaoBase  imp
         sc.setParameters("status", (Object[])statuses);
         sc.setParameters("pool", primaryStorageId);
         
-        List rs = searchAll(sc, null);
+        List rs = searchIncludingRemoved(sc, null);
         if (rs.size() == 0) {
             return 0;
         }
diff --git a/core/src/com/cloud/storage/dao/StoragePoolDetailsDaoImpl.java b/core/src/com/cloud/storage/dao/StoragePoolDetailsDaoImpl.java
index a063a212e0e..97e3b19a529 100644
--- a/core/src/com/cloud/storage/dao/StoragePoolDetailsDaoImpl.java
+++ b/core/src/com/cloud/storage/dao/StoragePoolDetailsDaoImpl.java
@@ -48,7 +48,7 @@ public class StoragePoolDetailsDaoImpl extends GenericDaoBase entry : details.entrySet()) {
             StoragePoolDetailVO detail = new StoragePoolDetailVO(poolId, entry.getKey(), entry.getValue());
             persist(detail);
@@ -61,7 +61,7 @@ public class StoragePoolDetailsDaoImpl extends GenericDaoBase sc = PoolSearch.create();
     	sc.setParameters("pool", poolId);
     	
-    	List details = listActiveBy(sc);
+    	List details = listBy(sc);
     	Map detailsMap = new HashMap();
     	for (StoragePoolDetailVO detail : details) {
     		detailsMap.put(detail.getName(), detail.getValue());
diff --git a/core/src/com/cloud/storage/dao/StoragePoolHostDaoImpl.java b/core/src/com/cloud/storage/dao/StoragePoolHostDaoImpl.java
index 35891cb95ca..6b1dbcb0d8e 100644
--- a/core/src/com/cloud/storage/dao/StoragePoolHostDaoImpl.java
+++ b/core/src/com/cloud/storage/dao/StoragePoolHostDaoImpl.java
@@ -91,14 +91,14 @@ public class StoragePoolHostDaoImpl extends GenericDaoBase listByPoolId(long id) {
 	    SearchCriteria sc = PoolSearch.create();
 	    sc.setParameters("pool_id", id);
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 
 	@Override
 	public List listByHostId(long hostId) {
 	    SearchCriteria sc = HostSearch.create();
 	    sc.setParameters("host_id", hostId);
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -106,7 +106,7 @@ public class StoragePoolHostDaoImpl extends GenericDaoBase sc = PoolHostSearch.create();
 	    sc.setParameters("pool_id", poolId);
 	    sc.setParameters("host_id", hostId);
-	    return findOneBy(sc);
+	    return findOneIncludingRemovedBy(sc);
 	}
 	
 	@Override
diff --git a/core/src/com/cloud/storage/dao/UploadDao.java b/core/src/com/cloud/storage/dao/UploadDao.java
new file mode 100755
index 00000000000..95637029bd2
--- /dev/null
+++ b/core/src/com/cloud/storage/dao/UploadDao.java
@@ -0,0 +1,17 @@
+package com.cloud.storage.dao;
+
+import java.util.List;
+
+import com.cloud.storage.UploadVO;
+import com.cloud.storage.Upload.Status;
+import com.cloud.storage.Upload.Type;
+import com.cloud.utils.db.GenericDao;
+
+public interface UploadDao extends GenericDao {		
+
+	List listByTypeUploadStatus(long typeId, Type type,
+			Status uploadState);
+
+	List listByHostAndUploadStatus(long sserverId, Status uploadInProgress);
+
+}
diff --git a/core/src/com/cloud/storage/dao/UploadDaoImpl.java b/core/src/com/cloud/storage/dao/UploadDaoImpl.java
new file mode 100755
index 00000000000..9d4da07c223
--- /dev/null
+++ b/core/src/com/cloud/storage/dao/UploadDaoImpl.java
@@ -0,0 +1,58 @@
+package com.cloud.storage.dao;
+import java.util.List;
+import javax.ejb.Local;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.storage.UploadVO;
+import com.cloud.storage.Upload.Status;
+import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.SearchBuilder;
+import com.cloud.utils.db.SearchCriteria;
+
+@Local(value={UploadDao.class})
+public class UploadDaoImpl extends GenericDaoBase implements UploadDao {
+	public static final Logger s_logger = Logger.getLogger(UploadDaoImpl.class.getName());
+	protected final SearchBuilder typeUploadStatusSearch;
+	protected final SearchBuilder typeHostAndUploadStatusSearch;
+	
+	protected static final String UPDATE_UPLOAD_INFO =
+		"UPDATE upload SET upload_state = ?, upload_pct= ?, last_updated = ? "
+	+   ", upload_error_str = ?, upload_job_id = ? "
+	+   "WHERE host_id = ? and type_id = ? and type = ?";
+	
+	protected static final String UPLOADS_STATE_DC=
+		"SELECT * FROM upload t, host h where t.host_id = h.id and h.data_center_id=? "
+	+	" and t.type_id=? and t.upload_state = ?" ;
+	
+	
+	public UploadDaoImpl() {
+		typeUploadStatusSearch = createSearchBuilder();
+		typeUploadStatusSearch.and("type_id", typeUploadStatusSearch.entity().getTypeId(), SearchCriteria.Op.EQ);
+		typeUploadStatusSearch.and("upload_state", typeUploadStatusSearch.entity().getUploadState(), SearchCriteria.Op.EQ);
+		typeUploadStatusSearch.and("type", typeUploadStatusSearch.entity().getType(), SearchCriteria.Op.EQ);
+		typeUploadStatusSearch.done();
+		
+		typeHostAndUploadStatusSearch = createSearchBuilder();
+		typeHostAndUploadStatusSearch.and("host_id", typeHostAndUploadStatusSearch.entity().getHostId(), SearchCriteria.Op.EQ);
+		typeHostAndUploadStatusSearch.and("upload_state", typeHostAndUploadStatusSearch.entity().getUploadState(), SearchCriteria.Op.EQ);
+		typeHostAndUploadStatusSearch.done();
+	}
+	
+	@Override
+	public List listByTypeUploadStatus(long typeId, UploadVO.Type type, UploadVO.Status uploadState) {
+		SearchCriteria sc = typeUploadStatusSearch.create();
+		sc.setParameters("type_id", typeId);
+		sc.setParameters("type", type);
+		sc.setParameters("upload_state", uploadState.toString());
+		return listBy(sc);
+	}
+	
+	@Override
+    public List listByHostAndUploadStatus(long sserverId, Status uploadState){	    
+        SearchCriteria sc = typeHostAndUploadStatusSearch.create();
+        sc.setParameters("host_id", sserverId);
+        sc.setParameters("upload_state", uploadState.toString());
+        return listBy(sc);
+	}
+}
\ No newline at end of file
diff --git a/core/src/com/cloud/storage/dao/VMTemplateDaoImpl.java b/core/src/com/cloud/storage/dao/VMTemplateDaoImpl.java
index e929a440328..ea3d24ea32c 100644
--- a/core/src/com/cloud/storage/dao/VMTemplateDaoImpl.java
+++ b/core/src/com/cloud/storage/dao/VMTemplateDaoImpl.java
@@ -74,21 +74,21 @@ public class VMTemplateDaoImpl extends GenericDaoBase implem
     public List listByPublic() {
     	SearchCriteria sc = PublicSearch.create();
     	sc.setParameters("public", 1);
-	    return listActiveBy(sc);
+	    return listBy(sc);
 	}
     
 	@Override
 	public VMTemplateVO findByName(String templateName) {
 		SearchCriteria sc = UniqueNameSearch.create();
 		sc.setParameters("uniqueName", templateName);
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 
 	@Override
 	public VMTemplateVO findByTemplateName(String templateName) {
 		SearchCriteria sc = NameSearch.create();
 		sc.setParameters("name", templateName);
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 
     @Override
@@ -103,14 +103,14 @@ public class VMTemplateDaoImpl extends GenericDaoBase implem
 	public VMTemplateVO findRoutingTemplate() {
 		SearchCriteria sc = UniqueNameSearch.create();
 		sc.setParameters("uniqueName", routerTmpltName);
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 	
 	@Override
 	public VMTemplateVO findConsoleProxyTemplate() {
 		SearchCriteria sc = UniqueNameSearch.create();
 		sc.setParameters("uniqueName", consoleProxyTmpltName);
-		return findOneBy(sc);
+		return findOneIncludingRemovedBy(sc);
 	}
 	
 	@Override
@@ -118,7 +118,7 @@ public class VMTemplateDaoImpl extends GenericDaoBase implem
 		SearchCriteria sc = createSearchCriteria();
 		sc.addAnd("ready", SearchCriteria.Op.EQ, true);
 		sc.addAnd("format", SearchCriteria.Op.NEQ, Storage.ImageFormat.ISO);
-		return listBy(sc);
+		return listIncludingRemovedBy(sc);
 	}
 	
 	@Override
@@ -131,14 +131,14 @@ public class VMTemplateDaoImpl extends GenericDaoBase implem
 			sc.addAnd("accountId", SearchCriteria.Op.EQ, accountId);
 		if (path != null)
 			sc.addAnd("path", SearchCriteria.Op.EQ, path);
-		return listBy(sc);
+		return listIncludingRemovedBy(sc);
 	}
 
 	@Override
 	public List listByAccountId(long accountId) {
         SearchCriteria sc = AccountIdSearch.create();
         sc.setParameters("accountId", accountId);
-        return listActiveBy(sc);
+        return listBy(sc);
 	}
 
 	@Override
@@ -195,7 +195,7 @@ public class VMTemplateDaoImpl extends GenericDaoBase implem
         	String accountId = null;
         	if (account != null) {
         		accountType = account.getType();
-        		accountId = account.getId().toString();
+        		accountId = Long.toString(account.getId());
         	} else {
         		accountType = Account.ACCOUNT_TYPE_ADMIN;
         	}
diff --git a/core/src/com/cloud/storage/dao/VMTemplateHostDao.java b/core/src/com/cloud/storage/dao/VMTemplateHostDao.java
old mode 100644
new mode 100755
index 5a2b26cd5c2..2d21481d69d
--- a/core/src/com/cloud/storage/dao/VMTemplateHostDao.java
+++ b/core/src/com/cloud/storage/dao/VMTemplateHostDao.java
@@ -18,9 +18,11 @@
 
 package com.cloud.storage.dao;
 
+import java.util.Date;
 import java.util.List;
 
 import com.cloud.storage.VMTemplateHostVO;
+import com.cloud.storage.VMTemplateStorageResourceAssoc.Status;
 import com.cloud.utils.db.GenericDao;
 
 public interface VMTemplateHostDao extends GenericDao {
@@ -40,7 +42,7 @@ public interface VMTemplateHostDao extends GenericDao {
 
     List listByTemplatePool(long templateId, long poolId);
 
-    void update(VMTemplateHostVO instance);
+    void update(VMTemplateHostVO instance);    
 
     List listByTemplateStatus(long templateId, VMTemplateHostVO.Status downloadState);
 
@@ -53,4 +55,5 @@ public interface VMTemplateHostDao extends GenericDao {
     List listDestroyed(long hostId);
 
     boolean templateAvailable(long templateId, long hostId);
+	
 }
diff --git a/core/src/com/cloud/storage/dao/VMTemplateHostDaoImpl.java b/core/src/com/cloud/storage/dao/VMTemplateHostDaoImpl.java
old mode 100644
new mode 100755
index 48f3d939624..83d2a15a7de
--- a/core/src/com/cloud/storage/dao/VMTemplateHostDaoImpl.java
+++ b/core/src/com/cloud/storage/dao/VMTemplateHostDaoImpl.java
@@ -49,13 +49,14 @@ public class VMTemplateHostDaoImpl extends GenericDaoBase PoolTemplateSearch;
 	protected final SearchBuilder HostTemplatePoolSearch;
 	protected final SearchBuilder TemplateStatusSearch;
-	protected final SearchBuilder TemplateStatesSearch;
-	
-	protected static final String UPDATE_TEMPLATE_HOST_REF =
-		"UPDATE template_host_ref SET download_state = ?, download_pct= ?, last_updated = ? "
-	+   ", error_str = ?, local_path = ?, job_id = ? "
-	+   "WHERE host_id = ? and template_id = ?";
-	
+	protected final SearchBuilder TemplateStatesSearch;
+	
+	
+	protected static final String UPDATE_TEMPLATE_HOST_REF =
+		"UPDATE type_host_ref SET download_state = ?, download_pct= ?, last_updated = ? "
+	+   ", error_str = ?, local_path = ?, job_id = ? "
+	+   "WHERE host_id = ? and type_id = ?";
+			
 	protected static final String DOWNLOADS_STATE_DC=
 		"SELECT * FROM template_host_ref t, host h where t.host_id = h.id and h.data_center_id=? "
 	+	" and t.template_id=? and t.download_state = ?" ;
@@ -67,7 +68,9 @@ public class VMTemplateHostDaoImpl extends GenericDaoBase listByHostId(long id) {
 	    SearchCriteria sc = HostSearch.create();
 	    sc.setParameters("host_id", id);
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -143,14 +147,14 @@ public class VMTemplateHostDaoImpl extends GenericDaoBase sc = TemplateSearch.create();
 	    sc.setParameters("template_id", templateId);
 	    sc.setParameters("destroyed", false);
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 	
 	@Override
 	public List listByOnlyTemplateId(long templateId) {
 	    SearchCriteria sc = TemplateSearch.create();
 	    sc.setParameters("template_id", templateId);	    
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -158,15 +162,15 @@ public class VMTemplateHostDaoImpl extends GenericDaoBase sc = HostTemplateSearch.create();
 	    sc.setParameters("host_id", hostId);
 	    sc.setParameters("template_id", templateId);
-	    return findOneBy(sc);
+	    return findOneIncludingRemovedBy(sc);
 	}
-	
+
 	@Override
 	public List listByTemplateStatus(long templateId, VMTemplateHostVO.Status downloadState) {
 		SearchCriteria sc = TemplateStatusSearch.create();
 		sc.setParameters("template_id", templateId);
 		sc.setParameters("download_state", downloadState.toString());
-		return listBy(sc);
+		return listIncludingRemovedBy(sc);
 	}
 	
 	@Override
@@ -252,7 +256,7 @@ public class VMTemplateHostDaoImpl extends GenericDaoBase sc = HostTemplateSearch.create();
 	    sc.setParameters("host_id", hostId);
 	    sc.setParameters("template_id", templateId);
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -268,7 +272,7 @@ public class VMTemplateHostDaoImpl extends GenericDaoBase sc = PoolTemplateSearch.create();
 	    sc.setParameters("pool_id", poolId);
 	    sc.setParameters("template_id", templateId);
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 	
 	@Override
@@ -276,7 +280,7 @@ public class VMTemplateHostDaoImpl extends GenericDaoBase sc = HostDestroyedSearch.create();
 		sc.setParameters("host_id", hostId);
 		sc.setParameters("destroyed", true);
-		return listBy(sc);
+		return listIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -285,7 +289,7 @@ public class VMTemplateHostDaoImpl extends GenericDaoBase listByPoolId(long id) {
 	    SearchCriteria sc = PoolSearch.create();
 	    sc.setParameters("pool_id", id);
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 
 	@Override
 	public List listByTemplateId(long templateId) {
 	    SearchCriteria sc = TemplateSearch.create();
 	    sc.setParameters("template_id", templateId);
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -115,7 +115,7 @@ public class VMTemplatePoolDaoImpl extends GenericDaoBase sc = PoolTemplateSearch.create();
 	    sc.setParameters("pool_id", hostId);
 	    sc.setParameters("template_id", templateId);
-	    return findOneBy(sc);
+	    return findOneIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -123,7 +123,7 @@ public class VMTemplatePoolDaoImpl extends GenericDaoBase sc = TemplateStatusSearch.create();
 		sc.setParameters("template_id", templateId);
 		sc.setParameters("download_state", downloadState.toString());
-		return listBy(sc);
+		return listIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -132,7 +132,7 @@ public class VMTemplatePoolDaoImpl extends GenericDaoBase listByZoneId(long id) {
 	    SearchCriteria sc = ZoneSearch.create();
 	    sc.setParameters("zone_id", id);
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 
 	@Override
 	public List listByTemplateId(long templateId) {
 	    SearchCriteria sc = TemplateSearch.create();
 	    sc.setParameters("template_id", templateId);
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -73,7 +73,7 @@ public class VMTemplateZoneDaoImpl extends GenericDaoBase sc = ZoneTemplateSearch.create();
 	    sc.setParameters("zone_id", zoneId);
 	    sc.setParameters("template_id", templateId);
-	    return findOneBy(sc);
+	    return findOneIncludingRemovedBy(sc);
 	}
 
 	@Override
@@ -81,7 +81,7 @@ public class VMTemplateZoneDaoImpl extends GenericDaoBase sc = ZoneTemplateSearch.create();
 	    sc.setParameters("zone_id", zoneId);
 	    sc.setParameters("template_id", templateId);
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}	
 
 }
diff --git a/core/src/com/cloud/storage/dao/VolumeDao.java b/core/src/com/cloud/storage/dao/VolumeDao.java
index 39979744aab..3bd43faf9bc 100755
--- a/core/src/com/cloud/storage/dao/VolumeDao.java
+++ b/core/src/com/cloud/storage/dao/VolumeDao.java
@@ -46,4 +46,5 @@ public interface VolumeDao extends GenericDao {
     List listRemovedButNotDestroyed();
     List findCreatedByInstance(long id);
     List findByPoolId(long poolId);
+	List findByInstanceAndDeviceId(long instanceId, long deviceId);
 }
diff --git a/core/src/com/cloud/storage/dao/VolumeDaoImpl.java b/core/src/com/cloud/storage/dao/VolumeDaoImpl.java
index 976c6445483..dc82102a1a5 100755
--- a/core/src/com/cloud/storage/dao/VolumeDaoImpl.java
+++ b/core/src/com/cloud/storage/dao/VolumeDaoImpl.java
@@ -30,18 +30,18 @@ import org.apache.log4j.Logger;
 
 import com.cloud.async.AsyncInstanceCreateStatus;
 import com.cloud.storage.Volume;
-import com.cloud.storage.VolumeVO;
 import com.cloud.storage.Volume.MirrorState;
 import com.cloud.storage.Volume.VolumeType;
+import com.cloud.storage.VolumeVO;
 import com.cloud.utils.Pair;
 import com.cloud.utils.db.DB;
 import com.cloud.utils.db.GenericDaoBase;
 import com.cloud.utils.db.GenericSearchBuilder;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
-import com.cloud.utils.db.Transaction;
 import com.cloud.utils.db.SearchCriteria.Func;
 import com.cloud.utils.db.SearchCriteria.Op;
+import com.cloud.utils.db.Transaction;
 import com.cloud.utils.exception.CloudRuntimeException;
 
 @Local(value=VolumeDao.class) @DB(txn=false)
@@ -61,6 +61,7 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
     protected final GenericSearchBuilder ActiveTemplateSearch;
     protected final SearchBuilder RemovedButNotDestroyedSearch;
     protected final SearchBuilder PoolIdSearch;
+    protected final SearchBuilder InstanceAndDeviceIdSearch;
     
     protected static final String SELECT_VM_SQL = "SELECT DISTINCT instance_id from volumes v where v.host_id = ? and v.mirror_state = ?";
     protected static final String SELECT_VM_ID_SQL = "SELECT DISTINCT instance_id from volumes v where v.host_id = ?";
@@ -70,7 +71,7 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
         SearchCriteria sc = RemovedButNotDestroyedSearch.create();
         sc.setParameters("destroyed", false);
         
-        return searchAll(sc, null, null, false);
+        return searchIncludingRemoved(sc, null, null, false);
     }
     
     @Override @DB
@@ -100,7 +101,7 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
     	SearchCriteria sc = DetachedAccountIdSearch.create();
     	sc.setParameters("accountId", accountId);
     	sc.setParameters("destroyed", false);
-    	return listActiveBy(sc);
+    	return listBy(sc);
     }
     
     @Override
@@ -108,21 +109,29 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
         SearchCriteria sc = AccountIdSearch.create();
         sc.setParameters("accountId", accountId);
         sc.setParameters("destroyed", false);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
     public List findByInstance(long id) {
         SearchCriteria sc = InstanceIdSearch.create();
         sc.setParameters("instanceId", id);
-	    return listActiveBy(sc);
+	    return listBy(sc);
 	}
+   
+    @Override
+    public List findByInstanceAndDeviceId(long instanceId, long deviceId){
+    	SearchCriteria sc = InstanceAndDeviceIdSearch.create();
+    	sc.setParameters("instanceId", instanceId);
+    	sc.setParameters("deviceId", deviceId);
+    	return listBy(sc);
+    }
     
     @Override
     public List findByPoolId(long poolId) {
         SearchCriteria sc = PoolIdSearch.create();
         sc.setParameters("poolId", poolId);
-	    return listActiveBy(sc);
+	    return listBy(sc);
 	}
     
     @Override 
@@ -131,7 +140,7 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
         sc.setParameters("instanceId", id);
         sc.setParameters("status", AsyncInstanceCreateStatus.Created);
         sc.setParameters("destroyed", false);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
 	@Override
@@ -139,7 +148,7 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
         SearchCriteria sc = InstanceAndTypeSearch.create();
         sc.setParameters("instanceId", id);
         sc.setParameters("vType", vType.toString());
-	    return listActiveBy(sc);
+	    return listBy(sc);
 	}
 	
 	@Override
@@ -147,14 +156,14 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
 		SearchCriteria sc = InstanceIdDestroyedSearch.create();
 		sc.setParameters("instanceId", vmId);
 		sc.setParameters("destroyed", true);
-		return listBy(sc);
+		return listIncludingRemovedBy(sc);
 	}
 	
 	@Override
 	public List findByDetachedDestroyed() {
 		SearchCriteria sc = DetachedDestroyedSearch.create();
 		sc.setParameters("destroyed", true);
-		return listActiveBy(sc);
+		return listBy(sc);
 	}
 	
 	@Override
@@ -165,7 +174,7 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
         sc.setParameters("destroyed", false);
         sc.setParameters("status", AsyncInstanceCreateStatus.Created);
         
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
 	}
 	
 	@Override
@@ -174,7 +183,7 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
 		sc.setParameters("template", templateId);
 		sc.setParameters("zone", zoneId);
 		
-		return listBy(sc);
+		return listIncludingRemovedBy(sc);
 	}
 
 	@Override @DB
@@ -206,7 +215,7 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
 		SearchCriteria sc = MirrorSearch.create();
         sc.setParameters("mirrorState", MirrorState.ACTIVE.toString());
 
-	    return listBy(sc);
+	    return listIncludingRemovedBy(sc);
 	}
 	
 	@Override
@@ -215,7 +224,7 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
 	    sc.setParameters("template", templateId);
 	    sc.setParameters("pool", poolId);
 	    
-	    List results = searchAll(sc, null);
+	    List results = searchIncludingRemoved(sc, null);
 	    assert results.size() > 0 : "How can this return a size of " + results.size();
 	    
 	    return results.get(0) > 0;
@@ -225,7 +234,7 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
     public void deleteVolumesByInstance(long instanceId) {
         SearchCriteria sc = InstanceIdSearch.create();
         sc.setParameters("instanceId", instanceId);
-        delete(sc);
+        expunge(sc);
     }
     
     @Override
@@ -234,6 +243,7 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
     	volume.setInstanceId(vmId);
     	volume.setDeviceId(deviceId);
     	volume.setUpdated(new Date());
+    	volume.setAttached(new Date());
     	update(volumeId, volume);
     }
     
@@ -243,6 +253,7 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
     	volume.setInstanceId(null);
         volume.setDeviceId(null);
     	volume.setUpdated(new Date());
+    	volume.setAttached(null);
     	update(volumeId, volume);
     }
     
@@ -302,6 +313,11 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
         InstanceIdSearch.and("instanceId", InstanceIdSearch.entity().getInstanceId(), SearchCriteria.Op.EQ);
         InstanceIdSearch.done();
 
+        InstanceAndDeviceIdSearch = createSearchBuilder();
+        InstanceAndDeviceIdSearch.and("instanceId", InstanceAndDeviceIdSearch.entity().getInstanceId(), SearchCriteria.Op.EQ);
+        InstanceAndDeviceIdSearch.and("deviceId", InstanceAndDeviceIdSearch.entity().getDeviceId(), SearchCriteria.Op.EQ);
+        InstanceAndDeviceIdSearch.done();
+        
         PoolIdSearch = createSearchBuilder();
         PoolIdSearch.and("poolId", PoolIdSearch.entity().getPoolId(), SearchCriteria.Op.EQ);
         PoolIdSearch.done();
@@ -343,7 +359,7 @@ public class VolumeDaoImpl extends GenericDaoBase implements Vol
 	public Pair getCountAndTotalByPool(long poolId) {
         SearchCriteria sc = TotalSizeByPoolSearch.create();
         sc.setParameters("poolId", poolId);
-        List results = searchAll(sc, null);
+        List results = searchIncludingRemoved(sc, null);
         SumCount sumCount = results.get(0);
         return new Pair(sumCount.count, sumCount.sum);
 	}
diff --git a/core/src/com/cloud/storage/resource/DummySecondaryStorageResource.java b/core/src/com/cloud/storage/resource/DummySecondaryStorageResource.java
index b15a538de78..611dc1c5450 100644
--- a/core/src/com/cloud/storage/resource/DummySecondaryStorageResource.java
+++ b/core/src/com/cloud/storage/resource/DummySecondaryStorageResource.java
@@ -42,7 +42,7 @@ import com.cloud.host.Host;
 import com.cloud.host.Host.Type;
 import com.cloud.resource.ServerResource;
 import com.cloud.resource.ServerResourceBase;
-import com.cloud.storage.Volume;
+import com.cloud.storage.Storage;
 import com.cloud.storage.Storage.StoragePoolType;
 import com.cloud.storage.template.TemplateInfo;
 
@@ -102,7 +102,7 @@ public class DummySecondaryStorageResource extends ServerResourceBase implements
         	StoragePoolType.NetworkFilesystem, 1024*1024*1024*100L,
         	new HashMap());
         
-        cmd.setResourceType(Volume.StorageResourceType.SECONDARY_STORAGE);
+        cmd.setResourceType(Storage.StorageResourceType.SECONDARY_STORAGE);
         cmd.setIqn(null);
         cmd.setNfsShare(_guid);
         
diff --git a/core/src/com/cloud/storage/resource/NfsSecondaryStorageResource.java b/core/src/com/cloud/storage/resource/NfsSecondaryStorageResource.java
old mode 100644
new mode 100755
index dc00d8983d0..b99d867978f
--- a/core/src/com/cloud/storage/resource/NfsSecondaryStorageResource.java
+++ b/core/src/com/cloud/storage/resource/NfsSecondaryStorageResource.java
@@ -48,16 +48,19 @@ import com.cloud.agent.api.SecStorageFirewallCfgCommand.PortConfig;
 import com.cloud.agent.api.storage.DeleteTemplateCommand;
 import com.cloud.agent.api.storage.DownloadCommand;
 import com.cloud.agent.api.storage.DownloadProgressCommand;
+import com.cloud.agent.api.storage.UploadCommand;
 import com.cloud.host.Host;
 import com.cloud.host.Host.Type;
 import com.cloud.resource.ServerResource;
 import com.cloud.resource.ServerResourceBase;
+import com.cloud.storage.Storage;
 import com.cloud.storage.StorageLayer;
-import com.cloud.storage.Volume;
 import com.cloud.storage.Storage.StoragePoolType;
 import com.cloud.storage.template.DownloadManager;
 import com.cloud.storage.template.DownloadManagerImpl;
 import com.cloud.storage.template.TemplateInfo;
+import com.cloud.storage.template.UploadManager;
+import com.cloud.storage.template.UploadManagerImpl;
 import com.cloud.utils.NumbersUtil;
 import com.cloud.utils.component.ComponentLocator;
 import com.cloud.utils.exception.CloudRuntimeException;
@@ -85,6 +88,7 @@ public class NfsSecondaryStorageResource extends ServerResourceBase implements S
     Random _rand = new Random(System.currentTimeMillis());
     
     DownloadManager _dlMgr;
+    UploadManager _upldMgr;
 	private String _configSslScr;
 	private String _configAuthScr;
 	private String _publicIp;
@@ -111,6 +115,8 @@ public class NfsSecondaryStorageResource extends ServerResourceBase implements S
             return _dlMgr.handleDownloadCommand((DownloadProgressCommand)cmd);
         } else if (cmd instanceof DownloadCommand) {
             return _dlMgr.handleDownloadCommand((DownloadCommand)cmd);
+        }else if (cmd instanceof UploadCommand) {        	
+            return _upldMgr.handleUploadCommand((UploadCommand)cmd);
         } else if (cmd instanceof GetStorageStatsCommand) {
         	return execute((GetStorageStatsCommand)cmd);
         } else if (cmd instanceof CheckHealthCommand) {
@@ -413,6 +419,8 @@ public class NfsSecondaryStorageResource extends ServerResourceBase implements S
             _params.put(StorageLayer.InstanceConfigKey, _storage);
             _dlMgr = new DownloadManagerImpl();
             _dlMgr.configure("DownloadManager", _params);
+            _upldMgr = new UploadManagerImpl();
+            _upldMgr.configure("UploadManager", params);
         } catch (ConfigurationException e) {
             s_logger.warn("Caught problem while configuring DownloadManager", e);
             return false;
@@ -600,7 +608,7 @@ public class NfsSecondaryStorageResource extends ServerResourceBase implements S
         
         final StartupStorageCommand cmd = new StartupStorageCommand(_parent, StoragePoolType.NetworkFilesystem, getTotalSize(), new HashMap());
         
-        cmd.setResourceType(Volume.StorageResourceType.SECONDARY_STORAGE);
+        cmd.setResourceType(Storage.StorageResourceType.SECONDARY_STORAGE);
         cmd.setIqn(null);
         
         fillNetworkInformation(cmd);
diff --git a/core/src/com/cloud/storage/template/DownloadManagerImpl.java b/core/src/com/cloud/storage/template/DownloadManagerImpl.java
index 2ec957a6d8a..2d37f630fe3 100644
--- a/core/src/com/cloud/storage/template/DownloadManagerImpl.java
+++ b/core/src/com/cloud/storage/template/DownloadManagerImpl.java
@@ -303,7 +303,7 @@ public class DownloadManagerImpl implements DownloadManager {
         }
 
         // add options common to ISO and template
-        String extension = dnld.getFormat().toString().toLowerCase();
+        String extension = dnld.getFormat().getFileExtension();
         String templateName = "";
         if( extension.equals("iso")) {
             templateName = jobs.get(jobId).getTmpltName().trim().replace(" ", "_");
@@ -353,6 +353,7 @@ public class DownloadManagerImpl implements DownloadManager {
 			try {
 				info = processor.process(templatePath, null, templateName);
 			} catch (InternalErrorException e) {
+				s_logger.error("Template process exception ", e);
 				return e.toString();
 			}
             if (info != null) {
@@ -781,6 +782,11 @@ public class DownloadManagerImpl implements DownloadManager {
         processor = new QCOW2Processor();
         processor.configure("QCOW2 Processor", params);
         processors.add(processor);
+
+        processor = new VmdkProcessor();
+        processor.configure("VMDK Processor", params);
+        processors.add(processor);
+        
         // Add more processors here.
         threadPool = Executors.newFixedThreadPool(numInstallThreads);
         return true;
diff --git a/core/src/com/cloud/storage/template/FtpTemplateUploader.java b/core/src/com/cloud/storage/template/FtpTemplateUploader.java
new file mode 100755
index 00000000000..aafb294a042
--- /dev/null
+++ b/core/src/com/cloud/storage/template/FtpTemplateUploader.java
@@ -0,0 +1,216 @@
+package com.cloud.storage.template;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.net.URLConnection;
+import java.util.Date;
+
+import org.apache.log4j.Logger;
+
+
+public class FtpTemplateUploader implements TemplateUploader {
+	
+	public static final Logger s_logger = Logger.getLogger(FtpTemplateUploader.class.getName());
+	public TemplateUploader.Status status = TemplateUploader.Status.NOT_STARTED;
+	public String errorString = "";
+	public long totalBytes = 0;
+	public long entitySizeinBytes;
+	private String sourcePath;
+	private String ftpUrl;	
+	private UploadCompleteCallback completionCallback;
+	private boolean resume;
+    private BufferedInputStream inputStream = null;
+    private BufferedOutputStream outputStream = null;
+	private static final int CHUNK_SIZE = 1024*1024; //1M
+	
+	public FtpTemplateUploader(String sourcePath, String url, UploadCompleteCallback callback, long entitySizeinBytes){
+		
+		this.sourcePath = sourcePath;
+		this.ftpUrl = url;
+		this.completionCallback = callback;
+		this.entitySizeinBytes = entitySizeinBytes;
+		
+	}
+	
+	public long upload(UploadCompleteCallback callback )
+		   {
+		
+				switch (status) {
+				case ABORTED:
+				case UNRECOVERABLE_ERROR:
+				case UPLOAD_FINISHED:
+					return 0;
+				default:
+		
+				}
+				
+	             Date start = new Date();
+				 
+		         StringBuffer sb = new StringBuffer(ftpUrl);
+		         // check for authentication else assume its anonymous access.
+		        /* if (user != null && password != null)
+		         {
+		            sb.append( user );
+		            sb.append( ':' );
+		            sb.append( password );
+		            sb.append( '@' );
+		         }*/		         
+		         /*
+		          * type ==> a=ASCII mode, i=image (binary) mode, d= file directory
+		          * listing
+		          */
+		         sb.append( ";type=i" );
+
+		         try
+		         {
+		            URL url = new URL( sb.toString() );
+		            URLConnection urlc = url.openConnection();
+		            File sourceFile = new File(sourcePath);
+		            entitySizeinBytes = sourceFile.length();
+
+		            outputStream = new BufferedOutputStream( urlc.getOutputStream() );
+		            inputStream = new BufferedInputStream( new FileInputStream(sourceFile) );            
+
+		            status = TemplateUploader.Status.IN_PROGRESS;
+
+		            int bytes = 0;
+		            byte[] block = new byte[CHUNK_SIZE];
+		            boolean done=false;
+		            while (!done && status != Status.ABORTED ) {
+		            	if ( (bytes = inputStream.read(block, 0, CHUNK_SIZE)) > -1) {
+		            		outputStream.write(block,0, bytes);		            			            				            			            		
+		            		totalBytes += bytes;
+		            	} else {
+		            		done = true;
+		            	}
+		            }		            
+		            status = TemplateUploader.Status.UPLOAD_FINISHED;		            
+		            return totalBytes;
+		         } catch (MalformedURLException e) {
+		        	status = TemplateUploader.Status.UNRECOVERABLE_ERROR;
+		 			errorString = e.getMessage();
+		 			s_logger.error(errorString);
+				} catch (IOException e) {
+					status = TemplateUploader.Status.UNRECOVERABLE_ERROR;
+		 			errorString = e.getMessage();
+		 			s_logger.error(errorString);
+				}
+		         finally
+		         {
+		           try		         
+	               {
+		            if (inputStream != null){		               
+		            	inputStream.close();
+		            }
+		            if (outputStream != null){		               
+		                  outputStream.close();
+		            }
+	               }catch (IOException ioe){
+	            	   s_logger.error(" Caught exception while closing the resources" ); 		                  
+	               }
+				   if (callback != null) {
+					   callback.uploadComplete(status);
+				   }
+		         }
+
+				return 0;
+		   }
+
+	@Override
+	public void run() {
+		try {
+			upload(completionCallback);
+		} catch (Throwable t) {
+			s_logger.warn("Caught exception during upload "+ t.getMessage(), t);
+			errorString = "Failed to install: " + t.getMessage();
+			status = TemplateUploader.Status.UNRECOVERABLE_ERROR;
+		}
+		
+	}
+
+	@Override
+	public Status getStatus() {
+		return status;
+	}
+
+	@Override
+	public String getUploadError() {
+		return errorString;
+	}
+
+	@Override
+	public String getUploadLocalPath() {
+		return sourcePath;
+	}
+
+	@Override
+	public int getUploadPercent() {
+		if (entitySizeinBytes == 0) {
+			return 0;
+		}		
+		return (int)(100.0*totalBytes/entitySizeinBytes);
+	}
+
+	@Override
+	public long getUploadTime() {
+		// TODO
+		return 0;
+	}
+
+	@Override
+	public long getUploadedBytes() {
+		return totalBytes;
+	}
+
+	@Override
+	public void setResume(boolean resume) {
+		this.resume = resume;
+		
+	}
+
+	@Override
+	public void setStatus(Status status) {
+		this.status = status;		
+	}
+
+	@Override
+	public void setUploadError(String string) {
+		errorString = string;		
+	}
+
+	@Override
+	public boolean stopUpload() {
+		switch (getStatus()) {
+		case IN_PROGRESS:
+			try {
+				if(outputStream != null) {
+					outputStream.close();
+				}
+				if (inputStream != null){				
+					inputStream.close();					
+				}
+			} catch (IOException e) {
+				s_logger.error(" Caught exception while closing the resources" );
+			}
+			status = TemplateUploader.Status.ABORTED;
+			return true;
+		case UNKNOWN:
+		case NOT_STARTED:
+		case RECOVERABLE_ERROR:
+		case UNRECOVERABLE_ERROR:
+		case ABORTED:
+			status = TemplateUploader.Status.ABORTED;
+		case UPLOAD_FINISHED:
+			return true;
+
+		default:
+			return true;
+		}
+	}	
+
+}
diff --git a/core/src/com/cloud/storage/template/TemplateUploader.java b/core/src/com/cloud/storage/template/TemplateUploader.java
new file mode 100755
index 00000000000..90ac2dee953
--- /dev/null
+++ b/core/src/com/cloud/storage/template/TemplateUploader.java
@@ -0,0 +1,74 @@
+package com.cloud.storage.template;
+
+import com.cloud.storage.template.TemplateUploader.UploadCompleteCallback;
+import com.cloud.storage.template.TemplateUploader.Status;
+
+public interface TemplateUploader extends Runnable{
+
+	/**
+	 * Callback used to notify completion of upload
+	 * @author nitin
+	 *
+	 */
+	public interface UploadCompleteCallback {
+		void uploadComplete( Status status);
+
+	}
+
+	public static enum Status  {UNKNOWN, NOT_STARTED, IN_PROGRESS, ABORTED, UNRECOVERABLE_ERROR, RECOVERABLE_ERROR, UPLOAD_FINISHED, POST_UPLOAD_FINISHED}
+
+	
+	/**
+	 * Initiate upload
+	 * @param callback completion callback to be called after upload is complete
+	 * @return bytes uploaded
+	 */
+	public long upload(UploadCompleteCallback callback);
+	
+	/**
+	 * @return
+	 */
+	public boolean stopUpload();
+	
+	/**
+	 * @return percent of file uploaded
+	 */
+	public int getUploadPercent();
+
+	/**
+	 * Get the status of the upload
+	 * @return status of upload
+	 */
+	public TemplateUploader.Status getStatus();
+
+
+	/**
+	 * Get time taken to upload so far
+	 * @return time in seconds taken to upload
+	 */
+	public long getUploadTime();
+
+	/**
+	 * Get bytes uploaded
+	 * @return bytes uploaded so far
+	 */
+	public long getUploadedBytes();
+
+	/**
+	 * Get the error if any
+	 * @return error string if any
+	 */
+	public String getUploadError();
+
+	/** Get local path of the uploaded file
+	 * @return local path of the file uploaded
+	 */
+	public String getUploadLocalPath();
+
+	public void setStatus(TemplateUploader.Status status);
+
+	public void setUploadError(String string);
+
+	public void setResume(boolean resume);	
+
+}
diff --git a/core/src/com/cloud/storage/template/UploadManager.java b/core/src/com/cloud/storage/template/UploadManager.java
new file mode 100644
index 00000000000..df096235c85
--- /dev/null
+++ b/core/src/com/cloud/storage/template/UploadManager.java
@@ -0,0 +1,64 @@
+package com.cloud.storage.template;
+
+import com.cloud.agent.api.storage.UploadAnswer;
+import com.cloud.agent.api.storage.UploadCommand;
+import com.cloud.storage.StorageResource;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.Upload.Status;
+import com.cloud.utils.component.Manager;
+
+public interface UploadManager extends Manager {
+
+
+	/**
+	 * Get the status of a upload job
+	 * @param jobId job Id
+	 * @return status of the upload job
+	 */
+	public TemplateUploader.Status getUploadStatus(String jobId);
+	
+	/**
+	 * Get the status of a upload job
+	 * @param jobId job Id
+	 * @return status of the upload job
+	 */
+	public Status getUploadStatus2(String jobId);
+
+	/**
+	 * Get the upload percent of a upload job
+	 * @param jobId job Id
+	 * @return
+	 */
+	public int getUploadPct(String jobId);
+
+	/**
+	 * Get the upload error if any
+	 * @param jobId job Id
+	 * @return
+	 */
+	public String getUploadError(String jobId);
+
+	/**
+	 * Get the local path for the upload
+	 * @param jobId job Id
+	 * @return
+	public String getUploadLocalPath(String jobId);
+     */
+	
+	/** Handle upload commands from the management server
+	 * @param cmd cmd from server
+	 * @return answer representing status of upload.
+	 */
+	public UploadAnswer handleUploadCommand(UploadCommand cmd);		
+	
+	public String setRootDir(String rootDir, StorageResource storage);
+    
+    public String getPublicTemplateRepo();
+
+
+	String uploadPublicTemplate(long id, String url, String name,
+			ImageFormat format, Long accountId, String descr,
+			String cksum, String installPathPrefix, String user,
+			String password, long maxTemplateSizeInBytes);
+	
+}
diff --git a/core/src/com/cloud/storage/template/UploadManagerImpl.java b/core/src/com/cloud/storage/template/UploadManagerImpl.java
new file mode 100755
index 00000000000..b6c8ffd128d
--- /dev/null
+++ b/core/src/com/cloud/storage/template/UploadManagerImpl.java
@@ -0,0 +1,594 @@
+package com.cloud.storage.template;
+
+import java.io.File;
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+
+import javax.naming.ConfigurationException;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.storage.UploadAnswer;
+import com.cloud.agent.api.storage.UploadProgressCommand;
+import com.cloud.agent.api.storage.UploadCommand;
+import com.cloud.agent.api.storage.UploadAnswer;
+import com.cloud.agent.api.storage.UploadCommand;
+import com.cloud.storage.StorageLayer;
+import com.cloud.storage.StorageResource;
+import com.cloud.storage.UploadVO;
+import com.cloud.storage.VMTemplateHostVO;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.template.TemplateUploader.UploadCompleteCallback;
+import com.cloud.storage.template.TemplateUploader.Status;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.UUID;
+import com.cloud.utils.component.Adapters;
+import com.cloud.utils.component.ComponentLocator;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+
+public class UploadManagerImpl implements UploadManager {
+
+
+   public class Completion implements UploadCompleteCallback {
+        private final String jobId;
+
+        public Completion(String jobId) {
+            this.jobId = jobId;
+        }
+
+        @Override
+        public void uploadComplete(Status status) {
+            setUploadStatus(jobId, status);
+        }
+    }
+   
+   private static class UploadJob {
+       private final TemplateUploader tu;
+       private final String jobId;
+       private final String name;
+       private final ImageFormat format;
+       private String tmpltPath;
+       private String description;
+       private String checksum;
+       private Long accountId;
+       private String installPathPrefix;
+       private long templatesize;
+       private long id;
+
+       public UploadJob(TemplateUploader tu, String jobId, long id, String name, ImageFormat format, boolean hvm, Long accountId, String descr, String cksum, String installPathPrefix) {
+           super();
+           this.tu = tu;
+           this.jobId = jobId;
+           this.name = name;
+           this.format = format;
+           this.accountId = accountId;
+           this.description = descr;
+           this.checksum = cksum;
+           this.installPathPrefix = installPathPrefix;
+           this.templatesize = 0;
+           this.id = id;
+       }
+
+       public TemplateUploader getTd() {
+           return tu;
+       }
+
+       public String getDescription() {
+           return description;
+       }
+
+       public String getChecksum() {
+           return checksum;
+       }
+
+       public UploadJob(TemplateUploader td, String jobId, UploadCommand cmd) {
+           this.tu = td;
+           this.jobId = jobId;
+           this.name = cmd.getName();
+           this.format = cmd.getFormat();           
+       }
+
+       public TemplateUploader getTemplateUploader() {
+           return tu;
+       }
+
+       public String getJobId() {
+           return jobId;
+       }
+
+       public String getTmpltName() {
+           return name;
+       }
+
+       public ImageFormat getFormat() {
+           return format;
+       }
+
+       public Long getAccountId() {
+           return accountId;
+       }
+
+       public long getId() {
+           return id;
+       }
+
+       public void setTmpltPath(String tmpltPath) {
+           this.tmpltPath = tmpltPath;
+       }
+
+       public String getTmpltPath() {
+           return tmpltPath;
+       }
+
+       public String getInstallPathPrefix() {
+           return installPathPrefix;
+       }
+
+       public void cleanup() {
+           if (tu != null) {
+               String upldPath = tu.getUploadLocalPath();
+               if (upldPath != null) {
+                   File f = new File(upldPath);
+                   f.delete();
+               }
+           }
+       }
+
+       public void setTemplatesize(long templatesize) {
+           this.templatesize = templatesize;
+       }
+
+       public long getTemplatesize() {
+           return templatesize;
+       }
+   }
+   public static final Logger s_logger = Logger.getLogger(UploadManagerImpl.class);
+   private ExecutorService threadPool;
+   private final Map jobs = new ConcurrentHashMap();
+	private String parentDir;
+	private Adapters _processors;
+	private String publicTemplateRepo;
+	private StorageLayer _storage;
+	private int installTimeoutPerGig;
+	private boolean _sslCopy;
+	private String _name;
+	private boolean hvm;
+   
+	@Override
+	public String uploadPublicTemplate(long id, String url, String name,
+			ImageFormat format, Long accountId, String descr,
+			String cksum, String installPathPrefix, String userName,
+			String passwd, long templateSizeInBytes) {		
+		
+        UUID uuid = new UUID();
+        String jobId = uuid.toString();
+
+        String completePath = parentDir + File.separator + installPathPrefix;
+        s_logger.debug("Starting upload from " + completePath);
+        
+        URI uri;
+		try {
+		    uri = new URI(url);
+		} catch (URISyntaxException e) {
+		    s_logger.error("URI is incorrect: " + url);
+		    throw new CloudRuntimeException("URI is incorrect: " + url);
+		}
+		TemplateUploader tu;
+		if ((uri != null) && (uri.getScheme() != null)) {
+		    if (uri.getScheme().equalsIgnoreCase("ftp")) {
+		        tu = new FtpTemplateUploader(completePath, url, new Completion(jobId), templateSizeInBytes);                                
+		    } else {
+		    	s_logger.error("Scheme is not supported " + url);
+		        throw new CloudRuntimeException("Scheme is not supported " + url);
+		    }
+		} else {
+		    s_logger.error("Unable to download from URL: " + url);
+		    throw new CloudRuntimeException("Unable to download from URL: " + url);
+		}
+		UploadJob uj = new UploadJob(tu, jobId, id, name, format, hvm, accountId, descr, cksum, installPathPrefix);
+		jobs.put(jobId, uj);
+		threadPool.execute(tu);
+
+		return jobId;
+				
+	}
+
+	@Override
+	public String getUploadError(String jobId) {
+        UploadJob uj = jobs.get(jobId);
+        if (uj != null) {
+            return uj.getTemplateUploader().getUploadError();
+        }
+        return null;
+	}
+
+	@Override
+	public int getUploadPct(String jobId) {
+		UploadJob uj = jobs.get(jobId);
+        if (uj != null) {
+            return uj.getTemplateUploader().getUploadPercent();
+        }
+        return 0;
+	}
+
+	@Override
+	public Status getUploadStatus(String jobId) {
+        UploadJob job = jobs.get(jobId);
+        if (job != null) {
+            TemplateUploader tu = job.getTemplateUploader();
+            if (tu != null) {
+                return tu.getStatus();
+            }
+        }
+        return Status.UNKNOWN;
+	}
+	
+    public static UploadVO.Status convertStatus(Status tds) {
+        switch (tds) {
+        case ABORTED:
+            return UploadVO.Status.NOT_UPLOADED;
+        case UPLOAD_FINISHED:
+            return UploadVO.Status.UPLOAD_IN_PROGRESS;
+        case IN_PROGRESS:
+            return UploadVO.Status.UPLOAD_IN_PROGRESS;
+        case NOT_STARTED:
+            return UploadVO.Status.NOT_UPLOADED;
+        case RECOVERABLE_ERROR:
+            return UploadVO.Status.NOT_UPLOADED;
+        case UNKNOWN:
+            return UploadVO.Status.UNKNOWN;
+        case UNRECOVERABLE_ERROR:
+            return UploadVO.Status.UPLOAD_ERROR;
+        case POST_UPLOAD_FINISHED:
+            return UploadVO.Status.UPLOADED;
+        default:
+            return UploadVO.Status.UNKNOWN;
+        }
+    }
+
+    @Override
+    public com.cloud.storage.UploadVO.Status getUploadStatus2(String jobId) {
+        return convertStatus(getUploadStatus(jobId));
+    }
+	@Override
+	public String getPublicTemplateRepo() {
+		// TODO Auto-generated method stub
+		return null;
+	}
+
+    private UploadAnswer handleUploadProgressCmd(UploadProgressCommand cmd) {
+        String jobId = cmd.getJobId();
+        UploadAnswer answer;
+        UploadJob uj = null;
+        if (jobId != null)
+            uj = jobs.get(jobId);
+        if (uj == null) {           
+           return new UploadAnswer(null, 0, "Cannot find job", com.cloud.storage.UploadVO.Status.UNKNOWN, "", "", 0);            
+        }
+        TemplateUploader td = uj.getTemplateUploader();
+        switch (cmd.getRequest()) {
+        case GET_STATUS:
+            break;
+        case ABORT:
+            td.stopUpload();
+            sleep();
+            break;
+        /*case RESTART:
+            td.stopUpload();
+            sleep();
+            threadPool.execute(td);
+            break;*/
+        case PURGE:
+            td.stopUpload();
+            answer = new UploadAnswer(jobId, getUploadPct(jobId), getUploadError(jobId), getUploadStatus2(jobId), getUploadLocalPath(jobId), getInstallPath(jobId), getUploadTemplateSize(jobId));
+            jobs.remove(jobId);
+            return answer;
+        default:
+            break; // TODO
+        }
+        return new UploadAnswer(jobId, getUploadPct(jobId), getUploadError(jobId), getUploadStatus2(jobId), getUploadLocalPath(jobId), getInstallPath(jobId),
+                getUploadTemplateSize(jobId));
+    }	
+	
+    @Override
+    public UploadAnswer handleUploadCommand(UploadCommand cmd) {
+    	s_logger.warn("Handling the upload " +cmd.getInstallPath() + " " + cmd.getId());
+        if (cmd instanceof UploadProgressCommand) {
+            return handleUploadProgressCmd((UploadProgressCommand) cmd);
+        }
+
+        String user = null;
+        String password = null;
+        String jobId = uploadPublicTemplate(cmd.getId(), cmd.getUrl(), cmd.getName(), 
+        									cmd.getFormat(), cmd.getAccountId(), cmd.getDescription(),
+        									cmd.getChecksum(), cmd.getInstallPath(), user, password,
+        									cmd.getTemplateSizeInBytes());
+        sleep();
+        if (jobId == null) {
+            return new UploadAnswer(null, 0, "Internal Error", com.cloud.storage.UploadVO.Status.UPLOAD_ERROR, "", "", 0);
+        }
+        return new UploadAnswer(jobId, getUploadPct(jobId), getUploadError(jobId), getUploadStatus2(jobId), getUploadLocalPath(jobId), getInstallPath(jobId),
+                getUploadTemplateSize(jobId));
+    }
+
+	private String getInstallPath(String jobId) {
+		// TODO Auto-generated method stub
+		return null;
+	}
+
+	private String getUploadLocalPath(String jobId) {
+		// TODO Auto-generated method stub
+		return null;
+	}
+
+	private long getUploadTemplateSize(String jobId){
+		UploadJob uj = jobs.get(jobId);
+        if (uj != null) {
+            return uj.getTemplatesize();
+        }
+        return 0;
+	}
+
+	@Override
+	public String setRootDir(String rootDir, StorageResource storage) {
+        this.publicTemplateRepo = rootDir + publicTemplateRepo;
+        return null;
+	}
+
+	@Override
+	public boolean configure(String name, Map params)
+			throws ConfigurationException {
+        _name = name;
+
+        String value = null;
+
+        _storage = (StorageLayer) params.get(StorageLayer.InstanceConfigKey);
+        if (_storage == null) {
+            value = (String) params.get(StorageLayer.ClassConfigKey);
+            if (value == null) {
+                throw new ConfigurationException("Unable to find the storage layer");
+            }
+
+            Class clazz;
+            try {
+                clazz = (Class) Class.forName(value);
+            } catch (ClassNotFoundException e) {
+                throw new ConfigurationException("Unable to instantiate " + value);
+            }
+            _storage = ComponentLocator.inject(clazz);
+        }
+        String useSsl = (String)params.get("sslcopy");
+        if (useSsl != null) {
+        	_sslCopy = Boolean.parseBoolean(useSsl);
+        	
+        }
+        configureFolders(name, params);
+        String inSystemVM = (String)params.get("secondary.storage.vm");
+        if (inSystemVM != null && "true".equalsIgnoreCase(inSystemVM)) {
+        	s_logger.info("UploadManager: starting additional services since we are inside system vm");
+        	startAdditionalServices();
+        	blockOutgoingOnPrivate();
+        }
+
+        value = (String) params.get("install.timeout.pergig");
+        this.installTimeoutPerGig = NumbersUtil.parseInt(value, 15 * 60) * 1000;
+
+        value = (String) params.get("install.numthreads");
+        final int numInstallThreads = NumbersUtil.parseInt(value, 10);        
+
+        String scriptsDir = (String) params.get("template.scripts.dir");
+        if (scriptsDir == null) {
+            scriptsDir = "scripts/storage/secondary";
+        }
+
+        List processors = new ArrayList();
+        _processors = new Adapters("processors", processors);
+        Processor processor = new VhdProcessor();
+        
+        processor.configure("VHD Processor", params);
+        processors.add(processor);
+        
+        processor = new IsoProcessor();
+        processor.configure("ISO Processor", params);
+        processors.add(processor);
+        
+        processor = new QCOW2Processor();
+        processor.configure("QCOW2 Processor", params);
+        processors.add(processor);
+        // Add more processors here.
+        threadPool = Executors.newFixedThreadPool(numInstallThreads);
+        return true;
+	}
+	
+	protected void configureFolders(String name, Map params) throws ConfigurationException {
+        parentDir = (String) params.get("template.parent");
+        if (parentDir == null) {
+            throw new ConfigurationException("Unable to find the parent root for the templates");
+        }
+
+        String value = (String) params.get("public.templates.root.dir");
+        if (value == null) {
+            value = TemplateConstants.DEFAULT_TMPLT_ROOT_DIR;
+        }
+               
+		if (value.startsWith(File.separator)) {
+            publicTemplateRepo = value;
+        } else {
+            publicTemplateRepo = parentDir + File.separator + value;
+        }
+        
+        if (!publicTemplateRepo.endsWith(File.separator)) {
+            publicTemplateRepo += File.separator;
+        }
+        
+        publicTemplateRepo += TemplateConstants.DEFAULT_TMPLT_FIRST_LEVEL_DIR;
+        
+        if (!_storage.mkdirs(publicTemplateRepo)) {
+            throw new ConfigurationException("Unable to create public templates directory");
+        }
+    }
+
+	@Override
+	public String getName() {
+		return _name;
+	}
+
+	@Override
+	public boolean start() {
+		return true;
+	}
+
+	@Override
+	public boolean stop() {
+		return true;
+	}
+
+    /**
+     * Get notified of change of job status. Executed in context of uploader thread
+     * 
+     * @param jobId
+     *            the id of the job
+     * @param status
+     *            the status of the job
+     */
+    public void setUploadStatus(String jobId, Status status) {
+        UploadJob uj = jobs.get(jobId);
+        if (uj == null) {
+            s_logger.warn("setUploadStatus for jobId: " + jobId + ", status=" + status + " no job found");
+            return;
+        }
+        TemplateUploader tu = uj.getTemplateUploader();
+        s_logger.warn("Upload Completion for jobId: " + jobId + ", status=" + status);
+        s_logger.warn("UploadedBytes=" + tu.getUploadedBytes() + ", error=" + tu.getUploadError() + ", pct=" + tu.getUploadPercent());
+
+        switch (status) {
+        case ABORTED:
+        case NOT_STARTED:
+        case UNRECOVERABLE_ERROR:
+            // TODO
+            uj.cleanup();
+            break;
+        case UNKNOWN:
+            return;
+        case IN_PROGRESS:
+            s_logger.info("Resuming jobId: " + jobId + ", status=" + status);
+            tu.setResume(true);
+            threadPool.execute(tu);
+            break;
+        case RECOVERABLE_ERROR:
+            threadPool.execute(tu);
+            break;
+        case UPLOAD_FINISHED:
+            tu.setUploadError("Upload success, starting install ");
+            String result = postUpload(jobId);
+            if (result != null) {
+                s_logger.error("Failed post upload script: " + result);
+                tu.setStatus(Status.UNRECOVERABLE_ERROR);
+                tu.setUploadError("Failed post upload script: " + result);
+            } else {
+            	s_logger.warn("Upload completed successfully at " + new SimpleDateFormat().format(new Date()));
+                tu.setStatus(Status.POST_UPLOAD_FINISHED);
+                tu.setUploadError("Upload completed successfully at " + new SimpleDateFormat().format(new Date()));
+            }
+            uj.cleanup();
+            break;
+        default:
+            break;
+        }
+    }
+
+	private String postUpload(String jobId) {
+		return null;
+	}
+
+    private void sleep() {
+        try {
+            Thread.sleep(3000);
+        } catch (InterruptedException e) {
+            // ignore
+        }
+    }
+
+    private void blockOutgoingOnPrivate() {
+    	Script command = new Script("/bin/bash", s_logger);
+    	String intf = "eth1";
+    	command.add("-c");
+    	command.add("iptables -A OUTPUT -o " + intf + " -p tcp -m state --state NEW -m tcp --dport " + "80" + " -j REJECT;" +
+    			"iptables -A OUTPUT -o " + intf + " -p tcp -m state --state NEW -m tcp --dport " + "443" + " -j REJECT;");
+
+    	String result = command.execute();
+    	if (result != null) {
+    		s_logger.warn("Error in blocking outgoing to port 80/443 err=" + result );
+    		return;
+    	}		
+	}
+
+  private void startAdditionalServices() {
+    	
+    	Script command = new Script("/bin/bash", s_logger);
+		command.add("-c");
+    	command.add("service httpd stop ");
+    	String result = command.execute();
+    	if (result != null) {
+    		s_logger.warn("Error in stopping httpd service err=" + result );
+    	}
+    	String port = Integer.toString(TemplateConstants.DEFAULT_TMPLT_COPY_PORT);
+    	String intf = TemplateConstants.DEFAULT_TMPLT_COPY_INTF;
+    	
+    	command = new Script("/bin/bash", s_logger);
+		command.add("-c");
+    	command.add("iptables -D INPUT -i " + intf + " -p tcp -m state --state NEW -m tcp --dport " + port + " -j DROP;" +
+			        "iptables -D INPUT -i " + intf + " -p tcp -m state --state NEW -m tcp --dport " + port + " -j HTTP;" +
+			        "iptables -D INPUT -i " + intf + " -p tcp -m state --state NEW -m tcp --dport " + "443" + " -j DROP;" +
+			        "iptables -D INPUT -i " + intf + " -p tcp -m state --state NEW -m tcp --dport " + "443" + " -j HTTP;" +
+			        "iptables -F HTTP;" +
+			        "iptables -X HTTP;" +
+			        "iptables -N HTTP;" +
+    			    "iptables -I INPUT -i " + intf + " -p tcp -m state --state NEW -m tcp --dport " + port + " -j DROP;" +
+    			    "iptables -I INPUT -i " + intf + " -p tcp -m state --state NEW -m tcp --dport " + "443" + " -j DROP;" +
+    			    "iptables -I INPUT -i " + intf + " -p tcp -m state --state NEW -m tcp --dport " + port + " -j HTTP;" +
+    	            "iptables -I INPUT -i " + intf + " -p tcp -m state --state NEW -m tcp --dport " + "443" + " -j HTTP;");
+
+    	result = command.execute();
+    	if (result != null) {
+    		s_logger.warn("Error in opening up httpd port err=" + result );
+    		return;
+    	}
+    	
+    	command = new Script("/bin/bash", s_logger);
+		command.add("-c");
+    	command.add("service httpd start ");
+    	result = command.execute();
+    	if (result != null) {
+    		s_logger.warn("Error in starting httpd service err=" + result );
+    		return;
+    	}
+    	command = new Script("mkdir", s_logger);
+		command.add("-p");
+    	command.add("/var/www/html/copy/template");
+    	result = command.execute();
+    	if (result != null) {
+    		s_logger.warn("Error in creating directory =" + result );
+    		return;
+    	}
+    	
+    	command = new Script("/bin/bash", s_logger);
+		command.add("-c");
+    	command.add("ln -sf " + publicTemplateRepo + " /var/www/html/copy/template");
+    	result = command.execute();
+    	if (result != null) {
+    		s_logger.warn("Error in linking  err=" + result );
+    		return;
+    	}
+	}
+}    
diff --git a/core/src/com/cloud/storage/template/VhdProcessor.java b/core/src/com/cloud/storage/template/VhdProcessor.java
index 36ec0bf6ef8..6ed4752903a 100644
--- a/core/src/com/cloud/storage/template/VhdProcessor.java
+++ b/core/src/com/cloud/storage/template/VhdProcessor.java
@@ -47,7 +47,7 @@ public class VhdProcessor implements Processor {
     private int vhd_footer_creator_app_offset = 28;
     private int vhd_footer_creator_ver_offset = 32;
     private int vhd_footer_current_size_offset = 48;
-    private byte[] citrix_creator_app = {0x74, 0x61, 0x70, 0x00}; /*"tap "*/
+    private byte[][] citrix_creator_app = {{0x74, 0x61, 0x70, 0x00},{0x43, 0x54, 0x58, 0x53}}; /*"tap ", and "CTXS"*/
 
     @Override
     public FormatInfo process(String templatePath, ImageFormat format, String templateName) throws InternalErrorException {
@@ -91,10 +91,7 @@ public class VhdProcessor implements Processor {
             }
         }
         
-        if (!Arrays.equals(creatorApp, citrix_creator_app)) {
-        	/*Only support VHD image created by citrix xenserver*/
-        	throw new InternalErrorException("Image creator is:" + creatorApp.toString() +", is not supported");
-        }
+        //imageSignatureCheck(creatorApp);
         
         long templateSize = NumbersUtil.bytesToLong(currentSize);
         info.virtualSize = templateSize;
@@ -127,4 +124,22 @@ public class VhdProcessor implements Processor {
     public boolean stop() {
         return true;
     }
+    
+    private void imageSignatureCheck(byte[] creatorApp) throws InternalErrorException {
+    	boolean findKnownCreator = false;
+    	for (int i = 0; i < citrix_creator_app.length; i++) {
+    		if (Arrays.equals(creatorApp, citrix_creator_app[i])) {
+    			findKnownCreator = true;
+    			break;
+    		}
+    	}
+    	if (!findKnownCreator) {
+    		/*Only support VHD image created by citrix xenserver, and xenconverter*/
+    		String readableCreator = "";
+    		for (int j = 0; j < creatorApp.length; j++) {
+    			readableCreator += (char)creatorApp[j];
+    		}
+    		throw new InternalErrorException("Image creator is:" + readableCreator +", is not supported");
+    	}
+    }
 }
diff --git a/core/src/com/cloud/storage/template/VmdkProcessor.java b/core/src/com/cloud/storage/template/VmdkProcessor.java
new file mode 100644
index 00000000000..d1e522196b1
--- /dev/null
+++ b/core/src/com/cloud/storage/template/VmdkProcessor.java
@@ -0,0 +1,69 @@
+package com.cloud.storage.template;
+
+import java.io.File;
+import java.util.Map;
+
+import javax.naming.ConfigurationException;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.exception.InternalErrorException;
+import com.cloud.storage.StorageLayer;
+import com.cloud.storage.Storage.ImageFormat;
+
+public class VmdkProcessor implements Processor {
+    private static final Logger s_logger = Logger.getLogger(VmdkProcessor.class);
+
+    String _name;
+    StorageLayer _storage;
+	
+    @Override
+    public FormatInfo process(String templatePath, ImageFormat format, String templateName) throws InternalErrorException {
+        if (format != null) {
+        	if(s_logger.isInfoEnabled())
+        		s_logger.info("We currently don't handle conversion from " + format + " to VMDK.");
+            return null;
+        }
+        
+        s_logger.info("Template processing. templatePath: " + templatePath + ", templateName: " + templateName);
+        String templateFilePath = templatePath + File.separator + templateName + "." + ImageFormat.VMDK.getFileExtension();
+        if (!_storage.exists(templateFilePath)) {
+        	if(s_logger.isInfoEnabled())
+        		s_logger.info("Unable to find the vmware template file: " + templateFilePath);
+            return null;
+        }
+        
+        FormatInfo info = new FormatInfo();
+        info.format = ImageFormat.VMDK;
+        info.filename = templateName + "." + ImageFormat.VMDK.getFileExtension();
+        info.size = _storage.getSize(templateFilePath);
+        info.virtualSize = info.size;
+        return info;
+    }
+    
+    @Override
+    public boolean configure(String name, Map params) throws ConfigurationException {
+        _name = name;
+        _storage = (StorageLayer)params.get(StorageLayer.InstanceConfigKey);
+        if (_storage == null) {
+            throw new ConfigurationException("Unable to get storage implementation");
+        }
+    	
+    	return true;
+    }
+    
+    @Override
+    public String getName() {
+        return _name;
+    }
+    
+    @Override
+    public boolean start() {
+        return true;
+    }
+
+    @Override
+    public boolean stop() {
+        return true;
+    }
+}
diff --git a/core/src/com/cloud/user/AccountVO.java b/core/src/com/cloud/user/AccountVO.java
index 42c30ee7ca3..58f0c89b3e8 100644
--- a/core/src/com/cloud/user/AccountVO.java
+++ b/core/src/com/cloud/user/AccountVO.java
@@ -18,15 +18,15 @@
 
 package com.cloud.user;
 
-import java.util.Date;
-
-import javax.persistence.Column;
-import javax.persistence.Entity;
-import javax.persistence.GeneratedValue;
-import javax.persistence.GenerationType;
-import javax.persistence.Id;
-import javax.persistence.Table;
-
+import java.util.Date;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+
 import com.cloud.utils.db.GenericDao;
 
 @Entity
@@ -35,7 +35,7 @@ public class AccountVO implements Account {
     @Id
     @GeneratedValue(strategy=GenerationType.IDENTITY)
     @Column(name="id")
-    private Long id = null;
+    private long id;
 
     @Column(name="account_name")
     private String accountName = null;
@@ -44,7 +44,7 @@ public class AccountVO implements Account {
     private short type = ACCOUNT_TYPE_NORMAL;
 
     @Column(name="domain_id")
-    private Long domainId = null;
+    private long domainId;
 
     @Column(name="state")
     private String state;
@@ -56,7 +56,7 @@ public class AccountVO implements Account {
     private boolean needsCleanup = false;
 
     public AccountVO() {}
-    public AccountVO(Long id) {
+    public AccountVO(long id) {
         this.id = id;
     }
     
@@ -67,8 +67,9 @@ public class AccountVO implements Account {
     public boolean getNeedsCleanup() {
     	return needsCleanup;
     }
-
-    public Long getId() {
+
+    @Override
+    public long getId() {
         return id;
     }
 
@@ -85,10 +86,11 @@ public class AccountVO implements Account {
         this.type = type;
     }
 
-    public Long getDomainId() {
+    public long getDomainId() {
         return domainId;
-    }
-    public void setDomainId(Long domainId) {
+    }
+    
+    public void setDomainId(long domainId) {
         this.domainId = domainId;
     }
 
diff --git a/core/src/com/cloud/user/dao/AccountDaoImpl.java b/core/src/com/cloud/user/dao/AccountDaoImpl.java
index c9dd9c0b401..1b60f22d2b4 100644
--- a/core/src/com/cloud/user/dao/AccountDaoImpl.java
+++ b/core/src/com/cloud/user/dao/AccountDaoImpl.java
@@ -36,8 +36,8 @@ import com.cloud.utils.db.Filter;
 import com.cloud.utils.db.GenericDaoBase;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
-import com.cloud.utils.db.Transaction;
 import com.cloud.utils.db.SearchCriteria.Op;
+import com.cloud.utils.db.Transaction;
 
 @Local(value={AccountDao.class})
 public class AccountDaoImpl extends GenericDaoBase implements AccountDao {
@@ -74,7 +74,7 @@ public class AccountDaoImpl extends GenericDaoBase implements A
     	SearchCriteria sc = CleanupSearch.create();
     	sc.setParameters("cleanup", true);
     	
-    	return searchAll(sc, null, null, false);
+    	return searchIncludingRemoved(sc, null, null, false);
     }
     
     public Pair findUserAccountByApiKey(String apiKey) {
@@ -94,7 +94,7 @@ public class AccountDaoImpl extends GenericDaoBase implements A
                 u.setSecretKey(rs.getString(4));
                 u.setState(rs.getString(5));
 
-                Account a = new AccountVO(rs.getLong(6));
+                AccountVO a = new AccountVO(rs.getLong(6));
                 a.setAccountName(rs.getString(7));
                 a.setType(rs.getShort(8));
                 a.setDomainId(rs.getLong(9));
@@ -112,26 +112,26 @@ public class AccountDaoImpl extends GenericDaoBase implements A
     public List findAccountsLike(String accountName) {
         SearchCriteria sc = createSearchCriteria();
         sc.addAnd("accountName", SearchCriteria.Op.LIKE, "%"+accountName+"%");
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
     public Account findActiveAccount(String accountName, Long domainId) {
         SearchCriteria sc = AccountNameSearch.create("accountName", accountName);
         sc.addAnd("domainId", SearchCriteria.Op.EQ, domainId);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 
     @Override
     public Account findAccount(String accountName, Long domainId) {
         SearchCriteria sc = AccountNameSearch.create("accountName", accountName);
         sc.addAnd("domainId", SearchCriteria.Op.EQ, domainId);
-        return findOneBy(sc);
+        return findOneIncludingRemovedBy(sc);
     }
     
     public Account findActiveAccountByName(String accountName) {
     	SearchCriteria sc = AccountNameSearch.create("accountName", accountName);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 
     public List findActiveAccounts(Long maxAccountId, Filter filter) {
@@ -140,7 +140,7 @@ public class AccountDaoImpl extends GenericDaoBase implements A
         SearchCriteria sc = createSearchCriteria();
         sc.addAnd("id", SearchCriteria.Op.LTEQ, maxAccountId);
 
-        return listActiveBy(sc, filter);
+        return listBy(sc, filter);
     }
 
     public List findRecentlyDeletedAccounts(Long maxAccountId, Date earliestRemovedDate, Filter filter) {
@@ -152,7 +152,7 @@ public class AccountDaoImpl extends GenericDaoBase implements A
         sc.addAnd("removed", SearchCriteria.Op.NNULL);
         sc.addAnd("removed", SearchCriteria.Op.GTEQ, earliestRemovedDate);
 
-        return listBy(sc, filter);
+        return listIncludingRemovedBy(sc, filter);
     }
 
     public List findNewAccounts(Long minAccountId, Filter filter) {
@@ -161,7 +161,7 @@ public class AccountDaoImpl extends GenericDaoBase implements A
         SearchCriteria sc = createSearchCriteria();
         sc.addAnd("id", SearchCriteria.Op.GT, minAccountId);
 
-        return listBy(sc, filter);
+        return listIncludingRemovedBy(sc, filter);
     }
 
 	@Override
diff --git a/core/src/com/cloud/user/dao/UserAccountDaoImpl.java b/core/src/com/cloud/user/dao/UserAccountDaoImpl.java
index 175f5ca3c7e..8e147389a7e 100644
--- a/core/src/com/cloud/user/dao/UserAccountDaoImpl.java
+++ b/core/src/com/cloud/user/dao/UserAccountDaoImpl.java
@@ -36,7 +36,7 @@ public class UserAccountDaoImpl extends GenericDaoBase impl
         SearchCriteria sc = createSearchCriteria();
         sc.addAnd("username", SearchCriteria.Op.EQ, username);
         sc.addAnd("domainId", SearchCriteria.Op.EQ, domainId);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 
     @Override
diff --git a/core/src/com/cloud/user/dao/UserDaoImpl.java b/core/src/com/cloud/user/dao/UserDaoImpl.java
index 7fefd2e3c24..4b4a53b11a8 100644
--- a/core/src/com/cloud/user/dao/UserDaoImpl.java
+++ b/core/src/com/cloud/user/dao/UserDaoImpl.java
@@ -75,41 +75,41 @@ public class UserDaoImpl extends GenericDaoBase implements UserDao
 	    SearchCriteria sc = UsernamePasswordSearch.create();
 	    sc.setParameters("username", username);
 	    sc.setParameters("password", password);
-	    return findOneActiveBy(sc);
+	    return findOneBy(sc);
 	}
 	
 	public List listByAccount(long accountId) {
 	    SearchCriteria sc = AccountIdSearch.create();
 	    sc.setParameters("account", accountId);
-	    return listActiveBy(sc, null);
+	    return listBy(sc, null);
 	}
 
 	@Override
 	public UserVO getUser(String username) {
 	    SearchCriteria sc = UsernameSearch.create();
 	    sc.setParameters("username", username);
-	    return findOneActiveBy(sc);
+	    return findOneBy(sc);
 	}
 
     @Override
     public UserVO getUser(long userId) {
         SearchCriteria sc = UserIdSearch.create();
         sc.setParameters("id", userId);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 
 	@Override
 	public List findUsersLike(String username) {
         SearchCriteria sc = UsernameLikeSearch.create();
         sc.setParameters("username", "%" + username + "%");
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 	
     @Override
     public UserVO findUserBySecretKey(String secretKey) {
         SearchCriteria sc = SecretKeySearch.create();
         sc.setParameters("secretKey", secretKey);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
     
     @Override
diff --git a/core/src/com/cloud/user/dao/UserStatisticsDaoImpl.java b/core/src/com/cloud/user/dao/UserStatisticsDaoImpl.java
index 72298ba8848..956fe6e1761 100644
--- a/core/src/com/cloud/user/dao/UserStatisticsDaoImpl.java
+++ b/core/src/com/cloud/user/dao/UserStatisticsDaoImpl.java
@@ -62,7 +62,7 @@ public class UserStatisticsDaoImpl extends GenericDaoBase sc = UserDcSearch.create();
         sc.setParameters("account", accountId);
         sc.setParameters("dc", dcId);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 
     @Override
diff --git a/core/src/com/cloud/vm/ConsoleProxyVO.java b/core/src/com/cloud/vm/ConsoleProxyVO.java
index 20499b437b8..c2e13672b0f 100644
--- a/core/src/com/cloud/vm/ConsoleProxyVO.java
+++ b/core/src/com/cloud/vm/ConsoleProxyVO.java
@@ -93,9 +93,18 @@ public class ConsoleProxyVO extends VMInstanceVO implements ConsoleProxy {
     
 	@Transient
     private int port;
-    
+
+	/**
+	 * Correct constructor to use.
+	 */
+	public ConsoleProxyVO(long id, long serviceOfferingId, String name, long templateId, long guestOSId, long dataCenterId, long domainId, long accountId, int activeSession) {
+	    super(id, serviceOfferingId, name, name, Type.ConsoleProxy, templateId, guestOSId, domainId, accountId, false);
+	    this.activeSession = activeSession;
+	}
+	
     public ConsoleProxyVO(
-    		long id,
+    		long id,
+    		long serviceOfferingId,
             String name,
             String guestMacAddress,
             String guestIpAddress,
@@ -111,7 +120,9 @@ public class ConsoleProxyVO extends VMInstanceVO implements ConsoleProxy {
             Long vlanDbId,
             String vlanId,
             long podId,
-            long dataCenterId,
+            long dataCenterId,
+            long domainId,
+            long accountId,
             String gateway,
             Long hostId,
             String dns1,
@@ -119,8 +130,8 @@ public class ConsoleProxyVO extends VMInstanceVO implements ConsoleProxy {
             String domain,
             int ramSize,
             int activeSession) {
-    	super(id, name, name, Type.ConsoleProxy, templateId, guestOSId,
-    			privateMacAddress, privateIpAddress, privateNetmask, dataCenterId, podId, true, hostId);
+    	super(id, serviceOfferingId, name, name, Type.ConsoleProxy, templateId, guestOSId,
+    			privateMacAddress, privateIpAddress, privateNetmask, dataCenterId, podId, domainId, accountId, true, hostId);
     	this.gateway = gateway;
     	this.publicIpAddress = publicIpAddress;
     	this.publicNetmask = publicNetmask;
diff --git a/core/src/com/cloud/vm/DomainRouterVO.java b/core/src/com/cloud/vm/DomainRouterVO.java
index 13f6c94c939..12f4cafb1cb 100755
--- a/core/src/com/cloud/vm/DomainRouterVO.java
+++ b/core/src/com/cloud/vm/DomainRouterVO.java
@@ -35,12 +35,6 @@ import com.cloud.utils.net.NetUtils;
 @PrimaryKeyJoinColumn(name="id")
 @DiscriminatorValue(value="DomainRouter")
 public class DomainRouterVO extends VMInstanceVO implements DomainRouter {
-    @Column(name="account_id", updatable=false, nullable=false)
-    private long accountId = -1;
-
-    @Column(name="domain_id", updatable=false, nullable=false)
-    private long domainId = 0;
-
     @Column(name="ram_size", updatable=false, nullable=false)
     private int ramSize;
     
@@ -94,6 +88,7 @@ public class DomainRouterVO extends VMInstanceVO implements DomainRouter {
     private Role role = Role.DHCP_FIREWALL_LB_PASSWD_USERDATA;
     
     public DomainRouterVO(long id,
+                          long serviceOfferingId,
                           String name,
                           String instanceName,
                           String privateMacAddress,
@@ -120,7 +115,7 @@ public class DomainRouterVO extends VMInstanceVO implements DomainRouter {
                           Long hostId,
                           String dns1,
                           String dns2) {
-        super(id, name, instanceName, Type.DomainRouter, templateId, guestOSId, privateMacAddress, privateIpAddress, privateNetmask, dataCenterId, podId, true, hostId);
+        super(id, serviceOfferingId, name, instanceName, Type.DomainRouter, templateId, guestOSId, privateMacAddress, privateIpAddress, privateNetmask, dataCenterId, podId, domainId, accountId, true, hostId);
         this.privateMacAddress = privateMacAddress;
         this.guestMacAddress = guestMacAddress;
         this.guestIpAddress = guestIpAddress;
@@ -142,6 +137,7 @@ public class DomainRouterVO extends VMInstanceVO implements DomainRouter {
     }
     
     public DomainRouterVO(long id,
+                          long serviceOfferingId,
                           String name,
                           String privateMacAddress,
                           String privateIpAddress,
@@ -164,15 +160,7 @@ public class DomainRouterVO extends VMInstanceVO implements DomainRouter {
                           String domain,
                           String dns1,
                           String dns2) {
-        this(id, name, name, privateMacAddress, privateIpAddress, privateNetmask, templateId, guestOSId, guestMacAddress, guestIpAddress, guestNetmask, null, accountId, domainId, publicMacAddress, publicIpAddress, publicNetMask, vlanDbId, vlanId, podId, dataCenterId, ramSize, gateway, domain, null, dns1, dns2);
-    }
-
-    public long getAccountId() {
-        return accountId;
-    }
-
-    public long getDomainId() {
-        return domainId;
+        this(id, serviceOfferingId, name, name, privateMacAddress, privateIpAddress, privateNetmask, templateId, guestOSId, guestMacAddress, guestIpAddress, guestNetmask, null, accountId, domainId, publicMacAddress, publicIpAddress, publicNetMask, vlanDbId, vlanId, podId, dataCenterId, ramSize, gateway, domain, null, dns1, dns2);
     }
 
     public void setGateway(String gateway) {
@@ -319,6 +307,7 @@ public class DomainRouterVO extends VMInstanceVO implements DomainRouter {
         this.zoneVlan = zoneVlan;
     }
 
+    @Override
     public String getZoneVlan() {
         return zoneVlan;
     }
@@ -327,6 +316,7 @@ public class DomainRouterVO extends VMInstanceVO implements DomainRouter {
         this.guestZoneMacAddress = guestZoneMacAddress;
     }
 
+    @Override
     public String getGuestZoneMacAddress() {
         return guestZoneMacAddress;
     }
diff --git a/core/src/com/cloud/vm/InstanceGroupVMMapVO.java b/core/src/com/cloud/vm/InstanceGroupVMMapVO.java
new file mode 100644
index 00000000000..f51092c5dc7
--- /dev/null
+++ b/core/src/com/cloud/vm/InstanceGroupVMMapVO.java
@@ -0,0 +1,53 @@
+package com.cloud.vm;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.PrimaryKeyJoinColumn;
+import javax.persistence.SecondaryTable;
+import javax.persistence.SecondaryTables;
+import javax.persistence.Table;
+
+@Entity
+@Table(name=("instance_group_vm_map"))
+@SecondaryTables({
+@SecondaryTable(name="user_vm",
+        pkJoinColumns={@PrimaryKeyJoinColumn(name="instance_id", referencedColumnName="id")}),      
+@SecondaryTable(name="instance_group", 
+		pkJoinColumns={@PrimaryKeyJoinColumn(name="group_id", referencedColumnName="id")})
+		})
+public class InstanceGroupVMMapVO {
+    @Id
+    @GeneratedValue(strategy=GenerationType.IDENTITY)
+    @Column(name="id")
+    private Long id;
+
+    @Column(name="group_id")
+    private long groupId;
+
+    @Column(name="instance_id")
+    private long instanceId;
+    
+
+    public InstanceGroupVMMapVO() { }
+
+    public InstanceGroupVMMapVO(long groupId, long instanceId) {
+        this.groupId = groupId;
+        this.instanceId = instanceId;
+    }
+
+    public Long getId() {
+        return id;
+    }
+
+    public long getGroupId() {
+        return groupId;
+    }
+
+    public long getInstanceId() {
+        return instanceId;
+    }
+
+}
\ No newline at end of file
diff --git a/core/src/com/cloud/vm/InstanceGroupVO.java b/core/src/com/cloud/vm/InstanceGroupVO.java
new file mode 100644
index 00000000000..a16d784c3fb
--- /dev/null
+++ b/core/src/com/cloud/vm/InstanceGroupVO.java
@@ -0,0 +1,79 @@
+package com.cloud.vm;
+
+import java.util.Date;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.PrimaryKeyJoinColumn;
+import javax.persistence.SecondaryTable;
+import javax.persistence.Table;
+
+import com.cloud.user.OwnedBy;
+import com.cloud.utils.db.GenericDao;
+
+@Entity
+@Table(name="instance_group")
+@SecondaryTable(name="account",
+        pkJoinColumns={@PrimaryKeyJoinColumn(name="account_id", referencedColumnName="id")})
+public class InstanceGroupVO implements OwnedBy{
+    @Id
+    @GeneratedValue(strategy=GenerationType.IDENTITY)
+    @Column(name="id")
+    private long id;
+    
+    @Column(name="name")
+    String name;
+    
+    @Column(name="account_id")
+    private long accountId;
+    
+    @Column(name="domain_id", table="account", insertable=false, updatable=false)
+    private long domainId;
+    
+    @Column(name=GenericDao.REMOVED_COLUMN)
+    private Date removed;
+
+    @Column(name=GenericDao.CREATED_COLUMN)
+    private Date created;
+    
+    public InstanceGroupVO(String name, long accountId) {
+        this.name = name;
+        this.accountId = accountId;
+    }
+    
+    protected InstanceGroupVO() {
+        super();
+    }
+    
+    public long getId() {
+    	return id;
+    }
+    
+    public String getName() {
+    	return name; 
+    }
+    
+    public long getAccountId() {
+        return accountId;
+    }
+    
+    public long getDomainId() {
+        return domainId;
+    }
+    
+    public Date getRemoved() {
+        return removed;
+    }
+    
+	public Date getCreated() {
+		return created;
+	}
+    
+    public void setName(String name) {
+    	this.name = name;
+    }
+
+}
diff --git a/core/src/com/cloud/vm/SecondaryStorageVmVO.java b/core/src/com/cloud/vm/SecondaryStorageVmVO.java
index 57d9167b17d..cb00d26350c 100644
--- a/core/src/com/cloud/vm/SecondaryStorageVmVO.java
+++ b/core/src/com/cloud/vm/SecondaryStorageVmVO.java
@@ -89,7 +89,8 @@ public class SecondaryStorageVmVO extends VMInstanceVO implements SecondaryStora
     
     
     public SecondaryStorageVmVO(
-    		long id,
+    		long id,
+    		long serviceOfferingId,
             String name,
             String guestMacAddress,
             String guestIpAddress,
@@ -105,7 +106,9 @@ public class SecondaryStorageVmVO extends VMInstanceVO implements SecondaryStora
             Long vlanDbId,
             String vlanId,
             long podId,
-            long dataCenterId,
+            long dataCenterId,
+            long domainId,
+            long accountId,
             String gateway,
             Long hostId,
             String dns1,
@@ -114,8 +117,8 @@ public class SecondaryStorageVmVO extends VMInstanceVO implements SecondaryStora
             int ramSize,
             String guid,
             String nfsShare) {
-    	super(id, name, name, Type.SecondaryStorageVm, templateId, guestOSId,
-    			privateMacAddress, privateIpAddress, privateNetmask, dataCenterId, podId, true, hostId);
+    	super(id, serviceOfferingId, name, name, Type.SecondaryStorageVm, templateId, guestOSId,
+    			privateMacAddress, privateIpAddress, privateNetmask, dataCenterId, podId, domainId, accountId, true, hostId);
     	this.gateway = gateway;
     	this.publicIpAddress = publicIpAddress;
     	this.publicNetmask = publicNetmask;
diff --git a/core/src/com/cloud/vm/UserVmVO.java b/core/src/com/cloud/vm/UserVmVO.java
index 0943ff72f2b..b4c9049fa94 100755
--- a/core/src/com/cloud/vm/UserVmVO.java
+++ b/core/src/com/cloud/vm/UserVmVO.java
@@ -31,18 +31,9 @@ import com.cloud.uservm.UserVm;
 @PrimaryKeyJoinColumn(name="id")
 public class UserVmVO extends VMInstanceVO implements UserVm {
 
-    @Column(name="account_id", updatable=false, nullable=false)
-    private long accountId = -1;
-    
-    @Column(name="domain_id", updatable=false, nullable=false)
-    private long domainId = -1;
-    
     @Column(name="domain_router_id", updatable=true, nullable=true)
     Long domainRouterId;
 
-    @Column(name="service_offering_id", updatable=true, nullable=false)
-    long serviceOfferingId;
-    
     @Column(name="vnet", length=10, updatable=true, nullable=true)
     String vnet;
 
@@ -60,9 +51,6 @@ public class UserVmVO extends VMInstanceVO implements UserVm {
     
     @Column(name="external_ip_address")
 	String externalIpAddress;
-    
-    @Column(name="group", updatable=true, nullable=true)
-    private String group;
 
     @Column(name="external_mac_address")
 	String externalMacAddress;
@@ -77,15 +65,6 @@ public class UserVmVO extends VMInstanceVO implements UserVm {
     private String displayName;
 
     @Override
-    public long getAccountId() {
-        return accountId;
-    }
-
-    @Override
-    public long getDomainId() {
-        return domainId;
-    }
-
     public String getGuestIpAddress() {
 		return guestIpAddress;
 	}
@@ -95,7 +74,8 @@ public class UserVmVO extends VMInstanceVO implements UserVm {
 		setPrivateIpAddress(guestIpAddress);
 	}
 
-	public String getGuestMacAddress() {
+	@Override
+    public String getGuestMacAddress() {
 		return guestMacAddress;
 	}
 
@@ -146,15 +126,6 @@ public class UserVmVO extends VMInstanceVO implements UserVm {
         return vnet;
     }
     
-    @Override
-    public String getGroup() {
-        return group;
-    }
-    
-    public void setGroup(String group) {
-        this.group = group;
-    }
-    
     public UserVmVO(long id,
                     String instanceName,
                     String displayName,
@@ -164,10 +135,8 @@ public class UserVmVO extends VMInstanceVO implements UserVm {
                     long domainId,
                     long accountId,
                     long serviceOfferingId,
-                    String group,
                     String userData) {
-        super(id, displayName, instanceName, Type.User, templateId, guestOsId, haEnabled);
-        this.group = group;
+        super(id, serviceOfferingId, displayName, instanceName, Type.User, templateId, guestOsId, domainId, accountId, haEnabled);
         this.userData = userData;
         this.displayName = displayName;
         
@@ -191,13 +160,9 @@ public class UserVmVO extends VMInstanceVO implements UserVm {
                     long dcId,
                     boolean haEnabled,
                     String displayName,
-                    String group,
                     String userData) {
-        super(id, name, name, Type.User, templateId, guestOSId, guestMacAddress, guestIpAddress, guestNetMask, dcId, podId, haEnabled, null);
-        this.serviceOfferingId = serviceOfferingId;
+        super(id, serviceOfferingId, name, name, Type.User, templateId, guestOSId, guestMacAddress, guestIpAddress, guestNetMask, dcId, podId, domainId, accountId, haEnabled, null);
         this.domainRouterId = routerId;
-        this.accountId = accountId;
-        this.domainId = domainId;
         this.guestIpAddress = guestIpAddress;
         this.guestNetmask = guestNetMask;
         this.guestMacAddress = guestMacAddress;
@@ -205,10 +170,8 @@ public class UserVmVO extends VMInstanceVO implements UserVm {
         this.externalMacAddress = externalMacAddress;
         this.setUserData(userData);
         this.setExternalVlanDbId(vlanDbId);
-        this.group = group;
         this.isoId = null;
         this.displayName = displayName;
-        this.group = group;
     }
 
     protected UserVmVO() {
diff --git a/core/src/com/cloud/vm/VMInstanceVO.java b/core/src/com/cloud/vm/VMInstanceVO.java
index 84c91116737..640dbc10926 100644
--- a/core/src/com/cloud/vm/VMInstanceVO.java
+++ b/core/src/com/cloud/vm/VMInstanceVO.java
@@ -36,12 +36,14 @@ import javax.persistence.Temporal;
 import javax.persistence.TemporalType;
 
 import com.cloud.utils.db.GenericDao;
+import com.cloud.utils.db.StateMachine;
+import com.cloud.utils.fsm.FiniteStateObject;
 
 @Entity
 @Table(name="vm_instance")
 @Inheritance(strategy=InheritanceType.JOINED)
 @DiscriminatorColumn(name="type", discriminatorType=DiscriminatorType.STRING, length=32)
-public class VMInstanceVO implements VirtualMachine {
+public class VMInstanceVO implements VirtualMachine, FiniteStateObject {
     @Id
     @TableGenerator(name="vm_instance_sq", table="sequence", pkColumnName="name", valueColumnName="value", pkColumnValue="vm_instance_seq", allocationSize=1)
     @Column(name="id", updatable=false, nullable = false)
@@ -65,7 +67,8 @@ public class VMInstanceVO implements VirtualMachine {
      * the state machine needs to go through the DAO object because someone
      * else could be updating it as well.
      */
-    @Enumerated(value=EnumType.STRING)
+    @Enumerated(value=EnumType.STRING)
+    @StateMachine(state=State.class, event=Event.class)
     @Column(name="state", updatable=true, nullable=false, length=32)
     private State state = null;
 
@@ -122,12 +125,24 @@ public class VMInstanceVO implements VirtualMachine {
     @Temporal(value=TemporalType.TIMESTAMP)
     Date updateTime;
     
+    @Column(name="domain_id")
+    long domainId;
+    
+    @Column(name="account_id")
+    long accountId;
+    
+    @Column(name="service_offering_id")
+    long serviceOfferingId;
+    
     public VMInstanceVO(long id,
+                        long serviceOfferingId,
                         String name,
                         String instanceName,
                         Type type,
                         Long vmTemplateId,
                         long guestOSId,
+                        long domainId,
+                        long accountId,
                         boolean haEnabled) {
         this.id = id;
         this.name = name;
@@ -138,10 +153,17 @@ public class VMInstanceVO implements VirtualMachine {
         this.type = type;
         this.guestOSId = guestOSId;
         this.haEnabled = haEnabled;
+        this.mirroredVols = false;
+        this.vncPassword = Long.toHexString(new Random().nextLong());
+        this.state = State.Creating;
+        this.accountId = accountId;
+        this.domainId = domainId;
+        this.serviceOfferingId = serviceOfferingId;
     }
                        
 	
-    public VMInstanceVO(long id,
+    public VMInstanceVO(long id,
+                        long serviceOfferingId,
                         String name,
                         String instanceName,
                         Type type,
@@ -151,7 +173,9 @@ public class VMInstanceVO implements VirtualMachine {
                         String privateIpAddress,
                         String privateNetmask,
                         long dataCenterId,
-                        long podId,
+                        long podId,
+                        long domainId,
+                        long accountId,
                         boolean haEnabled,
                         Long hostId) {
         super();
@@ -175,6 +199,9 @@ public class VMInstanceVO implements VirtualMachine {
         this.updateTime = new Date();
 		this.vncPassword = Long.toHexString(new Random().nextLong());
 		this.state = State.Creating;
+		this.serviceOfferingId = serviceOfferingId;
+		this.domainId = domainId;
+		this.accountId =  accountId;
     }
 
     protected VMInstanceVO() {
@@ -182,6 +209,16 @@ public class VMInstanceVO implements VirtualMachine {
     
     public Date getRemoved() {
     	return removed;
+    }
+    
+    @Override
+    public long getDomainId() {
+        return domainId;
+    }
+    
+    @Override
+    public long getAccountId() {
+        return accountId;
     }
     
     @Override
@@ -193,11 +230,13 @@ public class VMInstanceVO implements VirtualMachine {
     	return updated;
     }
     
-	public long getId() {
+	@Override
+    public long getId() {
 		return id;
 	}
 	
-	public Date getCreated() {
+	@Override
+    public Date getCreated() {
 		return created;
 	}
 	
@@ -205,7 +244,8 @@ public class VMInstanceVO implements VirtualMachine {
 		return updateTime;
 	}
 	
-	public long getDataCenterId() {
+	@Override
+    public long getDataCenterId() {
 	    return dataCenterId;
 	}
 	
@@ -242,7 +282,8 @@ public class VMInstanceVO implements VirtualMachine {
 	}
 	
 	// don't use this directly, use VM state machine instead, this method is added for migration tool only
-	public void setState(State state) {
+	@Override
+    public void setState(State state) {
 		this.state = state;
 	}
 	
@@ -264,6 +305,10 @@ public class VMInstanceVO implements VirtualMachine {
         return vncPassword;
     }
     
+    public long getServiceOfferingId() {
+        return serviceOfferingId;
+    }
+    
 	public Long getProxyId() {
     	return proxyId;
     }
@@ -292,7 +337,8 @@ public class VMInstanceVO implements VirtualMachine {
 		this.templateId = templateId;
 	}
 
-	public long getGuestOSId() {
+	@Override
+    public long getGuestOSId() {
 		return guestOSId;
 	}
 	
diff --git a/core/src/com/cloud/vm/VirtualNetwork.java b/core/src/com/cloud/vm/VirtualNetwork.java
index fd882a0b7e9..44651f65b3b 100644
--- a/core/src/com/cloud/vm/VirtualNetwork.java
+++ b/core/src/com/cloud/vm/VirtualNetwork.java
@@ -17,25 +17,15 @@
  */
 package com.cloud.vm;
 
+import com.cloud.network.Network.IsolationType;
+import com.cloud.network.Network.Mode;
+
 /**
  * VirtualNetwork describes from a management level the
  * things needed to provide the network to the virtual
  * machine.
  */
 public class VirtualNetwork {
-    public enum Mode {
-        None,
-        Local,
-        Static,
-        Dhcp;
-    }
-    
-    public enum Isolation {
-        VNET,
-        VLAN,
-        OSWITCH,
-    }
-    
     /**
      * The gateway for this network.
      */
@@ -74,7 +64,7 @@ public class VirtualNetwork {
     /**
      * Isolation method for networking.
      */
-    public Isolation method;
+    public IsolationType method;
     
     public boolean firewalled;
     
diff --git a/core/src/com/cloud/vm/dao/ConsoleProxyDaoImpl.java b/core/src/com/cloud/vm/dao/ConsoleProxyDaoImpl.java
index 2d33f98aeb4..7936c22a370 100644
--- a/core/src/com/cloud/vm/dao/ConsoleProxyDaoImpl.java
+++ b/core/src/com/cloud/vm/dao/ConsoleProxyDaoImpl.java
@@ -231,21 +231,21 @@ public class ConsoleProxyDaoImpl extends GenericDaoBase im
         SearchCriteria sc = DataCenterStatusSearch.create();
         sc.setParameters("states", (Object[])states);
         sc.setParameters("dc", dataCenterId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
     public List getProxyListInStates(State... states) {
         SearchCriteria sc = StateSearch.create();
         sc.setParameters("states", (Object[])states);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
     public List listByHostId(long hostId) {
         SearchCriteria sc = HostSearch.create();
         sc.setParameters("host", hostId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
@@ -253,7 +253,7 @@ public class ConsoleProxyDaoImpl extends GenericDaoBase im
         SearchCriteria sc = HostUpSearch.create();
         sc.setParameters("host", hostId);
         sc.setParameters("states", new Object[] {State.Destroyed, State.Stopped, State.Expunging}); 
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
diff --git a/core/src/com/cloud/vm/dao/DomainRouterDaoImpl.java b/core/src/com/cloud/vm/dao/DomainRouterDaoImpl.java
index 2a12035ded4..c6589d48715 100755
--- a/core/src/com/cloud/vm/dao/DomainRouterDaoImpl.java
+++ b/core/src/com/cloud/vm/dao/DomainRouterDaoImpl.java
@@ -123,7 +123,7 @@ public class DomainRouterDaoImpl extends GenericDaoBase im
     public DomainRouterVO findByPublicIpAddress(String ipAddress) {
         SearchCriteria sc = IpSearch.create();
         sc.setParameters("ip", ipAddress);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 
     @Override
@@ -196,7 +196,7 @@ public class DomainRouterDaoImpl extends GenericDaoBase im
     public List listByDataCenter(long dcId) {
         SearchCriteria sc = DcSearch.create();
         sc.setParameters("dc", dcId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -205,7 +205,7 @@ public class DomainRouterDaoImpl extends GenericDaoBase im
         sc.setParameters("account", accountId);
         sc.setParameters("dc", dcId);
         sc.setParameters("role", Role.DHCP_FIREWALL_LB_PASSWD_USERDATA);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
     
     @Override
@@ -214,21 +214,21 @@ public class DomainRouterDaoImpl extends GenericDaoBase im
         sc.setParameters("account", accountId);
         sc.setParameters("dc", dcId);
         sc.setParameters("role", role);
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
     }
 
     @Override
     public List listBy(long accountId) {
         SearchCriteria sc = AccountSearch.create();
         sc.setParameters("account", accountId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
     public List listByHostId(Long hostId) {
         SearchCriteria sc = HostSearch.create();
         sc.setParameters("host", hostId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
@@ -238,7 +238,7 @@ public class DomainRouterDaoImpl extends GenericDaoBase im
             sc.setParameters("host", hostId);
         }
         sc.setParameters("states", new Object[] {State.Destroyed, State.Stopped, State.Expunging});
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -287,13 +287,13 @@ public class DomainRouterDaoImpl extends GenericDaoBase im
     public List listByDomain(Long domainId) {
         SearchCriteria sc = DomainIdSearch.create();
         sc.setParameters("domainId", domainId);
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
     }
 
     @Override
     public List listByVlanDbId(Long vlanDbId) {
         SearchCriteria sc = VlanDbIdSearch.create();
         sc.setParameters("vlanDbId", vlanDbId);
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
     }
 }
diff --git a/core/src/com/cloud/vm/dao/InstanceGroupDao.java b/core/src/com/cloud/vm/dao/InstanceGroupDao.java
new file mode 100644
index 00000000000..6cfadce311b
--- /dev/null
+++ b/core/src/com/cloud/vm/dao/InstanceGroupDao.java
@@ -0,0 +1,20 @@
+package com.cloud.vm.dao;
+
+import java.util.List;
+
+import com.cloud.utils.db.GenericDao;
+import com.cloud.vm.InstanceGroupVO;
+
+public interface InstanceGroupDao extends GenericDao{
+	List listByAccountId(long id);
+	boolean isNameInUse(Long accountId, String name);
+    InstanceGroupVO findByAccountAndName(Long accountId, String name);
+    
+    /**
+     * Updates name for the vm
+     * @param id - group id
+     * @param name
+     */
+    void updateVmGroup(long id, String name);
+
+}
diff --git a/core/src/com/cloud/vm/dao/InstanceGroupDaoImpl.java b/core/src/com/cloud/vm/dao/InstanceGroupDaoImpl.java
new file mode 100644
index 00000000000..ad38184c6e8
--- /dev/null
+++ b/core/src/com/cloud/vm/dao/InstanceGroupDaoImpl.java
@@ -0,0 +1,61 @@
+package com.cloud.vm.dao;
+
+import java.util.List;
+
+import javax.ejb.Local;
+
+import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.SearchBuilder;
+import com.cloud.utils.db.SearchCriteria;
+import com.cloud.vm.InstanceGroupVO;
+
+@Local (value={InstanceGroupDao.class})
+public class InstanceGroupDaoImpl extends GenericDaoBase implements InstanceGroupDao{
+	private SearchBuilder AccountIdNameSearch;
+	protected final SearchBuilder AccountSearch;
+	
+	protected InstanceGroupDaoImpl() {
+        AccountSearch = createSearchBuilder();
+        AccountSearch.and("account", AccountSearch.entity().getAccountId(), SearchCriteria.Op.EQ);
+        AccountSearch.done();
+        
+        AccountIdNameSearch = createSearchBuilder();
+        AccountIdNameSearch.and("accountId", AccountIdNameSearch.entity().getAccountId(), SearchCriteria.Op.EQ);
+        AccountIdNameSearch.and("groupName", AccountIdNameSearch.entity().getName(), SearchCriteria.Op.EQ);
+        AccountIdNameSearch.done();
+        
+	}
+	
+    @Override
+    public boolean isNameInUse(Long accountId, String name) {
+        SearchCriteria sc = createSearchCriteria();
+        sc.addAnd("name", SearchCriteria.Op.EQ, name);
+        if (accountId != null) {
+            sc.addAnd("accountId", SearchCriteria.Op.EQ, accountId);
+        }
+        List vmGroups = listBy(sc);
+        return ((vmGroups != null) && !vmGroups.isEmpty());
+    }
+    
+	@Override
+	public InstanceGroupVO findByAccountAndName(Long accountId, String name) {
+		SearchCriteria sc = AccountIdNameSearch.create();
+		sc.setParameters("accountId", accountId);
+		sc.setParameters("groupName", name);
+		return findOneBy(sc);
+	}
+	
+    @Override
+    public void updateVmGroup(long id, String name) {
+        InstanceGroupVO vo = createForUpdate();
+        vo.setName(name);
+        update(id, vo);
+    }
+    
+    @Override
+    public List listByAccountId(long id) {
+        SearchCriteria sc = AccountSearch.create();
+        sc.setParameters("account", id);
+        return listBy(sc);
+    }
+}
diff --git a/core/src/com/cloud/vm/dao/InstanceGroupVMMapDao.java b/core/src/com/cloud/vm/dao/InstanceGroupVMMapDao.java
new file mode 100644
index 00000000000..14d05cb4f7a
--- /dev/null
+++ b/core/src/com/cloud/vm/dao/InstanceGroupVMMapDao.java
@@ -0,0 +1,12 @@
+package com.cloud.vm.dao;
+
+import java.util.List;
+
+import com.cloud.utils.db.GenericDao;
+import com.cloud.vm.InstanceGroupVMMapVO;
+
+public interface InstanceGroupVMMapDao extends GenericDao{
+	List listByInstanceId(long instanceId);
+	List listByGroupId(long groupId);
+	InstanceGroupVMMapVO findByVmIdGroupId(long instanceId, long groupId);
+}
diff --git a/core/src/com/cloud/vm/dao/InstanceGroupVMMapDaoImpl.java b/core/src/com/cloud/vm/dao/InstanceGroupVMMapDaoImpl.java
new file mode 100644
index 00000000000..7e61dad390a
--- /dev/null
+++ b/core/src/com/cloud/vm/dao/InstanceGroupVMMapDaoImpl.java
@@ -0,0 +1,56 @@
+package com.cloud.vm.dao;
+
+import java.util.List;
+
+import javax.ejb.Local;
+
+import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.SearchBuilder;
+import com.cloud.utils.db.SearchCriteria;
+import com.cloud.vm.InstanceGroupVMMapVO;
+
+@Local(value={InstanceGroupVMMapDao.class})
+public class InstanceGroupVMMapDaoImpl extends GenericDaoBase implements InstanceGroupVMMapDao{
+	
+	private SearchBuilder ListByVmId;
+	private SearchBuilder ListByGroupId;
+    private SearchBuilder ListByVmIdGroupId;
+	
+	protected InstanceGroupVMMapDaoImpl() {
+		ListByVmId  = createSearchBuilder();
+		ListByVmId.and("instanceId", ListByVmId.entity().getInstanceId(), SearchCriteria.Op.EQ);
+		ListByVmId.done();
+		
+		ListByGroupId  = createSearchBuilder();
+		ListByGroupId.and("groupId", ListByGroupId.entity().getGroupId(), SearchCriteria.Op.EQ);
+		ListByGroupId.done();
+		
+        ListByVmIdGroupId  = createSearchBuilder();
+        ListByVmIdGroupId.and("instanceId", ListByVmIdGroupId.entity().getInstanceId(), SearchCriteria.Op.EQ);
+        ListByVmIdGroupId.and("groupId", ListByVmIdGroupId.entity().getGroupId(), SearchCriteria.Op.EQ);
+        ListByVmIdGroupId.done();
+	}
+	
+    @Override
+    public List listByInstanceId(long vmId) {
+        SearchCriteria sc = ListByVmId.create();
+        sc.setParameters("instanceId", vmId);
+        return listBy(sc);
+    }
+    
+    @Override
+    public List listByGroupId(long groupId) {
+        SearchCriteria sc = ListByGroupId.create();
+        sc.setParameters("groupId", groupId);
+        return listBy(sc);
+    }
+    
+	@Override
+	public InstanceGroupVMMapVO findByVmIdGroupId(long instanceId, long groupId) {
+        SearchCriteria sc = ListByVmIdGroupId.create();
+        sc.setParameters("groupId", groupId);
+        sc.setParameters("instanceId", instanceId);
+		return findOneBy(sc);
+	}
+
+}
diff --git a/core/src/com/cloud/vm/dao/SecondaryStorageVmDaoImpl.java b/core/src/com/cloud/vm/dao/SecondaryStorageVmDaoImpl.java
index 2005b324888..d5ff13312b8 100644
--- a/core/src/com/cloud/vm/dao/SecondaryStorageVmDaoImpl.java
+++ b/core/src/com/cloud/vm/dao/SecondaryStorageVmDaoImpl.java
@@ -158,21 +158,21 @@ public class SecondaryStorageVmDaoImpl extends GenericDaoBase sc = DataCenterStatusSearch.create();
         sc.setParameters("states", (Object[])states);
         sc.setParameters("dc", dataCenterId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
     public List getSecStorageVmListInStates(State... states) {
         SearchCriteria sc = StateSearch.create();
         sc.setParameters("states", (Object[])states);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
     public List listByHostId(long hostId) {
         SearchCriteria sc = HostSearch.create();
         sc.setParameters("host", hostId);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
@@ -180,7 +180,7 @@ public class SecondaryStorageVmDaoImpl extends GenericDaoBase sc = HostUpSearch.create();
         sc.setParameters("host", hostId);
         sc.setParameters("states", new Object[] {State.Destroyed, State.Stopped, State.Expunging});        
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
@@ -208,6 +208,6 @@ public class SecondaryStorageVmDaoImpl extends GenericDaoBase listByZoneId(long zoneId) {
 		SearchCriteria sc = ZoneSearch.create();
         sc.setParameters("zone", zoneId);
-        return listActiveBy(sc);
+        return listBy(sc);
 	}
 }
diff --git a/core/src/com/cloud/vm/dao/UserVmDao.java b/core/src/com/cloud/vm/dao/UserVmDao.java
index 3719dfffd17..1ca3aa874f9 100755
--- a/core/src/com/cloud/vm/dao/UserVmDao.java
+++ b/core/src/com/cloud/vm/dao/UserVmDao.java
@@ -62,9 +62,9 @@ public interface UserVmDao extends GenericDao {
     /**
      * Updates display name and group for vm; enables/disables ha
      * @param id vm id.
-     * @param displan name, group and enable for ha
+     * @param displan name and enable for ha
      */
-    void updateVM(long id, String displayName, String group, boolean enable);
+    void updateVM(long id, String displayName, boolean enable);
     
     List findDestroyedVms(Date date);
 
diff --git a/core/src/com/cloud/vm/dao/UserVmDaoImpl.java b/core/src/com/cloud/vm/dao/UserVmDaoImpl.java
index 92f7203e7e1..b75808a6ba7 100755
--- a/core/src/com/cloud/vm/dao/UserVmDaoImpl.java
+++ b/core/src/com/cloud/vm/dao/UserVmDaoImpl.java
@@ -24,13 +24,14 @@ import javax.ejb.Local;
 
 import org.apache.log4j.Logger;
 
-import com.cloud.offering.ServiceOffering;
+import com.cloud.offering.NetworkOffering;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.service.dao.ServiceOfferingDao;
 import com.cloud.uservm.UserVm;
 import com.cloud.utils.component.ComponentLocator;
 import com.cloud.utils.db.Attribute;
 import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.JoinBuilder;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
 import com.cloud.utils.db.UpdateBuilder;
@@ -133,7 +134,7 @@ public class UserVmDaoImpl extends GenericDaoBase implements Use
     	sc.setParameters("account", accountId);
     	sc.setParameters("pod", podId);
     	
-    	return listBy(sc);
+    	return listIncludingRemovedBy(sc);
     }
     
     public List listByAccountAndDataCenter(long accountId, long dcId) {
@@ -141,7 +142,7 @@ public class UserVmDaoImpl extends GenericDaoBase implements Use
         sc.setParameters("account", accountId);
         sc.setParameters("dc", dcId);
         
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
     }
     
     @Override
@@ -154,14 +155,13 @@ public class UserVmDaoImpl extends GenericDaoBase implements Use
             ssc.addOr("state", SearchCriteria.Op.EQ, state.toString());
         }
         sc.addAnd("state", SearchCriteria.Op.SC, ssc);
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
     }
     
     @Override
-    public void updateVM(long id, String displayName, String group, boolean enable) {
+    public void updateVM(long id, String displayName, boolean enable) {
         UserVmVO vo = createForUpdate();
         vo.setDisplayName(displayName);
-        vo.setGroup(group);
         vo.setHaEnabled(enable);
         update(id, vo);
     }
@@ -172,7 +172,7 @@ public class UserVmDaoImpl extends GenericDaoBase implements Use
         
         sc.setParameters("router", routerId);
         
-        return listBy(sc);
+        return listIncludingRemovedBy(sc);
     }
 
     @Override
@@ -232,20 +232,20 @@ public class UserVmDaoImpl extends GenericDaoBase implements Use
     	sc.setParameters("state", State.Destroyed, State.Expunging);
     	sc.setParameters("updateTime", date);
     	
-    	return listActiveBy(sc);
+    	return listBy(sc);
     }
     
     public List listByAccountId(long id) {
         SearchCriteria sc = AccountSearch.create();
         sc.setParameters("account", id);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     public List listByHostId(Long id) {
         SearchCriteria sc = HostSearch.create();
         sc.setParameters("host", id);
         
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
@@ -253,7 +253,7 @@ public class UserVmDaoImpl extends GenericDaoBase implements Use
         SearchCriteria sc = HostUpSearch.create();
         sc.setParameters("host", hostId);
         sc.setParameters("states", new Object[] {State.Destroyed, State.Stopped, State.Expunging});
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     public List listRunningByHostId(long hostId) {
@@ -261,13 +261,13 @@ public class UserVmDaoImpl extends GenericDaoBase implements Use
         sc.setParameters("host", hostId);
         sc.setParameters("state", State.Running);
         
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     public UserVmVO findByName(String name) {
         SearchCriteria sc = NameSearch.create();
         sc.setParameters("name", name);
-        return findOneBy(sc);
+        return findOneIncludingRemovedBy(sc);
     }
 
     @Override
@@ -280,16 +280,16 @@ public class UserVmDaoImpl extends GenericDaoBase implements Use
             AccountDataCenterVirtualSearch = createSearchBuilder();
             AccountDataCenterVirtualSearch.and("account", AccountDataCenterVirtualSearch.entity().getAccountId(), SearchCriteria.Op.EQ);
             AccountDataCenterVirtualSearch.and("dc", AccountDataCenterVirtualSearch.entity().getDataCenterId(), SearchCriteria.Op.EQ);
-            AccountDataCenterVirtualSearch.join("offeringSearch", offeringSearch, AccountDataCenterVirtualSearch.entity().getServiceOfferingId(), offeringSearch.entity().getId());
+            AccountDataCenterVirtualSearch.join("offeringSearch", offeringSearch, AccountDataCenterVirtualSearch.entity().getServiceOfferingId(), offeringSearch.entity().getId(), JoinBuilder.JoinType.INNER);
             AccountDataCenterVirtualSearch.done();
         }
 
         SearchCriteria sc = AccountDataCenterVirtualSearch.create();
         sc.setParameters("account", accountId);
         sc.setParameters("dc", dcId);
-        sc.setJoinParameters("offeringSearch", "guestIpType", ServiceOffering.GuestIpType.Virtualized);
+        sc.setJoinParameters("offeringSearch", "guestIpType", NetworkOffering.GuestIpType.Virtualized);
 
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
 	@Override
@@ -299,7 +299,7 @@ public class UserVmDaoImpl extends GenericDaoBase implements Use
     	sc.setParameters("ip", ipAddress);
     	sc.setParameters("states", new Object[] {State.Destroyed,  State.Expunging});
     	
-    	return listActiveBy(sc);
+    	return listBy(sc);
 	}
 
 	@Override
@@ -309,6 +309,6 @@ public class UserVmDaoImpl extends GenericDaoBase implements Use
         sc.setParameters("accountId", accountId);
         sc.setParameters("guestIpAddress", ipAddress);
 
-        return findOneActiveBy(sc);
+        return findOneBy(sc);
 	}
 }
diff --git a/core/src/com/cloud/vm/dao/VMInstanceDao.java b/core/src/com/cloud/vm/dao/VMInstanceDao.java
index 653445ad331..d496a80f4f3 100644
--- a/core/src/com/cloud/vm/dao/VMInstanceDao.java
+++ b/core/src/com/cloud/vm/dao/VMInstanceDao.java
@@ -80,4 +80,5 @@ public interface VMInstanceDao extends GenericDao {
     List listByHostIdTypes(long hostid, VirtualMachine.Type... types);
     
     List listUpByHostIdTypes(long hostid, VirtualMachine.Type... types);
+    List listByZoneIdAndType(long zoneId, VirtualMachine.Type type);
 }
diff --git a/core/src/com/cloud/vm/dao/VMInstanceDaoImpl.java b/core/src/com/cloud/vm/dao/VMInstanceDaoImpl.java
index 3ec6b2a12dc..813e056c2ea 100644
--- a/core/src/com/cloud/vm/dao/VMInstanceDaoImpl.java
+++ b/core/src/com/cloud/vm/dao/VMInstanceDaoImpl.java
@@ -51,6 +51,7 @@ public class VMInstanceDaoImpl extends GenericDaoBase implem
     protected final SearchBuilder HostSearch;
     protected final SearchBuilder LastHostSearch;
     protected final SearchBuilder ZoneSearch;
+    protected final SearchBuilder ZoneVmTypeSearch;
     protected final SearchBuilder ZoneTemplateNonExpungedSearch;
     protected final SearchBuilder NameLikeSearch;
     protected final SearchBuilder StateChangeSearch;
@@ -79,6 +80,11 @@ public class VMInstanceDaoImpl extends GenericDaoBase implem
         ZoneSearch = createSearchBuilder();
         ZoneSearch.and("zone", ZoneSearch.entity().getDataCenterId(), SearchCriteria.Op.EQ);
         ZoneSearch.done();
+        
+        ZoneVmTypeSearch = createSearchBuilder();
+        ZoneVmTypeSearch.and("zone", ZoneVmTypeSearch.entity().getDataCenterId(), SearchCriteria.Op.EQ);
+        ZoneVmTypeSearch.and("type", ZoneVmTypeSearch.entity().getType(), SearchCriteria.Op.EQ);
+        ZoneVmTypeSearch.done();
 
         ZoneTemplateNonExpungedSearch = createSearchBuilder();
         ZoneTemplateNonExpungedSearch.and("zone", ZoneTemplateNonExpungedSearch.entity().getDataCenterId(), SearchCriteria.Op.EQ);
@@ -130,7 +136,7 @@ public class VMInstanceDaoImpl extends GenericDaoBase implem
     public List findVMInstancesLike(String name) {
         SearchCriteria sc = NameLikeSearch.create();
         sc.setParameters("name", "%" + name + "%");
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -174,7 +180,7 @@ public class VMInstanceDaoImpl extends GenericDaoBase implem
         SearchCriteria sc = HostSearch.create();
         sc.setParameters("host", hostid);
 
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
@@ -182,7 +188,7 @@ public class VMInstanceDaoImpl extends GenericDaoBase implem
         SearchCriteria sc = LastHostSearch.create();
         sc.setParameters("lastHost", hostId);
         
-        return listActiveBy(sc);
+        return listBy(sc);
 	}
 
     @Override
@@ -190,9 +196,18 @@ public class VMInstanceDaoImpl extends GenericDaoBase implem
         SearchCriteria sc = ZoneSearch.create();
         sc.setParameters("zone", zoneId);
 
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
+    @Override
+    public List listByZoneIdAndType(long zoneId, VirtualMachine.Type type) {
+        SearchCriteria sc = ZoneVmTypeSearch.create();
+        sc.setParameters("zone", zoneId);
+        sc.setParameters("type", type.toString());
+        return listBy(sc);
+    }
+    
+    
     @Override
     public List listNonExpungedByZoneAndTemplate(long zoneId, long templateId) {
         SearchCriteria sc = ZoneTemplateNonExpungedSearch.create();
@@ -201,7 +216,7 @@ public class VMInstanceDaoImpl extends GenericDaoBase implem
         sc.setParameters("template", templateId);
         sc.setParameters("state", State.Expunging);
 
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -240,7 +255,7 @@ public class VMInstanceDaoImpl extends GenericDaoBase implem
         SearchCriteria sc = HostIdTypesSearch.create();
         sc.setParameters("hostid", hostid);
         sc.setParameters("types", (Object[]) types);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -249,14 +264,14 @@ public class VMInstanceDaoImpl extends GenericDaoBase implem
         sc.setParameters("hostid", hostid);
         sc.setParameters("types", (Object[]) types);
         sc.setParameters("states", new Object[] {State.Destroyed, State.Stopped, State.Expunging}); 
-        return listActiveBy(sc);
+        return listBy(sc);
     }
     
     @Override
     public List listByTypes(Type... types) {
         SearchCriteria sc = TypesSearch.create();
         sc.setParameters("types", (Object[]) types);
-        return listActiveBy(sc);
+        return listBy(sc);
     }
 
     @Override
@@ -264,7 +279,7 @@ public class VMInstanceDaoImpl extends GenericDaoBase implem
         SearchCriteria sc = IdTypesSearch.create();
         sc.setParameters("id", id);
         sc.setParameters("types", (Object[]) types);
-        return findOneBy(sc);
+        return findOneIncludingRemovedBy(sc);
     }
 
    
diff --git a/core/test/com/cloud/vmware/TestVMWare.java b/core/test/com/cloud/vmware/TestVMWare.java
index 1c7f4aa633b..6ec9c9f3d3e 100644
--- a/core/test/com/cloud/vmware/TestVMWare.java
+++ b/core/test/com/cloud/vmware/TestVMWare.java
@@ -10,22 +10,35 @@ import java.util.Map;
 import org.apache.log4j.xml.DOMConfigurator;
 
 import com.cloud.utils.PropertiesUtil;
-import com.vmware.apputils.AppUtil;
-import com.vmware.vim.ArrayOfManagedObjectReference;
-import com.vmware.vim.DatastoreInfo;
-import com.vmware.vim.DynamicProperty;
-import com.vmware.vim.InvalidProperty;
-import com.vmware.vim.ManagedObjectReference;
-import com.vmware.vim.ObjectContent;
-import com.vmware.vim.ObjectSpec;
-import com.vmware.vim.PropertyFilterSpec;
-import com.vmware.vim.PropertySpec;
-import com.vmware.vim.RuntimeFault;
-import com.vmware.vim.SelectionSpec;
-import com.vmware.vim.TraversalSpec;
+import com.vmware.apputils.version.ExtendedAppUtil;
+import com.vmware.vim25.HostIpConfig;
+import com.vmware.vim25.HostVirtualNicSpec;
+import com.vmware.vim25.HostConfigManager;
+import com.vmware.vim25.HostPortGroupSpec;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualNicManagerNetConfig;
+import com.vmware.vim25.VirtualPCNet32;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualMachineCloneSpec;
+import com.vmware.vim25.VirtualMachineRelocateSpec;
+import com.vmware.vim25.ArrayOfManagedObjectReference;
+import com.vmware.vim25.DatastoreInfo;
+import com.vmware.vim25.DynamicProperty;
+import com.vmware.vim25.InvalidProperty;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.ObjectContent;
+import com.vmware.vim25.ObjectSpec;
+import com.vmware.vim25.PropertyFilterSpec;
+import com.vmware.vim25.PropertySpec;
+import com.vmware.vim25.RuntimeFault;
+import com.vmware.vim25.SelectionSpec;
+import com.vmware.vim25.TraversalSpec;
 
 public class TestVMWare {
-	private static AppUtil cb;
+	private static ExtendedAppUtil cb;
 	
 	private static void setupLog4j() {
 	   File file = PropertiesUtil.findConfigFile("log4j-cloud.xml");
@@ -98,7 +111,7 @@ public class TestVMWare {
       PropertyFilterSpec spec = new PropertyFilterSpec();
       spec.setPropSet(propspecary);
       spec.setObjectSet(new ObjectSpec[] { new ObjectSpec() });
-      spec.getObjectSet(0).setObj(cb.getConnection().getRootFolder());
+      spec.getObjectSet(0).setObj(cb.getServiceConnection3().getRootFolder());
       spec.getObjectSet(0).setSkip(new Boolean(false));
       spec.getObjectSet(0).setSelectSet(
       new SelectionSpec[] { folderTraversalSpec });      
@@ -106,8 +119,8 @@ public class TestVMWare {
       // Recursively get all ManagedEntity ManagedObjectReferences 
       // and the "name" property for all ManagedEntities retrieved
       ObjectContent[] ocary = 
-        cb.getConnection().getService().retrieveProperties(
-        cb.getConnection().getServiceContent().getPropertyCollector(), 
+        cb.getServiceConnection3().getService().retrieveProperties(
+        cb.getServiceConnection3().getServiceContent().getPropertyCollector(), 
            new PropertyFilterSpec[] { spec }
       );
 
@@ -253,15 +266,15 @@ public class TestVMWare {
 	    PropertyFilterSpec filterSpec = new PropertyFilterSpec();
 	    filterSpec.setPropSet(propSpecs);
 	    filterSpec.setObjectSet(new ObjectSpec[] { new ObjectSpec() });
-	    filterSpec.getObjectSet(0).setObj(cb.getConnection().getRootFolder());
+	    filterSpec.getObjectSet(0).setObj(cb.getServiceConnection3().getRootFolder());
 	    filterSpec.getObjectSet(0).setSkip(new Boolean(false));
 	    filterSpec.getObjectSet(0).setSelectSet(
     		new SelectionSpec[] { folderTraversalSpec }
     	);      
 	    
 	    try {
-			ObjectContent[] objContent = cb.getConnection().getService().retrieveProperties(
-				cb.getConnection().getServiceContent().getPropertyCollector(), 
+			ObjectContent[] objContent = cb.getServiceConnection3().getService().retrieveProperties(
+				cb.getServiceConnection3().getServiceContent().getPropertyCollector(), 
 				new PropertyFilterSpec[] { filterSpec } 
 			);
 			printContent(objContent);
@@ -293,7 +306,7 @@ public class TestVMWare {
 	    pSpec.setPathSet(new String[] { "name"} );
 
 	    ObjectSpec oSpec = new ObjectSpec();
-	    oSpec.setObj(cb.getConnection().getRootFolder());
+	    oSpec.setObj(cb.getServiceConnection3().getRootFolder());
 	    oSpec.setSkip(Boolean.TRUE);
 	    oSpec.setSelectSet(new SelectionSpec[] { getFolderRecursiveTraversalSpec() });
 
@@ -301,8 +314,8 @@ public class TestVMWare {
 	    pfSpec.setPropSet(new PropertySpec[] { pSpec });
 	    pfSpec.setObjectSet(new ObjectSpec[] { oSpec });
 	      
-	    ObjectContent[] ocs = cb.getConnection().getService().retrieveProperties(
-            cb.getConnection().getServiceContent().getPropertyCollector(),
+	    ObjectContent[] ocs = cb.getServiceConnection3().getService().retrieveProperties(
+            cb.getServiceConnection3().getServiceContent().getPropertyCollector(),
             new PropertyFilterSpec[] { pfSpec });
 	    
 	    if(ocs != null) {
@@ -336,8 +349,8 @@ public class TestVMWare {
 	    pfSpec.setPropSet(new PropertySpec[] { pSpec });
 	    pfSpec.setObjectSet(new ObjectSpec[] { oSpec });
 	      
-	    ObjectContent[] ocs = cb.getConnection().getService().retrieveProperties(
-            cb.getConnection().getServiceContent().getPropertyCollector(),
+	    ObjectContent[] ocs = cb.getServiceConnection3().getService().retrieveProperties(
+            cb.getServiceConnection3().getServiceContent().getPropertyCollector(),
             new PropertyFilterSpec[] { pfSpec });
 	    
 	    if(ocs != null) {
@@ -380,8 +393,8 @@ public class TestVMWare {
 	    pfSpec.setPropSet(new PropertySpec[] { pSpec });
 	    pfSpec.setObjectSet(new ObjectSpec[] { oSpec });
 	      
-	    ObjectContent[] ocs = cb.getConnection().getService().retrieveProperties(
-            cb.getConnection().getServiceContent().getPropertyCollector(),
+	    ObjectContent[] ocs = cb.getServiceConnection3().getService().retrieveProperties(
+            cb.getServiceConnection3().getServiceContent().getPropertyCollector(),
             new PropertyFilterSpec[] { pfSpec });
 	    
 	    if(ocs != null) {
@@ -416,8 +429,8 @@ public class TestVMWare {
 	    pfSpec.setPropSet(new PropertySpec[] { pSpec });
 	    pfSpec.setObjectSet(new ObjectSpec[] { oSpec });
 	      
-	    ObjectContent[] ocs = cb.getConnection().getService().retrieveProperties(
-            cb.getConnection().getServiceContent().getPropertyCollector(),
+	    ObjectContent[] ocs = cb.getServiceConnection3().getService().retrieveProperties(
+            cb.getServiceConnection3().getServiceContent().getPropertyCollector(),
             new PropertyFilterSpec[] { pfSpec });
 	    
 	    if(ocs != null) {
@@ -461,8 +474,8 @@ public class TestVMWare {
 	    pfSpec.setPropSet(new PropertySpec[] { pSpec });
 	    pfSpec.setObjectSet(new ObjectSpec[] { oSpec });
 	    
-	    ObjectContent[] ocs = cb.getConnection().getService().retrieveProperties(
-            cb.getConnection().getServiceContent().getPropertyCollector(),
+	    ObjectContent[] ocs = cb.getServiceConnection3().getService().retrieveProperties(
+            cb.getServiceConnection3().getServiceContent().getPropertyCollector(),
             new PropertyFilterSpec[] { pfSpec });
 	    
 	    if(ocs != null) {
@@ -495,7 +508,7 @@ public class TestVMWare {
 	    folder2childEntity.setSelectSet(new SelectionSpec[] { recurseFolders });
 
 	    ObjectSpec oSpec = new ObjectSpec();
-	    oSpec.setObj(cb.getConnection().getRootFolder());
+	    oSpec.setObj(cb.getServiceConnection3().getRootFolder());
 	    oSpec.setSkip(Boolean.TRUE);
 	    oSpec.setSelectSet(new SelectionSpec[] { folder2childEntity });
 
@@ -503,8 +516,8 @@ public class TestVMWare {
 	    pfSpec.setPropSet(new PropertySpec[] { pSpec });
 	    pfSpec.setObjectSet(new ObjectSpec[] { oSpec });
 	      
-	    return cb.getConnection().getService().retrieveProperties(
-            cb.getConnection().getServiceContent().getPropertyCollector(),
+	    return cb.getServiceConnection3().getService().retrieveProperties(
+            cb.getServiceConnection3().getServiceContent().getPropertyCollector(),
             new PropertyFilterSpec[] { pfSpec });
 	}
 	
@@ -556,8 +569,8 @@ public class TestVMWare {
 		pfSpec.setPropSet(new PropertySpec[] {pSpec} );
 		pfSpec.setObjectSet(new ObjectSpec[] {oSpec} );
 		
-		ObjectContent[] ocs = cb.getConnection().getService().retrieveProperties(
-	         cb.getConnection().getServiceContent().getPropertyCollector(),
+		ObjectContent[] ocs = cb.getServiceConnection3().getService().retrieveProperties(
+	         cb.getServiceConnection3().getServiceContent().getPropertyCollector(),
 	         new PropertyFilterSpec[] {pfSpec} );
 		
 		if(ocs != null) {
@@ -584,8 +597,8 @@ public class TestVMWare {
 		PropertyFilterSpec pfSpec = new PropertyFilterSpec();
 		pfSpec.setPropSet(new PropertySpec[] {pSpec} );
 		pfSpec.setObjectSet(new ObjectSpec[] {oSpec} );
-		ObjectContent[] ocs = cb.getConnection().getService().retrieveProperties(
-	         cb.getConnection().getServiceContent().getPropertyCollector(),
+		ObjectContent[] ocs = cb.getServiceConnection3().getService().retrieveProperties(
+	         cb.getServiceConnection3().getServiceContent().getPropertyCollector(),
 	         new PropertyFilterSpec[] {pfSpec} );
 
 		Object[] ret = new Object[properties.length];
@@ -613,7 +626,7 @@ public class TestVMWare {
 		morVm.setType("VirtualMachine");
 		morVm.set_value("vm-66");
 		
-		cb.getConnection().getService().powerOnVM_Task(morVm, null);
+		cb.getServiceConnection3().getService().powerOnVM_Task(morVm, null);
 	}
 	
 	private void powerOffVm() throws Exception {
@@ -621,7 +634,192 @@ public class TestVMWare {
 		morVm.setType("VirtualMachine");
 		morVm.set_value("vm-66");
 		
-		cb.getConnection().getService().powerOffVM_Task(morVm);
+		cb.getServiceConnection3().getService().powerOffVM_Task(morVm);
+	}
+	
+	private void createSnapshot() throws Exception {
+		ManagedObjectReference morVm = new ManagedObjectReference();
+		morVm.setType("VirtualMachine");
+		morVm.set_value("vm-66");
+		cb.getServiceConnection3().getService().createSnapshot_Task(morVm, "RunningSnapshotProg", "", false, false);
+	}
+	
+	private void registerTemplate() throws Exception {
+      ManagedObjectReference morFolder = new ManagedObjectReference();
+      morFolder.setType("Folder");
+      morFolder.set_value("group-v3");
+      
+      ManagedObjectReference morHost = new ManagedObjectReference();
+      morHost.setType("HostSystem");
+      morHost.set_value("host-48");
+
+      System.out.println("Begin registerVM_Task");
+      ManagedObjectReference taskmor = cb.getServiceConnection3().getService().registerVM_Task(
+    		  morFolder, "[NFS datastore] Template-Fedora/Template-Fedora.vmtx", "Template-Fedora", true, 
+		  null, morHost);
+      System.out.println("End registerVM_Task");
+
+      String result = cb.getServiceUtil3().waitForTask(taskmor);
+      if (result.equalsIgnoreCase("Sucess")) {
+    	  System.out.println("Registering The Virtual Machine ..........Done");
+      } else {
+    	  System.out.println("Some Exception While Registering The VM");
+      }
+	}
+	
+	private void createVmFromTemplate() throws Exception {
+	     VirtualMachineCloneSpec cloneSpec = new VirtualMachineCloneSpec();
+	     
+	     ManagedObjectReference morDatastore = new ManagedObjectReference();
+	     morDatastore.setType("Datastore");
+	     morDatastore.set_value("datastore-30");
+	     
+	     ManagedObjectReference morHost = new ManagedObjectReference();
+	     morHost.setType("HostSystem");
+	     morHost.set_value("host-48");
+	     
+	     ManagedObjectReference morPool = new ManagedObjectReference();
+	     morPool.setType("ResourcePool");
+	     morPool.set_value("resgroup-41");
+	     
+	     VirtualMachineRelocateSpec relocSpec = new VirtualMachineRelocateSpec();
+	     cloneSpec.setLocation(relocSpec);
+	     cloneSpec.setPowerOn(false);
+	     cloneSpec.setTemplate(false);
+	     
+	     relocSpec.setDatastore(morDatastore);
+	     relocSpec.setHost(morHost);
+	     relocSpec.setPool(morPool);
+	     
+	     ManagedObjectReference morTemplate = new ManagedObjectReference();
+	     morTemplate.setType("VirtualMachine");
+	     morTemplate.set_value("vm-76");
+	     
+	     ManagedObjectReference morFolder = new ManagedObjectReference();
+	     morFolder.setType("Folder");
+	     morFolder.set_value("group-v3");
+	      
+         ManagedObjectReference cloneTask 
+            = cb.getServiceConnection3().getService().cloneVM_Task(morTemplate, morFolder, 
+            	"Fedora-clone-test", cloneSpec);
+         
+         String status = cb.getServiceUtil3().waitForTask(cloneTask);
+         if(status.equalsIgnoreCase("failure")) {
+            System.out.println("Failure -: Virtual Machine cannot be cloned");
+         }
+                  
+         if(status.equalsIgnoreCase("sucess")) {
+            System.out.println("Virtual Machine Cloned  successfully.");
+         }
+	}
+	
+	private void addNic() throws Exception {
+		ManagedObjectReference morVm = new ManagedObjectReference();
+		morVm.setType("VirtualMachine");
+		morVm.set_value("vm-77");
+		
+		ManagedObjectReference morNetwork = new ManagedObjectReference();
+		morNetwork.setType("DistributedVirtualPortgroup");
+		morNetwork.set_value("dvportgroup-56");
+		
+		VirtualDeviceConfigSpec nicSpec = new VirtualDeviceConfigSpec();
+        nicSpec.setOperation(VirtualDeviceConfigSpecOperation.add);
+        VirtualEthernetCard nic =  new VirtualPCNet32();
+        VirtualEthernetCardNetworkBackingInfo nicBacking 
+           = new VirtualEthernetCardNetworkBackingInfo();
+        nicBacking.setDeviceName("Adapter to dSwitch-vlan26");
+        nicBacking.setNetwork(morNetwork);
+        
+        nic.setAddressType("generated");
+        nic.setBacking(nicBacking);
+        nic.setKey(4);
+        nicSpec.setDevice(nic);
+        
+        VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+        VirtualDeviceConfigSpec [] nicSpecArray = {nicSpec};                     
+        vmConfigSpec.setDeviceChange(nicSpecArray);
+        
+        ManagedObjectReference tmor 
+        	= cb.getServiceConnection3().getService().reconfigVM_Task(
+            morVm, vmConfigSpec);
+        
+        String status = cb.getServiceUtil3().waitForTask(tmor);
+        if(status.equalsIgnoreCase("failure")) {
+           System.out.println("Failure -: Virtual Machine cannot be cloned");
+        }
+                 
+        if(status.equalsIgnoreCase("sucess")) {
+           System.out.println("Virtual Machine Cloned  successfully.");
+        }
+	}
+	
+	// add virtual NIC to vmkernel
+	private void addNicToNetwork() throws Exception {
+		ManagedObjectReference morHost = new ManagedObjectReference();
+		morHost.setType("HostSystem");
+		morHost.set_value("host-48");
+		
+        HostPortGroupSpec portgrp = new HostPortGroupSpec();
+        portgrp.setName("VM Network vlan26");
+		
+        Object cmobj = cb.getServiceUtil3().getDynamicProperty(morHost, "configManager");
+        HostConfigManager configMgr = (HostConfigManager)cmobj;
+        ManagedObjectReference nwSystem = configMgr.getNetworkSystem();
+        
+        HostVirtualNicSpec vNicSpec = new HostVirtualNicSpec();
+        HostIpConfig ipConfig = new HostIpConfig();
+        ipConfig.setDhcp(false);
+        ipConfig.setIpAddress("192.168.26.177");
+        ipConfig.setSubnetMask("255.255.255.0");
+        
+        vNicSpec.setIp(ipConfig);
+        vNicSpec.setPortgroup("VM Network vlan26");
+        
+        cb.getServiceConnection3().getService().addVirtualNic(nwSystem, 
+        		"dvPortGroup-vlan26", vNicSpec);
+	}
+	
+	private void createDatacenter() throws Exception {
+		cb.getServiceConnection3().getService().createDatacenter(
+			cb.getServiceConnection3().getRootFolder(), 
+			"cloud.dc.test");
+	}
+	
+	private void getPropertyWithPath() throws Exception {
+		ManagedObjectReference morHost = new ManagedObjectReference();
+		morHost.setType("HostSystem");
+		morHost.set_value("host-161");
+		
+		VirtualNicManagerNetConfig[] netConfigs = (VirtualNicManagerNetConfig[])cb.getServiceUtil3().getDynamicProperty(morHost, "config.virtualNicManagerInfo.netConfig");
+	}
+	
+	private void getHostVMs() throws Exception {
+		ManagedObjectReference morHost = new ManagedObjectReference();
+		morHost.setType("HostSystem");
+		morHost.set_value("host-48");
+		
+		PropertySpec pSpec = new PropertySpec();
+		pSpec.setType("VirtualMachine");
+		pSpec.setPathSet(new String[] { "name", "runtime.powerState", "config.template" });
+		
+	    TraversalSpec host2VmTraversal = new TraversalSpec();
+	    host2VmTraversal.setType("HostSystem");
+	    host2VmTraversal.setPath("vm");
+	    host2VmTraversal.setName("host2VmTraversal");
+
+	    ObjectSpec oSpec = new ObjectSpec();
+	    oSpec.setObj(morHost);
+	    oSpec.setSkip(Boolean.TRUE);
+	    oSpec.setSelectSet(new SelectionSpec[] { host2VmTraversal });
+
+	    PropertyFilterSpec pfSpec = new PropertyFilterSpec();
+	    pfSpec.setPropSet(new PropertySpec[] { pSpec });
+	    pfSpec.setObjectSet(new ObjectSpec[] { oSpec });
+		      
+	    ObjectContent[] ocs = cb.getServiceConnection3().getService().retrieveProperties(
+            cb.getServiceConnection3().getServiceContent().getPropertyCollector(),
+            new PropertyFilterSpec[] { pfSpec });
+	    this.printContent(ocs);
 	}
 	
 	public static void main(String[] args) throws Exception {
@@ -631,21 +829,31 @@ public class TestVMWare {
 		// skip certificate check
 		System.setProperty("axis.socketSecureFactory", "org.apache.axis.components.net.SunFakeTrustSocketFactory");
 		 
-		String serviceUrl = "https://vsphere-1.lab.vmops.com/sdk/vimService";
+		String serviceUrl = "https://" + args[0] + "/sdk/vimService";
+		
 		try {
-			String[] params = new String[] {"--url", serviceUrl, "--username", "Administrator", "--password", "Suite219" };
+			String[] params = new String[] {"--url", serviceUrl, "--username", args[1], "--password", args[2] };
 		 
-			cb = AppUtil.initialize("Connect", params);
+			cb = ExtendedAppUtil.initialize("Connect", params);
 			cb.connect();
 			System.out.println("Connection Succesful.");
 
 			// client.listInventoryFolders();
 			// client.listDataCenters();
-			client.powerOnVm();
-			
+			// client.powerOnVm();
+			// client.createSnapshot();
+			// client.registerTemplate();
+			// client.createVmFromTemplate();
+			// client.addNic();
+			// client.addNicToNetwork();
+
+			// client.createDatacenter();
+			// client.getPropertyWithPath();
+			client.getHostVMs();
+		
 			cb.disConnect();
 		} catch (Exception e) {
-			System.out.println("Failed to connect to " + serviceUrl);
+			e.printStackTrace();
 		}
 	}
 }
diff --git a/daemonize/wscript_build b/daemonize/wscript_build
new file mode 100644
index 00000000000..ce5741a741a
--- /dev/null
+++ b/daemonize/wscript_build
@@ -0,0 +1,7 @@
+if bld.env.DISTRO not in ['Windows','Mac']:
+	# build / install declarations of the daemonization utility - except for Windows
+	bld(
+		name='daemonize',
+		features='cc cprogram',
+		source='daemonize.c',
+		target='cloud-daemonize')
\ No newline at end of file
diff --git a/debian/cloud-agent-scripts.install b/debian/cloud-agent-scripts.install
index 48058f0e9f0..eb0c1589ee0 100644
--- a/debian/cloud-agent-scripts.install
+++ b/debian/cloud-agent-scripts.install
@@ -1,2 +1,28 @@
-/usr/lib/cloud/agent/scripts/*
+/usr/lib/cloud/agent/scripts/installer/*
+/usr/lib/cloud/agent/scripts/network/*
+/usr/lib/cloud/agent/scripts/storage/*
+/usr/lib/cloud/agent/scripts/util/*
+/usr/lib/cloud/agent/scripts/vm/network/*
+/usr/lib/cloud/agent/scripts/vm/pingtest.sh
+/usr/lib/cloud/agent/scripts/vm/storage/*
+/usr/lib/cloud/agent/scripts/vm/hypervisor/kvm/*
+/usr/lib/cloud/agent/scripts/vm/hypervisor/versions.sh
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xen/*
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/hostvmstats.py
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/id_rsa.cloud
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/make_migratable.sh
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/network_info.sh
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/setup_iscsi.sh
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/setupxenserver.sh
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/vmops
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/vmopsSnapshot
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/xcpserver/*
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/xenserver56/cleanup.py
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/xenserver56/ISCSISR.py
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/xenserver56/LUNperVDI.py
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/xenserver56/nfs.py
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/xenserver56/NFSSR.py
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/xenserver56/patch
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/xenserver56/scsiutil.py
 /usr/lib/cloud/agent/vms/systemvm.zip
+/usr/lib/cloud/agent/vms/systemvm.iso
diff --git a/debian/cloud-client.postinst b/debian/cloud-client.postinst
index ce3ebc3da6d..af731f19be7 100644
--- a/debian/cloud-client.postinst
+++ b/debian/cloud-client.postinst
@@ -17,8 +17,6 @@ case "$1" in
 		chgrp cloud $i
 	done
 
-	test -f /var/lib/cloud/management/.ssh/id_rsa || su - cloud -c 'yes "" | ssh-keygen -t rsa -q -N ""' < /dev/null
-
 	for i in /etc/cloud/management/db.properties
 	do
 		chmod 0640 $i
diff --git a/debian/cloud-premium.install b/debian/cloud-premium.install
index 8e7bba9de55..db394b490fa 100644
--- a/debian/cloud-premium.install
+++ b/debian/cloud-premium.install
@@ -4,6 +4,12 @@
 /etc/cloud/management/components-premium.xml
 /usr/share/cloud/setup/create-database-premium.sql
 /usr/share/cloud/setup/create-schema-premium.sql
-/usr/lib/cloud/agent/scripts/vm/hypervisor/xen/*
-/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/*
-/usr/lib/cloud/agent/vms/systemvm-premium.zip
+/usr/lib/cloud/agent/vms/systemvm-premium.iso
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/check_heartbeat.sh
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/find_bond.sh
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/launch_hb.sh
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/setup_heartbeat_sr.sh
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/vmopspremium
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/xenheartbeat.sh
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/xenserver56/patch-premium
+/usr/lib/cloud/agent/scripts/vm/hypervisor/xenserver/xs_cleanup.sh
diff --git a/debian/cloud-setup.install b/debian/cloud-setup.install
index 542cf84199e..ff795672cfe 100644
--- a/debian/cloud-setup.install
+++ b/debian/cloud-setup.install
@@ -4,11 +4,15 @@
 /usr/share/cloud/setup/create-index-fk.sql
 /usr/share/cloud/setup/create-schema.sql
 /usr/share/cloud/setup/server-setup.sql
-/usr/share/cloud/setup/templates.kvm.sql
-/usr/share/cloud/setup/templates.xenserver.sql
+/usr/share/cloud/setup/templates.*.sql
 /usr/share/cloud/setup/deploy-db-dev.sh
 /usr/share/cloud/setup/server-setup.xml
 /usr/share/cloud/setup/data-20to21.sql
 /usr/share/cloud/setup/index-20to21.sql
+/usr/share/cloud/setup/index-212to213.sql
 /usr/share/cloud/setup/postprocess-20to21.sql
 /usr/share/cloud/setup/schema-20to21.sql
+/usr/share/cloud/setup/schema-level.sql
+/usr/share/cloud/setup/schema-21to22.sql
+/usr/share/cloud/setup/data-21to22.sql
+/usr/share/cloud/setup/index-21to22.sql
diff --git a/debian/cloud-utils.install b/debian/cloud-utils.install
index 14204856b57..b552626efcd 100644
--- a/debian/cloud-utils.install
+++ b/debian/cloud-utils.install
@@ -3,3 +3,5 @@
 /usr/share/doc/cloud/sccs-info
 /usr/share/doc/cloud/version-info
 /usr/share/doc/cloud/configure-info
+/usr/bin/cloud-sccs
+/usr/bin/cloud-gitrevs
diff --git a/debian/control b/debian/control
index ef4e379e20d..91a8186c662 100644
--- a/debian/control
+++ b/debian/control
@@ -2,7 +2,7 @@ Source: cloud
 Section: libs
 Priority: extra
 Maintainer: Manuel Amador (Rudd-O) 
-Build-Depends: debhelper (>= 7), openjdk-6-jdk, tomcat6, libws-commons-util-java, libcommons-dbcp-java, libcommons-collections-java, libcommons-httpclient-java, libservlet2.5-java
+Build-Depends: debhelper (>= 7), openjdk-6-jdk, tomcat6, libws-commons-util-java, libcommons-dbcp-java, libcommons-collections-java, libcommons-httpclient-java, libservlet2.5-java, genisoimage, python-mysqldb
 Standards-Version: 3.8.1
 Homepage: http://techcenter.cloud.com/software/cloudstack
 
@@ -22,7 +22,7 @@ Provides: vmops-utils
 Conflicts: vmops-utils
 Replaces: vmops-utils
 Architecture: any
-Depends: openjdk-6-jre
+Depends: openjdk-6-jre, python
 Description: Cloud.com utility library
  The Cloud.com utility libraries provide a set of Java classes used
  in the Cloud.com Cloud Stack.
@@ -128,7 +128,7 @@ Provides: vmops-setup
 Conflicts: vmops-setup
 Replaces: vmops-setup
 Architecture: any
-Depends: openjdk-6-jre, python, cloud-utils (= ${source:Version}), mysql-client, cloud-deps (= ${source:Version}), cloud-server (= ${source:Version}), cloud-python (= ${source:Version}), python-mysqldb
+Depends: openjdk-6-jre, python, cloud-utils (= ${source:Version}), cloud-deps (= ${source:Version}), cloud-server (= ${source:Version}), cloud-python (= ${source:Version}), python-mysqldb
 Description: Cloud.com client
  The Cloud.com setup tools let you set up your Management Server and Usage Server.
 
diff --git a/debian/rules b/debian/rules
index c99b62b85a7..4f0fa109a82 100755
--- a/debian/rules
+++ b/debian/rules
@@ -91,7 +91,7 @@ binary-common:
 	dh_testdir
 	dh_testroot
 	dh_installchangelogs 
-	dh_installdocs -A README INSTALL HACKING README.html
+	dh_installdocs -A README.html
 #	dh_installexamples
 #	dh_installmenu
 #	dh_installdebconf
diff --git a/deps/wscript_build b/deps/wscript_build
new file mode 100644
index 00000000000..11697814dd0
--- /dev/null
+++ b/deps/wscript_build
@@ -0,0 +1 @@
+bld.install_files('${JAVADIR}','*.jar')
diff --git a/patches/kvm/etc/hosts b/patches/kvm/etc/hosts
deleted file mode 100755
index 28486e43ca8..00000000000
--- a/patches/kvm/etc/hosts
+++ /dev/null
@@ -1,2 +0,0 @@
-# @VERSION@
-10.1.1.1 gateway
diff --git a/patches/kvm/etc/init.d/domr_webserver b/patches/kvm/etc/init.d/domr_webserver
deleted file mode 100755
index fef8ee25321..00000000000
--- a/patches/kvm/etc/init.d/domr_webserver
+++ /dev/null
@@ -1,7 +0,0 @@
-#!/bin/bash
-# chkconfig: 345 98 02
-# description: Web server that sends passwords to User VMs
-#   This file exists in /etc/init.d/ in the routing domain
-# @VERSION@
-
-bash /root/run_domr_webserver&
diff --git a/patches/kvm/etc/init.d/seteth1 b/patches/kvm/etc/init.d/seteth1
deleted file mode 100755
index 0dd61a77f9b..00000000000
--- a/patches/kvm/etc/init.d/seteth1
+++ /dev/null
@@ -1,223 +0,0 @@
-
-
-
-#! /bin/bash
-# chkconfig: 35 09 90
-# description: pre-boot configuration using boot line parameters 
-#   This file exists in /etc/init.d/ 
-
-replace_in_file() {
-  local filename=$1
-  local keyname=$2
-  local value=$3
-  sed -i /$keyname=/d $filename
-  echo "$keyname=$value" >> $filename
-  return $?
-}
-
-setup_interface() {
-  local intfnum=$1
-  local ip=$2
-  local mask=$3
-  
-  cfg=/etc/sysconfig/network-scripts/ifcfg-eth${intfnum} 
-  replace_in_file ${cfg} IPADDR ${ip}
-  replace_in_file ${cfg} NETMASK ${mask}
-  replace_in_file ${cfg} BOOTPROTO STATIC
-  if [ "$ip" == "0.0.0.0" ]
-  then
-    replace_in_file ${cfg} ONBOOT No
-  else
-    replace_in_file ${cfg} ONBOOT Yes
-  fi
-}
-
-setup_common() {
-  setup_interface "0" $ETH0_IP $ETH0_MASK
-  setup_interface "1" $ETH1_IP $ETH1_MASK
-  setup_interface "2" $ETH2_IP $ETH2_MASK
-  
-  replace_in_file /etc/sysconfig/network GATEWAY $GW
-  replace_in_file /etc/sysconfig/network HOSTNAME $NAME
-  echo "NOZEROCONF=yes" >> /etc/sysconfig/network
-  hostname $NAME
-  
-  #Nameserver
-  if [ -n "$NS1" ]
-  then
-    echo "nameserver $NS1" > /etc/dnsmasq-resolv.conf
-    echo "nameserver $NS1" > /etc/resolv.conf
-  fi
-  
-  if [ -n "$NS2" ]
-  then
-    echo "nameserver $NS2" >> /etc/dnsmasq-resolv.conf
-    echo "nameserver $NS2" >> /etc/resolv.conf
-  fi
-  if [[ -n "$MGMTNET"  && -n "$LOCAL_GW" ]]
-  then
-    echo "$MGMTNET via $LOCAL_GW dev eth1" > /etc/sysconfig/network-scripts/route-eth1
-  fi
-}
-
-setup_router() {
-  setup_common
-  [ -z $DHCP_RANGE ] && DHCP_RANGE=$ETH0_IP
-  if [ -n "$DOMAIN" ]
-  then
-    #send domain name to dhcp clients
-    sed -i s/[#]*dhcp-option=15.*$/dhcp-option=15,\"$DOMAIN\"/ /etc/dnsmasq.conf
-    #DNS server will append $DOMAIN to local queries
-    sed -r -i s/^[#]?domain=.*$/domain=$DOMAIN/ /etc/dnsmasq.conf
-    #answer all local domain queries
-    sed  -i -e "s/^[#]*local=.*$/local=\/$DOMAIN\//" /etc/dnsmasq.conf
-  fi
-  sed -i -e "s/^dhcp-range=.*$/dhcp-range=$DHCP_RANGE,static/" /etc/dnsmasq.conf
-  sed -i -e "s/^[#]*listen-address=.*$/listen-address=$ETH0_IP/" /etc/dnsmasq.conf
-  sed -i  /gateway/d /etc/hosts
-  echo "$ETH0_IP $NAME" >> /etc/hosts
-  [ -f /etc/httpd/conf/httpd.conf ] && sed -i -e "s/^Listen.*$/Listen $ETH0_IP:80/" /etc/httpd/conf/httpd.conf
-  [ -f /etc/httpd/conf.d/ssl.conf ] && mv /etc/httpd/conf.d/ssl.conf /etc/httpd/conf.d/ssl.conf.bak
-  [ -f /etc/ssh/sshd_config ] && sed -i -e "s/^[#]*ListenAddress.*$/ListenAddress $ETH1_IP/" /etc/ssh/sshd_config
-}
-
-setup_dhcpsrvr() {
-  setup_common
-  [ -z $DHCP_RANGE ] && DHCP_RANGE=$ETH0_IP
-  if [ -n "$DOMAIN" ]
-  then
-    #send domain name to dhcp clients
-    sed -i s/[#]*dhcp-option=15.*$/dhcp-option=15,\"$DOMAIN\"/ /etc/dnsmasq.conf
-    #DNS server will append $DOMAIN to local queries
-    sed -r -i s/^[#]?domain=.*$/domain=$DOMAIN/ /etc/dnsmasq.conf
-    #answer all local domain queries
-    sed  -i -e "s/^[#]*local=.*$/local=\/$DOMAIN\//" /etc/dnsmasq.conf
-  else
-    #delete domain option
-    sed -i /^dhcp-option=15.*$/d /etc/dnsmasq.conf
-    sed -i /^domain=.*$/d /etc/dnsmasq.conf
-    sed  -i -e "/^local=.*$/d" /etc/dnsmasq.conf
-  fi
-  sed -i -e "s/^dhcp-range=.*$/dhcp-range=$DHCP_RANGE,static/" /etc/dnsmasq.conf
-  sed -i -e "s/^[#]*dhcp-option=option:router.*$/dhcp-option=option:router,$GW/" /etc/dnsmasq.conf
-  echo "dhcp-option=6,$NS1,$NS2" >> /etc/dnsmasq.conf
-  sed -i  /gateway/d /etc/hosts
-  echo "$ETH0_IP $NAME" >> /etc/hosts
-  [ -f /etc/httpd/conf/httpd.conf ] && sed -i -e "s/^Listen.*$/Listen $ETH0_IP:80/" /etc/httpd/conf/httpd.conf
-  [ -f /etc/httpd/conf.d/ssl.conf ] && mv /etc/httpd/conf.d/ssl.conf /etc/httpd/conf.d/ssl.conf.bak
-}
-
-setup_secstorage() {
-  setup_common
-  sed -i  /gateway/d /etc/hosts
-  public_ip=$ETH2_IP
-  [ "$ETH2_IP" == "0.0.0.0" ] && public_ip=$ETH1_IP
-  echo "$public_ip $NAME" >> /etc/hosts
-  [ -f /etc/httpd/conf/httpd.conf ] && sed -i -e "s/^Listen.*:80$/Listen $public_ip:80/" /etc/httpd/conf/httpd.conf
-  [ -f /etc/httpd/conf/httpd.conf ] && sed -i -e "s/^Listen.*:443$/Listen $public_ip:443/" /etc/httpd/conf/httpd.conf
-}
-
-setup_console_proxy() {
-  setup_common
-  public_ip=$ETH2_IP
-  [ "$ETH2_IP" == "0.0.0.0" ] && public_ip=$ETH1_IP
-  sed -i  /gateway/d /etc/hosts
-  echo "$public_ip $NAME" >> /etc/hosts
-}
-
-if [ -f /mnt/cmdline ]
-then
-    CMDLINE=$(cat /mnt/cmdline)
-else
-    CMDLINE=$(cat /proc/cmdline)
-fi
-
-TYPE="router"
-
-for i in $CMDLINE
-  do
-    # search for foo=bar pattern and cut out foo
-    KEY=$(echo $i | cut -d= -f1)
-    VALUE=$(echo $i | cut -d= -f2)
-    case $KEY in 
-      eth0ip)
-          ETH0_IP=$VALUE
-          ;;
-      eth1ip)
-          ETH1_IP=$VALUE
-          ;;
-      eth2ip)
-          ETH2_IP=$VALUE
-          ;;
-      gateway)
-          GW=$VALUE
-          ;;
-      eth0mask)
-          ETH0_MASK=$VALUE
-          ;;
-      eth1mask)
-          ETH1_MASK=$VALUE
-          ;;
-      eth2mask)
-          ETH2_MASK=$VALUE
-          ;;
-      dns1)
-          NS1=$VALUE
-          ;;
-      dns2)
-          NS2=$VALUE
-          ;;
-      domain)
-          DOMAIN=$VALUE
-          ;;
-      mgmtcidr)
-          MGMTNET=$VALUE
-          ;;
-      localgw)
-          LOCAL_GW=$VALUE
-          ;;
-      template)
-        TEMPLATE=$VALUE
-      	;;
-      name)
-	NAME=$VALUE
-	;;
-      dhcprange)
-        DHCP_RANGE=$(echo $VALUE | tr ':' ',')
-      	;;
-      type)
-        TYPE=$VALUE	
-	;;
-    esac
-done
-
-case $TYPE in 
-   router)
-       [ "$NAME" == "" ] && NAME=router
-       setup_router
-	;;
-   dhcpsrvr)
-       [ "$NAME" == "" ] && NAME=dhcpsrvr
-       setup_dhcpsrvr
-	;;
-   secstorage)
-       [ "$NAME" == "" ] && NAME=secstorage
-       setup_secstorage;
-	;;
-   consoleproxy)
-       [ "$NAME" == "" ] && NAME=consoleproxy
-       setup_console_proxy;
-	;;
-esac
-
-if [ ! -d /root/.ssh ]
-then
-   mkdir /root/.ssh
-   chmod 700 /root/.ssh
-fi
-if [ -f /mnt/id_rsa.pub ]
-then
-   cat /mnt/id_rsa.pub > /root/.ssh/authorized_keys
-   chmod 600 /root/.ssh/authorized_keys
-fi
-
diff --git a/patches/kvm/etc/init.d/vmops b/patches/kvm/etc/init.d/vmops
deleted file mode 100755
index f0c4b3e2c0f..00000000000
--- a/patches/kvm/etc/init.d/vmops
+++ /dev/null
@@ -1,140 +0,0 @@
-#!/bin/bash
-#
-# vmops		Script to start and stop VMOps console proxy in domR/domP.
-#
-# Author:       Chiradeep Vittal 
-# chkconfig: 2345 99 01
-# description: 	Start up the VMOps agent
-#
-#   This file exists in /etc/init.d/ in the domR/DomP
-#   with a software link /etc/rc.d/rc3.d/S99vmops pointed to it
-#
-# @VERSION@
-
-if [ -f /mnt/cmdline ]
-then
-    CMDLINE=$(cat /mnt/cmdline)
-else
-    CMDLINE=$(cat /proc/cmdline)
-fi
-TEMPLATE="domR"
-for i in $CMDLINE
-  do
-    # search for foo=bar pattern and cut out foo
-    FIRSTPATTERN=$(echo $i | cut -d= -f1)
-    case $FIRSTPATTERN in 
-      template)
-          TEMPLATE=$(echo $i | cut -d= -f2)
-      ;;
-    esac
-done
-
-# Source function library.
-if [ -f /etc/init.d/functions ]
-then
-  . /etc/init.d/functions
-fi
-
-_success() {
-  if [ -f /etc/init.d/functions ]
-  then
-    success
-  else
-    echo "Success"
-  fi
-}
-
-_failure() {
-  if [ -f /etc/init.d/functions ]
-  then
-    failure
-  else
-    echo "Failed"
-  fi
-}
-RETVAL=$?
-VMOPS_HOME="/usr/local/vmops"
-
-# mkdir -p /var/log/vmops
-
-get_pids() {
-  local i
-  for i in $(ps -ef| grep java | grep -v grep | awk '{print $2}'); 
-  do 
-    echo $(pwdx $i) | grep "$VMOPS_HOME" | grep -i console | awk -F: '{print $1}'; 
-  done
-}
-
-start() {
-   if [ "$TEMPLATE" == "domP" ];
-   then
-	   local pid=$(get_pids)
-	   echo -n "Starting VMOps Console Proxy: "
-	   if [ -f $VMOPS_HOME/consoleproxy/run.sh ];
-	   then
-	     if [ "$pid" == "" ]
-	     then
-            if [ ! -d /var/log/vmops ]
-            then
-                mkdir -p /var/log/vmops
-            fi
-            if [ ! -f /var/log/vmops/vmops.out ]
-            then
-                touch /var/log/vmops/vmops.out
-            fi
-	       (cd $VMOPS_HOME/consoleproxy; nohup ./run.sh > /var/log/vmops/vmops.out 2>&1 & )
-	       pid=$(get_pids)
-	       echo $pid > /var/run/vmops.pid 
-	     fi
-	     _success
-	   else
-	     _failure
-	   fi
-	   echo
-   fi
-}
-
-stop() {
-   if [ "$TEMPLATE" == "domP" ];
-   then
-	  local pid
-	  echo -n  "Stopping VMOps agent: "
-	  for pid in $(get_pids)
-	  do
-	    kill $pid
-	  done
-	  _success
-	  echo
-	fi
-}
-
-status() {
-   if [ "$TEMPLATE" == "domP" ];
-   then
-	  local pids=$(get_pids)
-	  if [ "$pids" == "" ]
-	  then
-	    echo "VMOps agent is not running"
-	    return 1
-	  fi
-	  echo "VMOps agent is running: process id: $pids"
-  fi
-  return 0
-}
-
-case "$1" in
-   start) start
-	  ;;
-    stop) stop
- 	  ;;
-    status) status
- 	  ;;
- restart) stop
-          start
- 	  ;;
-       *) echo $"Usage: $0 {start|stop|status|restart}"
-	  exit 1
-	  ;;
-esac
-
-exit $RETVAL
diff --git a/patches/kvm/etc/rc.local b/patches/kvm/etc/rc.local
deleted file mode 100755
index 6a6c84a2117..00000000000
--- a/patches/kvm/etc/rc.local
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/sh
-#
-# This script will be executed *after* all the other init scripts.
-# You can put your own initialization stuff in here if you don't
-# want to do the full Sys V style init stuff.
-# @VERSION@
-
-touch /var/lock/subsys/local
-ethtool -K eth0 tx off
-ethtool -K eth1 tx off
diff --git a/patches/kvm/etc/sysconfig/iptables b/patches/kvm/etc/sysconfig/iptables
deleted file mode 100755
index 5048fb6d670..00000000000
--- a/patches/kvm/etc/sysconfig/iptables
+++ /dev/null
@@ -1,33 +0,0 @@
-# Generated by iptables-save v1.3.8 on Thu Oct  1 18:16:05 2009
-# @VERSION@
-*nat
-:PREROUTING ACCEPT [499:70846]
-:POSTROUTING ACCEPT [1:85]
-:OUTPUT ACCEPT [1:85]
-COMMIT
-# Completed on Thu Oct  1 18:16:06 2009
-# Generated by iptables-save v1.3.8 on Thu Oct  1 18:16:06 2009
-*filter
-#:INPUT DROP [288:42467]
-:FORWARD DROP [0:0]
-:OUTPUT ACCEPT [65:9665]
--A INPUT -i eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT 
--A INPUT -i eth2 -m state --state RELATED,ESTABLISHED -j ACCEPT 
--A INPUT -p icmp -j ACCEPT 
--A INPUT -i eth0 -p udp -m udp --dport 67 -j ACCEPT 
--A INPUT -i eth0 -p udp -m udp --dport 53 -j ACCEPT 
--A INPUT -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT 
--A INPUT -i eth1 -p tcp -m tcp --dport 3922 --tcp-flags FIN,SYN,RST,ACK SYN -j ACCEPT 
--A INPUT -i eth0 -p tcp -m tcp --dport 8080 --tcp-flags FIN,SYN,RST,ACK SYN -j ACCEPT
--A INPUT -p tcp -m tcp --dport 8001 --tcp-flags FIN,SYN,RST,ACK SYN -j ACCEPT
--A INPUT -p tcp -m tcp --dport 443 --tcp-flags FIN,SYN,RST,ACK SYN -j ACCEPT
--A INPUT -p tcp -m tcp --dport 80 --tcp-flags FIN,SYN,RST,ACK SYN -j ACCEPT
--A INPUT -i eth1 -p tcp -m state --state NEW -m tcp --dport 8001 -j ACCEPT
--A INPUT -i eth2 -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
--A INPUT -i eth2 -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
--A FORWARD -i eth0 -o eth1 -j ACCEPT 
--A FORWARD -i eth0 -o eth2 -j ACCEPT 
--A FORWARD -i eth1 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT 
--A FORWARD -i eth2 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT 
-COMMIT
-# Completed on Thu Oct  1 18:16:06 2009
diff --git a/patches/kvm/etc/sysconfig/iptables-config b/patches/kvm/etc/sysconfig/iptables-config
deleted file mode 100644
index c8a02b4a306..00000000000
--- a/patches/kvm/etc/sysconfig/iptables-config
+++ /dev/null
@@ -1,48 +0,0 @@
-# Load additional iptables modules (nat helpers)
-#   Default: -none-
-# Space separated list of nat helpers (e.g. 'ip_nat_ftp ip_nat_irc'), which
-# are loaded after the firewall rules are applied. Options for the helpers are
-# stored in /etc/modprobe.conf.
-IPTABLES_MODULES="ip_conntrack_ftp nf_nat_ftp"
-
-# Unload modules on restart and stop
-#   Value: yes|no,  default: yes
-# This option has to be 'yes' to get to a sane state for a firewall
-# restart or stop. Only set to 'no' if there are problems unloading netfilter
-# modules.
-IPTABLES_MODULES_UNLOAD="yes"
-
-# Save current firewall rules on stop.
-#   Value: yes|no,  default: no
-# Saves all firewall rules to /etc/sysconfig/iptables if firewall gets stopped
-# (e.g. on system shutdown).
-IPTABLES_SAVE_ON_STOP="no"
-
-# Save current firewall rules on restart.
-#   Value: yes|no,  default: no
-# Saves all firewall rules to /etc/sysconfig/iptables if firewall gets
-# restarted.
-IPTABLES_SAVE_ON_RESTART="no"
-
-# Save (and restore) rule and chain counter.
-#   Value: yes|no,  default: no
-# Save counters for rules and chains to /etc/sysconfig/iptables if
-# 'service iptables save' is called or on stop or restart if SAVE_ON_STOP or
-# SAVE_ON_RESTART is enabled.
-IPTABLES_SAVE_COUNTER="no"
-
-# Numeric status output
-#   Value: yes|no,  default: yes
-# Print IP addresses and port numbers in numeric format in the status output.
-IPTABLES_STATUS_NUMERIC="yes"
-
-# Verbose status output
-#   Value: yes|no,  default: yes
-# Print info about the number of packets and bytes plus the "input-" and
-# "outputdevice" in the status output.
-IPTABLES_STATUS_VERBOSE="no"
-
-# Status output with numbered lines
-#   Value: yes|no,  default: yes
-# Print a counter/number for every rule in the status output.
-IPTABLES_STATUS_LINENUMBERS="yes"
diff --git a/patches/kvm/etc/sysconfig/iptables-domp b/patches/kvm/etc/sysconfig/iptables-domp
deleted file mode 100755
index 0a29cd3454f..00000000000
--- a/patches/kvm/etc/sysconfig/iptables-domp
+++ /dev/null
@@ -1,18 +0,0 @@
-# @VERSION@
-*nat
-:PREROUTING ACCEPT [499:70846]
-:POSTROUTING ACCEPT [1:85]
-:OUTPUT ACCEPT [1:85]
-COMMIT
-*filter
-:INPUT DROP [288:42467]
-:FORWARD DROP [0:0]
-:OUTPUT ACCEPT [65:9665]
--A INPUT -i lo  -j ACCEPT 
--A INPUT -i eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT 
--A INPUT -i eth2 -m state --state RELATED,ESTABLISHED -j ACCEPT 
--A INPUT -p icmp -j ACCEPT 
--A INPUT -i eth1 -p tcp -m state --state NEW -m tcp --dport 8001 -j ACCEPT
--A INPUT -i eth2 -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
--A INPUT -i eth2 -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-COMMIT
diff --git a/patches/kvm/etc/sysctl.conf b/patches/kvm/etc/sysctl.conf
deleted file mode 100755
index 69704598684..00000000000
--- a/patches/kvm/etc/sysctl.conf
+++ /dev/null
@@ -1,27 +0,0 @@
-# Kernel sysctl configuration file for Red Hat Linux
-#
-# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
-# sysctl.conf(5) for more details.
-# @VERSION@
-
-# Controls IP packet forwarding
-net.ipv4.ip_forward = 1
-
-# Controls source route verification
-net.ipv4.conf.default.rp_filter = 1
-
-# Do not accept source routing
-net.ipv4.conf.default.accept_source_route = 0
-
-# Controls the System Request debugging functionality of the kernel
-kernel.sysrq = 0
-
-# Controls whether core dumps will append the PID to the core filename.
-# Useful for debugging multi-threaded applications.
-kernel.core_uses_pid = 1
-
-# Controls the use of TCP syncookies
-net.ipv4.tcp_syncookies = 1
-
-# VMOps Rudd-O increase conntrack limits, fix http://bugzilla.lab.vmops.com/show_bug.cgi?id=2008
-net.ipv4.netfilter.ip_conntrack_max=65536
diff --git a/patches/kvm/root/edithosts.sh b/patches/kvm/root/edithosts.sh
deleted file mode 100755
index 06c961cf8a5..00000000000
--- a/patches/kvm/root/edithosts.sh
+++ /dev/null
@@ -1,51 +0,0 @@
-#!/usr/bin/env bash
-# $Id: edithosts.sh 9947 2010-06-25 19:34:24Z manuel $ $HeadURL: svn://svn.lab.vmops.com/repos/vmdev/java/patches/kvm/root/edithosts.sh $
-# edithosts.sh -- edit the dhcphosts file on the routing domain
-# $1 : the mac address
-# $2 : the associated ip address
-# $3 : the hostname
-# @VERSION@
-
-wait_for_dnsmasq () {
-  local _pid=$(/sbin/pidof dnsmasq)
-  for i in 0 1 2 3 4 5 6 7 8 9 10
-  do
-    sleep 1
-    _pid=$(/sbin/pidof dnsmasq)
-    [ "$_pid" != "" ] && break;
-  done
-  [ "$_pid" != "" ] && return 0;
-  echo "edithosts: timed out waiting for dnsmasq to start"
-  return 1
-}
-
-#delete any previous entries from the dhcp hosts file
-sed -i  /$1/d /etc/dhcphosts.txt 
-sed -i  /$2,/d /etc/dhcphosts.txt 
-sed -i  /$3,/d /etc/dhcphosts.txt 
-
-#put in the new entry
-echo "$1,$2,$3,infinite" >>/etc/dhcphosts.txt
-
-#delete leases to supplied mac and ip addresses
-sed -i  /$1/d /var/lib/misc/dnsmasq.leases 
-sed -i  /"$2 "/d /var/lib/misc/dnsmasq.leases 
-sed -i  /"$3 "/d /var/lib/misc/dnsmasq.leases 
-
-#put in the new entry
-echo "0 $1 $2 $3 *" >> /var/lib/misc/dnsmasq.leases
-
-#edit hosts file as well
-sed -i  /"$2 "/d /etc/hosts
-sed -i  /"$3"/d /etc/hosts
-echo "$2 $3" >> /etc/hosts
-
-# send SIGHUP to make dnsmasq re-read files
-pid=$(/sbin/pidof dnsmasq)
-if [ "$pid" != "" ]
-then
-  kill -1 $(/sbin/pidof dnsmasq)
-else
-  wait_for_dnsmasq
-fi
-
diff --git a/patches/kvm/root/run_domr_webserver b/patches/kvm/root/run_domr_webserver
deleted file mode 100755
index 39ee48417e2..00000000000
--- a/patches/kvm/root/run_domr_webserver
+++ /dev/null
@@ -1,17 +0,0 @@
-#!/bin/bash
-# @VERSION@
-
-guestIp=$(ifconfig eth0 | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}')
-
-while true
-do
-	/usr/bin/socat TCP4-LISTEN:8080,reuseaddr,crnl,bind=$guestIp SYSTEM:"/root/send_password_to_domu.sh \"\$SOCAT_PEERADDR\""
-
-	rc=$?
-	if [ $rc -ne 0 ]
-	then
-		logger "Socat failed with error code $rc. Restarting socat..."
-		sleep 3
-	fi
-
-done
diff --git a/patches/shared/var/www/html/latest/.htaccess b/patches/shared/var/www/html/latest/.htaccess
deleted file mode 100644
index c62613d1fda..00000000000
--- a/patches/shared/var/www/html/latest/.htaccess
+++ /dev/null
@@ -1,4 +0,0 @@
-Options +FollowSymLinks
-Options -Indexes
-RewriteEngine On
-
diff --git a/patches/shared/var/www/html/userdata/.htaccess b/patches/shared/var/www/html/userdata/.htaccess
deleted file mode 100644
index 5a928f6da25..00000000000
--- a/patches/shared/var/www/html/userdata/.htaccess
+++ /dev/null
@@ -1 +0,0 @@
-Options -Indexes
diff --git a/patches/systemvm/root/clearUsageRules.sh b/patches/systemvm/root/clearUsageRules.sh
index 2517d42e2e0..1582612fd99 100755
--- a/patches/systemvm/root/clearUsageRules.sh
+++ b/patches/systemvm/root/clearUsageRules.sh
@@ -16,6 +16,8 @@ then
             # remove rules
             iptables -D NETWORK_STATS -i eth0 -o $i > /dev/null;
             iptables -D NETWORK_STATS -i $i -o eth0 > /dev/null;
+            iptables -D NETWORK_STATS -o $pubIf ! -i eth0 -p tcp > /dev/null;
+            iptables -D NETWORK_STATS -i $pubIf ! -o eth0 -p tcp > /dev/null;
         fi
     done
 rm /root/removedVifs
diff --git a/patches/wscript_build b/patches/wscript_build
new file mode 100644
index 00000000000..a28272fb8e4
--- /dev/null
+++ b/patches/wscript_build
@@ -0,0 +1,18 @@
+import os, Utils, glob, re
+
+bld.substitute("*/**",name="patchsubst")
+
+for virttech in Utils.to_list(bld.path.ant_glob("*",dir=True)):
+	if virttech in ["shared","wscript_build"]: continue
+	patchfiles = bld.path.ant_glob('shared/** %s/**'%virttech,src=False,bld=True,dir=False,flat=True)
+	tgen = bld(
+		features  = 'tar',#Utils.tar_up,
+		source = patchfiles,
+		target = '%s-patch.tgz'%virttech,
+		name   = '%s-patch_tgz'%virttech,
+		root = os.path.join("patches",virttech),
+		rename = lambda x: re.sub(".subst$","",x),
+	)
+	if virttech != "xenserver":
+		# xenserver uses the patch.tgz file later to make an ISO, so we do not need to install it
+		bld.install_as("${AGENTLIBDIR}/scripts/vm/hypervisor/%s/patch.tgz"%virttech, "%s-patch.tgz"%virttech)
diff --git a/patches/xenserver/etc/init.d/seteth1 b/patches/xenserver/etc/init.d/seteth1
deleted file mode 100755
index 7c5e8d79c08..00000000000
--- a/patches/xenserver/etc/init.d/seteth1
+++ /dev/null
@@ -1,221 +0,0 @@
-#! /bin/bash
-# chkconfig: 35 09 90
-# description: pre-boot configuration using boot line parameters 
-#   This file exists in /etc/init.d/ 
-
-replace_in_file() {
-  local filename=$1
-  local keyname=$2
-  local value=$3
-  sed -i /$keyname=/d $filename
-  echo "$keyname=$value" >> $filename
-  return $?
-}
-
-setup_interface() {
-  local intfnum=$1
-  local ip=$2
-  local mask=$3
-  
-  cfg=/etc/sysconfig/network-scripts/ifcfg-eth${intfnum} 
-
-  if [ "$BOOTPROTO" == "dhcp" ]
-  then
-    if [ "$intfnum" != "0" ]
-    then
-    	replace_in_file ${cfg} BOOTPROTO dhcp
-    	replace_in_file ${cfg} ONBOOT Yes
-    	sed -i /IPADDR/d ${cfg}
-    	sed -i /NETMASK/d ${cfg}
-    	return
-    fi
-  fi
-
-  replace_in_file ${cfg} IPADDR ${ip}
-  replace_in_file ${cfg} NETMASK ${mask}
-  if [ "$ip" == "0.0.0.0" ]
-  then
-    replace_in_file ${cfg} ONBOOT No
-  else
-    replace_in_file ${cfg} ONBOOT Yes
-  fi
-
-}
-
-setup_common() {
-  setup_interface "0" $ETH0_IP $ETH0_MASK
-  setup_interface "1" $ETH1_IP $ETH1_MASK
-  setup_interface "2" $ETH2_IP $ETH2_MASK
-  
-  if [ "$BOOTPROTO" == "static" ]
-  then
-  	replace_in_file /etc/sysconfig/network GATEWAY $GW
-  else
-    sed -i /GATEWAY/d /etc/sysconfig/network
-  fi
-  replace_in_file /etc/sysconfig/network HOSTNAME $NAME
-  replace_in_file /etc/sysconfig/network NOZEROCONF yes
-  hostname $NAME
-  
-  #Nameserver
-  if [ -n "$NS1" ]
-  then
-    echo "nameserver $NS1" > /etc/dnsmasq-resolv.conf
-    echo "nameserver $NS1" > /etc/resolv.conf
-  fi
-  
-  if [ -n "$NS2" ]
-  then
-    echo "nameserver $NS2" >> /etc/dnsmasq-resolv.conf
-    echo "nameserver $NS2" >> /etc/resolv.conf
-  fi
-  if [[ -n "$MGMTNET"  && -n "$LOCAL_GW" ]]
-  then
-    echo "$MGMTNET via $LOCAL_GW dev eth1" > /etc/sysconfig/network-scripts/route-eth1
-  fi
-}
-
-setup_router() {
-  setup_common
-  [ -z $DHCP_RANGE ] && DHCP_RANGE=$ETH0_IP
-  if [ -n "$DOMAIN" ]
-  then
-    #send domain name to dhcp clients
-    sed -i s/[#]*dhcp-option=15.*$/dhcp-option=15,\"$DOMAIN\"/ /etc/dnsmasq.conf
-    #DNS server will append $DOMAIN to local queries
-    sed -r -i s/^[#]?domain=.*$/domain=$DOMAIN/ /etc/dnsmasq.conf
-    #answer all local domain queries
-    sed  -i -e "s/^[#]*local=.*$/local=\/$DOMAIN\//" /etc/dnsmasq.conf
-  fi
-  sed -i -e "s/^dhcp-range=.*$/dhcp-range=$DHCP_RANGE,static/" /etc/dnsmasq.conf
-  sed -i -e "s/^[#]*listen-address=.*$/listen-address=$ETH0_IP/" /etc/dnsmasq.conf
-  sed -i  /gateway/d /etc/hosts
-  echo "$ETH0_IP $NAME" >> /etc/hosts
-  [ -f /etc/httpd/conf/httpd.conf ] && sed -i -e "s/^Listen.*$/Listen $ETH0_IP:80/" /etc/httpd/conf/httpd.conf
-  [ -f /etc/httpd/conf.d/ssl.conf ] && mv /etc/httpd/conf.d/ssl.conf /etc/httpd/conf.d/ssl.conf.bak
-  [ -f /etc/ssh/sshd_config ] && sed -i -e "s/^[#]*ListenAddress.*$/ListenAddress $ETH1_IP/" /etc/ssh/sshd_config
-}
-
-setup_dhcpsrvr() {
-  setup_common
-  [ -z $DHCP_RANGE ] && DHCP_RANGE=$ETH0_IP
-  [ -z $DOMAIN ] && DOMAIN="cloudnine.internal"
-  if [ -n "$DOMAIN" ]
-  then
-    #send domain name to dhcp clients
-    sed -i s/[#]*dhcp-option=15.*$/dhcp-option=15,\"$DOMAIN\"/ /etc/dnsmasq.conf
-    #DNS server will append $DOMAIN to local queries
-    sed -r -i s/^[#]?domain=.*$/domain=$DOMAIN/ /etc/dnsmasq.conf
-    #answer all local domain queries
-    sed  -i -e "s/^[#]*local=.*$/local=\/$DOMAIN\//" /etc/dnsmasq.conf
-  fi
-  sed -i -e "s/^dhcp-range=.*$/dhcp-range=$DHCP_RANGE,static/" /etc/dnsmasq.conf
-  sed -i -e "s/^[#]*dhcp-option=option:router.*$/dhcp-option=option:router,$GW/" /etc/dnsmasq.conf
-  #for now set up ourself as the dns server as well
-  #echo "dhcp-option=6,$NS1,$NS2" >> /etc/dnsmasq.conf
-  sed -i  /gateway/d /etc/hosts
-  echo "$ETH0_IP $NAME" >> /etc/hosts
-  [ -f /etc/httpd/conf/httpd.conf ] && sed -i -e "s/^Listen.*$/Listen $ETH0_IP:80/" /etc/httpd/conf/httpd.conf
-  [ -f /etc/httpd/conf.d/ssl.conf ] && mv /etc/httpd/conf.d/ssl.conf /etc/httpd/conf.d/ssl.conf.bak
-}
-
-setup_secstorage() {
-  setup_common
-  sed -i  /gateway/d /etc/hosts
-  public_ip=$ETH2_IP
-  [ "$ETH2_IP" == "0.0.0.0" ] && public_ip=$ETH1_IP
-  echo "$public_ip $NAME" >> /etc/hosts
-  [ -f /etc/httpd/conf/httpd.conf ] && sed -i -e "s/^Listen.*:80$/Listen $public_ip:80/" /etc/httpd/conf/httpd.conf
-  [ -f /etc/httpd/conf/httpd.conf ] && sed -i -e "s/^Listen.*:443$/Listen $public_ip:443/" /etc/httpd/conf/httpd.conf
-}
-
-setup_console_proxy() {
-  setup_common
-  public_ip=$ETH2_IP
-  [ "$ETH2_IP" == "0.0.0.0" ] && public_ip=$ETH1_IP
-  sed -i  /gateway/d /etc/hosts
-  echo "$public_ip $NAME" >> /etc/hosts
-}
-
-CMDLINE=$(cat /proc/cmdline)
-TYPE="router"
-BOOTPROTO="static"
-
-for i in $CMDLINE
-  do
-    # search for foo=bar pattern and cut out foo
-    KEY=$(echo $i | cut -d= -f1)
-    VALUE=$(echo $i | cut -d= -f2)
-    case $KEY in 
-      eth0ip)
-          ETH0_IP=$VALUE
-          ;;
-      eth1ip)
-          ETH1_IP=$VALUE
-          ;;
-      eth2ip)
-          ETH2_IP=$VALUE
-          ;;
-      gateway)
-          GW=$VALUE
-          ;;
-      eth0mask)
-          ETH0_MASK=$VALUE
-          ;;
-      eth1mask)
-          ETH1_MASK=$VALUE
-          ;;
-      eth2mask)
-          ETH2_MASK=$VALUE
-          ;;
-      dns1)
-          NS1=$VALUE
-          ;;
-      dns2)
-          NS2=$VALUE
-          ;;
-      domain)
-          DOMAIN=$VALUE
-          ;;
-      mgmtcidr)
-          MGMTNET=$VALUE
-          ;;
-      localgw)
-          LOCAL_GW=$VALUE
-          ;;
-      template)
-        TEMPLATE=$VALUE
-      	;;
-      name)
-	NAME=$VALUE
-	;;
-      dhcprange)
-        DHCP_RANGE=$(echo $VALUE | tr ':' ',')
-      	;;
-      bootproto)
-        BOOTPROTO=$VALUE 
-      	;;
-      type)
-        TYPE=$VALUE	
-	;;
-    esac
-done
-
-case $TYPE in 
-   router)
-       [ "$NAME" == "" ] && NAME=router
-       setup_router
-	;;
-   dhcpsrvr)
-       [ "$NAME" == "" ] && NAME=dhcpsrvr
-       setup_dhcpsrvr
-	;;
-   secstorage)
-       [ "$NAME" == "" ] && NAME=secstorage
-       setup_secstorage;
-	;;
-   consoleproxy)
-       [ "$NAME" == "" ] && NAME=consoleproxy
-       setup_console_proxy;
-	;;
-esac
diff --git a/patches/xenserver/etc/sysconfig/iptables-config b/patches/xenserver/etc/sysconfig/iptables-config
deleted file mode 100644
index c8a02b4a306..00000000000
--- a/patches/xenserver/etc/sysconfig/iptables-config
+++ /dev/null
@@ -1,48 +0,0 @@
-# Load additional iptables modules (nat helpers)
-#   Default: -none-
-# Space separated list of nat helpers (e.g. 'ip_nat_ftp ip_nat_irc'), which
-# are loaded after the firewall rules are applied. Options for the helpers are
-# stored in /etc/modprobe.conf.
-IPTABLES_MODULES="ip_conntrack_ftp nf_nat_ftp"
-
-# Unload modules on restart and stop
-#   Value: yes|no,  default: yes
-# This option has to be 'yes' to get to a sane state for a firewall
-# restart or stop. Only set to 'no' if there are problems unloading netfilter
-# modules.
-IPTABLES_MODULES_UNLOAD="yes"
-
-# Save current firewall rules on stop.
-#   Value: yes|no,  default: no
-# Saves all firewall rules to /etc/sysconfig/iptables if firewall gets stopped
-# (e.g. on system shutdown).
-IPTABLES_SAVE_ON_STOP="no"
-
-# Save current firewall rules on restart.
-#   Value: yes|no,  default: no
-# Saves all firewall rules to /etc/sysconfig/iptables if firewall gets
-# restarted.
-IPTABLES_SAVE_ON_RESTART="no"
-
-# Save (and restore) rule and chain counter.
-#   Value: yes|no,  default: no
-# Save counters for rules and chains to /etc/sysconfig/iptables if
-# 'service iptables save' is called or on stop or restart if SAVE_ON_STOP or
-# SAVE_ON_RESTART is enabled.
-IPTABLES_SAVE_COUNTER="no"
-
-# Numeric status output
-#   Value: yes|no,  default: yes
-# Print IP addresses and port numbers in numeric format in the status output.
-IPTABLES_STATUS_NUMERIC="yes"
-
-# Verbose status output
-#   Value: yes|no,  default: yes
-# Print info about the number of packets and bytes plus the "input-" and
-# "outputdevice" in the status output.
-IPTABLES_STATUS_VERBOSE="no"
-
-# Status output with numbered lines
-#   Value: yes|no,  default: yes
-# Print a counter/number for every rule in the status output.
-IPTABLES_STATUS_LINENUMBERS="yes"
diff --git a/python/incubation/cloud-web-ipallocator b/python/incubation/cloud-web-ipallocator
new file mode 100755
index 00000000000..3a278d29e56
--- /dev/null
+++ b/python/incubation/cloud-web-ipallocator
@@ -0,0 +1,136 @@
+#! /usr/bin/python
+import web
+import socket, struct
+import cloud_utils
+from cloud_utils import Command
+urls = ("/ipallocator", "ipallocator")
+app = web.application(urls, globals())
+
+augtool = Command("augtool")
+service = Command("service")
+class dhcp:
+	_instance = None
+	def __init__(self):
+		self.availIP=[]
+		self.router=None
+		self.netmask=None
+		self.initialized=False
+
+		options = augtool.match("/files/etc/dnsmasq.conf/dhcp-option").stdout.strip()
+		for option in options.splitlines():
+			if option.find("option:router") != -1:
+				self.router = option.split("=")[1].strip().split(",")[1]
+				print self.router
+
+		dhcp_range = augtool.get("/files/etc/dnsmasq.conf/dhcp-range").stdout.strip()
+		dhcp_start = dhcp_range.split("=")[1].strip().split(",")[0]
+		dhcp_end = dhcp_range.split("=")[1].strip().split(",")[1]
+		self.netmask = dhcp_range.split("=")[1].strip().split(",")[2]
+		print dhcp_start, dhcp_end, self.netmask
+
+ 		start_ip_num = self.ipToNum(dhcp_start);
+		end_ip_num =  self.ipToNum(dhcp_end)
+		print start_ip_num, end_ip_num
+	
+		for ip in range(start_ip_num, end_ip_num + 1):
+			self.availIP.append(ip)	
+		print self.availIP[0], self.availIP[len(self.availIP) - 1]	
+		
+		#load the ip already allocated
+		self.reloadAllocatedIP()
+
+	def ipToNum(self, ip):
+		return struct.unpack("!I", socket.inet_aton(ip))[0]
+
+	def numToIp(self, num):
+		return socket.inet_ntoa(struct.pack('!I', num))
+
+	def getFreeIP(self):
+		if len(self.availIP) > 0:
+			ip = self.numToIp(self.availIP[0])
+			self.availIP.remove(self.availIP[0])	
+			return ip
+		else:
+			return None
+
+	def getNetmask(self):
+		return self.netmask
+
+	def getRouter(self):
+		return self.router
+
+	def getInstance():
+		if not dhcp._instance:
+			dhcp._instance = dhcp()
+		return dhcp._instance
+	getInstance = staticmethod(getInstance)
+
+	def reloadAllocatedIP(self):
+		dhcp_hosts = augtool.match("/files/etc/dnsmasq.conf/dhcp-host").stdout.strip().splitlines()
+		
+		for host in dhcp_hosts:
+			if host.find("dhcp-host") != -1:
+				allocatedIP = self.ipToNum(host.split("=")[1].strip().split(",")[1])
+				if allocatedIP in self.availIP:	
+					self.availIP.remove(allocatedIP)
+		
+	def allocateIP(self, mac):
+		newIP = self.getFreeIP()
+		dhcp_host = augtool.match("/files/etc/dnsmasq.conf/dhcp-host").stdout.strip()
+		cnt = len(dhcp_host.splitlines()) + 1
+		script = """set %s %s
+			    save"""%("/files/etc/dnsmasq.conf/dhcp-host[" + str(cnt) + "]", str(mac) + "," + newIP)
+		augtool < script
+		#reset dnsmasq
+		service("dnsmasq", "restart", stdout=None, stderr=None)
+		return newIP
+
+	def releaseIP(self, ip):
+		dhcp_host = augtool.match("/files/etc/dnsmasq.conf/dhcp-host").stdout.strip()
+		path = None
+		for host in dhcp_host.splitlines():
+			if host.find(ip) != -1:
+				path = host.split("=")[0].strip()
+				
+		if path == None:
+			print "Can't find " + str(ip) + " in conf file"
+			return None
+
+		print path
+		script = """rm %s
+			    save"""%(path)
+		augtool < script
+		
+		#reset dnsmasq
+		service("dnsmasq", "restart", stdout=None, stderr=None)
+
+class ipallocator:
+	def GET(self):
+		try:
+			user_data = web.input()
+			command = user_data.command
+			print "Processing: " + command
+
+			dhcpInit = dhcp.getInstance()
+
+			if command == "getIpAddr":
+				mac = user_data.mac
+				zone_id = user_data.dc
+				pod_id = user_data.pod
+				print mac, zone_id, pod_id
+				freeIP = dhcpInit.allocateIP(mac)
+				if not freeIP:
+					return "0,0,0"
+				print "Find an available IP: " + freeIP
+		
+				return freeIP + "," + dhcpInit.getNetmask() + "," + dhcpInit.getRouter()
+			elif command == "releaseIpAddr":
+				ip = user_data.ip
+				zone_id = user_data.dc
+				pod_id = user_data.pod
+				dhcpInit.releaseIP(ip)
+		except:
+			return None
+
+if __name__ == "__main__":
+	app.run()
diff --git a/python/lib/cloud_utils.py b/python/lib/cloud_utils.py
index a8d6cfbd72f..27fed1f86c6 100644
--- a/python/lib/cloud_utils.py
+++ b/python/lib/cloud_utils.py
@@ -1,1135 +1,1164 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-"""Cloud.com Python utility library"""
-
-import sys, os, subprocess, errno, re, time, glob
-import urllib2
-import xml.dom.minidom
-import logging
-import socket
-
-# exit() error constants
-E_GENERIC= 1
-E_NOKVM = 2
-E_NODEFROUTE = 3
-E_DHCP = 4
-E_NOPERSISTENTNET = 5
-E_NETRECONFIGFAILED = 6
-E_VIRTRECONFIGFAILED = 7
-E_FWRECONFIGFAILED = 8
-E_AGENTRECONFIGFAILED = 9
-E_AGENTFAILEDTOSTART = 10
-E_NOFQDN = 11
-E_SELINUXENABLED = 12
-E_USAGE = os.EX_USAGE
-
-E_NEEDSMANUALINTERVENTION = 13
-E_INTERRUPTED = 14
-E_SETUPFAILED = 15
-E_UNHANDLEDEXCEPTION = 16
-E_MISSINGDEP = 17
-
-Unknown = 0
-Fedora = 1
-CentOS = 2
-Ubuntu = 3
-
-IPV4 = 4
-IPV6 = 6
-
-#=================== DISTRIBUTION DETECTION =================
-
-if os.path.exists("/etc/fedora-release"): distro = Fedora
-elif os.path.exists("/etc/centos-release"): distro = CentOS
-elif os.path.exists("/etc/redhat-release") and not os.path.exists("/etc/fedora-release"): distro = CentOS
-elif os.path.exists("/etc/legal") and "Ubuntu" in file("/etc/legal").read(-1): distro = Ubuntu
-else: distro = Unknown
-
-
-# ==================  LIBRARY UTILITY CODE=============
-
-def read_properties(propfile):
-	if not hasattr(propfile,"read"): propfile = file(propfile)
-	properties = propfile.read().splitlines()
-	properties = [ s.strip() for s in properties ]
-	properties = [ s for s in properties if
-			s and
-			not s.startswith("#") and
-			not s.startswith(";") ]
-	#[ logging.debug("Valid config file line: %s",s) for s in properties ]
-	proppairs = [ s.split("=",1) for s in properties ]
-	return dict(proppairs)
-
-def stderr(msgfmt,*args):
-	"""Print a message to stderr, optionally interpolating the arguments into it"""
-	msgfmt += "\n"
-	if args: sys.stderr.write(msgfmt%args)
-	else: sys.stderr.write(msgfmt)
-	sys.stderr.flush()
-
-def exit(errno=E_GENERIC,message=None,*args):
-	"""Exit with an error status code, printing a message to stderr if specified"""
-	if message: stderr(message,*args)
-	sys.exit(errno)
-
-def resolve(host,port):
-	return [ (x[4][0],len(x[4])+2) for x in socket.getaddrinfo(host,port,socket.AF_UNSPEC,socket.SOCK_STREAM, 0, socket.AI_PASSIVE) ]
-	
-def resolves_to_ipv6(host,port):
-	return resolve(host,port)[0][1] == IPV6
-
-###add this to Python 2.4, patching the subprocess module at runtime
-if hasattr(subprocess,"check_call"):
-	from subprocess import CalledProcessError, check_call
-else:
-	class CalledProcessError(Exception):
-		def __init__(self, returncode, cmd):
-			self.returncode = returncode ; self.cmd = cmd
-		def __str__(self): return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
-	subprocess.CalledProcessError = CalledProcessError
-	
-	def check_call(*popenargs, **kwargs):
-		retcode = subprocess.call(*popenargs, **kwargs)
-		cmd = kwargs.get("args")
-		if cmd is None: cmd = popenargs[0]
-		if retcode: raise subprocess.CalledProcessError(retcode, cmd)
-		return retcode
-	subprocess.check_call = check_call
-
-# python 2.4 does not have this
-try:
-	any = any
-	all = all
-except NameError:
-	def any(sequence):
-		for i in sequence:
-			if i: return True
-		return False
-	def all(sequence):
-		for i in sequence:
-			if not i: return False
-		return True
-
-class Command:
-	"""This class simulates a shell command"""
-	def __init__(self,name,parent=None):
-		self.__name = name
-		self.__parent = parent
-	def __getattr__(self,name):
-		if name == "_print": name = "print"
-		return Command(name,self)
-	def __call__(self,*args,**kwargs):
-		cmd = self.__get_recursive_name() + list(args)
-		#print "	",cmd
-		kwargs = dict(kwargs)
-		if "stdout" not in kwargs: kwargs["stdout"] = subprocess.PIPE
-		if "stderr" not in kwargs: kwargs["stderr"] = subprocess.PIPE
-		popen = subprocess.Popen(cmd,**kwargs)
-		m = popen.communicate()
-		ret = popen.wait()
-		if ret:
-			e = CalledProcessError(ret,cmd)
-			e.stdout,e.stderr = m
-			raise e
-		class CommandOutput:
-			def __init__(self,stdout,stderr):
-				self.stdout = stdout
-				self.stderr = stderr
-		return CommandOutput(*m)
-	def __lt__(self,other):
-		cmd = self.__get_recursive_name()
-		#print "	",cmd,"<",other
-		popen = subprocess.Popen(cmd,stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
-		m = popen.communicate(other)
-		ret = popen.wait()
-		if ret:
-			e = CalledProcessError(ret,cmd)
-			e.stdout,e.stderr = m
-			raise e
-		class CommandOutput:
-			def __init__(self,stdout,stderr):
-				self.stdout = stdout
-				self.stderr = stderr
-		return CommandOutput(*m)
-		
-	def __get_recursive_name(self,sep=None):
-		m = self
-		l = []
-		while m is not None:
-			l.append(m.__name)
-			m = m.__parent
-		l.reverse()
-		if sep: return sep.join(l)
-		else: return l
-	def __str__(self):
-		return ''%self.__get_recursive_name(sep=" ")
-		
-	def __repr__(self): return self.__str__()
-
-kvmok = Command("kvm-ok")
-getenforce = Command("/usr/sbin/getenforce")
-ip = Command("ip")
-service = Command("service")
-chkconfig = Command("chkconfig")
-updatercd = Command("update-rc.d")
-ufw = Command("ufw")
-iptables = Command("iptables")
-iptablessave = Command("iptables-save")
-augtool = Command("augtool")
-ifconfig = Command("ifconfig")
-ifdown = Command("ifdown")
-ifup = Command("ifup")
-brctl = Command("brctl")
-uuidgen = Command("uuidgen")
-
-
-def is_service_running(servicename):
-	try:
-		o = service(servicename,"status")
-		if distro is Ubuntu:
-			# status in ubuntu does not signal service status via return code
-			if "start/running" in o.stdout: return True
-			return False
-		else:
-			# retcode 0, service running
-			return True
-	except CalledProcessError,e:
-		# retcode nonzero, service not running
-		return False
-
-
-def stop_service(servicename,force=False):
-	# This function is idempotent.  N number of calls have the same result as N+1 number of calls.
-	if is_service_running(servicename) or force: service(servicename,"stop",stdout=None,stderr=None)
-
-
-def disable_service(servicename):
-	# Stops AND disables the service
-	stop_service(servicename)
-	if distro is Ubuntu:
-		updatercd("-f",servicename,"remove",stdout=None,stderr=None)
-	else:
-		chkconfig("--del",servicename,stdout=None,stderr=None)
-
-
-def start_service(servicename,force=False):
-	# This function is idempotent unless force is True.  N number of calls have the same result as N+1 number of calls.
-	if not is_service_running(servicename) or force: service(servicename,"start",stdout=None,stderr=None)
-
-
-def enable_service(servicename,forcestart=False):
-	# Stops AND disables the service
-	if distro is Ubuntu:
-		updatercd("-f",servicename,"remove",stdout=None,stderr=None)
-		updatercd("-f",servicename,"start","2","3","4","5",".",stdout=None,stderr=None)
-	else:
-		chkconfig("--add",servicename,stdout=None,stderr=None)
-		chkconfig("--level","345",servicename,"on",stdout=None,stderr=None)
-	start_service(servicename,force=forcestart)
-
-
-def replace_line(f,startswith,stanza,always_add=False):
-	lines = [ s.strip() for s in file(f).readlines() ]
-	newlines = []
-	replaced = False
-	for line in lines:
-		if line.startswith(startswith):
-			newlines.append(stanza)
-			replaced = True
-		else: newlines.append(line)
-	if not replaced and always_add: newlines.append(stanza)
-	newlines = [ s + '\n' for s in newlines ]
-	file(f,"w").writelines(newlines)
-
-def replace_or_add_line(f,startswith,stanza):
-	return replace_line(f,startswith,stanza,always_add=True)
-	
-# ==================================== CHECK FUNCTIONS ==========================
-
-# If they return without exception, it's okay.  If they raise a CheckFailed exception, that means a condition
-# (generallly one that needs administrator intervention) was detected.
-
-class CheckFailed(Exception): pass
-
-#check function
-def check_hostname():
-	"""If the hostname is a non-fqdn, fail with CalledProcessError.  Else return 0."""
-	try: check_call(["hostname",'--fqdn'])
-	except CalledProcessError:
-		raise CheckFailed("This machine does not have an FQDN (fully-qualified domain name) for a hostname")
-
-#check function
-def check_kvm():
-	if distro in (Fedora,CentOS):
-		if os.path.exists("/dev/kvm"): return True
-		raise CheckFailed("KVM is not correctly installed on this system, or support for it is not enabled in the BIOS")
-	else:
-		try:
-			kvmok()
-			return True
-		except CalledProcessError:
-			raise CheckFailed("KVM is not correctly installed on this system, or support for it is not enabled in the BIOS")
-		except OSError,e:
-			if e.errno is errno.ENOENT: raise CheckFailed("KVM is not correctly installed on this system, or support for it is not enabled in the BIOS")
-			raise
-		return True
-	raise AssertionError, "check_kvm() should have never reached this part"
-
-def check_cgroups():
-	return glob.glob("/*/cpu.shares")
-
-#check function
-def check_selinux():
-	if distro not in [Fedora,CentOS]: return # no selinux outside of those
-	enforcing = False
-	try:
-		output = getenforce().stdout.strip()
-		if "nforcing" in output:
-			enforcing = True
-		if any ( [ s.startswith("SELINUX=enforcing") for s in file("/etc/selinux/config").readlines() ] ):
-			enforcing = True
-	except (IOError,OSError),e:
-		if e.errno == 2: pass
-		else: raise CheckFailed("An unknown error (%s) took place while checking for SELinux"%str(e))
-	if enforcing:
-		raise CheckFailed("SELinux is set to enforcing, please set it to permissive in /etc/selinux/config, then reboot the machine or type setenforce Permissive, after which you can run this program again.")
-
-
-def preflight_checks(do_check_kvm=True):
-	if distro is Ubuntu:
-		preflight_checks = [
-			(check_hostname,"Checking hostname"),
-		]
-	else:
-		preflight_checks = [
-			(check_hostname,"Checking hostname"),
-			(check_selinux,"Checking if SELinux is disabled"),
-		]
-	#preflight_checks.append( (check_cgroups,"Checking if the control groups /cgroup filesystem is mounted") )
-	if do_check_kvm: preflight_checks.append( (check_kvm,"Checking for KVM") )
-	return preflight_checks
-
-
-# ========================== CONFIGURATION TASKS ================================
-
-# A Task is a function that runs within the context of its run() function that runs the function execute(), which does several things, reporting back to the caller as it goes with the use of yield
-# the done() method ought to return true if the task has run in the past
-# the execute() method must implement the configuration act itself
-# run() wraps the output of execute() within a Starting taskname and a Completed taskname message
-# tasks have a name
-
-class TaskFailed(Exception): pass
-	#def __init__(self,code,msg):
-		#Exception.__init__(self,msg)
-		#self.code = code
-
-class ConfigTask:
-	name = "generic config task"
-	def __init__(self): pass
-	def done(self):
-		"""Returns true if the config task has already been done in the past, false if it hasn't"""
-		return False
-	def execute(self):
-		"""Executes the configuration task.  Must not be run if test() returned true.
-		Must yield strings that describe the steps in the task.
-		Raises TaskFailed if the task failed at some step.
-		"""
-	def run (self):
-		stderr("Starting %s"%self.name)
-		it = self.execute()
-		if not it:
-			pass # not a yielding iterable
-		else:
-			for msg in it: stderr(msg)
-		stderr("Completed %s"%self.name)
-
-
-# ============== these are some configuration tasks ==================
-
-class SetupNetworking(ConfigTask):
-	name = "network setup"
-	def __init__(self,brname):
-		ConfigTask.__init__(self)
-		self.brname = brname
-		self.runtime_state_changed = False
-		self.was_nm_service_running = None
-		self.was_net_service_running = None
-		if distro in (Fedora, CentOS):
-			self.nmservice = 'NetworkManager'
-			self.netservice = 'network'
-		else:
-			self.nmservice = 'network-manager'
-			self.netservice = 'networking'
-		
-		
-	def done(self):
-		try:
-			if distro in (Fedora,CentOS):
-				alreadysetup = augtool._print("/files/etc/sysconfig/network-scripts/ifcfg-%s"%self.brname).stdout.strip()
-			else:
-				alreadysetup = augtool.match("/files/etc/network/interfaces/iface",self.brname).stdout.strip()
-			return alreadysetup
-		except OSError,e:
-			if e.errno is 2: raise TaskFailed("augtool has not been properly installed on this system")
-			raise
-
-	def restore_state(self):
-		if not self.runtime_state_changed: return
-		
-		try:
-			o = ifconfig(self.brname)
-			bridge_exists = True
-		except CalledProcessError,e:
-			print e.stdout + e.stderr
-			bridge_exists = False
-			
-		if bridge_exists:
-			ifconfig(self.brname,"0.0.0.0")
-			if hasattr(self,"old_net_device"):
-				ifdown(self.old_net_device)
-				ifup(self.old_net_device)
-			try: ifdown(self.brname)
-			except CalledProcessError: pass
-			try: ifconfig(self.brname,"down")
-			except CalledProcessError: pass
-			try: brctl("delbr",self.brname)
-			except CalledProcessError: pass
-			try: ifdown("--force",self.brname)
-			except CalledProcessError: pass
-		
-		
-		if self.was_net_service_running is None:
-			# we do nothing
-			pass
-		elif self.was_net_service_running == False:
-			stop_service(self.netservice,force=True)
-			time.sleep(1)
-		else:
-			# we altered service configuration
-			stop_service(self.netservice,force=True)
-			time.sleep(1)
-			try: start_service(self.netservice,force=True)
-			except CalledProcessError,e:
-				if e.returncode == 1: pass
-				else: raise
-			time.sleep(1)
-		
-		if self.was_nm_service_running is None:
-			 # we do nothing
-			 pass
-		elif self.was_nm_service_running == False:
-			stop_service(self.nmservice,force=True)
-			time.sleep(1)
-		else:
-			# we altered service configuration
-			stop_service(self.nmservice,force=True)
-			time.sleep(1)
-			start_service(self.nmservice,force=True)
-			time.sleep(1)
-		
-		self.runtime_state_changed = False
-
-	def execute(self):
-		yield "Determining default route"
-		routes = ip.route().stdout.splitlines()
-		defaultroute = [ x for x in routes if x.startswith("default") ]
-		if not defaultroute: raise TaskFailed("Your network configuration does not have a default route")
-		
-		dev = defaultroute[0].split()[4]
-		yield "Default route assigned to device %s"%dev
-		
-		self.old_net_device = dev
-		
-		if distro in (Fedora, CentOS):
-			inconfigfile = "/".join(augtool.match("/files/etc/sysconfig/network-scripts/*/DEVICE",dev).stdout.strip().split("/")[:-1])
-			if not inconfigfile: raise TaskFailed("Device %s has not been set up in /etc/sysconfig/network-scripts"%dev)
-			pathtoconfigfile = inconfigfile[6:]
-
-		if distro in (Fedora, CentOS):
-			automatic = augtool.match("%s/ONBOOT"%inconfigfile,"yes").stdout.strip()
-		else:
-			automatic = augtool.match("/files/etc/network/interfaces/auto/*/",dev).stdout.strip()
-		if not automatic:
-			if distro is Fedora: raise TaskFailed("Device %s has not been set up in %s as automatic on boot"%dev,pathtoconfigfile)
-			else: raise TaskFailed("Device %s has not been set up in /etc/network/interfaces as automatic on boot"%dev)
-			
-		if distro not in (Fedora , CentOS):
-			inconfigfile = augtool.match("/files/etc/network/interfaces/iface",dev).stdout.strip()
-			if not inconfigfile: raise TaskFailed("Device %s has not been set up in /etc/network/interfaces"%dev)
-
-		if distro in (Fedora, CentOS):
-			isstatic = augtool.match(inconfigfile + "/BOOTPROTO","none").stdout.strip()
-			if not isstatic: isstatic = augtool.match(inconfigfile + "/BOOTPROTO","static").stdout.strip()
-		else:
-			isstatic = augtool.match(inconfigfile + "/method","static").stdout.strip()
-		if not isstatic:
-			if distro in (Fedora, CentOS): raise TaskFailed("Device %s has not been set up as a static device in %s"%(dev,pathtoconfigfile))
-			else: raise TaskFailed("Device %s has not been set up as a static device in /etc/network/interfaces"%dev)
-
-		if is_service_running(self.nmservice):
-			self.was_nm_service_running = True
-			yield "Stopping NetworkManager to avoid automatic network reconfiguration"
-			disable_service(self.nmservice)
-		else:
-			self.was_nm_service_running = False
-			
-		if is_service_running(self.netservice):
-			self.was_net_service_running = True
-		else:
-			self.was_net_service_running = False
-			
-		yield "Creating Cloud bridging device and making device %s member of this bridge"%dev
-
-		if distro in (Fedora, CentOS):
-			ifcfgtext = file(pathtoconfigfile).read()
-			newf = "/etc/sysconfig/network-scripts/ifcfg-%s"%self.brname
-			#def restore():
-				#try: os.unlink(newf)
-				#except OSError,e:
-					#if errno == 2: pass
-					#raise
-				#try: file(pathtoconfigfile,"w").write(ifcfgtext)
-				#except OSError,e: raise
-
-			f = file(newf,"w") ; f.write(ifcfgtext) ; f.flush() ; f.close()
-			innewconfigfile = "/files" + newf
-
-			script = """set %s/DEVICE %s
-set %s/NAME %s
-set %s/BRIDGE_PORTS %s
-set %s/TYPE Bridge
-rm %s/HWADDR
-rm %s/UUID
-rm %s/HWADDR
-rm %s/IPADDR
-rm %s/DEFROUTE
-rm %s/NETMASK
-rm %s/GATEWAY
-rm %s/BROADCAST
-rm %s/NETWORK
-set %s/BRIDGE %s
-save"""%(innewconfigfile,self.brname,innewconfigfile,self.brname,innewconfigfile,dev,
-			innewconfigfile,innewconfigfile,innewconfigfile,innewconfigfile,
-			inconfigfile,inconfigfile,inconfigfile,inconfigfile,inconfigfile,inconfigfile,
-			inconfigfile,self.brname)
-			
-			yield "Executing the following reconfiguration script:\n%s"%script
-			
-			try:
-				returned = augtool < script
-				if "Saved 2 file" not in returned.stdout:
-					print returned.stdout + returned.stderr
-					#restore()
-					raise TaskFailed("Network reconfiguration failed.")
-				else:
-					yield "Network reconfiguration complete"
-			except CalledProcessError,e:
-				#restore()
-				print e.stdout + e.stderr
-				raise TaskFailed("Network reconfiguration failed")
-		else: # Not fedora
-			backup = file("/etc/network/interfaces").read(-1)
-			#restore = lambda: file("/etc/network/interfaces","w").write(backup)
-
-			script = """set %s %s
-set %s %s
-set %s/bridge_ports %s
-save"""%(automatic,self.brname,inconfigfile,self.brname,inconfigfile,dev)
-			
-			yield "Executing the following reconfiguration script:\n%s"%script
-			
-			try:
-				returned = augtool < script
-				if "Saved 1 file" not in returned.stdout:
-					#restore()
-					raise TaskFailed("Network reconfiguration failed.")
-				else:
-					yield "Network reconfiguration complete"
-			except CalledProcessError,e:
-				#restore()
-				print e.stdout + e.stderr
-				raise TaskFailed("Network reconfiguration failed")
-		
-		yield "We are going to restart network services now, to make the network changes take effect.  Hit ENTER when you are ready."
-		raw_input()
-		
-		# if we reach here, then if something goes wrong we should attempt to revert the runinng state
-		# if not, then no point
-		self.runtime_state_changed = True
-		
-		yield "Enabling and restarting non-NetworkManager networking"
-		if distro is Ubuntu: ifup(self.brname,stdout=None,stderr=None)
-		stop_service(self.netservice)
-		try: enable_service(self.netservice,forcestart=True)
-		except CalledProcessError,e:
-			if e.returncode == 1: pass
-			else: raise
-		
-		yield "Verifying that the bridge is up"
-		try:
-			o = ifconfig(self.brname)
-		except CalledProcessError,e:
-			print e.stdout + e.stderr
-			raise TaskFailed("The bridge could not be set up properly")
-		
-		yield "Networking restart done"
-
-
-class SetupCgConfig(ConfigTask):
-	name = "control groups configuration"
-	
-	def done(self):
-		
-		try:
-			return "group virt" in file("/etc/cgconfig.conf","r").read(-1)
-		except IOError,e:
-			if e.errno is 2: raise TaskFailed("cgconfig has not been properly installed on this system")
-			raise
-		
-	def execute(self):
-		cgconfig = file("/etc/cgconfig.conf","r").read(-1)
-		cgconfig = cgconfig + """
-group virt {
-	cpu {
-		cpu.shares = 9216;
-	}
-}
-"""
-		file("/etc/cgconfig.conf","w").write(cgconfig)
-		
-		stop_service("cgconfig")
-		enable_service("cgconfig",forcestart=True)
-
-
-class SetupCgRules(ConfigTask):
-	name = "control group rules setup"
-	cfgline = "root:/usr/sbin/libvirtd	cpu	virt/"
-	
-	def done(self):
-		try:
-			return self.cfgline in file("/etc/cgrules.conf","r").read(-1)
-		except IOError,e:
-			if e.errno is 2: raise TaskFailed("cgrulesd has not been properly installed on this system")
-			raise
-	
-	def execute(self):
-		cgrules = file("/etc/cgrules.conf","r").read(-1)
-		cgrules = cgrules + "\n" + self.cfgline + "\n"
-		file("/etc/cgrules.conf","w").write(cgrules)
-		
-		stop_service("cgred")
-		enable_service("cgred")
-
-
-class SetupCgroupControllers(ConfigTask):
-	name = "qemu cgroup controllers setup"
-	cfgline = "cgroup_controllers = [ \"cpu\" ]"
-	filename = "/etc/libvirt/qemu.conf"
-	
-	def done(self):
-		try:
-			return self.cfgline in file(self.filename,"r").read(-1)
-		except IOError,e:
-			if e.errno is 2: raise TaskFailed("qemu has not been properly installed on this system")
-			raise
-	
-	def execute(self):
-		libvirtqemu = file(self.filename,"r").read(-1)
-		libvirtqemu = libvirtqemu + "\n" + self.cfgline + "\n"
-		file("/etc/libvirt/qemu.conf","w").write(libvirtqemu)
-
-
-class SetupSecurityDriver(ConfigTask):
-	name = "security driver setup"
-	cfgline = "security_driver = \"none\""
-	filename = "/etc/libvirt/qemu.conf"
-	
-	def done(self):
-		try:
-			return self.cfgline in file(self.filename,"r").read(-1)
-		except IOError,e:
-			if e.errno is 2: raise TaskFailed("qemu has not been properly installed on this system")
-			raise
-	
-	def execute(self):
-		libvirtqemu = file(self.filename,"r").read(-1)
-		libvirtqemu = libvirtqemu + "\n" + self.cfgline + "\n"
-		file("/etc/libvirt/qemu.conf","w").write(libvirtqemu)
-
-
-class SetupLibvirt(ConfigTask):
-	name = "libvirt setup"
-	cfgline = "export CGROUP_DAEMON='cpu:/virt'"
-	def done(self):
-		try:
-			if distro in (Fedora,CentOS): 	 libvirtfile = "/etc/sysconfig/libvirtd"
-			elif distro is Ubuntu:	 libvirtfile = "/etc/default/libvirt-bin"
-			else: raise AssertionError, "We should not reach this"
-			return self.cfgline in file(libvirtfile,"r").read(-1)
-		except IOError,e:
-			if e.errno is 2: raise TaskFailed("libvirt has not been properly installed on this system")
-			raise
-	
-	def execute(self):
-		if distro in (Fedora,CentOS): 	 libvirtfile = "/etc/sysconfig/libvirtd"
-		elif distro is Ubuntu:	 libvirtfile = "/etc/default/libvirt-bin"
-		else: raise AssertionError, "We should not reach this"
-		libvirtbin = file(libvirtfile,"r").read(-1)
-		libvirtbin = libvirtbin + "\n" + self.cfgline + "\n"
-		file(libvirtfile,"w").write(libvirtbin)
-		
-		if distro in (CentOS, Fedora):	svc = "libvirtd"
-		else:					svc = "libvirt-bin"
-		stop_service(svc)
-		enable_service(svc)
-
-class SetupLiveMigration(ConfigTask):
-	name = "live migration setup"
-	stanzas = (
-			"listen_tcp=1",
-			'tcp_port="16509"',
-			'auth_tcp="none"',
-			"listen_tls=0",
-	)
-	
-	def done(self):
-		try:
-			lines = [ s.strip() for s in file("/etc/libvirt/libvirtd.conf").readlines() ]
-			if all( [ stanza in lines for stanza in self.stanzas ] ): return True
-		except IOError,e:
-			if e.errno is 2: raise TaskFailed("libvirt has not been properly installed on this system")
-			raise
-	
-	def execute(self):
-		
-		for stanza in self.stanzas:
-			startswith = stanza.split("=")[0] + '='
-			replace_or_add_line("/etc/libvirt/libvirtd.conf",startswith,stanza)
-
-		if distro is Fedora:
-			replace_or_add_line("/etc/sysconfig/libvirtd","LIBVIRTD_ARGS=","LIBVIRTD_ARGS=-l")
-		
-		elif distro is Ubuntu:
-			if os.path.exists("/etc/init/libvirt-bin.conf"):
-				replace_line("/etc/init/libvirt-bin.conf", "exec /usr/sbin/libvirtd","exec /usr/sbin/libvirtd -d -l")
-			else:
-				replace_or_add_line("/etc/default/libvirt-bin","libvirtd_opts=","libvirtd_opts='-l -d'")
-			
-		else:
-			raise AssertionError("Unsupported distribution")
-		
-		if distro in (CentOS, Fedora):	svc = "libvirtd"
-		else:						svc = "libvirt-bin"
-		stop_service(svc)
-		enable_service(svc)
-
-
-class SetupRequiredServices(ConfigTask):
-	name = "required services setup"
-	
-	def done(self):
-		if distro is Fedora:  nfsrelated = "rpcbind nfslock"
-		elif distro is CentOS: nfsrelated = "portmap nfslock"
-		else: return True
-		return all( [ is_service_running(svc) for svc in nfsrelated.split() ] )
-		
-	def execute(self):
-
-		if distro is Fedora:  nfsrelated = "rpcbind nfslock"
-		elif distro is CentOS: nfsrelated = "portmap nfslock"
-		else: raise AssertionError("Unsupported distribution")
-
-		for svc in nfsrelated.split(): enable_service(svc)
-
-
-class SetupFirewall(ConfigTask):
-	name = "firewall setup"
-	
-	def done(self):
-		
-		if distro in (Fedora, CentOS):
-			if not os.path.exists("/etc/sysconfig/iptables"): return True
-			if ":on" not in chkconfig("--list","iptables").stdout: return True
-		else:
-			if "Status: active" not in ufw.status().stdout: return True
-			if not os.path.exists("/etc/ufw/before.rules"): return True
-		rule = "-p tcp -m tcp --dport 16509 -j ACCEPT"
-		if rule in iptablessave().stdout: return True
-		return False
-	
-	def execute(self):
-		ports = "22 1798 16509".split()
-		if distro in (Fedora , CentOS):
-			for p in ports: iptables("-I","INPUT","1","-p","tcp","--dport",p,'-j','ACCEPT')
-			o = service.iptables.save() ; print o.stdout + o.stderr
-		else:
-			for p in ports: ufw.allow(p)
-
-
-class SetupFirewall2(ConfigTask):
-	# this closes bug 4371
-	name = "additional firewall setup"
-	def __init__(self,brname):
-		ConfigTask.__init__(self)
-		self.brname = brname
-	
-	def done(self):
-		
-		if distro in (Fedora, CentOS):
-			if not os.path.exists("/etc/sysconfig/iptables"): return True
-			if ":on" not in chkconfig("--list","iptables").stdout: return True
-			rule = "FORWARD -i %s -o %s -j ACCEPT"%(self.brname,self.brname)
-			if rule in iptablessave().stdout: return True
-			return False
-		else:
-			if "Status: active" not in ufw.status().stdout: return True
-			if not os.path.exists("/etc/ufw/before.rules"): return True
-			rule = "-A ufw-before-forward -i %s -o %s -j ACCEPT"%(self.brname,self.brname)
-			if rule in file("/etc/ufw/before.rules").read(-1): return True
-			return False
-		
-	def execute(self):
-		
-		yield "Permitting traffic in the bridge interface, migration port and for VNC ports"
-		
-		if distro in (Fedora , CentOS):
-			
-			for rule in (
-				"-I FORWARD -i %s -o %s -j ACCEPT"%(self.brname,self.brname),
-				"-I INPUT 1 -p tcp --dport 5900:6100 -j ACCEPT",
-				"-I INPUT 1 -p tcp --dport 49152:49216 -j ACCEPT",
-				):
-				args = rule.split()
-				o = iptables(*args)
-			service.iptables.save(stdout=None,stderr=None)
-			
-		else:
-			
-			rule = "-A ufw-before-forward -i %s -o %s -j ACCEPT"%(self.brname,self.brname)
-			text = file("/etc/ufw/before.rules").readlines()
-			newtext = []
-			for line in text:
-				if line.startswith("COMMIT"):
-					newtext.append(rule + "\n")
-				newtext.append(line)
-			file("/etc/ufw/before.rules","w").writelines(newtext)
-			ufw.allow.proto.tcp("from","any","to","any","port","5900:6100")
-			ufw.allow.proto.tcp("from","any","to","any","port","49152:49216")
-
-			stop_service("ufw")
-			start_service("ufw")
-
-
-# Tasks according to distribution -- at some point we will split them in separate modules
-
-def config_tasks(brname):
-	if distro is CentOS:
-		config_tasks = (
-			SetupNetworking(brname),
-			SetupLibvirt(),
-			SetupRequiredServices(),
-			SetupFirewall(),
-			SetupFirewall2(brname),
-		)
-	elif distro in (Ubuntu,Fedora):
-		config_tasks = (
-			SetupNetworking(brname),
-			SetupCgConfig(),
-			SetupCgRules(),
-			SetupCgroupControllers(),
-			SetupSecurityDriver(),
-			SetupLibvirt(),
-			SetupLiveMigration(),
-			SetupRequiredServices(),
-			SetupFirewall(),
-			SetupFirewall2(brname),
-		)
-	else:
-		raise AssertionError("Unknown distribution")
-	return config_tasks
-
-
-def backup_etc(targetdir):
-	if not targetdir.endswith("/"): targetdir += "/"
-	check_call( ["mkdir","-p",targetdir] )
-	rsynccall = ["rsync","-ax","--delete"] + ["/etc/",targetdir]
-	check_call( rsynccall )
-def restore_etc(targetdir):
-	if not targetdir.endswith("/"): targetdir += "/"
-	rsynccall = ["rsync","-ax","--delete"] + [targetdir,"/etc/"]
-	check_call( rsynccall )
-def remove_backup(targetdir):
-	check_call( ["rm","-rf",targetdir] )
-
-def list_zonespods(host):
-	text = urllib2.urlopen('http://%s:8096/client/api?command=listPods'%host).read(-1)
-	dom = xml.dom.minidom.parseString(text) 
-	x = [ (zonename,podname)
-		for pod in dom.childNodes[0].childNodes  
-		for podname in [ x.childNodes[0].wholeText for x in pod.childNodes if x.tagName == "name" ] 
-		for zonename in  [ x.childNodes[0].wholeText for x in pod.childNodes if x.tagName == "zonename" ]
-		]
-	return x
-	
-def prompt_for_hostpods(zonespods):
-	"""Ask user to select one from those zonespods
-	Returns (zone,pod) or None if the user made the default selection."""
-	while True:
-		stderr("Type the number of the zone and pod combination this host belongs to (hit ENTER to skip this step)")
-		print "  N) ZONE, POD" 
-		print "================"
-		for n,(z,p) in enumerate(zonespods):
-			print "%3d) %s, %s"%(n,z,p)
-		print "================"
-		zoneandpod = raw_input().strip()
-		
-		if not zoneandpod:
-			# we go with default, do not touch anything, just break
-			return None
-		
-		try:
-			# if parsing fails as an int, just vomit and retry
-			zoneandpod = int(zoneandpod)
-			if zoneandpod >= len(zonespods) or zoneandpod < 0: raise ValueError, "%s out of bounds"%zoneandpod
-		except ValueError,e:
-			stderr(str(e))
-			continue # re-ask
-		
-		# oh yeah, the int represents an valid zone and pod index in the array
-		return zonespods[zoneandpod]
-	
-# this configures the agent
-
-def setup_agent_config(configfile):
-	stderr("Examining Agent configuration")
-	fn = configfile
-	text = file(fn).read(-1)
-	lines = [ s.strip() for s in text.splitlines() ]
-	confopts = dict([ m.split("=",1) for m in lines if "=" in m and not m.startswith("#") ])
-	confposes = dict([ (m.split("=",1)[0],n) for n,m in enumerate(lines) if "=" in m and not m.startswith("#") ])
-	
-	if not "guid" in confopts:
-		stderr("Generating GUID for this Agent")
-		confopts['guid'] = uuidgen().stdout.strip()
-	
-	try: host = confopts["host"]
-	except KeyError: host = "localhost"
-	stderr("Please enter the host name of the management server that this agent will connect to: (just hit ENTER to go with %s)",host)
-	newhost = raw_input().strip()
-	if newhost: host = newhost
-	confopts["host"] = host
-	
-	stderr("Querying %s for zones and pods",host)
-	
-	try:
-		x = list_zonespods(confopts['host'])
-		zoneandpod = prompt_for_hostpods(x)
-		if zoneandpod:
-			confopts["zone"],confopts["pod"] = zoneandpod
-			stderr("You selected zone %s pod %s",confopts["zone"],confopts["pod"])
-		else:
-			stderr("Skipped -- using the previous zone %s pod %s",confopts["zone"],confopts["pod"])
-	except (urllib2.URLError,urllib2.HTTPError),e:
-		stderr("Query failed: %s.  Defaulting to zone %s pod %s",confopts["zone"],confopts["pod"])
-
-	for opt,val in confopts.items():
-		line = "=".join([opt,val])
-		if opt not in confposes: lines.append(line)
-		else: lines[confposes[opt]] = line
-	
-	text = "\n".join(lines)
-	file(fn,"w").write(text)
-
-def setup_consoleproxy_config(configfile):
-	stderr("Examining Console Proxy configuration")
-	fn = configfile
-	text = file(fn).read(-1)
-	lines = [ s.strip() for s in text.splitlines() ]
-	confopts = dict([ m.split("=",1) for m in lines if "=" in m and not m.startswith("#") ])
-	confposes = dict([ (m.split("=",1)[0],n) for n,m in enumerate(lines) if "=" in m and not m.startswith("#") ])
-
-	if not "guid" in confopts:
-		stderr("Generating GUID for this Console Proxy")
-		confopts['guid'] = uuidgen().stdout.strip()
-
-	try: host = confopts["host"]
-	except KeyError: host = "localhost"
-	stderr("Please enter the host name of the management server that this console-proxy will connect to: (just hit ENTER to go with %s)",host)
-	newhost = raw_input().strip()
-	if newhost: host = newhost
-	confopts["host"] = host
-
-	stderr("Querying %s for zones and pods",host)
-	
-	try:
-		x = list_zonespods(confopts['host'])
-		zoneandpod = prompt_for_hostpods(x)
-		if zoneandpod:
-			confopts["zone"],confopts["pod"] = zoneandpod
-			stderr("You selected zone %s pod %s",confopts["zone"],confopts["pod"])
-		else:
-			stderr("Skipped -- using the previous zone %s pod %s",confopts["zone"],confopts["pod"])
-	except (urllib2.URLError,urllib2.HTTPError),e:
-		stderr("Query failed: %s.  Defaulting to zone %s pod %s",e,confopts["zone"],confopts["pod"])
-
-	for opt,val in confopts.items():
-		line = "=".join([opt,val])
-		if opt not in confposes: lines.append(line)
-		else: lines[confposes[opt]] = line
-	
-	text = "\n".join(lines)
-	file(fn,"w").write(text)
-
-# =========================== DATABASE MIGRATION SUPPORT CODE ===================
-
-# Migrator, Migratee and Evolvers -- this is the generic infrastructure.
-# To actually implement Cloud.com-specific code, search "Cloud.com-specific evolvers and context"
-
-
-class MigratorException(Exception): pass
-class NoMigrationPath(MigratorException): pass
-class NoMigrator(MigratorException): pass
-
-INITIAL_LEVEL = '-'
-
-class Migrator:
-	"""Migrator class.
-	
-	The migrator gets a list of Python objects, and discovers MigrationSteps in it. It then sorts the steps into a chain, based on the attributes from_level and to_level in each one of the steps.
-	
-	When the migrator's run(context) is called, the chain of steps is applied sequentially on the context supplied to run(), in the order of the chain of steps found at discovery time.  See the documentation for the MigrationStep class for information on how that happens.
-	"""
-	
-	def __init__(self,evolver_source):
-		self.discover_evolvers(evolver_source)
-		self.sort_evolvers()
-		
-	def discover_evolvers(self,source):
-		self.evolvers = []
-		for val in source:
-			if hasattr(val,"from_level") and hasattr(val,"to_level") and val.to_level:
-				self.evolvers.append(val)
-	
-	def sort_evolvers(self):
-		new = []
-		while self.evolvers:
-			if not new:
-				try: idx= [ i for i,s in enumerate(self.evolvers)
-					if s.from_level == INITIAL_LEVEL ][0] # initial evolver
-				except IndexError,e:
-					raise IndexError, "no initial evolver (from_level is None) could be found"
-			else:
-				try: idx= [ i for i,s in enumerate(self.evolvers)
-					if new[-1].to_level == s.from_level ][0]
-				except IndexError,e:
-					raise IndexError, "no evolver could be found to evolve from level %s"%new[-1].to_level
-			new.append(self.evolvers.pop(idx))
-		self.evolvers = new
-	
-	def get_evolver_chain(self):
-		return [ (s.from_level, s.to_level, s) for s in self.evolvers ]
-		
-	def get_evolver_by_starting_level(self,level):
-		try: return [ s for s in self.evolvers if s.from_level == level][0]
-		except IndexError: raise NoMigrator, "No evolver knows how to evolve the database from schema level %r"%level
-	
-	def get_evolver_by_ending_level(self,level):
-		try: return [ s for s in self.evolvers if s.to_level == level][0]
-		except IndexError: raise NoMigrator, "No evolver knows how to evolve the database to schema level %r"%level
-	
-	def run(self, context, dryrun = False, starting_level = None, ending_level = None):
-		"""Runs each one of the steps in sequence, passing the migration context to each. At the end of the process, context.commit() is called to save the changes, or context.rollback() is called if dryrun = True.
-		
-		If starting_level is not specified, then the context.get_schema_level() is used to find out at what level the context is at.  Then starting_level is set to that.
-		
-		If ending_level is not specified, then the evolvers will run till the end of the chain."""
-		
-		assert dryrun is False # NOT IMPLEMENTED, prolly gonna implement by asking the context itself to remember its state
-		
-		starting_level = starting_level or context.get_schema_level() or self.evolvers[0].from_level
-		ending_level = ending_level or self.evolvers[-1].to_level
-		
-		evolution_path = self.evolvers
-		idx = evolution_path.index(self.get_evolver_by_starting_level(starting_level))
-		evolution_path = evolution_path[idx:]
-		try: idx = evolution_path.index(self.get_evolver_by_ending_level(ending_level))
-		except ValueError:
-			raise NoEvolutionPath, "No evolution path from schema level %r to schema level %r" % \
-				(starting_level,ending_level)
-		evolution_path = evolution_path[:idx+1]
-		
-		logging.info("Starting migration on %s"%context)
-		
-		for ec in evolution_path:
-			assert ec.from_level == context.get_schema_level()
-			evolver = ec(context=context)
-			logging.info("%s (from level %s to level %s)",
-				evolver,
-				evolver.from_level,
-				evolver.to_level)
-			#try:
-			evolver.run()
-			#except:
-				#context.rollback()
-				#raise
-			context.set_schema_level(evolver.to_level)
-			#context.commit()
-			logging.info("%s is now at level %s",context,context.get_schema_level())
-		
-		#if dryrun: # implement me with backup and restore
-			#logging.info("Rolling back changes on %s",context)
-			#context.rollback()
-		#else:
-			#logging.info("Committing changes on %s",context)
-			#context.commit()
-		
-		logging.info("Migration finished")
-		
-
-class MigrationStep:
-	"""Base MigrationStep class, aka evolver.
-	
-	You develop your own steps, and then pass a list of those steps to the
-	Migrator instance that will run them in order.
-	
-	When the migrator runs, it will take the list of steps you gave him,
-	and, for each step:
-	
-	a) instantiate it, passing the context you gave to the migrator
-	   into the step's __init__().
-	b) run() the method in the migration step.
-	
-	As you can see, the default MigrationStep constructor makes the passed
-	context available as self.context in the methods of your step.
-	
-	Each step has two member vars that determine in which order they
-	are run, and if they need to run:
-	
-	- from_level = the schema level that the database should be at,
-		       before running the evolver
-		       The value None has special meaning here, it
-		       means the first evolver that should be run if the
-		       database does not have a schema level yet.
-	- to_level =   the schema level number that the database will be at
-		       after the evolver has run
-	"""
-	
-	# Implement these attributes in your steps
-	from_level = None
-	to_level = None
-	
-	def __init__(self,context):
-		self.context = context
-		
-	def run(self):
-		raise NotImplementedError
-
-
-class MigrationContext:
-	def __init__(self): pass
-	def commit(self):raise NotImplementedError
-	def rollback(self):raise NotImplementedError
-	def get_schema_level(self):raise NotImplementedError
-	def set_schema_level(self,l):raise NotImplementedError
-
-
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+"""Cloud.com Python utility library"""
+
+import sys, os, subprocess, errno, re, time, glob
+import urllib2
+import xml.dom.minidom
+import logging
+import socket
+
+# exit() error constants
+E_GENERIC= 1
+E_NOKVM = 2
+E_NODEFROUTE = 3
+E_DHCP = 4
+E_NOPERSISTENTNET = 5
+E_NETRECONFIGFAILED = 6
+E_VIRTRECONFIGFAILED = 7
+E_FWRECONFIGFAILED = 8
+E_AGENTRECONFIGFAILED = 9
+E_AGENTFAILEDTOSTART = 10
+E_NOFQDN = 11
+E_SELINUXENABLED = 12
+try: E_USAGE = os.EX_USAGE
+except AttributeError: E_USAGE = 64
+
+E_NEEDSMANUALINTERVENTION = 13
+E_INTERRUPTED = 14
+E_SETUPFAILED = 15
+E_UNHANDLEDEXCEPTION = 16
+E_MISSINGDEP = 17
+
+Unknown = 0
+Fedora = 1
+CentOS = 2
+Ubuntu = 3
+
+IPV4 = 4
+IPV6 = 6
+
+#=================== DISTRIBUTION DETECTION =================
+
+if os.path.exists("/etc/fedora-release"): distro = Fedora
+elif os.path.exists("/etc/centos-release"): distro = CentOS
+elif os.path.exists("/etc/redhat-release") and not os.path.exists("/etc/fedora-release"): distro = CentOS
+elif os.path.exists("/etc/legal") and "Ubuntu" in file("/etc/legal").read(-1): distro = Ubuntu
+else: distro = Unknown
+
+logFileName=None
+# ==================  LIBRARY UTILITY CODE=============
+def setLogFile(logFile):
+	global logFileName
+	logFileName=logFile
+def read_properties(propfile):
+	if not hasattr(propfile,"read"): propfile = file(propfile)
+	properties = propfile.read().splitlines()
+	properties = [ s.strip() for s in properties ]
+	properties = [ s for s in properties if
+			s and
+			not s.startswith("#") and
+			not s.startswith(";") ]
+	#[ logging.debug("Valid config file line: %s",s) for s in properties ]
+	proppairs = [ s.split("=",1) for s in properties ]
+	return dict(proppairs)
+
+def stderr(msgfmt,*args):
+	"""Print a message to stderr, optionally interpolating the arguments into it"""
+	msgfmt += "\n"
+	if logFileName != None:
+		sys.stderr = open(logFileName, 'a+')
+	if args: sys.stderr.write(msgfmt%args)
+	else: sys.stderr.write(msgfmt)
+
+def exit(errno=E_GENERIC,message=None,*args):
+	"""Exit with an error status code, printing a message to stderr if specified"""
+	if message: stderr(message,*args)
+	sys.exit(errno)
+
+def resolve(host,port):
+	return [ (x[4][0],len(x[4])+2) for x in socket.getaddrinfo(host,port,socket.AF_UNSPEC,socket.SOCK_STREAM, 0, socket.AI_PASSIVE) ]
+	
+def resolves_to_ipv6(host,port):
+	return resolve(host,port)[0][1] == IPV6
+
+###add this to Python 2.4, patching the subprocess module at runtime
+if hasattr(subprocess,"check_call"):
+	from subprocess import CalledProcessError, check_call
+else:
+	class CalledProcessError(Exception):
+		def __init__(self, returncode, cmd):
+			self.returncode = returncode ; self.cmd = cmd
+		def __str__(self): return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
+	subprocess.CalledProcessError = CalledProcessError
+	
+	def check_call(*popenargs, **kwargs):
+		retcode = subprocess.call(*popenargs, **kwargs)
+		cmd = kwargs.get("args")
+		if cmd is None: cmd = popenargs[0]
+		if retcode: raise subprocess.CalledProcessError(retcode, cmd)
+		return retcode
+	subprocess.check_call = check_call
+
+# python 2.4 does not have this
+try:
+	any = any
+	all = all
+except NameError:
+	def any(sequence):
+		for i in sequence:
+			if i: return True
+		return False
+	def all(sequence):
+		for i in sequence:
+			if not i: return False
+		return True
+
+class Command:
+	"""This class simulates a shell command"""
+	def __init__(self,name,parent=None):
+		self.__name = name
+		self.__parent = parent
+	def __getattr__(self,name):
+		if name == "_print": name = "print"
+		return Command(name,self)
+	def __call__(self,*args,**kwargs):
+		cmd = self.__get_recursive_name() + list(args)
+		#print "	",cmd
+		kwargs = dict(kwargs)
+		if "stdout" not in kwargs: kwargs["stdout"] = subprocess.PIPE
+		if "stderr" not in kwargs: kwargs["stderr"] = subprocess.PIPE
+		popen = subprocess.Popen(cmd,**kwargs)
+		m = popen.communicate()
+		ret = popen.wait()
+		if ret:
+			e = CalledProcessError(ret,cmd)
+			e.stdout,e.stderr = m
+			raise e
+		class CommandOutput:
+			def __init__(self,stdout,stderr):
+				self.stdout = stdout
+				self.stderr = stderr
+		return CommandOutput(*m)
+	def __lt__(self,other):
+		cmd = self.__get_recursive_name()
+		#print "	",cmd,"<",other
+		popen = subprocess.Popen(cmd,stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
+		m = popen.communicate(other)
+		ret = popen.wait()
+		if ret:
+			e = CalledProcessError(ret,cmd)
+			e.stdout,e.stderr = m
+			raise e
+		class CommandOutput:
+			def __init__(self,stdout,stderr):
+				self.stdout = stdout
+				self.stderr = stderr
+		return CommandOutput(*m)
+		
+	def __get_recursive_name(self,sep=None):
+		m = self
+		l = []
+		while m is not None:
+			l.append(m.__name)
+			m = m.__parent
+		l.reverse()
+		if sep: return sep.join(l)
+		else: return l
+	def __str__(self):
+		return ''%self.__get_recursive_name(sep=" ")
+		
+	def __repr__(self): return self.__str__()
+
+kvmok = Command("kvm-ok")
+getenforce = Command("/usr/sbin/getenforce")
+ip = Command("ip")
+service = Command("service")
+chkconfig = Command("chkconfig")
+updatercd = Command("update-rc.d")
+ufw = Command("ufw")
+iptables = Command("iptables")
+iptablessave = Command("iptables-save")
+augtool = Command("augtool")
+ifconfig = Command("ifconfig")
+ifdown = Command("ifdown")
+ifup = Command("ifup")
+brctl = Command("brctl")
+uuidgen = Command("uuidgen")
+
+
+def is_service_running(servicename):
+	try:
+		o = service(servicename,"status")
+		if distro is Ubuntu:
+			# status in ubuntu does not signal service status via return code
+			if "start/running" in o.stdout: return True
+			return False
+		else:
+			# retcode 0, service running
+			return True
+	except CalledProcessError,e:
+		# retcode nonzero, service not running
+		return False
+
+
+def stop_service(servicename,force=False):
+	# This function is idempotent.  N number of calls have the same result as N+1 number of calls.
+	if is_service_running(servicename) or force: service(servicename,"stop",stdout=None,stderr=None)
+
+
+def disable_service(servicename):
+	# Stops AND disables the service
+	stop_service(servicename)
+	if distro is Ubuntu:
+		updatercd("-f",servicename,"remove",stdout=None,stderr=None)
+	else:
+		chkconfig("--del",servicename,stdout=None,stderr=None)
+
+
+def start_service(servicename,force=False):
+	# This function is idempotent unless force is True.  N number of calls have the same result as N+1 number of calls.
+	if not is_service_running(servicename) or force: service(servicename,"start",stdout=None,stderr=None)
+
+
+def enable_service(servicename,forcestart=False):
+	# Stops AND disables the service
+	if distro is Ubuntu:
+		updatercd("-f",servicename,"remove",stdout=None,stderr=None)
+		updatercd("-f",servicename,"start","2","3","4","5",".",stdout=None,stderr=None)
+	else:
+		chkconfig("--add",servicename,stdout=None,stderr=None)
+		chkconfig("--level","345",servicename,"on",stdout=None,stderr=None)
+	start_service(servicename,force=forcestart)
+
+
+def replace_line(f,startswith,stanza,always_add=False):
+	lines = [ s.strip() for s in file(f).readlines() ]
+	newlines = []
+	replaced = False
+	for line in lines:
+		if line.startswith(startswith):
+			newlines.append(stanza)
+			replaced = True
+		else: newlines.append(line)
+	if not replaced and always_add: newlines.append(stanza)
+	newlines = [ s + '\n' for s in newlines ]
+	file(f,"w").writelines(newlines)
+
+def replace_or_add_line(f,startswith,stanza):
+	return replace_line(f,startswith,stanza,always_add=True)
+	
+# ==================================== CHECK FUNCTIONS ==========================
+
+# If they return without exception, it's okay.  If they raise a CheckFailed exception, that means a condition
+# (generallly one that needs administrator intervention) was detected.
+
+class CheckFailed(Exception): pass
+
+#check function
+def check_hostname():
+	"""If the hostname is a non-fqdn, fail with CalledProcessError.  Else return 0."""
+	try: check_call(["hostname",'--fqdn'])
+	except CalledProcessError:
+		raise CheckFailed("This machine does not have an FQDN (fully-qualified domain name) for a hostname")
+
+#check function
+def check_kvm():
+	if distro in (Fedora,CentOS):
+		if os.path.exists("/dev/kvm"): return True
+		raise CheckFailed("KVM is not correctly installed on this system, or support for it is not enabled in the BIOS")
+	else:
+		try:
+			kvmok()
+			return True
+		except CalledProcessError:
+			raise CheckFailed("KVM is not correctly installed on this system, or support for it is not enabled in the BIOS")
+		except OSError,e:
+			if e.errno is errno.ENOENT: raise CheckFailed("KVM is not correctly installed on this system, or support for it is not enabled in the BIOS")
+			raise
+		return True
+	raise AssertionError, "check_kvm() should have never reached this part"
+
+def check_cgroups():
+	return glob.glob("/*/cpu.shares")
+
+#check function
+def check_selinux():
+	if distro not in [Fedora,CentOS]: return # no selinux outside of those
+	enforcing = False
+	try:
+		output = getenforce().stdout.strip()
+		if "nforcing" in output:
+			enforcing = True
+		if any ( [ s.startswith("SELINUX=enforcing") for s in file("/etc/selinux/config").readlines() ] ):
+			enforcing = True
+	except (IOError,OSError),e:
+		if e.errno == 2: pass
+		else: raise CheckFailed("An unknown error (%s) took place while checking for SELinux"%str(e))
+	if enforcing:
+		raise CheckFailed("SELinux is set to enforcing, please set it to permissive in /etc/selinux/config, then reboot the machine or type setenforce Permissive, after which you can run this program again.")
+
+
+def preflight_checks(do_check_kvm=True):
+	if distro is Ubuntu:
+		preflight_checks = [
+			(check_hostname,"Checking hostname"),
+		]
+	else:
+		preflight_checks = [
+			(check_hostname,"Checking hostname"),
+			(check_selinux,"Checking if SELinux is disabled"),
+		]
+	#preflight_checks.append( (check_cgroups,"Checking if the control groups /cgroup filesystem is mounted") )
+	if do_check_kvm: preflight_checks.append( (check_kvm,"Checking for KVM") )
+	return preflight_checks
+
+
+# ========================== CONFIGURATION TASKS ================================
+
+# A Task is a function that runs within the context of its run() function that runs the function execute(), which does several things, reporting back to the caller as it goes with the use of yield
+# the done() method ought to return true if the task has run in the past
+# the execute() method must implement the configuration act itself
+# run() wraps the output of execute() within a Starting taskname and a Completed taskname message
+# tasks have a name
+
+class TaskFailed(Exception): pass
+	#def __init__(self,code,msg):
+		#Exception.__init__(self,msg)
+		#self.code = code
+
+class ConfigTask:
+	name = "generic config task"
+	autoMode=False
+	def __init__(self): pass
+	def done(self):
+		"""Returns true if the config task has already been done in the past, false if it hasn't"""
+		return False
+	def execute(self):
+		"""Executes the configuration task.  Must not be run if test() returned true.
+		Must yield strings that describe the steps in the task.
+		Raises TaskFailed if the task failed at some step.
+		"""
+	def run (self):
+		stderr("Starting %s"%self.name)
+		it = self.execute()
+		if not it:
+			pass # not a yielding iterable
+		else:
+			for msg in it: stderr(msg)
+		stderr("Completed %s"%self.name)
+	def setAutoMode(self, autoMode):
+		self.autoMode = autoMode
+	def  isAutoMode(self):
+		return self.autoMode
+
+
+# ============== these are some configuration tasks ==================
+
+class SetupNetworking(ConfigTask):
+	name = "network setup"
+	def __init__(self,brname):
+		ConfigTask.__init__(self)
+		self.brname = brname
+		self.runtime_state_changed = False
+		self.was_nm_service_running = None
+		self.was_net_service_running = None
+		if distro in (Fedora, CentOS):
+			self.nmservice = 'NetworkManager'
+			self.netservice = 'network'
+		else:
+			self.nmservice = 'network-manager'
+			self.netservice = 'networking'
+		
+		
+	def done(self):
+		try:
+			if distro in (Fedora,CentOS):
+				alreadysetup = augtool._print("/files/etc/sysconfig/network-scripts/ifcfg-%s"%self.brname).stdout.strip()
+			else:
+				alreadysetup = augtool.match("/files/etc/network/interfaces/iface",self.brname).stdout.strip()
+			return alreadysetup
+		except OSError,e:
+			if e.errno is 2: raise TaskFailed("augtool has not been properly installed on this system")
+			raise
+
+	def restore_state(self):
+		if not self.runtime_state_changed: return
+		
+		try:
+			o = ifconfig(self.brname)
+			bridge_exists = True
+		except CalledProcessError,e:
+			print e.stdout + e.stderr
+			bridge_exists = False
+			
+		if bridge_exists:
+			ifconfig(self.brname,"0.0.0.0")
+			if hasattr(self,"old_net_device"):
+				ifdown(self.old_net_device)
+				ifup(self.old_net_device)
+			try: ifdown(self.brname)
+			except CalledProcessError: pass
+			try: ifconfig(self.brname,"down")
+			except CalledProcessError: pass
+			try: brctl("delbr",self.brname)
+			except CalledProcessError: pass
+			try: ifdown("--force",self.brname)
+			except CalledProcessError: pass
+		
+		
+		if self.was_net_service_running is None:
+			# we do nothing
+			pass
+		elif self.was_net_service_running == False:
+			stop_service(self.netservice,force=True)
+			time.sleep(1)
+		else:
+			# we altered service configuration
+			stop_service(self.netservice,force=True)
+			time.sleep(1)
+			try: start_service(self.netservice,force=True)
+			except CalledProcessError,e:
+				if e.returncode == 1: pass
+				else: raise
+			time.sleep(1)
+		
+		if self.was_nm_service_running is None:
+			 # we do nothing
+			 pass
+		elif self.was_nm_service_running == False:
+			stop_service(self.nmservice,force=True)
+			time.sleep(1)
+		else:
+			# we altered service configuration
+			stop_service(self.nmservice,force=True)
+			time.sleep(1)
+			start_service(self.nmservice,force=True)
+			time.sleep(1)
+		
+		self.runtime_state_changed = False
+
+	def execute(self):
+		yield "Determining default route"
+		routes = ip.route().stdout.splitlines()
+		defaultroute = [ x for x in routes if x.startswith("default") ]
+		if not defaultroute: raise TaskFailed("Your network configuration does not have a default route")
+		
+		dev = defaultroute[0].split()[4]
+		yield "Default route assigned to device %s"%dev
+		
+		self.old_net_device = dev
+		
+		if distro in (Fedora, CentOS):
+			inconfigfile = "/".join(augtool.match("/files/etc/sysconfig/network-scripts/*/DEVICE",dev).stdout.strip().split("/")[:-1])
+			if not inconfigfile: raise TaskFailed("Device %s has not been set up in /etc/sysconfig/network-scripts"%dev)
+			pathtoconfigfile = inconfigfile[6:]
+
+		if distro in (Fedora, CentOS):
+			automatic = augtool.match("%s/ONBOOT"%inconfigfile,"yes").stdout.strip()
+		else:
+			automatic = augtool.match("/files/etc/network/interfaces/auto/*/",dev).stdout.strip()
+		if not automatic:
+			if distro is Fedora: raise TaskFailed("Device %s has not been set up in %s as automatic on boot"%dev,pathtoconfigfile)
+			else: raise TaskFailed("Device %s has not been set up in /etc/network/interfaces as automatic on boot"%dev)
+			
+		if distro not in (Fedora , CentOS):
+			inconfigfile = augtool.match("/files/etc/network/interfaces/iface",dev).stdout.strip()
+			if not inconfigfile: raise TaskFailed("Device %s has not been set up in /etc/network/interfaces"%dev)
+
+		if distro in (Fedora, CentOS):
+			isstatic = augtool.match(inconfigfile + "/BOOTPROTO","none").stdout.strip()
+			if not isstatic: isstatic = augtool.match(inconfigfile + "/BOOTPROTO","static").stdout.strip()
+		else:
+			isstatic = augtool.match(inconfigfile + "/method","static").stdout.strip()
+		if not isstatic:
+			if distro in (Fedora, CentOS): raise TaskFailed("Device %s has not been set up as a static device in %s"%(dev,pathtoconfigfile))
+			else: raise TaskFailed("Device %s has not been set up as a static device in /etc/network/interfaces"%dev)
+
+		if is_service_running(self.nmservice):
+			self.was_nm_service_running = True
+			yield "Stopping NetworkManager to avoid automatic network reconfiguration"
+			disable_service(self.nmservice)
+		else:
+			self.was_nm_service_running = False
+			
+		if is_service_running(self.netservice):
+			self.was_net_service_running = True
+		else:
+			self.was_net_service_running = False
+			
+		yield "Creating Cloud bridging device and making device %s member of this bridge"%dev
+
+		if distro in (Fedora, CentOS):
+			ifcfgtext = file(pathtoconfigfile).read()
+			newf = "/etc/sysconfig/network-scripts/ifcfg-%s"%self.brname
+			#def restore():
+				#try: os.unlink(newf)
+				#except OSError,e:
+					#if errno == 2: pass
+					#raise
+				#try: file(pathtoconfigfile,"w").write(ifcfgtext)
+				#except OSError,e: raise
+
+			f = file(newf,"w") ; f.write(ifcfgtext) ; f.flush() ; f.close()
+			innewconfigfile = "/files" + newf
+
+			script = """set %s/DEVICE %s
+set %s/NAME %s
+set %s/BRIDGE_PORTS %s
+set %s/TYPE Bridge
+rm %s/HWADDR
+rm %s/UUID
+rm %s/HWADDR
+rm %s/IPADDR
+rm %s/DEFROUTE
+rm %s/NETMASK
+rm %s/GATEWAY
+rm %s/BROADCAST
+rm %s/NETWORK
+set %s/BRIDGE %s
+save"""%(innewconfigfile,self.brname,innewconfigfile,self.brname,innewconfigfile,dev,
+			innewconfigfile,innewconfigfile,innewconfigfile,innewconfigfile,
+			inconfigfile,inconfigfile,inconfigfile,inconfigfile,inconfigfile,inconfigfile,
+			inconfigfile,self.brname)
+			
+			yield "Executing the following reconfiguration script:\n%s"%script
+			
+			try:
+				returned = augtool < script
+				if "Saved 2 file" not in returned.stdout:
+					print returned.stdout + returned.stderr
+					#restore()
+					raise TaskFailed("Network reconfiguration failed.")
+				else:
+					yield "Network reconfiguration complete"
+			except CalledProcessError,e:
+				#restore()
+				print e.stdout + e.stderr
+				raise TaskFailed("Network reconfiguration failed")
+		else: # Not fedora
+			backup = file("/etc/network/interfaces").read(-1)
+			#restore = lambda: file("/etc/network/interfaces","w").write(backup)
+
+			script = """set %s %s
+set %s %s
+set %s/bridge_ports %s
+save"""%(automatic,self.brname,inconfigfile,self.brname,inconfigfile,dev)
+			
+			yield "Executing the following reconfiguration script:\n%s"%script
+			
+			try:
+				returned = augtool < script
+				if "Saved 1 file" not in returned.stdout:
+					#restore()
+					raise TaskFailed("Network reconfiguration failed.")
+				else:
+					yield "Network reconfiguration complete"
+			except CalledProcessError,e:
+				#restore()
+				print e.stdout + e.stderr
+				raise TaskFailed("Network reconfiguration failed")
+		
+		yield "We are going to restart network services now, to make the network changes take effect.  Hit ENTER when you are ready."
+		if self.isAutoMode(): pass
+        	else:
+		    raw_input()
+		
+		# if we reach here, then if something goes wrong we should attempt to revert the runinng state
+		# if not, then no point
+		self.runtime_state_changed = True
+		
+		yield "Enabling and restarting non-NetworkManager networking"
+		if distro is Ubuntu: ifup(self.brname,stdout=None,stderr=None)
+		stop_service(self.netservice)
+		try: enable_service(self.netservice,forcestart=True)
+		except CalledProcessError,e:
+			if e.returncode == 1: pass
+			else: raise
+		
+		yield "Verifying that the bridge is up"
+		try:
+			o = ifconfig(self.brname)
+		except CalledProcessError,e:
+			print e.stdout + e.stderr
+			raise TaskFailed("The bridge could not be set up properly")
+		
+		yield "Networking restart done"
+
+
+class SetupCgConfig(ConfigTask):
+	name = "control groups configuration"
+	
+	def done(self):
+		
+		try:
+			return "group virt" in file("/etc/cgconfig.conf","r").read(-1)
+		except IOError,e:
+			if e.errno is 2: raise TaskFailed("cgconfig has not been properly installed on this system")
+			raise
+		
+	def execute(self):
+		cgconfig = file("/etc/cgconfig.conf","r").read(-1)
+		cgconfig = cgconfig + """
+group virt {
+	cpu {
+		cpu.shares = 9216;
+	}
+}
+"""
+		file("/etc/cgconfig.conf","w").write(cgconfig)
+		
+		stop_service("cgconfig")
+		enable_service("cgconfig",forcestart=True)
+
+
+class SetupCgRules(ConfigTask):
+	name = "control group rules setup"
+	cfgline = "root:/usr/sbin/libvirtd	cpu	virt/"
+	
+	def done(self):
+		try:
+			return self.cfgline in file("/etc/cgrules.conf","r").read(-1)
+		except IOError,e:
+			if e.errno is 2: raise TaskFailed("cgrulesd has not been properly installed on this system")
+			raise
+	
+	def execute(self):
+		cgrules = file("/etc/cgrules.conf","r").read(-1)
+		cgrules = cgrules + "\n" + self.cfgline + "\n"
+		file("/etc/cgrules.conf","w").write(cgrules)
+		
+		stop_service("cgred")
+		enable_service("cgred")
+
+
+class SetupCgroupControllers(ConfigTask):
+	name = "qemu cgroup controllers setup"
+	cfgline = "cgroup_controllers = [ \"cpu\" ]"
+	filename = "/etc/libvirt/qemu.conf"
+	
+	def done(self):
+		try:
+			return self.cfgline in file(self.filename,"r").read(-1)
+		except IOError,e:
+			if e.errno is 2: raise TaskFailed("qemu has not been properly installed on this system")
+			raise
+	
+	def execute(self):
+		libvirtqemu = file(self.filename,"r").read(-1)
+		libvirtqemu = libvirtqemu + "\n" + self.cfgline + "\n"
+		file("/etc/libvirt/qemu.conf","w").write(libvirtqemu)
+
+
+class SetupSecurityDriver(ConfigTask):
+	name = "security driver setup"
+	cfgline = "security_driver = \"none\""
+	filename = "/etc/libvirt/qemu.conf"
+	
+	def done(self):
+		try:
+			return self.cfgline in file(self.filename,"r").read(-1)
+		except IOError,e:
+			if e.errno is 2: raise TaskFailed("qemu has not been properly installed on this system")
+			raise
+	
+	def execute(self):
+		libvirtqemu = file(self.filename,"r").read(-1)
+		libvirtqemu = libvirtqemu + "\n" + self.cfgline + "\n"
+		file("/etc/libvirt/qemu.conf","w").write(libvirtqemu)
+
+
+class SetupLibvirt(ConfigTask):
+	name = "libvirt setup"
+	cfgline = "export CGROUP_DAEMON='cpu:/virt'"
+	def done(self):
+		try:
+			if distro in (Fedora,CentOS): 	 libvirtfile = "/etc/sysconfig/libvirtd"
+			elif distro is Ubuntu:	 libvirtfile = "/etc/default/libvirt-bin"
+			else: raise AssertionError, "We should not reach this"
+			return self.cfgline in file(libvirtfile,"r").read(-1)
+		except IOError,e:
+			if e.errno is 2: raise TaskFailed("libvirt has not been properly installed on this system")
+			raise
+	
+	def execute(self):
+		if distro in (Fedora,CentOS): 	 libvirtfile = "/etc/sysconfig/libvirtd"
+		elif distro is Ubuntu:	 libvirtfile = "/etc/default/libvirt-bin"
+		else: raise AssertionError, "We should not reach this"
+		libvirtbin = file(libvirtfile,"r").read(-1)
+		libvirtbin = libvirtbin + "\n" + self.cfgline + "\n"
+		file(libvirtfile,"w").write(libvirtbin)
+		
+		if distro in (CentOS, Fedora):	svc = "libvirtd"
+		else:					svc = "libvirt-bin"
+		stop_service(svc)
+		enable_service(svc)
+
+class SetupLiveMigration(ConfigTask):
+	name = "live migration setup"
+	stanzas = (
+			"listen_tcp=1",
+			'tcp_port="16509"',
+			'auth_tcp="none"',
+			"listen_tls=0",
+	)
+	
+	def done(self):
+		try:
+			lines = [ s.strip() for s in file("/etc/libvirt/libvirtd.conf").readlines() ]
+			if all( [ stanza in lines for stanza in self.stanzas ] ): return True
+		except IOError,e:
+			if e.errno is 2: raise TaskFailed("libvirt has not been properly installed on this system")
+			raise
+	
+	def execute(self):
+		
+		for stanza in self.stanzas:
+			startswith = stanza.split("=")[0] + '='
+			replace_or_add_line("/etc/libvirt/libvirtd.conf",startswith,stanza)
+
+		if distro is Fedora:
+			replace_or_add_line("/etc/sysconfig/libvirtd","LIBVIRTD_ARGS=","LIBVIRTD_ARGS=-l")
+		
+		elif distro is Ubuntu:
+			if os.path.exists("/etc/init/libvirt-bin.conf"):
+				replace_line("/etc/init/libvirt-bin.conf", "exec /usr/sbin/libvirtd","exec /usr/sbin/libvirtd -d -l")
+			else:
+				replace_or_add_line("/etc/default/libvirt-bin","libvirtd_opts=","libvirtd_opts='-l -d'")
+			
+		else:
+			raise AssertionError("Unsupported distribution")
+		
+		if distro in (CentOS, Fedora):	svc = "libvirtd"
+		else:						svc = "libvirt-bin"
+		stop_service(svc)
+		enable_service(svc)
+
+
+class SetupRequiredServices(ConfigTask):
+	name = "required services setup"
+	
+	def done(self):
+		if distro is Fedora:  nfsrelated = "rpcbind nfslock"
+		elif distro is CentOS: nfsrelated = "portmap nfslock"
+		else: return True
+		return all( [ is_service_running(svc) for svc in nfsrelated.split() ] )
+		
+	def execute(self):
+
+		if distro is Fedora:  nfsrelated = "rpcbind nfslock"
+		elif distro is CentOS: nfsrelated = "portmap nfslock"
+		else: raise AssertionError("Unsupported distribution")
+
+		for svc in nfsrelated.split(): enable_service(svc)
+
+
+class SetupFirewall(ConfigTask):
+	name = "firewall setup"
+	
+	def done(self):
+		
+		if distro in (Fedora, CentOS):
+			if not os.path.exists("/etc/sysconfig/iptables"): return True
+			if ":on" not in chkconfig("--list","iptables").stdout: return True
+		else:
+			if "Status: active" not in ufw.status().stdout: return True
+			if not os.path.exists("/etc/ufw/before.rules"): return True
+		rule = "-p tcp -m tcp --dport 16509 -j ACCEPT"
+		if rule in iptablessave().stdout: return True
+		return False
+	
+	def execute(self):
+		ports = "22 1798 16509".split()
+		if distro in (Fedora , CentOS):
+			for p in ports: iptables("-I","INPUT","1","-p","tcp","--dport",p,'-j','ACCEPT')
+			o = service.iptables.save() ; print o.stdout + o.stderr
+		else:
+			for p in ports: ufw.allow(p)
+
+
+class SetupFirewall2(ConfigTask):
+	# this closes bug 4371
+	name = "additional firewall setup"
+	def __init__(self,brname):
+		ConfigTask.__init__(self)
+		self.brname = brname
+	
+	def done(self):
+		
+		if distro in (Fedora, CentOS):
+			if not os.path.exists("/etc/sysconfig/iptables"): return True
+			if ":on" not in chkconfig("--list","iptables").stdout: return True
+			rule = "FORWARD -i %s -o %s -j ACCEPT"%(self.brname,self.brname)
+			if rule in iptablessave().stdout: return True
+			return False
+		else:
+			if "Status: active" not in ufw.status().stdout: return True
+			if not os.path.exists("/etc/ufw/before.rules"): return True
+			rule = "-A ufw-before-forward -i %s -o %s -j ACCEPT"%(self.brname,self.brname)
+			if rule in file("/etc/ufw/before.rules").read(-1): return True
+			return False
+		
+	def execute(self):
+		
+		yield "Permitting traffic in the bridge interface, migration port and for VNC ports"
+		
+		if distro in (Fedora , CentOS):
+			
+			for rule in (
+				"-I FORWARD -i %s -o %s -j ACCEPT"%(self.brname,self.brname),
+				"-I INPUT 1 -p tcp --dport 5900:6100 -j ACCEPT",
+				"-I INPUT 1 -p tcp --dport 49152:49216 -j ACCEPT",
+				):
+				args = rule.split()
+				o = iptables(*args)
+			service.iptables.save(stdout=None,stderr=None)
+			
+		else:
+			
+			rule = "-A ufw-before-forward -i %s -o %s -j ACCEPT"%(self.brname,self.brname)
+			text = file("/etc/ufw/before.rules").readlines()
+			newtext = []
+			for line in text:
+				if line.startswith("COMMIT"):
+					newtext.append(rule + "\n")
+				newtext.append(line)
+			file("/etc/ufw/before.rules","w").writelines(newtext)
+			ufw.allow.proto.tcp("from","any","to","any","port","5900:6100")
+			ufw.allow.proto.tcp("from","any","to","any","port","49152:49216")
+
+			stop_service("ufw")
+			start_service("ufw")
+
+
+# Tasks according to distribution -- at some point we will split them in separate modules
+
+def config_tasks(brname):
+	if distro is CentOS:
+		config_tasks = (
+			SetupNetworking(brname),
+			SetupLibvirt(),
+			SetupRequiredServices(),
+			SetupFirewall(),
+			SetupFirewall2(brname),
+		)
+	elif distro in (Ubuntu,Fedora):
+		config_tasks = (
+			SetupNetworking(brname),
+			SetupCgConfig(),
+			SetupCgRules(),
+			SetupCgroupControllers(),
+			SetupSecurityDriver(),
+			SetupLibvirt(),
+			SetupLiveMigration(),
+			SetupRequiredServices(),
+			SetupFirewall(),
+			SetupFirewall2(brname),
+		)
+	else:
+		raise AssertionError("Unknown distribution")
+	return config_tasks
+
+
+def backup_etc(targetdir):
+	if not targetdir.endswith("/"): targetdir += "/"
+	check_call( ["mkdir","-p",targetdir] )
+	rsynccall = ["rsync","-ax","--delete"] + ["/etc/",targetdir]
+	check_call( rsynccall )
+def restore_etc(targetdir):
+	if not targetdir.endswith("/"): targetdir += "/"
+	rsynccall = ["rsync","-ax","--delete"] + [targetdir,"/etc/"]
+	check_call( rsynccall )
+def remove_backup(targetdir):
+	check_call( ["rm","-rf",targetdir] )
+
+def list_zonespods(host):
+	text = urllib2.urlopen('http://%s:8096/client/api?command=listPods'%host).read(-1)
+	dom = xml.dom.minidom.parseString(text) 
+	x = [ (zonename,podname)
+		for pod in dom.childNodes[0].childNodes  
+		for podname in [ x.childNodes[0].wholeText for x in pod.childNodes if x.tagName == "name" ] 
+		for zonename in  [ x.childNodes[0].wholeText for x in pod.childNodes if x.tagName == "zonename" ]
+		]
+	return x
+	
+def prompt_for_hostpods(zonespods):
+	"""Ask user to select one from those zonespods
+	Returns (zone,pod) or None if the user made the default selection."""
+	while True:
+		stderr("Type the number of the zone and pod combination this host belongs to (hit ENTER to skip this step)")
+		print "  N) ZONE, POD" 
+		print "================"
+		for n,(z,p) in enumerate(zonespods):
+			print "%3d) %s, %s"%(n,z,p)
+		print "================"
+		print "> ",
+		zoneandpod = raw_input().strip()
+		
+		if not zoneandpod:
+			# we go with default, do not touch anything, just break
+			return None
+		
+		try:
+			# if parsing fails as an int, just vomit and retry
+			zoneandpod = int(zoneandpod)
+			if zoneandpod >= len(zonespods) or zoneandpod < 0: raise ValueError, "%s out of bounds"%zoneandpod
+		except ValueError,e:
+			stderr(str(e))
+			continue # re-ask
+		
+		# oh yeah, the int represents an valid zone and pod index in the array
+		return zonespods[zoneandpod]
+	
+# this configures the agent
+
+def setup_agent_config(configfile, host, zone, pod, cluster, guid):
+	stderr("Examining Agent configuration")
+	fn = configfile
+	text = file(fn).read(-1)
+	lines = [ s.strip() for s in text.splitlines() ]
+	confopts = dict([ m.split("=",1) for m in lines if "=" in m and not m.startswith("#") ])
+	confposes = dict([ (m.split("=",1)[0],n) for n,m in enumerate(lines) if "=" in m and not m.startswith("#") ])
+	
+	if guid != None:
+		confopts['guid'] = guid
+	else:
+		if not "guid" in confopts:
+			stderr("Generating GUID for this Agent")
+			confopts['guid'] = uuidgen().stdout.strip()
+	
+	if host == None:
+		try: host = confopts["host"]
+		except KeyError: host = "localhost"
+		stderr("Please enter the host name of the management server that this agent will connect to: (just hit ENTER to go with %s)",host)
+		print "> ",
+		newhost = raw_input().strip()
+		if newhost: host = newhost
+
+	confopts["host"] = host
+	
+	stderr("Querying %s for zones and pods",host)
+	
+	try:
+	    if zone == None or pod == None:
+			x = list_zonespods(confopts['host'])
+			zoneandpod = prompt_for_hostpods(x)
+			if zoneandpod:
+				confopts["zone"],confopts["pod"] = zoneandpod
+				stderr("You selected zone %s pod %s",confopts["zone"],confopts["pod"])
+			else:
+				stderr("Skipped -- using the previous zone %s pod %s",confopts["zone"],confopts["pod"])
+	    else:
+			confopts["zone"] = zone
+			confopts["pod"] = pod
+			confopts["cluster"] = cluster
+	except (urllib2.URLError,urllib2.HTTPError),e:
+		stderr("Query failed: %s.  Defaulting to zone %s pod %s",str(e),confopts["zone"],confopts["pod"])
+
+	for opt,val in confopts.items():
+		line = "=".join([opt,val])
+		if opt not in confposes: lines.append(line)
+		else: lines[confposes[opt]] = line
+	
+	text = "\n".join(lines)
+	file(fn,"w").write(text)
+
+def setup_consoleproxy_config(configfile, host, zone, pod):
+	stderr("Examining Console Proxy configuration")
+	fn = configfile
+	text = file(fn).read(-1)
+	lines = [ s.strip() for s in text.splitlines() ]
+	confopts = dict([ m.split("=",1) for m in lines if "=" in m and not m.startswith("#") ])
+	confposes = dict([ (m.split("=",1)[0],n) for n,m in enumerate(lines) if "=" in m and not m.startswith("#") ])
+
+	if not "guid" in confopts:
+		stderr("Generating GUID for this Console Proxy")
+		confopts['guid'] = uuidgen().stdout.strip()
+
+        if host == None:
+		try: host = confopts["host"]
+		except KeyError: host = "localhost"
+		stderr("Please enter the host name of the management server that this console-proxy will connect to: (just hit ENTER to go with %s)",host)
+		print "> ",
+		newhost = raw_input().strip()
+		if newhost: host = newhost
+	confopts["host"] = host
+
+	stderr("Querying %s for zones and pods",host)
+	
+	try:
+                if zone == None or pod == None:
+			x = list_zonespods(confopts['host'])
+			zoneandpod = prompt_for_hostpods(x)
+			if zoneandpod:
+				confopts["zone"],confopts["pod"] = zoneandpod
+				stderr("You selected zone %s pod %s",confopts["zone"],confopts["pod"])
+			else:
+				stderr("Skipped -- using the previous zone %s pod %s",confopts["zone"],confopts["pod"])
+		else:
+			confopts["zone"] = zone
+			confopts["pod"] = pod
+	except (urllib2.URLError,urllib2.HTTPError),e:
+		stderr("Query failed: %s.  Defaulting to zone %s pod %s",str(e),confopts["zone"],confopts["pod"])
+
+	for opt,val in confopts.items():
+		line = "=".join([opt,val])
+		if opt not in confposes: lines.append(line)
+		else: lines[confposes[opt]] = line
+	
+	text = "\n".join(lines)
+	file(fn,"w").write(text)
+
+# =========================== DATABASE MIGRATION SUPPORT CODE ===================
+
+# Migrator, Migratee and Evolvers -- this is the generic infrastructure.
+# To actually implement Cloud.com-specific code, search "Cloud.com-specific evolvers and context"
+
+
+class MigratorException(Exception): pass
+class NoMigrationPath(MigratorException): pass
+class NoMigrator(MigratorException): pass
+
+INITIAL_LEVEL = '-'
+
+class Migrator:
+	"""Migrator class.
+	
+	The migrator gets a list of Python objects, and discovers MigrationSteps in it. It then sorts the steps into a chain, based on the attributes from_level and to_level in each one of the steps.
+	
+	When the migrator's run(context) is called, the chain of steps is applied sequentially on the context supplied to run(), in the order of the chain of steps found at discovery time.  See the documentation for the MigrationStep class for information on how that happens.
+	"""
+	
+	def __init__(self,evolver_source):
+		self.discover_evolvers(evolver_source)
+		self.sort_evolvers()
+		
+	def discover_evolvers(self,source):
+		self.evolvers = []
+		for val in source:
+			if hasattr(val,"from_level") and hasattr(val,"to_level") and val.to_level:
+				self.evolvers.append(val)
+	
+	def sort_evolvers(self):
+		new = []
+		while self.evolvers:
+			if not new:
+				try: idx= [ i for i,s in enumerate(self.evolvers)
+					if s.from_level == INITIAL_LEVEL ][0] # initial evolver
+				except IndexError,e:
+					raise IndexError, "no initial evolver (from_level is None) could be found"
+			else:
+				try: idx= [ i for i,s in enumerate(self.evolvers)
+					if new[-1].to_level == s.from_level ][0]
+				except IndexError,e:
+					raise IndexError, "no evolver could be found to evolve from level %s"%new[-1].to_level
+			new.append(self.evolvers.pop(idx))
+		self.evolvers = new
+	
+	def get_evolver_chain(self):
+		return [ (s.from_level, s.to_level, s) for s in self.evolvers ]
+		
+	def get_evolver_by_starting_level(self,level):
+		try: return [ s for s in self.evolvers if s.from_level == level][0]
+		except IndexError: raise NoMigrator, "No evolver knows how to evolve the database from schema level %r"%level
+	
+	def get_evolver_by_ending_level(self,level):
+		try: return [ s for s in self.evolvers if s.to_level == level][0]
+		except IndexError: raise NoMigrator, "No evolver knows how to evolve the database to schema level %r"%level
+	
+	def run(self, context, dryrun = False, starting_level = None, ending_level = None):
+		"""Runs each one of the steps in sequence, passing the migration context to each. At the end of the process, context.commit() is called to save the changes, or context.rollback() is called if dryrun = True.
+		
+		If starting_level is not specified, then the context.get_schema_level() is used to find out at what level the context is at.  Then starting_level is set to that.
+		
+		If ending_level is not specified, then the evolvers will run till the end of the chain."""
+		
+		assert dryrun is False # NOT IMPLEMENTED, prolly gonna implement by asking the context itself to remember its state
+		
+		starting_level = starting_level or context.get_schema_level() or self.evolvers[0].from_level
+		ending_level = ending_level or self.evolvers[-1].to_level
+		
+		evolution_path = self.evolvers
+		idx = evolution_path.index(self.get_evolver_by_starting_level(starting_level))
+		evolution_path = evolution_path[idx:]
+		try: idx = evolution_path.index(self.get_evolver_by_ending_level(ending_level))
+		except ValueError:
+			raise NoEvolutionPath, "No evolution path from schema level %r to schema level %r" % \
+				(starting_level,ending_level)
+		evolution_path = evolution_path[:idx+1]
+		
+		logging.info("Starting migration on %s"%context)
+		
+		for ec in evolution_path:
+			assert ec.from_level == context.get_schema_level()
+			evolver = ec(context=context)
+			logging.info("%s (from level %s to level %s)",
+				evolver,
+				evolver.from_level,
+				evolver.to_level)
+			#try:
+			evolver.run()
+			#except:
+				#context.rollback()
+				#raise
+			context.set_schema_level(evolver.to_level)
+			#context.commit()
+			logging.info("%s is now at level %s",context,context.get_schema_level())
+		
+		#if dryrun: # implement me with backup and restore
+			#logging.info("Rolling back changes on %s",context)
+			#context.rollback()
+		#else:
+			#logging.info("Committing changes on %s",context)
+			#context.commit()
+		
+		logging.info("Migration finished")
+		
+
+class MigrationStep:
+	"""Base MigrationStep class, aka evolver.
+	
+	You develop your own steps, and then pass a list of those steps to the
+	Migrator instance that will run them in order.
+	
+	When the migrator runs, it will take the list of steps you gave him,
+	and, for each step:
+	
+	a) instantiate it, passing the context you gave to the migrator
+	   into the step's __init__().
+	b) run() the method in the migration step.
+	
+	As you can see, the default MigrationStep constructor makes the passed
+	context available as self.context in the methods of your step.
+	
+	Each step has two member vars that determine in which order they
+	are run, and if they need to run:
+	
+	- from_level = the schema level that the database should be at,
+		       before running the evolver
+		       The value None has special meaning here, it
+		       means the first evolver that should be run if the
+		       database does not have a schema level yet.
+	- to_level =   the schema level number that the database will be at
+		       after the evolver has run
+	"""
+	
+	# Implement these attributes in your steps
+	from_level = None
+	to_level = None
+	
+	def __init__(self,context):
+		self.context = context
+		
+	def run(self):
+		raise NotImplementedError
+
+
+class MigrationContext:
+	def __init__(self): pass
+	def commit(self):raise NotImplementedError
+	def rollback(self):raise NotImplementedError
+	def get_schema_level(self):raise NotImplementedError
+	def set_schema_level(self,l):raise NotImplementedError
+
+
diff --git a/python/wscript_build b/python/wscript_build
new file mode 100644
index 00000000000..d3a80e70d26
--- /dev/null
+++ b/python/wscript_build
@@ -0,0 +1,2 @@
+obj = bld(features = 'py',name='pythonmodules')
+obj.find_sources_in_dirs('lib', exts=['.py'])
diff --git a/scripts/network/domr/call_firewall.sh b/scripts/network/domr/call_firewall.sh
index 8878ec833ba..287efa21f5d 100755
--- a/scripts/network/domr/call_firewall.sh
+++ b/scripts/network/domr/call_firewall.sh
@@ -85,7 +85,7 @@ do
   esac
 done
 
-CERT="$(dirname $0)/id_rsa"
+cert="/root/.ssh/id_rsa.cloud"
 
 # Check if DomR is up and running. If not, exit with error code 1.
 check_gw "$domRIp"
@@ -114,7 +114,7 @@ then
   exit 2
 fi
 
-ssh -p 3922 -q -o StrictHostKeyChecking=no -i $CERT root@$domRIp "/root/firewall.sh $*"
+ssh -p 3922 -q -o StrictHostKeyChecking=no -i $cert root@$domRIp "/root/firewall.sh $*"
 exit $?
 
 
diff --git a/scripts/network/domr/call_loadbalancer.sh b/scripts/network/domr/call_loadbalancer.sh
index fd9a8c02700..fed3abe3d80 100755
--- a/scripts/network/domr/call_loadbalancer.sh
+++ b/scripts/network/domr/call_loadbalancer.sh
@@ -26,7 +26,7 @@ copy_haproxy() {
   local domRIp=$1
   local cfg=$2
 
-  scp -P 3922 -q -o StrictHostKeyChecking=no -i $CERT $cfg root@$domRIp:/etc/haproxy/haproxy.cfg.new
+  scp -P 3922 -q -o StrictHostKeyChecking=no -i $cert $cfg root@$domRIp:/etc/haproxy/haproxy.cfg.new
   return $?
 }
 
@@ -56,7 +56,7 @@ do
   esac
 done
 
-CERT="$(dirname $0)/id_rsa"
+cert="/root/.ssh/id_rsa.cloud"
 
 if [ "$iflag$fflag" != "11" ]
 then
@@ -79,5 +79,5 @@ then
   exit 1
 fi
 	
-ssh -p 3922 -q -o StrictHostKeyChecking=no -i $CERT root@$domRIp "/root/loadbalancer.sh $*"
+ssh -p 3922 -q -o StrictHostKeyChecking=no -i $cert root@$domRIp "/root/loadbalancer.sh $*"
 exit $?	
diff --git a/scripts/network/domr/ipassoc.sh b/scripts/network/domr/ipassoc.sh
index 91b0f1d3698..2d460bf7e5a 100755
--- a/scripts/network/domr/ipassoc.sh
+++ b/scripts/network/domr/ipassoc.sh
@@ -50,6 +50,22 @@ check_gw() {
   return $?;
 }
 
+#Add 1:1 NAT entry
+add_one_to_one_nat_entry() {
+  local guestIp=$1
+  local publicIp=$2  
+  local dIp=$3
+  ssh -p 3922 -o StrictHostKeyChecking=no -i $cert root@$dIp "\
+  iptables -t nat -A PREROUTING -i eth2 -d $publicIp -j DNAT --to-destination $guestIp
+  iptables -t nat -A POSTROUTING -o eth2 -s $guestIp -j SNAT --to-source $publicIp
+  iptables -P FORWARD DROP
+  iptables -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
+  iptables -A FORWARD -i eth2 -o eth0 -d $guestIp -m state --state NEW -j ACCEPT
+  iptables -A FORWARD -i eth0 -o eth2 -s $guestIp -m state --state NEW -j ACCEPT
+  "
+  return $?
+}
+
 #Add the NAT entries into iptables in the routing domain
 add_nat_entry() {
   local dRIp=$1
@@ -57,7 +73,7 @@ add_nat_entry() {
    ssh -p 3922 -o StrictHostKeyChecking=no -i $cert root@$dRIp "\
       ip addr add dev $correctVif $pubIp
       iptables -t nat -I POSTROUTING   -j SNAT -o $correctVif --to-source $pubIp ;
-      /sbin/arping -c 3 -I $correctVif -A -U -s $pubIp $pubIp;
+      arping -c 3 -I $correctVif -A -U -s $pubIp $pubIp;
      "
   if [ $? -gt 0  -a $? -ne 2 ]
   then
@@ -91,7 +107,7 @@ add_an_ip () {
    ssh -p 3922 -o StrictHostKeyChecking=no -i $cert root@$dRIp "\
    	  ifconfig $correctVif up;
       ip addr add dev $correctVif $pubIp ;
-      /sbin/arping -c 3 -I $correctVif -A -U -s $pubIp $pubIp;
+      arping -c 3 -I $correctVif -A -U -s $pubIp $pubIp;
      "
    return $?
 }
@@ -120,9 +136,10 @@ vflag=
 gflag=
 nflag=
 cflag=
+Gflag=
 op=""
 
-while getopts 'fADr:i:a:l:v:g:n:c:' OPTION
+while getopts 'fADr:i:a:l:v:g:n:c:G:' OPTION
 do
   case $OPTION in
   A)	Aflag=1
@@ -157,12 +174,26 @@ do
   c)	cflag=1
   		correctVif="$OPTARG"
   		;;
+  G)    Gflag=1
+        guestIp="$OPTARG"
+        ;;
   ?)	usage
 		exit 2
 		;;
   esac
 done
 
+#1:1 NAT
+if [ "$Gflag" == "1" ]
+then
+  add_nat_entry $domRIp $publicIp 
+  if [ $? -eq 0 ]
+  then
+    add_one_to_one_nat_entry $guestIp $publicIp $domRIp
+  fi
+  exit $?
+fi
+
 #Either the A flag or the D flag but not both
 if [ "$Aflag$Dflag" != "1" ]
 then
@@ -182,7 +213,6 @@ then
    exit 3
 fi
 
-
 if [ "$fflag" == "1" ] && [ "$Aflag" == "1" ]
 then
   add_nat_entry $domRIp $publicIp 
@@ -208,4 +238,3 @@ then
 fi
 
 exit 0
-
diff --git a/scripts/vm/hypervisor/xenserver/networkUsage.sh b/scripts/network/domr/networkUsage.sh
similarity index 86%
rename from scripts/vm/hypervisor/xenserver/networkUsage.sh
rename to scripts/network/domr/networkUsage.sh
index 129aa76f05e..2aa76fe2d8c 100755
--- a/scripts/vm/hypervisor/xenserver/networkUsage.sh
+++ b/scripts/network/domr/networkUsage.sh
@@ -18,15 +18,19 @@ check_gw() {
   return $?;
 }
 
-cert="$(dirname $0)/id_rsa"
+cert="/root/.ssh/id_rsa.cloud"
 
 create_usage_rules () {
   local dRIp=$1
    ssh -p 3922 -o StrictHostKeyChecking=no -i $cert root@$dRIp "\
      iptables -N NETWORK_STATS > /dev/null;
      iptables -I FORWARD -j NETWORK_STATS > /dev/null;
+     iptables -I INPUT -j NETWORK_STATS > /dev/null;
+     iptables -I OUTPUT -j NETWORK_STATS > /dev/null;
      iptables -A NETWORK_STATS -i eth0 -o eth2 > /dev/null;
      iptables -A NETWORK_STATS -i eth2 -o eth0 > /dev/null;
+     iptables -A NETWORK_STATS -o eth2 ! -i eth0 -p tcp > /dev/null;
+     iptables -A NETWORK_STATS -i eth2 ! -o eth0 -p tcp > /dev/null;
      "
      return 1
 }
@@ -37,6 +41,8 @@ add_public_interface () {
    ssh -p 3922 -o StrictHostKeyChecking=no -i $cert root@$dRIp "\
      iptables -A NETWORK_STATS -i eth0 -o $pubIf > /dev/null;
      iptables -A NETWORK_STATS -i $pubIf -o eth0 > /dev/null;
+     iptables -A NETWORK_STATS -o $pubIf ! -i eth0 -p tcp > /dev/null;
+     iptables -A NETWORK_STATS -i $pubIf ! -o eth0 -p tcp > /dev/null;
      "
      return 1
 }
diff --git a/scripts/network/domr/vm_data.sh b/scripts/network/domr/vm_data.sh
index 78fc7f92c50..32b27bbdad9 100755
--- a/scripts/network/domr/vm_data.sh
+++ b/scripts/network/domr/vm_data.sh
@@ -10,7 +10,7 @@ usage() {
 }
 
 set -x
-CERT="/root/.ssh/id_rsa.cloud"
+cert="/root/.ssh/id_rsa.cloud"
 PORT=3922
 
 create_htaccess() {
@@ -24,7 +24,7 @@ create_htaccess() {
   entry="RewriteRule ^$file$  ../$folder/%{REMOTE_ADDR}/$file [L,NC,QSA]"
   htaccessFolder="/var/www/html/latest"
   htaccessFile=$htaccessFolder/.htaccess
-  ssh -p $PORT -o StrictHostKeyChecking=no -i $CERT root@$domrIp "mkdir -p $htaccessFolder; touch $htaccessFile; grep -F \"$entry\" $htaccessFile; if [ \$? -gt 0 ]; then echo -e \"$entry\" >> $htaccessFile; fi" >/dev/null
+  ssh -p $PORT -o StrictHostKeyChecking=no -i $cert root@$domrIp "mkdir -p $htaccessFolder; touch $htaccessFile; grep -F \"$entry\" $htaccessFile; if [ \$? -gt 0 ]; then echo -e \"$entry\" >> $htaccessFile; fi" >/dev/null
   result=$?
   
   if [ $result -eq 0 ]
@@ -32,7 +32,7 @@ create_htaccess() {
     entry="Options -Indexes\\nOrder Deny,Allow\\nDeny from all\\nAllow from $vmIp"
     htaccessFolder="/var/www/html/$folder/$vmIp"
     htaccessFile=$htaccessFolder/.htaccess
-    ssh -p $PORT -o StrictHostKeyChecking=no -i $CERT root@$domrIp "mkdir -p $htaccessFolder; echo -e \"$entry\" > $htaccessFile" >/dev/null
+    ssh -p $PORT -o StrictHostKeyChecking=no -i $cert root@$domrIp "mkdir -p $htaccessFolder; echo -e \"$entry\" > $htaccessFile" >/dev/null
     result=$?
   fi
   
@@ -47,7 +47,7 @@ copy_vm_data_file() {
   local dataFile=$5        
   
   chmod +r $dataFile
-  scp -P $PORT -o StrictHostKeyChecking=no -i $CERT $dataFile root@$domrIp:/var/www/html/$folder/$vmIp/$file >/dev/null
+  scp -P $PORT -o StrictHostKeyChecking=no -i $cert $dataFile root@$domrIp:/var/www/html/$folder/$vmIp/$file >/dev/null
   return $?
 }
 
@@ -58,7 +58,7 @@ delete_vm_data_file() {
   local file=$4
   
   vmDataFilePath="/var/www/html/$folder/$vmIp/$file"
-  ssh -p $PORT -o StrictHostKeyChecking=no -i $CERT root@$domrIp "if [ -f $vmDataFilePath ]; then rm -rf $vmDataFilePath; fi" >/dev/null
+  ssh -p $PORT -o StrictHostKeyChecking=no -i $cert root@$domrIp "if [ -f $vmDataFilePath ]; then rm -rf $vmDataFilePath; fi" >/dev/null
   return $?
 }
 
diff --git a/scripts/storage/qcow2/createtmplt.sh b/scripts/storage/qcow2/createtmplt.sh
index 550901882b1..0bce6b2d5a5 100755
--- a/scripts/storage/qcow2/createtmplt.sh
+++ b/scripts/storage/qcow2/createtmplt.sh
@@ -78,6 +78,23 @@ create_from_file() {
   then
     rm -f $tmpltimg
   fi
+  chmod a+r /$tmpltfs/$tmpltname
+}
+
+create_from_snapshot() {
+  local tmpltImg=$1
+  local snapshotName=$2
+  local tmpltfs=$3
+  local tmpltname=$4
+
+  cloud-qemu-img convert -f qcow2 -O qcow2 -s $snapshotName $tmpltImg /$tmpltfs/$tmpltname >& /dev/null
+  if [ $? -gt 0 ]
+  then
+     printf "Failed to create template /$tmplfs/$tmpltname from snapshot $snapshotName on disk $tmpltImg "
+     exit 2
+  fi
+
+  chmod a+r /$tmpltfs/$tmpltname
 }
 
 tflag=
@@ -89,8 +106,9 @@ hvm=false
 cleanup=false
 dflag=
 cflag=
+snapshotName=
 
-while getopts 'uht:n:f:s:c:d:' OPTION
+while getopts 'uht:n:f:sc:d:' OPTION
 do
   case $OPTION in
   t)	tflag=1
@@ -103,10 +121,10 @@ do
 		tmpltimg="$OPTARG"
 		;;
   s)	sflag=1
-		volsize="$OPTARG"
+		sflag=1
 		;;
   c)	cflag=1
-		cksum="$OPTARG"
+		snapshotName="$OPTARG"
 		;;
   d)	dflag=1
 		descr="$OPTARG"
@@ -119,12 +137,6 @@ do
   esac
 done
 
-if [ "$tflag$nflag$fflag" != "111" ]
-then
- usage
- exit 2
-fi
-
 
 if [ ! -d /$tmpltfs ] 
 then
@@ -148,9 +160,15 @@ then
   printf "failed to uncompress $tmpltimg\n"
 fi
 
-create_from_file $tmpltfs $tmpltimg $tmpltname
+if [ "$sflag" == "1" ]
+then
+   create_from_snapshot  $tmpltimg $snapshotName $tmpltfs $tmpltname
+else
+   create_from_file $tmpltfs $tmpltimg $tmpltname
+fi
 
 touch /$tmpltfs/template.properties
+chmod a+r /$tmpltfs/template.properties
 echo -n "" > /$tmpltfs/template.properties
 
 today=$(date '+%m_%d_%Y')
diff --git a/scripts/storage/qcow2/managesnapshot.sh b/scripts/storage/qcow2/managesnapshot.sh
index eb689a20ebe..e2cdd77205d 100755
--- a/scripts/storage/qcow2/managesnapshot.sh
+++ b/scripts/storage/qcow2/managesnapshot.sh
@@ -16,13 +16,20 @@ create_snapshot() {
   local snapshotname=$2
   local failed=0
 
-  qemu-img snapshot -c $snapshotname $disk
+  if [ ! -f $disk ]
+  then
+     failed=1
+     printf "No disk $disk exist\n" >&2
+     return $failed
+  fi
+
+  cloud-qemu-img snapshot -c $snapshotname $disk
   
   if [ $? -gt 0 ]
   then
-    failed=1
+    failed=2
     printf "***Failed to create snapshot $snapshotname for path $disk\n" >&2
-    qemu-img snapshot -d $snapshotname $disk
+    cloud-qemu-img snapshot -d $snapshotname $disk
     
     if [ $? -gt 0 ]
     then
@@ -34,21 +41,40 @@ create_snapshot() {
 }
 
 destroy_snapshot() {
-  local backupSnapDir=$1
+  local disk=$1
   local snapshotname=$2
+  local deleteDir=$3
   local failed=0
 
-  if [ -f $backupSnapDir/$snapshotname ]
+  if [ -d $disk ]
   then
-     rm -f $backupSnapDir/$snapshotname
-  
-     if [ $? -gt 0 ]
+     if [ -f $disk/$snapshotname ]
      then
-        printf "***Failed to delete snapshot $snapshotname for path $backupSnapDir\n" >&2
-        failed=1
+	rm -rf $disk/$snapshotname >& /dev/null
      fi
+
+     if [ "$deleteDir" == "1" ]
+     then
+	rm -rf %disk >& /dev/null
+     fi
+
+     return $failed
   fi
 
+  if [ ! -f $disk ]
+  then
+     failed=1
+     printf "No disk $disk exist\n" >&2
+     return $failed
+  fi
+
+  cloud-qemu-img snapshot -d $snapshotname $disk
+  if [ $? -gt 0 ]
+  then
+     failed=2
+     printf "Failed to delete snapshot $snapshotname for path $disk\n" >&2
+  fi	
+
   return $failed 
 }
 
@@ -71,6 +97,7 @@ backup_snapshot() {
   local disk=$1
   local snapshotname=$2
   local destPath=$3
+  local destName=$4
 
   if [ ! -d $destPath ]
   then
@@ -90,7 +117,7 @@ backup_snapshot() {
     return 1
   fi
 
-  cloud-qemu-img convert -f qcow2 -O qcow2 -s $snapshotname $disk $destPath/$snapshotname >& /dev/null
+  cloud-qemu-img convert -f qcow2 -O qcow2 -s $snapshotname $disk $destPath/$destName >& /dev/null
   if [ $? -gt 0 ]
   then
     printf "Failed to backup $snapshotname for disk $disk to $destPath" >&2
@@ -107,8 +134,10 @@ bflag=
 nflag=
 pathval=
 snapshot=
+tmplName=
+deleteDir=
 
-while getopts 'c:d:r:n:b:p:' OPTION
+while getopts 'c:d:r:n:b:p:t:f' OPTION
 do
   case $OPTION in
   c)	cflag=1
@@ -128,6 +157,10 @@ do
 	;;
   p)    destPath="$OPTARG"
         ;;
+  t)    tmplName="$OPTARG"
+	;;
+  f)    deleteDir=1
+	;;
   ?)	usage
 	;;
   esac
@@ -140,11 +173,11 @@ then
   exit $?
 elif [ "$dflag" == "1" ]
 then
-  destroy_snapshot $pathval $snapshot
+  destroy_snapshot $pathval $snapshot $deleteDir
   exit $?
 elif [ "$bflag" == "1" ]
 then
-  backup_snapshot $pathval $snapshot $destPath
+  backup_snapshot $pathval $snapshot $destPath $tmplName
   exit $?
 elif [ "$rflag" == "1" ]
 then
diff --git a/scripts/storage/secondary/createtmplt.sh b/scripts/storage/secondary/createtmplt.sh
index c686b8b7640..a2d296332df 100755
--- a/scripts/storage/secondary/createtmplt.sh
+++ b/scripts/storage/secondary/createtmplt.sh
@@ -3,7 +3,7 @@
 # createtmplt.sh -- install a template
 
 usage() {
-  printf "Usage: %s: -t  -n  -f  -s  -c  -d  -h  [-u]\n" $(basename $0) >&2
+  printf "Usage: %s: -t  -n  -f  -c  -d  -h  [-u]\n" $(basename $0) >&2
 }
 
 
@@ -47,8 +47,7 @@ untar() {
 
 uncompress() {
   local ft=$(file $1| awk -F" " '{print $2}')
-  local imgfile=${1%.*} #strip out trailing file suffix
-  local tmpfile=${imgfile}.tmp
+  local tmpfile=${1}.tmp
 
   case $ft in
   gzip)  gunzip -q -c $1 > $tmpfile
@@ -68,8 +67,8 @@ uncompress() {
     return 1 
   fi
  
-  mv $tmpfile $imgfile
-  printf "$imgfile"
+  rm -f $1
+  printf $tmpfile
 
   return 0
 }
@@ -78,16 +77,10 @@ create_from_file() {
   local tmpltfs=$1
   local tmpltimg=$2
   local tmpltname=$3
-  local volsize=$4
-  local cleanup=$5
 
   #copy the file to the disk
   mv $tmpltimg /$tmpltfs/$tmpltname
 
-#  if [ "$cleanup" == "true" ]
-#  then
-#    rm -f $tmpltimg
-#  fi
 }
 
 tflag=
@@ -113,7 +106,6 @@ do
 		tmpltimg="$OPTARG"
 		;;
   s)	sflag=1
-		volsize="$OPTARG"
 		;;
   c)	cflag=1
 		cksum="$OPTARG"
@@ -143,77 +135,48 @@ then
  exit 2
 fi
 
+mkdir -p $tmpltfs
+
+if [ ! -f $tmpltimg ] 
+then
+  printf "root disk file $tmpltimg doesn't exist\n"
+  exit 3
+fi
+
 if [ -n "$cksum" ]
 then
   verify_cksum $cksum $tmpltimg
 fi
 
-#if [ ! -d /$tmpltfs ] 
-#then
-#  mkdir /$tmpltfs
-#  if [ $? -gt 0 ] 
-#  then
-#    printf "Failed to create user fs $tmpltfs\n" >&2
-#    exit 1
-#  fi
-#fi
-
 tmpltimg2=$(uncompress $tmpltimg)
-if [ $? -ne 0 ]
-then
-  rollback_if_needed $tmpltfs 2 "failed to uncompress $tmpltimg\n"
-fi
+rollback_if_needed $tmpltfs $? "failed to uncompress $tmpltimg\n"
 
-tmpltimg2=$(untar $tmpltimg2 /$tmpltfs vmi-root)
-if [ $? -ne 0 ]
-then
-  rollback_if_needed $tmpltfs 2 "tar archives not supported\n"
-fi
+tmpltimg2=$(untar $tmpltimg2)
+rollback_if_needed $tmpltfs $? "tar archives not supported\n"
 
-if [ ! -f $tmpltimg2 ] 
+if [ ${tmpltname%.vhd} != ${tmpltname} ]
 then
-  rollback_if_needed $tmpltfs 2 "root disk file $tmpltimg doesn't exist\n"
-  exit 3
-fi
-
-# need the 'G' suffix on volume size
-if [ ${volsize:(-1)} != G ]
-then
-  volsize=${volsize}G
-fi
-
-#determine source file size -- it needs to be less than or equal to volsize
-imgsize=$(ls -lh $tmpltimg2| awk -F" " '{print $5}')
-if [ ${imgsize:(-1)} == G ] 
-then
-  imgsize=${imgsize%G} #strip out the G 
-  imgsize=${imgsize%.*} #...and any decimal part
-  let imgsize=imgsize+1 # add 1 to compensate for decimal part
-  volsizetmp=${volsize%G}
-  if [ $volsizetmp -lt $imgsize ]
-  then
-    volsize=${imgsize}G  
+  if  which  vhd-util 2>/dev/null
+  then 
+    vhd-util check -n ${tmpltimg2} > /dev/null
+    rollback_if_needed $tmpltfs $? "vhd tool check $tmpltimg2 failed\n"
   fi
 fi
 
-tgtfile=${tmpltfs}/vmi-root-${tmpltname}
+imgsize=$(ls -l $tmpltimg2| awk -F" " '{print $5}')
 
-create_from_file $tmpltfs $tmpltimg2 $tmpltname $volsize $cleanup
+create_from_file $tmpltfs $tmpltimg2 $tmpltname
 
-tgtfilename=$(echo $tmpltimg2 | awk -F"/" '{print $NF}') 
 touch /$tmpltfs/template.properties
 rollback_if_needed $tmpltfs $? "Failed to create template.properties file"
 echo -n "" > /$tmpltfs/template.properties
 
 today=$(date '+%m_%d_%Y')
 echo "filename=$tmpltname" > /$tmpltfs/template.properties
-echo "snapshot.name=$today" >> /$tmpltfs/template.properties
 echo "description=$descr" >> /$tmpltfs/template.properties
-echo "name=$tmpltname" >> /$tmpltfs/template.properties
 echo "checksum=$cksum" >> /$tmpltfs/template.properties
 echo "hvm=$hvm" >> /$tmpltfs/template.properties
-echo "volume.size=$volsize" >> /$tmpltfs/template.properties
-
+echo "size=$imgsize" >> /$tmpltfs/template.properties
 
 if [ "$cleanup" == "true" ]
 then
diff --git a/scripts/storage/secondary/installrtng.sh b/scripts/storage/secondary/installrtng.sh
index e2e423d0745..ff801b5d536 100755
--- a/scripts/storage/secondary/installrtng.sh
+++ b/scripts/storage/secondary/installrtng.sh
@@ -1,9 +1,9 @@
 #!/bin/bash
 # $Id: installrtng.sh 11251 2010-07-23 23:40:44Z abhishek $ $HeadURL: svn://svn.lab.vmops.com/repos/vmdev/java/scripts/storage/secondary/installrtng.sh $
 usage() {
-  printf "Usage: %s: -m  -f  [-F]\n" $(basename $0) >&2
+  printf "Usage: %s: -m  -f  [-F]\n" $(basename $0) >&2
   printf "or\n" >&2
-  printf "%s: -m  -u  [-F]\n" $(basename $0) >&2
+  printf "%s: -m  -u  [-F]\n" $(basename $0) >&2
 }
 
 mflag=
@@ -93,7 +93,7 @@ then
   wget -O $tmpfile $url
   if [ $? -ne 0 ]
   then
-    echo "Failed to fetch routing template from $url"
+    echo "Failed to fetch system vm template from $url"
     exit 5
   fi
 fi
@@ -109,20 +109,23 @@ then
 fi
 
 
-$(dirname $0)/createtmplt.sh -s 2 -d 'DomR Template' -n $localfile -t $destdir/ -f $tmpfile -u  &> /dev/null
+$(dirname $0)/createtmplt.sh -s 2 -d 'SystemVM Template' -n $localfile -t $destdir/ -f $tmpfile -u  &> /dev/null
 
 if [ $? -ne 0 ]
 then
-  echo "Failed to install routing template $tmpltimg to $destdir"
+  echo "Failed to install system vm template $tmpltimg to $destdir"
 fi
 
+tmpltfile=$destdir/$localfile
+tmpltsize=$(ls -l $tmpltfile| awk -F" " '{print $5}')
+
 echo "vhd=true" >> $destdir/template.properties
 echo "id=1" >> $destdir/template.properties
 echo "public=true" >> $destdir/template.properties
 echo "vhd.filename=$localfile" >> $destdir/template.properties
 echo "uniquename=routing" >> $destdir/template.properties
-echo "vhd.virtualsize=2147483648" >> $destdir/template.properties
-echo "virtualsize=2147483648" >> $destdir/template.properties
-echo "vhd.size=2101252608" >> $destdir/template.properties
+echo "vhd.virtualsize=$tmpltsize" >> $destdir/template.properties
+echo "virtualsize=$tmpltsize" >> $destdir/template.properties
+echo "vhd.size=$tmpltsize" >> $destdir/template.properties
 
-echo "Successfully installed routing template $tmpltimg to $destdir"
+echo "Successfully installed system VM template $tmpltimg to $destdir"
diff --git a/scripts/vm/hypervisor/kvm/setup_agent.sh b/scripts/vm/hypervisor/kvm/setup_agent.sh
new file mode 100755
index 00000000000..89bcfb0dc10
--- /dev/null
+++ b/scripts/vm/hypervisor/kvm/setup_agent.sh
@@ -0,0 +1,182 @@
+#! /bin/bash
+# Did cloud-agent installed
+#set -x
+install_cloud_agent() {
+    local dev=$1
+    local retry=10
+    which cloud-setup-agent
+    if [ $? -gt 0 ]
+    then 
+        # download  the repo
+        which wget
+        if [ $? -gt 0 ]
+        then
+            yum install wget -y
+            if [ $? -gt 0 ]
+            then
+                printf "failed to install wget"
+                exit 1
+            fi 
+        fi
+        wget -N -P /etc/yum.repos.d/ http://download.cloud.com/foss/fedora/cloud.repo
+        if [ $? -gt 0 ]
+        then
+            printf "Failed to download repo"
+            exit 1
+        fi
+        if [ "$dev" == "1" ]
+        then
+            sed -i 's/\(baseurl\)\(.*\)/\1=http:\/\/yumrepo.lab.vmops.com\/repositories\/fedora\/vmdev\/oss\//'	/etc/yum.repos.d/cloud.repo 
+        fi
+        while [ "$retry" -gt "0" ]
+        do
+            yum clean all
+            yum install cloud-agent -y
+            if [ $? -gt 0 ]
+            then
+                let retry=retry-1
+            else
+                break
+            fi
+        done
+    else
+	    # is there update?
+        while [ "$retry" -gt "0" ]
+        do
+	        yum clean all
+	        yum update cloud-agent -y
+            if [ $? -gt 0 ]
+            then
+                let retry=retry-1
+            else
+                break
+            fi
+
+        done
+    fi
+    
+    if [ $? -gt 0 ]
+    then
+        printf "Failed to install agent"
+        exit 2
+    fi
+}
+
+install_cloud_consoleP() {
+    local dev=$1
+    local retry=10
+    which cloud-setup-console-proxy
+    if [ $? -gt 0 ]
+    then 
+        # download  the repo
+        which wget
+        if [ $? -gt 0 ]
+        then
+            yum install wget -y
+            if [ $? -gt 0 ]
+            then
+                printf "failed to install wget"
+                exit 1
+            fi 
+        fi
+        wget -N -P=/etc/yum.repos.d/ http://download.cloud.com/foss/fedora/cloud.repo
+        if [ $? -gt 0 ]
+        then
+            printf "Failed to download repo"
+            exit 1
+        fi
+        if [ "$dev" == "1" ]
+        then
+            sed -i 's/\(baseurl\)\(.*\)/\1=http:\/\/yumrepo.lab.vmops.com\/repositories\/fedora\/vmdev\/oss\//'	/etc/yum.repos.d/cloud.repo 
+        fi
+        while [ "$retry" -gt "0" ]
+        do
+            yum clean all
+            yum install cloud-console-proxy -y
+            if [ $? -gt 0 ]
+            then
+                let retry=retry-1
+            else
+                break
+            fi
+        done
+    else
+	    # is there update?
+        while [ "$retry" -gt "0" ]
+        do
+	        yum clean all
+	        yum update cloud-console-proxy -y
+            if [ $? -gt 0 ]
+            then
+                let retry=retry-1
+            else
+                break
+            fi
+
+        done
+    fi
+    
+    if [ $? -gt 0 ]
+    then
+        printf "Failed to install console"
+        exit 2
+    fi
+}
+cloud_agent_setup() {
+    local host=$1
+    local zone=$2
+    local pod=$3
+    local cluster=$4
+    local guid=$5
+    # disable selinux
+    selenabled=`cat /selinux/enforce`
+    if [ "$selenabled" == "1" ]
+    then
+        sed -i  's/\(SELINUX\)\(.*\)/\1=permissive/' /etc/selinux/config
+        setenforce 0
+    fi
+    cloud-setup-agent --host=$host --zone=$zone --pod=$pod --cluster=$cluster --guid=$guid -a > /dev/null
+}
+
+cloud_consoleP_setup() {
+    local host=$1
+    local zone=$2
+    local pod=$3
+    cloud-setup-console-proxy --host=$host --zone=$zone --pod=$pod -a > /dev/null
+}
+
+host=
+zone=
+pod=
+cluster=
+guid=
+dflag=
+while getopts 'h:z:p:u:c:d' OPTION
+do
+  case $OPTION in
+  h) 
+        host="$OPTARG"
+        ;;
+  z)    
+        zone="$OPTARG"
+        ;;
+  p)    
+        pod="$OPTARG"
+        ;;
+  c)    
+        cluster="$OPTARG"
+        ;;
+  u)    
+        guid="$OPTARG"
+        ;;
+  d)    
+        dflag=1
+        ;;
+  *)    ;;
+  esac
+done
+
+#install_cloud_agent $dflag
+#install_cloud_consoleP $dflag
+cloud_agent_setup $host $zone $pod $cluster $guid
+#cloud_consoleP_setup $host $zone $pod
diff --git a/scripts/vm/hypervisor/xenserver/check_heartbeat.sh b/scripts/vm/hypervisor/xenserver/check_heartbeat.sh
deleted file mode 100755
index 2f5b40022c3..00000000000
--- a/scripts/vm/hypervisor/xenserver/check_heartbeat.sh
+++ /dev/null
@@ -1,58 +0,0 @@
-#!/bin/bash
-
-#set -x
- 
-usage() {
-  printf "Usage: %s [uuid of this host] [interval in seconds]\n" $(basename $0) >&2
-
-}
-
-if [ -z $1 ]; then
-  usage
-  exit 2
-fi
-
-if [ -z $2 ]; then
-  usage
-  exit 3
-fi
-
-
-date=`date +%s`
-hbs=`lvscan | grep hb-$1 | awk '{print $2}'`
-for hb in $hbs
-do
-  hb=${hb:1:`expr ${#hb} - 2`}
-  active=`lvscan | grep $hb | awk '{print $1}'`
-  if [ "$active" == "inactive" ]; then
-    lvchange -ay $hb
-    if [ ! -L $hb ]; then
-      continue;
-    fi
-  fi
-  ping=`dd if=$hb bs=1 count=100`
-  if [ $? -ne 0 ]; then
-    continue;
-  fi
-  diff=`expr $date - $ping`
-  if [ $diff -lt $2 ]; then
-    echo "=====> ALIVE <====="
-    exit 0;    
-  fi
-done
-
-hbs=`ls -l /var/run/sr-mount/*/hb-$1 | awk '{print $9}'`
-for hb in $hbs
-do
-  ping=`cat $hb`
-  if [ $? -ne 0 ]; then
-    continue;
-  fi
-  diff=`expr $date - $ping`
-  if [ $diff -lt $2 ]; then
-    echo "=====> ALIVE <====="
-    exit 0;    
-  fi
-done
-
-echo "=====> DEAD <======"
diff --git a/scripts/vm/hypervisor/xenserver/find_bond.sh b/scripts/vm/hypervisor/xenserver/find_bond.sh
deleted file mode 100755
index 2818259c06f..00000000000
--- a/scripts/vm/hypervisor/xenserver/find_bond.sh
+++ /dev/null
@@ -1,111 +0,0 @@
-#!/bin/sh
-# $Id: find_bond.sh 10510 2010-07-11 10:10:03Z alex $ $HeadURL: svn://svn.lab.vmops.com/repos/vmdev/java/scripts/vm/hypervisor/xenserver/find_bond.sh $
-
-#set -x
- 
-
-cleanup_vlan() {
-  for vlan in `xe vlan-list | grep uuid | awk '{print $NF}'`; do 
-    untagged=$(xe vlan-param-list uuid=$vlan | grep untagged | awk '{print $NF}')
-    network=$(xe pif-param-get param-name=network-uuid uuid=$untagged)
-    xe vlan-destroy uuid=$vlan
-    xe network-destroy uuid=$network
-  done
-}
-
-usage() {
-  echo "$0 device"
-  exit 1
-}
-
-sflag=
-dflag=
-
-while getopts 'sd' OPTION
-do
-  case $OPTION in
-  d)    dflag=1
-         ;;
-  s)    sflag=1
-         ;;
-  ?)    usage
-         exit 1
-         ;;
-  esac
-done
-
-shift $(($OPTIND - 1))
-nic=$1
-
-[ -z "$nic" ] && usage
-
-addr=$(ip addr | grep $nic | grep inet | awk '{print $2}')
-addr=${addr%/*}
-bridges=$(brctl show | grep -v bridge | awk '{print $1}')
-
-host_uuid=$(xe host-list hostname=$(hostname) | grep uuid | awk '{print $NF}')
-if [ -z "$host_uuid" ]; then
-  printf "Unable to find host uuid using $(hostname)\n" >&2
-  exit 2
-fi
-
-
-if [ -z "$addr" ]; then
-  printf "Unable to find an ip address for $nic\n" >&2
-  exit 3
-fi
-
-current=$(brctl show | grep $nic | awk '{print $NF}')
-for dev in `ip addr | grep mtu | grep -v -E "\.[0-9]*@|lo|$nic|$current" | awk '{print $2}'`
-do
-  dev=${dev%:}
-  echo $bridges | grep $dev >/dev/null 2>&1
-  br=$?
-  ifconfig $dev | grep UP >/dev/null 2>&1
-  rc=$?
-  if [ $rc -eq 1 ]; then
-    ifconfig $dev up
-    sleep 4
-  fi
-  arping -q -c 1 -w 2 -D -I $dev $addr >/dev/null 2>&1
-  rc=$?
-  if [ $rc -ne 1 ]; then
-    continue;
-  fi
-
-  if [ $br -ne 0 ]; then
-    # What we've found is the naked nic.
-    pif_uuid=$(xe pif-list device=$dev host-uuid=$host_uuid | grep -B 3 "( RO): -1" | grep uuid | awk '{print $NF}')
-    if [ -z "$pif_uuid" ]; then
-      mac=$(ifconfig $dev | grep HWaddr | awk '{print $NF}')
-      pif_uuid=$(xe pif-introduce host-uuid=$host_uuid device=$dev mac=$mac)
-    fi
-
-    if [ -z $pif_uuid ]; then
-      continue;
-    fi
-
-    bridge=$(xe network-list PIF-uuids=$pif_uuid | grep bridge | awk '{print $NF}')
-    if [ -z $bridge ]; then
-      continue;
-    fi
-
-    xe pif-plug uuid=$pif_uuid
-
-    echo ">>>$dev<<<"
-    exit 0    
-  else
-    # What we've found is the bridge
-    network_uuid=`xe network-list bridge=$dev | grep uuid | awk '{print $NF}'`
-    if [ -z "$network_uuid" ]; then
-      continue;
-    fi
-    pif=`xe pif-list network-uuid=$network_uuid host-uuid=$host_uuid VLAN=-1 | grep device | awk '{print $NF}'`
-    if [ -z "$pif" ]; then
-      continue;
-    fi
-    echo ">>>$pif<<<"
-    exit 0
-  fi
-done
-exit 4
diff --git a/scripts/vm/hypervisor/xenserver/hostvmstats.py b/scripts/vm/hypervisor/xenserver/hostvmstats.py
index 89af959db61..a5047697c9f 100644
--- a/scripts/vm/hypervisor/xenserver/hostvmstats.py
+++ b/scripts/vm/hypervisor/xenserver/hostvmstats.py
@@ -7,25 +7,19 @@ import time
 import logging
 logging.basicConfig(filename='/tmp/xapilog',level=logging.DEBUG)
                       
-def get_xapi_session():
-    xapi = XenAPI.xapi_local();
-    xapi.login_with_password("","")
-    return xapi._session
-
-def get_stats(collect_host_stats, consolidation_function, interval, start_time):
+def get_stats(session, collect_host_stats, consolidation_function, interval, start_time):
   try:
-    session = get_xapi_session()
     
     if collect_host_stats == "true" :
     	url = "http://localhost/rrd_updates?"
-   	url += "session_id=" + session
+   	url += "session_id=" + session._session
    	url += "&host=" + collect_host_stats
     	url += "&cf=" + consolidation_function
     	url += "&interval=" + str(interval)
     	url += "&start=" + str(int(time.time())-100)
     else :
     	url = "http://localhost/rrd_updates?"
-   	url += "session_id=" + session
+   	url += "session_id=" + session._session
    	url += "&host=" + collect_host_stats
     	url += "&cf=" + consolidation_function
     	url += "&interval=" + str(interval)
diff --git a/scripts/vm/hypervisor/xenserver/id_rsa.cloud b/scripts/vm/hypervisor/xenserver/id_rsa.cloud
new file mode 100644
index 00000000000..ea46a3cebb6
--- /dev/null
+++ b/scripts/vm/hypervisor/xenserver/id_rsa.cloud
@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEogIBAAKCAQEA3VD1tGRDn3stlJvPNXmQZdQCNjqcfY+xlitd5q0n3KYqJ5OB
+rty3/00XBUdLt31TbQ4dv+GR7uEr+ex7rm0jjmTFKV4rHYPi882CuC5+bkBp5R4k
++mpcyKbxb+IoNS9ItbiExQxMiiRQpHvNem0GGnNFO3lElRPwUFs8evTvZu5HcTj4
+k4RJLJ66jeIGJ3sMAJ03SICGwfEZjrsyeOMwJk7cH8WNeuNzxzoZd9v02eI0lHdK
+9O5z7FwrxvRBbzsmJ0EwuhbH8pR7WR6kGLTNP9KEwtrnzV1LYWd+rFoSeh6ImExG
+7fma3Ldydg8CPTQsjvCEQUxiuV1/x5am5VJlUwIBIwKCAQEA0KtrUk/n/MSYsLAp
+xLRyNB+qUGMl1Xjao4f5cxhKJ8/emlfgrC8xI+mZXL+QiG7ZoVZz0ixzprcMNMkG
+5kmlLnxE3dxxy18Xz+2nIq9+hTVrKHuB82uZT3jVAxcP96GcU5C3snlPeu8KNK8+
+FFgqU3P/cpbo5FSgwMsNI3k5fkyffYtmBdtjZhWXJqnA9+bMdCmYEKyQFWp18LvV
+pjGx1jLFZTx9+aDz7gdIk21zbVXmwQmnS1fVKJEByTMvokpvdJUvDedvpgqGqX/g
+IXkTXe49pYhYwxVguLK6FXyQBwOuUsnur2A79T3wBvzEMozkYLkEG/zcw0fyo3iC
+fdzc6wKBgQD2gq+kUc2r/+xE+smIej2ICvFZZlSh1ko2tVmVUHuuuMCuBt054Dq9
+mf8/yIbXSvVtuBMJ+jewVnKfhucEQKf6E1jBdQShezlomFLOQ8cFQJhT6tAwJl/k
+TR+OjeTuOcBknkE8nstNt7hAkZxY6h/Lu54OM9AkXyZ9skx7gHh+IwKBgQDl1f09
+YkoM9rqXM8lMKjF0z81T4ACCaFUA6ZKjSZelyG+azJDlRFNWX1In3Kq6aInpZPzs
+owwIS9tjkXIaLR1wDJ+K8IGJQ19sqCzv3/kBCDXA6mqXkkPR80xRi4wuZ3lETOdL
+OBXPffuQaKxk32esqsxK6As1LgH4+048JS23EQKBgQCpCSf7pc7cV7f0yTm8q5fo
+QgSVEvg0da87dQo6gFTPlKFhY8rl25X+WvgrvLQ726D6x12DLzwhJVXpu5cY2+Dl
+/qNC0+XrEqsF5MsRGIh4oVKCr6SzTYOVPDLlaJz7IElpkRbKe4QYCPNfecpLmTpf
+0Rvse0zlvZa8l4Tm+QIqmwKBgBOzQZeMFPnMAV1q1r1is8gvEZl5maTHHTqXrXu1
+2cxhoyqGkBOmxVCL09eH8WBvXEc0irUyjAC2C32QH7kZz1K/QOAF/Hl6zao6TP6e
+K0k7N861AdJ6QFPTBoqlj6w0wUBeXPfRm3gvXrSbQfoEhTqvjdqI6wSO6jnpp57B
+W7CbAoGABFHMVXEyT3SliMSRtiCuDOrtl9E/aiOByPulXolqth5WDSel31Lz+iY7
+ldOLNQO/oononTStdd0fDGChl3WXBSOToJJ/HjIWH05bDY9n2EDAyZvmaW9rX3JQ
+pH9c/1vlD9lxDEBvq4JXmTtdL0Ho00F5vVHnWnwINtfx6c5BIjg=
+-----END RSA PRIVATE KEY-----
diff --git a/scripts/vm/hypervisor/xenserver/launch_hb.sh b/scripts/vm/hypervisor/xenserver/launch_hb.sh
deleted file mode 100755
index 849565e1e71..00000000000
--- a/scripts/vm/hypervisor/xenserver/launch_hb.sh
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/bin/bash
-
-#set -x
- 
-usage() {
-  printf "Usage: %s [uuid of this host] [interval in seconds]\n" $(basename $0)
-
-}
-
-if [ -z $1 ]; then
-  usage
-  exit 2
-fi
-
-if [ -z $2 ]; then
-  usage
-  exit 3
-fi
-
-if [ ! -f /opt/xensource/bin/xenheartbeat.sh ]; then
-  printf "Error: Unable to find xenheartbeat.sh to launch\n"
-  exit 4
-fi
-
-for psid in `ps -ef | grep xenheartbeat | grep -v grep | awk '{print $2}'`; do
-  kill $psid
-done
-
-nohup /opt/xensource/bin/xenheartbeat.sh $1 $2 >/var/log/heartbeat.log 2>&1 &
-echo "======> DONE <======"
diff --git a/scripts/vm/hypervisor/xenserver/prepsystemvm.sh b/scripts/vm/hypervisor/xenserver/prepsystemvm.sh
deleted file mode 100755
index 4a5b7f0e695..00000000000
--- a/scripts/vm/hypervisor/xenserver/prepsystemvm.sh
+++ /dev/null
@@ -1,232 +0,0 @@
-#/bin/bash
-# $Id: prepsystemvm.sh 10800 2010-07-16 13:48:39Z edison $ $HeadURL: svn://svn.lab.vmops.com/repos/vmdev/java/scripts/vm/hypervisor/xenserver/prepsystemvm.sh $
-
-#set -x
-
-mntpath() {
-  local vmname=$1
-  echo "/mnt/$vmname"
-}
-
-mount_local() {
-   local vmname=$1
-   local disk=$2
-   local path=$(mntpath $vmname)
-
-   mkdir -p ${path}
-   mount $disk ${path} 
-
-   return $?
-}
-
-umount_local() {
-   local vmname=$1
-   local path=$(mntpath $vmname)
-
-   umount  $path
-   local ret=$?
-   
-   rm -rf $path
-   return $ret
-}
-
-
-patch_scripts() {
-   local vmname=$1
-   local patchfile=$2
-   local path=$(mntpath $vmname)
-
-   local oldmd5=
-   local md5file=${path}/md5sum
-   [ -f ${md5file} ] && oldmd5=$(cat ${md5file})
-   local newmd5=$(md5sum $patchfile | awk '{print $1}')
-
-   if [ "$oldmd5" != "$newmd5" ]
-   then
-     tar xzf $patchfile -C ${path}
-     echo ${newmd5} > ${md5file}
-   fi
-
-   return 0
-}
-
-#
-# To use existing console proxy .zip-based package file
-#
-patch_console_proxy() {
-   local vmname=$1
-   local patchfile=$2
-   local path=$(mntpath $vmname)
-   local oldmd5=
-   local md5file=${path}/usr/local/cloud/systemvm/md5sum
-
-   [ -f ${md5file} ] && oldmd5=$(cat ${md5file})
-   local newmd5=$(md5sum $patchfile | awk '{print $1}')
-
-   if [ "$oldmd5" != "$newmd5" ]
-   then
-     echo "All" | unzip $patchfile -d ${path}/usr/local/cloud/systemvm >/dev/null 2>&1
-     chmod 555 ${path}/usr/local/cloud/systemvm/run.sh
-     find ${path}/usr/local/cloud/systemvm/ -name \*.sh | xargs chmod 555
-     echo ${newmd5} > ${md5file}
-   fi
-
-   return 0
-}
-
-consoleproxy_svcs() {
-   local vmname=$1
-   local path=$(mntpath $vmname)
-
-   chroot ${path} /sbin/chkconfig cloud on
-   chroot ${path} /sbin/chkconfig postinit on
-   chroot ${path} /sbin/chkconfig domr_webserver off
-   chroot ${path} /sbin/chkconfig haproxy off ;
-   chroot ${path} /sbin/chkconfig dnsmasq off
-   chroot ${path} /sbin/chkconfig sshd on
-   chroot ${path} /sbin/chkconfig httpd off
-   chroot ${path} /sbin/chkconfig nfs off
-   chroot ${path} /sbin/chkconfig nfslock off
-   chroot ${path} /sbin/chkconfig rpcbind off
-   chroot ${path} /sbin/chkconfig rpcidmap off
-
-   cp ${path}/etc/sysconfig/iptables-consoleproxy ${path}/etc/sysconfig/iptables
-}
-
-secstorage_svcs() {
-   local vmname=$1
-   local path=$(mntpath $vmname)
-
-   chroot ${path} /sbin/chkconfig cloud on
-   chroot ${path} /sbin/chkconfig postinit on
-   chroot ${path} /sbin/chkconfig domr_webserver off
-   chroot ${path} /sbin/chkconfig haproxy off ;
-   chroot ${path} /sbin/chkconfig dnsmasq off
-   chroot ${path} /sbin/chkconfig sshd on
-   chroot ${path} /sbin/chkconfig httpd off
-    
-
-   cp ${path}/etc/sysconfig/iptables-secstorage ${path}/etc/sysconfig/iptables
-   mkdir -p ${path}/var/log/cloud
-}
-
-routing_svcs() {
-   local vmname=$1
-   local path=$(mntpath $vmname)
-
-   chroot ${path} /sbin/chkconfig cloud off
-   chroot ${path} /sbin/chkconfig domr_webserver on ; 
-   chroot ${path} /sbin/chkconfig haproxy on ; 
-   chroot ${path} /sbin/chkconfig dnsmasq on
-   chroot ${path} /sbin/chkconfig sshd on
-   chroot ${path} /sbin/chkconfig nfs off
-   chroot ${path} /sbin/chkconfig nfslock off
-   chroot ${path} /sbin/chkconfig rpcbind off
-   chroot ${path} /sbin/chkconfig rpcidmap off
-   cp ${path}/etc/sysconfig/iptables-domr ${path}/etc/sysconfig/iptables
-}
-
-lflag=
-dflag=
-
-while getopts 't:l:d:' OPTION
-do
-  case $OPTION in
-  l)	lflag=1
-	vmname="$OPTARG"
-        ;;
-  t)    tflag=1
-        vmtype="$OPTARG"
-        ;;
-  d)    dflag=1
-        rootdisk="$OPTARG"
-        ;;
-  *)    ;;
-  esac
-done
-
-if [ "$lflag$tflag$dflag" != "111" ]
-then
-  printf "Error: Not enough parameter\n" >&2
-  exit 1
-fi
-
-
-mount_local $vmname $rootdisk
-
-if [ $? -gt 0 ]
-then
-  printf "Failed to mount disk $rootdisk for $vmname\n" >&2
-  exit 1
-fi
-
-if [ -f $(dirname $0)/patch.tgz ]
-then
-  patch_scripts $vmname $(dirname $0)/patch.tgz
-  if [ $? -gt 0 ]
-  then
-    printf "Failed to apply patch patch.zip to $vmname\n" >&2
-    umount_local $vmname
-    exit 4
-  fi
-fi
-
-cpfile=$(dirname $0)/systemvm-premium.zip
-if [ "$vmtype" == "consoleproxy" ] || [ "$vmtype" == "secstorage" ]  && [ -f $cpfile ]
-then
-  patch_console_proxy $vmname $cpfile
-  if [ $? -gt 0 ]
-  then
-    printf "Failed to apply patch $patch $cpfile to $vmname\n" >&2
-    umount_local $vmname
-    exit 5
-  fi
-fi
-
-# domr is 64 bit, need to copy 32bit chkconfig to domr
-# this is workaroud, will use 32 bit domr
-dompath=$(mntpath $vmname)
-cp /sbin/chkconfig $dompath/sbin
-# copy public key to system vm
-cp $(dirname $0)/id_rsa.pub  $dompath/root/.ssh/authorized_keys
-#empty known hosts
-echo "" > $dompath/root/.ssh/known_hosts
-
-if [ "$vmtype" == "router" ]
-then
-  routing_svcs $vmname
-  if [ $? -gt 0 ]
-  then
-    printf "Failed to execute routing_svcs\n" >&2
-    umount_local $vmname
-    exit 6
-  fi
-fi
-
-
-if [ "$vmtype" == "consoleproxy" ]
-then
-  consoleproxy_svcs $vmname
-  if [ $? -gt 0 ]
-  then
-    printf "Failed to execute consoleproxy_svcs\n" >&2
-    umount_local $vmname
-    exit 7
-  fi
-fi
-
-if [ "$vmtype" == "secstorage" ]
-then
-  secstorage_svcs $vmname
-  if [ $? -gt 0 ]
-  then
-    printf "Failed to execute secstorage_svcs\n" >&2
-    umount_local $vmname
-    exit 8
-  fi
-fi
-
-
-umount_local $vmname
-
-exit $?
diff --git a/scripts/vm/hypervisor/xenserver/setup_heartbeat_sr.sh b/scripts/vm/hypervisor/xenserver/setup_heartbeat_sr.sh
deleted file mode 100755
index e6d394866ce..00000000000
--- a/scripts/vm/hypervisor/xenserver/setup_heartbeat_sr.sh
+++ /dev/null
@@ -1,83 +0,0 @@
-#!/bin/bash
-
-#set -x
- 
-usage() {
-  printf "Usage: %s [uuid of this host] [uuid of the sr to place the heartbeat]\n" $(basename $0) >&2
-
-}
-
-
-if [ -z $1 ]; then
-  usage
-  exit 2
-fi
-
-if [ -z $2 ]; then
-  usage
-  exit 3
-fi
-
-if [ `xe host-list | grep $1 | wc -l` -ne 1 ]; then
-  printf "Error: Unable to find the host uuid: $1\n" >&2
-  usage
-  exit 4
-fi
-
-if [ `xe sr-list uuid=$2 | wc -l`  -eq 0 ]; then 
-  printf "Error: Unable to find SR with uuid: $2\n" >&2
-  usage
-  exit 5
-fi
-
-if [ `xe pbd-list sr-uuid=$2 | grep -B 1 $1 | wc -l` -eq 0 ]; then
-  printf "Error: Unable to find a pbd for the SR: $2\n" >&2
-  usage
-  exit 6
-fi
-
-srtype=`xe sr-param-get param-name=type uuid=$2`
-
-if [ "$srtype" == "nfs" ];then
-  filename=/var/run/sr-mount/$2/hb-$1
-  files=`ls /var/run/sr-mount/$2 | grep "hb-$1"`
-  if [ -z "$files" ]; then
-    date=`date +%s`
-    echo "$date" > $filename
-  fi
-
-else 
-
-  link=/dev/VG_XenStorage-$2/hb-$1
-  lv=`lvscan | grep $link`
-  if [ -z "$lv" ]; then
-    lvcreate VG_XenStorage-$2 -n hb-$1 --size 1M
-    if [ $? -ne 0 ]; then
-      printf "Error: Unable to create heartbeat SR\n" >&2
-      exit 7
-    fi
-    lv=`lvscan | grep $link`
-    if [ -z "$lv" ]; then
-      printf "Error: Unable to create heartbeat SR\n" >&2
-      exit 8
-    fi
-  fi
-
-  if [ `echo $lv | awk '{print $1}'` == "inactive" ]; then
-    lvchange -ay $link
-    if [ $? -ne 0 ]; then
-      printf "Error: Unable to make the heartbeat SR active\n" >&2
-      exit 8
-    fi
-  fi
-
-  if [ ! -L $link ]; then
-    printf "Error: Unable to find the soft link $link\n" >&2
-    exit 9
-  fi
-
-  dd if=/dev/zero of=$link bs=1 count=100
-fi
-
-echo "======> DONE <======"
-exit 0
diff --git a/scripts/vm/hypervisor/xenserver/vmops b/scripts/vm/hypervisor/xenserver/vmops
index 3ac93f4abd5..4bffbef164f 100755
--- a/scripts/vm/hypervisor/xenserver/vmops
+++ b/scripts/vm/hypervisor/xenserver/vmops
@@ -4,6 +4,7 @@
 
 import os, sys, time
 import XenAPIPlugin
+import XenAPI
 sys.path.append("/opt/xensource/sm/")
 import SR, VDI, SRCommand, util, lvutil
 from util import CommandException
@@ -35,26 +36,23 @@ def echo(fn):
         return res
     return wrapped
 
+
+def get_xapi_session():
+    session = XenAPI.xapi_local();
+    session.login_with_password("","")
+    return session
+
 @echo
 def gethostvmstats(session, args):
     collect_host_stats = args['collectHostStats']
     consolidation_function = args['consolidationFunction']
     interval = args['interval']
     start_time = args['startTime']
-    result = hostvmstats.get_stats(collect_host_stats, consolidation_function, interval, start_time)
+    session = get_xapi_session()
+    result = hostvmstats.get_stats(session, collect_host_stats, consolidation_function, interval, start_time)
     return result
     
-@echo
-def find_bond(session, args):
-    nic = args['arg1']
-    try:
-        cmd = ["bash", "/opt/xensource/bin/find_bond.sh", nic]
-        txt = util.pread2(cmd)
-    except:
-        txt = ''
-    
-    return txt
-    
+   
 @echo
 def setup_iscsi(session, args):
    uuid=args['uuid']
@@ -420,41 +418,9 @@ def networkUsage(session, args):
 
     return txt
     
-@echo
-def setup_heartbeat_sr(session, args):
-    host = args['host']
-    sr = args['sr']
-    try:
-        cmd = ["bash", "/opt/xensource/bin/setup_heartbeat_sr.sh", host, sr]
-        txt = util.pread2(cmd)
-    except:
-        txt = ''
-    return txt
-   
-@echo
-def check_heartbeat(session, args):
-    host = args['host']
-    interval = args['interval']
-    try:
-       cmd = ["bash", "/opt/xensource/bin/check_heartbeat.sh", host, interval]
-       txt = util.pread2(cmd)
-    except:
-       txt=''
-    return txt
-    
-   
-@echo
-def heartbeat(session, args):
-    host = args['host']
-    interval = args['interval']
-    try: 
-       cmd = ["/bin/bash", "/opt/xensource/bin/launch_hb.sh", host, interval]
-       txt = util.pread2(cmd)
-    except:
-       txt='fail'
-    return txt
-
-def get_private_nic(session, args):
+def get_private_nic( args):
+    session = get_xapi_session()
+    vms = session.xenapi.VM.get_all()
     host_uuid = args.get('host_uuid')
     host = session.xenapi.host.get_by_uuid(host_uuid)
     piflist = session.xenapi.host.get_PIFs(host)
@@ -481,7 +447,7 @@ def can_bridge_firewall(session, args):
         util.pread2(['iptables', '-D', 'FORWARD',  '-j', 'RH-Firewall-1-INPUT'])
     except:
         util.SMlog('Chain BRIDGE-FIREWALL already exists')
-    privnic = get_private_nic(session, args)
+    privnic = get_private_nic(args)
     result = 'true'
     try:
         util.pread2(['/bin/bash', '-c', 'iptables -n -L FORWARD | grep BRIDGE-FIREWALL'])
@@ -496,8 +462,8 @@ def can_bridge_firewall(session, args):
     if not os.path.exists('/var/run/cloud'):
         os.makedirs('/var/run/cloud')
  
-    cleanup_rules_for_dead_vms(session)
-    cleanup_rules(session)
+    cleanup_rules_for_dead_vms()
+    cleanup_rules()
     
     return result
 
@@ -875,6 +841,8 @@ def get_rule_log_for_vm(session, vmName):
 def get_rule_logs_for_vms(session, args):
     host_uuid = args.pop('host_uuid')
     try:
+        session = get_xapi_session()
+
         thishost = session.xenapi.host.get_by_uuid(host_uuid)
         hostrec = session.xenapi.host.get_record(thishost)
         vms = hostrec.get('resident_VMs')
@@ -897,7 +865,9 @@ def get_rule_logs_for_vms(session, args):
     return ";".join(result)
 
 @echo
-def cleanup_rules_for_dead_vms(session):
+def cleanup_rules_for_dead_vms():
+  try:
+    session = get_xapi_session()
     vms = session.xenapi.VM.get_all()
     cleaned = 0
     for vm_name in [session.xenapi.VM.get_name_label(x) for x in vms]:
@@ -913,9 +883,16 @@ def cleanup_rules_for_dead_vms(session):
                 cleaned = cleaned+1
                 
     util.SMlog("Cleaned up rules for " + str(cleaned) + " vms")
+  except:
+    util.SMlog("Failed to cleanup rules for dead vms!")
+        
+
 
 @echo
-def cleanup_rules(session):
+def cleanup_rules():
+  try:
+    session = get_xapi_session()
+
     chainscmd = "iptables-save | grep '^:' | awk '{print $1}' | cut -d':' -f2"
     chains = util.pread2(['/bin/bash', '-c', chainscmd]).split('\n')
     cleaned = 0
@@ -942,6 +919,8 @@ def cleanup_rules(session):
         destroy_network_rules_for_vm(session, {'vmName':vmname})
                     
     util.SMlog("Cleaned up rules for " + str(len(cleanup)) + " chains")                
+  except:
+    util.SMlog("Failed to cleanup rules !")
 
 @echo
 def check_rule_log_for_vm(vmName, vmID, vmIP, domID, signature, seqno):
@@ -1005,6 +984,8 @@ def remove_rule_log_for_vm(vmName):
 
 @echo
 def network_rules(session, args):
+  try:
+    session = get_xapi_session()
     vm_name = args.get('vmName')
     vmName = vm_name
     vm_ip = args.get('vmIP')
@@ -1101,9 +1082,11 @@ def network_rules(session, args):
         return 'false'
     
     return 'true'
+  except:
+    util.SMlog("Failed to network rule !")
 
 
 
 if __name__ == "__main__":
-    XenAPIPlugin.dispatch({"pingtest": pingtest, "setup_heartbeat_sr":setup_heartbeat_sr, "check_heartbeat":check_heartbeat, "heartbeat": heartbeat, "setup_iscsi":setup_iscsi, "find_bond": find_bond, "gethostvmstats": gethostvmstats, "getvncport": getvncport, "getgateway": getgateway, "getnetwork": getnetwork, "preparemigration": preparemigration, "setIptables": setIptables, "patchdomr": patchdomr, "pingdomr": pingdomr, "pingxenserver": pingxenserver,  "ipassoc": ipassoc, "vm_data": vm_data, "savePassword": savePassword, "saveDhcpEntry": saveDhcpEntry, "setFirewallRule": setFirewallRule, "setLoadBalancerRule": setLoadBalancerRule, "createFile": createFile, "deleteFile": deleteFile, "checkMount": checkMount, "checkIscsi": checkIscsi, "networkUsage": networkUsage, "network_rules":network_rules, "can_bridge_firewall":can_bridge_firewall, "default_network_rules":default_network_rules, "destroy_network_rules_for_vm":destroy_network_rules_for_vm, "default_network_rules_systemvm":default_network_rules_systemvm, "get_rule_logs_for_vms":get_rule_logs_for_vms, "setLinkLocalIP":setLinkLocalIP})
+    XenAPIPlugin.dispatch({"pingtest": pingtest, "setup_iscsi":setup_iscsi, "gethostvmstats": gethostvmstats, "getvncport": getvncport, "getgateway": getgateway, "getnetwork": getnetwork, "preparemigration": preparemigration, "setIptables": setIptables, "patchdomr": patchdomr, "pingdomr": pingdomr, "pingxenserver": pingxenserver,  "ipassoc": ipassoc, "vm_data": vm_data, "savePassword": savePassword, "saveDhcpEntry": saveDhcpEntry, "setFirewallRule": setFirewallRule, "setLoadBalancerRule": setLoadBalancerRule, "createFile": createFile, "deleteFile": deleteFile, "checkMount": checkMount, "checkIscsi": checkIscsi, "networkUsage": networkUsage, "network_rules":network_rules, "can_bridge_firewall":can_bridge_firewall, "default_network_rules":default_network_rules, "destroy_network_rules_for_vm":destroy_network_rules_for_vm, "default_network_rules_systemvm":default_network_rules_systemvm, "get_rule_logs_for_vms":get_rule_logs_for_vms, "setLinkLocalIP":setLinkLocalIP})
 
diff --git a/scripts/vm/hypervisor/xenserver/vmopsSnapshot b/scripts/vm/hypervisor/xenserver/vmopsSnapshot
index 891257fb4a9..e4b62b3b160 100755
--- a/scripts/vm/hypervisor/xenserver/vmopsSnapshot
+++ b/scripts/vm/hypervisor/xenserver/vmopsSnapshot
@@ -458,12 +458,16 @@ def getIsTrueString(stringValue):
     return booleanValue 
 
 def makeUnavailable(uuid, primarySRPath, isISCSI):
+    if not isISCSI:
+        return
     VHD = getVHD(uuid, isISCSI)
     path = os.path.join(primarySRPath, VHD)
     manageAvailability(path, '-an')
     return
 
 def manageAvailability(path, value):
+    if path.__contains__("/var/run/sr-mount"):
+        return
     util.SMlog("Setting availability of " + path + " to " + value)
     try:
         cmd = ['/usr/sbin/lvchange', value, path]
diff --git a/scripts/vm/hypervisor/xenserver/xenheartbeat.sh b/scripts/vm/hypervisor/xenserver/xenheartbeat.sh
deleted file mode 100755
index 6c616f435ea..00000000000
--- a/scripts/vm/hypervisor/xenserver/xenheartbeat.sh
+++ /dev/null
@@ -1,62 +0,0 @@
-#!/bin/bash
-# Version @VERSION@
-
-#set -x
- 
-usage() {
-  printf "Usage: %s [uuid of this host] [interval in seconds]\n" $(basename $0) >&2
-
-}
-
-if [ -z $1 ]; then
-  usage
-  exit 2
-fi
-
-if [ -z $2 ]; then
-  usage
-  exit 3
-fi
-
-
-hbs=
-while true 
-do 
-  sleep $2
-
-  date=`date +%s`
-  lvscan
-  hbs=`ls -l /dev/VG*/hb-$1 | awk '{print $9}'`
-  for hb in $hbs
-  do
-    echo $date | dd of=$hb count=100 bs=1
-    if [ $? -ne 0 ]; then
-      reboot -f
-      echo "Problem with $hb";
-    fi
-  done
-
-
-  dirs=`ls /var/run/sr-mount`
-  if [ "$dirs" == "" ]; then
-    continue
-  fi
-
-  ls /var/run/sr-mount/* >/dev/null 2>&1 
-  if [ $? -ne 0 ]; then
-    reboot -f
-    echo "Problem with ls";
-  fi
-
-  hbs=`ls -l /var/run/sr-mount/*/hb-$1 | awk '{print $9}'`
-  for hb in $hbs
-  do
-    echo $date > $hb
-    if [ $? -ne 0 ]; then
-      reboot -f
-      echo "Problem with $hb";
-    fi
-  done
-
-done
-
diff --git a/scripts/vm/hypervisor/xenserver/xenserver56/NFSSR.py b/scripts/vm/hypervisor/xenserver/xenserver56/NFSSR.py
index 5fd672a67a6..1b600cfcf21 100755
--- a/scripts/vm/hypervisor/xenserver/xenserver56/NFSSR.py
+++ b/scripts/vm/hypervisor/xenserver/xenserver56/NFSSR.py
@@ -218,7 +218,6 @@ class NFSSR(FileSR.FileSR):
             if inst.code != errno.ENOENT:
                 raise xs_errors.XenError('NFSDelete')
 
-    @echo
     def vdi(self, uuid, loadLocked = False):
         if not loadLocked:
             return NFSFileVDINolock(self, uuid)
diff --git a/scripts/vm/hypervisor/xenserver/xenserver56/cleanup.py b/scripts/vm/hypervisor/xenserver/xenserver56/cleanup.py
index b17ec91823f..e80625c6c71 100755
--- a/scripts/vm/hypervisor/xenserver/xenserver56/cleanup.py
+++ b/scripts/vm/hypervisor/xenserver/xenserver56/cleanup.py
@@ -1576,7 +1576,7 @@ class FileSR(SR):
         for uuid, vhdInfo in vhds.iteritems():
             vdi = self.getVDI(uuid)
             if not vdi:
-                Util.log("Found new VDI when scanning: %s" % uuid)
+                #Util.log("Found new VDI when scanning: %s" % uuid)
                 vdi = FileVDI(self, uuid)
                 self.vdis[uuid] = vdi
             vdi.load(vhdInfo)
diff --git a/scripts/vm/hypervisor/xenserver/xenserver56/patch b/scripts/vm/hypervisor/xenserver/xenserver56/patch
index fe414c79b6f..29cbcd95d05 100644
--- a/scripts/vm/hypervisor/xenserver/xenserver56/patch
+++ b/scripts/vm/hypervisor/xenserver/xenserver56/patch
@@ -18,24 +18,21 @@ nfs.py=/opt/xensource/sm
 patch.tgz=..,0775,/opt/xensource/bin
 vmops=..,0755,/etc/xapi.d/plugins
 vmopsSnapshot=..,0755,/etc/xapi.d/plugins
-systemvm-premium.zip=../../../../../vms,0755,/opt/xensource/bin
+xs_cleanup.sh=..,0755,/opt/xensource/bin
+systemvm.iso=../../../../../vms,0644,/opt/xensource/packages/iso
 hostvmstats.py=..,0755,/opt/xensource/sm
+id_rsa.cloud=..,0600,/root/.ssh
 network_info.sh=..,0755,/opt/xensource/bin
 prepsystemvm.sh=..,0755,/opt/xensource/bin
 setupxenserver.sh=..,0755,/opt/xensource/bin
 make_migratable.sh=..,0755,/opt/xensource/bin
-networkUsage.sh=..,0755,/opt/xensource/bin
-find_bond.sh=..,0755,/opt/xensource/bin
 setup_iscsi.sh=..,0755,/opt/xensource/bin
 version=..,0755,/opt/xensource/bin
-setup_heartbeat_sr.sh=..,0755,/opt/xensource/bin
-check_heartbeat.sh=..,0755,/opt/xensource/bin
-xenheartbeat.sh=..,0755,/opt/xensource/bin
-launch_hb.sh=..,0755,/opt/xensource/bin
 pingtest.sh=../../..,0755,/opt/xensource/bin
 dhcp_entry.sh=../../../../network/domr/,0755,/opt/xensource/bin
 ipassoc.sh=../../../../network/domr/,0755,/opt/xensource/bin
 vm_data.sh=../../../../network/domr/,0755,/opt/xensource/bin
 save_password_to_domr.sh=../../../../network/domr/,0755,/opt/xensource/bin
+networkUsage.sh=../../../../network/domr/,0755,/opt/xensource/bin
 call_firewall.sh=../../../../network/domr/,0755,/opt/xensource/bin
 call_loadbalancer.sh=../../../../network/domr/,0755,/opt/xensource/bin
diff --git a/scripts/wscript_build b/scripts/wscript_build
new file mode 100644
index 00000000000..6730263e22f
--- /dev/null
+++ b/scripts/wscript_build
@@ -0,0 +1 @@
+bld.substitute('**',"${AGENTLIBDIR}/scripts",chmod=0755)
diff --git a/server/.classpath b/server/.classpath
index 52f2d3852ad..cefa0653a40 100644
--- a/server/.classpath
+++ b/server/.classpath
@@ -20,5 +20,6 @@
 	
 	
 	
+	
 	
 
diff --git a/server/conf/migration-components.xml b/server/conf/migration-components.xml
index 93e0d236b59..1a24e58eb0d 100644
--- a/server/conf/migration-components.xml
+++ b/server/conf/migration-components.xml
@@ -1,7 +1,7 @@
 
 
 
-    
+    
         
         
         
@@ -20,5 +20,9 @@
         
         
         
+        
+        
+        
+        
     
 
diff --git a/server/src/com/cloud/agent/manager/AgentManager.java b/server/src/com/cloud/agent/manager/AgentManager.java
index effccccd82f..d78e6f8eed0 100755
--- a/server/src/com/cloud/agent/manager/AgentManager.java
+++ b/server/src/com/cloud/agent/manager/AgentManager.java
@@ -47,11 +47,12 @@ import com.cloud.offering.ServiceOffering;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.VMTemplateVO;
-import com.cloud.storage.VirtualMachineTemplate;
+import com.cloud.template.VirtualMachineTemplate;
 import com.cloud.uservm.UserVm;
 import com.cloud.utils.Pair;
 import com.cloud.utils.component.Manager;
 import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachineProfile;
 
 /**
  * AgentManager manages hosts.  It directly coordinates between the
@@ -196,6 +197,8 @@ public interface AgentManager extends Manager {
 	 * @return
 	 */
     Pair findPod(VirtualMachineTemplate template, ServiceOfferingVO offering, DataCenterVO dc, long userId, Set avoids);
+    
+    Host findHost(VirtualMachineProfile vm, Set avoids);
 
     /**
      * Put the agent in maintenance mode.
@@ -233,4 +236,6 @@ public interface AgentManager extends Manager {
     public List discoverHosts(Long dcId, Long podId, Long clusterId, String clusterName, String url, String username, String password) throws DiscoveryException, InvalidParameterValueException;
     public List discoverHosts(AddHostCmd cmd) throws IllegalArgumentException, DiscoveryException, InvalidParameterValueException;
     public List discoverHosts(AddSecondaryStorageCmd cmd) throws IllegalArgumentException, DiscoveryException, InvalidParameterValueException;
+
+	Answer easySend(Long hostId, Command cmd, int timeout);
 }
diff --git a/server/src/com/cloud/agent/manager/AgentManagerImpl.java b/server/src/com/cloud/agent/manager/AgentManagerImpl.java
index dfd4e6f1bc0..ec49f02f299 100755
--- a/server/src/com/cloud/agent/manager/AgentManagerImpl.java
+++ b/server/src/com/cloud/agent/manager/AgentManagerImpl.java
@@ -63,12 +63,10 @@ import com.cloud.agent.api.StartupProxyCommand;
 import com.cloud.agent.api.StartupRoutingCommand;
 import com.cloud.agent.api.StartupStorageCommand;
 import com.cloud.agent.api.UnsupportedAnswer;
-import com.cloud.agent.api.UpgradeCommand;
 import com.cloud.agent.manager.allocator.HostAllocator;
 import com.cloud.agent.manager.allocator.PodAllocator;
 import com.cloud.agent.transport.Request;
 import com.cloud.agent.transport.Response;
-import com.cloud.agent.transport.UpgradeResponse;
 import com.cloud.alert.AlertManager;
 import com.cloud.api.BaseCmd;
 import com.cloud.api.ServerApiException;
@@ -83,9 +81,11 @@ import com.cloud.capacity.CapacityVO;
 import com.cloud.capacity.dao.CapacityDao;
 import com.cloud.configuration.dao.ConfigurationDao;
 import com.cloud.dc.ClusterVO;
+import com.cloud.dc.DataCenter;
 import com.cloud.dc.DataCenterIpAddressVO;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
+import com.cloud.dc.Pod;
 import com.cloud.dc.PodCluster;
 import com.cloud.dc.dao.ClusterDao;
 import com.cloud.dc.dao.DataCenterDao;
@@ -110,6 +110,7 @@ import com.cloud.host.Status.Event;
 import com.cloud.host.dao.DetailsDao;
 import com.cloud.host.dao.HostDao;
 import com.cloud.hypervisor.Hypervisor;
+import com.cloud.hypervisor.kvm.resource.KvmDummyResourceBase;
 import com.cloud.maid.StackMaid;
 import com.cloud.maint.UpgradeManager;
 import com.cloud.network.IPAddressVO;
@@ -120,12 +121,11 @@ import com.cloud.resource.Discoverer;
 import com.cloud.resource.ServerResource;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.storage.GuestOSCategoryVO;
+import com.cloud.storage.Storage;
 import com.cloud.storage.StorageManager;
 import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.VMTemplateHostVO;
 import com.cloud.storage.VMTemplateVO;
-import com.cloud.storage.VirtualMachineTemplate;
-import com.cloud.storage.Volume.StorageResourceType;
 import com.cloud.storage.dao.GuestOSCategoryDao;
 import com.cloud.storage.dao.StoragePoolDao;
 import com.cloud.storage.dao.StoragePoolHostDao;
@@ -133,6 +133,7 @@ import com.cloud.storage.dao.VMTemplateDao;
 import com.cloud.storage.dao.VMTemplateHostDao;
 import com.cloud.storage.dao.VolumeDao;
 import com.cloud.storage.resource.DummySecondaryStorageResource;
+import com.cloud.template.VirtualMachineTemplate;
 import com.cloud.user.dao.UserStatisticsDao;
 import com.cloud.uservm.UserVm;
 import com.cloud.utils.ActionDelegate;
@@ -154,7 +155,7 @@ import com.cloud.utils.nio.NioServer;
 import com.cloud.utils.nio.Task;
 import com.cloud.vm.State;
 import com.cloud.vm.VMInstanceVO;
-import com.cloud.vm.VmCharacteristics;
+import com.cloud.vm.VirtualMachineProfile;
 import com.cloud.vm.dao.VMInstanceDao;
 
 /**
@@ -413,11 +414,11 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
 	public Host findHost(final Host.Type type, final DataCenterVO dc, final HostPodVO pod, final StoragePoolVO sp,
     		final ServiceOffering offering, final VMTemplateVO template, VMInstanceVO vm,
     		Host currentHost, final Set avoid) {
-        VmCharacteristics vmc = new VmCharacteristics(vm.getType());
+        VirtualMachineProfile vmc = new VirtualMachineProfile(vm.getType());
         Enumeration en = _hostAllocators.enumeration();
         while (en.hasMoreElements()) {
             final HostAllocator allocator = en.nextElement();
-            final Host host = allocator.allocateTo(vmc, offering, type, dc, pod, sp, template, avoid);
+            final Host host = allocator.allocateTo(vmc, offering, type, dc, pod, sp.getClusterId(), template, avoid);
             if (host == null) {
                 continue;
             } else {
@@ -474,11 +475,15 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
         if (server == null) {
             return null;
         }
-
+        
         long id = server.getId();
 
         AgentAttache attache = createAttache(id, server, resource);
-        notifyMonitorsOfConnection(attache, startup);
+        if (!resource.IsRemoteAgent())
+        	notifyMonitorsOfConnection(attache, startup);
+        else {
+        	 _hostDao.updateStatus(server, Event.AgentConnected, _nodeId);
+        }
         return attache;
     }
 
@@ -573,7 +578,13 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
         Enumeration en = _discoverers.enumeration();
         while (en.hasMoreElements()) {
             Discoverer discoverer = en.nextElement();
-            Map> resources = discoverer.find(dcId, podId, clusterId, uri, username, password);
+            Map> resources = null;
+            
+            try {
+            	resources = discoverer.find(dcId, podId, clusterId, uri, username, password);
+            } catch(Exception e) {
+            	s_logger.info("Exception in host discovery process with discoverer: " + discoverer.getName() + ", skip to another discoverer if there is any");
+            }
             if (resources != null) {
                 for (Map.Entry> entry : resources.entrySet()) {
                     ServerResource resource = entry.getKey();
@@ -594,6 +605,7 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
         throw new DiscoveryException("Unable to add the host");
     }
     
+    @Override
     @DB
     public boolean deleteHost(long hostId) {
         Transaction txn = Transaction.currentTxn();
@@ -634,11 +646,10 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
             _dcDao.releasePrivateIpAddress(host.getPrivateIpAddress(), host.getDataCenterId(), null);
             AgentAttache attache = _agents.get(hostId);
             handleDisconnect(attache, Status.Event.Remove, false);
-            /*
             host.setGuid(null);
             host.setClusterId(null);
             _hostDao.update(host.getId(), host);
-            */
+            
             _hostDao.remove(hostId);
             
             //delete the associated primary storage from db
@@ -715,6 +726,8 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
             templateHostSC.addAnd("hostId", SearchCriteria.Op.EQ, secStorageHost.getId());
             _vmTemplateHostDao.remove(templateHostSC);
             
+            /*Disconnected agent needs special handling here*/
+    		secStorageHost.setGuid(null);
     		txn.commit();
     		return true;
     	}catch (Throwable t) {
@@ -788,11 +801,12 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
             long seq = _hostDao.getNextSequence(hostId);
             Request req = new Request(seq, hostId, _nodeId, new Command[] { new CheckHealthCommand() }, true, true);
             Answer[] answers = agent.send(req, 50 * 1000);
-            if (answers[0].getResult()) {
+            if (answers != null && answers[0] != null ) {
+                Status status = answers[0].getResult() ? Status.Up : Status.Down;
                 if (s_logger.isDebugEnabled()) {
-                    s_logger.debug("agent (" + hostId + ") responded to checkHeathCommand, reporting that agent is up");
+                    s_logger.debug("agent (" + hostId + ") responded to checkHeathCommand, reporting that agent is " + status);
                 }
-                return answers[0].getResult() ? Status.Up : Status.Down;
+                return status;
             }
         } catch (AgentUnavailableException e) {
             s_logger.debug("Agent is unavailable so we move on.");
@@ -1030,7 +1044,8 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
         	loadDirectlyConnectedHost(host, null);
         }
     }
-    
+
+    @SuppressWarnings("rawtypes")
     protected void loadDirectlyConnectedHost(HostVO host, ActionDelegate actionDelegate) {
         String resourceName = host.getResource();
         ServerResource resource = null;
@@ -1161,6 +1176,19 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
         }
         return null;
     }
+    
+    public Pod findPod(VirtualMachineProfile vm, DataCenter dc, Set avoids) {
+        for (PodAllocator allocator : _podAllocators) {
+            Pod pod = allocator.allocateTo(vm, dc, avoids);
+            if (pod != null) {
+                s_logger.debug("Pod " + pod.getId() + " is found by " + allocator.getName());
+                return pod;
+            }
+        }
+        
+        s_logger.debug("Unable to find any pod for " + vm);
+        return null;
+    }
 
     @Override
     public HostStats getHostStatistics(long hostId) throws InternalErrorException
@@ -1187,6 +1215,7 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
     	return null;
     }
     
+    @Override
     public Long getGuestOSCategoryId(long hostId) {
     	HostVO host = _hostDao.findById(hostId);
     	if (host == null) {
@@ -1218,6 +1247,7 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
             _investigate = investigate;
         }
 
+        @Override
         public void run() {
             try {
                 handleDisconnect(_attache, _event, _investigate);
@@ -1228,11 +1258,16 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
             }
         }
     }
+    
+    @Override
+    public Answer easySend(final Long hostId, final Command cmd) {   	
+    	return easySend(hostId, cmd, _wait);
+    }
 
     @Override
-    public Answer easySend(final Long hostId, final Command cmd) {
+    public Answer easySend(final Long hostId, final Command cmd, int timeout) { 
         try {
-            final Answer answer = send(hostId, cmd, _wait);
+            final Answer answer = send(hostId, cmd, timeout);
             if (answer == null) {
                 s_logger.warn("send returns null answer");
                 return null;
@@ -1572,7 +1607,7 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
         if (startup instanceof StartupStorageCommand) {
 
             StartupStorageCommand ssCmd = ((StartupStorageCommand) startup);
-            if (ssCmd.getResourceType() == StorageResourceType.SECONDARY_STORAGE) {
+            if (ssCmd.getResourceType() == Storage.StorageResourceType.SECONDARY_STORAGE) {
                 type = Host.Type.SecondaryStorage;
                 if (resource != null && resource instanceof DummySecondaryStorageResource){
                 	resource = null;
@@ -1719,7 +1754,7 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
 
     protected AgentAttache createAttache(long id, HostVO server, ServerResource resource) {
         s_logger.debug("Adding directly connect host for " + id);
-        if (resource instanceof DummySecondaryStorageResource) {
+        if (resource instanceof DummySecondaryStorageResource || resource instanceof KvmDummyResourceBase) {
         	return new DummyAttache(id, false);
         }
         final DirectAgentAttache attache = new DirectAgentAttache(id, resource, server.getStatus() == Status.Maintenance
@@ -1745,7 +1780,8 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
             return _hostDao.updateStatus(host, Event.UnableToMigrate, _nodeId);
         }
     }
-    
+
+    @Override
     public HostVO updateHost(UpdateHostCmd cmd) throws InvalidParameterValueException{
     	Long hostId = cmd.getId();
     	Long guestOSCategoryId = cmd.getOsCategoryId();
@@ -1790,7 +1826,7 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
         String cluster = startup.getCluster();
         
         if (pod != null && dataCenter != null && pod.equalsIgnoreCase("default") && dataCenter.equalsIgnoreCase("default")) {
-        	List pods = _podDao.listAll();
+        	List pods = _podDao.listAllIncludingRemoved();
         	for (HostPodVO hpv : pods) {
         		if (checkCIDR(type, hpv, startup.getPrivateIpAddress(), startup.getPrivateNetmask())) {
         			pod = hpv.getName();
@@ -1864,7 +1900,7 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
 
             // If this command is from a KVM agent, or from an agent that has a
             // null hypervisor type, don't do the CIDR check
-            if (hypervisorType == null || hypervisorType == Hypervisor.Type.KVM)
+            if (hypervisorType == null || hypervisorType == Hypervisor.Type.KVM || hypervisorType == Hypervisor.Type.VmWare)
                 doCidrCheck = false;
 
             if (doCidrCheck)
@@ -1927,6 +1963,11 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
         }
 
     }
+    
+    @Override
+    public Host findHost(VirtualMachineProfile vm, Set avoids) {
+        return null;
+    }
 
     // create capacity entries if none exist for this server
     private void createCapacityEntry(final StartupCommand startup, HostVO server) {
@@ -1945,7 +1986,7 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
 
         if (startup instanceof StartupStorageCommand) {
             StartupStorageCommand ssCmd = (StartupStorageCommand) startup;
-            if (ssCmd.getResourceType() == StorageResourceType.STORAGE_HOST) {
+            if (ssCmd.getResourceType() == Storage.StorageResourceType.STORAGE_HOST) {
                 CapacityVO capacity = new CapacityVO(server.getId(), server.getDataCenterId(), server.getPodId(), 0L, server.getTotalSize(),
                         CapacityVO.CAPACITY_TYPE_STORAGE);
                 _capacityDao.persist(capacity);
@@ -1966,28 +2007,28 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
         }
     }
 
-    protected void upgradeAgent(final Link link, final byte[] request, final String reason) {
-
-        if (reason == UnsupportedVersionException.IncompatibleVersion) {
-            final UpgradeResponse response = new UpgradeResponse(request, _upgradeMgr.getAgentUrl());
-            try {
-                s_logger.info("Asking for the agent to update due to incompatible version: " + response.toString());
-                link.send(response.toBytes());
-            } catch (final ClosedChannelException e) {
-                s_logger.warn("Unable to send response due to connection closed: " + response.toString());
-            }
-            return;
-        }
-
-        assert (reason == UnsupportedVersionException.UnknownVersion) : "Unknown reason: " + reason;
-        final UpgradeResponse response = new UpgradeResponse(request, _upgradeMgr.getAgentUrl());
-        try {
-            s_logger.info("Asking for the agent to update due to unknown version: " + response.toString());
-            link.send(response.toBytes());
-        } catch (final ClosedChannelException e) {
-            s_logger.warn("Unable to send response due to connection closed: " + response.toString());
-        }
-    }
+//    protected void upgradeAgent(final Link link, final byte[] request, final String reason) {
+//
+//        if (reason == UnsupportedVersionException.IncompatibleVersion) {
+//            final UpgradeResponse response = new UpgradeResponse(request, _upgradeMgr.getAgentUrl());
+//            try {
+//                s_logger.info("Asking for the agent to update due to incompatible version: " + response.toString());
+//                link.send(response.toBytes());
+//            } catch (final ClosedChannelException e) {
+//                s_logger.warn("Unable to send response due to connection closed: " + response.toString());
+//            }
+//            return;
+//        }
+//
+//        assert (reason == UnsupportedVersionException.UnknownVersion) : "Unknown reason: " + reason;
+//        final UpgradeResponse response = new UpgradeResponse(request, _upgradeMgr.getAgentUrl());
+//        try {
+//            s_logger.info("Asking for the agent to update due to unknown version: " + response.toString());
+//            link.send(response.toBytes());
+//        } catch (final ClosedChannelException e) {
+//            s_logger.warn("Unable to send response due to connection closed: " + response.toString());
+//        }
+//    }
 
     protected class SimulateStartTask implements Runnable {
         ServerResource resource;
@@ -2002,6 +2043,7 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
             this.actionDelegate = actionDelegate;
         }
 
+        @Override
         public void run() {
             try {
                 if (s_logger.isDebugEnabled()) {
@@ -2038,17 +2080,17 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
                     return;
                 }
                 StartupCommand startup = (StartupCommand) cmd;
-                if ((_upgradeMgr.registerForUpgrade(-1, startup.getVersion()) == UpgradeManager.State.RequiresUpdate) && (_upgradeMgr.getAgentUrl() != null)) {
-                    final UpgradeCommand upgrade = new UpgradeCommand(_upgradeMgr.getAgentUrl());
-                    final Request req = new Request(1, -1, -1, new Command[] { upgrade }, true, true);
-                    s_logger.info("Agent requires upgrade: " + req.toString());
-                    try {
-                        link.send(req.toBytes());
-                    } catch (ClosedChannelException e) {
-                        s_logger.warn("Unable to tell agent it should update.");
-                    }
-                    return;
-                }
+//                if ((_upgradeMgr.registerForUpgrade(-1, startup.getVersion()) == UpgradeManager.State.RequiresUpdate) && (_upgradeMgr.getAgentUrl() != null)) {
+//                    final UpgradeCommand upgrade = new UpgradeCommand(_upgradeMgr.getAgentUrl());
+//                    final Request req = new Request(1, -1, -1, new Command[] { upgrade }, true, true);
+//                    s_logger.info("Agent requires upgrade: " + req.toString());
+//                    try {
+//                        link.send(req.toBytes());
+//                    } catch (ClosedChannelException e) {
+//                        s_logger.warn("Unable to tell agent it should update.");
+//                    }
+//                    return;
+//                }
                 try {
                     StartupCommand[] startups = new StartupCommand[cmds.length];
                     for (int i = 0; i < cmds.length; i++)
@@ -2217,7 +2259,7 @@ public class AgentManagerImpl implements AgentManager, HandlerFactory {
 	                    }
 	                } catch (final UnsupportedVersionException e) {
 	                    s_logger.warn(e.getMessage());
-	                    upgradeAgent(task.getLink(), data, e.getReason());
+	                    //upgradeAgent(task.getLink(), data, e.getReason());
 	                }
 	            } else if (type == Task.Type.CONNECT) {
 	            } else if (type == Task.Type.DISCONNECT) {
diff --git a/server/src/com/cloud/agent/manager/DirectAgentAttache.java b/server/src/com/cloud/agent/manager/DirectAgentAttache.java
index e651c76813e..cafa42b9306 100644
--- a/server/src/com/cloud/agent/manager/DirectAgentAttache.java
+++ b/server/src/com/cloud/agent/manager/DirectAgentAttache.java
@@ -124,7 +124,11 @@ public class DirectAgentAttache extends AgentAttache {
 	    public synchronized void run() {
 	        try {
 	            ServerResource resource = _resource;
-
+	            
+	            if (resource.IsRemoteAgent()) {
+	            	return;
+	            }
+	            
 	            if (resource != null) {
         	        PingCommand cmd = resource.getCurrentStatus(_id);
         	        if (cmd == null) {
diff --git a/server/src/com/cloud/agent/manager/allocator/HostAllocator.java b/server/src/com/cloud/agent/manager/allocator/HostAllocator.java
index c4427b07be3..b4c0dd66b1e 100755
--- a/server/src/com/cloud/agent/manager/allocator/HostAllocator.java
+++ b/server/src/com/cloud/agent/manager/allocator/HostAllocator.java
@@ -22,14 +22,15 @@ import java.util.Set;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
 import com.cloud.host.Host;
+import com.cloud.host.Host.Type;
 import com.cloud.offering.ServiceOffering;
-import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.VMTemplateVO;
 import com.cloud.uservm.UserVm;
 import com.cloud.utils.component.Adapter;
-import com.cloud.vm.VmCharacteristics;
+import com.cloud.vm.VirtualMachineProfile;
 
 public interface HostAllocator extends Adapter {
-	Host allocateTo(VmCharacteristics vm, ServiceOffering offering, Host.Type type, DataCenterVO dc, HostPodVO pod, StoragePoolVO sp, VMTemplateVO template, Set avoid);
 	boolean isVirtualMachineUpgradable(final UserVm vm, final ServiceOffering offering);
+	Host allocateTo(VirtualMachineProfile vm, ServiceOffering offering, Type type, DataCenterVO dc, HostPodVO pod, Long clusterId, VMTemplateVO template, Set avoid);
+	
 }
diff --git a/server/src/com/cloud/agent/manager/allocator/PodAllocator.java b/server/src/com/cloud/agent/manager/allocator/PodAllocator.java
index ab567aab85f..f4775ac2c5d 100755
--- a/server/src/com/cloud/agent/manager/allocator/PodAllocator.java
+++ b/server/src/com/cloud/agent/manager/allocator/PodAllocator.java
@@ -19,13 +19,18 @@ package com.cloud.agent.manager.allocator;
 
 import java.util.Set;
 
+import com.cloud.dc.DataCenter;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
+import com.cloud.dc.Pod;
 import com.cloud.service.ServiceOfferingVO;
-import com.cloud.storage.VirtualMachineTemplate;
+import com.cloud.template.VirtualMachineTemplate;
 import com.cloud.utils.Pair;
 import com.cloud.utils.component.Adapter;
+import com.cloud.vm.VirtualMachineProfile;
 
 public interface PodAllocator extends Adapter {
     Pair allocateTo(VirtualMachineTemplate template, ServiceOfferingVO offering, DataCenterVO dc, long userId, Set avoids);
+    
+    Pod allocateTo(VirtualMachineProfile vm, DataCenter dc, Set avoids);
 }
diff --git a/server/src/com/cloud/agent/manager/allocator/impl/FirstFitAllocator.java b/server/src/com/cloud/agent/manager/allocator/impl/FirstFitAllocator.java
index 06569701554..3ea2fef067c 100755
--- a/server/src/com/cloud/agent/manager/allocator/impl/FirstFitAllocator.java
+++ b/server/src/com/cloud/agent/manager/allocator/impl/FirstFitAllocator.java
@@ -51,11 +51,12 @@ import com.cloud.storage.dao.StoragePoolHostDao;
 import com.cloud.uservm.UserVm;
 import com.cloud.utils.NumbersUtil;
 import com.cloud.utils.component.ComponentLocator;
+import com.cloud.utils.component.Inject;
 import com.cloud.vm.ConsoleProxyVO;
 import com.cloud.vm.DomainRouterVO;
 import com.cloud.vm.SecondaryStorageVmVO;
 import com.cloud.vm.UserVmVO;
-import com.cloud.vm.VmCharacteristics;
+import com.cloud.vm.VirtualMachineProfile;
 import com.cloud.vm.dao.ConsoleProxyDao;
 import com.cloud.vm.dao.DomainRouterDao;
 import com.cloud.vm.dao.SecondaryStorageVmDao;
@@ -68,23 +69,22 @@ import com.cloud.vm.dao.UserVmDao;
 public class FirstFitAllocator implements HostAllocator {
     private static final Logger s_logger = Logger.getLogger(FirstFitAllocator.class);
     private String _name;
-    protected HostDao _hostDao;
-    protected DetailsDao _hostDetailsDao;
-    protected UserVmDao _vmDao;
-    protected ServiceOfferingDao _offeringDao;
-    protected DomainRouterDao _routerDao;
-    protected ConsoleProxyDao _consoleProxyDao;
-    protected SecondaryStorageVmDao _secStorgaeVmDao;
-    protected StoragePoolHostDao _storagePoolHostDao;
-    protected ConfigurationDao _configDao;
-    protected GuestOSDao _guestOSDao;
-    protected GuestOSCategoryDao _guestOSCategoryDao;
+    @Inject HostDao _hostDao = null;
+    @Inject DetailsDao _hostDetailsDao = null;
+    @Inject UserVmDao _vmDao = null;
+    @Inject ServiceOfferingDao _offeringDao = null;
+    @Inject DomainRouterDao _routerDao = null;
+    @Inject ConsoleProxyDao _consoleProxyDao = null;
+    @Inject SecondaryStorageVmDao _secStorgaeVmDao = null;
+    @Inject ConfigurationDao _configDao = null;
+    @Inject GuestOSDao _guestOSDao = null; 
+    @Inject GuestOSCategoryDao _guestOSCategoryDao = null;
     float _factor = 1;
     protected String _allocationAlgorithm = "random";
     
 	@Override
-	public Host allocateTo(VmCharacteristics vm, ServiceOffering offering, Type type, DataCenterVO dc,
-			HostPodVO pod, StoragePoolVO sp, VMTemplateVO template,
+	public Host allocateTo(VirtualMachineProfile vm, ServiceOffering offering, Type type, DataCenterVO dc,
+			HostPodVO pod, Long clusterId, VMTemplateVO template,
 			Set avoid) {
 
         if (type == Host.Type.Storage) {
@@ -92,28 +92,21 @@ public class FirstFitAllocator implements HostAllocator {
             return null;
         }
         
-        s_logger.debug("Looking for hosts associated with storage pool " + sp.getId());
+        s_logger.debug("Looking for hosts in dc: " + dc.getId() + "  pod:" + pod.getId() + "  cluster:" + clusterId);
 
-        List poolhosts = _storagePoolHostDao.listByPoolId(sp.getId());
-        List hosts = new ArrayList();
-        for( StoragePoolHostVO poolhost : poolhosts ){
-            hosts.add(_hostDao.findById(poolhost.getHostId()));
-        }
-        
-        long podId = pod.getId();
-        List podHosts = new ArrayList(hosts.size());
-        Iterator it = hosts.iterator();
+        List clusterHosts = _hostDao.listBy(type, clusterId, pod.getId(), dc.getId());
+        Iterator it = clusterHosts.iterator();
         while (it.hasNext()) {
         	HostVO host = it.next();
-        	if (host.getPodId() == podId && !avoid.contains(host)) {
+        	if (avoid.contains(host)) {
+			it.remove();
+        	} else {
         	    if (s_logger.isDebugEnabled()) {
         	        s_logger.debug("Adding host " + host + " as possible pod host");
         	    }
-        		podHosts.add(host);
         	}
-        }
-        
-        return allocateTo(offering, template, avoid, podHosts);
+        }   
+        return allocateTo(offering, template, avoid, clusterHosts);
     }
 
     protected Host allocateTo(ServiceOffering offering, VMTemplateVO template, Set avoid, List hosts) {
@@ -335,17 +328,6 @@ public class FirstFitAllocator implements HostAllocator {
     public boolean configure(String name, Map params) {
         _name = name;
         ComponentLocator locator = ComponentLocator.getCurrentLocator();
-        _hostDao = locator.getDao(HostDao.class);
-        _hostDetailsDao = locator.getDao(DetailsDao.class);
-        _vmDao = locator.getDao(UserVmDao.class);
-        _offeringDao = locator.getDao(ServiceOfferingDao.class);
-        _routerDao = locator.getDao(DomainRouterDao.class);
-        _consoleProxyDao = locator.getDao(ConsoleProxyDao.class);
-        _secStorgaeVmDao = locator.getDao(SecondaryStorageVmDao.class);
-        _storagePoolHostDao  = locator.getDao(StoragePoolHostDao.class);
-        _configDao = locator.getDao(ConfigurationDao.class);
-        _guestOSDao = locator.getDao(GuestOSDao.class);
-        _guestOSCategoryDao = locator.getDao(GuestOSCategoryDao.class);
     	if (_configDao != null) {
     		Map configs = _configDao.getConfiguration(params);
             String opFactor = configs.get("cpu.overprovisioning.factor");
diff --git a/server/src/com/cloud/agent/manager/allocator/impl/FirstFitRoutingAllocator.java b/server/src/com/cloud/agent/manager/allocator/impl/FirstFitRoutingAllocator.java
index 90aa10e3c12..2c54bc46099 100644
--- a/server/src/com/cloud/agent/manager/allocator/impl/FirstFitRoutingAllocator.java
+++ b/server/src/com/cloud/agent/manager/allocator/impl/FirstFitRoutingAllocator.java
@@ -31,13 +31,13 @@ import com.cloud.host.Host;
 import com.cloud.offering.ServiceOffering;
 import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.VMTemplateVO;
-import com.cloud.vm.VmCharacteristics;
+import com.cloud.vm.VirtualMachineProfile;
 
 @Local(value={HostAllocator.class})
 public class FirstFitRoutingAllocator extends FirstFitAllocator {
     @Override
-    public Host allocateTo(VmCharacteristics vm, ServiceOffering offering, Host.Type type, DataCenterVO dc, HostPodVO pod,
-    		StoragePoolVO sp, VMTemplateVO template, Set avoid) {
+    public Host allocateTo(VirtualMachineProfile vm, ServiceOffering offering, Host.Type type, DataCenterVO dc, HostPodVO pod,
+    		Long clusterId, VMTemplateVO template, Set avoid) {
         try {
             NDC.push("FirstFitRoutingAllocator");
             if (type != Host.Type.Routing) {
@@ -45,7 +45,7 @@ public class FirstFitRoutingAllocator extends FirstFitAllocator {
                 return null;
             }
             //all hosts should be of type routing anyway.
-            return super.allocateTo(vm, offering, type, dc, pod, sp, template, avoid);
+            return super.allocateTo(vm, offering, type, dc, pod, clusterId, template, avoid);
         } finally {
             NDC.pop();
         }
diff --git a/server/src/com/cloud/agent/manager/allocator/impl/RandomAllocator.java b/server/src/com/cloud/agent/manager/allocator/impl/RandomAllocator.java
index 75c23fd225c..f4c4ac35e30 100755
--- a/server/src/com/cloud/agent/manager/allocator/impl/RandomAllocator.java
+++ b/server/src/com/cloud/agent/manager/allocator/impl/RandomAllocator.java
@@ -36,7 +36,7 @@ import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.VMTemplateVO;
 import com.cloud.uservm.UserVm;
 import com.cloud.utils.component.ComponentLocator;
-import com.cloud.vm.VmCharacteristics;
+import com.cloud.vm.VirtualMachineProfile;
 
 @Local(value=HostAllocator.class)
 public class RandomAllocator implements HostAllocator {
@@ -45,14 +45,14 @@ public class RandomAllocator implements HostAllocator {
     private HostDao _hostDao;
 
     @Override
-    public Host allocateTo(VmCharacteristics vm, ServiceOffering offering, Host.Type type, DataCenterVO dc, HostPodVO pod,
-    		StoragePoolVO sp, VMTemplateVO template, Set avoid) {
+    public Host allocateTo(VirtualMachineProfile vm, ServiceOffering offering, Host.Type type, DataCenterVO dc, HostPodVO pod,
+    		Long clusterId, VMTemplateVO template, Set avoid) {
         if (type == Host.Type.Storage) {
             return null;
         }
 
         // list all computing hosts, regardless of whether they support routing...it's random after all
-        List hosts = _hostDao.listBy(type, sp.getClusterId(), sp.getPodId(), sp.getDataCenterId());
+        List hosts = _hostDao.listBy(type, clusterId, pod.getId(), dc.getId());
         if (hosts.size() == 0) {
             return null;
         }
diff --git a/server/src/com/cloud/agent/manager/allocator/impl/RecreateHostAllocator.java b/server/src/com/cloud/agent/manager/allocator/impl/RecreateHostAllocator.java
index 3df93a18816..5372c9e0310 100644
--- a/server/src/com/cloud/agent/manager/allocator/impl/RecreateHostAllocator.java
+++ b/server/src/com/cloud/agent/manager/allocator/impl/RecreateHostAllocator.java
@@ -41,7 +41,7 @@ import com.cloud.storage.dao.StoragePoolDao;
 import com.cloud.utils.Pair;
 import com.cloud.utils.component.Inject;
 import com.cloud.vm.VirtualMachine;
-import com.cloud.vm.VmCharacteristics;
+import com.cloud.vm.VirtualMachineProfile;
 
 @Local(value=HostAllocator.class)
 public class RecreateHostAllocator extends FirstFitRoutingAllocator {
@@ -53,9 +53,9 @@ public class RecreateHostAllocator extends FirstFitRoutingAllocator {
     @Inject AgentManager _agentMgr;
     
     @Override
-    public Host allocateTo(VmCharacteristics vm, ServiceOffering offering, Host.Type type, DataCenterVO dc, HostPodVO pod,
-            StoragePoolVO sp, VMTemplateVO template, Set avoid) {
-        Host host = super.allocateTo(vm, offering, type, dc, pod, sp, template, avoid);
+    public Host allocateTo(VirtualMachineProfile vm, ServiceOffering offering, Host.Type type, DataCenterVO dc, HostPodVO pod,
+    		Long clusterId, VMTemplateVO template, Set avoid) {
+        Host host = super.allocateTo(vm, offering, type, dc, pod, clusterId, template, avoid);
         if (host != null) {
             return host;
         }
@@ -77,16 +77,15 @@ public class RecreateHostAllocator extends FirstFitRoutingAllocator {
             s_logger.debug("Removing " + pcId + " from the list of available pods");
             pcs.remove(new PodCluster(new HostPodVO(pcId.first()), pcId.second() != null ? new ClusterVO(pcId.second()) : null));
         }
-        
-        for (PodCluster p : pcs) {
-            List pools = _poolDao.listBy(dc.getId(), p.getPod().getId(), p.getCluster() == null ? null : p.getCluster().getId());
-            for (StoragePoolVO pool : pools) {
-                host = super.allocateTo(vm, offering, type, dc, p.getPod(), pool, template, avoid);
-                if (host != null) {
-                    return host;
-                }
-            }
-        }
+
+		for (PodCluster p : pcs) {
+			clusterId = p.getCluster() == null ? null : p.getCluster().getId();
+			host = super.allocateTo(vm, offering, type, dc, p.getPod(),
+					clusterId, template, avoid);
+			if (host != null) {
+				return host;
+			}
+		}
 
         s_logger.debug("Unable to find any available pods at all!");
         return null;
diff --git a/server/src/com/cloud/agent/manager/allocator/impl/TestingAllocator.java b/server/src/com/cloud/agent/manager/allocator/impl/TestingAllocator.java
index 2f89fb8922e..51d8d8afe43 100755
--- a/server/src/com/cloud/agent/manager/allocator/impl/TestingAllocator.java
+++ b/server/src/com/cloud/agent/manager/allocator/impl/TestingAllocator.java
@@ -32,7 +32,7 @@ import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.VMTemplateVO;
 import com.cloud.uservm.UserVm;
 import com.cloud.utils.component.ComponentLocator;
-import com.cloud.vm.VmCharacteristics;
+import com.cloud.vm.VirtualMachineProfile;
 
 /**
  * @author ahuang
@@ -47,8 +47,8 @@ public class TestingAllocator implements HostAllocator {
     String _name;
 
     @Override
-    public Host allocateTo(VmCharacteristics vm, ServiceOffering offering, Host.Type type, DataCenterVO dc, HostPodVO pod,
-    		StoragePoolVO sp, VMTemplateVO template, Set avoid) {
+    public Host allocateTo(VirtualMachineProfile vm, ServiceOffering offering, Host.Type type, DataCenterVO dc, HostPodVO pod,
+    		Long clusterId, VMTemplateVO template, Set avoid) {
         if (type == Host.Type.Routing && _routingHost != null) {
             return _hostDao.findById(_routingHost);
         } else if (type == Host.Type.Storage && _storageHost != null) {
diff --git a/server/src/com/cloud/agent/manager/allocator/impl/UserConcentratedAllocator.java b/server/src/com/cloud/agent/manager/allocator/impl/UserConcentratedAllocator.java
index 8c628dee712..1b7147211df 100755
--- a/server/src/com/cloud/agent/manager/allocator/impl/UserConcentratedAllocator.java
+++ b/server/src/com/cloud/agent/manager/allocator/impl/UserConcentratedAllocator.java
@@ -33,28 +33,30 @@ import com.cloud.agent.manager.allocator.PodAllocator;
 import com.cloud.capacity.CapacityVO;
 import com.cloud.capacity.dao.CapacityDao;
 import com.cloud.configuration.dao.ConfigurationDao;
+import com.cloud.dc.DataCenter;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
+import com.cloud.dc.Pod;
 import com.cloud.dc.dao.HostPodDao;
+import com.cloud.offering.NetworkOffering;
 import com.cloud.offering.ServiceOffering;
-import com.cloud.offering.ServiceOffering.GuestIpType;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.service.dao.ServiceOfferingDao;
-import com.cloud.storage.VirtualMachineTemplate;
 import com.cloud.storage.VolumeVO;
 import com.cloud.storage.dao.VMTemplateHostDao;
 import com.cloud.storage.dao.VMTemplatePoolDao;
 import com.cloud.storage.dao.VolumeDao;
+import com.cloud.template.VirtualMachineTemplate;
 import com.cloud.utils.DateUtil;
 import com.cloud.utils.NumbersUtil;
 import com.cloud.utils.Pair;
 import com.cloud.utils.component.Inject;
-import com.cloud.utils.db.GlobalLock;
 import com.cloud.utils.db.SearchCriteria;
 import com.cloud.vm.State;
 import com.cloud.vm.UserVmVO;
 import com.cloud.vm.VMInstanceVO;
 import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VirtualMachineProfile;
 import com.cloud.vm.dao.UserVmDao;
 import com.cloud.vm.dao.VMInstanceDao;
 
@@ -75,8 +77,8 @@ public class UserConcentratedAllocator implements PodAllocator {
     @Inject VMInstanceDao _vmInstanceDao;
     
     Random _rand = new Random(System.currentTimeMillis());
-    private final GlobalLock m_capacityCheckLock = GlobalLock.getInternLock("capacity.check");
     private int _hoursToSkipStoppedVMs = 24;
+    private int _hoursToSkipDestroyedVMs = 0;
     
     private int _secStorageVmRamSize = 1024;
     private int _proxyRamSize =  256;
@@ -142,7 +144,7 @@ public class UserConcentratedAllocator implements PodAllocator {
         }
         
         if (availablePods.size() == 0) {
-            s_logger.debug("There are no pods with enough memory/CPU capacity in zone" + zone.getName());
+            s_logger.debug("There are no pods with enough memory/CPU capacity in zone " + zone.getName());
             return null;
         } else {
         	// Return a random pod
@@ -154,54 +156,40 @@ public class UserConcentratedAllocator implements PodAllocator {
     }
 
     private boolean dataCenterAndPodHasEnoughCapacity(long dataCenterId, long podId, long capacityNeeded, short capacityType, long[] hostCandidate) {
-        if (m_capacityCheckLock.lock(120)) { // 2 minutes
-            try {
-                SearchCriteria sc = _capacityDao.createSearchCriteria();
-                sc.addAnd("capacityType", SearchCriteria.Op.EQ, capacityType);
-                sc.addAnd("dataCenterId", SearchCriteria.Op.EQ, dataCenterId);
-                sc.addAnd("podId", SearchCriteria.Op.EQ, podId);
-                List capacities = _capacityDao.search(sc, null);
-                boolean enoughCapacity = false;
-                if (capacities != null) {
-                    for (CapacityVO capacity : capacities) {
-                    	if(capacityType == CapacityVO.CAPACITY_TYPE_CPU || capacityType == CapacityVO.CAPACITY_TYPE_MEMORY) {
-                    		//
-                    		// for CPU/Memory, we now switch to static allocation
-                    		//
-	                        if ((capacity.getTotalCapacity() -
-	                        	calcHostAllocatedCpuMemoryCapacity(capacity.getHostOrPoolId(), capacityType)) >= capacityNeeded) {
+        List capacities = null;
+        
+        SearchCriteria sc = _capacityDao.createSearchCriteria();
+        sc.addAnd("capacityType", SearchCriteria.Op.EQ, capacityType);
+        sc.addAnd("dataCenterId", SearchCriteria.Op.EQ, dataCenterId);
+        sc.addAnd("podId", SearchCriteria.Op.EQ, podId);
+        s_logger.trace("Executing search");
+        capacities = _capacityDao.search(sc, null);
+        s_logger.trace("Done with a search");
 
-	                            hostCandidate[0] = capacity.getHostOrPoolId();
-	                            enoughCapacity = true;
-	                            break;
-	                        }
-                    	} else {
-	                        if ((capacity.getTotalCapacity() - capacity.getUsedCapacity()) >= capacityNeeded) {
-	                            hostCandidate[0] = capacity.getHostOrPoolId();
-	                            enoughCapacity = true;
-	                            break;
-	                        }
-                    	}
+        boolean enoughCapacity = false;
+        if (capacities != null) {
+            for (CapacityVO capacity : capacities) {
+                if(capacityType == CapacityVO.CAPACITY_TYPE_CPU || capacityType == CapacityVO.CAPACITY_TYPE_MEMORY) {
+                    //
+                    // for CPU/Memory, we now switch to static allocation
+                    //
+                    if ((capacity.getTotalCapacity() -
+                            calcHostAllocatedCpuMemoryCapacity(capacity.getHostOrPoolId(), capacityType)) >= capacityNeeded) {
+
+                        hostCandidate[0] = capacity.getHostOrPoolId();
+                        enoughCapacity = true;
+                        break;
+                    }
+                } else {
+                    if ((capacity.getTotalCapacity() - capacity.getUsedCapacity()) >= capacityNeeded) {
+                        hostCandidate[0] = capacity.getHostOrPoolId();
+                        enoughCapacity = true;
+                        break;
                     }
                 }
-                return enoughCapacity;
-            } finally {
-                m_capacityCheckLock.unlock();
             }
-        } else {
-        	s_logger.error("Unable to acquire synchronization lock for pod allocation");
-        	
-        	// we now try to enforce reservation-style allocation, waiting time has been adjusted
-        	// to 2 minutes
-        	return false;
-
-/*
-            // If we can't lock the table, just return that there is enough capacity and allow instance creation to fail on the agent
-            // if there is not enough capacity.  All that does is skip the optimization of checking for capacity before sending the
-            // command to the agent.
-            return true;
-*/
         }
+        return enoughCapacity;
     }
     
     private boolean skipCalculation(VMInstanceVO vm) {
@@ -215,15 +203,20 @@ public class UserConcentratedAllocator implements PodAllocator {
     		return true;
     	
     	if(vm.getState() == State.Stopped || vm.getState() == State.Destroyed) {
-    		// for stopped/Destroyed VMs, we will skip counting it if it hasn't been used for a while
+    		// for Stopped/Destroyed VMs, we will skip counting it if it hasn't been used for a while
+    		int _hoursToSkipVMs = _hoursToSkipStoppedVMs;
+    		
+    		if (vm.getState() == State.Destroyed)
+    			_hoursToSkipVMs = _hoursToSkipDestroyedVMs;
     		
     		long millisecondsSinceLastUpdate = DateUtil.currentGMTTime().getTime() - vm.getUpdateTime().getTime();
-    		if(millisecondsSinceLastUpdate > _hoursToSkipStoppedVMs*3600000L) {
+    		if(millisecondsSinceLastUpdate > _hoursToSkipVMs*3600000L) {
     			if(s_logger.isDebugEnabled())
-    				s_logger.debug("Skip counting vm " + vm.getInstanceName() + " in capacity allocation as it has been stopped for " + millisecondsSinceLastUpdate/60000 + " minutes");
+    				s_logger.debug("Skip counting " + vm.getState().toString() + " vm " + vm.getInstanceName() + " in capacity allocation as it has been " + vm.getState().toString().toLowerCase() + " for " + millisecondsSinceLastUpdate/60000 + " minutes");
     			return true;
     		}
     	}
+    	
     	return false;
     }
     
@@ -251,14 +244,14 @@ public class UserConcentratedAllocator implements PodAllocator {
         		so = _offeringDao.findById(userVm.getServiceOfferingId());
         	} else if(vm.getType() == VirtualMachine.Type.ConsoleProxy) {
         		so = new ServiceOfferingVO("Fake Offering For DomP", 1,
-    				_proxyRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized, false, true, null);
+    				_proxyRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, false, true, null);
         	} else if(vm.getType() == VirtualMachine.Type.SecondaryStorageVm) {
-        		so = new ServiceOfferingVO("Fake Offering For Secondary Storage VM", 1, _secStorageVmRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized, false, true, null);
+        		so = new ServiceOfferingVO("Fake Offering For Secondary Storage VM", 1, _secStorageVmRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, false, true, null);
         	} else if(vm.getType() == VirtualMachine.Type.DomainRouter) {
-                so = new ServiceOfferingVO("Fake Offering For DomR", 1, _routerRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized, false, true, null);
+                so = new ServiceOfferingVO("Fake Offering For DomR", 1, _routerRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, false, true, null);
         	} else {
         		assert(false) : "Unsupported system vm type";
-                so = new ServiceOfferingVO("Fake Offering For unknow system VM", 1, 128, 0, 0, 0, false, null, GuestIpType.Virtualized, false, true, null);
+                so = new ServiceOfferingVO("Fake Offering For unknow system VM", 1, 128, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, false, true, null);
         	}
             
             if(capacityType == CapacityVO.CAPACITY_TYPE_MEMORY) {
@@ -321,8 +314,10 @@ public class UserConcentratedAllocator implements PodAllocator {
         _name = name;
         
 		Map configs = _configDao.getConfiguration("management-server", params);
-		String value = configs.get("capacity.skipcounting.hours");
-		_hoursToSkipStoppedVMs = NumbersUtil.parseInt(value, 24);
+		String stoppedValue = configs.get("capacity.skipcounting.hours");
+		String destroyedValue = configs.get("capacity.skipcounting.destroyed.hours");
+		_hoursToSkipStoppedVMs = NumbersUtil.parseInt(stoppedValue, 24);
+		_hoursToSkipDestroyedVMs = NumbersUtil.parseInt(destroyedValue, 0);
 
 		// TODO this is not good, there should be one place to get these values
 		_secStorageVmRamSize = NumbersUtil.parseInt(configs.get("secstorage.vm.ram.size"), 256);
@@ -363,4 +358,9 @@ public class UserConcentratedAllocator implements PodAllocator {
 */
         return true;
     }
+
+    @Override
+    public Pod allocateTo(VirtualMachineProfile vm, DataCenter dc, Set avoids) {
+        return null;
+    }
 }
diff --git a/server/src/com/cloud/alert/AlertManagerImpl.java b/server/src/com/cloud/alert/AlertManagerImpl.java
index 1ad3fe7076b..bc8481c89e9 100644
--- a/server/src/com/cloud/alert/AlertManagerImpl.java
+++ b/server/src/com/cloud/alert/AlertManagerImpl.java
@@ -59,14 +59,17 @@ import com.cloud.offering.ServiceOffering;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.service.dao.ServiceOfferingDao;
 import com.cloud.storage.Storage.StoragePoolType;
+import com.cloud.storage.StorageManager;
 import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.dao.StoragePoolDao;
 import com.cloud.storage.dao.VolumeDao;
 import com.cloud.utils.NumbersUtil;
 import com.cloud.utils.Pair;
 import com.cloud.utils.component.ComponentLocator;
-import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.component.Inject;
+import com.cloud.utils.db.DB;
 import com.cloud.utils.db.SearchCriteria;
+import com.cloud.utils.db.Transaction;
 import com.cloud.vm.ConsoleProxyVO;
 import com.cloud.vm.DomainRouterVO;
 import com.cloud.vm.SecondaryStorageVmVO;
@@ -92,7 +95,8 @@ public class AlertManagerImpl implements AlertManager {
     private String _name = null;
     private EmailAlert _emailAlert;
     private AlertDao _alertDao;
-    private HostDao _hostDao;
+    private HostDao _hostDao;
+    @Inject protected StorageManager _storageMgr;
     private ServiceOfferingDao _offeringsDao;
     private CapacityDao _capacityDao;
     private VMInstanceDao _vmDao;
@@ -108,7 +112,6 @@ public class AlertManagerImpl implements AlertManager {
     private StoragePoolDao _storagePoolDao;
     
     private Timer _timer = null;
-    private int _overProvisioningFactor = 1;
     private float _cpuOverProvisioningFactor = 1;
     private long _capacityCheckPeriod = 60L * 60L * 1000L; // one hour by default
     private double _memoryCapacityThreshold = 0.75;
@@ -118,8 +121,6 @@ public class AlertManagerImpl implements AlertManager {
     private double _publicIPCapacityThreshold = 0.75;
     private double _privateIPCapacityThreshold = 0.75;
 
-    private final GlobalLock m_capacityCheckLock = GlobalLock.getInternLock("capacity.check");
-
     @Override
     public boolean configure(String name, Map params) throws ConfigurationException {
         _name = name;
@@ -153,7 +154,8 @@ public class AlertManagerImpl implements AlertManager {
             smtpDebug = Boolean.parseBoolean(smtpDebugStr);
         }
 
-        _emailAlert = new EmailAlert(emailAddresses, smtpHost, smtpPort, useAuth, smtpUsername, smtpPassword, emailSender, smtpDebug);
+        _emailAlert = new EmailAlert(emailAddresses, smtpHost, smtpPort, useAuth, smtpUsername, smtpPassword, emailSender, smtpDebug);
+        _emailAlert = null;
 
         String storageCapacityThreshold = configs.get("storage.capacity.threshold");
         String cpuCapacityThreshold = configs.get("cpu.capacity.threshold");
@@ -258,11 +260,6 @@ public class AlertManagerImpl implements AlertManager {
         if (capacityCheckPeriodStr != null) {
             _capacityCheckPeriod = Long.parseLong(capacityCheckPeriodStr);
         }
-
-        String overProvisioningFactorStr = configs.get("storage.overprovisioning.factor");
-        if (overProvisioningFactorStr != null) {
-            _overProvisioningFactor = Integer.parseInt(overProvisioningFactorStr);
-        }
         
         String cpuOverProvisioningFactorStr = configs.get("cpu.overprovisioning.factor");
         if (cpuOverProvisioningFactorStr != null) {
@@ -318,7 +315,7 @@ public class AlertManagerImpl implements AlertManager {
         }
     }
 
-    @Override
+    @Override @DB
     public void recalculateCapacity() {
         // FIXME: the right way to do this is to register a listener (see RouterStatsListener, VMSyncListener)
         //        for the vm sync state.  The listener model has connects/disconnects to keep things in sync much better
@@ -326,126 +323,124 @@ public class AlertManagerImpl implements AlertManager {
         //        is stopped we updated the amount allocated, and when VM sync reports a changed state, we update
         //        the amount allocated.  Hopefully it's limited to 3 entry points and will keep the amount allocated
         //        per host accurate.
-
-        if (m_capacityCheckLock.lock(5)) { // 5 second timeout
-            if (s_logger.isTraceEnabled()) {
-                s_logger.trace("recalculating system capacity");
-            }
-            try {
-                // delete the old records
-                _capacityDao.clearNonStorageCapacities();
-
-                // get all hosts..
-                SearchCriteria sc = _hostDao.createSearchCriteria();
-                sc.addAnd("status", SearchCriteria.Op.EQ, Status.Up.toString());
-                List hosts = _hostDao.search(sc, null);
-
-                // prep the service offerings
-                List offerings = _offeringsDao.listAll();
-                Map offeringsMap = new HashMap();
-                for (ServiceOfferingVO offering : offerings) {
-                    offeringsMap.put(offering.getId(), offering);
-                }
-                for (HostVO host : hosts) {
-                    if (host.getType() != Host.Type.Routing) {
-                        continue;
-                    }
-                    long cpu = 0;
-                    long usedMemory = 0;
-                    List domainRouters = _routerDao.listUpByHostId(host.getId());
-                    if (s_logger.isDebugEnabled()) {
-                        s_logger.debug("Found " + domainRouters.size() + " router domains on host " + host.getId());
-                    }
-                    for (DomainRouterVO router : domainRouters) {
-                        usedMemory += router.getRamSize() * 1024L * 1024L;
-                    }
 
-                    List proxys = _consoleProxyDao.listUpByHostId(host.getId());
-                    if (s_logger.isDebugEnabled()) {
-                        s_logger.debug("Found " + proxys.size() + " console proxy on host " + host.getId());
-                    }
-                    for(ConsoleProxyVO proxy : proxys) {
-                        usedMemory += proxy.getRamSize() * 1024L * 1024L;
-                    }
-                    
-                    List secStorageVms = _secStorgaeVmDao.listUpByHostId(host.getId());
-                    if (s_logger.isDebugEnabled()) {
-                        s_logger.debug("Found " + secStorageVms.size() + " secondary storage VM on host " + host.getId());
-                    }
-                    for(SecondaryStorageVmVO secStorageVm : secStorageVms) {
-                        usedMemory += secStorageVm.getRamSize() * 1024L * 1024L;
-                    }
-                            
-                    List vms = _userVmDao.listUpByHostId(host.getId());
-                    if (s_logger.isDebugEnabled()) {
-                        s_logger.debug("Found " + vms.size() + " user VM on host " + host.getId());
-                    }
-                    
-                    for (UserVmVO vm : vms) {
-                        ServiceOffering so = offeringsMap.get(vm.getServiceOfferingId());
-                        usedMemory += so.getRamSize() * 1024L * 1024L;
-                        cpu += so.getCpu() * (so.getSpeed() * 0.99);
-                    }
+        if (s_logger.isTraceEnabled()) {
+            s_logger.trace("recalculating system capacity");
+        }
+        List newCapacities = new ArrayList();
 
-                    long totalMemory = host.getTotalMemory();
+        // get all hosts..
+        SearchCriteria sc = _hostDao.createSearchCriteria();
+        sc.addAnd("status", SearchCriteria.Op.EQ, Status.Up.toString());
+        List hosts = _hostDao.search(sc, null);
 
-                    CapacityVO newMemoryCapacity = new CapacityVO(host.getId(), host.getDataCenterId(), host.getPodId(), usedMemory, totalMemory, CapacityVO.CAPACITY_TYPE_MEMORY);
-                    CapacityVO newCPUCapacity = new CapacityVO(host.getId(), host.getDataCenterId(), host.getPodId(), cpu, (long)(host.getCpus()*host.getSpeed()* _cpuOverProvisioningFactor), CapacityVO.CAPACITY_TYPE_CPU);
-                    _capacityDao.persist(newMemoryCapacity);
-                    _capacityDao.persist(newCPUCapacity);
-                }
-
-                // Calculate storage pool capacity
-                List storagePools = _storagePoolDao.listAllActive();
-                for (StoragePoolVO pool : storagePools) {
-                    long disk = 0l;
-                    Pair sizes = _volumeDao.getCountAndTotalByPool(pool.getId());
-                    disk = sizes.second();
-                    int provFactor = 1;
-                    if( pool.getPoolType() == StoragePoolType.NetworkFilesystem ) {
-                        provFactor = _overProvisioningFactor;
-                    }
-                    CapacityVO newStorageCapacity = new CapacityVO(pool.getId(), pool.getDataCenterId(), pool.getPodId(), disk, pool.getCapacityBytes() * provFactor, CapacityVO.CAPACITY_TYPE_STORAGE_ALLOCATED);
-                    _capacityDao.persist(newStorageCapacity);
-
-                    continue;
-                }
-
-                // Calculate new Public IP capacity
-                List datacenters = _dcDao.listAll();
-                for (DataCenterVO datacenter : datacenters) {
-                    long dcId = datacenter.getId();
-
-                    int totalPublicIPs = _publicIPAddressDao.countIPs(dcId, -1, false);
-                    int allocatedPublicIPs = _publicIPAddressDao.countIPs(dcId, -1, true);
-
-                    CapacityVO newPublicIPCapacity = new CapacityVO(null, dcId, null, allocatedPublicIPs, totalPublicIPs, CapacityVO.CAPACITY_TYPE_PUBLIC_IP);
-                    _capacityDao.persist(newPublicIPCapacity);
-                }
-
-                // Calculate new Private IP capacity
-                List pods = _podDao.listAll();
-                for (HostPodVO pod : pods) {
-                    long podId = pod.getId();
-                    long dcId = pod.getDataCenterId();
-                    
-                    int totalPrivateIPs = _privateIPAddressDao.countIPs(podId, dcId, false);
-                    int allocatedPrivateIPs = _privateIPAddressDao.countIPs(podId, dcId, true);
-
-                    CapacityVO newPrivateIPCapacity = new CapacityVO(null, dcId, podId, allocatedPrivateIPs, totalPrivateIPs, CapacityVO.CAPACITY_TYPE_PRIVATE_IP);
-                    _capacityDao.persist(newPrivateIPCapacity);
-                }
-            } finally {
-                m_capacityCheckLock.unlock();
+        // prep the service offerings
+        List offerings = _offeringsDao.listAllIncludingRemoved();
+        Map offeringsMap = new HashMap();
+        for (ServiceOfferingVO offering : offerings) {
+            offeringsMap.put(offering.getId(), offering);
+        }
+        for (HostVO host : hosts) {
+            if (host.getType() != Host.Type.Routing) {
+                continue;
             }
+
+            long cpu = 0;
+            long usedMemory = 0;
+            List domainRouters = _routerDao.listUpByHostId(host.getId());
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("Found " + domainRouters.size() + " router domains on host " + host.getId());
+            }
+            for (DomainRouterVO router : domainRouters) {
+                usedMemory += router.getRamSize() * 1024L * 1024L;
+            }
+
+            List proxys = _consoleProxyDao.listUpByHostId(host.getId());
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("Found " + proxys.size() + " console proxy on host " + host.getId());
+            }
+            for(ConsoleProxyVO proxy : proxys) {
+                usedMemory += proxy.getRamSize() * 1024L * 1024L;
+            }
+
+            List secStorageVms = _secStorgaeVmDao.listUpByHostId(host.getId());
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("Found " + secStorageVms.size() + " secondary storage VM on host " + host.getId());
+            }
+            for(SecondaryStorageVmVO secStorageVm : secStorageVms) {
+                usedMemory += secStorageVm.getRamSize() * 1024L * 1024L;
+            }
+
+            List vms = _userVmDao.listUpByHostId(host.getId());
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("Found " + vms.size() + " user VM on host " + host.getId());
+            }
+
+            for (UserVmVO vm : vms) {
+                ServiceOffering so = offeringsMap.get(vm.getServiceOfferingId());
+                usedMemory += so.getRamSize() * 1024L * 1024L;
+                cpu += so.getCpu() * (so.getSpeed() * 0.99);
+            }
+
+            long totalMemory = host.getTotalMemory();
 
-            if (s_logger.isTraceEnabled()) {
-                s_logger.trace("done recalculating system capacity");
-            }
-        } else {
-            if (s_logger.isTraceEnabled()) {
-                s_logger.trace("Skipping capacity check, unable to lock the capacity table for recalculation.");
-            }
+            CapacityVO newMemoryCapacity = new CapacityVO(host.getId(), host.getDataCenterId(), host.getPodId(), usedMemory, totalMemory, CapacityVO.CAPACITY_TYPE_MEMORY);
+            CapacityVO newCPUCapacity = new CapacityVO(host.getId(), host.getDataCenterId(), host.getPodId(), cpu, (long)(host.getCpus()*host.getSpeed()* _cpuOverProvisioningFactor), CapacityVO.CAPACITY_TYPE_CPU);
+            newCapacities.add(newMemoryCapacity);
+            newCapacities.add(newCPUCapacity);
+        }
+
+        // Calculate storage pool capacity
+        List storagePools = _storagePoolDao.listAll();
+        for (StoragePoolVO pool : storagePools) {
+            long disk = 0l;
+            Pair sizes = _volumeDao.getCountAndTotalByPool(pool.getId());
+            disk = sizes.second();
+            _storageMgr.createCapacityEntry(pool, disk);
+        }
+
+        // Calculate new Public IP capacity
+        List datacenters = _dcDao.listAllIncludingRemoved();
+        for (DataCenterVO datacenter : datacenters) {
+            long dcId = datacenter.getId();
+
+            int totalPublicIPs = _publicIPAddressDao.countIPs(dcId, -1, false);
+            int allocatedPublicIPs = _publicIPAddressDao.countIPs(dcId, -1, true);
+
+            CapacityVO newPublicIPCapacity = new CapacityVO(null, dcId, null, allocatedPublicIPs, totalPublicIPs, CapacityVO.CAPACITY_TYPE_PUBLIC_IP);
+            newCapacities.add(newPublicIPCapacity);
+        }
+
+        // Calculate new Private IP capacity
+        List pods = _podDao.listAllIncludingRemoved();
+        for (HostPodVO pod : pods) {
+            long podId = pod.getId();
+            long dcId = pod.getDataCenterId();
+
+            int totalPrivateIPs = _privateIPAddressDao.countIPs(podId, dcId, false);
+            int allocatedPrivateIPs = _privateIPAddressDao.countIPs(podId, dcId, true);
+
+            CapacityVO newPrivateIPCapacity = new CapacityVO(null, dcId, podId, allocatedPrivateIPs, totalPrivateIPs, CapacityVO.CAPACITY_TYPE_PRIVATE_IP);
+            newCapacities.add(newPrivateIPCapacity);
+        }
+
+        Transaction txn = Transaction.currentTxn();
+        try {
+        	txn.start();
+        	// delete the old records
+            _capacityDao.clearNonStorageCapacities();
+
+            for (CapacityVO newCapacity : newCapacities) {
+            	s_logger.trace("Executing capacity update");
+                _capacityDao.persist(newCapacity);
+                s_logger.trace("Done with capacity update");
+            }
+            txn.commit();
+        } catch (Exception ex) {
+        	txn.rollback();
+        	s_logger.error("Unable to start transaction for capacity update");
+        }finally {
+        	txn.close();
         }
     }
 
@@ -460,7 +455,7 @@ public class AlertManagerImpl implements AlertManager {
             }
 
             try {
-                List capacityList = _capacityDao.listAll();
+                List capacityList = _capacityDao.listAllIncludingRemoved();
                 Map> capacityDcTypeMap = new HashMap>();
 
                 for (CapacityVO capacity : capacityList) {
diff --git a/server/src/com/cloud/api/ApiDBUtils.java b/server/src/com/cloud/api/ApiDBUtils.java
index 08e58cd4a86..0ef97d93e80 100644
--- a/server/src/com/cloud/api/ApiDBUtils.java
+++ b/server/src/com/cloud/api/ApiDBUtils.java
@@ -63,6 +63,8 @@ import com.cloud.user.dao.UserStatisticsDao;
 import com.cloud.uservm.UserVm;
 import com.cloud.utils.DateUtil;
 import com.cloud.utils.component.ComponentLocator;
+import com.cloud.vm.InstanceGroupVO;
+import com.cloud.vm.UserVmManager;
 import com.cloud.vm.UserVmVO;
 import com.cloud.vm.VMInstanceVO;
 import com.cloud.vm.VmStats;
@@ -76,6 +78,7 @@ public class ApiDBUtils {
     private static NetworkGroupManager _networkGroupMgr;
     private static SnapshotManager _snapMgr;
     private static StorageManager _storageMgr;
+    private static UserVmManager _userVmMgr;
     private static StatsCollector _statsCollector;
 
     private static AccountDao _accountDao;
@@ -110,6 +113,7 @@ public class ApiDBUtils {
         _networkGroupMgr = locator.getManager(NetworkGroupManager.class);
         _snapMgr = locator.getManager(SnapshotManager.class);
         _storageMgr = locator.getManager(StorageManager.class);
+        _userVmMgr = locator.getManager(UserVmManager.class);
 
         _accountDao = locator.getDao(AccountDao.class);
         _accountVlanMapDao = locator.getDao(AccountVlanMapDao.class);
@@ -157,6 +161,10 @@ public class ApiDBUtils {
         return _ms.getPodIdForVlan(vlanDbId);
     }
 
+    public static String getVersion() {
+        return _ms.getVersion();
+    }
+
     public static List searchForUserVMs(Criteria c) {
         return _ms.searchForUserVMs(c);
     }
@@ -222,6 +230,10 @@ public class ApiDBUtils {
         return _storageMgr.isLocalStorageActiveOnHost(host);
     }
 
+    public static InstanceGroupVO findInstanceGroupForVM(long vmId) {
+        return _userVmMgr.getGroupForVm(vmId);
+    }
+
     /////////////////////////////////////////////////////////////
     //                    Misc methods                         //
     /////////////////////////////////////////////////////////////
@@ -379,7 +391,7 @@ public class ApiDBUtils {
     }
 
     public static List listZones() {
-        return _zoneDao.listAllActive();
+        return _zoneDao.listAll();
     }
 
     public static boolean volumeIsOnSharedStorage(long volumeId) throws InvalidParameterValueException {
diff --git a/server/src/com/cloud/api/ApiServer.java b/server/src/com/cloud/api/ApiServer.java
index 3a67119e642..f4bcb0fe6d2 100644
--- a/server/src/com/cloud/api/ApiServer.java
+++ b/server/src/com/cloud/api/ApiServer.java
@@ -82,6 +82,7 @@ import com.cloud.async.AsyncJobManager;
 import com.cloud.async.AsyncJobVO;
 import com.cloud.configuration.ConfigurationVO;
 import com.cloud.configuration.dao.ConfigurationDao;
+import com.cloud.domain.Domain;
 import com.cloud.domain.DomainVO;
 import com.cloud.exception.CloudAuthenticationException;
 import com.cloud.maid.StackMaid;
@@ -360,7 +361,7 @@ public class ApiServer implements HttpRequestHandler {
                 params.put("ctxUserId", userId.toString());
             }
             if (account != null) {
-                params.put("ctxAccountId", account.getId().toString());
+                params.put("ctxAccountId", String.valueOf(account.getId()));
             }
 
             AsyncJobVO job = new AsyncJobVO();
@@ -468,7 +469,7 @@ public class ApiServer implements HttpRequestHandler {
                 /*
     			requestParameters.put(BaseCmd.Properties.USER_ID.getName(), new String[] { user.getId().toString() });
                 requestParameters.put(BaseCmd.Properties.ACCOUNT.getName(), new String[] { account.getAccountName() });
-                requestParameters.put(BaseCmd.Properties.DOMAIN_ID.getName(), new String[] { account.getDomainId().toString() });
+                requestParameters.put(BaseCmd.Properties.DOMAIN_ID.getName(), new String[] { Long.toString(account.getDomainId()) });
         		requestParameters.put(BaseCmd.Properties.ACCOUNT_OBJ.getName(), new Object[] { account });
         		*/
     		} else {
@@ -516,7 +517,7 @@ public class ApiServer implements HttpRequestHandler {
         	if (domainPath == null || domainPath.trim().length() == 0) {
         		domainId = DomainVO.ROOT_DOMAIN;
         	} else {
-                DomainVO domainObj = _ms.findDomainByPath(domainPath);
+                Domain domainObj = _ms.findDomainByPath(domainPath);
         		if (domainObj != null) {
         			domainId = domainObj.getId();
         		} else { // if an unknown path is passed in, fail the login call
@@ -562,20 +563,21 @@ public class ApiServer implements HttpRequestHandler {
             if (systemVmUseLocalStorage == null) 
             	systemVmUseLocalStorage = "false";            
 
-            // set the userId and account object for everyone
+            // set the userId and account object for everyone
             session.setAttribute("userid", userAcct.getId().toString());
             session.setAttribute("username", userAcct.getUsername());
             session.setAttribute("firstname", userAcct.getFirstname());
             session.setAttribute("lastname", userAcct.getLastname());
             session.setAttribute("accountobj", account);
             session.setAttribute("account", account.getAccountName());
-            session.setAttribute("domainid", account.getDomainId().toString());
+            session.setAttribute("domainid", account.getDomainId());
             session.setAttribute("type", Short.valueOf(account.getType()).toString());
             session.setAttribute("networktype", networkType);
             session.setAttribute("hypervisortype", hypervisorType);
             session.setAttribute("directattachnetworkgroupsenabled", directAttachNetworkGroupsEnabled);
             session.setAttribute("directattacheduntaggedenabled", directAttachedUntaggedEnabled);
             session.setAttribute("systemvmuselocalstorage", systemVmUseLocalStorage);
+
             if (timezone != null) {
                 session.setAttribute("timezone", timezone);
                 session.setAttribute("timezoneoffset", Float.valueOf(offsetInHrs).toString());
diff --git a/server/src/com/cloud/api/BaseAsyncCmd.java b/server/src/com/cloud/api/BaseAsyncCmd.java
index 0fa8348d793..a216e0f6809 100644
--- a/server/src/com/cloud/api/BaseAsyncCmd.java
+++ b/server/src/com/cloud/api/BaseAsyncCmd.java
@@ -22,6 +22,10 @@ public abstract class BaseAsyncCmd extends BaseCmd {
         return response;
     }
 
+    public AsyncJobManager getAsyncJobManager() {
+        return _asyncJobMgr;
+    }
+
     public void setAsyncJobManager(AsyncJobManager mgr) {
         _asyncJobMgr = mgr;
     }
@@ -30,6 +34,10 @@ public abstract class BaseAsyncCmd extends BaseCmd {
         _asyncJobMgr.syncAsyncJobExecution(_job, syncObjType, syncObjId);
     }
 
+    public AsyncJobVO getJob() {
+        return _job;
+    }
+
     public void setJob(AsyncJobVO job) {
         _job = job;
     }
diff --git a/server/src/com/cloud/api/BaseCmd.java b/server/src/com/cloud/api/BaseCmd.java
old mode 100644
new mode 100755
index 3c4addaea7c..5b7891cf38b
--- a/server/src/com/cloud/api/BaseCmd.java
+++ b/server/src/com/cloud/api/BaseCmd.java
@@ -27,6 +27,7 @@ import java.util.Map;
 
 import org.apache.log4j.Logger;
 
+import com.cloud.server.ManagementServerImpl;
 import com.cloud.user.Account;
 import com.cloud.utils.Pair;
 
@@ -298,13 +299,14 @@ public abstract class BaseCmd {
         StringBuffer sb = new StringBuffer();
         if (RESPONSE_TYPE_JSON.equalsIgnoreCase(responseType)) {
             // JSON response
-            sb.append("{ \"" + getName() + "\" : { \"errorcode\" : \"" + apiException.getErrorCode() + "\", \"description\" : \"" + apiException.getDescription() + "\" } }");
+            sb.append("{ \"" + getName() + "\" : { " + "\"@attributes\":{\"cloud-stack-version\":\""+ApiDBUtils.getVersion()+"\"},");
+            sb.append("\"errorcode\" : \"" + apiException.getErrorCode() + "\", \"description\" : \"" + apiException.getDescription() + "\" } }");
         } else {
             sb.append("");
             sb.append("<" + getName() + ">");
             sb.append("" + apiException.getErrorCode() + "");
             sb.append("" + escapeXml(apiException.getDescription()) + "");
-            sb.append("");
+            sb.append("");
         }
         return sb.toString();
     }
@@ -314,10 +316,10 @@ public abstract class BaseCmd {
 
         // set up the return value with the name of the response
         if (RESPONSE_TYPE_JSON.equalsIgnoreCase(responseType)) {
-            sb.append("{ \"" + getName() + "\" : { ");
+            sb.append("{ \"" + getName() + "\" : { \"@attributes\":{\"cloud-stack-version\":\""+ApiDBUtils.getVersion()+"\"},");
         } else {
             sb.append("");
-            sb.append("<" + getName() + ">");
+            sb.append("<" + getName() + " cloud-stack-version=\""+ApiDBUtils.getVersion()+ "\">");
         }
 
         int i = 0;
diff --git a/server/src/com/cloud/api/commands/AddConfigCmd.java b/server/src/com/cloud/api/commands/AddConfigCmd.java
index 9b397252dc1..ac2cc21fa81 100644
--- a/server/src/com/cloud/api/commands/AddConfigCmd.java
+++ b/server/src/com/cloud/api/commands/AddConfigCmd.java
@@ -15,8 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -27,7 +25,6 @@ import com.cloud.api.Implementation;
 import com.cloud.api.Parameter;
 import com.cloud.api.ResponseObject;
 import com.cloud.api.ServerApiException;
-import com.cloud.api.response.ApiResponseSerializer;
 import com.cloud.api.response.ConfigurationResponse;
 import com.cloud.configuration.ConfigurationVO;
 
diff --git a/server/src/com/cloud/api/commands/AddHostCmd.java b/server/src/com/cloud/api/commands/AddHostCmd.java
index a9d4071bbba..258a2a1a261 100644
--- a/server/src/com/cloud/api/commands/AddHostCmd.java
+++ b/server/src/com/cloud/api/commands/AddHostCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.text.DecimalFormat;
@@ -48,6 +47,7 @@ import com.cloud.vm.UserVmVO;
 @Implementation(method="discoverHosts", manager=Manager.AgentManager)
 public class AddHostCmd extends BaseCmd {
     public static final Logger s_logger = Logger.getLogger(AddHostCmd.class.getName());
+
     private static final String s_name = "addhostresponse";
     
     /////////////////////////////////////////////////////
@@ -219,6 +219,5 @@ public class AddHostCmd extends BaseCmd {
         response.setResponses(hostResponses);
         response.setResponseName(getName());
         return response;
-        //return ApiResponseSerializer.toSerializedString(response);
     }
 }
diff --git a/server/src/com/cloud/api/commands/AssignPortForwardingServiceCmd.java b/server/src/com/cloud/api/commands/AssignPortForwardingServiceCmd.java
index df0603c2aef..9a9d856454a 100644
--- a/server/src/com/cloud/api/commands/AssignPortForwardingServiceCmd.java
+++ b/server/src/com/cloud/api/commands/AssignPortForwardingServiceCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.List;
@@ -24,10 +23,10 @@ import org.apache.log4j.Logger;
 
 import com.cloud.api.BaseAsyncCmd;
 import com.cloud.api.BaseCmd.Manager;
-import com.cloud.api.response.SuccessResponse;
 import com.cloud.api.Implementation;
 import com.cloud.api.Parameter;
 import com.cloud.api.ResponseObject;
+import com.cloud.api.response.SuccessResponse;
 
 @Implementation(method="assignSecurityGroup", manager=Manager.ManagementServer)
 public class AssignPortForwardingServiceCmd extends BaseAsyncCmd {
diff --git a/server/src/com/cloud/api/commands/AssignToLoadBalancerRuleCmd.java b/server/src/com/cloud/api/commands/AssignToLoadBalancerRuleCmd.java
index 6c7e012421c..33f01e221da 100644
--- a/server/src/com/cloud/api/commands/AssignToLoadBalancerRuleCmd.java
+++ b/server/src/com/cloud/api/commands/AssignToLoadBalancerRuleCmd.java
@@ -16,7 +16,7 @@
  * 
  */
 package com.cloud.api.commands;
-
+
 import java.util.List;
 
 import org.apache.log4j.Logger;
diff --git a/server/src/com/cloud/api/commands/AssociateIPAddrCmd.java b/server/src/com/cloud/api/commands/AssociateIPAddrCmd.java
index 22ecd1bd008..094a59cfe54 100644
--- a/server/src/com/cloud/api/commands/AssociateIPAddrCmd.java
+++ b/server/src/com/cloud/api/commands/AssociateIPAddrCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -26,7 +25,6 @@ import com.cloud.api.BaseCmd.Manager;
 import com.cloud.api.Implementation;
 import com.cloud.api.Parameter;
 import com.cloud.api.ResponseObject;
-import com.cloud.api.response.ApiResponseSerializer;
 import com.cloud.api.response.IPAddressResponse;
 import com.cloud.dc.Vlan.VlanType;
 import com.cloud.dc.VlanVO;
@@ -37,7 +35,7 @@ import com.cloud.user.UserContext;
 @Implementation(method="associateIP", manager=Manager.NetworkManager)
 public class AssociateIPAddrCmd extends BaseCmd {
     public static final Logger s_logger = Logger.getLogger(AssociateIPAddrCmd.class.getName());
-    private static final String s_name = "associateipaddressresponse";
+    private static final String s_name = "associateipaddressresponse";
 
     /////////////////////////////////////////////////////
     //////////////// API parameters /////////////////////
diff --git a/server/src/com/cloud/api/commands/AttachIsoCmd.java b/server/src/com/cloud/api/commands/AttachIsoCmd.java
index 3e7bb18f5d5..5d068364b15 100644
--- a/server/src/com/cloud/api/commands/AttachIsoCmd.java
+++ b/server/src/com/cloud/api/commands/AttachIsoCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
diff --git a/server/src/com/cloud/api/commands/CopyIsoCmd.java b/server/src/com/cloud/api/commands/CopyIsoCmd.java
index f1343618812..1d048b28d92 100644
--- a/server/src/com/cloud/api/commands/CopyIsoCmd.java
+++ b/server/src/com/cloud/api/commands/CopyIsoCmd.java
@@ -128,7 +128,7 @@ public class CopyIsoCmd extends BaseAsyncCmd {
     		//Return download status for admin users
             VMTemplateHostVO templateHostRef = ApiDBUtils.findTemplateHostRef(iso.getId(), destZoneId);
             
-    		if (isAdmin || iso.getAccountId() == account.getId().longValue()) {
+    		if (isAdmin || iso.getAccountId() == account.getId()) {
                 if (templateHostRef.getDownloadState()!=Status.DOWNLOADED) {
                     String templateStatus = "Processing";
                     if (templateHostRef.getDownloadState() == VMTemplateHostVO.Status.DOWNLOAD_IN_PROGRESS) {
diff --git a/server/src/com/cloud/api/commands/CopyTemplateCmd.java b/server/src/com/cloud/api/commands/CopyTemplateCmd.java
index 2d69fb1ef9d..7da51b33366 100644
--- a/server/src/com/cloud/api/commands/CopyTemplateCmd.java
+++ b/server/src/com/cloud/api/commands/CopyTemplateCmd.java
@@ -131,7 +131,7 @@ public class CopyTemplateCmd extends BaseAsyncCmd {
     		//Return download status for admin users
             VMTemplateHostVO templateHostRef = ApiDBUtils.findTemplateHostRef(template.getId(), destZoneId);
             
-    		if (isAdmin || template.getAccountId() == account.getId().longValue()) {
+    		if (isAdmin || template.getAccountId() == account.getId()) {
                 if (templateHostRef.getDownloadState()!=Status.DOWNLOADED) {
                     String templateStatus = "Processing";
                     if (templateHostRef.getDownloadState() == VMTemplateHostVO.Status.DOWNLOAD_IN_PROGRESS) {
diff --git a/server/src/com/cloud/api/commands/CreateDiskOfferingCmd.java b/server/src/com/cloud/api/commands/CreateDiskOfferingCmd.java
old mode 100644
new mode 100755
index 0986164ae53..b0c98db232a
--- a/server/src/com/cloud/api/commands/CreateDiskOfferingCmd.java
+++ b/server/src/com/cloud/api/commands/CreateDiskOfferingCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
diff --git a/server/src/com/cloud/api/commands/CreateDomainCmd.java b/server/src/com/cloud/api/commands/CreateDomainCmd.java
index 3ca92993c05..5fabea53661 100644
--- a/server/src/com/cloud/api/commands/CreateDomainCmd.java
+++ b/server/src/com/cloud/api/commands/CreateDomainCmd.java
@@ -15,9 +15,8 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
-
+
 import org.apache.log4j.Logger;
 
 import com.cloud.api.ApiDBUtils;
diff --git a/server/src/com/cloud/api/commands/CreateNetworkGroupCmd.java b/server/src/com/cloud/api/commands/CreateNetworkGroupCmd.java
index 0eeea4f2120..2f987515141 100644
--- a/server/src/com/cloud/api/commands/CreateNetworkGroupCmd.java
+++ b/server/src/com/cloud/api/commands/CreateNetworkGroupCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -81,7 +80,7 @@ public class CreateNetworkGroupCmd extends BaseCmd {
         return s_name;
     }
 
-    @Override
+    @Override
     public ResponseObject getResponse() {
         NetworkGroupVO group = (NetworkGroupVO)getResponseObject();
 
diff --git a/server/src/com/cloud/api/commands/CreatePortForwardingServiceCmd.java b/server/src/com/cloud/api/commands/CreatePortForwardingServiceCmd.java
index d960bfad770..eea1dc9c856 100644
--- a/server/src/com/cloud/api/commands/CreatePortForwardingServiceCmd.java
+++ b/server/src/com/cloud/api/commands/CreatePortForwardingServiceCmd.java
@@ -17,7 +17,7 @@
  */
 
 package com.cloud.api.commands;
-
+
 import org.apache.log4j.Logger;
 
 import com.cloud.api.ApiDBUtils;
@@ -92,7 +92,7 @@ public class CreatePortForwardingServiceCmd extends BaseCmd {
         response.setAccountName(group.getAccountName());
         response.setDomainId(group.getDomainId());
         response.setDomainName(ApiDBUtils.findDomainById(group.getDomainId()).getName());
-
+
         response.setResponseName(getName());
         return response;
     }
diff --git a/server/src/com/cloud/api/commands/CreatePortForwardingServiceRuleCmd.java b/server/src/com/cloud/api/commands/CreatePortForwardingServiceRuleCmd.java
index a2cbf1bedbb..5f08767945a 100644
--- a/server/src/com/cloud/api/commands/CreatePortForwardingServiceRuleCmd.java
+++ b/server/src/com/cloud/api/commands/CreatePortForwardingServiceRuleCmd.java
@@ -84,7 +84,7 @@ public class CreatePortForwardingServiceRuleCmd extends BaseAsyncCreateCmd {
     	return "portforwardingservicerule";
     }
 
-    @Override
+    @Override
     public ResponseObject getResponse() {
         NetworkRuleConfigVO netRule = (NetworkRuleConfigVO)getResponseObject();
 
diff --git a/server/src/com/cloud/api/commands/CreateTemplateCmd.java b/server/src/com/cloud/api/commands/CreateTemplateCmd.java
index 7f0cc3f4707..9107364f2f0 100644
--- a/server/src/com/cloud/api/commands/CreateTemplateCmd.java
+++ b/server/src/com/cloud/api/commands/CreateTemplateCmd.java
@@ -75,7 +75,6 @@ public class CreateTemplateCmd extends BaseAsyncCreateCmd {
     @Parameter(name="volumeid", type=CommandType.LONG)
     private Long volumeId;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
@@ -120,7 +119,6 @@ public class CreateTemplateCmd extends BaseAsyncCreateCmd {
         return volumeId;
     }
 
-
     /////////////////////////////////////////////////////
     /////////////// API Implementation///////////////////
     /////////////////////////////////////////////////////
diff --git a/server/src/com/cloud/api/commands/CreateVMGroupCmd.java b/server/src/com/cloud/api/commands/CreateVMGroupCmd.java
new file mode 100644
index 00000000000..d17b862eb91
--- /dev/null
+++ b/server/src/com/cloud/api/commands/CreateVMGroupCmd.java
@@ -0,0 +1,95 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General License for more details.
+ * 
+ * You should have received a copy of the GNU General License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.api.commands;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.api.ApiDBUtils;
+import com.cloud.api.BaseCmd;
+import com.cloud.api.BaseCmd.Manager;
+import com.cloud.api.Implementation;
+import com.cloud.api.Parameter;
+import com.cloud.api.ResponseObject;
+import com.cloud.api.response.InstanceGroupResponse;
+import com.cloud.user.Account;
+import com.cloud.vm.InstanceGroupVO;
+
+@Implementation(method="createVmGroup", manager=Manager.UserVmManager)
+public class CreateVMGroupCmd extends BaseCmd{
+    public static final Logger s_logger = Logger.getLogger(CreateVMGroupCmd.class.getName());
+
+    private static final String s_name = "createinstancegroupresponse";
+
+    /////////////////////////////////////////////////////
+    //////////////// API parameters /////////////////////
+    /////////////////////////////////////////////////////
+
+    @Parameter(name="name", type=CommandType.STRING, required=true)
+    private String groupName;
+
+    @Parameter(name="account", type=CommandType.STRING)
+    private String accountName;
+
+    @Parameter(name="domainid", type=CommandType.LONG)
+    private Long domainId;
+
+    /////////////////////////////////////////////////////
+    /////////////////// Accessors ///////////////////////
+    /////////////////////////////////////////////////////
+
+    public String getGroupName() {
+        return groupName;
+    }
+
+    public String getAccountName() {
+        return accountName;
+    }
+
+    public Long getDomainId() {
+        return domainId;
+    }
+
+    /////////////////////////////////////////////////////
+    /////////////// API Implementation///////////////////
+    /////////////////////////////////////////////////////
+
+    @Override
+    public String getName() {
+        return s_name;
+    }
+
+    @Override
+    public ResponseObject getResponse() {
+        InstanceGroupVO group = (InstanceGroupVO)getResponseObject();
+
+        InstanceGroupResponse response = new InstanceGroupResponse();
+        response.setId(group.getId());
+        response.setName(group.getName());
+        response.setCreated(group.getCreated());
+
+        Account accountTemp = ApiDBUtils.findAccountById(group.getAccountId());
+        if (accountTemp != null) {
+            response.setAccountName(accountTemp.getAccountName());
+            response.setDomainId(accountTemp.getDomainId());
+            response.setDomainName(ApiDBUtils.findDomainById(accountTemp.getDomainId()).getName());
+        }
+
+        response.setResponseName(getName());
+        return response;
+    }
+}
diff --git a/server/src/com/cloud/api/commands/CreateVolumeCmd.java b/server/src/com/cloud/api/commands/CreateVolumeCmd.java
index 79f65fef02c..a1e27f5ca5e 100644
--- a/server/src/com/cloud/api/commands/CreateVolumeCmd.java
+++ b/server/src/com/cloud/api/commands/CreateVolumeCmd.java
@@ -121,12 +121,11 @@ public class CreateVolumeCmd extends BaseAsyncCreateCmd {
         response.setAccountName(ApiDBUtils.findAccountById(volume.getAccountId()).getAccountName());
         response.setDomainId(volume.getDomainId());
         response.setDiskOfferingId(volume.getDiskOfferingId());
-        
-        if (volume.getDiskOfferingId() != null) {
-            DiskOfferingVO diskOffering = ApiDBUtils.findDiskOfferingById(volume.getDiskOfferingId());
-            response.setDiskOfferingName(diskOffering.getName());
-            response.setDiskOfferingDisplayText(diskOffering.getDisplayText());
-        }
+
+        DiskOfferingVO diskOffering = ApiDBUtils.findDiskOfferingById(volume.getDiskOfferingId());
+        response.setDiskOfferingName(diskOffering.getName());
+        response.setDiskOfferingDisplayText(diskOffering.getDisplayText());
+
         response.setDomainName(ApiDBUtils.findDomainById(volume.getDomainId()).getName());
         response.setStorageType("shared"); // NOTE: You can never create a local disk volume but if that changes, we need to change this
         if (volume.getPoolId() != null) {
diff --git a/server/src/com/cloud/api/commands/DeleteDiskOfferingCmd.java b/server/src/com/cloud/api/commands/DeleteDiskOfferingCmd.java
index 41602513067..de537e285f4 100644
--- a/server/src/com/cloud/api/commands/DeleteDiskOfferingCmd.java
+++ b/server/src/com/cloud/api/commands/DeleteDiskOfferingCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
diff --git a/server/src/com/cloud/api/commands/DeleteDomainCmd.java b/server/src/com/cloud/api/commands/DeleteDomainCmd.java
index b7365a2c3b5..c3620cde427 100644
--- a/server/src/com/cloud/api/commands/DeleteDomainCmd.java
+++ b/server/src/com/cloud/api/commands/DeleteDomainCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
diff --git a/server/src/com/cloud/api/commands/DeleteIPForwardingRuleCmd.java b/server/src/com/cloud/api/commands/DeleteIPForwardingRuleCmd.java
index 9810d75e6fe..830f7e5cea4 100644
--- a/server/src/com/cloud/api/commands/DeleteIPForwardingRuleCmd.java
+++ b/server/src/com/cloud/api/commands/DeleteIPForwardingRuleCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -55,7 +54,7 @@ public class DeleteIPForwardingRuleCmd extends BaseCmd {
     @Override
     public String getName() {
         return s_name;
-    }
+    }
 
 	@Override
 	public ResponseObject getResponse() {
diff --git a/server/src/com/cloud/api/commands/DeleteIsoCmd.java b/server/src/com/cloud/api/commands/DeleteIsoCmd.java
index ba2404326ce..1981287d251 100644
--- a/server/src/com/cloud/api/commands/DeleteIsoCmd.java
+++ b/server/src/com/cloud/api/commands/DeleteIsoCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
diff --git a/server/src/com/cloud/api/commands/DeleteLoadBalancerRuleCmd.java b/server/src/com/cloud/api/commands/DeleteLoadBalancerRuleCmd.java
index 722bda23aa5..e1660208eac 100644
--- a/server/src/com/cloud/api/commands/DeleteLoadBalancerRuleCmd.java
+++ b/server/src/com/cloud/api/commands/DeleteLoadBalancerRuleCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -54,7 +53,7 @@ public class DeleteLoadBalancerRuleCmd extends BaseAsyncCmd {
     @Override
     public String getName() {
         return s_name;
-    }
+    }
 
 	@Override
 	public ResponseObject getResponse() {
diff --git a/server/src/com/cloud/api/commands/DeleteVMGroupCmd.java b/server/src/com/cloud/api/commands/DeleteVMGroupCmd.java
new file mode 100644
index 00000000000..56e810fab11
--- /dev/null
+++ b/server/src/com/cloud/api/commands/DeleteVMGroupCmd.java
@@ -0,0 +1,68 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General License for more details.
+ * 
+ * You should have received a copy of the GNU General License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.api.commands;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.api.BaseCmd;
+import com.cloud.api.BaseCmd.Manager;
+import com.cloud.api.Implementation;
+import com.cloud.api.Parameter;
+import com.cloud.api.ResponseObject;
+import com.cloud.api.response.SuccessResponse;
+
+@Implementation(method="deleteVmGroup", manager=Manager.UserVmManager)
+public class DeleteVMGroupCmd extends BaseCmd{
+    public static final Logger s_logger = Logger.getLogger(DeleteVMGroupCmd.class.getName());
+    private static final String s_name = "deleteinstancegroupresponse";
+
+    /////////////////////////////////////////////////////
+    //////////////// API parameters /////////////////////
+    /////////////////////////////////////////////////////
+
+    @Parameter(name="id", type=CommandType.LONG, required=true)
+    private Long id;
+
+    /////////////////////////////////////////////////////
+    /////////////////// Accessors ///////////////////////
+    /////////////////////////////////////////////////////
+
+    public Long getId() {
+        return id;
+    }
+
+    /////////////////////////////////////////////////////
+    /////////////// API Implementation///////////////////
+    /////////////////////////////////////////////////////
+
+    @Override
+    public String getName() {
+        return s_name;
+    }
+
+    @Override
+    public ResponseObject getResponse() {
+        Boolean success = (Boolean)getResponseObject();
+
+        SuccessResponse response = new SuccessResponse();
+        response.setSuccess(success);
+        response.setResponseName(getName());
+
+        return response;
+    }
+}
diff --git a/server/src/com/cloud/api/commands/DeployVMCmd.java b/server/src/com/cloud/api/commands/DeployVMCmd.java
index 61b3c02ade0..de74b3f7050 100644
--- a/server/src/com/cloud/api/commands/DeployVMCmd.java
+++ b/server/src/com/cloud/api/commands/DeployVMCmd.java
@@ -35,6 +35,7 @@ import com.cloud.user.Account;
 import com.cloud.user.User;
 import com.cloud.user.UserContext;
 import com.cloud.uservm.UserVm;
+import com.cloud.vm.InstanceGroupVO;
 
 @Implementation(method="deployVirtualMachine")
 public class DeployVMCmd extends BaseAsyncCmd {
@@ -166,7 +167,13 @@ public class DeployVMCmd extends BaseAsyncCmd {
         response.setPrivateIp(userVm.getPrivateIpAddress());
         response.setServiceOfferingId(userVm.getServiceOfferingId());
         response.setHaEnable(userVm.isHaEnabled());
-        response.setGroup(userVm.getGroup());
+
+        InstanceGroupVO group = ApiDBUtils.findInstanceGroupForVM(userVm.getId());
+        if (group != null) {
+            response.setGroup(group.getName());
+            response.setGroupId(group.getId());
+        }
+
         if (userVm.getDisplayName() == null || userVm.getDisplayName().length() == 0) {
             response.setDisplayName(userVm.getName());
         } else {
diff --git a/server/src/com/cloud/api/commands/DestroyVMCmd.java b/server/src/com/cloud/api/commands/DestroyVMCmd.java
index 86df37c735a..71338725a87 100644
--- a/server/src/com/cloud/api/commands/DestroyVMCmd.java
+++ b/server/src/com/cloud/api/commands/DestroyVMCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
diff --git a/server/src/com/cloud/api/commands/DetachIsoCmd.java b/server/src/com/cloud/api/commands/DetachIsoCmd.java
index cc8fc3aa045..5f11a157403 100644
--- a/server/src/com/cloud/api/commands/DetachIsoCmd.java
+++ b/server/src/com/cloud/api/commands/DetachIsoCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -40,7 +39,6 @@ public class DetachIsoCmd extends BaseAsyncCmd {
     @Parameter(name="virtualmachineid", type=CommandType.LONG, required=true)
     private Long virtualMachineId;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
@@ -49,7 +47,6 @@ public class DetachIsoCmd extends BaseAsyncCmd {
         return virtualMachineId;
     }
 
-
     /////////////////////////////////////////////////////
     /////////////// API Implementation///////////////////
     /////////////////////////////////////////////////////
diff --git a/server/src/com/cloud/api/commands/DetachVolumeCmd.java b/server/src/com/cloud/api/commands/DetachVolumeCmd.java
index 3feb8a22914..67707940c45 100644
--- a/server/src/com/cloud/api/commands/DetachVolumeCmd.java
+++ b/server/src/com/cloud/api/commands/DetachVolumeCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -36,9 +35,14 @@ public class DetachVolumeCmd extends BaseAsyncCmd {
     //////////////// API parameters /////////////////////
     /////////////////////////////////////////////////////
 
-    @Parameter(name="id", type=CommandType.LONG, required=true)
+    @Parameter(name="id", type=CommandType.LONG)
     private Long id;
 
+    @Parameter(name="deviceid", type=CommandType.LONG)
+    private Long deviceId;
+
+    @Parameter(name="virtualmachineid", type=CommandType.LONG)
+    private Long virtualMachineId;
 
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
@@ -48,6 +52,13 @@ public class DetachVolumeCmd extends BaseAsyncCmd {
         return id;
     }
 
+    public Long getDeviceId() {
+        return deviceId;
+    }
+
+    public Long getVirtualMachineId() {
+        return virtualMachineId;
+    }
 
     /////////////////////////////////////////////////////
     /////////////// API Implementation///////////////////
diff --git a/server/src/com/cloud/api/commands/DisableUserCmd.java b/server/src/com/cloud/api/commands/DisableUserCmd.java
index aea93abebe9..25dfe8770cb 100644
--- a/server/src/com/cloud/api/commands/DisableUserCmd.java
+++ b/server/src/com/cloud/api/commands/DisableUserCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -36,7 +35,6 @@ public class DisableUserCmd extends BaseCmd {
     @Parameter(name="id", type=CommandType.LONG, required=true)
     private Long id;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
@@ -45,7 +43,6 @@ public class DisableUserCmd extends BaseCmd {
         return id;
     }
 
-
     /////////////////////////////////////////////////////
     /////////////// API Implementation///////////////////
     /////////////////////////////////////////////////////
diff --git a/server/src/com/cloud/api/commands/DisassociateIPAddrCmd.java b/server/src/com/cloud/api/commands/DisassociateIPAddrCmd.java
index 2f6f31f5ff8..2f2df83647b 100644
--- a/server/src/com/cloud/api/commands/DisassociateIPAddrCmd.java
+++ b/server/src/com/cloud/api/commands/DisassociateIPAddrCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -40,7 +39,6 @@ public class DisassociateIPAddrCmd extends BaseCmd {
     @Parameter(name="ipaddress", type=CommandType.STRING, required=true)
     private String ipAddress;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
@@ -49,7 +47,6 @@ public class DisassociateIPAddrCmd extends BaseCmd {
         return ipAddress;
     }
 
-
     /////////////////////////////////////////////////////
     /////////////// API Implementation///////////////////
     /////////////////////////////////////////////////////
diff --git a/server/src/com/cloud/api/commands/EnableAccountCmd.java b/server/src/com/cloud/api/commands/EnableAccountCmd.java
index 609d3e08bf4..bb697364195 100644
--- a/server/src/com/cloud/api/commands/EnableAccountCmd.java
+++ b/server/src/com/cloud/api/commands/EnableAccountCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -43,7 +42,6 @@ public class EnableAccountCmd extends BaseCmd {
     @Parameter(name="domainid", type=CommandType.LONG)
     private Long domainId;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
@@ -56,7 +54,6 @@ public class EnableAccountCmd extends BaseCmd {
         return domainId;
     }
 
-
     /////////////////////////////////////////////////////
     /////////////// API Implementation///////////////////
     /////////////////////////////////////////////////////
diff --git a/server/src/com/cloud/api/commands/ExtractIsoCmd.java b/server/src/com/cloud/api/commands/ExtractIsoCmd.java
new file mode 100755
index 00000000000..60bba246ddb
--- /dev/null
+++ b/server/src/com/cloud/api/commands/ExtractIsoCmd.java
@@ -0,0 +1,83 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General License for more details.
+ * 
+ * You should have received a copy of the GNU General License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.api.commands;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.api.BaseAsyncCmd;
+import com.cloud.api.BaseCmd.Manager;
+import com.cloud.api.Implementation;
+import com.cloud.api.Parameter;
+import com.cloud.api.ResponseObject;
+import com.cloud.api.response.ExtractResponse;
+
+@Implementation(method="extract", manager=Manager.TemplateManager)
+public class ExtractIsoCmd extends BaseAsyncCmd {
+	public static final Logger s_logger = Logger.getLogger(ExtractIsoCmd.class.getName());
+
+    private static final String s_name = "extractisoresponse";
+
+    /////////////////////////////////////////////////////
+    //////////////// API parameters /////////////////////
+    /////////////////////////////////////////////////////
+
+    @Parameter(name="id", type=CommandType.LONG, required=true)
+    private Long id;
+
+    @Parameter(name="url", type=CommandType.STRING, required=true)
+    private String url;
+
+    @Parameter(name="zoneid", type=CommandType.LONG, required=true)
+    private Long zoneId;
+
+    /////////////////////////////////////////////////////
+    /////////////////// Accessors ///////////////////////
+    /////////////////////////////////////////////////////
+
+    public Long getId() {
+        return id;
+    }
+
+    public String getUrl() {
+        return url;
+    }
+
+    public Long getZoneId() {
+        return zoneId;
+    }
+
+    /////////////////////////////////////////////////////
+    /////////////// API Implementation///////////////////
+    /////////////////////////////////////////////////////
+
+    @Override
+    public String getName() {
+        return s_name;
+    }
+
+    @Override
+    public ResponseObject getResponse() {
+        ExtractResponse response = (ExtractResponse)getResponseObject();
+        response.setResponseName(getName());
+        return response;
+    }
+
+    public static String getStaticName() {
+        return "ExtractIso";
+    }
+}
diff --git a/server/src/com/cloud/api/commands/ExtractTemplateCmd.java b/server/src/com/cloud/api/commands/ExtractTemplateCmd.java
new file mode 100755
index 00000000000..37af6f637b5
--- /dev/null
+++ b/server/src/com/cloud/api/commands/ExtractTemplateCmd.java
@@ -0,0 +1,83 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General License for more details.
+ * 
+ * You should have received a copy of the GNU General License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.api.commands;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.api.BaseAsyncCmd;
+import com.cloud.api.BaseCmd.Manager;
+import com.cloud.api.Implementation;
+import com.cloud.api.Parameter;
+import com.cloud.api.ResponseObject;
+import com.cloud.api.response.ExtractResponse;
+
+@Implementation(method="extract", manager=Manager.TemplateManager)
+public class ExtractTemplateCmd extends BaseAsyncCmd {
+	public static final Logger s_logger = Logger.getLogger(ExtractTemplateCmd.class.getName());
+
+    private static final String s_name = "extracttemplateresponse";
+
+    /////////////////////////////////////////////////////
+    //////////////// API parameters /////////////////////
+    /////////////////////////////////////////////////////
+
+    @Parameter(name="id", type=CommandType.LONG, required=true)
+    private Long id;
+
+    @Parameter(name="url", type=CommandType.STRING, required=true)
+    private String url;
+
+    @Parameter(name="zoneid", type=CommandType.LONG, required=true)
+    private Long zoneId;
+
+    /////////////////////////////////////////////////////
+    /////////////////// Accessors ///////////////////////
+    /////////////////////////////////////////////////////
+
+    public Long getId() {
+        return id;
+    }
+
+    public String getUrl() {
+        return url;
+    }
+
+    public Long getZoneId() {
+        return zoneId;
+    }
+
+    /////////////////////////////////////////////////////
+    /////////////// API Implementation///////////////////
+    /////////////////////////////////////////////////////
+
+	@Override
+	public String getName() {
+		return s_name;
+	}
+	
+    public static String getStaticName() {
+        return "ExtractTemplate";
+    }
+
+	@Override
+	public ResponseObject getResponse() {
+        ExtractResponse response = (ExtractResponse)getResponseObject();
+        response.setResponseName(getName());
+        return response;
+	}
+}
diff --git a/server/src/com/cloud/api/commands/ExtractVolumeCmd.java b/server/src/com/cloud/api/commands/ExtractVolumeCmd.java
new file mode 100755
index 00000000000..6e98f52bedb
--- /dev/null
+++ b/server/src/com/cloud/api/commands/ExtractVolumeCmd.java
@@ -0,0 +1,82 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General License for more details.
+ * 
+ * You should have received a copy of the GNU General License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.api.commands;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.api.BaseAsyncCmd;
+import com.cloud.api.Implementation;
+import com.cloud.api.Parameter;
+import com.cloud.api.ResponseObject;
+import com.cloud.api.response.ExtractResponse;
+
+@Implementation(method="extractVolume")
+public class ExtractVolumeCmd extends BaseAsyncCmd {
+	public static final Logger s_logger = Logger.getLogger(ExtractVolumeCmd.class.getName());
+
+    private static final String s_name = "extractvolumeresponse";
+
+    /////////////////////////////////////////////////////
+    //////////////// API parameters /////////////////////
+    /////////////////////////////////////////////////////
+
+    @Parameter(name="id", type=CommandType.LONG, required=true)
+    private Long id;
+
+    @Parameter(name="url", type=CommandType.STRING, required=true)
+    private String url;
+
+    @Parameter(name="zoneid", type=CommandType.LONG, required=true)
+    private Long zoneId;
+
+    /////////////////////////////////////////////////////
+    /////////////////// Accessors ///////////////////////
+    /////////////////////////////////////////////////////
+
+    public Long getId() {
+        return id;
+    }
+
+    public String getUrl() {
+        return url;
+    }
+
+    public Long getZoneId() {
+        return zoneId;
+    }
+
+    /////////////////////////////////////////////////////
+    /////////////// API Implementation///////////////////
+    /////////////////////////////////////////////////////
+
+	@Override
+	public String getName() {
+		return s_name;
+	}
+
+	@Override
+	public ResponseObject getResponse() {
+	    ExtractResponse response = (ExtractResponse)getResponseObject();
+	    response.setResponseName(getName());
+	    return response;
+	}
+
+	public static String getStaticName() {
+		return "ExtractVolume";
+	}
+}
diff --git a/server/src/com/cloud/api/commands/ListAccountsCmd.java b/server/src/com/cloud/api/commands/ListAccountsCmd.java
index 79b4ccf43dc..269f780826c 100644
--- a/server/src/com/cloud/api/commands/ListAccountsCmd.java
+++ b/server/src/com/cloud/api/commands/ListAccountsCmd.java
@@ -15,9 +15,8 @@
  * along with this program.  If not, see .
  * 
  */
+package com.cloud.api.commands;
 
-package com.cloud.api.commands;
-
 import java.util.ArrayList;
 import java.util.Iterator;
 import java.util.List;
diff --git a/server/src/com/cloud/api/commands/ListAlertsCmd.java b/server/src/com/cloud/api/commands/ListAlertsCmd.java
index fac4328787a..5e082fb797a 100644
--- a/server/src/com/cloud/api/commands/ListAlertsCmd.java
+++ b/server/src/com/cloud/api/commands/ListAlertsCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
@@ -45,7 +44,6 @@ public class ListAlertsCmd extends BaseListCmd {
     @Parameter(name="type", type=CommandType.STRING)
     private String type;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
@@ -54,7 +52,6 @@ public class ListAlertsCmd extends BaseListCmd {
         return type;
     }
 
-
     /////////////////////////////////////////////////////
     /////////////// API Implementation///////////////////
     /////////////////////////////////////////////////////
diff --git a/server/src/com/cloud/api/commands/ListCapabilitiesCmd.java b/server/src/com/cloud/api/commands/ListCapabilitiesCmd.java
new file mode 100644
index 00000000000..c6d51038822
--- /dev/null
+++ b/server/src/com/cloud/api/commands/ListCapabilitiesCmd.java
@@ -0,0 +1,51 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.api.commands;
+
+import java.util.Map;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.api.BaseCmd;
+import com.cloud.api.Implementation;
+import com.cloud.api.ResponseObject;
+import com.cloud.api.response.CapabilitiesResponse;
+
+@Implementation(method="listCapabilities")
+public class ListCapabilitiesCmd extends BaseCmd {
+	public static final Logger s_logger = Logger.getLogger(ListCapabilitiesCmd.class.getName());
+
+    private static final String s_name = "listcapabilitiesresponse";
+
+    @Override
+    public String getName() {
+        return s_name;
+    }
+
+    @Override @SuppressWarnings("unchecked")
+    public ResponseObject getResponse() {
+        Map capabilities = (Map)getResponseObject();
+
+        CapabilitiesResponse response = new CapabilitiesResponse();
+        response.setNetworkGroupsEnabled(capabilities.get("networkGroupsEnabled"));
+        response.setCloudStackVersion(capabilities.get("cloudStackVersion"));
+        response.setResponseName(getName());
+
+        return response;
+    }
+}
diff --git a/server/src/com/cloud/api/commands/ListDiskOfferingsCmd.java b/server/src/com/cloud/api/commands/ListDiskOfferingsCmd.java
index c6752067d2a..b5adec0cd20 100644
--- a/server/src/com/cloud/api/commands/ListDiskOfferingsCmd.java
+++ b/server/src/com/cloud/api/commands/ListDiskOfferingsCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
@@ -52,7 +51,6 @@ public class ListDiskOfferingsCmd extends BaseListCmd {
     @Parameter(name="name", type=CommandType.STRING)
     private String diskOfferingName;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
@@ -69,7 +67,6 @@ public class ListDiskOfferingsCmd extends BaseListCmd {
         return diskOfferingName;
     }
 
-
     /////////////////////////////////////////////////////
     /////////////// API Implementation///////////////////
     /////////////////////////////////////////////////////
@@ -78,7 +75,7 @@ public class ListDiskOfferingsCmd extends BaseListCmd {
     public String getName() {
         return s_name;
     }
-
+
     @Override @SuppressWarnings("unchecked")
     public ResponseObject getResponse() {
         List offerings = (List)getResponseObject();
diff --git a/server/src/com/cloud/api/commands/ListDomainChildrenCmd.java b/server/src/com/cloud/api/commands/ListDomainChildrenCmd.java
index 4fdb6d342b6..141709ea131 100644
--- a/server/src/com/cloud/api/commands/ListDomainChildrenCmd.java
+++ b/server/src/com/cloud/api/commands/ListDomainChildrenCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
@@ -51,7 +50,6 @@ public class ListDomainChildrenCmd extends BaseListCmd {
     @Parameter(name="name", type=CommandType.STRING)
     private String domainName;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
@@ -68,7 +66,6 @@ public class ListDomainChildrenCmd extends BaseListCmd {
         return domainName;
     }
 
-
     /////////////////////////////////////////////////////
     /////////////// API Implementation///////////////////
     /////////////////////////////////////////////////////
diff --git a/server/src/com/cloud/api/commands/ListDomainsCmd.java b/server/src/com/cloud/api/commands/ListDomainsCmd.java
index 1a7a3068db2..537c2ecda08 100644
--- a/server/src/com/cloud/api/commands/ListDomainsCmd.java
+++ b/server/src/com/cloud/api/commands/ListDomainsCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
@@ -51,7 +50,6 @@ public class ListDomainsCmd extends BaseListCmd {
     @Parameter(name="name", type=CommandType.STRING)
     private String domainName;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
@@ -68,7 +66,6 @@ public class ListDomainsCmd extends BaseListCmd {
         return domainName;
     }
 
-
     /////////////////////////////////////////////////////
     /////////////// API Implementation///////////////////
     /////////////////////////////////////////////////////
diff --git a/server/src/com/cloud/api/commands/ListEventsCmd.java b/server/src/com/cloud/api/commands/ListEventsCmd.java
index c423a703525..483fcaf04c1 100644
--- a/server/src/com/cloud/api/commands/ListEventsCmd.java
+++ b/server/src/com/cloud/api/commands/ListEventsCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
@@ -68,7 +67,6 @@ public class ListEventsCmd extends BaseListCmd {
     @Parameter(name="type", type=CommandType.STRING)
     private String type;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
diff --git a/server/src/com/cloud/api/commands/ListIsosCmd.java b/server/src/com/cloud/api/commands/ListIsosCmd.java
index ac4981c0cc9..ff7aa2b2a93 100644
--- a/server/src/com/cloud/api/commands/ListIsosCmd.java
+++ b/server/src/com/cloud/api/commands/ListIsosCmd.java
@@ -231,7 +231,7 @@ public class ListIsosCmd extends BaseListCmd {
                 isoResponse.setZoneName(datacenter.getName());
                             
                 // If the user is an admin, add the template download status
-                if (isAdmin || account.getId().longValue() == iso.getAccountId()) {
+                if (isAdmin || account.getId() == iso.getAccountId()) {
                     // add download status
                     if (isoHost.getDownloadState()!=Status.DOWNLOADED) {
                         String isoStatus = "Processing";
diff --git a/server/src/com/cloud/api/commands/ListLoadBalancerRuleInstancesCmd.java b/server/src/com/cloud/api/commands/ListLoadBalancerRuleInstancesCmd.java
index 316b8e2487b..1e1f9820f0d 100644
--- a/server/src/com/cloud/api/commands/ListLoadBalancerRuleInstancesCmd.java
+++ b/server/src/com/cloud/api/commands/ListLoadBalancerRuleInstancesCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
@@ -49,7 +48,6 @@ public class ListLoadBalancerRuleInstancesCmd extends BaseListCmd {
     @Parameter(name="id", type=CommandType.LONG, required=true)
     private Long id;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
@@ -62,7 +60,6 @@ public class ListLoadBalancerRuleInstancesCmd extends BaseListCmd {
         return id;
     }
 
-
     /////////////////////////////////////////////////////
     /////////////// API Implementation///////////////////
     /////////////////////////////////////////////////////
diff --git a/server/src/com/cloud/api/commands/ListNetworkGroupsCmd.java b/server/src/com/cloud/api/commands/ListNetworkGroupsCmd.java
index a7c3e490b5e..9a7141dfe4d 100644
--- a/server/src/com/cloud/api/commands/ListNetworkGroupsCmd.java
+++ b/server/src/com/cloud/api/commands/ListNetworkGroupsCmd.java
@@ -1,3 +1,20 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
@@ -40,7 +57,6 @@ public class ListNetworkGroupsCmd extends BaseListCmd {
     @Parameter(name="virtualmachineid", type=CommandType.LONG)
     private Long virtualMachineId;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
@@ -102,7 +118,7 @@ public class ListNetworkGroupsCmd extends BaseListCmd {
                         ingressData.setStartPort(ingressRule.getStartPort());
                         ingressData.setEndPort(ingressRule.getEndPort());
                     }
-
+
                     if (ingressRule.getAllowedNetworkGroup() != null) {
                         ingressData.setNetworkGroupName(ingressRule.getAllowedNetworkGroup());
                         ingressData.setAccountName(ingressRule.getAllowedNetGroupAcct());
diff --git a/server/src/com/cloud/api/commands/ListPortForwardingRulesCmd.java b/server/src/com/cloud/api/commands/ListPortForwardingRulesCmd.java
index b6fd8c41cd2..015bae33d65 100644
--- a/server/src/com/cloud/api/commands/ListPortForwardingRulesCmd.java
+++ b/server/src/com/cloud/api/commands/ListPortForwardingRulesCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
@@ -51,7 +50,6 @@ public class ListPortForwardingRulesCmd extends BaseListCmd {
     @Parameter(name="ipaddress", type=CommandType.STRING, required=true)
     private String ipAddress;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
@@ -66,7 +64,7 @@ public class ListPortForwardingRulesCmd extends BaseListCmd {
 
     @Override
     public String getName() {
-        return s_name;
+        return s_name;
     }
 
     @Override @SuppressWarnings("unchecked")
diff --git a/server/src/com/cloud/api/commands/ListPortForwardingServiceRulesCmd.java b/server/src/com/cloud/api/commands/ListPortForwardingServiceRulesCmd.java
index 68a53afe875..22eb7bdf6db 100644
--- a/server/src/com/cloud/api/commands/ListPortForwardingServiceRulesCmd.java
+++ b/server/src/com/cloud/api/commands/ListPortForwardingServiceRulesCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
@@ -55,7 +54,6 @@ public class ListPortForwardingServiceRulesCmd extends BaseListCmd {
     @Parameter(name="portforwardingserviceid", type=CommandType.LONG)
     private Long portForwardingServiceId;
 
-
     /////////////////////////////////////////////////////
     /////////////////// Accessors ///////////////////////
     /////////////////////////////////////////////////////
@@ -82,7 +80,7 @@ public class ListPortForwardingServiceRulesCmd extends BaseListCmd {
 
     @Override
     public String getName() {
-        return s_name;
+        return s_name;
     }
 
     @Override @SuppressWarnings("unchecked")
diff --git a/server/src/com/cloud/api/commands/ListRecurringSnapshotScheduleCmd.java b/server/src/com/cloud/api/commands/ListRecurringSnapshotScheduleCmd.java
index 7aa66665c9c..aec1f8bf9a7 100644
--- a/server/src/com/cloud/api/commands/ListRecurringSnapshotScheduleCmd.java
+++ b/server/src/com/cloud/api/commands/ListRecurringSnapshotScheduleCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
@@ -62,7 +61,7 @@ public class ListRecurringSnapshotScheduleCmd extends BaseListCmd {
 
     @Override
     public String getName() {
-        return s_name;
+        return s_name;
     }
 
     @Override @SuppressWarnings("unchecked")
diff --git a/server/src/com/cloud/api/commands/ListServiceOfferingsCmd.java b/server/src/com/cloud/api/commands/ListServiceOfferingsCmd.java
index 1bf062dd9fa..70814e9f2fc 100644
--- a/server/src/com/cloud/api/commands/ListServiceOfferingsCmd.java
+++ b/server/src/com/cloud/api/commands/ListServiceOfferingsCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
@@ -29,7 +28,7 @@ import com.cloud.api.Parameter;
 import com.cloud.api.ResponseObject;
 import com.cloud.api.response.ListResponse;
 import com.cloud.api.response.ServiceOfferingResponse;
-import com.cloud.offering.ServiceOffering.GuestIpType;
+import com.cloud.offering.NetworkOffering.GuestIpType;
 import com.cloud.service.ServiceOfferingVO;
 
 @Implementation(method="searchForServiceOfferings")
diff --git a/server/src/com/cloud/api/commands/ListSnapshotsCmd.java b/server/src/com/cloud/api/commands/ListSnapshotsCmd.java
index 6bfcab5c469..fc073b87397 100644
--- a/server/src/com/cloud/api/commands/ListSnapshotsCmd.java
+++ b/server/src/com/cloud/api/commands/ListSnapshotsCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
diff --git a/server/src/com/cloud/api/commands/ListTemplateOrIsoPermissionsCmd.java b/server/src/com/cloud/api/commands/ListTemplateOrIsoPermissionsCmd.java
index fec6227cb21..deb8dbda8e9 100644
--- a/server/src/com/cloud/api/commands/ListTemplateOrIsoPermissionsCmd.java
+++ b/server/src/com/cloud/api/commands/ListTemplateOrIsoPermissionsCmd.java
@@ -1,3 +1,20 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
 package com.cloud.api.commands;
 
 import java.util.List;
diff --git a/server/src/com/cloud/api/commands/ListTemplatesCmd.java b/server/src/com/cloud/api/commands/ListTemplatesCmd.java
index cc6018d7272..4d3c51e7adf 100644
--- a/server/src/com/cloud/api/commands/ListTemplatesCmd.java
+++ b/server/src/com/cloud/api/commands/ListTemplatesCmd.java
@@ -192,7 +192,7 @@ public class ListTemplatesCmd extends BaseListCmd {
                 templateResponse.setZoneName(datacenter.getName());
                 
                 // If the user is an admin, add the template download status
-                if (isAdmin || account.getId().longValue() == template.getAccountId()) {
+                if (isAdmin || account.getId() == template.getAccountId()) {
                     // add download status
                     if (templateHostRef.getDownloadState()!=Status.DOWNLOADED) {
                         String templateStatus = "Processing";
diff --git a/server/src/com/cloud/api/commands/ListVMGroupsCmd.java b/server/src/com/cloud/api/commands/ListVMGroupsCmd.java
new file mode 100644
index 00000000000..9803029c7df
--- /dev/null
+++ b/server/src/com/cloud/api/commands/ListVMGroupsCmd.java
@@ -0,0 +1,114 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General License for more details.
+ * 
+ * You should have received a copy of the GNU General License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.api.commands;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.api.ApiDBUtils;
+import com.cloud.api.BaseListCmd;
+import com.cloud.api.Implementation;
+import com.cloud.api.Parameter;
+import com.cloud.api.ResponseObject;
+import com.cloud.api.response.InstanceGroupResponse;
+import com.cloud.api.response.ListResponse;
+import com.cloud.user.Account;
+import com.cloud.vm.InstanceGroupVO;
+
+@Implementation(method="searchForVmGroups")
+public class ListVMGroupsCmd extends BaseListCmd {
+    public static final Logger s_logger = Logger.getLogger(ListVMGroupsCmd.class.getName());
+
+    private static final String s_name = "listinstancegroupsresponse";
+
+    /////////////////////////////////////////////////////
+    //////////////// API parameters /////////////////////
+    /////////////////////////////////////////////////////
+
+    @Parameter(name="id", type=CommandType.LONG)
+    private Long id;
+
+    @Parameter(name="name", type=CommandType.STRING)
+    private String groupName;
+
+    @Parameter(name="account", type=CommandType.STRING)
+    private String accountName;
+
+    @Parameter(name="domainid", type=CommandType.LONG)
+    private Long domainId;
+
+    /////////////////////////////////////////////////////
+    /////////////////// Accessors ///////////////////////
+    /////////////////////////////////////////////////////
+
+    public Long getId() {
+        return id;
+    }
+
+    public String getGroupName() {
+        return groupName;
+    }
+
+    public String getAccountName() {
+        return accountName;
+    }
+
+    public Long getDomainId() {
+        return domainId;
+    }
+
+    /////////////////////////////////////////////////////
+    /////////////// API Implementation///////////////////
+    /////////////////////////////////////////////////////
+
+    @Override
+	public String getName() {
+        return s_name;
+    }
+
+    @Override @SuppressWarnings("unchecked")
+	public ResponseObject getResponse() {
+        List groups = (List)getResponseObject();
+
+        ListResponse response = new ListResponse();
+        List responses = new ArrayList();
+        for (InstanceGroupVO group : groups) {
+            InstanceGroupResponse groupResponse = new InstanceGroupResponse();
+
+            groupResponse.setId(group.getId());
+            groupResponse.setName(group.getName());
+            groupResponse.setCreated(group.getCreated());
+
+            Account accountTemp = ApiDBUtils.findAccountById(group.getAccountId());
+            if (accountTemp != null) {
+                groupResponse.setAccountName(accountTemp.getAccountName());
+                groupResponse.setDomainId(accountTemp.getDomainId());
+                groupResponse.setDomainName(ApiDBUtils.findDomainById(accountTemp.getDomainId()).getName());
+            }
+
+            groupResponse.setResponseName("instancegroup");
+            responses.add(groupResponse);
+        }
+
+        response.setResponses(responses);
+        response.setResponseName(getName());
+        return response;
+    }
+}
diff --git a/server/src/com/cloud/api/commands/ListVMsCmd.java b/server/src/com/cloud/api/commands/ListVMsCmd.java
index 9c4fb2f5f45..4edd0eb3b8b 100644
--- a/server/src/com/cloud/api/commands/ListVMsCmd.java
+++ b/server/src/com/cloud/api/commands/ListVMsCmd.java
@@ -35,6 +35,7 @@ import com.cloud.offering.ServiceOffering;
 import com.cloud.storage.VMTemplateVO;
 import com.cloud.user.Account;
 import com.cloud.user.UserContext;
+import com.cloud.vm.InstanceGroupVO;
 import com.cloud.vm.UserVmVO;
 import com.cloud.vm.VmStats;
 
@@ -154,7 +155,11 @@ public class ListVMsCmd extends BaseListCmd {
                 userVmResponse.setDisplayName(userVm.getName());
             }
 
-            userVmResponse.setGroup(userVm.getGroup());
+            InstanceGroupVO group = ApiDBUtils.findInstanceGroupForVM(userVm.getId());
+            if (group != null) {
+                userVmResponse.setGroup(group.getName());
+                userVmResponse.setGroupId(group.getId());
+            }
 
             // Data Center Info
             userVmResponse.setZoneId(userVm.getDataCenterId());
diff --git a/server/src/com/cloud/api/commands/ListVolumesCmd.java b/server/src/com/cloud/api/commands/ListVolumesCmd.java
old mode 100644
new mode 100755
index 1d4c61d0cc7..dc7b2bea731
--- a/server/src/com/cloud/api/commands/ListVolumesCmd.java
+++ b/server/src/com/cloud/api/commands/ListVolumesCmd.java
@@ -123,7 +123,7 @@ public class ListVolumesCmd extends BaseListCmd {
     public String getName() {
         return s_name;
     }
-
+
     @Override @SuppressWarnings("unchecked")
     public ResponseObject getResponse() {
         List volumes = (List)getResponseObject();
@@ -189,11 +189,10 @@ public class ListVolumesCmd extends BaseListCmd {
             volResponse.setStorageType(storageType);
             
             volResponse.setDiskOfferingId(volume.getDiskOfferingId());
-            if (volume.getDiskOfferingId() != null) {
-                DiskOfferingVO diskOffering = ApiDBUtils.findDiskOfferingById(volume.getDiskOfferingId());
-                volResponse.setDiskOfferingName(diskOffering.getName());
-                volResponse.setDiskOfferingDisplayText(diskOffering.getDisplayText());
-            }
+
+            DiskOfferingVO diskOffering = ApiDBUtils.findDiskOfferingById(volume.getDiskOfferingId());
+            volResponse.setDiskOfferingName(diskOffering.getName());
+            volResponse.setDiskOfferingDisplayText(diskOffering.getDisplayText());
 
             Long poolId = volume.getPoolId();
             String poolName = (poolId == null) ? "none" : ApiDBUtils.findStoragePoolById(poolId).getName();
diff --git a/server/src/com/cloud/api/commands/ListZonesByCmd.java b/server/src/com/cloud/api/commands/ListZonesByCmd.java
index 798a5576bd0..6ef465247bd 100644
--- a/server/src/com/cloud/api/commands/ListZonesByCmd.java
+++ b/server/src/com/cloud/api/commands/ListZonesByCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
diff --git a/server/src/com/cloud/api/commands/LockAccountCmd.java b/server/src/com/cloud/api/commands/LockAccountCmd.java
index 27520155e9a..15578e66e8f 100644
--- a/server/src/com/cloud/api/commands/LockAccountCmd.java
+++ b/server/src/com/cloud/api/commands/LockAccountCmd.java
@@ -16,7 +16,7 @@
  * 
  */
 package com.cloud.api.commands;
-
+
 import org.apache.log4j.Logger;
 
 import com.cloud.api.BaseCmd;
@@ -60,7 +60,7 @@ public class LockAccountCmd extends BaseCmd {
 
     @Override
     public String getName() {
-        return s_name;
+        return s_name;
     }
 
 	@Override
diff --git a/server/src/com/cloud/api/commands/PrepareForMaintenanceCmd.java b/server/src/com/cloud/api/commands/PrepareForMaintenanceCmd.java
index 34fa2900e2e..836fdfbe9fc 100644
--- a/server/src/com/cloud/api/commands/PrepareForMaintenanceCmd.java
+++ b/server/src/com/cloud/api/commands/PrepareForMaintenanceCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -29,7 +28,8 @@ import com.cloud.api.response.SuccessResponse;
 
 @Implementation(method="maintain", manager=Manager.AgentManager)
 public class PrepareForMaintenanceCmd extends BaseAsyncCmd {
-	public static final Logger s_logger = Logger.getLogger(PrepareForMaintenanceCmd.class.getName());	
+	public static final Logger s_logger = Logger.getLogger(PrepareForMaintenanceCmd.class.getName());
+	
     private static final String s_name = "preparehostformaintenanceresponse";
 
     /////////////////////////////////////////////////////
diff --git a/server/src/com/cloud/api/commands/PreparePrimaryStorageForMaintenanceCmd.java b/server/src/com/cloud/api/commands/PreparePrimaryStorageForMaintenanceCmd.java
index 2abe76e6116..f20fd7b5f25 100644
--- a/server/src/com/cloud/api/commands/PreparePrimaryStorageForMaintenanceCmd.java
+++ b/server/src/com/cloud/api/commands/PreparePrimaryStorageForMaintenanceCmd.java
@@ -15,24 +15,22 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
 
 import com.cloud.api.BaseAsyncCmd;
 import com.cloud.api.BaseCmd.Manager;
-import com.cloud.api.ResponseObject;
-import com.cloud.api.response.SuccessResponse;
 import com.cloud.api.Implementation;
 import com.cloud.api.Parameter;
+import com.cloud.api.ResponseObject;
+import com.cloud.api.response.SuccessResponse;
 
 @Implementation(method="preparePrimaryStorageForMaintenance", manager=Manager.StorageManager)
 public class PreparePrimaryStorageForMaintenanceCmd extends BaseAsyncCmd {
 	public static final Logger s_logger = Logger.getLogger(PreparePrimaryStorageForMaintenanceCmd.class.getName());	
     private static final String s_name = "prepareprimarystorageformaintenanceresponse";
 
-
     /////////////////////////////////////////////////////
     //////////////// API parameters /////////////////////
     /////////////////////////////////////////////////////
diff --git a/server/src/com/cloud/api/commands/RebootVMCmd.java b/server/src/com/cloud/api/commands/RebootVMCmd.java
index 98f0baa11ac..5017c0701cc 100644
--- a/server/src/com/cloud/api/commands/RebootVMCmd.java
+++ b/server/src/com/cloud/api/commands/RebootVMCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
diff --git a/server/src/com/cloud/api/commands/RegisterIsoCmd.java b/server/src/com/cloud/api/commands/RegisterIsoCmd.java
index b6c51e32a5b..759d398e697 100644
--- a/server/src/com/cloud/api/commands/RegisterIsoCmd.java
+++ b/server/src/com/cloud/api/commands/RegisterIsoCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
@@ -23,14 +22,14 @@ import java.util.List;
 
 import org.apache.log4j.Logger;
 
+import com.cloud.api.ApiDBUtils;
 import com.cloud.api.BaseCmd;
 import com.cloud.api.BaseCmd.Manager;
-import com.cloud.api.response.ListResponse;
-import com.cloud.api.response.TemplateResponse;
-import com.cloud.api.ApiDBUtils;
 import com.cloud.api.Implementation;
 import com.cloud.api.Parameter;
 import com.cloud.api.ResponseObject;
+import com.cloud.api.response.ListResponse;
+import com.cloud.api.response.TemplateResponse;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.storage.VMTemplateHostVO;
 import com.cloud.storage.VMTemplateStorageResourceAssoc.Status;
diff --git a/server/src/com/cloud/api/commands/RegisterTemplateCmd.java b/server/src/com/cloud/api/commands/RegisterTemplateCmd.java
index f7693ec1102..0c5b1c5917f 100644
--- a/server/src/com/cloud/api/commands/RegisterTemplateCmd.java
+++ b/server/src/com/cloud/api/commands/RegisterTemplateCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.ArrayList;
@@ -23,19 +22,19 @@ import java.util.List;
 
 import org.apache.log4j.Logger;
 
+import com.cloud.api.ApiDBUtils;
 import com.cloud.api.BaseCmd;
 import com.cloud.api.BaseCmd.Manager;
-import com.cloud.api.response.ListResponse;
-import com.cloud.api.response.TemplateResponse;
-import com.cloud.api.ApiDBUtils;
 import com.cloud.api.Implementation;
 import com.cloud.api.Parameter;
 import com.cloud.api.ResponseObject;
+import com.cloud.api.response.ListResponse;
+import com.cloud.api.response.TemplateResponse;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.storage.GuestOS;
 import com.cloud.storage.VMTemplateHostVO;
-import com.cloud.storage.VMTemplateVO;
 import com.cloud.storage.VMTemplateStorageResourceAssoc.Status;
+import com.cloud.storage.VMTemplateVO;
 import com.cloud.user.Account;
 
 @Implementation(method="registerTemplate", manager=Manager.TemplateManager)
diff --git a/server/src/com/cloud/api/commands/RemoveFromLoadBalancerRuleCmd.java b/server/src/com/cloud/api/commands/RemoveFromLoadBalancerRuleCmd.java
index 7112124a773..dc8a15d93ed 100644
--- a/server/src/com/cloud/api/commands/RemoveFromLoadBalancerRuleCmd.java
+++ b/server/src/com/cloud/api/commands/RemoveFromLoadBalancerRuleCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import java.util.List;
@@ -24,10 +23,10 @@ import org.apache.log4j.Logger;
 
 import com.cloud.api.BaseAsyncCmd;
 import com.cloud.api.BaseCmd.Manager;
-import com.cloud.api.response.SuccessResponse;
 import com.cloud.api.Implementation;
 import com.cloud.api.Parameter;
 import com.cloud.api.ResponseObject;
+import com.cloud.api.response.SuccessResponse;
 
 @Implementation(method="removeFromLoadBalancer", manager=Manager.NetworkManager)
 public class RemoveFromLoadBalancerRuleCmd extends BaseAsyncCmd {
diff --git a/server/src/com/cloud/api/commands/RemovePortForwardingServiceCmd.java b/server/src/com/cloud/api/commands/RemovePortForwardingServiceCmd.java
index 354dbb03551..ef51005e6c5 100644
--- a/server/src/com/cloud/api/commands/RemovePortForwardingServiceCmd.java
+++ b/server/src/com/cloud/api/commands/RemovePortForwardingServiceCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
diff --git a/server/src/com/cloud/api/commands/ResetVMPasswordCmd.java b/server/src/com/cloud/api/commands/ResetVMPasswordCmd.java
index fbc248fac9b..37360ac0b5d 100644
--- a/server/src/com/cloud/api/commands/ResetVMPasswordCmd.java
+++ b/server/src/com/cloud/api/commands/ResetVMPasswordCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
diff --git a/server/src/com/cloud/api/commands/StartVMCmd.java b/server/src/com/cloud/api/commands/StartVMCmd.java
index df3119843bf..d698fa08764 100644
--- a/server/src/com/cloud/api/commands/StartVMCmd.java
+++ b/server/src/com/cloud/api/commands/StartVMCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -33,6 +32,7 @@ import com.cloud.offering.ServiceOffering;
 import com.cloud.storage.VMTemplateVO;
 import com.cloud.user.Account;
 import com.cloud.uservm.UserVm;
+import com.cloud.vm.InstanceGroupVO;
 
 @Implementation(method="startVirtualMachine", manager=Manager.UserVmManager)
 public class StartVMCmd extends BaseAsyncCmd {
@@ -87,8 +87,10 @@ public class StartVMCmd extends BaseAsyncCmd {
             response.setDisplayName(vm.getDisplayName());
         }
 
-        if (vm.getGroup() != null) {
-            response.setGroup(vm.getGroup());
+        InstanceGroupVO group = ApiDBUtils.findInstanceGroupForVM(vm.getId());
+        if (group != null) {
+            response.setGroup(group.getName());
+            response.setGroupId(group.getId());
         }
 
         if (vm.getState() != null) {
diff --git a/server/src/com/cloud/api/commands/StopVMCmd.java b/server/src/com/cloud/api/commands/StopVMCmd.java
index 63bf6806945..bebbc564798 100644
--- a/server/src/com/cloud/api/commands/StopVMCmd.java
+++ b/server/src/com/cloud/api/commands/StopVMCmd.java
@@ -15,23 +15,23 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
 
-import com.cloud.api.BaseAsyncCmd;
-import com.cloud.api.BaseCmd.Manager;
-import com.cloud.api.response.UserVmResponse;
 import com.cloud.api.ApiDBUtils;
+import com.cloud.api.BaseAsyncCmd;
 import com.cloud.api.BaseCmd;
+import com.cloud.api.BaseCmd.Manager;
 import com.cloud.api.Implementation;
 import com.cloud.api.Parameter;
 import com.cloud.api.ResponseObject;
+import com.cloud.api.response.UserVmResponse;
 import com.cloud.offering.ServiceOffering;
 import com.cloud.storage.VMTemplateVO;
 import com.cloud.user.Account;
 import com.cloud.uservm.UserVm;
+import com.cloud.vm.InstanceGroupVO;
 
 @Implementation(method="stopVirtualMachine", manager=Manager.UserVmManager)
 public class StopVMCmd extends BaseAsyncCmd {
@@ -61,7 +61,7 @@ public class StopVMCmd extends BaseAsyncCmd {
     @Override
     public String getName() {
         return s_name;
-    }
+    }
 
 	@Override
 	public ResponseObject getResponse() {
@@ -82,8 +82,10 @@ public class StopVMCmd extends BaseAsyncCmd {
             response.setDisplayName(vm.getDisplayName());
         }
 
-        if (vm.getGroup() != null) {
-            response.setGroup(vm.getGroup());
+        InstanceGroupVO group = ApiDBUtils.findInstanceGroupForVM(vm.getId());
+        if (group != null) {
+            response.setGroup(group.getName());
+            response.setGroupId(group.getId());
         }
 
         if (vm.getState() != null) {
diff --git a/server/src/com/cloud/api/commands/UpdateAccountCmd.java b/server/src/com/cloud/api/commands/UpdateAccountCmd.java
index 9b91e966282..98f51cdcdb2 100644
--- a/server/src/com/cloud/api/commands/UpdateAccountCmd.java
+++ b/server/src/com/cloud/api/commands/UpdateAccountCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -68,9 +67,9 @@ public class UpdateAccountCmd extends BaseCmd{
 
     @Override
     public String getName() {
-        return s_name;
+        return s_name;
     }
-    
+
     @Override
     public ResponseObject getResponse() {
         SuccessResponse response = new SuccessResponse();
diff --git a/server/src/com/cloud/api/commands/UpdateDomainCmd.java b/server/src/com/cloud/api/commands/UpdateDomainCmd.java
index 5218d77ee60..2ccb29343d2 100644
--- a/server/src/com/cloud/api/commands/UpdateDomainCmd.java
+++ b/server/src/com/cloud/api/commands/UpdateDomainCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -29,7 +28,7 @@ import com.cloud.api.ServerApiException;
 import com.cloud.api.response.SuccessResponse;
 
 @Implementation(method="updateDomain", manager=Manager.ManagementServer)
-public class UpdateDomainCmd extends BaseCmd{
+public class UpdateDomainCmd extends BaseCmd {
     public static final Logger s_logger = Logger.getLogger(UpdateDomainCmd.class.getName());
     private static final String s_name = "updatedomainresponse";
 
@@ -61,7 +60,7 @@ public class UpdateDomainCmd extends BaseCmd{
 
     @Override
     public String getName() {
-        return s_name;
+        return s_name;
     }
     
     @Override
diff --git a/server/src/com/cloud/api/commands/UpdateIPForwardingRuleCmd.java b/server/src/com/cloud/api/commands/UpdateIPForwardingRuleCmd.java
index 89821aaf9a4..a0cdef8110e 100644
--- a/server/src/com/cloud/api/commands/UpdateIPForwardingRuleCmd.java
+++ b/server/src/com/cloud/api/commands/UpdateIPForwardingRuleCmd.java
@@ -71,7 +71,7 @@ public class UpdateIPForwardingRuleCmd extends BaseAsyncCmd {
     /////////////// API Implementation///////////////////
     /////////////////////////////////////////////////////
 
-    @Override
+    @Override
     public String getName() {
         return s_name;
     }
diff --git a/server/src/com/cloud/api/commands/UpdateIsoCmd.java b/server/src/com/cloud/api/commands/UpdateIsoCmd.java
old mode 100644
new mode 100755
index 060fe2a13c2..b6bc0420478
--- a/server/src/com/cloud/api/commands/UpdateIsoCmd.java
+++ b/server/src/com/cloud/api/commands/UpdateIsoCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
diff --git a/server/src/com/cloud/api/commands/UpdateLoadBalancerRuleCmd.java b/server/src/com/cloud/api/commands/UpdateLoadBalancerRuleCmd.java
index 8b60e7e84ce..8a57de021d7 100644
--- a/server/src/com/cloud/api/commands/UpdateLoadBalancerRuleCmd.java
+++ b/server/src/com/cloud/api/commands/UpdateLoadBalancerRuleCmd.java
@@ -1,14 +1,31 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
 
+import com.cloud.api.ApiDBUtils;
 import com.cloud.api.BaseAsyncCmd;
 import com.cloud.api.BaseCmd.Manager;
-import com.cloud.api.response.LoadBalancerResponse;
-import com.cloud.api.ApiDBUtils;
 import com.cloud.api.Implementation;
 import com.cloud.api.Parameter;
 import com.cloud.api.ResponseObject;
+import com.cloud.api.response.LoadBalancerResponse;
 import com.cloud.network.LoadBalancerVO;
 
 @Implementation(method="updateLoadBalancerRule", manager=Manager.NetworkManager)
@@ -63,7 +80,7 @@ public class UpdateLoadBalancerRuleCmd extends BaseAsyncCmd {
     /////////////// API Implementation///////////////////
     /////////////////////////////////////////////////////
 
-    @Override
+    @Override
     public String getName() {
         return s_name;
     }
diff --git a/server/src/com/cloud/api/commands/UpdateServiceOfferingCmd.java b/server/src/com/cloud/api/commands/UpdateServiceOfferingCmd.java
index 157df9e7729..d4e68d63d8f 100644
--- a/server/src/com/cloud/api/commands/UpdateServiceOfferingCmd.java
+++ b/server/src/com/cloud/api/commands/UpdateServiceOfferingCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -26,12 +25,11 @@ import com.cloud.api.Implementation;
 import com.cloud.api.Parameter;
 import com.cloud.api.ResponseObject;
 import com.cloud.api.response.ServiceOfferingResponse;
-import com.cloud.offering.ServiceOffering.GuestIpType;
+import com.cloud.offering.NetworkOffering.GuestIpType;
 import com.cloud.service.ServiceOfferingVO;
 
 @Implementation(method="updateServiceOffering", manager=Manager.ConfigManager)
-public class UpdateServiceOfferingCmd extends BaseCmd
-{
+public class UpdateServiceOfferingCmd extends BaseCmd {
     public static final Logger s_logger = Logger.getLogger(UpdateServiceOfferingCmd.class.getName());
     private static final String s_name = "updateserviceofferingresponse";
 
@@ -91,7 +89,7 @@ public class UpdateServiceOfferingCmd extends BaseCmd
 
     @Override
     public String getName() {
-        return s_name;
+        return s_name;
     }
 
     @Override
diff --git a/server/src/com/cloud/api/commands/UpdateTemplateCmd.java b/server/src/com/cloud/api/commands/UpdateTemplateCmd.java
index 7d27a7aa589..84fe561ab7a 100644
--- a/server/src/com/cloud/api/commands/UpdateTemplateCmd.java
+++ b/server/src/com/cloud/api/commands/UpdateTemplateCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
diff --git a/server/src/com/cloud/api/commands/UpdateVMCmd.java b/server/src/com/cloud/api/commands/UpdateVMCmd.java
index 1fef5ad7347..16c700c73c9 100644
--- a/server/src/com/cloud/api/commands/UpdateVMCmd.java
+++ b/server/src/com/cloud/api/commands/UpdateVMCmd.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
 
 import org.apache.log4j.Logger;
@@ -74,7 +73,7 @@ public class UpdateVMCmd extends BaseCmd{
 
     @Override
     public String getName() {
-        return s_name;
+        return s_name;
     }
     
     public static String getResultObjectName() {
diff --git a/server/src/com/cloud/api/commands/UpdateVMGroupCmd.java b/server/src/com/cloud/api/commands/UpdateVMGroupCmd.java
new file mode 100644
index 00000000000..3147e8b8e99
--- /dev/null
+++ b/server/src/com/cloud/api/commands/UpdateVMGroupCmd.java
@@ -0,0 +1,87 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General License for more details.
+ * 
+ * You should have received a copy of the GNU General License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.api.commands;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.api.ApiDBUtils;
+import com.cloud.api.BaseCmd;
+import com.cloud.api.Implementation;
+import com.cloud.api.Parameter;
+import com.cloud.api.ResponseObject;
+import com.cloud.api.response.InstanceGroupResponse;
+import com.cloud.user.Account;
+import com.cloud.vm.InstanceGroupVO;
+
+@Implementation(method="updateVmGroup")
+public class UpdateVMGroupCmd extends BaseCmd{
+
+    private static final String s_name = "updateinstancegroupresponse";
+    public static final Logger s_logger = Logger.getLogger(UpdateVMGroupCmd.class.getName());
+
+    /////////////////////////////////////////////////////
+    //////////////// API parameters /////////////////////
+    /////////////////////////////////////////////////////
+
+    @Parameter(name="id", type=CommandType.LONG, required=true)
+    private Long id;
+
+    @Parameter(name="name", type=CommandType.STRING)
+    private String groupName;
+
+    /////////////////////////////////////////////////////
+    /////////////////// Accessors ///////////////////////
+    /////////////////////////////////////////////////////
+
+    public Long getId() {
+        return id;
+    }
+
+    public String getGroupName() {
+        return groupName;
+    }
+
+    /////////////////////////////////////////////////////
+    /////////////// API Implementation///////////////////
+    /////////////////////////////////////////////////////
+
+    @Override
+    public String getName() {
+        return s_name;
+    }
+
+    @Override
+    public ResponseObject getResponse() {
+        InstanceGroupVO group = (InstanceGroupVO)getResponseObject();
+
+        InstanceGroupResponse response = new InstanceGroupResponse();
+        response.setId(group.getId());
+        response.setName(group.getName());
+        response.setCreated(group.getCreated());
+
+        Account accountTemp = ApiDBUtils.findAccountById(group.getAccountId());
+        if (accountTemp != null) {
+            response.setAccountName(accountTemp.getAccountName());
+            response.setDomainId(accountTemp.getDomainId());
+            response.setDomainName(ApiDBUtils.findDomainById(accountTemp.getDomainId()).getName());
+        }
+
+        response.setResponseName(getName());
+        return response;
+    }
+}
diff --git a/server/src/com/cloud/api/commands/UpgradeVMCmd.java b/server/src/com/cloud/api/commands/UpgradeVMCmd.java
index 7e5646fb316..0ae7afaae0a 100644
--- a/server/src/com/cloud/api/commands/UpgradeVMCmd.java
+++ b/server/src/com/cloud/api/commands/UpgradeVMCmd.java
@@ -15,9 +15,8 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.api.commands;
-
+
 import org.apache.log4j.Logger;
 
 import com.cloud.api.ApiDBUtils;
@@ -73,7 +72,7 @@ public class UpgradeVMCmd extends BaseCmd {
 
     public static String getResultObjectName() {
     	return "virtualmachine";
-    }    
+    }
     
     @Override
     public ResponseObject getResponse() {
diff --git a/server/src/com/cloud/api/response/CapabilitiesResponse.java b/server/src/com/cloud/api/response/CapabilitiesResponse.java
new file mode 100644
index 00000000000..6b4114c8f8c
--- /dev/null
+++ b/server/src/com/cloud/api/response/CapabilitiesResponse.java
@@ -0,0 +1,27 @@
+package com.cloud.api.response;
+
+import com.google.gson.annotations.SerializedName;
+
+public class CapabilitiesResponse extends BaseResponse {
+    @SerializedName("networkgroupsenabled")
+    private String networkGroupsEnabled;
+
+    @SerializedName("cloudstackversion")
+    private String cloudStackVersion;
+
+    public String getNetworkGroupsEnabled() {
+        return networkGroupsEnabled;
+    }
+
+    public void setNetworkGroupsEnabled(String networkGroupsEnabled) {
+        this.networkGroupsEnabled = networkGroupsEnabled;
+    }
+
+    public String getCloudStackVersion() {
+        return cloudStackVersion;
+    }
+
+    public void setCloudStackVersion(String cloudStackVersion) {
+        this.cloudStackVersion = cloudStackVersion;
+    }
+}
diff --git a/server/src/com/cloud/api/response/ExtractResponse.java b/server/src/com/cloud/api/response/ExtractResponse.java
new file mode 100644
index 00000000000..b3249dc7eae
--- /dev/null
+++ b/server/src/com/cloud/api/response/ExtractResponse.java
@@ -0,0 +1,156 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General License for more details.
+ * 
+ * You should have received a copy of the GNU General License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.api.response;
+
+import java.util.Date;
+
+import com.google.gson.annotations.SerializedName;
+
+public class ExtractResponse extends BaseResponse {
+    @SerializedName("id")
+    private long id;
+    
+    @SerializedName("name")
+    private String name;
+    
+    @SerializedName("uploadpercentage")
+    private int uploadPercent;
+    
+    @SerializedName("uploadstatus")
+    private String uploadStatus;
+    
+    @SerializedName("accountid")
+    private long accountId;    
+ 
+    @SerializedName("resultstring")
+    private String resultString;    
+
+    @SerializedName("created")
+    private Date createdDate;
+
+    @SerializedName("state")
+    private String state;
+    
+    @SerializedName("storagetype")
+    private String storageType;
+    
+    @SerializedName("storage")
+    private String storage;
+    
+    @SerializedName("zoneid")
+    private Long zoneId;
+
+    @SerializedName("zonename")
+    private String zoneName;
+
+    public long getId() {
+        return id;
+    }
+
+    public void setId(long id) {
+        this.id = id;
+    }
+
+    public String getName() {
+        return name;
+    }
+
+    public void setName(String name) {
+        this.name = name;
+    }
+
+    public int getUploadPercent() {
+        return uploadPercent;
+    }
+
+    public void setUploadPercent(int uploadPercent) {
+        this.uploadPercent = uploadPercent;
+    }
+
+    public String getUploadStatus() {
+        return uploadStatus;
+    }
+
+    public void setUploadStatus(String uploadStatus) {
+        this.uploadStatus = uploadStatus;
+    }
+
+    public long getAccountId() {
+        return accountId;
+    }
+
+    public void setAccountId(long accountId) {
+        this.accountId = accountId;
+    }
+
+    public String getResultString() {
+        return resultString;
+    }
+
+    public void setResultString(String resultString) {
+        this.resultString = resultString;
+    }
+
+    public Date getCreatedDate() {
+        return createdDate;
+    }
+
+    public void setCreatedDate(Date createdDate) {
+        this.createdDate = createdDate;
+    }
+
+    public String getState() {
+        return state;
+    }
+
+    public void setState(String state) {
+        this.state = state;
+    }
+
+    public String getStorageType() {
+        return storageType;
+    }
+
+    public void setStorageType(String storageType) {
+        this.storageType = storageType;
+    }
+
+    public String getStorage() {
+        return storage;
+    }
+
+    public void setStorage(String storage) {
+        this.storage = storage;
+    }
+
+    public Long getZoneId() {
+        return zoneId;
+    }
+
+    public void setZoneId(Long zoneId) {
+        this.zoneId = zoneId;
+    }
+
+    public String getZoneName() {
+        return zoneName;
+    }
+
+    public void setZoneName(String zoneName) {
+        this.zoneName = zoneName;
+    }
+}
diff --git a/server/src/com/cloud/api/response/InstanceGroupResponse.java b/server/src/com/cloud/api/response/InstanceGroupResponse.java
new file mode 100644
index 00000000000..25222777b31
--- /dev/null
+++ b/server/src/com/cloud/api/response/InstanceGroupResponse.java
@@ -0,0 +1,73 @@
+package com.cloud.api.response;
+
+import java.util.Date;
+
+import com.google.gson.annotations.SerializedName;
+
+public class InstanceGroupResponse extends BaseResponse {
+    @SerializedName("id")
+    private Long id;
+
+    @SerializedName("name")
+    private String name;
+
+    @SerializedName("created")
+    private Date created;
+
+    @SerializedName("account")
+    private String accountName;
+
+    @SerializedName("domainid")
+    private Long domainId;
+
+    @SerializedName("domain")
+    private String domainName;
+
+    public Long getId() {
+        return id;
+    }
+
+    public void setId(Long id) {
+        this.id = id;
+    }
+
+    public String getName() {
+        return name;
+    }
+
+    public void setName(String name) {
+        this.name = name;
+    }
+
+    public Date getCreated() {
+        return created;
+    }
+
+    public void setCreated(Date created) {
+        this.created = created;
+    }
+
+    public String getAccountName() {
+        return accountName;
+    }
+
+    public void setAccountName(String accountName) {
+        this.accountName = accountName;
+    }
+
+    public Long getDomainId() {
+        return domainId;
+    }
+
+    public void setDomainId(Long domainId) {
+        this.domainId = domainId;
+    }
+
+    public String getDomainName() {
+        return domainName;
+    }
+
+    public void setDomainName(String domainName) {
+        this.domainName = domainName;
+    }
+}
diff --git a/server/src/com/cloud/api/response/UserVmResponse.java b/server/src/com/cloud/api/response/UserVmResponse.java
index fba665107b1..dcc799ede7a 100644
--- a/server/src/com/cloud/api/response/UserVmResponse.java
+++ b/server/src/com/cloud/api/response/UserVmResponse.java
@@ -52,6 +52,9 @@ public class UserVmResponse extends BaseResponse {
     @SerializedName("haenable")
     private Boolean haEnable;
 
+    @SerializedName("groupid")
+    private Long groupId;
+
     @SerializedName("group")
     private String group;
 
@@ -207,6 +210,14 @@ public class UserVmResponse extends BaseResponse {
         this.haEnable = haEnable;
     }
 
+    public Long getGroupId() {
+        return groupId;
+    }
+
+    public void setGroupId(Long groupId) {
+        this.groupId = groupId;
+    }
+
     public String getGroup() {
         return group;
     }
diff --git a/server/src/com/cloud/async/AsyncJobManagerImpl.java b/server/src/com/cloud/async/AsyncJobManagerImpl.java
index 14a2f5c664b..c8fdc8fc343 100644
--- a/server/src/com/cloud/async/AsyncJobManagerImpl.java
+++ b/server/src/com/cloud/async/AsyncJobManagerImpl.java
@@ -506,7 +506,7 @@ public class AsyncJobManagerImpl implements AsyncJobManager {
 					List l = _jobDao.getExpiredJobs(cutTime, 100);
 					if(l != null && l.size() > 0) {
 						for(AsyncJobVO job : l) {
-							_jobDao.delete(job.getId());
+							_jobDao.expunge(job.getId());
 						}
 					}
 					
diff --git a/server/src/com/cloud/async/SyncQueueManagerImpl.java b/server/src/com/cloud/async/SyncQueueManagerImpl.java
index b50fbc74598..cf3097f5b7f 100644
--- a/server/src/com/cloud/async/SyncQueueManagerImpl.java
+++ b/server/src/com/cloud/async/SyncQueueManagerImpl.java
@@ -184,7 +184,7 @@ public class SyncQueueManagerImpl implements SyncQueueManager {
 			if(itemVO != null) {
 				SyncQueueVO queueVO = _syncQueueDao.lock(itemVO.getQueueId(), true);
 				
-				_syncQueueItemDao.delete(itemVO.getId());
+				_syncQueueItemDao.expunge(itemVO.getId());
 				
 				queueVO.setLastProcessTime(null);
 				queueVO.setLastUpdated(DateUtil.currentGMTTime());
diff --git a/server/src/com/cloud/async/executor/CopyTemplateExecutor.java b/server/src/com/cloud/async/executor/CopyTemplateExecutor.java
index 2652a4cd9b0..f41aaadc210 100644
--- a/server/src/com/cloud/async/executor/CopyTemplateExecutor.java
+++ b/server/src/com/cloud/async/executor/CopyTemplateExecutor.java
@@ -27,7 +27,7 @@ import com.cloud.async.AsyncJobResult;
 import com.cloud.async.AsyncJobVO;
 import com.cloud.async.BaseAsyncJobExecutor;
 import com.cloud.dc.DataCenterVO;
-import com.cloud.domain.DomainVO;
+import com.cloud.domain.Domain;
 import com.cloud.serializer.GsonHelper;
 import com.cloud.server.ManagementServer;
 import com.cloud.storage.VMTemplateHostVO;
@@ -81,7 +81,7 @@ public class CopyTemplateExecutor extends BaseAsyncJobExecutor {
 	public void processTimeout(VMOperationListener listener, long agentId, long seq) {
 	}
 	
-	private CopyTemplateResultObject composeResultObject(VMTemplateVO template, VMTemplateHostVO templateHostRef, DataCenterVO destZone, String guestOSName, Account owner, DomainVO domain) {
+	private CopyTemplateResultObject composeResultObject(VMTemplateVO template, VMTemplateHostVO templateHostRef, DataCenterVO destZone, String guestOSName, Account owner, Domain domain) {
 		CopyTemplateResultObject resultObject = new CopyTemplateResultObject();
 		
 		
@@ -91,7 +91,7 @@ public class CopyTemplateExecutor extends BaseAsyncJobExecutor {
 		if(owner.getType() == Account.ACCOUNT_TYPE_ADMIN)
 			isAdmin = true;
 		
-		if (isAdmin || owner.getId().longValue() == template.getAccountId()) {
+		if (isAdmin || owner.getId() == template.getAccountId()) {
             // add download status
             if (templateHostRef.getDownloadState()!=Status.DOWNLOADED) {
                 String templateStatus = "Processing";
diff --git a/server/src/com/cloud/async/executor/CreatePrivateTemplateExecutor.java b/server/src/com/cloud/async/executor/CreatePrivateTemplateExecutor.java
index d27057e64b4..fabb36929c8 100644
--- a/server/src/com/cloud/async/executor/CreatePrivateTemplateExecutor.java
+++ b/server/src/com/cloud/async/executor/CreatePrivateTemplateExecutor.java
@@ -144,7 +144,7 @@ public class CreatePrivateTemplateExecutor extends VolumeOperationExecutor {
             				    jobStatus = AsyncJobResult.STATUS_SUCCEEDED;
             				    resultCode = 0;
             				    details = null;
-            				    String eventParams = "id="+template.getId()+"\nname=" + template.getName() +"\nsize="+volume.getSize();
+            				    String eventParams = "id="+template.getId()+"\nname=" + template.getName() +"\nsize="+volume.getSize()+ "\ndcId="+volume.getDataCenterId();
             				    managerServer.saveEvent(param.getUserId(), param.getAccountId(), EventVO.LEVEL_INFO, EventTypes.EVENT_TEMPLATE_CREATE, "Successfully created Template " +param.getName(), eventParams ,param.getEventId());
             				    resultObject = composeResultObject(template, templateHostRef, volume.getDataCenterId());
             				} 
diff --git a/server/src/com/cloud/async/executor/DeployVMExecutor.java b/server/src/com/cloud/async/executor/DeployVMExecutor.java
index 00247666c1d..adef8fda909 100644
--- a/server/src/com/cloud/async/executor/DeployVMExecutor.java
+++ b/server/src/com/cloud/async/executor/DeployVMExecutor.java
@@ -38,11 +38,15 @@ import com.cloud.exception.ResourceAllocationException;
 import com.cloud.serializer.GsonHelper;
 import com.cloud.server.ManagementServer;
 import com.cloud.service.ServiceOfferingVO;
+import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.VMTemplateVO;
+import com.cloud.storage.VolumeVO;
 import com.cloud.user.Account;
 import com.cloud.user.User;
 import com.cloud.uservm.UserVm;
+import com.cloud.utils.Pair;
 import com.cloud.utils.exception.ExecutionException;
+import com.cloud.vm.InstanceGroupVO;
 import com.google.gson.Gson;
 
 public class DeployVMExecutor extends VMOperationExecutor {
@@ -155,14 +159,17 @@ public class DeployVMExecutor extends VMOperationExecutor {
 			resultObject.setDisplayName(vm.getDisplayName());
 		}
 		
-		if (vm.getGroup() != null) {
-			resultObject.setGroup(vm.getGroup());
-		}
-		
 		if(vm.getState() != null)
 			resultObject.setState(vm.getState().toString());
 		
-		ManagementServer managementServer = getAsyncJobMgr().getExecutorContext().getManagementServer();
+		ManagementServer managementServer = getAsyncJobMgr().getExecutorContext().getManagementServer();
+		
+		InstanceGroupVO group = managementServer.getGroupForVm(vm.getId());
+		if (group != null) {
+			resultObject.setGroupId(group.getId());
+			resultObject.setGroup(group.getName());
+		}
+		
         VMTemplateVO template = managementServer.findTemplateById(vm.getTemplateId());
         
         Account acct = managementServer.findAccountById(Long.valueOf(vm.getAccountId()));
@@ -229,7 +236,16 @@ public class DeployVMExecutor extends VMOperationExecutor {
 
         resultObject.setCpuNumber(String.valueOf(offering.getCpu()));
         resultObject.setCpuSpeed(String.valueOf(offering.getSpeed()));
-        resultObject.setMemory(String.valueOf(offering.getRamSize()));
+        resultObject.setMemory(String.valueOf(offering.getRamSize()));
+        
+        //root device related
+        VolumeVO rootVolume = managementServer.findRootVolume(vm.getId());
+        if(rootVolume!=null)
+        {
+        	resultObject.setRootDeviceId(rootVolume.getDeviceId());
+        	StoragePoolVO storagePool = managementServer.findPoolById(rootVolume.getPoolId());
+        	resultObject.setRootDeviceType(storagePool.getPoolType().toString());
+        }
         
 //        resultObject.setNetworkGroupList(managementServer.getNetworkGroupsNamesForVm(vm.getId()));
 		return resultObject;
diff --git a/server/src/com/cloud/async/executor/DeployVMResultObject.java b/server/src/com/cloud/async/executor/DeployVMResultObject.java
index 3ef9bad1b3e..43e4a7f321a 100644
--- a/server/src/com/cloud/async/executor/DeployVMResultObject.java
+++ b/server/src/com/cloud/async/executor/DeployVMResultObject.java
@@ -101,7 +101,10 @@ public class DeployVMResultObject {
 	private String displayName;
 	
 	@Param(name="group")
-	private String group;
+	private String group;
+	
+	@Param(name="groupid")
+	private Long groupId;
 	
 	@Param(name="domainid")
 	private Long domainId;
@@ -121,6 +124,28 @@ public class DeployVMResultObject {
 	@Param(name="networkgrouplist")
 	private String networkGroupList;
 	
+	@Param(name="rootdeviceid")
+	private Long rootDeviceId;
+	
+	@Param(name="rootdevicetype")
+	private String rootDeviceType;
+	
+	public Long getRootDeviceId(){
+		return this.rootDeviceId;
+	}
+	
+	public void setRootDeviceId(Long rootDeviceId){
+		this.rootDeviceId = rootDeviceId;
+	}
+	
+	public String getRootDeviceType(){
+		return this.rootDeviceType;
+	}
+	
+	public void setRootDeviceType(String deviceType){
+		this.rootDeviceType = deviceType;
+	}
+	
 	public String getNetworkGroupList(){
 		return this.networkGroupList;
 	}
@@ -335,6 +360,14 @@ public class DeployVMResultObject {
 
 	public void setGroup(String group) {
 		this.group = group;
+	}
+	
+	public Long getGroupId() {
+		return groupId;
+	}
+
+	public void setGroupId(Long groupId) {
+		this.groupId = groupId;
 	}
 	
 	public Long getDomainId() {
diff --git a/server/src/com/cloud/async/executor/ExtractJobResultObject.java b/server/src/com/cloud/async/executor/ExtractJobResultObject.java
new file mode 100644
index 00000000000..61474ea8962
--- /dev/null
+++ b/server/src/com/cloud/async/executor/ExtractJobResultObject.java
@@ -0,0 +1,153 @@
+package com.cloud.async.executor;
+
+import java.util.Date;
+
+import com.cloud.async.AsyncInstanceCreateStatus;
+import com.cloud.serializer.Param;
+import com.cloud.storage.Volume.VolumeType;
+import com.cloud.storage.upload.UploadState;
+
+public class ExtractJobResultObject {
+	
+	public ExtractJobResultObject(Long accountId, String typeName, String currState, int i, Long uploadId){
+		this.accountId = accountId;
+		this.name = typeName;
+		this.state = currState;
+		this.id = uploadId;
+		this.uploadPercent = i;
+	}
+
+	public ExtractJobResultObject(){		
+	}
+	
+	@Param(name="id")
+	private long id;
+    
+	@Param(name="name")
+	private String name;
+	
+    @Param(name="uploadPercentage")
+	private int uploadPercent;
+    
+    @Param(name="uploadStatus")
+	private String uploadStatus;
+    
+    @Param(name="accountid")
+    long accountId;    
+ 
+    @Param(name="result_string")
+    String result_string;    
+
+    public int getUploadPercent() {
+		return uploadPercent;
+	}
+
+	public void setUploadPercent(int i) {
+		this.uploadPercent = i;
+	}
+
+	public String getUploadStatus() {
+		return uploadStatus;
+	}
+
+	public void setUploadStatus(String uploadStatus) {
+		this.uploadStatus = uploadStatus;
+	}
+
+	public String getResult_string() {
+		return result_string;
+	}
+
+	public void setResult_string(String resultString) {
+		result_string = resultString;
+	}
+
+    @Param(name="created")
+    private Date createdDate;
+
+    @Param(name="state")
+    private String state;
+    
+    @Param(name="storagetype")
+	String storageType;
+    
+    @Param(name="storage")
+    private String storage;
+    
+    @Param(name="zoneid")
+    private Long zoneId;
+
+    public Long getZoneId() {
+		return zoneId;
+	}
+
+	public void setZoneId(Long zoneId) {
+		this.zoneId = zoneId;
+	}
+
+	public String getZoneName() {
+		return zoneName;
+	}
+
+	public void setZoneName(String zoneName) {
+		this.zoneName = zoneName;
+	}
+
+	@Param(name="zonename")
+    private String zoneName;
+
+	private long size;
+
+	public String getStorage() {
+		return storage;
+	}
+
+	public void setStorage(String storage) {
+		this.storage = storage;
+	}
+
+	public void setId(long id) {
+		this.id = id;
+	}
+	
+	public long getId() {
+		return id;
+	}
+
+	public void setName(String name) {
+		this.name = name;
+	}
+	
+	public String getName() {
+		return name;
+	}
+
+	public void setSize(long size) {
+		this.size = size;
+	}
+          
+    public void setCreatedDate(Date createdDate) {
+        this.createdDate = createdDate;
+    }
+
+    public Date getCreatedDate() {
+        return createdDate;
+    }
+
+    public void setState(String status) {
+        this.state = status;
+    }
+
+    public String getState() {
+        return state;
+    }
+   
+    public void setStorageType (String storageType) {
+    	this.storageType = storageType;
+    }
+    
+    public String getStorageType() {
+    	return storageType;
+    }
+    
+}
diff --git a/server/src/com/cloud/async/executor/ExtractTemplateExecutor.java b/server/src/com/cloud/async/executor/ExtractTemplateExecutor.java
new file mode 100644
index 00000000000..141f905649f
--- /dev/null
+++ b/server/src/com/cloud/async/executor/ExtractTemplateExecutor.java
@@ -0,0 +1,39 @@
+package com.cloud.async.executor;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.api.BaseCmd;
+import com.cloud.async.AsyncJobManager;
+import com.cloud.async.AsyncJobResult;
+import com.cloud.async.AsyncJobVO;
+import com.cloud.async.BaseAsyncJobExecutor;
+import com.cloud.serializer.GsonHelper;
+import com.cloud.server.ManagementServer;
+import com.google.gson.Gson;
+
+public class ExtractTemplateExecutor extends BaseAsyncJobExecutor {
+
+	public static final Logger s_logger = Logger.getLogger(ExtractTemplateExecutor.class.getName());
+	@Override
+	public boolean execute() {
+	   	Gson gson = GsonHelper.getBuilder().create();
+    	AsyncJobManager asyncMgr = getAsyncJobMgr();
+    	AsyncJobVO job = getJob();
+
+    	ExtractTemplateParam param = gson.fromJson(job.getCmdInfo(), ExtractTemplateParam.class);   	
+    	ManagementServer managementServer = asyncMgr.getExecutorContext().getManagementServer();
+ 
+    	/*
+		try {		
+				managementServer.extractTemplate(param.getUrl(), param.getTemplateId(), param.getZoneId(), param.getEventId(), getJob().getId());						
+		
+		} catch (Exception e) {
+			s_logger.warn("Unable to extract template: " + e.getMessage(), e);
+    		asyncMgr.completeAsyncJob(getJob().getId(), AsyncJobResult.STATUS_FAILED, BaseCmd.INTERNAL_ERROR, e.getMessage());
+		}
+		*/
+		
+    	return true;
+	}
+
+}
diff --git a/server/src/com/cloud/async/executor/ExtractTemplateParam.java b/server/src/com/cloud/async/executor/ExtractTemplateParam.java
new file mode 100644
index 00000000000..b42242e39c8
--- /dev/null
+++ b/server/src/com/cloud/async/executor/ExtractTemplateParam.java
@@ -0,0 +1,50 @@
+package com.cloud.async.executor;
+
+public class ExtractTemplateParam {
+	
+	private long userId;
+	private long templateId;
+	private Long zoneId;
+	private long eventId;
+	private String url;
+
+	public ExtractTemplateParam() {
+	}
+
+	public ExtractTemplateParam(long userId, long templateId, Long zoneId, long eventId, String url) {
+		this.userId = userId;
+		this.templateId = templateId;
+		this.zoneId = zoneId;
+		this.eventId = eventId;
+		this.url = url;
+	}
+	
+	public String getUrl() {
+		return url;
+	}
+
+	public void setUrl(String url) {
+		this.url = url;
+	}
+
+	public long getUserId() {
+		return userId;
+	}
+	
+	public long getTemplateId() {
+		return templateId;
+	}
+	
+	public Long getZoneId() {
+		return zoneId;
+	}
+
+    public void setEventId(long eventId) {
+        this.eventId = eventId;
+    }
+
+    public long getEventId() {
+        return eventId;
+    }
+
+}
diff --git a/server/src/com/cloud/async/executor/ExtractVolumeExecutor.java b/server/src/com/cloud/async/executor/ExtractVolumeExecutor.java
new file mode 100644
index 00000000000..71ee35f1506
--- /dev/null
+++ b/server/src/com/cloud/async/executor/ExtractVolumeExecutor.java
@@ -0,0 +1,40 @@
+package com.cloud.async.executor;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.api.BaseCmd;
+import com.cloud.async.AsyncJobManager;
+import com.cloud.async.AsyncJobResult;
+import com.cloud.async.AsyncJobVO;
+import com.cloud.async.BaseAsyncJobExecutor;
+import com.cloud.serializer.GsonHelper;
+import com.cloud.server.ManagementServer;
+import com.google.gson.Gson;
+
+public class ExtractVolumeExecutor extends BaseAsyncJobExecutor {
+
+	public static final Logger s_logger = Logger.getLogger(ExtractVolumeExecutor.class.getName());
+	
+	@Override
+	public boolean execute() {
+		Gson gson = GsonHelper.getBuilder().create();
+    	AsyncJobManager asyncMgr = getAsyncJobMgr();
+    	AsyncJobVO job = getJob();
+
+    	ExtractTemplateParam param = gson.fromJson(job.getCmdInfo(), ExtractTemplateParam.class);   	
+    	ManagementServer managementServer = asyncMgr.getExecutorContext().getManagementServer();
+ 
+    	/*
+		try {
+				managementServer.extractVolume(param.getUrl(), param.getTemplateId(), param.getZoneId(), param.getEventId(), getJob().getId());					
+		
+		} catch (Exception e) {
+			s_logger.warn("Unable to extract volume: " + e.getMessage(), e);
+    		asyncMgr.completeAsyncJob(getJob().getId(), AsyncJobResult.STATUS_FAILED, BaseCmd.INTERNAL_ERROR, e.getMessage());
+		}
+		*/
+		
+    	return true;
+	}
+
+}
diff --git a/server/src/com/cloud/async/executor/RebootRouterExecutor.java b/server/src/com/cloud/async/executor/RebootRouterExecutor.java
index e5f6298b8b1..2c8c71e3b4e 100644
--- a/server/src/com/cloud/async/executor/RebootRouterExecutor.java
+++ b/server/src/com/cloud/async/executor/RebootRouterExecutor.java
@@ -40,7 +40,8 @@ public class RebootRouterExecutor extends VMOperationExecutor {
     	AsyncJobVO job = getJob();
 		ManagementServer managementServer = asyncMgr.getExecutorContext().getManagementServer();
     	VMOperationParam param = gson.fromJson(job.getCmdInfo(), VMOperationParam.class);
-    	
+    	
+    	/*
 		if(getSyncSource() == null) {
 	    	asyncMgr.syncAsyncJobExecution(job, "Router", param.getVmId());
 			return true;
@@ -66,7 +67,9 @@ public class RebootRouterExecutor extends VMOperationExecutor {
 						e.getMessage());
 			}
 	    	return true;
-		}
+		}
+		*/
+    	return true;
 	}
 	
 	public void processAnswer(VMOperationListener listener, long agentId, long seq, Answer answer) {
diff --git a/server/src/com/cloud/async/executor/RouterExecutorHelper.java b/server/src/com/cloud/async/executor/RouterExecutorHelper.java
index cdc6b37d4f8..12b0d9c1b6f 100644
--- a/server/src/com/cloud/async/executor/RouterExecutorHelper.java
+++ b/server/src/com/cloud/async/executor/RouterExecutorHelper.java
@@ -44,7 +44,8 @@ public class RouterExecutorHelper {
 		resultObject.setGuestMacAddress(router.getGuestMacAddress());
 		resultObject.setTemplateId(router.getTemplateId());
 		resultObject.setCreated(router.getCreated());
-		
+		resultObject.setGuestNetmask(router.getGuestNetmask());
+        
 		if (router.getHostId() != null) {
         	resultObject.setHostname(managementServer.getHostBy(router.getHostId()).getName());
         	resultObject.setHostId(router.getHostId());
diff --git a/server/src/com/cloud/async/executor/RouterOperationResultObject.java b/server/src/com/cloud/async/executor/RouterOperationResultObject.java
index 7945535a11c..74a0fb7a067 100644
--- a/server/src/com/cloud/async/executor/RouterOperationResultObject.java
+++ b/server/src/com/cloud/async/executor/RouterOperationResultObject.java
@@ -96,8 +96,19 @@ public class RouterOperationResultObject {
 	private String state;
 	
 	@Param(name="hostname")
-	private String hostname;
-
+	private String hostname;
+	
+	@Param(name="guestnetmask")
+	private String guestNetmask;
+	
+	public String getGuestNetmask(){
+		return this.guestNetmask;
+	}
+	
+	public void setGuestNetmask(String guestNetmask){
+		this.guestNetmask = guestNetmask;
+	}
+	
 	public long getId() {
 		return id;
 	}
diff --git a/server/src/com/cloud/async/executor/StartRouterExecutor.java b/server/src/com/cloud/async/executor/StartRouterExecutor.java
index 2c314d23d7b..d8c9a8093a6 100644
--- a/server/src/com/cloud/async/executor/StartRouterExecutor.java
+++ b/server/src/com/cloud/async/executor/StartRouterExecutor.java
@@ -44,7 +44,8 @@ public class StartRouterExecutor extends VMOperationExecutor {
     	AsyncJobVO job = getJob();
 		ManagementServer managementServer = asyncMgr.getExecutorContext().getManagementServer();
     	VMOperationParam param = gson.fromJson(job.getCmdInfo(), VMOperationParam.class);
-    	
+    	
+    	/*
 		if(getSyncSource() == null) {
 	    	asyncMgr.syncAsyncJobExecution(job, "Router", param.getVmId());
 			return true;
@@ -74,7 +75,9 @@ public class StartRouterExecutor extends VMOperationExecutor {
 			    }
 			}
 	    	return true;
-		}
+		}
+		*/
+    	return true;
 	}
 	
 	public void processAnswer(VMOperationListener listener, long agentId, long seq, Answer answer) {
diff --git a/server/src/com/cloud/async/executor/StopRouterExecutor.java b/server/src/com/cloud/async/executor/StopRouterExecutor.java
index a002444ed80..321f37398eb 100644
--- a/server/src/com/cloud/async/executor/StopRouterExecutor.java
+++ b/server/src/com/cloud/async/executor/StopRouterExecutor.java
@@ -39,7 +39,8 @@ public class StopRouterExecutor extends VMOperationExecutor {
     	AsyncJobVO job = getJob();
 		ManagementServer managementServer = asyncMgr.getExecutorContext().getManagementServer();
     	VMOperationParam param = gson.fromJson(job.getCmdInfo(), VMOperationParam.class);
-    	
+    	
+    	/*
 		if(getSyncSource() == null) {
 	    	asyncMgr.syncAsyncJobExecution(job, "Router", param.getVmId());
 			return true;
@@ -59,7 +60,9 @@ public class StopRouterExecutor extends VMOperationExecutor {
 					e.getMessage());
 	    	}
 	    	return true;
-		}
+		}
+		*/
+    	return true;
 	}
 	
 	public void processAnswer(VMOperationListener listener, long agentId, long seq, Answer answer) {
diff --git a/server/src/com/cloud/async/executor/UpdatePortForwardingRuleExecutor.java b/server/src/com/cloud/async/executor/UpdatePortForwardingRuleExecutor.java
index a1810c74c0c..fbfb24a1ff8 100644
--- a/server/src/com/cloud/async/executor/UpdatePortForwardingRuleExecutor.java
+++ b/server/src/com/cloud/async/executor/UpdatePortForwardingRuleExecutor.java
@@ -1,3 +1,4 @@
+
 package com.cloud.async.executor;
 
 import java.util.List;
@@ -77,6 +78,7 @@ public class UpdatePortForwardingRuleExecutor extends BaseAsyncJobExecutor {
             UserVmVO userVM = userVMs.get(0);
             resultObject.setVirtualMachineId(userVM.getId());
             resultObject.setVirtualMachineName(userVM.getName());
+            resultObject.setVirtualMachineDisplayName(userVM.getDisplayName());
         }
 
         resultObject.setId(firewallRule.getId());
diff --git a/server/src/com/cloud/async/executor/UpdatePortForwardingRuleResultObject.java b/server/src/com/cloud/async/executor/UpdatePortForwardingRuleResultObject.java
index 1743174da0d..a5e81001963 100644
--- a/server/src/com/cloud/async/executor/UpdatePortForwardingRuleResultObject.java
+++ b/server/src/com/cloud/async/executor/UpdatePortForwardingRuleResultObject.java
@@ -27,6 +27,17 @@ public class UpdatePortForwardingRuleResultObject {
     @Param(name="vmname")
     private String virtualMachineName;
 
+    @Param(name="vmdisplayname")
+    private String virtualMachineDisplayName;
+    
+    public String getVirtualMachineDisplayName(){
+    	return this.virtualMachineDisplayName;
+    }
+    
+    public void setVirtualMachineDisplayName(String name){
+    	this.virtualMachineDisplayName = name;
+    }
+    
     public long getId() {
         return id;
     }
diff --git a/server/src/com/cloud/async/executor/VMExecutorHelper.java b/server/src/com/cloud/async/executor/VMExecutorHelper.java
index 469d70e4593..6ec5d41339d 100644
--- a/server/src/com/cloud/async/executor/VMExecutorHelper.java
+++ b/server/src/com/cloud/async/executor/VMExecutorHelper.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.async.executor;
 
 import com.cloud.api.BaseCmd;
@@ -45,12 +44,14 @@ public class VMExecutorHelper {
 			resultObject.setDisplayName(vm.getDisplayName());
 		}
 		
-		if (vm.getGroup() != null) {
-			resultObject.setGroup(vm.getGroup());
-		}
-		
 		if(vm.getState() != null)
-			resultObject.setState(vm.getState().toString());
+			resultObject.setState(vm.getState().toString());
+		
+//		InstanceGroupVO group = managementServer.getGroupForVm(vm.getId());
+//		if (group != null) {
+//			resultObject.setGroupId(group.getId());
+//			resultObject.setGroup(group.getName());
+//		}
 		
         VMTemplateVO template = managementServer.findTemplateById(vm.getTemplateId());
         
@@ -85,8 +86,8 @@ public class VMExecutorHelper {
         resultObject.setPasswordEnabled(templatePasswordEnabled);
         if(templatePasswordEnabled)
         	resultObject.setPassword(vmPassword);
-        else
-        	resultObject.setPassword("");
+//        else
+//        	resultObject.setPassword("");
         
         String isoName = null;
         if (vm.getIsoId() != null) {
diff --git a/server/src/com/cloud/async/executor/VMOperationResultObject.java b/server/src/com/cloud/async/executor/VMOperationResultObject.java
index 19403dcfc66..12da4875eab 100644
--- a/server/src/com/cloud/async/executor/VMOperationResultObject.java
+++ b/server/src/com/cloud/async/executor/VMOperationResultObject.java
@@ -97,7 +97,10 @@ public class VMOperationResultObject {
 	private String displayName;
 	
 	@Param(name="group")
-	private String group;
+	private String group;
+	
+	@Param(name="groupid")
+	private Long groupId;
 	
 	@Param(name="domainid")
 	private Long domainId;
@@ -117,6 +120,28 @@ public class VMOperationResultObject {
 	@Param(name="networkgrouplist")
 	private String networkGroupList;
 	
+	@Param(name="rootdeviceid")
+	private Long rootDeviceId;
+	
+	@Param(name="rootdevicetype")
+	private String rootDeviceType;
+	
+	public Long getRootDeviceId(){
+		return this.rootDeviceId;
+	}
+	
+	public void setRootDeviceId(Long rootDeviceId){
+		this.rootDeviceId = rootDeviceId;
+	}
+	
+	public String getRootDeviceType(){
+		return this.rootDeviceType;
+	}
+	
+	public void setRootDeviceType(String deviceType){
+		this.rootDeviceType = deviceType;
+	}
+	
 	public String getNetworkGroupList(){
 		return this.networkGroupList;
 	}
@@ -324,6 +349,14 @@ public class VMOperationResultObject {
 	public void setGroup(String group) {
 		this.group = group;
 	}
+	
+	public Long getGroupId() {
+		return groupId;
+	}
+
+	public void setGroupId(Long groupId) {
+		this.groupId = groupId;
+	}
 	
 	public Long getDomainId() {
 		return domainId;
diff --git a/server/src/com/cloud/async/executor/VmResultObject.java b/server/src/com/cloud/async/executor/VmResultObject.java
index 930281823a1..eaa32520e65 100644
--- a/server/src/com/cloud/async/executor/VmResultObject.java
+++ b/server/src/com/cloud/async/executor/VmResultObject.java
@@ -8,7 +8,7 @@ public class VmResultObject
 {
 	
 	@Param(name="id")
-	private long id;
+	private Long id;
 	
 	public String getName() {
 		return name;
@@ -98,11 +98,11 @@ public class VmResultObject
 		this.zoneName = zoneName;
 	}
 
-	public long getHostId() {
+	public Long getHostId() {
 		return hostId;
 	}
 
-	public void setHostId(long hostId) {
+	public void setHostId(Long hostId) {
 		this.hostId = hostId;
 	}
 
@@ -202,11 +202,15 @@ public class VmResultObject
 		this.networkKbsWrite = networkKbsWrite;
 	}
 
-	public long isId()
+	public Long getId()
 	{
 		return id;
 	}
 	
+	public void setId(Long id){
+		this.id = id;
+	}
+	
 	@Param(name="name")
 	private String name;
 
@@ -241,7 +245,7 @@ public class VmResultObject
 	private String zoneName;
 	
 	@Param(name="hostid")
-	private long hostId;
+	private Long hostId;
 
 	@Param(name="hostname")
 	private String hostName;
diff --git a/server/src/com/cloud/async/executor/VolumeOperationExecutor.java b/server/src/com/cloud/async/executor/VolumeOperationExecutor.java
index 43425158051..25cb50212a1 100644
--- a/server/src/com/cloud/async/executor/VolumeOperationExecutor.java
+++ b/server/src/com/cloud/async/executor/VolumeOperationExecutor.java
@@ -86,7 +86,7 @@ public class VolumeOperationExecutor extends BaseAsyncJobExecutor {
                     eventType = EventTypes.EVENT_VOLUME_DETACH;
                     failureDescription = "Failed to detach volume";
 
-    				asyncMgr.getExecutorContext().getManagementServer().detachVolumeFromVM(param.getVolumeId(), param.getEventId());
+    				asyncMgr.getExecutorContext().getManagementServer().detachVolumeFromVM(param.getVolumeId(), param.getEventId(),param.getDeviceId(),param.getVmId());
     				success = true;
     				asyncMgr.completeAsyncJob(getJob().getId(), AsyncJobResult.STATUS_SUCCEEDED, 0, null);
     			} else {
diff --git a/server/src/com/cloud/async/executor/VolumeOperationParam.java b/server/src/com/cloud/async/executor/VolumeOperationParam.java
index 4ee39e087a0..b59ddd51f2f 100644
--- a/server/src/com/cloud/async/executor/VolumeOperationParam.java
+++ b/server/src/com/cloud/async/executor/VolumeOperationParam.java
@@ -41,7 +41,7 @@ public class VolumeOperationParam {
 	private long volumeId;
 	private long eventId;
 	private Long deviceId;
-
+	
 	public VolumeOperationParam() {
 	}
 	
diff --git a/server/src/com/cloud/configuration/Config.java b/server/src/com/cloud/configuration/Config.java
index faa05cf60ca..a8e89b1221e 100644
--- a/server/src/com/cloud/configuration/Config.java
+++ b/server/src/com/cloud/configuration/Config.java
@@ -47,7 +47,7 @@ public enum Config {
 	
 	StorageOverprovisioningFactor("Storage", StoragePoolAllocator.class, String.class, "storage.overprovisioning.factor", "2", "Used for storage overprovisioning calculation; available storage will be (actualStorageSize * storage.overprovisioning.factor)", null),
 	StorageStatsInterval("Storage", ManagementServer.class, String.class, "storage.stats.interval", "60000", "The interval in milliseconds when storage stats (per host) are retrieved from agents.", null),
-	MaxVolumeSize("Storage", ManagementServer.class, Integer.class, "max.volume.size.gb", "2000", "The maximum size for a volume in Gb.", null),
+	MaxVolumeSize("Storage", ManagementServer.class, Integer.class, "max.volume.size.gb", "2000", "The maximum size for a volume in gigabytes.", null),
 	TotalRetries("Storage", AgentManager.class, Integer.class, "total.retries", "4", "The number of times each command sent to a host should be retried in case of failure.", null),
 	
 	// Network
@@ -61,7 +61,8 @@ public enum Config {
 	// Usage
 	
 	CapacityCheckPeriod("Usage", ManagementServer.class, Integer.class, "capacity.check.period", "300000", "The interval in milliseconds between capacity checks", null),
-	CapacitySkipCountingHours("Usage", ManagementServer.class, Integer.class, "capacity.skipcounting.hours", "24", "The interval in hours since VM has stopped to skip counting its allocated CPU/Memory capacity", null),
+	CapacitySkipCountingHours("Usage", ManagementServer.class, Integer.class, "capacity.skipcounting.hours", "24", "The interval in hours since VM has stopped to skip counting its allocated CPU/Memory capacity. Applies to vms in Stopped state", null),
+	CapacitySkipCountingDestroyedHours("Usage", ManagementServer.class, Integer.class, "capacity.skipcounting.destroyed.hours", "0", "The interval in hours since VM has stopped to skip counting its allocated CPU/Memory capacity. Applies to vms in Destroyed state", null),
 	StorageAllocatedCapacityThreshold("Usage", ManagementServer.class, Float.class, "storage.allocated.capacity.threshold", "0.85", "Percentage (as a value between 0 and 1) of allocated storage utilization above which alerts will be sent about low storage available.", null),
 	StorageCapacityThreshold("Usage", ManagementServer.class, Float.class, "storage.capacity.threshold", "0.85", "Percentage (as a value between 0 and 1) of storage utilization above which alerts will be sent about low storage available.", null),
 	CPUCapacityThreshold("Usage", ManagementServer.class, Float.class, "cpu.capacity.threshold", "0.85", "Percentage (as a value between 0 and 1) of cpu utilization above which alerts will be sent about low cpu available.", null),
@@ -126,7 +127,7 @@ public enum Config {
 	Wait("Advanced", AgentManager.class, Integer.class, "wait", "1800", "Time to wait for control commands to return", null),
 	Workers("Advanced", AgentManager.class, Integer.class, "workers", "5", "Number of worker threads.", null),
 	MountParent("Advanced", ManagementServer.class, String.class, "mount.parent", "/var/lib/cloud/mnt", "The mount point on the Management Server for Secondary Storage.", null),
-	UpgradeURL("Advanced", ManagementServer.class, String.class, "upgrade.url", "http://example.com:8080/client/agent/update.zip", "The upgrade URL is the URL of the management server that agents will connect to in order to automatically upgrade.", null),
+//	UpgradeURL("Advanced", ManagementServer.class, String.class, "upgrade.url", "http://example.com:8080/client/agent/update.zip", "The upgrade URL is the URL of the management server that agents will connect to in order to automatically upgrade.", null),
 	SystemVMUseLocalStorage("Advanced", ManagementServer.class, Boolean.class, "system.vm.use.local.storage", "false", "Indicates whether to use local storage pools or shared storage pools for system VMs.", null),
 	CPUOverprovisioningFactor("Advanced", ManagementServer.class, String.class, "cpu.overprovisioning.factor", "1", "Used for CPU overprovisioning calculation; available CPU will be (actualCpuCapacity * cpu.overprovisioning.factor)", null),
 	NetworkType("Advanced", ManagementServer.class, String.class, "network.type", "vlan", "The type of network that this deployment will use.", "vlan,direct"),
@@ -146,6 +147,11 @@ public enum Config {
 	DirectAttachNetworkExternalAPIURL("Advanced", ManagementServer.class, String.class, "direct.attach.network.externalIpAllocator.url", null, "Direct-attach VMs using external DHCP server (API url)", null),
 	DirectAttachUntaggedVlanEnabled("Advanced", ManagementServer.class, String.class, "direct.attach.untagged.vlan.enabled", "false", "Indicate whether the system supports direct-attached untagged vlan", "true,false"),
 	CheckPodCIDRs("Advanced", ManagementServer.class, String.class, "check.pod.cidrs", "true", "If true, different pods must belong to different CIDR subnets.", "true,false"),
+	MD5Hashed("Advanced", ManagementServer.class, Boolean.class, "security.password.md5hashed", "true", "If set to false password is sent in clear text or else md5hashed", null),
+	
+	ControlCidr("Advanced", ManagementServer.class, String.class, "control.cidr", "169.254.0.0/16", "Changes the cidr for the control network traffic.  Defaults to using link local.  Must be unique within pods", null),
+	ControlGateway("Advanced", ManagementServer.class, String.class, "control.gateway", "169.254.0.1", "gateway for the control network traffic", null),
+	
 	
 	// XenServer
     VmAllocationAlgorithm("Advanced", ManagementServer.class, String.class, "vm.allocation.algorithm", "random", "If 'random', hosts within a pod will be randomly considered for VM/volume allocation. If 'firstfit', they will be considered on a first-fit basis.", null),
@@ -167,7 +173,7 @@ public enum Config {
     
 	// Premium
 	
-	UsageAggregationTimezone("Premium", ManagementServer.class, String.class, "usage.aggregation.timezone", "GMT", "The timezone to use when aggregating user statistics", null),
+	UsageExecutionTimezone("Premium", ManagementServer.class, String.class, "usage.execution.timezone", null, "The timezone to use for usage job execution time", null),
 	UsageStatsJobAggregationRange("Premium", ManagementServer.class, Integer.class, "usage.stats.job.aggregation.range", "1440", "The range of time for aggregating the user statistics specified in minutes (e.g. 1440 for daily, 60 for hourly.", null),
 	UsageStatsJobExecTime("Premium", ManagementServer.class, String.class, "usage.stats.job.exec.time", "00:15", "The time at which the usage statistics aggregation job will run as an HH24:MM time, e.g. 00:30 to run at 12:30am.", null),
     EnableUsageServer("Premium", ManagementServer.class, Boolean.class, "enable.usage.server", "true", "Flag for enabling usage", null),
diff --git a/server/src/com/cloud/configuration/ConfigurationManagerImpl.java b/server/src/com/cloud/configuration/ConfigurationManagerImpl.java
index 7917d76a523..872f0ed5b45 100644
--- a/server/src/com/cloud/configuration/ConfigurationManagerImpl.java
+++ b/server/src/com/cloud/configuration/ConfigurationManagerImpl.java
@@ -15,7 +15,6 @@
  * along with this program.  If not, see .
  * 
  */
-
 package com.cloud.configuration;
 
 import java.sql.PreparedStatement;
@@ -75,7 +74,8 @@ import com.cloud.exception.InternalErrorException;
 import com.cloud.exception.InvalidParameterValueException;
 import com.cloud.network.NetworkManager;
 import com.cloud.network.dao.IPAddressDao;
-import com.cloud.offering.ServiceOffering.GuestIpType;
+import com.cloud.offering.NetworkOffering;
+import com.cloud.offering.NetworkOffering.GuestIpType;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.service.dao.ServiceOfferingDao;
 import com.cloud.storage.DiskOfferingVO;
@@ -93,10 +93,15 @@ import com.cloud.utils.component.Inject;
 import com.cloud.utils.db.DB;
 import com.cloud.utils.db.Transaction;
 import com.cloud.utils.net.NetUtils;
+import com.cloud.vm.ConsoleProxyVO;
 import com.cloud.vm.DomainRouterVO;
+import com.cloud.vm.SecondaryStorageVmVO;
 import com.cloud.vm.State;
 import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.ConsoleProxyDao;
 import com.cloud.vm.dao.DomainRouterDao;
+import com.cloud.vm.dao.SecondaryStorageVmDao;
 import com.cloud.vm.dao.VMInstanceDao;
 
 @Local(value={ConfigurationManager.class})
@@ -123,6 +128,8 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
 	@Inject HostPodDao _hostPodDao;
 	@Inject AccountManager _accountMgr;
 	@Inject NetworkManager _networkMgr;
+	@Inject ConsoleProxyDao _consoleDao;
+	@Inject SecondaryStorageVmDao _secStorageDao;
 	public boolean _premium;
 
 	private int _maxVolumeSizeInGb;
@@ -162,6 +169,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
         return true;
     }
     
+    @Override
     public void updateConfiguration(long userId, String name, String value) throws InvalidParameterValueException, InternalErrorException {
     	if (value != null && (value.trim().isEmpty() || value.equals("null"))) {
     		value = null;
@@ -390,6 +398,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
 		}
     }
     
+    @Override
     @DB
     public boolean deletePod(DeletePodCmd cmd) throws InvalidParameterValueException, InternalErrorException {
     	Long podId = cmd.getId();
@@ -402,22 +411,22 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
     	if (!validPod(podId)) {
     		throw new InvalidParameterValueException("A pod with ID: " + podId + " does not exist.");
     	}
-    	
+
     	checkIfPodIsDeletable(podId);
-    	
+
     	HostPodVO pod = _podDao.findById(podId);
     	DataCenterVO zone = _zoneDao.findById(pod.getDataCenterId());
-    	
+
     	//Delete the pod and private IP addresses in the pod
-    	if (_podDao.delete(podId) && _privateIpAddressDao.deleteIpAddressByPod(podId)) {
+    	if (_podDao.expunge(podId) && _privateIpAddressDao.deleteIpAddressByPod(podId)) {
     		saveConfigurationEvent(userId, null, EventTypes.EVENT_POD_DELETE, "Successfully deleted pod with name: " + pod.getName() + " in zone: " + zone.getName() + ".", "podId=" + podId, "dcId=" + zone.getId());
     		return true;
-    		
     	} else {
     		return false;
     	}
     }
-    
+
+    @Override
     @DB
     public HostPodVO editPod(UpdatePodCmd cmd) throws InvalidParameterValueException, InternalErrorException 
     {
@@ -481,7 +490,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
     	checkPodAttributes(id, name, pod.getDataCenterId(), gateway, cidr, startIp, endIp, checkForDuplicates);
     	
     	String cidrAddress = getCidrAddress(cidr);
-    	long cidrSize = getCidrSize(cidr);
+    	int cidrSize = getCidrSize(cidr);
     	
     	if (startIp != null && endIp == null) {
     		endIp = NetUtils.getIpRangeEndIpFromCidr(cidrAddress, cidrSize);
@@ -562,7 +571,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
     	checkPodAttributes(-1, podName, zoneId, gateway, cidr, startIp, endIp, true);
 		
 		String cidrAddress = getCidrAddress(cidr);
-		long cidrSize = getCidrSize(cidr);
+		int cidrSize = getCidrSize(cidr);
 		
 		if (startIp != null) {
 			if (endIp == null) {
@@ -753,6 +762,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
     	
     }
     
+    @Override
     @DB
     public void deleteZone(DeleteZoneCmd cmd) throws InvalidParameterValueException, InternalErrorException {
     	
@@ -772,7 +782,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
     	
     	DataCenterVO zone = _zoneDao.findById(zoneId);
     	
-    	_zoneDao.delete(zoneId);
+    	_zoneDao.expunge(zoneId);
     	
     	// Delete vNet
         _zoneDao.deleteVnet(zoneId);
@@ -810,43 +820,26 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
     	if (!validZone(zoneId)) {
     		throw new InvalidParameterValueException("A zone with ID: " + zoneId + " does not exist.");
     	}
-    	
-    	// If DNS values are being changed, make sure there are no VMs in this zone
-    	if (dns1 != null || dns2 != null || internalDns1 != null || internalDns2 != null) {
-    		if (zoneHasVMs(zoneId)) {
-    			throw new InternalErrorException("The zone is not editable because there are VMs in the zone.");
-    		}
-    	}
-    	
+
     	// If the Vnet range is being changed, make sure there are no allocated VNets
     	if (vnetRange != null) {
     		if (zoneHasAllocatedVnets(zoneId)) {
     			throw new InternalErrorException("The vlan range is not editable because there are allocated vlans.");
     		}
     	}
-    	
-    	//To modify a zone, we need to make sure there are no domr's associated with it
-    	//1. List all the domain router objs
-    	//2. Check if any of these has the current data center associated
-    	//3. If yes, throw exception
-    	//4, If no, edit
-    	List allDomainRoutersAvailable = _domrDao.listAll();
-    	
-    	for(DomainRouterVO domR : allDomainRoutersAvailable)
-    	{
-    		if(domR.getDataCenterId() == zoneId)
-    		{
-    			throw new InternalErrorException("The zone is not editable because there are domR's associated with the zone.");
-    		}
-    	}
-    	
-    	//5. Reached here, hence editable   	
+
     	String oldZoneName = zone.getName();
     	
     	if (zoneName == null) {
     		zoneName = oldZoneName;
     	}
     	
+    	boolean dnsUpdate = false;
+    	
+    	if(dns1 != null || dns2 != null){
+    	    dnsUpdate = true;
+    	}
+    	
     	if (dns1 == null) {
     		dns1 = zone.getDns1();
     	}
@@ -886,6 +879,48 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
 	    	_zoneDao.addVnet(zone.getId(), begin, end);
     	}
     	
+    	if(dnsUpdate){
+    	    
+    	    //Update dns for domRs in zone
+    	    
+    	    List DomainRouters = _domrDao.listByDataCenter(zoneId);
+
+    	    for(DomainRouterVO domR : DomainRouters)
+    	    {
+    	        domR.setDns1(dns1);
+    	        domR.setDns2(dns2);
+    	        _domrDao.update(domR.getId(), domR);
+    	    }
+    	    
+    	  //Update dns for console proxies in zone
+            List ConsoleProxies = _vmInstanceDao.listByZoneIdAndType(zoneId, VirtualMachine.Type.ConsoleProxy);
+            
+            for(VMInstanceVO consoleVm : ConsoleProxies)
+            {
+                ConsoleProxyVO proxy = _consoleDao.findById(consoleVm.getId());
+                if( proxy!= null ){
+                    proxy.setDns1(dns1);
+                    proxy.setDns2(dns2);
+                    _consoleDao.update(proxy.getId(), proxy);
+                }
+            }    	    
+            
+          //Update dns for secondary storage Vms in zone
+            List storageVms = _vmInstanceDao.listByZoneIdAndType(zoneId, VirtualMachine.Type.SecondaryStorageVm);
+            
+            for(VMInstanceVO storageVm : storageVms)
+            {
+                SecondaryStorageVmVO secStorageVm = _secStorageDao.findById(storageVm.getId());
+                if( secStorageVm!= null ){
+                    secStorageVm.setDns1(dns1);
+                    secStorageVm.setDns2(dns2);
+                    _secStorageDao.update(secStorageVm.getId(), secStorageVm);
+                }
+            }           
+            
+    	}
+
+    	
     	saveConfigurationEvent(userId, null, EventTypes.EVENT_ZONE_EDIT, "Successfully edited zone with name: " + zone.getName() + ".", "dcId=" + zone.getId(), "dns1=" + dns1, "dns2=" + dns2, "internalDns1=" + internalDns1, "internalDns2=" + internalDns2, "vnetRange=" + vnetRange, "guestCidr=" + guestCidr);
     	
     	return zone;
@@ -926,14 +961,14 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
         checkZoneParameters(zoneName, dns1, dns2, internalDns1, internalDns2, true);
 
         // Create the new zone in the database
-        DataCenterVO zone = new DataCenterVO(null, zoneName, null, dns1, dns2, internalDns1, internalDns2, vnetRange, guestCidr);
+        DataCenterVO zone = new DataCenterVO(zoneName, null, dns1, dns2, internalDns1, internalDns2, vnetRange, guestCidr);
         zone = _zoneDao.persist(zone);
 
         // Add vnet entries for the new zone
         _zoneDao.addVnet(zone.getId(), vnetStart, vnetEnd);
 
         saveConfigurationEvent(userId, null, EventTypes.EVENT_ZONE_CREATE, "Successfully created new zone with name: " + zoneName + ".", "dcId=" + zone.getId(), "dns1=" + dns1, "dns2=" + dns2, "internalDns1=" + internalDns1, "internalDns2=" + internalDns2, "vnetRange=" + vnetRange, "guestCidr=" + guestCidr);
-        
+
         return zone;
     }
 
@@ -1020,7 +1055,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
     	String multicastRateStr = _configDao.getValue("multicast.throttling.rate");
     	int networkRate = ((networkRateStr == null) ? 200 : Integer.parseInt(networkRateStr));
     	int multicastRate = ((multicastRateStr == null) ? 10 : Integer.parseInt(multicastRateStr));
-    	GuestIpType guestIpType = useVirtualNetwork ? GuestIpType.Virtualized : GuestIpType.DirectSingle;        
+    	NetworkOffering.GuestIpType guestIpType = useVirtualNetwork ? NetworkOffering.GuestIpType.Virtualized : NetworkOffering.GuestIpType.DirectSingle;        
     	tags = cleanupTags(tags);
     	ServiceOfferingVO offering = new ServiceOfferingVO(name, cpu, ramSize, speed, networkRate, multicastRate, offerHA, displayText, guestIpType, localStorageRequired, false, tags);
     	
@@ -1032,7 +1067,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
     		return null;
     	}
     }
-    
+
     public ServiceOfferingVO updateServiceOffering(UpdateServiceOfferingCmd cmd) {
     	String displayText = cmd.getDisplayText();
     	Long id = cmd.getId();
@@ -1072,7 +1107,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
         }
 	    
         if (useVirtualNetwork != null) {
-        	GuestIpType guestIpType = useVirtualNetwork ? GuestIpType.Virtualized : GuestIpType.DirectSingle;
+        	NetworkOffering.GuestIpType guestIpType = useVirtualNetwork ? NetworkOffering.GuestIpType.Virtualized : NetworkOffering.GuestIpType.DirectSingle;
             offering.setGuestIpType(guestIpType);
         }
         
@@ -1087,7 +1122,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
         if (_serviceOfferingDao.update(id, offering)) {
         	offering = _serviceOfferingDao.findById(id);
     		saveConfigurationEvent(userId, null, EventTypes.EVENT_SERVICE_OFFERING_EDIT, "Successfully updated service offering with name: " + offering.getName() + ".", "soId=" + offering.getId(), "name=" + offering.getName(),
-    				"displayText=" + offering.getDisplayText(), "offerHA=" + offering.getOfferHA(), "useVirtualNetwork=" + (offering.getGuestIpType() == GuestIpType.Virtualized), "tags=" + offering.getTags());
+    				"displayText=" + offering.getDisplayText(), "offerHA=" + offering.getOfferHA(), "useVirtualNetwork=" + (offering.getGuestIpType() == NetworkOffering.GuestIpType.Virtualized), "tags=" + offering.getTags());
         	return offering;
         } else {
         	return null;
@@ -1122,7 +1157,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
 
         return createDiskOffering(domainId, name, description, numGibibytes, tags);
     }
-    
+
     public DiskOfferingVO updateDiskOffering(UpdateDiskOfferingCmd cmd) throws InvalidParameterValueException{
     	Long diskOfferingId = cmd.getId();
     	String name = cmd.getName();
@@ -1158,15 +1193,16 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
         		diskOffering.setTagsArray(csvTagsToList(tags));
         	}     	
         }
-    	
+
     	if (_diskOfferingDao.update(diskOfferingId, diskOffering)) {
+            saveConfigurationEvent(UserContext.current().getUserId(), null, EventTypes.EVENT_DISK_OFFERING_EDIT, "Successfully updated disk offering with name: " + diskOffering.getName() + ".", "doId=" + diskOffering.getId(), "name=" + diskOffering.getName(),
+                    "displayText=" + diskOffering.getDisplayText(), "diskSize=" + diskOffering.getDiskSize(),"tags=" + diskOffering.getTags());
     		return _diskOfferingDao.findById(diskOfferingId);
     	} else { 
     		return null;
     	}
     }
-    
-    
+
     public boolean deleteDiskOffering(DeleteDiskOfferingCmd cmd) throws InvalidParameterValueException{
     	Long diskOfferingId = cmd.getId();
     	
@@ -1183,7 +1219,6 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
     	}
     }
 
-    
     public boolean deleteServiceOffering(DeleteServiceOfferingCmd cmd) throws InvalidParameterValueException{
     	
         Long offeringId = cmd.getId();
@@ -1229,7 +1264,6 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
     }
 
     @Override
-//    public VlanVO createVlanAndPublicIpRange(long userId, VlanType vlanType, Long zoneId, Long accountId, Long podId, String vlanId, String vlanGateway, String vlanNetmask, String startIP, String endIP) throws InvalidParameterValueException, InternalErrorException {    		
     public VlanVO createVlanAndPublicIpRange(CreateVlanIpRangeCmd cmd) throws InvalidParameterValueException, InternalErrorException, InsufficientCapacityException {
         Long userId = UserContext.current().getUserId();
         if (userId == null) {
@@ -1282,6 +1316,16 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
 	    	
 		}
 
+        //if we have an ip range for vlan id=x, vlantype=y; we should
+	    //only allow adding another range with id=x for same type y
+        if (!vlanId.equals(Vlan.UNTAGGED)) {
+            VlanVO vlanHandle = _vlanDao.findByZoneAndVlanId(zoneId, vlanId);
+
+            if (vlanHandle!=null && !vlanHandle.getVlanType().equals(vlanType))
+                throw new InvalidParameterValueException("This vlan id is already associated with the vlan type "+vlanHandle.getVlanType().toString()
+                        +",whilst you are trying to associate it with vlan type "+vlanType.toString());
+        }
+
     	DataCenterVO zone;
     	if (zoneId == null || ((zone = _zoneDao.findById(zoneId)) == null)) {
 			throw new InvalidParameterValueException("Please specify a valid zone.");
@@ -1320,15 +1364,6 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
         				throw new InvalidParameterValueException("Zone " + zone.getName() + " already has pod-wide IP ranges. A zone may contain either pod-wide IP ranges or account-wide IP ranges, but not both.");
         			}
         		}
-        		
-        		// Make sure the specified account isn't already assigned to a VLAN in this zone
-        		List accountVlanMaps = _accountVlanMapDao.listAccountVlanMapsByAccount(account.getId());
-        		for (AccountVlanMapVO accountVlanMap : accountVlanMaps) {
-        			VlanVO vlan = _vlanDao.findById(accountVlanMap.getVlanDbId());
-        			if (vlan.getDataCenterId() == zone.getId().longValue()) {
-        				throw new InvalidParameterValueException("The account " + account.getAccountName() + " is already assigned to an IP range in zone " + zone.getName() + ".");
-        			}
-        		}
     		} else if (podId != null) {
     			// Pod-wide VLANs must be untagged
         		if (!vlanId.equals(Vlan.UNTAGGED)) {
@@ -1341,10 +1376,10 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
         		}
 
         		// Make sure there aren't any account VLANs in this zone
-        		List accountVlanMaps = _accountVlanMapDao.listAll();
+        		List accountVlanMaps = _accountVlanMapDao.listAllIncludingRemoved();
         		for (AccountVlanMapVO accountVlanMap : accountVlanMaps) {
         			VlanVO vlan = _vlanDao.findById(accountVlanMap.getVlanDbId());
-        			if (vlan.getDataCenterId() == zone.getId().longValue()) {
+        			if (vlan.getDataCenterId() == zone.getId()) {
         				throw new InvalidParameterValueException("Zone " + zone.getName() + " already has account-wide IP ranges. A zone may contain either pod-wide IP ranges or account-wide IP ranges, but not both.");
         			}
         		}
@@ -1435,12 +1470,12 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
 		if (account != null && vlanType.equals(VlanType.VirtualNetwork)){
 			if(!savePublicIPRangeForAccount(startIP, endIP, zoneId, vlan.getId(), account.getId(), account.getDomainId())) {
 				deletePublicIPRange(vlan.getId());
-				_vlanDao.delete(vlan.getId());
+				_vlanDao.expunge(vlan.getId());
 				throw new InternalErrorException("Failed to save IP range. Please contact Cloud Support."); //It can be Direct IP or Public IP.
 			}				
 		}else if (!savePublicIPRange(startIP, endIP, zoneId, vlan.getId())) {
 			deletePublicIPRange(vlan.getId());
-			_vlanDao.delete(vlan.getId());
+			_vlanDao.expunge(vlan.getId());
 			throw new InternalErrorException("Failed to save IP range. Please contact Cloud Support."); //It can be Direct IP or Public IP.
 		}
 		
@@ -1510,7 +1545,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
                 txn.start();
                 List ipAddrsList = new ArrayList();
                 for (VlanVO vlan : vlansForAccount){
-                    ipAddrsList.addAll(_publicIpAddressDao.assignAcccountSpecificIps(accountId, account.getDomainId().longValue(), vlan.getId(), false));
+                    ipAddrsList.addAll(_publicIpAddressDao.assignAcccountSpecificIps(accountId, account.getDomainId(), vlan.getId(), false));
 
                     long size = ipAddrsList.size();
                     _accountMgr.incrementResourceCount(accountId, ResourceType.public_ip, size);
@@ -1522,7 +1557,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
                 String params = "\nsourceNat=" + false + "\ndcId=" + zoneId;
 
                 // Associate the IP's to DomR
-                boolean success = _networkMgr.associateIP(router,ipAddrsList, true);
+                boolean success = _networkMgr.associateIP(router,ipAddrsList, true, 0);
                 String errorMsg = "Unable to assign public IP address pool";
                 if (!success) {
                     s_logger.debug(errorMsg);
@@ -1551,6 +1586,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
     
     }
     
+    @Override
     public boolean deleteVlanAndPublicIpRange(long userId, long vlanDbId) throws InvalidParameterValueException {
     	VlanVO vlan = _vlanDao.findById(vlanDbId);
     	if (vlan == null) {
@@ -1586,7 +1622,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
     	}
     	
 		// Delete the VLAN
-		boolean success = _vlanDao.delete(vlanDbId);
+		boolean success = _vlanDao.expunge(vlanDbId);
 		
 		if (success) {
 			String[] ipRange = vlan.getIpRange().split("\\-");
@@ -1607,6 +1643,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
 		return success;
     }
     
+    @Override
     public List csvTagsToList(String tags) {
     	List tagsList = new ArrayList();
     	
@@ -1620,6 +1657,7 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
     	return tagsList;
     }
     
+    @Override
     public String listToCsvTags(List tagsList) {
     	String tags = "";
     	if (tagsList.size() > 0) {
@@ -1906,9 +1944,9 @@ public class ConfigurationManagerImpl implements ConfigurationManager {
 		return cidrPair[0];
 	}
 	
-	private long getCidrSize(String cidr) {
+	private int getCidrSize(String cidr) {
 		String[] cidrPair = cidr.split("\\/");
-		return Long.parseLong(cidrPair[1]);
+		return Integer.parseInt(cidrPair[1]);
 	}
 	
 	private String getCidrAddress(long podId) {
diff --git a/server/src/com/cloud/consoleproxy/AgentBasedConsoleProxyManager.java b/server/src/com/cloud/consoleproxy/AgentBasedConsoleProxyManager.java
index 8f1383c3e74..3ba5f01d800 100644
--- a/server/src/com/cloud/consoleproxy/AgentBasedConsoleProxyManager.java
+++ b/server/src/com/cloud/consoleproxy/AgentBasedConsoleProxyManager.java
@@ -16,8 +16,8 @@
  * 
  */
 
-package com.cloud.consoleproxy;
-
+package com.cloud.consoleproxy;
+
 import java.util.Map;
 
 import javax.ejb.Local;
@@ -47,8 +47,10 @@ import com.cloud.ha.HighAvailabilityManager;
 import com.cloud.host.HostVO;
 import com.cloud.host.Status;
 import com.cloud.host.dao.HostDao;
+import com.cloud.info.ConsoleProxyInfo;
 import com.cloud.utils.NumbersUtil;
 import com.cloud.utils.component.ComponentLocator;
+import com.cloud.utils.component.Inject;
 import com.cloud.vm.ConsoleProxyVO;
 import com.cloud.vm.UserVmVO;
 import com.cloud.vm.VMInstanceVO;
@@ -57,311 +59,267 @@ import com.cloud.vm.VirtualMachineManager;
 import com.cloud.vm.VirtualMachineName;
 import com.cloud.vm.dao.UserVmDao;
 import com.cloud.vm.dao.VMInstanceDao;
-
-@Local(value={ConsoleProxyManager.class})
-public class AgentBasedConsoleProxyManager implements ConsoleProxyManager, VirtualMachineManager {
-	private static final Logger s_logger = Logger.getLogger(AgentBasedConsoleProxyManager.class);
-	
-    private String _name;
-	protected HostDao _hostDao;
-	protected UserVmDao _userVmDao;
-	private String _instance;
-	private VMInstanceDao _instanceDao;
-	private ConsoleProxyListener _listener;
-	
-	protected int _consoleProxyUrlPort = ConsoleProxyManager.DEFAULT_PROXY_URL_PORT;
-	protected boolean _sslEnabled = false;
-	AgentManager _agentMgr;
-	
-	public int getVncPort(VMInstanceVO vm) {
-        if (vm.getHostId() == null) {
-            return -1;
-        }
-        GetVncPortAnswer answer = (GetVncPortAnswer)_agentMgr.easySend(vm.getHostId(), new GetVncPortCommand(vm.getId(), vm.getName()));
-        return answer == null ? -1 : answer.getPort();
-    }
-	
-    @Override
-    public boolean configure(String name, Map params) throws ConfigurationException {
-
-    	if(s_logger.isInfoEnabled())
-    		s_logger.info("Start configuring AgentBasedConsoleProxyManager");
-    	
-        _name = name;
-        
-		ComponentLocator locator = ComponentLocator.getCurrentLocator();
-		ConfigurationDao configDao = locator.getDao(ConfigurationDao.class);
-		if (configDao == null) {
-			throw new ConfigurationException(
-					"Unable to get the configuration dao.");
-		}
-		
-		Map configs = configDao.getConfiguration(
-				"management-server", params);
-		String value = configs.get("consoleproxy.url.port");
-		if (value != null)
-			_consoleProxyUrlPort = NumbersUtil.parseInt(value, ConsoleProxyManager.DEFAULT_PROXY_URL_PORT);
-        
-		_hostDao = locator.getDao(HostDao.class);
-		if (_hostDao == null) {
-			throw new ConfigurationException("Unable to get "
-					+ HostDao.class.getName());
-		}
-		
-		_instanceDao = locator.getDao(VMInstanceDao.class);
-		if (_instanceDao == null)
-			throw new ConfigurationException("Unable to get " + VMInstanceDao.class.getName());
-		
-		_userVmDao = locator.getDao(UserVmDao.class);
-		if (_userVmDao == null)
-			throw new ConfigurationException("Unable to get " + UserVmDao.class.getName());
-		
-		_agentMgr = locator.getManager(AgentManager.class);
-		if (_agentMgr == null)
-            throw new ConfigurationException("Unable to get " + AgentManager.class.getName());
-		
-		value = configs.get("consoleproxy.sslEnabled");
-		if(value != null && value.equalsIgnoreCase("true"))
-			_sslEnabled = true;
-		
-		_instance = configs.get("instance.name");
-		
-		_listener = new ConsoleProxyListener(this);
-		_agentMgr.registerForHostEvents(_listener, true, true, false);
-		
-		HighAvailabilityManager haMgr = locator.getManager(HighAvailabilityManager.class);
-		haMgr.registerHandler(Type.ConsoleProxy, this);
-
-    	if(s_logger.isInfoEnabled())
-    		s_logger.info("AgentBasedConsoleProxyManager has been configured. SSL enabled: " + _sslEnabled);
-        return true;
-    }
-
-    @Override
-    public boolean start() {
-        return true;
-    }
-
-    @Override
-    public boolean stop() {
-        return true;
-    }
-
-    HostVO findHost(VMInstanceVO vm) {
-        return _hostDao.findById(vm.getHostId());
-    }
-    
-    protected ConsoleProxyVO allocateProxy(HostVO host, long dataCenterId) {
-        // only private IP, public IP, host id have meaningful values, rest of all are place-holder values
-        String publicIp = host.getPublicIpAddress();
-        if(publicIp == null) {
-            if(s_logger.isDebugEnabled())
-                s_logger.debug("Host " + host.getName() + "/" + host.getPrivateIpAddress() +
-                    " does not have public interface, we will return its private IP for cosole proxy.");
-            publicIp = host.getPrivateIpAddress();
-        }
+
+@Local(value = { ConsoleProxyManager.class })
+public class AgentBasedConsoleProxyManager implements ConsoleProxyManager, VirtualMachineManager, AgentHook {
+    private static final Logger s_logger = Logger.getLogger(AgentBasedConsoleProxyManager.class);
+
+    private String _name;
+    @Inject
+    protected HostDao _hostDao;
+    @Inject
+    protected UserVmDao _userVmDao;
+    private String _instance;
+    @Inject
+    private VMInstanceDao _instanceDao;
+    private ConsoleProxyListener _listener;
+
+    protected int _consoleProxyUrlPort = ConsoleProxyManager.DEFAULT_PROXY_URL_PORT;
+    protected int _consoleProxyPort = ConsoleProxyManager.DEFAULT_PROXY_VNC_PORT;
+    protected boolean _sslEnabled = false;
+    @Inject
+    AgentManager _agentMgr;
+
+    public int getVncPort(VMInstanceVO vm) {
+        if (vm.getHostId() == null) {
+            return -1;
+        }
+        GetVncPortAnswer answer = (GetVncPortAnswer) _agentMgr.easySend(vm.getHostId(), new GetVncPortCommand(vm.getId(), vm.getName()));
+        return answer == null ? -1 : answer.getPort();
+    }
+
+    @Override
+    public boolean configure(String name, Map params) throws ConfigurationException {
+
+        if (s_logger.isInfoEnabled())
+            s_logger.info("Start configuring AgentBasedConsoleProxyManager");
+
+        _name = name;
+
+        ComponentLocator locator = ComponentLocator.getCurrentLocator();
+        ConfigurationDao configDao = locator.getDao(ConfigurationDao.class);
+        if (configDao == null) {
+            throw new ConfigurationException("Unable to get the configuration dao.");
+        }
+
+        Map configs = configDao.getConfiguration("management-server", params);
+        String value = configs.get("consoleproxy.url.port");
+        if (value != null)
+            _consoleProxyUrlPort = NumbersUtil.parseInt(value, ConsoleProxyManager.DEFAULT_PROXY_URL_PORT);
         
-        // FIXME: Removed State.Running does this affect the console proxy?
-        return new ConsoleProxyVO(1l, "EmbeddedProxy", null, null, null,
-                "02:02:02:02:02:02",
-                host.getPrivateIpAddress(),
-                "255.255.255.0",
-                1l,
-                1l,
-                "03:03:03:03:03:03",
-                publicIp,
-                "255.255.255.0",
-                null,
-                "untagged",
-                1l,
-                dataCenterId,
-                "0.0.0.0",
-                host.getId(),
-                "dns1",
-                "dn2",
-                "domain",
-                0,
-                0);
-    }
-    
-    @Override
-    public ConsoleProxyVO assignProxy(long dataCenterId, long userVmId) {
-    	UserVmVO userVm = _userVmDao.findById(userVmId);
-		if (userVm == null) {
-			s_logger.warn("User VM " + userVmId
-					+ " no longer exists, return a null proxy for user vm:"
-					+ userVmId);
-			return null;
-		}
-		
-		HostVO host = findHost(userVm);
-		if(host != null) {
-			if(s_logger.isDebugEnabled())
-				s_logger.debug("Assign embedded console proxy running at " + host.getName() + " to user vm " + userVmId + " with public IP " + host.getPublicIpAddress());
-			
-			// only private IP, public IP, host id have meaningful values, rest of all are place-holder values
-			String publicIp = host.getPublicIpAddress();
-			if(publicIp == null) {
-				if(s_logger.isDebugEnabled())
-					s_logger.debug("Host " + host.getName() + "/" + host.getPrivateIpAddress() +
-						" does not have public interface, we will return its private IP for cosole proxy.");
-				publicIp = host.getPrivateIpAddress();
-			}
-			
-			ConsoleProxyVO proxy = allocateProxy(host, dataCenterId);
-			
-			if(host.getProxyPort() != null && host.getProxyPort().intValue() > 0)
-				proxy.setPort(host.getProxyPort().intValue());
-			else
-				proxy.setPort(_consoleProxyUrlPort);
-			
-			proxy.setSslEnabled(_sslEnabled);
-			return proxy;
-		} else {
-			s_logger.warn("Host that VM is running is no longer available, console access to VM " + userVmId + " will be temporarily unavailable.");
-		}
-    	return null;
-    }
-
-    @Override
-	public void onLoadReport(ConsoleProxyLoadReportCommand cmd) {
-    }
-
-    @Override
-	public AgentControlAnswer onConsoleAccessAuthentication(ConsoleAccessAuthenticationCommand cmd) {
-    	long vmId = 0;
-		
-		if(cmd.getVmId() != null && cmd.getVmId().isEmpty()) {
-			if(s_logger.isTraceEnabled())
-				s_logger.trace("Invalid vm id sent from proxy(happens when proxy session has terminated)");
-			return new ConsoleAccessAuthenticationAnswer(cmd, false);
-		}
-		
-		try {
-			vmId = Long.parseLong(cmd.getVmId());
-		} catch(NumberFormatException e) {
-			s_logger.error("Invalid vm id " + cmd.getVmId() + " sent from console access authentication", e);
-			return new ConsoleAccessAuthenticationAnswer(cmd, false);
-		}
-		
-		// TODO authentication channel between console proxy VM and management server needs to be secured,
-		// the data is now being sent through private network, but this is apparently not enough
-		VMInstanceVO vm = _instanceDao.findById(vmId);
-		if(vm == null) {
-			return new ConsoleAccessAuthenticationAnswer(cmd, false);
-		}
-		
-		if(vm.getHostId() == null) {
-			s_logger.warn("VM " + vmId + " lost host info, failed authentication request");
-			return new ConsoleAccessAuthenticationAnswer(cmd, false);
-		}
-		
-		HostVO host = _hostDao.findById(vm.getHostId());
-		if(host == null) {
-			s_logger.warn("VM " + vmId + "'s host does not exist, fail authentication request");
-			return new ConsoleAccessAuthenticationAnswer(cmd, false);
-		}
-		
-		String sid = cmd.getSid();
-		if(sid == null || !sid.equals(vm.getVncPassword())) {
-			s_logger.warn("sid " + sid + " in url does not match stored sid " + vm.getVncPassword());
-			return new ConsoleAccessAuthenticationAnswer(cmd, false);
-		}
-		
-		return new ConsoleAccessAuthenticationAnswer(cmd, true);
+        value = configs.get("consoleproxy.port");
+        if (value != null)
+            _consoleProxyPort = NumbersUtil.parseInt(value, ConsoleProxyManager.DEFAULT_PROXY_VNC_PORT);
+
+        value = configs.get("consoleproxy.sslEnabled");
+        if (value != null && value.equalsIgnoreCase("true"))
+            _sslEnabled = true;
+
+        _instance = configs.get("instance.name");
+
+        _listener = new ConsoleProxyListener(this);
+        _agentMgr.registerForHostEvents(_listener, true, true, false);
+
+        HighAvailabilityManager haMgr = locator.getManager(HighAvailabilityManager.class);
+        haMgr.registerHandler(Type.ConsoleProxy, this);
+
+        if (s_logger.isInfoEnabled())
+            s_logger.info("AgentBasedConsoleProxyManager has been configured. SSL enabled: " + _sslEnabled);
+        return true;
+    }
+
+    @Override
+    public boolean start() {
+        return true;
+    }
+
+    @Override
+    public boolean stop() {
+        return true;
+    }
+
+    HostVO findHost(VMInstanceVO vm) {
+        return _hostDao.findById(vm.getHostId());
+    }
+
+    @Override
+    public ConsoleProxyInfo assignProxy(long dataCenterId, long userVmId) {
+        UserVmVO userVm = _userVmDao.findById(userVmId);
+        if (userVm == null) {
+            s_logger.warn("User VM " + userVmId + " no longer exists, return a null proxy for user vm:" + userVmId);
+            return null;
+        }
+
+        HostVO host = findHost(userVm);
+        if (host != null) {
+            if (s_logger.isDebugEnabled())
+                s_logger.debug("Assign embedded console proxy running at " + host.getName() + " to user vm " + userVmId + " with public IP "
+                        + host.getPublicIpAddress());
+
+            // only private IP, public IP, host id have meaningful values, rest
+            // of all are place-holder values
+            String publicIp = host.getPublicIpAddress();
+            if (publicIp == null) {
+                if (s_logger.isDebugEnabled())
+                    s_logger.debug("Host " + host.getName() + "/" + host.getPrivateIpAddress()
+                            + " does not have public interface, we will return its private IP for cosole proxy.");
+                publicIp = host.getPrivateIpAddress();
+            }
+            
+            int urlPort = _consoleProxyUrlPort;
+
+            if (host.getProxyPort() != null && host.getProxyPort().intValue() > 0)
+                urlPort = host.getProxyPort().intValue();
+            
+            return new ConsoleProxyInfo(_sslEnabled, publicIp, _consoleProxyPort, urlPort);
+        } else {
+            s_logger.warn("Host that VM is running is no longer available, console access to VM " + userVmId + " will be temporarily unavailable.");
+        }
+        return null;
+    }
+
+    @Override
+    public void onLoadReport(ConsoleProxyLoadReportCommand cmd) {
+    }
+
+    @Override
+    public AgentControlAnswer onConsoleAccessAuthentication(ConsoleAccessAuthenticationCommand cmd) {
+        long vmId = 0;
+
+        if (cmd.getVmId() != null && cmd.getVmId().isEmpty()) {
+            if (s_logger.isTraceEnabled())
+                s_logger.trace("Invalid vm id sent from proxy(happens when proxy session has terminated)");
+            return new ConsoleAccessAuthenticationAnswer(cmd, false);
+        }
+
+        try {
+            vmId = Long.parseLong(cmd.getVmId());
+        } catch (NumberFormatException e) {
+            s_logger.error("Invalid vm id " + cmd.getVmId() + " sent from console access authentication", e);
+            return new ConsoleAccessAuthenticationAnswer(cmd, false);
+        }
+
+        // TODO authentication channel between console proxy VM and management
+        // server needs to be secured,
+        // the data is now being sent through private network, but this is
+        // apparently not enough
+        VMInstanceVO vm = _instanceDao.findById(vmId);
+        if (vm == null) {
+            return new ConsoleAccessAuthenticationAnswer(cmd, false);
+        }
+
+        if (vm.getHostId() == null) {
+            s_logger.warn("VM " + vmId + " lost host info, failed authentication request");
+            return new ConsoleAccessAuthenticationAnswer(cmd, false);
+        }
+
+        HostVO host = _hostDao.findById(vm.getHostId());
+        if (host == null) {
+            s_logger.warn("VM " + vmId + "'s host does not exist, fail authentication request");
+            return new ConsoleAccessAuthenticationAnswer(cmd, false);
+        }
+
+        String sid = cmd.getSid();
+        if (sid == null || !sid.equals(vm.getVncPassword())) {
+            s_logger.warn("sid " + sid + " in url does not match stored sid " + vm.getVncPassword());
+            return new ConsoleAccessAuthenticationAnswer(cmd, false);
+        }
+
+        return new ConsoleAccessAuthenticationAnswer(cmd, true);
     }
 
     @Override
     public void onAgentConnect(HostVO host, StartupCommand cmd) {
     }
-    
+
     @Override
-	public void onAgentDisconnect(long agentId, Status state) {
+    public void onAgentDisconnect(long agentId, Status state) {
     }
-
-    @Override
-    public ConsoleProxyVO startProxy(long proxyVmId, long startEventId) {
-        return null;
+
+    @Override
+    public ConsoleProxyVO startProxy(long proxyVmId, long startEventId) {
+        return null;
+    }
+
+    @Override
+    public boolean destroyProxy(long proxyVmId, long startEventId) {
+        return false;
+    }
+
+    @Override
+    public boolean rebootProxy(long proxyVmId, long startEventId) {
+        return false;
+    }
+
+    @Override
+    public boolean stopProxy(long proxyVmId, long startEventId) {
+        return false;
+    }
+
+    @Override
+    public String getName() {
+        return _name;
+    }
+
+    @Override
+    public Command cleanup(ConsoleProxyVO vm, String vmName) {
+        return new StopCommand(vm, vmName, null);
+    }
+
+    @Override
+    public boolean completeMigration(ConsoleProxyVO vm, HostVO host) throws AgentUnavailableException, OperationTimedoutException {
+        return false;
+    }
+
+    @Override
+    public void completeStartCommand(ConsoleProxyVO vm) {
+    }
+
+    @Override
+    public void completeStopCommand(ConsoleProxyVO vm) {
+    }
+
+    @Override
+    public Long convertToId(String vmName) {
+        if (!VirtualMachineName.isValidConsoleProxyName(vmName, _instance)) {
+            return null;
+        }
+        return VirtualMachineName.getConsoleProxyId(vmName);
+    }
+
+    @Override
+    public boolean destroy(ConsoleProxyVO vm) throws AgentUnavailableException {
+        return false;
+    }
+
+    @Override
+    public ConsoleProxyVO get(long id) {
+        return null;
+    }
+
+    @Override
+    public boolean migrate(ConsoleProxyVO vm, HostVO host) throws AgentUnavailableException, OperationTimedoutException {
+        return false;
+    }
+
+    @Override
+    public HostVO prepareForMigration(ConsoleProxyVO vm) throws InsufficientCapacityException, StorageUnavailableException {
+        return null;
+    }
+
+    @Override
+    public ConsoleProxyVO start(long vmId, long startEventId) throws InsufficientCapacityException, StorageUnavailableException,
+            ConcurrentOperationException {
+        return null;
+    }
+
+    @Override
+    public boolean stop(ConsoleProxyVO vm, long startEventId) throws AgentUnavailableException {
+        return false;
     }
 
     @Override
     public boolean destroyConsoleProxy(DestroyConsoleProxyCmd cmd) throws ServerApiException {
         return false;
-    }
-
-	@Override
-	public boolean destroyProxy(long proxyVmId, long startEventId) {
-		return false;
-	}
-
-	@Override
-	public boolean rebootProxy(long proxyVmId, long startEventId) {
-		return false;
-	}
-
-	@Override
-	public boolean stopProxy(long proxyVmId, long startEventId) {
-		return false;
-	}
-
-	@Override
-	public String getName() {
-		return _name;
-	}
-
-    @Override
-    public Command cleanup(ConsoleProxyVO vm, String vmName) {
-        return new StopCommand(vm, vmName, null);
-    }
-
-    @Override
-    public boolean completeMigration(ConsoleProxyVO vm, HostVO host) throws AgentUnavailableException, OperationTimedoutException {
-        return false;
-    }
-
-    @Override
-    public void completeStartCommand(ConsoleProxyVO vm) {
-    }
-
-    @Override
-    public void completeStopCommand(ConsoleProxyVO vm) {
-    }
-
-    @Override
-    public Long convertToId(String vmName) {
-        if (!VirtualMachineName.isValidConsoleProxyName(vmName, _instance)) {
-            return null;
-        }
-        return VirtualMachineName.getConsoleProxyId(vmName);
-    }
-
-    @Override
-    public boolean destroy(ConsoleProxyVO vm) throws AgentUnavailableException {
-        return false;
-    }
-
-    @Override
-    public ConsoleProxyVO get(long id) {
-        return null;
-    }
-
-    @Override
-    public boolean migrate(ConsoleProxyVO vm, HostVO host) throws AgentUnavailableException, OperationTimedoutException {
-        return false;
-    }
-
-    @Override
-    public HostVO prepareForMigration(ConsoleProxyVO vm) throws InsufficientCapacityException, StorageUnavailableException {
-        return null;
-    }
-
-    @Override
-    public ConsoleProxyVO start(long vmId, long startEventId) throws InsufficientCapacityException, StorageUnavailableException, ConcurrentOperationException {
-        return null;
-    }
-
-    @Override
-    public boolean stop(ConsoleProxyVO vm, long startEventId) throws AgentUnavailableException {
-        return false;
     }
 }
diff --git a/server/src/com/cloud/consoleproxy/AgentBasedStandaloneConsoleProxyManager.java b/server/src/com/cloud/consoleproxy/AgentBasedStandaloneConsoleProxyManager.java
index 4b5f2fb4d4c..5646bb16f18 100644
--- a/server/src/com/cloud/consoleproxy/AgentBasedStandaloneConsoleProxyManager.java
+++ b/server/src/com/cloud/consoleproxy/AgentBasedStandaloneConsoleProxyManager.java
@@ -26,7 +26,7 @@ import org.apache.log4j.Logger;
 
 import com.cloud.host.Host;
 import com.cloud.host.HostVO;
-import com.cloud.vm.ConsoleProxyVO;
+import com.cloud.info.ConsoleProxyInfo;
 import com.cloud.vm.UserVmVO;
 
 @Local(value={ConsoleProxyManager.class})
@@ -35,7 +35,7 @@ AgentBasedConsoleProxyManager {
 	private static final Logger s_logger = Logger.getLogger(AgentBasedStandaloneConsoleProxyManager.class);
 
 	@Override
-	public ConsoleProxyVO assignProxy(long dataCenterId, long userVmId) {
+	public ConsoleProxyInfo assignProxy(long dataCenterId, long userVmId) {
 		UserVmVO userVm = _userVmDao.findById(userVmId);
 		if (userVm == null) {
 			s_logger.warn("User VM " + userVmId
@@ -48,7 +48,7 @@ AgentBasedConsoleProxyManager {
 		if(host != null) {
 			HostVO allocatedHost = null;
 			/*Is there a consoleproxy agent running on the same machine?*/
-			List hosts = _hostDao.listAll();
+			List hosts = _hostDao.listAllIncludingRemoved();
 			for (HostVO hv : hosts) {
 				if (hv.getType() == Host.Type.ConsoleProxy && hv.getPublicIpAddress().equalsIgnoreCase(host.getPublicIpAddress())) {
 					allocatedHost = hv;
@@ -81,15 +81,11 @@ AgentBasedConsoleProxyManager {
 				publicIp = allocatedHost.getPrivateIpAddress();
 			}
 
-			ConsoleProxyVO proxy = allocateProxy(allocatedHost, dataCenterId);
-
-			if(allocatedHost.getProxyPort() != null && allocatedHost.getProxyPort().intValue() > 0)
-				proxy.setPort(allocatedHost.getProxyPort().intValue());
-			else
-				proxy.setPort(_consoleProxyUrlPort);
-
-			proxy.setSslEnabled(_sslEnabled);
-			return proxy;
+			int urlPort = _consoleProxyUrlPort;
+            if(allocatedHost.getProxyPort() != null && allocatedHost.getProxyPort().intValue() > 0)
+                urlPort = allocatedHost.getProxyPort().intValue();
+            
+            return new ConsoleProxyInfo(_sslEnabled, publicIp, _consoleProxyPort, urlPort);
 		} else {
 			s_logger.warn("Host that VM is running is no longer available, console access to VM " + userVmId + " will be temporarily unavailable.");
 		}
diff --git a/server/src/com/cloud/consoleproxy/AgentHook.java b/server/src/com/cloud/consoleproxy/AgentHook.java
new file mode 100644
index 00000000000..a25e555e92f
--- /dev/null
+++ b/server/src/com/cloud/consoleproxy/AgentHook.java
@@ -0,0 +1,19 @@
+/**
+ * 
+ */
+package com.cloud.consoleproxy;
+
+import com.cloud.agent.api.AgentControlAnswer;
+import com.cloud.agent.api.ConsoleAccessAuthenticationCommand;
+import com.cloud.agent.api.ConsoleProxyLoadReportCommand;
+import com.cloud.agent.api.StartupCommand;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+
+public interface AgentHook {
+    void onLoadReport(ConsoleProxyLoadReportCommand cmd);
+    AgentControlAnswer onConsoleAccessAuthentication(ConsoleAccessAuthenticationCommand cmd);
+    void onAgentConnect(HostVO host, StartupCommand cmd);
+    
+    public void onAgentDisconnect(long agentId, Status state);
+}
diff --git a/server/src/com/cloud/consoleproxy/ConsoleProxyListener.java b/server/src/com/cloud/consoleproxy/ConsoleProxyListener.java
index def13ab2209..43f06cc5c73 100644
--- a/server/src/com/cloud/consoleproxy/ConsoleProxyListener.java
+++ b/server/src/com/cloud/consoleproxy/ConsoleProxyListener.java
@@ -29,12 +29,12 @@ import com.cloud.host.HostVO;
 import com.cloud.host.Status;
 
 public class ConsoleProxyListener implements Listener {
-    ConsoleProxyManager _proxyMgr = null;
+    AgentHook _proxyMgr = null;
 
-    public ConsoleProxyListener(ConsoleProxyManager proxyMgr) {
+    public ConsoleProxyListener(AgentHook proxyMgr) {
         _proxyMgr = proxyMgr;
     }
-    
+
     @Override
     public boolean isRecurring() {
         return true;
@@ -42,46 +42,46 @@ public class ConsoleProxyListener implements Listener {
 
     @Override
     public boolean processAnswer(long agentId, long seq, Answer[] answers) {
-    	return true;
+        return true;
     }
 
     @Override
     public boolean processCommand(long agentId, long seq, Command[] commands) {
         return false;
     }
-    
+
     @Override
     public AgentControlAnswer processControlCommand(long agentId, AgentControlCommand cmd) {
-    	if(cmd instanceof ConsoleProxyLoadReportCommand) {
-    		_proxyMgr.onLoadReport((ConsoleProxyLoadReportCommand)cmd);
-    		
-    		// return dummy answer
-    		return new AgentControlAnswer(cmd);
-    	} else if(cmd instanceof ConsoleAccessAuthenticationCommand) {
-    		return _proxyMgr.onConsoleAccessAuthentication((ConsoleAccessAuthenticationCommand)cmd);
-    	}
-    	return null;
+        if (cmd instanceof ConsoleProxyLoadReportCommand) {
+            _proxyMgr.onLoadReport((ConsoleProxyLoadReportCommand) cmd);
+
+            // return dummy answer
+            return new AgentControlAnswer(cmd);
+        } else if (cmd instanceof ConsoleAccessAuthenticationCommand) {
+            return _proxyMgr.onConsoleAccessAuthentication((ConsoleAccessAuthenticationCommand) cmd);
+        }
+        return null;
     }
 
     @Override
     public boolean processConnect(HostVO host, StartupCommand cmd) {
-    	_proxyMgr.onAgentConnect(host, cmd);
+        _proxyMgr.onAgentConnect(host, cmd);
         return true;
     }
-    
+
     @Override
     public boolean processDisconnect(long agentId, Status state) {
-    	_proxyMgr.onAgentDisconnect(agentId, state);
+        _proxyMgr.onAgentDisconnect(agentId, state);
         return true;
     }
-    
+
     @Override
     public boolean processTimeout(long agentId, long seq) {
-    	return true;
+        return true;
     }
-    
+
     @Override
     public int getTimeout() {
-    	return -1;
+        return -1;
     }
 }
diff --git a/server/src/com/cloud/consoleproxy/ConsoleProxyManager.java b/server/src/com/cloud/consoleproxy/ConsoleProxyManager.java
index 2d531e543e3..b9e22488e58 100644
--- a/server/src/com/cloud/consoleproxy/ConsoleProxyManager.java
+++ b/server/src/com/cloud/consoleproxy/ConsoleProxyManager.java
@@ -26,6 +26,7 @@ import com.cloud.api.ServerApiException;
 import com.cloud.api.commands.DestroyConsoleProxyCmd;
 import com.cloud.host.HostVO;
 import com.cloud.host.Status;
+import com.cloud.info.ConsoleProxyInfo;
 import com.cloud.utils.component.Manager;
 import com.cloud.vm.ConsoleProxyVO;
 public interface ConsoleProxyManager extends Manager {
@@ -41,7 +42,7 @@ public interface ConsoleProxyManager extends Manager {
 	
 	public static final String ALERT_SUBJECT = "proxy-alert";
 	
-	public ConsoleProxyVO assignProxy(long dataCenterId, long userVmId);
+	public ConsoleProxyInfo assignProxy(long dataCenterId, long userVmId);
 	
 	public ConsoleProxyVO startProxy(long proxyVmId, long startEventId);
 	public boolean stopProxy(long proxyVmId, long startEventId);
diff --git a/server/src/com/cloud/consoleproxy/ConsoleProxyManagerImpl.java b/server/src/com/cloud/consoleproxy/ConsoleProxyManagerImpl.java
index 63ce5568fd7..b6b4f997c32 100644
--- a/server/src/com/cloud/consoleproxy/ConsoleProxyManagerImpl.java
+++ b/server/src/com/cloud/consoleproxy/ConsoleProxyManagerImpl.java
@@ -22,6 +22,7 @@ import java.io.IOException;
 import java.net.InetSocketAddress;
 import java.nio.channels.SocketChannel;
 import java.nio.charset.Charset;
+import java.util.ArrayList;
 import java.util.Date;
 import java.util.Enumeration;
 import java.util.HashMap;
@@ -64,7 +65,6 @@ import com.cloud.async.AsyncJobExecutor;
 import com.cloud.async.AsyncJobManager;
 import com.cloud.async.AsyncJobVO;
 import com.cloud.async.BaseAsyncJobExecutor;
-import com.cloud.capacity.dao.CapacityDao;
 import com.cloud.cluster.ClusterManager;
 import com.cloud.configuration.Config;
 import com.cloud.configuration.dao.ConfigurationDao;
@@ -75,6 +75,8 @@ import com.cloud.dc.VlanVO;
 import com.cloud.dc.dao.DataCenterDao;
 import com.cloud.dc.dao.HostPodDao;
 import com.cloud.dc.dao.VlanDao;
+import com.cloud.deploy.DataCenterDeployment;
+import com.cloud.deploy.DeploymentPlan;
 import com.cloud.domain.DomainVO;
 import com.cloud.event.EventState;
 import com.cloud.event.EventTypes;
@@ -87,12 +89,12 @@ import com.cloud.exception.InsufficientCapacityException;
 import com.cloud.exception.OperationTimedoutException;
 import com.cloud.exception.StorageUnavailableException;
 import com.cloud.ha.HighAvailabilityManager;
-import com.cloud.ha.dao.HighAvailabilityDao;
 import com.cloud.host.Host;
 import com.cloud.host.Host.Type;
 import com.cloud.host.HostVO;
 import com.cloud.host.dao.HostDao;
 import com.cloud.info.ConsoleProxyConnectionInfo;
+import com.cloud.info.ConsoleProxyInfo;
 import com.cloud.info.ConsoleProxyLoadInfo;
 import com.cloud.info.ConsoleProxyStatus;
 import com.cloud.info.RunningHostCountInfo;
@@ -101,9 +103,12 @@ import com.cloud.info.RunningHostInfoAgregator.ZoneHostInfo;
 import com.cloud.maid.StackMaid;
 import com.cloud.network.IpAddrAllocator;
 import com.cloud.network.IpAddrAllocator.networkInfo;
+import com.cloud.network.NetworkConfigurationVO;
 import com.cloud.network.NetworkManager;
 import com.cloud.network.dao.IPAddressDao;
-import com.cloud.offering.ServiceOffering.GuestIpType;
+import com.cloud.offering.NetworkOffering;
+import com.cloud.offerings.NetworkOfferingVO;
+import com.cloud.offerings.dao.NetworkOfferingDao;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.service.dao.ServiceOfferingDao;
 import com.cloud.storage.StorageManager;
@@ -112,11 +117,11 @@ import com.cloud.storage.VMTemplateHostVO;
 import com.cloud.storage.VMTemplateStorageResourceAssoc.Status;
 import com.cloud.storage.VMTemplateVO;
 import com.cloud.storage.VolumeVO;
-import com.cloud.storage.dao.StoragePoolDao;
 import com.cloud.storage.dao.VMTemplateDao;
 import com.cloud.storage.dao.VMTemplateHostDao;
 import com.cloud.storage.dao.VolumeDao;
 import com.cloud.user.Account;
+import com.cloud.user.AccountManager;
 import com.cloud.user.AccountVO;
 import com.cloud.user.User;
 import com.cloud.user.dao.AccountDao;
@@ -141,6 +146,8 @@ import com.cloud.vm.VirtualMachine;
 import com.cloud.vm.VirtualMachine.Event;
 import com.cloud.vm.VirtualMachineManager;
 import com.cloud.vm.VirtualMachineName;
+import com.cloud.vm.VirtualMachineProfile;
+import com.cloud.vm.VmManager;
 import com.cloud.vm.dao.ConsoleProxyDao;
 import com.cloud.vm.dao.VMInstanceDao;
 import com.google.gson.Gson;
@@ -165,2218 +172,2167 @@ import com.google.gson.GsonBuilder;
 // because sooner or later, it will be driven into Running state
 //
 @Local(value = { ConsoleProxyManager.class })
-public class ConsoleProxyManagerImpl implements ConsoleProxyManager,
-		VirtualMachineManager {
-	private static final Logger s_logger = Logger
-			.getLogger(ConsoleProxyManagerImpl.class);
-
-	private static final int DEFAULT_FIND_HOST_RETRY_COUNT = 2;
-	private static final int DEFAULT_CAPACITY_SCAN_INTERVAL = 30000; // 30
-	// seconds
-	private static final int EXECUTOR_SHUTDOWN_TIMEOUT = 1000; // 1 second
-
-	private static final int ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_COOPERATION = 3; // 3
-	// seconds
-	private static final int ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC = 180; // 3
-	// minutes
-
-	private static final int API_WAIT_TIMEOUT = 5000; // 5 seconds (in
-	// milliseconds)
-	private static final int STARTUP_DELAY = 60000; // 60 seconds
-
-	private int _consoleProxyPort = ConsoleProxyManager.DEFAULT_PROXY_VNC_PORT;
-	private int _consoleProxyUrlPort = ConsoleProxyManager.DEFAULT_PROXY_URL_PORT;
-
-	private String _mgmt_host;
-	private int _mgmt_port = 8250;
-
-	private String _name;
-	private Adapters _consoleProxyAllocators;
-
-	private ConsoleProxyDao _consoleProxyDao;
-	private DataCenterDao _dcDao;
-	private VlanDao _vlanDao;
-	private VMTemplateDao _templateDao;
-	private IPAddressDao _ipAddressDao;
-	private VolumeDao _volsDao;
-	private HostPodDao _podDao;
-	private HostDao _hostDao;
-	private StoragePoolDao _storagePoolDao;
-	private ConfigurationDao _configDao;
-	
-	private VMInstanceDao _instanceDao;
-	private AccountDao _accountDao;
-
-	private VMTemplateHostDao _vmTemplateHostDao;
-	private CapacityDao _capacityDao;
-	private HighAvailabilityDao _haDao;
-
-	private AgentManager _agentMgr;
-	private NetworkManager _networkMgr;
-	private StorageManager _storageMgr;
-	private HighAvailabilityManager _haMgr;
-	private EventDao _eventDao;
-	@Inject
-	ServiceOfferingDao _offeringDao;
-	private IpAddrAllocator _IpAllocator;
-
-	private ConsoleProxyListener _listener;
-
-	private ServiceOfferingVO _serviceOffering;
-	private VMTemplateVO _template;
-
-	private AsyncJobManager _asyncMgr;
-
-	private final ScheduledExecutorService _capacityScanScheduler = Executors
-			.newScheduledThreadPool(1, new NamedThreadFactory("CP-Scan"));
-	private final ExecutorService _requestHandlerScheduler = Executors
-			.newCachedThreadPool(new NamedThreadFactory("Request-handler"));
-
-	private long _capacityScanInterval = DEFAULT_CAPACITY_SCAN_INTERVAL;
-	private int _capacityPerProxy = ConsoleProxyManager.DEFAULT_PROXY_CAPACITY;
-	private int _standbyCapacity = ConsoleProxyManager.DEFAULT_STANDBY_CAPACITY;
-
-	private int _proxyRamSize;
-	private int _find_host_retry = DEFAULT_FIND_HOST_RETRY_COUNT;
-	private int _ssh_retry;
-	private int _ssh_sleep;
-	private boolean _use_lvm;
-	private boolean _use_storage_vm;
-
-	private String _domain;
-	private String _instance;
-
-	// private String _privateNetmask;
-	private int _proxyCmdPort = DEFAULT_PROXY_CMD_PORT;
-	private int _proxySessionTimeoutValue = DEFAULT_PROXY_SESSION_TIMEOUT;
-	private boolean _sslEnabled = false;
-
-	private final GlobalLock _capacityScanLock = GlobalLock
-			.getInternLock(getCapacityScanLockName());
-	private final GlobalLock _allocProxyLock = GlobalLock
-			.getInternLock(getAllocProxyLockName());
-
-	public ConsoleProxyVO assignProxy(final long dataCenterId, final long vmId) {
-
-		final Pair result = new Pair(
-				this, null);
-
-		_requestHandlerScheduler.execute(new Runnable() {
-			public void run() {
-				Transaction txn = Transaction.open(Transaction.CLOUD_DB);
-				try {
-					ConsoleProxyVO proxy = doAssignProxy(dataCenterId, vmId);
-					synchronized (result) {
-						result.second(proxy);
-						result.notifyAll();
-					}
-				} catch (Throwable e) {
-					s_logger.warn("Unexpected exception " + e.getMessage(), e);
-				} finally {
-					StackMaid.current().exitCleanup();
-					txn.close();
-				}
-			}
-		});
-
-		synchronized (result) {
-			try {
-				result.wait(API_WAIT_TIMEOUT);
-			} catch (InterruptedException e) {
-				s_logger.info("Waiting for console proxy assignment is interrupted");
-			}
-		}
-		return result.second();
-	}
-
-	public ConsoleProxyVO doAssignProxy(long dataCenterId, long vmId) {
-		ConsoleProxyVO proxy = null;
-		VMInstanceVO vm = _instanceDao.findById(vmId);
-		if (vm == null) {
-			s_logger.warn("VM " + vmId + " no longer exists, return a null proxy for vm:" + vmId);
-			return null;
-		}
-
-		Boolean[] proxyFromStoppedPool = new Boolean[1];
-		if (_allocProxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
-			try {
-				proxy = getOrAllocProxyResource(dataCenterId, vmId, proxyFromStoppedPool);
-			} finally {
-				_allocProxyLock.unlock();
-			}
-		} else {
-			s_logger.error("Unable to acquire synchronization lock to get/allocate proxy resource for vm :"
-				+ vmId + ". Previous console proxy allocation is taking too long");
-		}
-
-		if (proxy == null) {
-			s_logger.warn("Unable to find or allocate console proxy resource");
-			return null;
-		}
-
-		long proxyVmId = proxy.getId();
-		GlobalLock proxyLock = GlobalLock.getInternLock(getProxyLockName(proxyVmId));
-		try {
-			if (proxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
-				try {
-					proxy = startProxy(proxyVmId, 0);
-					
-					if (proxy == null) {
-						//
-						// We had a situation with multi-pod configuration, where
-						// storage allocation of the console proxy VM may succeed, but later-on starting of it
-						// may fail because of running out of computing resource (CPU/memory). We
-						// currently don't support moving storage to another pod on the fly, to deal
-						// with the situation we will destroy this proxy VM and let it the whole proxy VM
-						// creation process re-start again, by hoping that new storage and computing
-						// resource may be allocated and assigned in another pod
-						//
-						if (s_logger.isInfoEnabled())
-							s_logger.info("Unable to start console proxy, proxy vm Id : " + proxyVmId + " will recycle it and restart a new one");
-						destroyProxy(proxyVmId, 0);
-						return null;
-					} else {
-						if (s_logger.isTraceEnabled())
-							s_logger.trace("Console proxy " + proxy.getName() + " is started");
-
-						// if it is a new assignment or a changed assignment, update the
-						// record
-						if (vm.getProxyId() == null || vm.getProxyId().longValue() != proxy.getId())
-							_instanceDao.updateProxyId(vmId, proxy.getId(), DateUtil.currentGMTTime());
-
-						proxy.setSslEnabled(_sslEnabled);
-						if (_sslEnabled)
-							proxy.setPort(443);
-						else
-							proxy.setPort(80);
-						return proxy;
-					}
-				} finally {
-					proxyLock.unlock();
-				}
-			} else {
-				s_logger.error("Unable to acquire synchronization lock to start console proxy "
-					+ proxyVmId + " for vm: " + vmId + ". It takes too long to start the proxy");
-				
-				return null;
-			}
-		} finally {
-			proxyLock.releaseRef();
-		}
-	}
-
-	private ConsoleProxyVO getOrAllocProxyResource(long dataCenterId,
-			long vmId, Boolean[] proxyFromStoppedPool) {
-		ConsoleProxyVO proxy = null;
-		VMInstanceVO vm = this._instanceDao.findById(vmId);
-
-		if (vm != null && vm.getState() != State.Running) {
-			if (s_logger.isInfoEnabled())
-				s_logger.info("Detected that vm : " + vmId + " is not currently at running state, we will fail the proxy assignment for it");
-			return null;
-		}
-
-		if (vm != null && vm.getProxyId() != null) {
-			proxy = _consoleProxyDao.findById(vm.getProxyId());
-
-			if (proxy != null) {
-				if (!isInAssignableState(proxy)) {
-					if (s_logger.isInfoEnabled())
-						s_logger.info("A previous assigned proxy is not assignable now, reassign console proxy for user vm : " + vmId);
-					proxy = null;
-				} else {
-					if (_consoleProxyDao.getProxyActiveLoad(proxy.getId()) < _capacityPerProxy
-							|| hasPreviousSession(proxy, vm)) {
-						if (s_logger.isTraceEnabled())
-							s_logger.trace("Assign previous allocated console proxy for user vm : " + vmId);
-
-						if (proxy.getActiveSession() >= _capacityPerProxy)
-							s_logger.warn("Assign overloaded proxy to user VM as previous session exists, user vm : " + vmId);
-					} else {
-						proxy = null;
-					}
-				}
-			}
-		}
-
-		if (proxy == null)
-			proxy = assignProxyFromRunningPool(dataCenterId);
-
-		if (proxy == null) {
-			if (s_logger.isInfoEnabled())
-				s_logger.info("No running console proxy is available, check to see if we can bring up a stopped one for data center : " + dataCenterId);
-
-			proxy = assignProxyFromStoppedPool(dataCenterId);
-			if (proxy == null) {
-				if (s_logger.isInfoEnabled())
-					s_logger.info("No stopped console proxy is available, need to allocate a new console proxy for data center : " + dataCenterId);
-
-				proxy = startNew(dataCenterId);
-			} else {
-				if (s_logger.isInfoEnabled())
-					s_logger.info("Found a stopped console proxy, bring it up to running pool. proxy vm id : "
-							+ proxy.getId() + ", data center : " + dataCenterId);
-
-				proxyFromStoppedPool[0] = new Boolean(true);
-			}
-		}
-
-		return proxy;
-	}
-
-	private static boolean isInAssignableState(ConsoleProxyVO proxy) {
-		// console proxies that are in states of being able to serve user VM
-		State state = proxy.getState();
-		if (state == State.Running || state == State.Starting
-				|| state == State.Creating || state == State.Migrating)
-			return true;
-
-		return false;
-	}
-
-	private boolean hasPreviousSession(ConsoleProxyVO proxy, VMInstanceVO vm) {
-
-		ConsoleProxyStatus status = null;
-		try {
-			GsonBuilder gb = new GsonBuilder();
-			gb.setVersion(1.3);
-			Gson gson = gb.create();
-
-			byte[] details = proxy.getSessionDetails();
-			status = gson.fromJson(details != null ? new String(details,
-					Charset.forName("US-ASCII")) : null,
-					ConsoleProxyStatus.class);
-		} catch (Throwable e) {
-			s_logger.warn("Unable to parse proxy session details : "
-				+ proxy.getSessionDetails());
-		}
-
-		if (status != null && status.getConnections() != null) {
-			ConsoleProxyConnectionInfo[] connections = status.getConnections();
-			for (int i = 0; i < connections.length; i++) {
-				long taggedVmId = 0;
-				if (connections[i].tag != null) {
-					try {
-						taggedVmId = Long.parseLong(connections[i].tag);
-					} catch (NumberFormatException e) {
-						s_logger.warn(
-								"Unable to parse console proxy connection info passed through tag: "
-										+ connections[i].tag, e);
-					}
-				}
-				if (taggedVmId == vm.getId())
-					return true;
-			}
-
-			//
-			// even if we are not in the list, it may because we haven't
-			// received load-update yet
-			// wait until session time
-			//
-			if (DateUtil.currentGMTTime().getTime() - vm.getProxyAssignTime().getTime() < _proxySessionTimeoutValue)
-				return true;
-
-			return false;
-		} else {
-			s_logger.error("No proxy load info on an overloaded proxy ?");
-			return false;
-		}
-	}
-
-	@Override
-	public ConsoleProxyVO startProxy(long proxyVmId, long startEventId) {
-		try {
-			return start(proxyVmId, startEventId);
-		} catch (StorageUnavailableException e) {
-			s_logger.warn("Exception while trying to start console proxy", e);
-			return null;
-		} catch (InsufficientCapacityException e) {
-			s_logger.warn("Exception while trying to start console proxy", e);
-			return null;
-		} catch (ConcurrentOperationException e) {
-			s_logger.warn("Exception while trying to start console proxy", e);
-			return null;
-		}
-	}
-	
-	@Override
-	@DB
-	public ConsoleProxyVO start(long proxyId, long startEventId)
-			throws StorageUnavailableException, InsufficientCapacityException,
-			ConcurrentOperationException {
-
-		AsyncJobExecutor asyncExecutor = BaseAsyncJobExecutor.getCurrentExecutor();
-		if (asyncExecutor != null) {
-			AsyncJobVO job = asyncExecutor.getJob();
-
-			if (s_logger.isInfoEnabled())
-				s_logger.info("Start console proxy " + proxyId + ", update async job-" + job.getId());
-			_asyncMgr.updateAsyncJobAttachment(job.getId(), "console_proxy", proxyId);
-		}
-
-		ConsoleProxyVO proxy = _consoleProxyDao.findById(proxyId);
-		if (proxy == null || proxy.getRemoved() != null) {
-			s_logger.debug("proxy is not found: " + proxyId);
-			return null;
-		}
-/*
- 		// don't insert event here, it may be called multiple times!
-		saveStartedEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM,
-				EventTypes.EVENT_PROXY_START,
-				"Starting console proxy with Id: " + proxyId, startEventId);
-*/				
-
-		if (s_logger.isTraceEnabled()) {
-			s_logger.trace("Starting console proxy if it is not started, proxy vm id : " + proxyId);
-		}
-
-		for (int i = 0; i < 2; i++) {
-
-			State state = proxy.getState();
-
-			if (state == State.Starting /* || state == State.Migrating */) {
-				if (s_logger.isDebugEnabled())
-					s_logger.debug("Waiting console proxy to be ready, proxy vm id : " + proxyId + " proxy VM state : " + state.toString());
-
-				if (proxy.getPrivateIpAddress() == null || connect(proxy.getPrivateIpAddress(), _proxyCmdPort) != null) {
-					if (proxy.getPrivateIpAddress() == null)
-						s_logger.warn("Retruning a proxy that is being started but private IP has not been allocated yet, proxy vm id : " + proxyId);
-					else
-						s_logger.warn("Waiting console proxy to be ready timed out, proxy vm id : " + proxyId);
-
-					// TODO, it is very tricky here, if the startup process
-					// takes too long and it timed out here,
-					// we may give back a proxy that is not fully ready for
-					// functioning
-				}
-				return proxy;
-			}
-
-			if (state == State.Running) {
-				if (s_logger.isTraceEnabled())
-					s_logger.trace("Console proxy is already started: " + proxy.getName());
-				return proxy;
-			}
-
-			DataCenterVO dc = _dcDao.findById(proxy.getDataCenterId());
-			HostPodVO pod = _podDao.findById(proxy.getPodId());
-			List sps = _storageMgr.getStoragePoolsForVm(proxy.getId());
-			StoragePoolVO sp = sps.get(0); // FIXME
-
-			HashSet avoid = new HashSet();
-			HostVO routingHost = (HostVO) _agentMgr.findHost(Host.Type.Routing,
-					dc, pod, sp, _serviceOffering, _template, proxy, null,
-					avoid);
-
-			if (routingHost == null) {
-				if (s_logger.isDebugEnabled()) {
-					s_logger.debug("Unable to find a routing host for " + proxy.toString());
-					continue;
-				}
-			}
-			// to ensure atomic state transition to Starting state
-			if (!_consoleProxyDao.updateIf(proxy, Event.StartRequested, routingHost.getId())) {
-				if (s_logger.isDebugEnabled()) {
-					ConsoleProxyVO temp = _consoleProxyDao.findById(proxyId);
-					s_logger.debug("Unable to start console proxy " + proxy.getName() + " because it is not in a startable state : "
-							+ ((temp != null) ? temp.getState().toString() : "null"));
-				}
-				continue;
-			}
-
-			try {
-				Answer answer = null;
-				int retry = _find_host_retry;
-
-				// Console proxy VM will be running at routing hosts as routing
-				// hosts have public access to outside network
-				do {
-					if (s_logger.isDebugEnabled()) {
-						s_logger.debug("Trying to start console proxy on host " + routingHost.getName());
-					}
-
-					String privateIpAddress = allocPrivateIpAddress(
-							proxy.getDataCenterId(), routingHost.getPodId(),
-							proxy.getId(), proxy.getPrivateMacAddress());
-					if (privateIpAddress == null && (_IpAllocator != null && !_IpAllocator.exteralIpAddressAllocatorEnabled())) {
-						s_logger.debug("Not enough ip addresses in " + routingHost.getPodId());
-						avoid.add(routingHost);
-						continue;
-					}
-					
-					proxy.setPrivateIpAddress(privateIpAddress);
-					String guestIpAddress = _dcDao.allocateLinkLocalPrivateIpAddress(proxy.getDataCenterId(), routingHost.getPodId(), proxy.getId());
-					proxy.setGuestIpAddress(guestIpAddress);
-
-					_consoleProxyDao.updateIf(proxy, Event.OperationRetry, routingHost.getId());
-					proxy = _consoleProxyDao.findById(proxy.getId());
-
-					List vols = _storageMgr.prepare(proxy, routingHost);
-					if (vols == null) {
-						s_logger.debug("Unable to prepare storage for " + routingHost);
-						avoid.add(routingHost);
-						continue;
-					}
-
-					// _storageMgr.share(proxy, vols, null, true);
-
-					// carry the console proxy port info over so that we don't
-					// need to configure agent on this
-					StartConsoleProxyCommand cmdStart = new StartConsoleProxyCommand(
-							_proxyCmdPort, proxy, proxy.getName(), "", vols,
-							Integer.toString(_consoleProxyPort), 
-							Integer.toString(_consoleProxyUrlPort),
-							_mgmt_host, _mgmt_port, _sslEnabled);
-
-					if (s_logger.isDebugEnabled())
-						s_logger.debug("Sending start command for console proxy " + proxy.getName() + " to " + routingHost.getName());
-					try {
-						answer = _agentMgr.send(routingHost.getId(), cmdStart);
-						
-						s_logger.debug("StartConsoleProxy Answer: " + (answer != null ? answer : "null"));
-
-						if (s_logger.isDebugEnabled())
-							s_logger.debug("Received answer on starting console proxy " + proxy.getName() + " on " + routingHost.getName());
-
-						if (answer != null && answer.getResult()) {
-							if (s_logger.isDebugEnabled()) {
-								s_logger.debug("Console proxy " + proxy.getName() + " started on " + routingHost.getName());
-							}
-
-							if (answer instanceof StartConsoleProxyAnswer) {
-								StartConsoleProxyAnswer rAnswer = (StartConsoleProxyAnswer) answer;
-								if (rAnswer.getPrivateIpAddress() != null) {
-									proxy.setPrivateIpAddress(rAnswer.getPrivateIpAddress());
-								}
-								if (rAnswer.getPrivateMacAddress() != null) {
-									proxy.setPrivateMacAddress(rAnswer.getPrivateMacAddress());
-								}
-							}
-
-							final EventVO event = new EventVO();
-							event.setUserId(User.UID_SYSTEM);
-							event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-							event.setType(EventTypes.EVENT_PROXY_START);
-							event.setLevel(EventVO.LEVEL_INFO);
-							event.setStartId(startEventId);
-							event.setDescription("Console proxy started - " + proxy.getName());
-							_eventDao.persist(event);
-							break;
-						}
-						s_logger.debug("Unable to start " + proxy.toString() + " on host " + routingHost.toString() + " due to " + answer.getDetails());
-					} catch (OperationTimedoutException e) {
-						if (e.isActive()) {
-							s_logger.debug("Unable to start vm " + proxy.getName() + " due to operation timed out and it is active so scheduling a restart.");
-							_haMgr.scheduleRestart(proxy, true);
-							return null;
-						}
-					} catch (AgentUnavailableException e) {
-						s_logger.debug("Agent " + routingHost.toString() + " was unavailable to start VM "
-							+ proxy.getName());
-					}
-
-					avoid.add(routingHost);
-					proxy.setPrivateIpAddress(null);
-					freePrivateIpAddress(privateIpAddress, proxy.getDataCenterId(), proxy.getId());
-					proxy.setGuestIpAddress(null);
-					_dcDao.releaseLinkLocalPrivateIpAddress(guestIpAddress, proxy.getDataCenterId(), proxy.getId());
-					_storageMgr.unshare(proxy, vols, routingHost);
-				} while (--retry > 0 && (routingHost = (HostVO) _agentMgr.findHost(Host.Type.Routing, dc, pod, sp,
-						_serviceOffering, _template, proxy, null, avoid)) != null);
-				if (routingHost == null || retry <= 0) {
-
-					SubscriptionMgr.getInstance().notifySubscribers(
-							ConsoleProxyManager.ALERT_SUBJECT,
-							this, 
-							new ConsoleProxyAlertEventArgs(
-								ConsoleProxyAlertEventArgs.PROXY_START_FAILURE,
-								proxy.getDataCenterId(), proxy.getId(), proxy,
-								"Unable to find a routing host to run")
-							);
-
-					final EventVO event = new EventVO();
-					event.setUserId(User.UID_SYSTEM);
-					event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-					event.setType(EventTypes.EVENT_PROXY_START);
-					event.setLevel(EventVO.LEVEL_ERROR);
-					event.setStartId(startEventId);
-					event.setDescription("Starting console proxy failed due to unable to find a host - " + proxy.getName());
-					_eventDao.persist(event);
-					throw new ExecutionException("Couldn't find a routingHost to run console proxy");
-				}
-
-				_consoleProxyDao.updateIf(proxy, Event.OperationSucceeded, routingHost.getId());
-				if (s_logger.isDebugEnabled()) {
-					s_logger.debug("Console proxy is now started, vm id : " + proxy.getId());
-				}
-
-				// If starting the console proxy failed due to the external
-				// firewall not being reachable, send an alert.
-				if (answer != null && answer.getDetails() != null && answer.getDetails().equals("firewall")) {
-
-					SubscriptionMgr.getInstance().notifySubscribers(
-						ConsoleProxyManager.ALERT_SUBJECT,
-						this,
-						new ConsoleProxyAlertEventArgs(
-							ConsoleProxyAlertEventArgs.PROXY_FIREWALL_ALERT,
-							proxy.getDataCenterId(), proxy
-							.getId(), proxy, null)
-						);
-				}
-
-				SubscriptionMgr.getInstance().notifySubscribers(
-					ConsoleProxyManager.ALERT_SUBJECT,
-					this,
-					new ConsoleProxyAlertEventArgs(
-						ConsoleProxyAlertEventArgs.PROXY_UP, proxy.getDataCenterId(), proxy.getId(),
-						proxy, null)
-					);
-
-				return proxy;
-			} catch (Throwable thr) {
-				s_logger.warn("Unexpected exception: ", thr);
-
-				SubscriptionMgr.getInstance().notifySubscribers(
-						ConsoleProxyManager.ALERT_SUBJECT,
-						this,
-						new ConsoleProxyAlertEventArgs(
-							ConsoleProxyAlertEventArgs.PROXY_START_FAILURE,
-							proxy.getDataCenterId(), proxy.getId(), proxy,
-							"Unexpected exception: " + thr.getMessage()));
-
-/*				
-				final EventVO event = new EventVO();
-				event.setUserId(User.UID_SYSTEM);
-				event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-				event.setType(EventTypes.EVENT_PROXY_START);
-				event.setLevel(EventVO.LEVEL_ERROR);
-				event.setStartId(startEventId);
-				event.setDescription("Starting console proxy failed due to unhandled exception - "
-					+ proxy.getName());
-				_eventDao.persist(event);
-*/				
-
-				Transaction txn = Transaction.currentTxn();
-				try {
-					txn.start();
-					String privateIpAddress = proxy.getPrivateIpAddress();
-					if (privateIpAddress != null) {
-						proxy.setPrivateIpAddress(null);
-						freePrivateIpAddress(privateIpAddress, proxy.getDataCenterId(), proxy.getId());
-					}
-
-					_consoleProxyDao.updateIf(proxy, Event.OperationFailed, null);
-					txn.commit();
-				} catch (Exception e) {
-					s_logger.error("Caught exception during error recovery");
-				}
-
-				if (thr instanceof StorageUnavailableException) {
-					throw (StorageUnavailableException) thr;
-				} else if (thr instanceof ConcurrentOperationException) {
-					throw (ConcurrentOperationException) thr;
-				} else if (thr instanceof ExecutionException) {
-					s_logger.error("Error while starting console proxy due to " + thr.getMessage());
-				} else {
-					s_logger.error("Error while starting console proxy ", thr);
-				}
-				return null;
-			}
-		}
-
-		s_logger.warn("Starting console proxy encounters non-startable situation");
-		return null;
-	}
-
-	public ConsoleProxyVO assignProxyFromRunningPool(long dataCenterId) {
-
-		if (s_logger.isTraceEnabled())
-			s_logger.trace("Assign console proxy from running pool for request from data center : " + dataCenterId);
-
-		ConsoleProxyAllocator allocator = getCurrentAllocator();
-		assert (allocator != null);
-		List runningList = _consoleProxyDao.getProxyListInStates(dataCenterId, State.Running);
-		if (runningList != null && runningList.size() > 0) {
-			if (s_logger.isTraceEnabled()) {
-				s_logger.trace("Running proxy pool size : "
-						+ runningList.size());
-				for (ConsoleProxyVO proxy : runningList)
-					s_logger.trace("Running proxy instance : "
-							+ proxy.getName());
-			}
-
-			List> l = _consoleProxyDao.getProxyLoadMatrix();
-			Map loadInfo = new HashMap();
-			if (l != null) {
-				for (Pair p : l) {
-					loadInfo.put(p.first(), p.second());
-
-					if (s_logger.isTraceEnabled()) {
-						s_logger.trace("Running proxy instance allocation load { proxy id : "
-							+ p.first() + ", load : " + p.second() + "}");
-					}
-				}
-			}
-			return allocator.allocProxy(runningList, loadInfo, dataCenterId);
-		} else {
-			if (s_logger.isTraceEnabled())
-				s_logger.trace("Empty running proxy pool for now in data center : " + dataCenterId);
-		}
-		return null;
-	}
-
-	public ConsoleProxyVO assignProxyFromStoppedPool(long dataCenterId) {
-		List l = _consoleProxyDao.getProxyListInStates(
-				dataCenterId, State.Creating, State.Starting, State.Stopped,
-				State.Migrating);
-		if (l != null && l.size() > 0)
-			return l.get(0);
-
-		return null;
-	}
-
-	public ConsoleProxyVO startNew(long dataCenterId) {
-
-		if (s_logger.isDebugEnabled())
-			s_logger.debug("Assign console proxy from a newly started instance for request from data center : " + dataCenterId);
-
-		Map context = createProxyInstance(dataCenterId);
-
-		long proxyVmId = (Long) context.get("proxyVmId");
-		if (proxyVmId == 0) {
-			if (s_logger.isTraceEnabled())
-				s_logger.trace("Creating proxy instance failed, data center id : " + dataCenterId);
-
-			// release critical system resource on failure
-			if (context.get("publicIpAddress") != null)
-				freePublicIpAddress((String) context.get("publicIpAddress"), dataCenterId, 0);
-
-			return null;
-		}
-
-		ConsoleProxyVO proxy = allocProxyStorage(dataCenterId, proxyVmId);
-		if (proxy != null) {
-			SubscriptionMgr.getInstance().notifySubscribers(
-					ConsoleProxyManager.ALERT_SUBJECT,
-					this,
-					new ConsoleProxyAlertEventArgs(
-							ConsoleProxyAlertEventArgs.PROXY_CREATED,
-							dataCenterId, proxy.getId(), proxy, null));
-			return proxy;
-		} else {
-			if (s_logger.isDebugEnabled())
-				s_logger.debug("Unable to allocate console proxy storage, remove the console proxy record from DB, proxy id: " + proxyVmId);
-
-			SubscriptionMgr.getInstance().notifySubscribers(
-					ConsoleProxyManager.ALERT_SUBJECT,
-					this,
-					new ConsoleProxyAlertEventArgs(
-							ConsoleProxyAlertEventArgs.PROXY_CREATE_FAILURE,
-							dataCenterId, proxyVmId, null,
-							"Unable to allocate storage"));
-
-			destroyProxyDBOnly(proxyVmId);
-		}
-		return null;
-	}
-
-	@DB
-	protected Map createProxyInstance(long dataCenterId) {
-
-		Map context = new HashMap();
-		String publicIpAddress = null;
-
-		Transaction txn = Transaction.currentTxn();
-		try {
-			DataCenterVO dc = _dcDao.findById(dataCenterId);
-			assert (dc != null);
-			context.put("dc", dc);
-
-			// this will basically allocate the pod based on data center id as
-			// we use system user id here
-			Set avoidPods = new HashSet();
-			Pair pod = null;
-			networkInfo publicIpAndVlan = null;
-			
-			// About MAC address allocation
-			// MAC address used by User VM is inherited from DomR MAC address,
-			// with the least 16 bits overrided. to avoid
-			// potential conflicts, domP will mask bit 31
-			//
-			String[] macAddresses = _dcDao.getNextAvailableMacAddressPair(dataCenterId, (1L << 31));
-			String privateMacAddress = macAddresses[0];
-			String publicMacAddress = macAddresses[1];
-			macAddresses = _dcDao.getNextAvailableMacAddressPair(dataCenterId, (1L << 31));
-			String guestMacAddress = macAddresses[0];
-			while ((pod = _agentMgr.findPod(_template, _serviceOffering, dc,
-					Account.ACCOUNT_ID_SYSTEM, avoidPods)) != null) {
-				publicIpAndVlan = allocPublicIpAddress(dataCenterId, pod.first().getId(), publicMacAddress);
-				if (publicIpAndVlan == null) {
-					s_logger.warn("Unable to allocate public IP address for console proxy vm in data center : "
-						+ dataCenterId + ", pod=" + pod.first().getId());
-					avoidPods.add(pod.first().getId());
-				} else {
-					break;
-				}
-			}
-
-			if (pod == null || publicIpAndVlan == null) {
-				s_logger.warn("Unable to allocate pod for console proxy vm in data center : " + dataCenterId);
-
-				context.put("proxyVmId", (long) 0);
-				return context;
-			}
-			
-			long id = _consoleProxyDao.getNextInSequence(Long.class, "id");
-
-			context.put("publicIpAddress", publicIpAndVlan._ipAddr);
-			context.put("pod", pod);
-			if (s_logger.isDebugEnabled()) {
-				s_logger.debug("Pod allocated " + pod.first().getName());
-			}
-
-			String cidrNetmask = NetUtils.getCidrNetmask(pod.first().getCidrSize());
-
-			// Find the VLAN ID, VLAN gateway, and VLAN netmask for
-			// publicIpAddress
-			publicIpAddress = publicIpAndVlan._ipAddr;
-			
-			String vlanGateway = publicIpAndVlan._gateWay;
-			String vlanNetmask = publicIpAndVlan._netMask;
-
-			txn.start();
-			ConsoleProxyVO proxy;
-			String name = VirtualMachineName.getConsoleProxyName(id, _instance).intern();
-			proxy = new ConsoleProxyVO(id, name, guestMacAddress, null, NetUtils.getLinkLocalNetMask(),
-					privateMacAddress, null, cidrNetmask, _template.getId(),
-					_template.getGuestOSId(), publicMacAddress,
-					publicIpAddress, vlanNetmask, publicIpAndVlan._vlanDbId, publicIpAndVlan._vlanid, pod.first().getId(), dataCenterId,
-					vlanGateway, null, dc.getDns1(), dc.getDns2(), _domain,
-					_proxyRamSize, 0);
-			
-			proxy.setLastHostId(pod.second());
-			proxy = _consoleProxyDao.persist(proxy);
-			long proxyVmId = proxy.getId();
-
-			final EventVO event = new EventVO();
-			event.setUserId(User.UID_SYSTEM);
-			event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-			event.setType(EventTypes.EVENT_PROXY_CREATE);
-			event.setLevel(EventVO.LEVEL_INFO);
-			event.setDescription("New console proxy created - "
-					+ proxy.getName());
-			_eventDao.persist(event);
-			txn.commit();
-
-			context.put("proxyVmId", proxyVmId);
-			return context;
-		} catch (Throwable e) {
-			s_logger.error("Unexpected exception : ", e);
-
-			context.put("proxyVmId", (long) 0);
-			return context;
-		}
-	}
-
-	@DB
-	protected ConsoleProxyVO allocProxyStorage(long dataCenterId, long proxyVmId) {
-		ConsoleProxyVO proxy = _consoleProxyDao.findById(proxyVmId);
-		assert (proxy != null);
-
-		DataCenterVO dc = _dcDao.findById(dataCenterId);
-		HostPodVO pod = _podDao.findById(proxy.getPodId());
-		final AccountVO account = _accountDao.findById(Account.ACCOUNT_ID_SYSTEM);
-
-		try {
-			List vols = _storageMgr.create(account, proxy, _template, dc, pod, _serviceOffering, null,0);
-			if (vols == null) {
-				s_logger.error("Unable to alloc storage for console proxy");
-				return null;
-			}
-
-			Transaction txn = Transaction.currentTxn();
-			txn.start();
-
-			// update pool id
-			ConsoleProxyVO vo = _consoleProxyDao.findById(proxy.getId());
-			_consoleProxyDao.update(proxy.getId(), vo);
-
-			// kick the state machine
-			_consoleProxyDao.updateIf(proxy, Event.OperationSucceeded, null);
-
-			txn.commit();
-			return proxy;
-		} catch (StorageUnavailableException e) {
-			s_logger.error("Unable to alloc storage for console proxy: ", e);
-			return null;
-		} catch (ExecutionException e) {
-			s_logger.error("Unable to alloc storage for console proxy: ", e);
-			return null;
-		}
-	}
-
-	private networkInfo allocPublicIpAddress(long dcId, long podId, String macAddr) {
-		
-		if (_IpAllocator != null && _IpAllocator.exteralIpAddressAllocatorEnabled()) {
-			IpAddrAllocator.IpAddr ip = _IpAllocator.getPublicIpAddress(macAddr, dcId, podId);
-			networkInfo net = new networkInfo(ip.ipaddr, ip.netMask, ip.gateway, null, "untagged");
-			return net;
-		}
-
-		Pair ipAndVlan = _vlanDao.assignIpAddress(dcId,
-				Account.ACCOUNT_ID_SYSTEM, DomainVO.ROOT_DOMAIN,
-				VlanType.VirtualNetwork, true);
-
-		if (ipAndVlan == null) {
-			s_logger.debug("Unable to get public ip address (type=Virtual) for console proxy vm for data center  : " + dcId);
-			ipAndVlan = _vlanDao.assignPodDirectAttachIpAddress(dcId, podId, Account.ACCOUNT_ID_SYSTEM, DomainVO.ROOT_DOMAIN);
-			if (ipAndVlan == null)
-				s_logger.debug("Unable to get public ip address (type=DirectAttach) for console proxy vm for data center  : " + dcId);
-		}
-		if (ipAndVlan != null) {
-			VlanVO vlan = ipAndVlan.second();
-			networkInfo net = new networkInfo(ipAndVlan.first(), vlan.getVlanNetmask(), vlan.getVlanGateway(), vlan.getId(), vlan.getVlanId());
-			return net;
-		}
-		return null;
-	}
-
-	private String allocPrivateIpAddress(Long dcId, Long podId, Long proxyId, String macAddr) {
-		if (_IpAllocator != null && _IpAllocator.exteralIpAddressAllocatorEnabled()) {
-			return _IpAllocator.getPrivateIpAddress(macAddr, dcId, podId).ipaddr;
-		} else {
-			return _dcDao.allocatePrivateIpAddress(dcId, podId, proxyId);
-		}
-	}
-	
-	private void freePrivateIpAddress(String ipAddress, Long dcId, Long podId) {
-		if (_IpAllocator != null && _IpAllocator.exteralIpAddressAllocatorEnabled()) {
-			 _IpAllocator.releasePrivateIpAddress(ipAddress, dcId, podId);
-		} else {
-			_dcDao.releasePrivateIpAddress(ipAddress, dcId, podId);
-		}
-	}
-	
-	private void freePublicIpAddress(String ipAddress, long dcId, long podId) {
-		if (_IpAllocator != null && _IpAllocator.exteralIpAddressAllocatorEnabled()) {
-			 _IpAllocator.releasePublicIpAddress(ipAddress, dcId, podId);
-		} else {
-			_ipAddressDao.unassignIpAddress(ipAddress);
-		}
-	}
-
-	private ConsoleProxyAllocator getCurrentAllocator() {
-		// for now, only one adapter is supported
-		Enumeration it = _consoleProxyAllocators.enumeration();
-		if (it.hasMoreElements())
-			return it.nextElement();
-
-		return null;
-	}
-
-	protected String connect(String ipAddress, int port) {
-		for (int i = 0; i <= _ssh_retry; i++) {
-			SocketChannel sch = null;
-			try {
-				if (s_logger.isDebugEnabled()) {
-					s_logger.debug("Trying to connect to " + ipAddress);
-				}
-				sch = SocketChannel.open();
-				sch.configureBlocking(true);
-				sch.socket().setSoTimeout(5000);
-
-				InetSocketAddress addr = new InetSocketAddress(ipAddress, port);
-				sch.connect(addr);
-				return null;
-			} catch (IOException e) {
-				if (s_logger.isDebugEnabled()) {
-					s_logger.debug("Could not connect to " + ipAddress);
-				}
-			} finally {
-				if (sch != null) {
-					try {
-						sch.close();
-					} catch (IOException e) {
-					}
-				}
-			}
-			try {
-				Thread.sleep(_ssh_sleep);
-			} catch (InterruptedException ex) {
-			}
-		}
-
-		s_logger.debug("Unable to logon to " + ipAddress);
-
-		return "Unable to connect";
-	}
-
-	public void onLoadAnswer(ConsoleProxyLoadAnswer answer) {
-		if (answer.getDetails() == null)
-			return;
-
-		ConsoleProxyStatus status = null;
-		try {
-			GsonBuilder gb = new GsonBuilder();
-			gb.setVersion(1.3);
-			Gson gson = gb.create();
-			status = gson.fromJson(answer.getDetails(), ConsoleProxyStatus.class);
-		} catch (Throwable e) {
-			s_logger.warn("Unable to parse load info from proxy, proxy vm id : "
-					+ answer.getProxyVmId() + ", info : " + answer.getDetails());
-		}
-
-		if (status != null) {
-			int count = 0;
-			if (status.getConnections() != null)
-				count = status.getConnections().length;
-
-			byte[] details = null;
-			if (answer.getDetails() != null)
-				details = answer.getDetails().getBytes(Charset.forName("US-ASCII"));
-			_consoleProxyDao.update(answer.getProxyVmId(), count, DateUtil.currentGMTTime(), details);
-		} else {
-			if (s_logger.isTraceEnabled())
-				s_logger.trace("Unable to get console proxy load info, id : " + answer.getProxyVmId());
-
-			_consoleProxyDao.update(answer.getProxyVmId(), 0, DateUtil.currentGMTTime(), null);
-			// TODO : something is wrong with the VM, restart it?
-		}
-	}
-
-	public void onLoadReport(ConsoleProxyLoadReportCommand cmd) {
-		if (cmd.getLoadInfo() == null)
-			return;
-
-		ConsoleProxyStatus status = null;
-		try {
-			GsonBuilder gb = new GsonBuilder();
-			gb.setVersion(1.3);
-			Gson gson = gb.create();
-			status = gson.fromJson(cmd.getLoadInfo(), ConsoleProxyStatus.class);
-		} catch (Throwable e) {
-			s_logger.warn("Unable to parse load info from proxy, proxy vm id : "
-				+ cmd.getProxyVmId() + ", info : " + cmd.getLoadInfo());
-		}
-
-		if (status != null) {
-			int count = 0;
-			if (status.getConnections() != null)
-				count = status.getConnections().length;
-
-			byte[] details = null;
-			if (cmd.getLoadInfo() != null)
-				details = cmd.getLoadInfo().getBytes(Charset.forName("US-ASCII"));
-			_consoleProxyDao.update(cmd.getProxyVmId(), count, DateUtil.currentGMTTime(), details);
-		} else {
-			if (s_logger.isTraceEnabled())
-				s_logger.trace("Unable to get console proxy load info, id : " + cmd.getProxyVmId());
-
-			_consoleProxyDao.update(cmd.getProxyVmId(), 0, DateUtil.currentGMTTime(), null);
-		}
-	}
-
-	public AgentControlAnswer onConsoleAccessAuthentication(
-			ConsoleAccessAuthenticationCommand cmd) {
-		long vmId = 0;
-
-		if (cmd.getVmId() != null && cmd.getVmId().isEmpty()) {
-			if (s_logger.isTraceEnabled())
-				s_logger.trace("Invalid vm id sent from proxy(happens when proxy session has terminated)");
-			return new ConsoleAccessAuthenticationAnswer(cmd, false);
-		}
-
-		try {
-			vmId = Long.parseLong(cmd.getVmId());
-		} catch (NumberFormatException e) {
-			s_logger.error("Invalid vm id " + cmd.getVmId() + " sent from console access authentication", e);
-			return new ConsoleAccessAuthenticationAnswer(cmd, false);
-		}
-
-		// TODO authentication channel between console proxy VM and management
-		// server needs to be secured,
-		// the data is now being sent through private network, but this is
-		// apparently not enough
-		VMInstanceVO vm = _instanceDao.findById(vmId);
-		if (vm == null) {
-			return new ConsoleAccessAuthenticationAnswer(cmd, false);
-		}
-
-		if (vm.getHostId() == null) {
-			s_logger.warn("VM " + vmId + " lost host info, failed authentication request");
-			return new ConsoleAccessAuthenticationAnswer(cmd, false);
-		}
-
-		HostVO host = _hostDao.findById(vm.getHostId());
-		if (host == null) {
-			s_logger.warn("VM " + vmId + "'s host does not exist, fail authentication request");
-			return new ConsoleAccessAuthenticationAnswer(cmd, false);
-		}
-
-		String sid = cmd.getSid();
-		if (sid == null || !sid.equals(vm.getVncPassword())) {
-			s_logger.warn("sid " + sid + " in url does not match stored sid " + vm.getVncPassword());
-			return new ConsoleAccessAuthenticationAnswer(cmd, false);
-		}
-
-		return new ConsoleAccessAuthenticationAnswer(cmd, true);
-	}
-
-	private ConsoleProxyVO findConsoleProxyByHost(HostVO host) throws NumberFormatException {
-		String name = host.getName();
-		long proxyVmId = 0;
-		ConsoleProxyVO proxy = null;
-		if (name != null && name.startsWith("v-")) {
-			String[] tokens = name.split("-");
-			proxyVmId = Long.parseLong(tokens[1]);
-			proxy = this._consoleProxyDao.findById(proxyVmId);
-		}
-		return proxy;
-	}
-	@Override
-	public void onAgentConnect(HostVO host, StartupCommand cmd) {
-		if (host.getType() == Type.ConsoleProxy) {
-			// TODO we can use this event to mark the proxy is up and
-			// functioning instead of
-			// pinging the console proxy VM command port
-			//
-			// for now, just log a message
-			if (s_logger.isInfoEnabled())
-				s_logger.info("Console proxy agent is connected. proxy: " + host.getName());
-			
-			/* update public/private ip address */
-			if (_IpAllocator != null && _IpAllocator.exteralIpAddressAllocatorEnabled()) {
-				try {
-					ConsoleProxyVO console = findConsoleProxyByHost(host);
-					if (console == null) {
-						s_logger.debug("Can't find console proxy ");
-						return;
-					}
-					console.setPrivateIpAddress(cmd.getPrivateIpAddress());
-					console.setPrivateNetmask(cmd.getPrivateNetmask());
-					console.setPublicIpAddress(cmd.getPublicIpAddress());
-					console.setPublicNetmask(cmd.getPublicNetmask());
-					_consoleProxyDao.persist(console);
-				} catch (NumberFormatException e) {
-				}
-			}
-		}
-	}
-
-	@Override
-	public void onAgentDisconnect(long agentId, com.cloud.host.Status state) {
-		if (state == com.cloud.host.Status.Alert || state == com.cloud.host.Status.Disconnected) {
-			// be it either in alert or in disconnected state, the agent process
-			// may be gone in the VM,
-			// we will be reacting to stop the corresponding VM and let the scan
-			// process to
-			HostVO host = _hostDao.findById(agentId);
-			if (host.getType() == Type.ConsoleProxy) {
-				String name = host.getName();
-				if (s_logger.isInfoEnabled())
-					s_logger.info("Console proxy agent disconnected, proxy: " + name);
-				if (name != null && name.startsWith("v-")) {
-					String[] tokens = name.split("-");
-					long proxyVmId = 0;
-					try {
-						proxyVmId = Long.parseLong(tokens[1]);
-					} catch (NumberFormatException e) {
-						s_logger.error("Unexpected exception " + e.getMessage(), e);
-						return;
-					}
-
-					final ConsoleProxyVO proxy = this._consoleProxyDao.findById(proxyVmId);
-					if (proxy != null) {
-						Long hostId = proxy.getHostId();
-
-						// Disable this feature for now, as it conflicts with
-						// the case of allowing user to reboot console proxy
-						// when rebooting happens, we will receive disconnect
-						// here and we can't enter into stopping process,
-						// as when the rebooted one comes up, it will kick off a
-						// newly started one and trigger the process
-						// continue on forever
-
-						/*
-						 * _capacityScanScheduler.execute(new Runnable() {
-						 * public void run() { if(s_logger.isInfoEnabled())
-						 * s_logger.info("Stop console proxy " + proxy.getName()
-						 * +
-						 * " VM because of that the agent running inside it has disconnected"
-						 * ); stopProxy(proxy.getId()); } });
-						 */
-					} else {
-						if (s_logger.isInfoEnabled())
-							s_logger.info("Console proxy agent disconnected but corresponding console proxy VM no longer exists in DB, proxy: " + name);
-					}
-				} else {
-					assert (false) : "Invalid console proxy name: " + name;
-				}
-			}
-		}
-	}
-
-	private void checkPendingProxyVMs() {
-		// drive state to change away from transient states
-		List l = _consoleProxyDao.getProxyListInStates(State.Creating);
-		if (l != null && l.size() > 0) {
-			for (ConsoleProxyVO proxy : l) {
-				if (proxy.getLastUpdateTime() == null
-						|| (proxy.getLastUpdateTime() != null && System.currentTimeMillis() - proxy.getLastUpdateTime().getTime() > 60000)) {
-					try {
-						ConsoleProxyVO readyProxy = null;
-						if (_allocProxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
-							try {
-								readyProxy = allocProxyStorage(proxy.getDataCenterId(), proxy.getId());
-							} finally {
-								_allocProxyLock.unlock();
-							}
-
-							if (readyProxy != null) {
-								GlobalLock proxyLock = GlobalLock.getInternLock(getProxyLockName(readyProxy.getId()));
-								try {
-									if (proxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
-										try {
-											readyProxy = start(readyProxy.getId(), 0);
-										} finally {
-											proxyLock.unlock();
-										}
-									} else {
-										if (s_logger.isInfoEnabled())
-											s_logger.info("Unable to acquire synchronization lock to start console proxy : " + readyProxy.getName());
-									}
-								} finally {
-									proxyLock.releaseRef();
-								}
-							}
-						} else {
-							if (s_logger.isInfoEnabled())
-								s_logger.info("Unable to acquire synchronization lock to allocate proxy storage, wait for next turn");
-						}
-					} catch (StorageUnavailableException e) {
-						s_logger.warn("Storage unavailable", e);
-					} catch (InsufficientCapacityException e) {
-						s_logger.warn("insuffiient capacity", e);
-					} catch (ConcurrentOperationException e) {
-						s_logger.debug("Concurrent operation: "
-								+ e.getMessage());
-					}
-				}
-			}
-		}
-	}
-
-	private Runnable getCapacityScanTask() {
-		return new Runnable() {
-
-			@Override
-			public void run() {
-				Transaction txn = Transaction.open(Transaction.CLOUD_DB);
-				try {
-					reallyRun();
-				} catch (Throwable e) {
-					s_logger.warn("Unexpected exception " + e.getMessage(), e);
-				} finally {
-					StackMaid.current().exitCleanup();
-					txn.close();
-				}
-			}
-
-			private void reallyRun() {
-				if (s_logger.isTraceEnabled())
-					s_logger.trace("Begin console proxy capacity scan");
-
-				// config var for consoleproxy.restart check
-				String restart = _configDao.getValue("consoleproxy.restart");
-				if(restart != null && restart.equalsIgnoreCase("false"))
-				{
-					s_logger.debug("Capacity scan disabled purposefully, consoleproxy.restart = false. This happens when the primarystorage is in maintenance mode");
-					return;
-				}
-					
-				Map zoneHostInfoMap = getZoneHostInfo();
-				if (isServiceReady(zoneHostInfoMap)) {
-					if (s_logger.isTraceEnabled())
-						s_logger.trace("Service is ready, check to see if we need to allocate standby capacity");
-
-					if (!_capacityScanLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_COOPERATION)) {
-						if (s_logger.isTraceEnabled())
-							s_logger.trace("Capacity scan lock is used by others, skip and wait for my turn");
-						return;
-					}
-
-					if (s_logger.isTraceEnabled())
-						s_logger.trace("*** Begining capacity scan... ***");
-
-					try {
-						checkPendingProxyVMs();
-
-						// scan default data center first
-						long defaultId = 0;
-
-						// proxy count info by data-centers (zone-id, zone-name,
-						// count)
-						List l = _consoleProxyDao.getDatacenterProxyLoadMatrix();
-
-						// running VM session count by data-centers (zone-id,
-						// zone-name, count)
-						List listVmCounts = _consoleProxyDao.getDatacenterSessionLoadMatrix();
-
-						// indexing load info by data-center id
-						Map mapVmCounts = new HashMap();
-						if (listVmCounts != null)
-							for (ConsoleProxyLoadInfo info : listVmCounts)
-								mapVmCounts.put(info.getId(), info);
-
-						for (ConsoleProxyLoadInfo info : l) {
-							if (info.getName().equals(_instance)) {
-								ConsoleProxyLoadInfo vmInfo = mapVmCounts.get(info.getId());
-
-								if (!checkCapacity(info, vmInfo != null ? vmInfo : new ConsoleProxyLoadInfo())) {
-									if (isZoneReady(zoneHostInfoMap, info.getId())) {
-										allocCapacity(info.getId());
-									} else {
-										if (s_logger.isTraceEnabled())
-											s_logger.trace("Zone " + info.getId() + " is not ready to alloc standy console proxy");
-									}
-								}
-
-								defaultId = info.getId();
-								break;
-							}
-						}
-
-						// scan rest of data-centers
-						for (ConsoleProxyLoadInfo info : l) {
-							if (info.getId() != defaultId) {
-								ConsoleProxyLoadInfo vmInfo = mapVmCounts.get(info.getId());
-
-								if (!checkCapacity(info, vmInfo != null ? vmInfo : new ConsoleProxyLoadInfo())) {
-									if (isZoneReady(zoneHostInfoMap, info.getId())) {
-										allocCapacity(info.getId());
-									} else {
-										if (s_logger.isTraceEnabled())
-											s_logger.trace("Zone " + info.getId() + " is not ready to alloc standy console proxy");
-									}
-								}
-							}
-						}
-
-						if (s_logger.isTraceEnabled())
-							s_logger.trace("*** Stop capacity scan ***");
-					} finally {
-						_capacityScanLock.unlock();
-					}
-
-				} else {
-					if (s_logger.isTraceEnabled())
-						s_logger.trace("Service is not ready for capacity preallocation, wait for next time");
-				}
-
-				if (s_logger.isTraceEnabled())
-					s_logger.trace("End of console proxy capacity scan");
-			}
-		};
-	}
-
-	private boolean checkCapacity(ConsoleProxyLoadInfo proxyCountInfo,
-			ConsoleProxyLoadInfo vmCountInfo) {
-
-		if (proxyCountInfo.getCount() * _capacityPerProxy
-				- vmCountInfo.getCount() <= _standbyCapacity)
-			return false;
-
-		return true;
-	}
-
-	private void allocCapacity(long dataCenterId) {
-		if (s_logger.isTraceEnabled())
-			s_logger.trace("Allocate console proxy standby capacity for data center : " + dataCenterId);
-
-		boolean proxyFromStoppedPool = false;
-		ConsoleProxyVO proxy = assignProxyFromStoppedPool(dataCenterId);
-		if (proxy == null) {
-			if (s_logger.isInfoEnabled())
-				s_logger.info("No stopped console proxy is available, need to allocate a new console proxy");
-
-			if (_allocProxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
-				try {
-					proxy = startNew(dataCenterId);
-				} finally {
-					_allocProxyLock.unlock();
-				}
-			} else {
-				if (s_logger.isInfoEnabled())
-					s_logger.info("Unable to acquire synchronization lock to allocate proxy resource for standby capacity, wait for next scan");
-				return;
-			}
-		} else {
-			if (s_logger.isInfoEnabled())
-				s_logger.info("Found a stopped console proxy, bring it up to running pool. proxy vm id : " + proxy.getId());
-			proxyFromStoppedPool = true;
-		}
-
-		if (proxy != null) {
-			long proxyVmId = proxy.getId();
-			GlobalLock proxyLock = GlobalLock.getInternLock(getProxyLockName(proxyVmId));
-			try {
-				if (proxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
-					try {
-						proxy = startProxy(proxyVmId, 0);
-					} finally {
-						proxyLock.unlock();
-					}
-				} else {
-					if (s_logger.isInfoEnabled())
-						s_logger.info("Unable to acquire synchronization lock to start proxy for standby capacity, proxy vm id : " + proxy.getId());
-					return;
-				}
-			} finally {
-				proxyLock.releaseRef();
-			}
-
-			if (proxy == null) {
-				if (s_logger.isInfoEnabled())
-					s_logger.info("Unable to start console proxy for standby capacity, proxy vm Id : "
-									+ proxyVmId + ", will recycle it and start a new one");
-
-				if (proxyFromStoppedPool)
-					destroyProxy(proxyVmId, 0);
-			} else {
-				if (s_logger.isInfoEnabled())
-					s_logger.info("Console proxy " + proxy.getName() + " is started");
-			}
-		}
-	}
-
-	public boolean isServiceReady(Map zoneHostInfoMap) {
-		for (ZoneHostInfo zoneHostInfo : zoneHostInfoMap.values()) {
-			if (isZoneHostReady(zoneHostInfo)) {
-				if (s_logger.isInfoEnabled())
-					s_logger.info("Zone " + zoneHostInfo.getDcId() + " is ready to launch");
-				return true;
-			}
-		}
-
-		return false;
-	}
-
-	public boolean isZoneReady(Map zoneHostInfoMap,
-			long dataCenterId) {
-		ZoneHostInfo zoneHostInfo = zoneHostInfoMap.get(dataCenterId);
-		if (zoneHostInfo != null && isZoneHostReady(zoneHostInfo)) {
-			VMTemplateVO template = _templateDao.findConsoleProxyTemplate();
-			HostVO secondaryStorageHost = _storageMgr.getSecondaryStorageHost(dataCenterId);
-			boolean templateReady = false;
-
-			if (template != null && secondaryStorageHost != null) {
-				VMTemplateHostVO templateHostRef = _vmTemplateHostDao.findByHostTemplate(secondaryStorageHost.getId(), template.getId());
-				templateReady = (templateHostRef != null) && (templateHostRef.getDownloadState() == Status.DOWNLOADED);
-			}
-
-			if (templateReady) {
-				List> l = _consoleProxyDao.getDatacenterStoragePoolHostInfo(dataCenterId, _use_lvm);
-				if (l != null && l.size() > 0 && l.get(0).second().intValue() > 0) {
-					return true;
-				} else {
-					if (s_logger.isTraceEnabled())
-						s_logger.trace("Primary storage is not ready, wait until it is ready to launch console proxy");
-				}
-			} else {
-				if (s_logger.isTraceEnabled())
-					s_logger.trace("Zone host is ready, but console proxy template is not ready");
-			}
-		}
-		return false;
-	}
-
-	private boolean isZoneHostReady(ZoneHostInfo zoneHostInfo) {
-		int expectedFlags = 0;
-		if (_use_storage_vm)
-			expectedFlags = RunningHostInfoAgregator.ZoneHostInfo.ROUTING_HOST_MASK;
-		else
-			expectedFlags = RunningHostInfoAgregator.ZoneHostInfo.ALL_HOST_MASK;
-
-		return (zoneHostInfo.getFlags() & expectedFlags) == expectedFlags;
-	}
-
-	private synchronized Map getZoneHostInfo() {
-		Date cutTime = DateUtil.currentGMTTime();
-		List l = _hostDao.getRunningHostCounts(new Date(cutTime.getTime() - ClusterManager.DEFAULT_HEARTBEAT_THRESHOLD));
-
-		RunningHostInfoAgregator aggregator = new RunningHostInfoAgregator();
-		if (l.size() > 0)
-			for (RunningHostCountInfo countInfo : l)
-				aggregator.aggregate(countInfo);
-
-		return aggregator.getZoneHostInfoMap();
-	}
-
-	@Override
-	public String getName() {
-		return _name;
-	}
-
-	@Override
-	public boolean start() {
-		if (s_logger.isInfoEnabled())
-			s_logger.info("Start console proxy manager");
-
-		return true;
-	}
-
-	@Override
-	public boolean stop() {
-		if (s_logger.isInfoEnabled())
-			s_logger.info("Stop console proxy manager");
-		_capacityScanScheduler.shutdownNow();
-
-		try {
-			_capacityScanScheduler.awaitTermination(EXECUTOR_SHUTDOWN_TIMEOUT,
-					TimeUnit.MILLISECONDS);
-		} catch (InterruptedException e) {
-		}
-
-		_capacityScanLock.releaseRef();
-		_allocProxyLock.releaseRef();
-		return true;
-	}
-
-	@Override
-	public boolean configure(String name, Map params)
-			throws ConfigurationException {
-		if (s_logger.isInfoEnabled())
-			s_logger.info("Start configuring console proxy manager : " + name);
-
-		_name = name;
-
-		ComponentLocator locator = ComponentLocator.getCurrentLocator();
-		ConfigurationDao configDao = locator.getDao(ConfigurationDao.class);
-		if (configDao == null) {
-			throw new ConfigurationException(
-					"Unable to get the configuration dao.");
-		}
-
-		Map configs = configDao.getConfiguration(
-				"management-server", params);
-
-		_proxyRamSize = NumbersUtil.parseInt(configs
-				.get("consoleproxy.ram.size"), DEFAULT_PROXY_VM_RAMSIZE);
-
-		String value = configs.get("start.retry");
-		_find_host_retry = NumbersUtil.parseInt(value,
-				DEFAULT_FIND_HOST_RETRY_COUNT);
-
-		value = configs.get("consoleproxy.cmd.port");
-		_proxyCmdPort = NumbersUtil.parseInt(value, DEFAULT_PROXY_CMD_PORT);
-
-		value = configs.get("consoleproxy.sslEnabled");
-		if (value != null && value.equalsIgnoreCase("true"))
-			_sslEnabled = true;
-
-		value = configs.get("consoleproxy.capacityscan.interval");
-		_capacityScanInterval = NumbersUtil.parseLong(value, DEFAULT_CAPACITY_SCAN_INTERVAL);
-
-		_capacityPerProxy = NumbersUtil.parseInt(configs.get("consoleproxy.session.max"), DEFAULT_PROXY_CAPACITY);
-		_standbyCapacity = NumbersUtil.parseInt(configs.get("consoleproxy.capacity.standby"), DEFAULT_STANDBY_CAPACITY);
-		_proxySessionTimeoutValue = NumbersUtil.parseInt(configs.get("consoleproxy.session.timeout"), DEFAULT_PROXY_SESSION_TIMEOUT);
-
-		value = configs.get("consoleproxy.port");
-		if (value != null)
-			_consoleProxyPort = NumbersUtil.parseInt(value, ConsoleProxyManager.DEFAULT_PROXY_VNC_PORT);
-
-		value = configs.get("consoleproxy.url.port");
-		if (value != null)
-			_consoleProxyUrlPort = NumbersUtil.parseInt(value, ConsoleProxyManager.DEFAULT_PROXY_URL_PORT);
-
-		value = configs.get("system.vm.use.local.storage");
-		if (value != null && value.equalsIgnoreCase("true"))
-			_use_lvm = true;
-
-		value = configs.get("secondary.storage.vm");
-		if (value != null && value.equalsIgnoreCase("true"))
-			_use_storage_vm = true;
-
-		if (s_logger.isInfoEnabled()) {
-			s_logger.info("Console proxy max session soft limit : " + _capacityPerProxy);
-			s_logger.info("Console proxy standby capacity : " + _standbyCapacity);
-		}
-
-		_domain = configs.get("domain");
-		if (_domain == null) {
-			_domain = "foo.com";
-		}
-
-		_instance = configs.get("instance.name");
-		if (_instance == null) {
-			_instance = "DEFAULT";
-		}
-
-		value = (String) params.get("ssh.sleep");
-		_ssh_sleep = NumbersUtil.parseInt(value, 5) * 1000;
-
-		value = (String) params.get("ssh.retry");
-		_ssh_retry = NumbersUtil.parseInt(value, 3);
-
-		Map agentMgrConfigs = configDao.getConfiguration("AgentManager", params);
-		_mgmt_host = agentMgrConfigs.get("host");
-		if (_mgmt_host == null) {
-			s_logger.warn("Critical warning! Please configure your management server host address right after you have started your management server and then restart it, otherwise you won't be able to do console access");
-		}
-
-		value = agentMgrConfigs.get("port");
-		_mgmt_port = NumbersUtil.parseInt(value, 8250);
-
-		_consoleProxyDao = locator.getDao(ConsoleProxyDao.class);
-		if (_consoleProxyDao == null) {
-			throw new ConfigurationException("Unable to get " + ConsoleProxyDao.class.getName());
-		}
-
-		_consoleProxyAllocators = locator.getAdapters(ConsoleProxyAllocator.class);
-		if (_consoleProxyAllocators == null || !_consoleProxyAllocators.isSet()) {
-			throw new ConfigurationException("Unable to get proxy allocators");
-		}
-
-		_dcDao = locator.getDao(DataCenterDao.class);
-		if (_dcDao == null) {
-			throw new ConfigurationException("Unable to get " + DataCenterDao.class.getName());
-		}
-
-		_templateDao = locator.getDao(VMTemplateDao.class);
-		if (_templateDao == null) {
-			throw new ConfigurationException("Unable to get " + VMTemplateDao.class.getName());
-		}
-
-		_ipAddressDao = locator.getDao(IPAddressDao.class);
-		if (_ipAddressDao == null) {
-			throw new ConfigurationException("Unable to get " + IPAddressDao.class.getName());
-		}
-
-		_volsDao = locator.getDao(VolumeDao.class);
-		if (_volsDao == null) {
-			throw new ConfigurationException("Unable to get " + VolumeDao.class.getName());
-		}
-
-		_podDao = locator.getDao(HostPodDao.class);
-		if (_podDao == null) {
-			throw new ConfigurationException("Unable to get " + HostPodDao.class.getName());
-		}
-
-		_hostDao = locator.getDao(HostDao.class);
-		if (_hostDao == null) {
-			throw new ConfigurationException("Unable to get " + HostDao.class.getName());
-		}
-
-		_eventDao = locator.getDao(EventDao.class);
-		if (_eventDao == null) {
-			throw new ConfigurationException("Unable to get " + EventDao.class.getName());
-		}
-
-		_storagePoolDao = locator.getDao(StoragePoolDao.class);
-		if (_storagePoolDao == null) {
-			throw new ConfigurationException("Unable to find " + StoragePoolDao.class);
-		}
-		
-		_configDao = locator.getDao(ConfigurationDao.class);
-		if (_configDao == null) {
-			throw new ConfigurationException("Unable to find " + ConfigurationDao.class);
-		}
-
-		_vmTemplateHostDao = locator.getDao(VMTemplateHostDao.class);
-		if (_vmTemplateHostDao == null) {
-			throw new ConfigurationException("Unable to get " + VMTemplateHostDao.class.getName());
-		}
-
-		_instanceDao = locator.getDao(VMInstanceDao.class);
-		if (_instanceDao == null)
-			throw new ConfigurationException("Unable to get " + VMInstanceDao.class.getName());
-
-		_capacityDao = locator.getDao(CapacityDao.class);
-		if (_capacityDao == null) {
-			throw new ConfigurationException("Unable to get " + CapacityDao.class.getName());
-		}
-
-		_haDao = locator.getDao(HighAvailabilityDao.class);
-		if (_haDao == null) {
-			throw new ConfigurationException("Unable to get " + HighAvailabilityDao.class.getName());
-		}
-
-		_accountDao = locator.getDao(AccountDao.class);
-		if (_accountDao == null) {
-			throw new ConfigurationException("Unable to get " + AccountDao.class.getName());
-		}
-
-		_vlanDao = locator.getDao(VlanDao.class);
-		if (_vlanDao == null) {
-			throw new ConfigurationException("Unable to get " + VlanDao.class.getName());
-		}
-
-		_agentMgr = locator.getManager(AgentManager.class);
-		if (_agentMgr == null) {
-			throw new ConfigurationException("Unable to get " + AgentManager.class.getName());
-		}
-
-		_networkMgr = locator.getManager(NetworkManager.class);
-		if (_networkMgr == null) {
-			throw new ConfigurationException("Unable to get " + NetworkManager.class.getName());
-		}
-
-		_listener = new ConsoleProxyListener(this);
-		_agentMgr.registerForHostEvents(_listener, true, true, false);
-
-		_haMgr = locator.getManager(HighAvailabilityManager.class);
-		if (_haMgr == null) {
-			throw new ConfigurationException("Unable to get "
-					+ HighAvailabilityManager.class.getName());
-		}
-
-		_storageMgr = locator.getManager(StorageManager.class);
-		if (_storageMgr == null) {
-			throw new ConfigurationException("Unable to get "
-					+ StorageManager.class.getName());
-		}
-
-		_asyncMgr = locator.getManager(AsyncJobManager.class);
-		if (_asyncMgr == null) {
-			throw new ConfigurationException("Unable to get "
-					+ AsyncJobManager.class.getName());
-		}
-		
-		Adapters ipAllocators = locator.getAdapters(IpAddrAllocator.class);
-		if (ipAllocators != null && ipAllocators.isSet()) {
-			Enumeration it = ipAllocators.enumeration();
-			_IpAllocator = it.nextElement();
-		}
-
-		HighAvailabilityManager haMgr = locator.getManager(HighAvailabilityManager.class);
-		if (haMgr != null) {
-			haMgr.registerHandler(VirtualMachine.Type.ConsoleProxy, this);
-		}
-
-		boolean useLocalStorage = Boolean.parseBoolean((String) params.get(Config.SystemVMUseLocalStorage.key()));
-		_serviceOffering = new ServiceOfferingVO("Fake Offering For DomP", 1,
-				_proxyRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized,
-				useLocalStorage, true, null);
-		_serviceOffering.setUniqueName("Cloud.com-ConsoleProxy");
-		_serviceOffering = _offeringDao.persistSystemServiceOffering(_serviceOffering);
-		_template = _templateDao.findConsoleProxyTemplate();
-		if (_template == null) {
-			throw new ConfigurationException(
-					"Unable to find the template for console proxy VMs");
-		}
-
-		_capacityScanScheduler.scheduleAtFixedRate(getCapacityScanTask(),
-				STARTUP_DELAY, _capacityScanInterval, TimeUnit.MILLISECONDS);
-
-		if (s_logger.isInfoEnabled())
-			s_logger.info("Console Proxy Manager is configured.");
-		return true;
-	}
-
-	protected ConsoleProxyManagerImpl() {
-	}
-
-	@Override
-	public Command cleanup(ConsoleProxyVO vm, String vmName) {
-		if (vmName != null) {
-			return new StopCommand(vm, vmName, VirtualMachineName.getVnet(vmName));
-		} else if (vm != null) {
-			ConsoleProxyVO vo = vm;
-			return new StopCommand(vo, null);
-		} else {
-			throw new CloudRuntimeException("Shouldn't even be here!");
-		}
-	}
-
-	@Override
-	public void completeStartCommand(ConsoleProxyVO vm) {
-		_consoleProxyDao.updateIf(vm, Event.AgentReportRunning, vm.getHostId());
-	}
-
-	@Override
-	public void completeStopCommand(ConsoleProxyVO vm) {
-		completeStopCommand(vm, Event.AgentReportStopped);
-	}
-
-	@DB
-	protected void completeStopCommand(ConsoleProxyVO proxy, Event ev) {
-		Transaction txn = Transaction.currentTxn();
-		try {
-			txn.start();
-			String privateIpAddress = proxy.getPrivateIpAddress();
-			if (privateIpAddress != null) {
-				proxy.setPrivateIpAddress(null);
-				freePrivateIpAddress(privateIpAddress, proxy.getDataCenterId(), proxy.getId());
-			}
-			String guestIpAddress = proxy.getGuestIpAddress();
-			if (guestIpAddress != null) {
-				proxy.setGuestIpAddress(null);
-				_dcDao.releaseLinkLocalPrivateIpAddress(guestIpAddress, proxy.getDataCenterId(), proxy.getId());
-			}
-
-			if (!_consoleProxyDao.updateIf(proxy, ev, null)) {
-				s_logger.debug("Unable to update the console proxy");
-				return;
-			}
-			txn.commit();
-		} catch (Exception e) {
-			s_logger.error("Unable to complete stop command due to ", e);
-		}
-
-		if (_storageMgr.unshare(proxy, null) == null) {
-			s_logger.warn("Unable to set share to false for " + proxy.getId());
-		}
-	}
-
-	@Override
-	public ConsoleProxyVO get(long id) {
-		return _consoleProxyDao.findById(id);
-	}
-
-	@Override
-	public Long convertToId(String vmName) {
-		if (!VirtualMachineName.isValidConsoleProxyName(vmName, _instance)) {
-			return null;
-		}
-		return VirtualMachineName.getConsoleProxyId(vmName);
-	}
-
-	@Override
-	public boolean stopProxy(long proxyVmId, long startEventId) {
-
-		AsyncJobExecutor asyncExecutor = BaseAsyncJobExecutor.getCurrentExecutor();
-		if (asyncExecutor != null) {
-			AsyncJobVO job = asyncExecutor.getJob();
-
-			if (s_logger.isInfoEnabled())
-				s_logger.info("Stop console proxy " + proxyVmId + ", update async job-" + job.getId());
-			_asyncMgr.updateAsyncJobAttachment(job.getId(), "console_proxy", proxyVmId);
-		}
-
-		ConsoleProxyVO proxy = _consoleProxyDao.findById(proxyVmId);
-		if (proxy == null) {
-			if (s_logger.isDebugEnabled())
-				s_logger.debug("Stopping console proxy failed: console proxy " + proxyVmId + " no longer exists");
-			return false;
-		}
-/*		
-		saveStartedEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM,
-				EventTypes.EVENT_PROXY_STOP, "Stopping console proxy with Id: "
-						+ proxyVmId, startEventId);
-*/						
-		try {
-			return stop(proxy, startEventId);
-		} catch (AgentUnavailableException e) {
-			if (s_logger.isDebugEnabled())
-				s_logger.debug("Stopping console proxy " + proxy.getName() + " failed : exception " + e.toString());
-			return false;
-		}
-	}
-
-	@Override
-	public boolean rebootProxy(long proxyVmId, long startEventId) {
-		AsyncJobExecutor asyncExecutor = BaseAsyncJobExecutor.getCurrentExecutor();
-		if (asyncExecutor != null) {
-			AsyncJobVO job = asyncExecutor.getJob();
-
-			if (s_logger.isInfoEnabled())
-				s_logger.info("Reboot console proxy " + proxyVmId + ", update async job-" + job.getId());
-			_asyncMgr.updateAsyncJobAttachment(job.getId(), "console_proxy", proxyVmId);
-		}
-
-		final ConsoleProxyVO proxy = _consoleProxyDao.findById(proxyVmId);
-
-		if (proxy == null || proxy.getState() == State.Destroyed) {
-			return false;
-		}
-
-/*
-		saveStartedEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM,
-				EventTypes.EVENT_PROXY_REBOOT,
-				"Rebooting console proxy with Id: " + proxyVmId, startEventId);
-*/
-		if (proxy.getState() == State.Running && proxy.getHostId() != null) {
-			final RebootCommand cmd = new RebootCommand(proxy.getInstanceName());
-			final Answer answer = _agentMgr.easySend(proxy.getHostId(), cmd);
-
-			if (answer != null) {
-				if (s_logger.isDebugEnabled())
-					s_logger.debug("Successfully reboot console proxy " + proxy.getName());
-
-				SubscriptionMgr.getInstance().notifySubscribers(
-						ConsoleProxyManager.ALERT_SUBJECT,
-						this,
-						new ConsoleProxyAlertEventArgs(
-								ConsoleProxyAlertEventArgs.PROXY_REBOOTED,
-								proxy.getDataCenterId(), proxy.getId(), proxy,
-								null)
-						);
-
-				final EventVO event = new EventVO();
-				event.setUserId(User.UID_SYSTEM);
-				event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-				event.setType(EventTypes.EVENT_PROXY_REBOOT);
-				event.setLevel(EventVO.LEVEL_INFO);
-				event.setStartId(startEventId);
-				event.setDescription("Console proxy rebooted - " + proxy.getName());
-				_eventDao.persist(event);
-				return true;
-			} else {
-				if (s_logger.isDebugEnabled())
-					s_logger.debug("failed to reboot console proxy : " + proxy.getName());
-
-				final EventVO event = new EventVO();
-				event.setUserId(User.UID_SYSTEM);
-				event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-				event.setType(EventTypes.EVENT_PROXY_REBOOT);
-				event.setLevel(EventVO.LEVEL_ERROR);
-				event.setStartId(startEventId);
-				event.setDescription("Rebooting console proxy failed - " + proxy.getName());
-				_eventDao.persist(event);
-				return false;
-			}
-		} else {
-			return startProxy(proxyVmId, 0) != null;
-		}
-	}
-
-	@Override
-	public boolean destroy(ConsoleProxyVO proxy) throws AgentUnavailableException {
-		return destroyProxy(proxy.getId(), 0);
-	}
-
-	@Override
-	@DB
-	public boolean destroyProxy(long vmId, long startEventId) {
-		AsyncJobExecutor asyncExecutor = BaseAsyncJobExecutor.getCurrentExecutor();
-		if (asyncExecutor != null) {
-			AsyncJobVO job = asyncExecutor.getJob();
-
-			if (s_logger.isInfoEnabled())
-				s_logger.info("Destroy console proxy " + vmId + ", update async job-" + job.getId());
-			_asyncMgr.updateAsyncJobAttachment(job.getId(), "console_proxy", vmId);
-		}
-
-		ConsoleProxyVO vm = _consoleProxyDao.findById(vmId);
-		if (vm == null || vm.getState() == State.Destroyed) {
-			if (s_logger.isDebugEnabled()) {
-				s_logger.debug("Unable to find vm or vm is destroyed: " + vmId);
-			}
-			return true;
-		}
-/*		
-		saveStartedEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM,
-				EventTypes.EVENT_PROXY_DESTROY,
-				"Destroying console proxy with Id: " + vmId, startEventId);
-*/				
-		if (s_logger.isDebugEnabled()) {
-			s_logger.debug("Destroying console proxy vm " + vmId);
-		} 
-
-		if (!_consoleProxyDao.updateIf(vm, Event.DestroyRequested, null)) {
-			s_logger.debug("Unable to destroy the vm because it is not in the correct state: " + vmId);
-			return false;
-		}
-
-		Transaction txn = Transaction.currentTxn();
-		List vols = null;
-		try {
-			vols = _volsDao.findByInstance(vmId);
-			if (vols.size() != 0) {
-				_storageMgr.destroy(vm, vols);
-			}
-
-			return true;
-		} finally {
-			try {
-				txn.start();
-				// release critical system resources used by the VM before we
-				// delete them
-				if (vm.getPublicIpAddress() != null)
-					freePublicIpAddress(vm.getPublicIpAddress(), vm.getDataCenterId(), vm.getPodId());
-				vm.setPublicIpAddress(null);
-
-				_consoleProxyDao.remove(vm.getId());
-
-				final EventVO event = new EventVO();
-				event.setUserId(User.UID_SYSTEM);
-				event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-				event.setType(EventTypes.EVENT_PROXY_DESTROY);
-				event.setLevel(EventVO.LEVEL_INFO);
-				event.setStartId(startEventId);
-				event.setDescription("Console proxy destroyed - "
-						+ vm.getName());
-				_eventDao.persist(event);
-
-				txn.commit();
-			} catch (Exception e) {
-				s_logger.error("Caught this error: ", e);
-				txn.rollback();
-				return false;
-			} finally {
-				s_logger.debug("console proxy vm is destroyed : "
-						+ vm.getName());
-			}
-		}
-	}
-
-	@DB
-	public boolean destroyProxyDBOnly(long vmId) {
-		Transaction txn = Transaction.currentTxn();
-		try {
-			txn.start();
-			_volsDao.deleteVolumesByInstance(vmId);
-
-			ConsoleProxyVO proxy = _consoleProxyDao.findById(vmId);
-			if (proxy != null) {
-				if (proxy.getPublicIpAddress() != null)
-					freePublicIpAddress(proxy.getPublicIpAddress(), proxy.getDataCenterId(), proxy.getPodId());
-
-				_consoleProxyDao.remove(vmId);
-
-				final EventVO event = new EventVO();
-				event.setUserId(User.UID_SYSTEM);
-				event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-				event.setType(EventTypes.EVENT_PROXY_DESTROY);
-				event.setLevel(EventVO.LEVEL_INFO);
-				event.setDescription("Console proxy destroyed - "
-						+ proxy.getName());
-				_eventDao.persist(event);
-			}
-
-			txn.commit();
-			return true;
-		} catch (Exception e) {
-			s_logger.error("Caught this error: ", e);
-			txn.rollback();
-			return false;
-		} finally {
-			s_logger.debug("console proxy vm is destroyed from DB : " + vmId);
-		}
-	}
-
-	@Override
-	public boolean stop(ConsoleProxyVO proxy, long startEventId)
-			throws AgentUnavailableException {
-		if (!_consoleProxyDao.updateIf(proxy, Event.StopRequested, proxy.getHostId())) {
-			s_logger.debug("Unable to stop console proxy: " + proxy.toString());
-			return false;
-		}
-
-		// IPAddressVO ip = _ipAddressDao.findById(proxy.getPublicIpAddress());
-		// VlanVO vlan = _vlanDao.findById(new Long(ip.getVlanDbId()));
-
-		GlobalLock proxyLock = GlobalLock.getInternLock(getProxyLockName(proxy.getId()));
-		try {
-			if (proxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
-				try {
-					StopCommand cmd = new StopCommand(proxy, true, Integer.toString(_consoleProxyPort), 
-							Integer.toString(_consoleProxyUrlPort), proxy.getPublicIpAddress());
-					try {
-						Long proxyHostId = proxy.getHostId();
-						if (proxyHostId == null) {
-							s_logger.debug("Unable to stop due to proxy " + proxy.getId()
-								+ " as host is no longer available, proxy may already have been stopped");
-							return false;
-						}
-						StopAnswer answer = (StopAnswer) _agentMgr.send(proxyHostId, cmd);
-						if (answer == null || !answer.getResult()) {
-							s_logger.debug("Unable to stop due to " + (answer == null ? "answer is null" : answer.getDetails()));
-
-							final EventVO event = new EventVO();
-							event.setUserId(User.UID_SYSTEM);
-							event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-							event.setType(EventTypes.EVENT_PROXY_STOP);
-							event.setLevel(EventVO.LEVEL_ERROR);
-							event.setStartId(startEventId);
-							event.setDescription("Stopping console proxy failed due to negative answer from agent - " + proxy.getName());
-							_eventDao.persist(event);
-							return false;
-						}
-						completeStopCommand(proxy, Event.OperationSucceeded);
-
-						SubscriptionMgr.getInstance().notifySubscribers(
-								ConsoleProxyManager.ALERT_SUBJECT,
-								this,
-								new ConsoleProxyAlertEventArgs(
-										ConsoleProxyAlertEventArgs.PROXY_DOWN,
-										proxy.getDataCenterId(), proxy.getId(),
-										proxy, null));
-
-						final EventVO event = new EventVO();
-						event.setUserId(User.UID_SYSTEM);
-						event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-						event.setType(EventTypes.EVENT_PROXY_STOP);
-						event.setLevel(EventVO.LEVEL_INFO);
-						event.setStartId(startEventId);
-						event.setDescription("Console proxy stopped - " + proxy.getName());
-						_eventDao.persist(event);
-						return true;
-					} catch (OperationTimedoutException e) {
-						final EventVO event = new EventVO();
-						event.setUserId(User.UID_SYSTEM);
-						event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-						event.setType(EventTypes.EVENT_PROXY_STOP);
-						event.setLevel(EventVO.LEVEL_ERROR);
-						event.setStartId(startEventId);
-						event.setDescription("Stopping console proxy failed due to operation time out - " + proxy.getName());
-						_eventDao.persist(event);
-						throw new AgentUnavailableException(proxy.getHostId());
-					}
-				} finally {
-					proxyLock.unlock();
-				}
-			} else {
-				s_logger.debug("Unable to acquire console proxy lock : " + proxy.toString());
-				return false;
-			}
-		} finally {
-			proxyLock.releaseRef();
-		}
-	}
-
-	@Override
-	public boolean migrate(ConsoleProxyVO proxy, HostVO host) {
-		HostVO fromHost = _hostDao.findById(proxy.getId());
-
-		if (!_consoleProxyDao.updateIf(proxy, Event.MigrationRequested, proxy.getHostId())) {
-			s_logger.debug("State for " + proxy.toString() + " has changed so migration can not take place.");
-			return false;
-		}
-
-		MigrateCommand cmd = new MigrateCommand(proxy.getInstanceName(), host.getPrivateIpAddress(), false);
-		Answer answer = _agentMgr.easySend(fromHost.getId(), cmd);
-		if (answer == null) {
-			return false;
-		}
-
-		_storageMgr.unshare(proxy, fromHost);
-
-		return true;
-	}
-
-	@Override
-	public boolean completeMigration(ConsoleProxyVO proxy, HostVO host)
-			throws AgentUnavailableException, OperationTimedoutException {
-
-		CheckVirtualMachineCommand cvm = new CheckVirtualMachineCommand(proxy.getInstanceName());
-		CheckVirtualMachineAnswer answer = (CheckVirtualMachineAnswer) _agentMgr.send(host.getId(), cvm);
-		if (!answer.getResult()) {
-			s_logger.debug("Unable to complete migration for " + proxy.getId());
-			_consoleProxyDao.updateIf(proxy, Event.AgentReportStopped, null);
-			return false;
-		}
-
-		State state = answer.getState();
-		if (state == State.Stopped) {
-			s_logger.warn("Unable to complete migration as we can not detect it on " + host.getId());
-			_consoleProxyDao.updateIf(proxy, Event.AgentReportStopped, null);
-			return false;
-		}
-
-		_consoleProxyDao.updateIf(proxy, Event.OperationSucceeded, host.getId());
-		return true;
-	}
-
-	@Override
-	public HostVO prepareForMigration(ConsoleProxyVO proxy)
-			throws StorageUnavailableException {
-
-		VMTemplateVO template = _templateDao.findById(proxy.getTemplateId());
-		long routerId = proxy.getId();
-		boolean mirroredVols = proxy.isMirroredVols();
-		DataCenterVO dc = _dcDao.findById(proxy.getDataCenterId());
-		HostPodVO pod = _podDao.findById(proxy.getPodId());
-		List sps = _storageMgr.getStoragePoolsForVm(proxy.getId());
-		StoragePoolVO sp = sps.get(0); // FIXME
-
-		List vols = _volsDao.findCreatedByInstance(routerId);
-
-		String[] storageIps = new String[2];
-		VolumeVO vol = vols.get(0);
-		storageIps[0] = vol.getHostIp();
-		if (mirroredVols && (vols.size() == 2)) {
-			storageIps[1] = vols.get(1).getHostIp();
-		}
-
-		PrepareForMigrationCommand cmd = new PrepareForMigrationCommand(proxy.getName(), null, storageIps, vols, mirroredVols);
-
-		HostVO routingHost = null;
-		HashSet avoid = new HashSet();
-
-		HostVO fromHost = _hostDao.findById(proxy.getHostId());
-		if (fromHost.getClusterId() == null) {
-			s_logger.debug("The host is not in a cluster");
-			return null;
-		}
-		avoid.add(fromHost);
-
-		while ((routingHost = (HostVO) _agentMgr.findHost(Host.Type.Routing, dc, pod, sp, _serviceOffering, template, proxy, fromHost, avoid)) != null) {
-			avoid.add(routingHost);
-
-			if (s_logger.isDebugEnabled()) {
-				s_logger.debug("Trying to migrate router to host " + routingHost.getName());
-			}
-
-			if (!_storageMgr.share(proxy, vols, routingHost, false)) {
-				s_logger.warn("Can not share " + proxy.getName());
-				throw new StorageUnavailableException(vol.getPoolId());
-			}
-
-			Answer answer = _agentMgr.easySend(routingHost.getId(), cmd);
-			if (answer != null && answer.getResult()) {
-				return routingHost;
-			}
-			_storageMgr.unshare(proxy, vols, routingHost);
-		}
-
-		return null;
-	}
-
-	private String getCapacityScanLockName() {
-		// to improve security, it may be better to return a unique mashed
-		// name(for example MD5 hashed)
-		return "consoleproxy.capacity.scan";
-	}
-
-	private String getAllocProxyLockName() {
-		// to improve security, it may be better to return a unique mashed
-		// name(for example MD5 hashed)
-		return "consoleproxy.alloc";
-	}
-
-	private String getProxyLockName(long id) {
-		return "consoleproxy." + id;
-	}
-
-	private Long saveStartedEvent(Long userId, Long accountId, String type,
-			String description, long startEventId) {
-		EventVO event = new EventVO();
-		event.setUserId(userId);
-		event.setAccountId(accountId);
-		event.setType(type);
-		event.setState(EventState.Started);
-		event.setDescription(description);
-		event.setStartId(startEventId);
-		event = _eventDao.persist(event);
-		if (event != null)
-			return event.getId();
-		return null;
-	}
-	
-	@Override
-	public boolean destroyConsoleProxy(DestroyConsoleProxyCmd cmd) throws ServerApiException{
-	    Long proxyId = cmd.getId();
-	    
-	    // verify parameters
+public class ConsoleProxyManagerImpl implements ConsoleProxyManager, VirtualMachineManager, AgentHook {
+    private static final Logger s_logger = Logger.getLogger(ConsoleProxyManagerImpl.class);
+
+    private static final int DEFAULT_FIND_HOST_RETRY_COUNT = 2;
+    private static final int DEFAULT_CAPACITY_SCAN_INTERVAL = 30000; // 30
+    // seconds
+    private static final int EXECUTOR_SHUTDOWN_TIMEOUT = 1000; // 1 second
+
+    private static final int ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_COOPERATION = 3; // 3
+    // seconds
+    private static final int ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC = 180; // 3
+    // minutes
+
+    private static final int API_WAIT_TIMEOUT = 5000; // 5 seconds (in
+    // milliseconds)
+    private static final int STARTUP_DELAY = 60000; // 60 seconds
+
+    private int _consoleProxyPort = ConsoleProxyManager.DEFAULT_PROXY_VNC_PORT;
+    private int _consoleProxyUrlPort = ConsoleProxyManager.DEFAULT_PROXY_URL_PORT;
+
+    private String _mgmt_host;
+    private int _mgmt_port = 8250;
+
+    private String _name;
+    private Adapters _consoleProxyAllocators;
+
+    @Inject
+    private ConsoleProxyDao _consoleProxyDao;
+    @Inject
+    private DataCenterDao _dcDao;
+    @Inject
+    private VlanDao _vlanDao;
+    @Inject
+    private VMTemplateDao _templateDao;
+    @Inject
+    private IPAddressDao _ipAddressDao;
+    @Inject
+    private VolumeDao _volsDao;
+    @Inject
+    private HostPodDao _podDao;
+    @Inject
+    private HostDao _hostDao;
+    @Inject
+    private ConfigurationDao _configDao;
+
+    @Inject
+    private VMInstanceDao _instanceDao;
+    @Inject
+    private AccountDao _accountDao;
+
+    @Inject
+    private VMTemplateHostDao _vmTemplateHostDao;
+
+    @Inject
+    private AgentManager _agentMgr;
+    @Inject
+    private StorageManager _storageMgr;
+    @Inject
+    private HighAvailabilityManager _haMgr;
+    @Inject NetworkManager _networkMgr;
+    
+    @Inject AccountManager _accountMgr;
+    @Inject
+    private EventDao _eventDao;
+    @Inject
+    ServiceOfferingDao _offeringDao;
+    @Inject
+    NetworkOfferingDao _networkOfferingDao;
+    private IpAddrAllocator _IpAllocator;
+
+    private ConsoleProxyListener _listener;
+
+    private ServiceOfferingVO _serviceOffering;
+    private int _networkRate;
+    private int _multicastRate;
+    private VMTemplateVO _template;
+    
+    NetworkOfferingVO _publicNetworkOffering;
+    NetworkOfferingVO _managementNetworkOffering;
+    NetworkOfferingVO _linkLocalNetworkOffering;
+
+    @Inject
+    private AsyncJobManager _asyncMgr;
+    
+    @Inject
+    private VmManager _vmMgr;
+
+    private final ScheduledExecutorService _capacityScanScheduler = Executors.newScheduledThreadPool(1, new NamedThreadFactory("CP-Scan"));
+    private final ExecutorService _requestHandlerScheduler = Executors.newCachedThreadPool(new NamedThreadFactory("Request-handler"));
+
+    private long _capacityScanInterval = DEFAULT_CAPACITY_SCAN_INTERVAL;
+    private int _capacityPerProxy = ConsoleProxyManager.DEFAULT_PROXY_CAPACITY;
+    private int _standbyCapacity = ConsoleProxyManager.DEFAULT_STANDBY_CAPACITY;
+
+    private int _proxyRamSize;
+    private int _find_host_retry = DEFAULT_FIND_HOST_RETRY_COUNT;
+    private int _ssh_retry;
+    private int _ssh_sleep;
+    private boolean _use_lvm;
+    private boolean _use_storage_vm;
+
+    private String _domain;
+    private String _instance;
+
+    // private String _privateNetmask;
+    private int _proxyCmdPort = DEFAULT_PROXY_CMD_PORT;
+    private int _proxySessionTimeoutValue = DEFAULT_PROXY_SESSION_TIMEOUT;
+    private boolean _sslEnabled = false;
+
+    private final GlobalLock _capacityScanLock = GlobalLock.getInternLock(getCapacityScanLockName());
+    private final GlobalLock _allocProxyLock = GlobalLock.getInternLock(getAllocProxyLockName());
+
+    @Override
+    public ConsoleProxyInfo assignProxy(final long dataCenterId, final long vmId) {
+
+        final Pair result = new Pair(this, null);
+
+        _requestHandlerScheduler.execute(new Runnable() {
+            @Override
+            public void run() {
+                Transaction txn = Transaction.open(Transaction.CLOUD_DB);
+                try {
+                    ConsoleProxyVO proxy = doAssignProxy(dataCenterId, vmId);
+                    synchronized (result) {
+                        result.second(proxy);
+                        result.notifyAll();
+                    }
+                } catch (Throwable e) {
+                    s_logger.warn("Unexpected exception " + e.getMessage(), e);
+                } finally {
+                    StackMaid.current().exitCleanup();
+                    txn.close();
+                }
+            }
+        });
+
+        synchronized (result) {
+            try {
+                result.wait(API_WAIT_TIMEOUT);
+            } catch (InterruptedException e) {
+                s_logger.info("Waiting for console proxy assignment is interrupted");
+            }
+        }
+        
+        ConsoleProxyVO proxy = result.second();
+        if (proxy == null)
+            return null;
+        
+        return new ConsoleProxyInfo(proxy.isSslEnabled(), proxy.getPublicIpAddress(), _consoleProxyPort, proxy.getPort());
+    }
+
+    public ConsoleProxyVO doAssignProxy(long dataCenterId, long vmId) {
+        ConsoleProxyVO proxy = null;
+        VMInstanceVO vm = _instanceDao.findById(vmId);
+        if (vm == null) {
+            s_logger.warn("VM " + vmId + " no longer exists, return a null proxy for vm:" + vmId);
+            return null;
+        }
+
+        Boolean[] proxyFromStoppedPool = new Boolean[1];
+        if (_allocProxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
+            try {
+                proxy = getOrAllocProxyResource(dataCenterId, vmId, proxyFromStoppedPool);
+            } finally {
+                _allocProxyLock.unlock();
+            }
+        } else {
+            s_logger.error("Unable to acquire synchronization lock to get/allocate proxy resource for vm :" + vmId
+                    + ". Previous console proxy allocation is taking too long");
+        }
+
+        if (proxy == null) {
+            s_logger.warn("Unable to find or allocate console proxy resource");
+            return null;
+        }
+
+        long proxyVmId = proxy.getId();
+        GlobalLock proxyLock = GlobalLock.getInternLock(getProxyLockName(proxyVmId));
+        try {
+            if (proxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
+                try {
+                    proxy = startProxy(proxyVmId, 0);
+
+                    if (proxy == null) {
+                        //
+                        // We had a situation with multi-pod configuration,
+                        // where
+                        // storage allocation of the console proxy VM may
+                        // succeed, but later-on starting of it
+                        // may fail because of running out of computing resource
+                        // (CPU/memory). We
+                        // currently don't support moving storage to another pod
+                        // on the fly, to deal
+                        // with the situation we will destroy this proxy VM and
+                        // let it the whole proxy VM
+                        // creation process re-start again, by hoping that new
+                        // storage and computing
+                        // resource may be allocated and assigned in another pod
+                        //
+                        if (s_logger.isInfoEnabled())
+                            s_logger.info("Unable to start console proxy, proxy vm Id : " + proxyVmId + " will recycle it and restart a new one");
+                        destroyProxy(proxyVmId, 0);
+                        return null;
+                    } else {
+                        if (s_logger.isTraceEnabled())
+                            s_logger.trace("Console proxy " + proxy.getName() + " is started");
+
+                        // if it is a new assignment or a changed assignment,
+                        // update the
+                        // record
+                        if (vm.getProxyId() == null || vm.getProxyId().longValue() != proxy.getId())
+                            _instanceDao.updateProxyId(vmId, proxy.getId(), DateUtil.currentGMTTime());
+
+                        proxy.setSslEnabled(_sslEnabled);
+                        if (_sslEnabled)
+                            proxy.setPort(443);
+                        else
+                            proxy.setPort(80);
+                        return proxy;
+                    }
+                } finally {
+                    proxyLock.unlock();
+                }
+            } else {
+                s_logger.error("Unable to acquire synchronization lock to start console proxy " + proxyVmId + " for vm: " + vmId
+                        + ". It takes too long to start the proxy");
+
+                return null;
+            }
+        } finally {
+            proxyLock.releaseRef();
+        }
+    }
+
+    private ConsoleProxyVO getOrAllocProxyResource(long dataCenterId, long vmId, Boolean[] proxyFromStoppedPool) {
+        ConsoleProxyVO proxy = null;
+        VMInstanceVO vm = this._instanceDao.findById(vmId);
+
+        if (vm != null && vm.getState() != State.Running) {
+            if (s_logger.isInfoEnabled())
+                s_logger.info("Detected that vm : " + vmId + " is not currently at running state, we will fail the proxy assignment for it");
+            return null;
+        }
+
+        if (vm != null && vm.getProxyId() != null) {
+            proxy = _consoleProxyDao.findById(vm.getProxyId());
+
+            if (proxy != null) {
+                if (!isInAssignableState(proxy)) {
+                    if (s_logger.isInfoEnabled())
+                        s_logger.info("A previous assigned proxy is not assignable now, reassign console proxy for user vm : " + vmId);
+                    proxy = null;
+                } else {
+                    if (_consoleProxyDao.getProxyActiveLoad(proxy.getId()) < _capacityPerProxy || hasPreviousSession(proxy, vm)) {
+                        if (s_logger.isTraceEnabled())
+                            s_logger.trace("Assign previous allocated console proxy for user vm : " + vmId);
+
+                        if (proxy.getActiveSession() >= _capacityPerProxy)
+                            s_logger.warn("Assign overloaded proxy to user VM as previous session exists, user vm : " + vmId);
+                    } else {
+                        proxy = null;
+                    }
+                }
+            }
+        }
+
+        if (proxy == null)
+            proxy = assignProxyFromRunningPool(dataCenterId);
+
+        if (proxy == null) {
+            if (s_logger.isInfoEnabled())
+                s_logger.info("No running console proxy is available, check to see if we can bring up a stopped one for data center : "
+                        + dataCenterId);
+
+            proxy = assignProxyFromStoppedPool(dataCenterId);
+            if (proxy == null) {
+                if (s_logger.isInfoEnabled())
+                    s_logger.info("No stopped console proxy is available, need to allocate a new console proxy for data center : " + dataCenterId);
+
+                proxy = startNew(dataCenterId);
+            } else {
+                if (s_logger.isInfoEnabled())
+                    s_logger.info("Found a stopped console proxy, bring it up to running pool. proxy vm id : " + proxy.getId() + ", data center : "
+                            + dataCenterId);
+
+                proxyFromStoppedPool[0] = new Boolean(true);
+            }
+        }
+
+        return proxy;
+    }
+
+    private static boolean isInAssignableState(ConsoleProxyVO proxy) {
+        // console proxies that are in states of being able to serve user VM
+        State state = proxy.getState();
+        if (state == State.Running || state == State.Starting || state == State.Creating || state == State.Migrating)
+            return true;
+
+        return false;
+    }
+
+    private boolean hasPreviousSession(ConsoleProxyVO proxy, VMInstanceVO vm) {
+
+        ConsoleProxyStatus status = null;
+        try {
+            GsonBuilder gb = new GsonBuilder();
+            gb.setVersion(1.3);
+            Gson gson = gb.create();
+
+            byte[] details = proxy.getSessionDetails();
+            status = gson.fromJson(details != null ? new String(details, Charset.forName("US-ASCII")) : null, ConsoleProxyStatus.class);
+        } catch (Throwable e) {
+            s_logger.warn("Unable to parse proxy session details : " + proxy.getSessionDetails());
+        }
+
+        if (status != null && status.getConnections() != null) {
+            ConsoleProxyConnectionInfo[] connections = status.getConnections();
+            for (int i = 0; i < connections.length; i++) {
+                long taggedVmId = 0;
+                if (connections[i].tag != null) {
+                    try {
+                        taggedVmId = Long.parseLong(connections[i].tag);
+                    } catch (NumberFormatException e) {
+                        s_logger.warn("Unable to parse console proxy connection info passed through tag: " + connections[i].tag, e);
+                    }
+                }
+                if (taggedVmId == vm.getId())
+                    return true;
+            }
+
+            //
+            // even if we are not in the list, it may because we haven't
+            // received load-update yet
+            // wait until session time
+            //
+            if (DateUtil.currentGMTTime().getTime() - vm.getProxyAssignTime().getTime() < _proxySessionTimeoutValue)
+                return true;
+
+            return false;
+        } else {
+            s_logger.error("No proxy load info on an overloaded proxy ?");
+            return false;
+        }
+    }
+    
+    @Override
+    public ConsoleProxyVO startProxy(long proxyVmId, long startEventId) {
+        try {
+            return start(proxyVmId, startEventId);
+        } catch (StorageUnavailableException e) {
+            s_logger.warn("Exception while trying to start console proxy", e);
+            return null;
+        } catch (InsufficientCapacityException e) {
+            s_logger.warn("Exception while trying to start console proxy", e);
+            return null;
+        } catch (ConcurrentOperationException e) {
+            s_logger.warn("Exception while trying to start console proxy", e);
+            return null;
+        }
+    }
+
+    public ConsoleProxyVO start2(long proxyVmId, long startEventId) throws StorageUnavailableException, InsufficientCapacityException, ConcurrentOperationException {
+        ConsoleProxyVO proxy = _consoleProxyDao.findById(proxyVmId);
+        DeploymentPlan plan = new DataCenterDeployment(proxy.getDataCenterId(), 1);
+        return _vmMgr.start(proxy, plan);
+    }
+
+    @Override
+    @DB
+    public ConsoleProxyVO start(long proxyId, long startEventId) throws StorageUnavailableException, InsufficientCapacityException,
+            ConcurrentOperationException {
+
+        AsyncJobExecutor asyncExecutor = BaseAsyncJobExecutor.getCurrentExecutor();
+        if (asyncExecutor != null) {
+            AsyncJobVO job = asyncExecutor.getJob();
+
+            if (s_logger.isInfoEnabled())
+                s_logger.info("Start console proxy " + proxyId + ", update async job-" + job.getId());
+            _asyncMgr.updateAsyncJobAttachment(job.getId(), "console_proxy", proxyId);
+        }
+
+        ConsoleProxyVO proxy = _consoleProxyDao.findById(proxyId);
+        if (proxy == null || proxy.getRemoved() != null) {
+            s_logger.debug("proxy is not found: " + proxyId);
+            return null;
+        }
+        /*
+         * // don't insert event here, it may be called multiple times!
+         * saveStartedEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM,
+         * EventTypes.EVENT_PROXY_START, "Starting console proxy with Id: " +
+         * proxyId, startEventId);
+         */
+
+        if (s_logger.isTraceEnabled()) {
+            s_logger.trace("Starting console proxy if it is not started, proxy vm id : " + proxyId);
+        }
+
+        for (int i = 0; i < 2; i++) {
+
+            State state = proxy.getState();
+
+            if (state == State.Starting /* || state == State.Migrating */) {
+                if (s_logger.isDebugEnabled())
+                    s_logger.debug("Waiting console proxy to be ready, proxy vm id : " + proxyId + " proxy VM state : " + state.toString());
+
+                if (proxy.getPrivateIpAddress() == null || connect(proxy.getPrivateIpAddress(), _proxyCmdPort) != null) {
+                    if (proxy.getPrivateIpAddress() == null)
+                        s_logger.warn("Retruning a proxy that is being started but private IP has not been allocated yet, proxy vm id : " + proxyId);
+                    else
+                        s_logger.warn("Waiting console proxy to be ready timed out, proxy vm id : " + proxyId);
+
+                    // TODO, it is very tricky here, if the startup process
+                    // takes too long and it timed out here,
+                    // we may give back a proxy that is not fully ready for
+                    // functioning
+                }
+                return proxy;
+            }
+
+            if (state == State.Running) {
+                if (s_logger.isTraceEnabled())
+                    s_logger.trace("Console proxy is already started: " + proxy.getName());
+                return proxy;
+            }
+
+            DataCenterVO dc = _dcDao.findById(proxy.getDataCenterId());
+            HostPodVO pod = _podDao.findById(proxy.getPodId());
+            List sps = _storageMgr.getStoragePoolsForVm(proxy.getId());
+            StoragePoolVO sp = sps.get(0); // FIXME
+
+            HashSet avoid = new HashSet();
+            HostVO routingHost = (HostVO) _agentMgr.findHost(Host.Type.Routing, dc, pod, sp, _serviceOffering, _template, proxy, null, avoid);
+
+            if (routingHost == null) {
+                if (s_logger.isDebugEnabled()) {
+                    String msg = "Unable to find a routing host for " + proxy.toString() + " in pod " + pod.getId();
+                    s_logger.debug(msg);
+                    throw new CloudRuntimeException(msg);
+                }
+            }
+            // to ensure atomic state transition to Starting state
+            if (!_consoleProxyDao.updateIf(proxy, Event.StartRequested, routingHost.getId())) {
+                if (s_logger.isDebugEnabled()) {
+                    ConsoleProxyVO temp = _consoleProxyDao.findById(proxyId);
+                    s_logger.debug("Unable to start console proxy " + proxy.getName() + " because it is not in a startable state : "
+                            + ((temp != null) ? temp.getState().toString() : "null"));
+                }
+                continue;
+            }
+
+            try {
+                Answer answer = null;
+                int retry = _find_host_retry;
+
+                // Console proxy VM will be running at routing hosts as routing
+                // hosts have public access to outside network
+                do {
+                    if (s_logger.isDebugEnabled()) {
+                        s_logger.debug("Trying to start console proxy on host " + routingHost.getName());
+                    }
+
+                    String privateIpAddress = allocPrivateIpAddress(proxy.getDataCenterId(), routingHost.getPodId(), proxy.getId(),
+                            proxy.getPrivateMacAddress());
+                    if (privateIpAddress == null && (_IpAllocator != null && !_IpAllocator.exteralIpAddressAllocatorEnabled())) {
+                        String msg = "Unable to allocate private ip addresses for  " + proxy.getName() + " in pod " + pod.getId();
+                        s_logger.debug(msg);
+                        throw new CloudRuntimeException(msg);
+                    }
+
+                    proxy.setPrivateIpAddress(privateIpAddress);
+                    String guestIpAddress = _dcDao.allocateLinkLocalPrivateIpAddress(proxy.getDataCenterId(), routingHost.getPodId(), proxy.getId());
+                    proxy.setGuestIpAddress(guestIpAddress);
+
+                    _consoleProxyDao.updateIf(proxy, Event.OperationRetry, routingHost.getId());
+                    proxy = _consoleProxyDao.findById(proxy.getId());
+
+                    List vols = _storageMgr.prepare(proxy, routingHost);
+                    if (vols == null || vols.size() == 0) {
+                        String msg = "Unable to prepare storage for " + proxy.getName() + " in pod " + pod.getId();
+                        s_logger.debug(msg);
+                        throw new CloudRuntimeException(msg);
+                    }
+
+                    // _storageMgr.share(proxy, vols, null, true);
+
+                    // carry the console proxy port info over so that we don't
+                    // need to configure agent on this
+                    StartConsoleProxyCommand cmdStart = new StartConsoleProxyCommand(_networkRate, _multicastRate,
+                            _proxyCmdPort, proxy, proxy.getName(), "", vols, Integer.toString(_consoleProxyPort),
+                            Integer.toString(_consoleProxyUrlPort), _mgmt_host, _mgmt_port, _sslEnabled);
+                    if (s_logger.isDebugEnabled())
+                        s_logger.debug("Sending start command for console proxy " + proxy.getName() + " to " + routingHost.getName());
+                    try {
+                        answer = _agentMgr.send(routingHost.getId(), cmdStart);
+
+                        s_logger.debug("StartConsoleProxy Answer: " + (answer != null ? answer : "null"));
+
+                        if (s_logger.isDebugEnabled())
+                            s_logger.debug("Received answer on starting console proxy " + proxy.getName() + " on " + routingHost.getName());
+
+                        if (answer != null && answer.getResult()) {
+                            if (s_logger.isDebugEnabled()) {
+                                s_logger.debug("Console proxy " + proxy.getName() + " started on " + routingHost.getName());
+                            }
+
+                            if (answer instanceof StartConsoleProxyAnswer) {
+                                StartConsoleProxyAnswer rAnswer = (StartConsoleProxyAnswer) answer;
+                                if (rAnswer.getPrivateIpAddress() != null) {
+                                    proxy.setPrivateIpAddress(rAnswer.getPrivateIpAddress());
+                                }
+                                if (rAnswer.getPrivateMacAddress() != null) {
+                                    proxy.setPrivateMacAddress(rAnswer.getPrivateMacAddress());
+                                }
+                            }
+
+                            final EventVO event = new EventVO();
+                            event.setUserId(User.UID_SYSTEM);
+                            event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
+                            event.setType(EventTypes.EVENT_PROXY_START);
+                            event.setLevel(EventVO.LEVEL_INFO);
+                            event.setStartId(startEventId);
+                            event.setDescription("Console proxy started - " + proxy.getName());
+                            _eventDao.persist(event);
+                            break;
+                        }
+                        s_logger.debug("Unable to start " + proxy.toString() + " on host " + routingHost.toString() + " due to "
+                                + answer.getDetails());
+                    } catch (OperationTimedoutException e) {
+                        if (e.isActive()) {
+                            s_logger.debug("Unable to start vm " + proxy.getName()
+                                    + " due to operation timed out and it is active so scheduling a restart.");
+                            _haMgr.scheduleRestart(proxy, true);
+                            return null;
+                        }
+                    } catch (AgentUnavailableException e) {
+                        s_logger.debug("Agent " + routingHost.toString() + " was unavailable to start VM " + proxy.getName());
+                    }
+
+                    avoid.add(routingHost);
+                    proxy.setPrivateIpAddress(null);
+                    freePrivateIpAddress(privateIpAddress, proxy.getDataCenterId(), proxy.getId());
+                    proxy.setGuestIpAddress(null);
+                    _dcDao.releaseLinkLocalPrivateIpAddress(guestIpAddress, proxy.getDataCenterId(), proxy.getId());
+                    _storageMgr.unshare(proxy, vols, routingHost);
+                } while (--retry > 0
+                        && (routingHost = (HostVO) _agentMgr
+                                .findHost(Host.Type.Routing, dc, pod, sp, _serviceOffering, _template, proxy, null, avoid)) != null);
+                if (routingHost == null || retry <= 0) {
+
+                    SubscriptionMgr.getInstance().notifySubscribers(
+                            ConsoleProxyManager.ALERT_SUBJECT,
+                            this,
+                            new ConsoleProxyAlertEventArgs(ConsoleProxyAlertEventArgs.PROXY_START_FAILURE, proxy.getDataCenterId(), proxy.getId(),
+                                    proxy, "Unable to find a routing host to run"));
+
+                    final EventVO event = new EventVO();
+                    event.setUserId(User.UID_SYSTEM);
+                    event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
+                    event.setType(EventTypes.EVENT_PROXY_START);
+                    event.setLevel(EventVO.LEVEL_ERROR);
+                    event.setStartId(startEventId);
+                    event.setDescription("Starting console proxy failed due to unable to find a host - " + proxy.getName());
+                    _eventDao.persist(event);
+                    throw new ExecutionException("Couldn't find a routingHost to run console proxy");
+                }
+
+                _consoleProxyDao.updateIf(proxy, Event.OperationSucceeded, routingHost.getId());
+                if (s_logger.isDebugEnabled()) {
+                    s_logger.debug("Console proxy is now started, vm id : " + proxy.getId());
+                }
+
+                // If starting the console proxy failed due to the external
+                // firewall not being reachable, send an alert.
+                if (answer != null && answer.getDetails() != null && answer.getDetails().equals("firewall")) {
+
+                    SubscriptionMgr.getInstance().notifySubscribers(
+                            ConsoleProxyManager.ALERT_SUBJECT,
+                            this,
+                            new ConsoleProxyAlertEventArgs(ConsoleProxyAlertEventArgs.PROXY_FIREWALL_ALERT, proxy.getDataCenterId(), proxy.getId(),
+                                    proxy, null));
+                }
+
+                SubscriptionMgr.getInstance().notifySubscribers(ConsoleProxyManager.ALERT_SUBJECT, this,
+                        new ConsoleProxyAlertEventArgs(ConsoleProxyAlertEventArgs.PROXY_UP, proxy.getDataCenterId(), proxy.getId(), proxy, null));
+
+                return proxy;
+            } catch (Throwable thr) {
+                s_logger.warn("Unexpected exception: ", thr);
+
+                SubscriptionMgr.getInstance().notifySubscribers(
+                        ConsoleProxyManager.ALERT_SUBJECT,
+                        this,
+                        new ConsoleProxyAlertEventArgs(ConsoleProxyAlertEventArgs.PROXY_START_FAILURE, proxy.getDataCenterId(), proxy.getId(), proxy,
+                                "Unexpected exception: " + thr.getMessage()));
+
+                /*
+                 * final EventVO event = new EventVO();
+                 * event.setUserId(User.UID_SYSTEM);
+                 * event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
+                 * event.setType(EventTypes.EVENT_PROXY_START);
+                 * event.setLevel(EventVO.LEVEL_ERROR);
+                 * event.setStartId(startEventId); event.setDescription(
+                 * "Starting console proxy failed due to unhandled exception - "
+                 * + proxy.getName()); _eventDao.persist(event);
+                 */
+
+                Transaction txn = Transaction.currentTxn();
+                try {
+                    txn.start();
+                    String privateIpAddress = proxy.getPrivateIpAddress();
+                    if (privateIpAddress != null) {
+                        proxy.setPrivateIpAddress(null);
+                        freePrivateIpAddress(privateIpAddress, proxy.getDataCenterId(), proxy.getId());
+                    }
+
+                    _consoleProxyDao.updateIf(proxy, Event.OperationFailed, null);
+                    txn.commit();
+                } catch (Exception e) {
+                    s_logger.error("Caught exception during error recovery");
+                }
+
+                if (thr instanceof StorageUnavailableException) {
+                    throw (StorageUnavailableException) thr;
+                } else if (thr instanceof ConcurrentOperationException) {
+                    throw (ConcurrentOperationException) thr;
+                } else if (thr instanceof ExecutionException) {
+                    s_logger.error("Error while starting console proxy due to " + thr.getMessage());
+                } else {
+                    s_logger.error("Error while starting console proxy ", thr);
+                }
+                return null;
+            }
+        }
+
+        s_logger.warn("Starting console proxy encounters non-startable situation");
+        return null;
+    }
+
+    public ConsoleProxyVO assignProxyFromRunningPool(long dataCenterId) {
+
+        if (s_logger.isTraceEnabled())
+            s_logger.trace("Assign console proxy from running pool for request from data center : " + dataCenterId);
+
+        ConsoleProxyAllocator allocator = getCurrentAllocator();
+        assert (allocator != null);
+        List runningList = _consoleProxyDao.getProxyListInStates(dataCenterId, State.Running);
+        if (runningList != null && runningList.size() > 0) {
+            if (s_logger.isTraceEnabled()) {
+                s_logger.trace("Running proxy pool size : " + runningList.size());
+                for (ConsoleProxyVO proxy : runningList)
+                    s_logger.trace("Running proxy instance : " + proxy.getName());
+            }
+
+            List> l = _consoleProxyDao.getProxyLoadMatrix();
+            Map loadInfo = new HashMap();
+            if (l != null) {
+                for (Pair p : l) {
+                    loadInfo.put(p.first(), p.second());
+
+                    if (s_logger.isTraceEnabled()) {
+                        s_logger.trace("Running proxy instance allocation load { proxy id : " + p.first() + ", load : " + p.second() + "}");
+                    }
+                }
+            }
+            return allocator.allocProxy(runningList, loadInfo, dataCenterId);
+        } else {
+            if (s_logger.isTraceEnabled())
+                s_logger.trace("Empty running proxy pool for now in data center : " + dataCenterId);
+        }
+        return null;
+    }
+
+    public ConsoleProxyVO assignProxyFromStoppedPool(long dataCenterId) {
+        List l = _consoleProxyDao.getProxyListInStates(dataCenterId, State.Creating, State.Starting, State.Stopped, State.Migrating);
+        if (l != null && l.size() > 0)
+            return l.get(0);
+
+        return null;
+    }
+
+    public ConsoleProxyVO startNewConsoleProxy(long dataCenterId) {
+        if (s_logger.isDebugEnabled())
+            s_logger.debug("Assign console proxy from a newly started instance for request from data center : " + dataCenterId);
+
+        Map context = createProxyInstance(dataCenterId);
+
+        long proxyVmId = (Long) context.get("proxyVmId");
+        if (proxyVmId == 0) {
+            if (s_logger.isTraceEnabled())
+                s_logger.trace("Creating proxy instance failed, data center id : " + dataCenterId);
+
+            // release critical system resource on failure
+            if (context.get("publicIpAddress") != null)
+                freePublicIpAddress((String) context.get("publicIpAddress"), dataCenterId, 0);
+
+            return null;
+        }
+
+        ConsoleProxyVO proxy = _consoleProxyDao.findById(proxyVmId); 
+            //allocProxyStorage(dataCenterId, proxyVmId);
+        if (proxy != null) {
+            SubscriptionMgr.getInstance().notifySubscribers(ConsoleProxyManager.ALERT_SUBJECT, this,
+                    new ConsoleProxyAlertEventArgs(ConsoleProxyAlertEventArgs.PROXY_CREATED, dataCenterId, proxy.getId(), proxy, null));
+            return proxy;
+        } else {
+            if (s_logger.isDebugEnabled())
+                s_logger.debug("Unable to allocate console proxy storage, remove the console proxy record from DB, proxy id: " + proxyVmId);
+
+            SubscriptionMgr.getInstance().notifySubscribers(
+                    ConsoleProxyManager.ALERT_SUBJECT,
+                    this,
+                    new ConsoleProxyAlertEventArgs(ConsoleProxyAlertEventArgs.PROXY_CREATE_FAILURE, dataCenterId, proxyVmId, null,
+                            "Unable to allocate storage"));
+
+            destroyProxyDBOnly(proxyVmId);
+        }
+        return null;
+    }
+
+    public ConsoleProxyVO startNew(long dataCenterId) {
+
+        if (s_logger.isDebugEnabled())
+            s_logger.debug("Assign console proxy from a newly started instance for request from data center : " + dataCenterId);
+
+        Map context = createProxyInstance(dataCenterId);
+
+        long proxyVmId = (Long) context.get("proxyVmId");
+        if (proxyVmId == 0) {
+            if (s_logger.isTraceEnabled())
+                s_logger.trace("Creating proxy instance failed, data center id : " + dataCenterId);
+
+            // release critical system resource on failure
+            if (context.get("publicIpAddress") != null)
+                freePublicIpAddress((String) context.get("publicIpAddress"), dataCenterId, 0);
+
+            return null;
+        }
+
+        ConsoleProxyVO proxy = _consoleProxyDao.findById(proxyVmId); 
+            //allocProxyStorage(dataCenterId, proxyVmId);
+        if (proxy != null) {
+            SubscriptionMgr.getInstance().notifySubscribers(ConsoleProxyManager.ALERT_SUBJECT, this,
+                    new ConsoleProxyAlertEventArgs(ConsoleProxyAlertEventArgs.PROXY_CREATED, dataCenterId, proxy.getId(), proxy, null));
+            return proxy;
+        } else {
+            if (s_logger.isDebugEnabled())
+                s_logger.debug("Unable to allocate console proxy storage, remove the console proxy record from DB, proxy id: " + proxyVmId);
+
+            SubscriptionMgr.getInstance().notifySubscribers(
+                    ConsoleProxyManager.ALERT_SUBJECT,
+                    this,
+                    new ConsoleProxyAlertEventArgs(ConsoleProxyAlertEventArgs.PROXY_CREATE_FAILURE, dataCenterId, proxyVmId, null,
+                            "Unable to allocate storage"));
+
+            destroyProxyDBOnly(proxyVmId);
+        }
+        return null;
+    }
+
+    @DB
+    protected Map createProxyInstance(long dataCenterId) {
+
+        Map context = new HashMap();
+        String publicIpAddress = null;
+
+        Transaction txn = Transaction.currentTxn();
+        try {
+            DataCenterVO dc = _dcDao.findById(dataCenterId);
+            assert (dc != null);
+            context.put("dc", dc);
+
+            // this will basically allocate the pod based on data center id as
+            // we use system user id here
+            Set avoidPods = new HashSet();
+            Pair pod = null;
+            networkInfo publicIpAndVlan = null;
+
+            // About MAC address allocation
+            // MAC address used by User VM is inherited from DomR MAC address,
+            // with the least 16 bits overrided. to avoid
+            // potential conflicts, domP will mask bit 31
+            //
+            String[] macAddresses = _dcDao.getNextAvailableMacAddressPair(dataCenterId, (1L << 31));
+            String privateMacAddress = macAddresses[0];
+            String publicMacAddress = macAddresses[1];
+            macAddresses = _dcDao.getNextAvailableMacAddressPair(dataCenterId, (1L << 31));
+            String guestMacAddress = macAddresses[0];
+            while ((pod = _agentMgr.findPod(_template, _serviceOffering, dc, Account.ACCOUNT_ID_SYSTEM, avoidPods)) != null) {
+                publicIpAndVlan = allocPublicIpAddress(dataCenterId, pod.first().getId(), publicMacAddress);
+                if (publicIpAndVlan == null) {
+                    s_logger.warn("Unable to allocate public IP address for console proxy vm in data center : " + dataCenterId + ", pod="
+                            + pod.first().getId());
+                    avoidPods.add(pod.first().getId());
+                } else {
+                    break;
+                }
+            }
+
+            if (pod == null || publicIpAndVlan == null) {
+                String msg = "Unable to allocate pod for console proxy vm in data center : " + dataCenterId;
+                s_logger.warn(msg);
+                throw new CloudRuntimeException(msg);
+            }
+
+            long id = _consoleProxyDao.getNextInSequence(Long.class, "id");
+
+            context.put("publicIpAddress", publicIpAndVlan._ipAddr);
+            context.put("pod", pod);
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("Pod allocated " + pod.first().getName());
+            }
+
+            String cidrNetmask = NetUtils.getCidrNetmask(pod.first().getCidrSize());
+
+            // Find the VLAN ID, VLAN gateway, and VLAN netmask for
+            // publicIpAddress
+            publicIpAddress = publicIpAndVlan._ipAddr;
+
+            String vlanGateway = publicIpAndVlan._gateWay;
+            String vlanNetmask = publicIpAndVlan._netMask;
+            
+            AccountVO systemAccount = _accountMgr.getSystemAccount();
+
+            txn.start();
+            ConsoleProxyVO proxy;
+            String name = VirtualMachineName.getConsoleProxyName(id, _instance).intern();
+            proxy = new ConsoleProxyVO(id, _serviceOffering.getId(), name, guestMacAddress, null, NetUtils.getLinkLocalNetMask(), privateMacAddress, null, cidrNetmask,
+                    _template.getId(), _template.getGuestOSId(), publicMacAddress, publicIpAddress, vlanNetmask, publicIpAndVlan._vlanDbId,
+                    publicIpAndVlan._vlanid, pod.first().getId(), dataCenterId, systemAccount.getDomainId(), systemAccount.getId(), vlanGateway, null, dc.getDns1(), dc.getDns2(), _domain,
+                    _proxyRamSize, 0);
+
+            proxy.setLastHostId(pod.second());
+            proxy = _consoleProxyDao.persist(proxy);
+            long proxyVmId = proxy.getId();
+
+            final EventVO event = new EventVO();
+            event.setUserId(User.UID_SYSTEM);
+            event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
+            event.setType(EventTypes.EVENT_PROXY_CREATE);
+            event.setLevel(EventVO.LEVEL_INFO);
+            event.setDescription("New console proxy created - " + proxy.getName());
+            _eventDao.persist(event);
+            txn.commit();
+
+            context.put("proxyVmId", proxyVmId);
+            return context;
+        } catch (Throwable e) {
+            s_logger.error("Unexpected exception : ", e);
+
+            context.put("proxyVmId", (long) 0);
+            return context;
+        }
+    }
+
+    protected Map createProxyInstance2(long dataCenterId) {
+
+        long id = _consoleProxyDao.getNextInSequence(Long.class, "id");
+        String name = VirtualMachineName.getConsoleProxyName(id, _instance);
+        DataCenterVO dc = _dcDao.findById(dataCenterId);
+        AccountVO systemAcct = _accountMgr.getSystemAccount();
+        
+        DataCenterDeployment plan = new DataCenterDeployment(dataCenterId, 1);
+        
+        List offerings = _networkMgr.getSystemAccountNetworkOfferings(NetworkOfferingVO.SystemVmControlNetwork, NetworkOfferingVO.SystemVmManagementNetwork, NetworkOfferingVO.SystemVmPublicNetwork);
+        List profiles = new ArrayList(offerings.size());
+        for (NetworkOfferingVO offering : offerings) {
+            profiles.add(_networkMgr.setupNetworkConfiguration(_accountMgr.getSystemAccount(), offering, plan));
+        }
+        ConsoleProxyVO proxy = new ConsoleProxyVO(id, _serviceOffering.getId(), name, _template.getId(), _template.getGuestOSId(), dataCenterId, systemAcct.getDomainId(), systemAcct.getId(), 0);
+        proxy = _consoleProxyDao.persist(proxy);
+        try {
+            VirtualMachineProfile vmProfile = _vmMgr.allocate(proxy, _template, _serviceOffering, profiles, plan, _accountMgr.getSystemAccount());
+        } catch (InsufficientCapacityException e) {
+            s_logger.warn("InsufficientCapacity", e);
+            throw new CloudRuntimeException("Insufficient capacity exception", e);
+        } catch (StorageUnavailableException e) {
+            s_logger.warn("Unable to contact storage", e);
+            throw new CloudRuntimeException("Unable to contact storage", e);
+        }
+        
+        Map context = new HashMap();
+        context.put("dc", dc);
+//        context.put("publicIpAddress", publicIpAndVlan._ipAddr);
+        HostPodVO pod = _podDao.findById(proxy.getPodId());
+        context.put("pod", pod);
+        context.put("proxyVmId", proxy.getId());
+
+        return context;
+    }
+    
+    @DB
+    protected ConsoleProxyVO allocProxyStorage(long dataCenterId, long proxyVmId) {
+        ConsoleProxyVO proxy = _consoleProxyDao.findById(proxyVmId);
+        assert (proxy != null);
+
+        DataCenterVO dc = _dcDao.findById(dataCenterId);
+        HostPodVO pod = _podDao.findById(proxy.getPodId());
+        final AccountVO account = _accountDao.findById(Account.ACCOUNT_ID_SYSTEM);
+
+        try {
+            List vols = _storageMgr.create(account, proxy, _template, dc, pod, _serviceOffering, null, 0);
+            if (vols == null) {
+                s_logger.error("Unable to alloc storage for console proxy");
+                return null;
+            }
+
+            Transaction txn = Transaction.currentTxn();
+            txn.start();
+
+            // update pool id
+            ConsoleProxyVO vo = _consoleProxyDao.findById(proxy.getId());
+            _consoleProxyDao.update(proxy.getId(), vo);
+
+            // kick the state machine
+            _consoleProxyDao.updateIf(proxy, Event.OperationSucceeded, null);
+
+            txn.commit();
+            return proxy;
+        } catch (StorageUnavailableException e) {
+            s_logger.error("Unable to alloc storage for console proxy: ", e);
+            return null;
+        } catch (ExecutionException e) {
+            s_logger.error("Unable to alloc storage for console proxy: ", e);
+            return null;
+        }
+    }
+
+    private networkInfo allocPublicIpAddress(long dcId, long podId, String macAddr) {
+
+        if (_IpAllocator != null && _IpAllocator.exteralIpAddressAllocatorEnabled()) {
+            IpAddrAllocator.IpAddr ip = _IpAllocator.getPublicIpAddress(macAddr, dcId, podId);
+            networkInfo net = new networkInfo(ip.ipaddr, ip.netMask, ip.gateway, null, "untagged");
+            return net;
+        }
+
+        Pair ipAndVlan = _vlanDao.assignIpAddress(dcId, Account.ACCOUNT_ID_SYSTEM, DomainVO.ROOT_DOMAIN, VlanType.VirtualNetwork,
+                true);
+
+        if (ipAndVlan == null) {
+            s_logger.debug("Unable to get public ip address (type=Virtual) for console proxy vm for data center  : " + dcId);
+            ipAndVlan = _vlanDao.assignPodDirectAttachIpAddress(dcId, podId, Account.ACCOUNT_ID_SYSTEM, DomainVO.ROOT_DOMAIN);
+            if (ipAndVlan == null)
+                s_logger.debug("Unable to get public ip address (type=DirectAttach) for console proxy vm for data center  : " + dcId);
+        }
+        if (ipAndVlan != null) {
+            VlanVO vlan = ipAndVlan.second();
+            networkInfo net = new networkInfo(ipAndVlan.first(), vlan.getVlanNetmask(), vlan.getVlanGateway(), vlan.getId(), vlan.getVlanId());
+            return net;
+        }
+        return null;
+    }
+
+    private String allocPrivateIpAddress(Long dcId, Long podId, Long proxyId, String macAddr) {
+        if (_IpAllocator != null && _IpAllocator.exteralIpAddressAllocatorEnabled()) {
+            return _IpAllocator.getPrivateIpAddress(macAddr, dcId, podId).ipaddr;
+        } else {
+            return _dcDao.allocatePrivateIpAddress(dcId, podId, proxyId);
+        }
+    }
+
+    private void freePrivateIpAddress(String ipAddress, Long dcId, Long podId) {
+        if (_IpAllocator != null && _IpAllocator.exteralIpAddressAllocatorEnabled()) {
+            _IpAllocator.releasePrivateIpAddress(ipAddress, dcId, podId);
+        } else {
+            _dcDao.releasePrivateIpAddress(ipAddress, dcId, podId);
+        }
+    }
+
+    private void freePublicIpAddress(String ipAddress, long dcId, long podId) {
+        if (_IpAllocator != null && _IpAllocator.exteralIpAddressAllocatorEnabled()) {
+            _IpAllocator.releasePublicIpAddress(ipAddress, dcId, podId);
+        } else {
+            _ipAddressDao.unassignIpAddress(ipAddress);
+        }
+    }
+
+    private ConsoleProxyAllocator getCurrentAllocator() {
+        // for now, only one adapter is supported
+        Enumeration it = _consoleProxyAllocators.enumeration();
+        if (it.hasMoreElements())
+            return it.nextElement();
+
+        return null;
+    }
+
+    protected String connect(String ipAddress, int port) {
+        for (int i = 0; i <= _ssh_retry; i++) {
+            SocketChannel sch = null;
+            try {
+                if (s_logger.isDebugEnabled()) {
+                    s_logger.debug("Trying to connect to " + ipAddress);
+                }
+                sch = SocketChannel.open();
+                sch.configureBlocking(true);
+                sch.socket().setSoTimeout(5000);
+
+                InetSocketAddress addr = new InetSocketAddress(ipAddress, port);
+                sch.connect(addr);
+                return null;
+            } catch (IOException e) {
+                if (s_logger.isDebugEnabled()) {
+                    s_logger.debug("Could not connect to " + ipAddress);
+                }
+            } finally {
+                if (sch != null) {
+                    try {
+                        sch.close();
+                    } catch (IOException e) {
+                    }
+                }
+            }
+            try {
+                Thread.sleep(_ssh_sleep);
+            } catch (InterruptedException ex) {
+            }
+        }
+
+        s_logger.debug("Unable to logon to " + ipAddress);
+
+        return "Unable to connect";
+    }
+
+    public void onLoadAnswer(ConsoleProxyLoadAnswer answer) {
+        if (answer.getDetails() == null)
+            return;
+
+        ConsoleProxyStatus status = null;
+        try {
+            GsonBuilder gb = new GsonBuilder();
+            gb.setVersion(1.3);
+            Gson gson = gb.create();
+            status = gson.fromJson(answer.getDetails(), ConsoleProxyStatus.class);
+        } catch (Throwable e) {
+            s_logger.warn("Unable to parse load info from proxy, proxy vm id : " + answer.getProxyVmId() + ", info : " + answer.getDetails());
+        }
+
+        if (status != null) {
+            int count = 0;
+            if (status.getConnections() != null)
+                count = status.getConnections().length;
+
+            byte[] details = null;
+            if (answer.getDetails() != null)
+                details = answer.getDetails().getBytes(Charset.forName("US-ASCII"));
+            _consoleProxyDao.update(answer.getProxyVmId(), count, DateUtil.currentGMTTime(), details);
+        } else {
+            if (s_logger.isTraceEnabled())
+                s_logger.trace("Unable to get console proxy load info, id : " + answer.getProxyVmId());
+
+            _consoleProxyDao.update(answer.getProxyVmId(), 0, DateUtil.currentGMTTime(), null);
+            // TODO : something is wrong with the VM, restart it?
+        }
+    }
+
+    @Override
+    public void onLoadReport(ConsoleProxyLoadReportCommand cmd) {
+        if (cmd.getLoadInfo() == null)
+            return;
+
+        ConsoleProxyStatus status = null;
+        try {
+            GsonBuilder gb = new GsonBuilder();
+            gb.setVersion(1.3);
+            Gson gson = gb.create();
+            status = gson.fromJson(cmd.getLoadInfo(), ConsoleProxyStatus.class);
+        } catch (Throwable e) {
+            s_logger.warn("Unable to parse load info from proxy, proxy vm id : " + cmd.getProxyVmId() + ", info : " + cmd.getLoadInfo());
+        }
+
+        if (status != null) {
+            int count = 0;
+            if (status.getConnections() != null)
+                count = status.getConnections().length;
+
+            byte[] details = null;
+            if (cmd.getLoadInfo() != null)
+                details = cmd.getLoadInfo().getBytes(Charset.forName("US-ASCII"));
+            _consoleProxyDao.update(cmd.getProxyVmId(), count, DateUtil.currentGMTTime(), details);
+        } else {
+            if (s_logger.isTraceEnabled())
+                s_logger.trace("Unable to get console proxy load info, id : " + cmd.getProxyVmId());
+
+            _consoleProxyDao.update(cmd.getProxyVmId(), 0, DateUtil.currentGMTTime(), null);
+        }
+    }
+
+    @Override
+    public AgentControlAnswer onConsoleAccessAuthentication(ConsoleAccessAuthenticationCommand cmd) {
+        long vmId = 0;
+
+        if (cmd.getVmId() != null && cmd.getVmId().isEmpty()) {
+            if (s_logger.isTraceEnabled())
+                s_logger.trace("Invalid vm id sent from proxy(happens when proxy session has terminated)");
+            return new ConsoleAccessAuthenticationAnswer(cmd, false);
+        }
+
+        try {
+            vmId = Long.parseLong(cmd.getVmId());
+        } catch (NumberFormatException e) {
+            s_logger.error("Invalid vm id " + cmd.getVmId() + " sent from console access authentication", e);
+            return new ConsoleAccessAuthenticationAnswer(cmd, false);
+        }
+
+        // TODO authentication channel between console proxy VM and management
+        // server needs to be secured,
+        // the data is now being sent through private network, but this is
+        // apparently not enough
+        VMInstanceVO vm = _instanceDao.findById(vmId);
+        if (vm == null) {
+            return new ConsoleAccessAuthenticationAnswer(cmd, false);
+        }
+
+        if (vm.getHostId() == null) {
+            s_logger.warn("VM " + vmId + " lost host info, failed authentication request");
+            return new ConsoleAccessAuthenticationAnswer(cmd, false);
+        }
+
+        HostVO host = _hostDao.findById(vm.getHostId());
+        if (host == null) {
+            s_logger.warn("VM " + vmId + "'s host does not exist, fail authentication request");
+            return new ConsoleAccessAuthenticationAnswer(cmd, false);
+        }
+
+        String sid = cmd.getSid();
+        if (sid == null || !sid.equals(vm.getVncPassword())) {
+            s_logger.warn("sid " + sid + " in url does not match stored sid " + vm.getVncPassword());
+            return new ConsoleAccessAuthenticationAnswer(cmd, false);
+        }
+
+        return new ConsoleAccessAuthenticationAnswer(cmd, true);
+    }
+
+    private ConsoleProxyVO findConsoleProxyByHost(HostVO host) throws NumberFormatException {
+        String name = host.getName();
+        long proxyVmId = 0;
+        ConsoleProxyVO proxy = null;
+        if (name != null && name.startsWith("v-")) {
+            String[] tokens = name.split("-");
+            proxyVmId = Long.parseLong(tokens[1]);
+            proxy = this._consoleProxyDao.findById(proxyVmId);
+        }
+        return proxy;
+    }
+
+    @Override
+    public void onAgentConnect(HostVO host, StartupCommand cmd) {
+        if (host.getType() == Type.ConsoleProxy) {
+            // TODO we can use this event to mark the proxy is up and
+            // functioning instead of
+            // pinging the console proxy VM command port
+            //
+            // for now, just log a message
+            if (s_logger.isInfoEnabled())
+                s_logger.info("Console proxy agent is connected. proxy: " + host.getName());
+
+            /* update public/private ip address */
+            if (_IpAllocator != null && _IpAllocator.exteralIpAddressAllocatorEnabled()) {
+                try {
+                    ConsoleProxyVO console = findConsoleProxyByHost(host);
+                    if (console == null) {
+                        s_logger.debug("Can't find console proxy ");
+                        return;
+                    }
+                    console.setPrivateIpAddress(cmd.getPrivateIpAddress());
+                    console.setPrivateNetmask(cmd.getPrivateNetmask());
+                    console.setPublicIpAddress(cmd.getPublicIpAddress());
+                    console.setPublicNetmask(cmd.getPublicNetmask());
+                    _consoleProxyDao.persist(console);
+                } catch (NumberFormatException e) {
+                }
+            }
+        }
+    }
+
+    @Override
+    public void onAgentDisconnect(long agentId, com.cloud.host.Status state) {
+        if (state == com.cloud.host.Status.Alert || state == com.cloud.host.Status.Disconnected) {
+            // be it either in alert or in disconnected state, the agent process
+            // may be gone in the VM,
+            // we will be reacting to stop the corresponding VM and let the scan
+            // process to
+            HostVO host = _hostDao.findById(agentId);
+            if (host.getType() == Type.ConsoleProxy) {
+                String name = host.getName();
+                if (s_logger.isInfoEnabled())
+                    s_logger.info("Console proxy agent disconnected, proxy: " + name);
+                if (name != null && name.startsWith("v-")) {
+                    String[] tokens = name.split("-");
+                    long proxyVmId = 0;
+                    try {
+                        proxyVmId = Long.parseLong(tokens[1]);
+                    } catch (NumberFormatException e) {
+                        s_logger.error("Unexpected exception " + e.getMessage(), e);
+                        return;
+                    }
+
+                    final ConsoleProxyVO proxy = this._consoleProxyDao.findById(proxyVmId);
+                    if (proxy != null) {
+                        Long hostId = proxy.getHostId();
+
+                        // Disable this feature for now, as it conflicts with
+                        // the case of allowing user to reboot console proxy
+                        // when rebooting happens, we will receive disconnect
+                        // here and we can't enter into stopping process,
+                        // as when the rebooted one comes up, it will kick off a
+                        // newly started one and trigger the process
+                        // continue on forever
+
+                        /*
+                         * _capacityScanScheduler.execute(new Runnable() {
+                         * public void run() { if(s_logger.isInfoEnabled())
+                         * s_logger.info("Stop console proxy " + proxy.getName()
+                         * +
+                         * " VM because of that the agent running inside it has disconnected"
+                         * ); stopProxy(proxy.getId()); } });
+                         */
+                    } else {
+                        if (s_logger.isInfoEnabled())
+                            s_logger.info("Console proxy agent disconnected but corresponding console proxy VM no longer exists in DB, proxy: "
+                                    + name);
+                    }
+                } else {
+                    assert (false) : "Invalid console proxy name: " + name;
+                }
+            }
+        }
+    }
+
+    private void checkPendingProxyVMs() {
+        // drive state to change away from transient states
+        List l = _consoleProxyDao.getProxyListInStates(State.Creating);
+        if (l != null && l.size() > 0) {
+            for (ConsoleProxyVO proxy : l) {
+                if (proxy.getLastUpdateTime() == null
+                        || (proxy.getLastUpdateTime() != null && System.currentTimeMillis() - proxy.getLastUpdateTime().getTime() > 60000)) {
+                    try {
+                        ConsoleProxyVO readyProxy = null;
+                        if (_allocProxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
+                            try {
+                                readyProxy = allocProxyStorage(proxy.getDataCenterId(), proxy.getId());
+                            } finally {
+                                _allocProxyLock.unlock();
+                            }
+
+                            if (readyProxy != null) {
+                                GlobalLock proxyLock = GlobalLock.getInternLock(getProxyLockName(readyProxy.getId()));
+                                try {
+                                    if (proxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
+                                        try {
+                                            readyProxy = start(readyProxy.getId(), 0);
+                                        } finally {
+                                            proxyLock.unlock();
+                                        }
+                                    } else {
+                                        if (s_logger.isInfoEnabled())
+                                            s_logger.info("Unable to acquire synchronization lock to start console proxy : " + readyProxy.getName());
+                                    }
+                                } finally {
+                                    proxyLock.releaseRef();
+                                }
+                            }
+                        } else {
+                            if (s_logger.isInfoEnabled())
+                                s_logger.info("Unable to acquire synchronization lock to allocate proxy storage, wait for next turn");
+                        }
+                    } catch (StorageUnavailableException e) {
+                        s_logger.warn("Storage unavailable", e);
+                    } catch (InsufficientCapacityException e) {
+                        s_logger.warn("insuffiient capacity", e);
+                    } catch (ConcurrentOperationException e) {
+                        s_logger.debug("Concurrent operation: " + e.getMessage());
+                    }
+                }
+            }
+        }
+    }
+
+    private Runnable getCapacityScanTask() {
+        return new Runnable() {
+
+            @Override
+            public void run() {
+                Transaction txn = Transaction.open(Transaction.CLOUD_DB);
+                try {
+                    reallyRun();
+                } catch (Throwable e) {
+                    s_logger.warn("Unexpected exception " + e.getMessage(), e);
+                } finally {
+                    StackMaid.current().exitCleanup();
+                    txn.close();
+                }
+            }
+
+            private void reallyRun() {
+                if (s_logger.isTraceEnabled())
+                    s_logger.trace("Begin console proxy capacity scan");
+
+                // config var for consoleproxy.restart check
+                String restart = _configDao.getValue("consoleproxy.restart");
+                if (restart != null && restart.equalsIgnoreCase("false")) {
+                    s_logger.debug("Capacity scan disabled purposefully, consoleproxy.restart = false. This happens when the primarystorage is in maintenance mode");
+                    return;
+                }
+
+                Map zoneHostInfoMap = getZoneHostInfo();
+                if (isServiceReady(zoneHostInfoMap)) {
+                    if (s_logger.isTraceEnabled())
+                        s_logger.trace("Service is ready, check to see if we need to allocate standby capacity");
+
+                    if (!_capacityScanLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_COOPERATION)) {
+                        if (s_logger.isTraceEnabled())
+                            s_logger.trace("Capacity scan lock is used by others, skip and wait for my turn");
+                        return;
+                    }
+
+                    if (s_logger.isTraceEnabled())
+                        s_logger.trace("*** Begining capacity scan... ***");
+
+                    try {
+                        checkPendingProxyVMs();
+
+                        // scan default data center first
+                        long defaultId = 0;
+
+                        // proxy count info by data-centers (zone-id, zone-name,
+                        // count)
+                        List l = _consoleProxyDao.getDatacenterProxyLoadMatrix();
+
+                        // running VM session count by data-centers (zone-id,
+                        // zone-name, count)
+                        List listVmCounts = _consoleProxyDao.getDatacenterSessionLoadMatrix();
+
+                        // indexing load info by data-center id
+                        Map mapVmCounts = new HashMap();
+                        if (listVmCounts != null)
+                            for (ConsoleProxyLoadInfo info : listVmCounts)
+                                mapVmCounts.put(info.getId(), info);
+
+                        for (ConsoleProxyLoadInfo info : l) {
+                            if (info.getName().equals(_instance)) {
+                                ConsoleProxyLoadInfo vmInfo = mapVmCounts.get(info.getId());
+
+                                if (!checkCapacity(info, vmInfo != null ? vmInfo : new ConsoleProxyLoadInfo())) {
+                                    if (isZoneReady(zoneHostInfoMap, info.getId())) {
+                                        allocCapacity(info.getId());
+                                    } else {
+                                        if (s_logger.isTraceEnabled())
+                                            s_logger.trace("Zone " + info.getId() + " is not ready to alloc standy console proxy");
+                                    }
+                                }
+
+                                defaultId = info.getId();
+                                break;
+                            }
+                        }
+
+                        // scan rest of data-centers
+                        for (ConsoleProxyLoadInfo info : l) {
+                            if (info.getId() != defaultId) {
+                                ConsoleProxyLoadInfo vmInfo = mapVmCounts.get(info.getId());
+
+                                if (!checkCapacity(info, vmInfo != null ? vmInfo : new ConsoleProxyLoadInfo())) {
+                                    if (isZoneReady(zoneHostInfoMap, info.getId())) {
+                                        allocCapacity(info.getId());
+                                    } else {
+                                        if (s_logger.isTraceEnabled())
+                                            s_logger.trace("Zone " + info.getId() + " is not ready to alloc standy console proxy");
+                                    }
+                                }
+                            }
+                        }
+
+                        if (s_logger.isTraceEnabled())
+                            s_logger.trace("*** Stop capacity scan ***");
+                    } finally {
+                        _capacityScanLock.unlock();
+                    }
+
+                } else {
+                    if (s_logger.isTraceEnabled())
+                        s_logger.trace("Service is not ready for capacity preallocation, wait for next time");
+                }
+
+                if (s_logger.isTraceEnabled())
+                    s_logger.trace("End of console proxy capacity scan");
+            }
+        };
+    }
+
+    private boolean checkCapacity(ConsoleProxyLoadInfo proxyCountInfo, ConsoleProxyLoadInfo vmCountInfo) {
+
+        if (proxyCountInfo.getCount() * _capacityPerProxy - vmCountInfo.getCount() <= _standbyCapacity)
+            return false;
+
+        return true;
+    }
+
+    private void allocCapacity(long dataCenterId) {
+        if (s_logger.isTraceEnabled())
+            s_logger.trace("Allocate console proxy standby capacity for data center : " + dataCenterId);
+
+        boolean proxyFromStoppedPool = false;
+        ConsoleProxyVO proxy = assignProxyFromStoppedPool(dataCenterId);
+        if (proxy == null) {
+            if (s_logger.isInfoEnabled())
+                s_logger.info("No stopped console proxy is available, need to allocate a new console proxy");
+
+            if (_allocProxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
+                try {
+                    proxy = startNew(dataCenterId);
+                } finally {
+                    _allocProxyLock.unlock();
+                }
+            } else {
+                if (s_logger.isInfoEnabled())
+                    s_logger.info("Unable to acquire synchronization lock to allocate proxy resource for standby capacity, wait for next scan");
+                return;
+            }
+        } else {
+            if (s_logger.isInfoEnabled())
+                s_logger.info("Found a stopped console proxy, bring it up to running pool. proxy vm id : " + proxy.getId());
+            proxyFromStoppedPool = true;
+        }
+
+        if (proxy != null) {
+            long proxyVmId = proxy.getId();
+            GlobalLock proxyLock = GlobalLock.getInternLock(getProxyLockName(proxyVmId));
+            try {
+                if (proxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
+                    try {
+                        proxy = startProxy(proxyVmId, 0);
+                    } finally {
+                        proxyLock.unlock();
+                    }
+                } else {
+                    if (s_logger.isInfoEnabled())
+                        s_logger.info("Unable to acquire synchronization lock to start proxy for standby capacity, proxy vm id : " + proxy.getId());
+                    return;
+                }
+            } finally {
+                proxyLock.releaseRef();
+            }
+
+            if (proxy == null) {
+                if (s_logger.isInfoEnabled())
+                    s_logger.info("Unable to start console proxy for standby capacity, proxy vm Id : " + proxyVmId
+                            + ", will recycle it and start a new one");
+
+                if (proxyFromStoppedPool)
+                    destroyProxy(proxyVmId, 0);
+            } else {
+                if (s_logger.isInfoEnabled())
+                    s_logger.info("Console proxy " + proxy.getName() + " is started");
+            }
+        }
+    }
+
+    public boolean isServiceReady(Map zoneHostInfoMap) {
+        for (ZoneHostInfo zoneHostInfo : zoneHostInfoMap.values()) {
+            if (isZoneHostReady(zoneHostInfo)) {
+                if (s_logger.isInfoEnabled())
+                    s_logger.info("Zone " + zoneHostInfo.getDcId() + " is ready to launch");
+                return true;
+            }
+        }
+
+        return false;
+    }
+
+    public boolean isZoneReady(Map zoneHostInfoMap, long dataCenterId) {
+        ZoneHostInfo zoneHostInfo = zoneHostInfoMap.get(dataCenterId);
+        if (zoneHostInfo != null && isZoneHostReady(zoneHostInfo)) {
+            VMTemplateVO template = _templateDao.findConsoleProxyTemplate();
+            HostVO secondaryStorageHost = _storageMgr.getSecondaryStorageHost(dataCenterId);
+            boolean templateReady = false;
+
+            if (template != null && secondaryStorageHost != null) {
+                VMTemplateHostVO templateHostRef = _vmTemplateHostDao.findByHostTemplate(secondaryStorageHost.getId(), template.getId());
+                templateReady = (templateHostRef != null) && (templateHostRef.getDownloadState() == Status.DOWNLOADED);
+            }
+
+            if (templateReady) {
+                List> l = _consoleProxyDao.getDatacenterStoragePoolHostInfo(dataCenterId, _use_lvm);
+                if (l != null && l.size() > 0 && l.get(0).second().intValue() > 0) {
+                    return true;
+                } else {
+                    if (s_logger.isTraceEnabled())
+                        s_logger.trace("Primary storage is not ready, wait until it is ready to launch console proxy");
+                }
+            } else {
+                if (s_logger.isTraceEnabled())
+                    s_logger.trace("Zone host is ready, but console proxy template is not ready");
+            }
+        }
+        return false;
+    }
+
+    private boolean isZoneHostReady(ZoneHostInfo zoneHostInfo) {
+        int expectedFlags = 0;
+        if (_use_storage_vm)
+            expectedFlags = RunningHostInfoAgregator.ZoneHostInfo.ROUTING_HOST_MASK;
+        else
+            expectedFlags = RunningHostInfoAgregator.ZoneHostInfo.ALL_HOST_MASK;
+
+        return (zoneHostInfo.getFlags() & expectedFlags) == expectedFlags;
+    }
+
+    private synchronized Map getZoneHostInfo() {
+        Date cutTime = DateUtil.currentGMTTime();
+        List l = _hostDao.getRunningHostCounts(new Date(cutTime.getTime() - ClusterManager.DEFAULT_HEARTBEAT_THRESHOLD));
+
+        RunningHostInfoAgregator aggregator = new RunningHostInfoAgregator();
+        if (l.size() > 0)
+            for (RunningHostCountInfo countInfo : l)
+                aggregator.aggregate(countInfo);
+
+        return aggregator.getZoneHostInfoMap();
+    }
+
+    @Override
+    public String getName() {
+        return _name;
+    }
+
+    @Override
+    public boolean start() {
+        if (s_logger.isInfoEnabled())
+            s_logger.info("Start console proxy manager");
+
+        return true;
+    }
+
+    @Override
+    public boolean stop() {
+        if (s_logger.isInfoEnabled())
+            s_logger.info("Stop console proxy manager");
+        _capacityScanScheduler.shutdownNow();
+
+        try {
+            _capacityScanScheduler.awaitTermination(EXECUTOR_SHUTDOWN_TIMEOUT, TimeUnit.MILLISECONDS);
+        } catch (InterruptedException e) {
+        }
+
+        _capacityScanLock.releaseRef();
+        _allocProxyLock.releaseRef();
+        return true;
+    }
+
+    @Override
+    public Command cleanup(ConsoleProxyVO vm, String vmName) {
+        if (vmName != null) {
+            return new StopCommand(vm, vmName, VirtualMachineName.getVnet(vmName));
+        } else if (vm != null) {
+            ConsoleProxyVO vo = vm;
+            return new StopCommand(vo, null);
+        } else {
+            throw new CloudRuntimeException("Shouldn't even be here!");
+        }
+    }
+
+    @Override
+    public void completeStartCommand(ConsoleProxyVO vm) {
+        _consoleProxyDao.updateIf(vm, Event.AgentReportRunning, vm.getHostId());
+    }
+
+    @Override
+    public void completeStopCommand(ConsoleProxyVO vm) {
+        completeStopCommand(vm, Event.AgentReportStopped);
+    }
+
+    @DB
+    protected void completeStopCommand(ConsoleProxyVO proxy, Event ev) {
+        Transaction txn = Transaction.currentTxn();
+        try {
+            txn.start();
+            String privateIpAddress = proxy.getPrivateIpAddress();
+            if (privateIpAddress != null) {
+                proxy.setPrivateIpAddress(null);
+                freePrivateIpAddress(privateIpAddress, proxy.getDataCenterId(), proxy.getId());
+            }
+            String guestIpAddress = proxy.getGuestIpAddress();
+            if (guestIpAddress != null) {
+                proxy.setGuestIpAddress(null);
+                _dcDao.releaseLinkLocalPrivateIpAddress(guestIpAddress, proxy.getDataCenterId(), proxy.getId());
+            }
+
+            if (!_consoleProxyDao.updateIf(proxy, ev, null)) {
+                s_logger.debug("Unable to update the console proxy");
+                return;
+            }
+            txn.commit();
+        } catch (Exception e) {
+            s_logger.error("Unable to complete stop command due to ", e);
+        }
+
+        if (_storageMgr.unshare(proxy, null) == null) {
+            s_logger.warn("Unable to set share to false for " + proxy.getId());
+        }
+    }
+
+    @Override
+    public ConsoleProxyVO get(long id) {
+        return _consoleProxyDao.findById(id);
+    }
+
+    @Override
+    public Long convertToId(String vmName) {
+        if (!VirtualMachineName.isValidConsoleProxyName(vmName, _instance)) {
+            return null;
+        }
+        return VirtualMachineName.getConsoleProxyId(vmName);
+    }
+
+    @Override
+    public boolean stopProxy(long proxyVmId, long startEventId) {
+
+        AsyncJobExecutor asyncExecutor = BaseAsyncJobExecutor.getCurrentExecutor();
+        if (asyncExecutor != null) {
+            AsyncJobVO job = asyncExecutor.getJob();
+
+            if (s_logger.isInfoEnabled())
+                s_logger.info("Stop console proxy " + proxyVmId + ", update async job-" + job.getId());
+            _asyncMgr.updateAsyncJobAttachment(job.getId(), "console_proxy", proxyVmId);
+        }
+
+        ConsoleProxyVO proxy = _consoleProxyDao.findById(proxyVmId);
+        if (proxy == null) {
+            if (s_logger.isDebugEnabled())
+                s_logger.debug("Stopping console proxy failed: console proxy " + proxyVmId + " no longer exists");
+            return false;
+        }
+        /*
+         * saveStartedEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM,
+         * EventTypes.EVENT_PROXY_STOP, "Stopping console proxy with Id: " +
+         * proxyVmId, startEventId);
+         */
+        try {
+            return stop(proxy, startEventId);
+        } catch (AgentUnavailableException e) {
+            if (s_logger.isDebugEnabled())
+                s_logger.debug("Stopping console proxy " + proxy.getName() + " failed : exception " + e.toString());
+            return false;
+        }
+    }
+
+    @Override
+    public boolean rebootProxy(long proxyVmId, long startEventId) {
+        AsyncJobExecutor asyncExecutor = BaseAsyncJobExecutor.getCurrentExecutor();
+        if (asyncExecutor != null) {
+            AsyncJobVO job = asyncExecutor.getJob();
+
+            if (s_logger.isInfoEnabled())
+                s_logger.info("Reboot console proxy " + proxyVmId + ", update async job-" + job.getId());
+            _asyncMgr.updateAsyncJobAttachment(job.getId(), "console_proxy", proxyVmId);
+        }
+
+        final ConsoleProxyVO proxy = _consoleProxyDao.findById(proxyVmId);
+
+        if (proxy == null || proxy.getState() == State.Destroyed) {
+            return false;
+        }
+
+        /*
+         * saveStartedEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM,
+         * EventTypes.EVENT_PROXY_REBOOT, "Rebooting console proxy with Id: " +
+         * proxyVmId, startEventId);
+         */
+        if (proxy.getState() == State.Running && proxy.getHostId() != null) {
+            final RebootCommand cmd = new RebootCommand(proxy.getInstanceName());
+            final Answer answer = _agentMgr.easySend(proxy.getHostId(), cmd);
+
+            if (answer != null) {
+                if (s_logger.isDebugEnabled())
+                    s_logger.debug("Successfully reboot console proxy " + proxy.getName());
+
+                SubscriptionMgr.getInstance()
+                        .notifySubscribers(
+                                ConsoleProxyManager.ALERT_SUBJECT,
+                                this,
+                                new ConsoleProxyAlertEventArgs(ConsoleProxyAlertEventArgs.PROXY_REBOOTED, proxy.getDataCenterId(), proxy.getId(),
+                                        proxy, null));
+
+                final EventVO event = new EventVO();
+                event.setUserId(User.UID_SYSTEM);
+                event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
+                event.setType(EventTypes.EVENT_PROXY_REBOOT);
+                event.setLevel(EventVO.LEVEL_INFO);
+                event.setStartId(startEventId);
+                event.setDescription("Console proxy rebooted - " + proxy.getName());
+                _eventDao.persist(event);
+                return true;
+            } else {
+                if (s_logger.isDebugEnabled())
+                    s_logger.debug("failed to reboot console proxy : " + proxy.getName());
+
+                final EventVO event = new EventVO();
+                event.setUserId(User.UID_SYSTEM);
+                event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
+                event.setType(EventTypes.EVENT_PROXY_REBOOT);
+                event.setLevel(EventVO.LEVEL_ERROR);
+                event.setStartId(startEventId);
+                event.setDescription("Rebooting console proxy failed - " + proxy.getName());
+                _eventDao.persist(event);
+                return false;
+            }
+        } else {
+            return startProxy(proxyVmId, 0) != null;
+        }
+    }
+
+    @Override
+    public boolean destroy(ConsoleProxyVO proxy) throws AgentUnavailableException {
+        return destroyProxy(proxy.getId(), 0);
+    }
+
+    @Override
+    @DB
+    public boolean destroyProxy(long vmId, long startEventId) {
+        AsyncJobExecutor asyncExecutor = BaseAsyncJobExecutor.getCurrentExecutor();
+        if (asyncExecutor != null) {
+            AsyncJobVO job = asyncExecutor.getJob();
+
+            if (s_logger.isInfoEnabled())
+                s_logger.info("Destroy console proxy " + vmId + ", update async job-" + job.getId());
+            _asyncMgr.updateAsyncJobAttachment(job.getId(), "console_proxy", vmId);
+        }
+
+        ConsoleProxyVO vm = _consoleProxyDao.findById(vmId);
+        if (vm == null || vm.getState() == State.Destroyed) {
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("Unable to find vm or vm is destroyed: " + vmId);
+            }
+            return true;
+        }
+        /*
+         * saveStartedEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM,
+         * EventTypes.EVENT_PROXY_DESTROY, "Destroying console proxy with Id: "
+         * + vmId, startEventId);
+         */
+        if (s_logger.isDebugEnabled()) {
+            s_logger.debug("Destroying console proxy vm " + vmId);
+        }
+
+        if (!_consoleProxyDao.updateIf(vm, Event.DestroyRequested, null)) {
+            s_logger.debug("Unable to destroy the vm because it is not in the correct state: " + vmId);
+            return false;
+        }
+
+        Transaction txn = Transaction.currentTxn();
+        List vols = null;
+        try {
+            vols = _volsDao.findByInstance(vmId);
+            if (vols.size() != 0) {
+                _storageMgr.destroy(vm, vols);
+            }
+
+            return true;
+        } finally {
+            try {
+                txn.start();
+                // release critical system resources used by the VM before we
+                // delete them
+                if (vm.getPublicIpAddress() != null)
+                    freePublicIpAddress(vm.getPublicIpAddress(), vm.getDataCenterId(), vm.getPodId());
+                vm.setPublicIpAddress(null);
+
+                _consoleProxyDao.remove(vm.getId());
+
+                final EventVO event = new EventVO();
+                event.setUserId(User.UID_SYSTEM);
+                event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
+                event.setType(EventTypes.EVENT_PROXY_DESTROY);
+                event.setLevel(EventVO.LEVEL_INFO);
+                event.setStartId(startEventId);
+                event.setDescription("Console proxy destroyed - " + vm.getName());
+                _eventDao.persist(event);
+
+                txn.commit();
+            } catch (Exception e) {
+                s_logger.error("Caught this error: ", e);
+                txn.rollback();
+                return false;
+            } finally {
+                s_logger.debug("console proxy vm is destroyed : " + vm.getName());
+            }
+        }
+    }
+
+    @DB
+    public boolean destroyProxyDBOnly(long vmId) {
+        Transaction txn = Transaction.currentTxn();
+        try {
+            txn.start();
+            _volsDao.deleteVolumesByInstance(vmId);
+
+            ConsoleProxyVO proxy = _consoleProxyDao.findById(vmId);
+            if (proxy != null) {
+                if (proxy.getPublicIpAddress() != null)
+                    freePublicIpAddress(proxy.getPublicIpAddress(), proxy.getDataCenterId(), proxy.getPodId());
+
+                _consoleProxyDao.remove(vmId);
+
+                final EventVO event = new EventVO();
+                event.setUserId(User.UID_SYSTEM);
+                event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
+                event.setType(EventTypes.EVENT_PROXY_DESTROY);
+                event.setLevel(EventVO.LEVEL_INFO);
+                event.setDescription("Console proxy destroyed - " + proxy.getName());
+                _eventDao.persist(event);
+            }
+
+            txn.commit();
+            return true;
+        } catch (Exception e) {
+            s_logger.error("Caught this error: ", e);
+            txn.rollback();
+            return false;
+        } finally {
+            s_logger.debug("console proxy vm is destroyed from DB : " + vmId);
+        }
+    }
+
+    @Override
+    public boolean stop(ConsoleProxyVO proxy, long startEventId) throws AgentUnavailableException {
+        if (!_consoleProxyDao.updateIf(proxy, Event.StopRequested, proxy.getHostId())) {
+            s_logger.debug("Unable to stop console proxy: " + proxy.toString());
+            return false;
+        }
+
+        // IPAddressVO ip = _ipAddressDao.findById(proxy.getPublicIpAddress());
+        // VlanVO vlan = _vlanDao.findById(new Long(ip.getVlanDbId()));
+
+        GlobalLock proxyLock = GlobalLock.getInternLock(getProxyLockName(proxy.getId()));
+        try {
+            if (proxyLock.lock(ACQUIRE_GLOBAL_LOCK_TIMEOUT_FOR_SYNC)) {
+                try {
+                    StopCommand cmd = new StopCommand(proxy, true, Integer.toString(_consoleProxyPort), Integer.toString(_consoleProxyUrlPort),
+                            proxy.getPublicIpAddress());
+                    try {
+                        Long proxyHostId = proxy.getHostId();
+                        if (proxyHostId == null) {
+                            s_logger.debug("Unable to stop due to proxy " + proxy.getId()
+                                    + " as host is no longer available, proxy may already have been stopped");
+                            return false;
+                        }
+                        StopAnswer answer = (StopAnswer) _agentMgr.send(proxyHostId, cmd);
+                        if (answer == null || !answer.getResult()) {
+                            s_logger.debug("Unable to stop due to " + (answer == null ? "answer is null" : answer.getDetails()));
+
+                            final EventVO event = new EventVO();
+                            event.setUserId(User.UID_SYSTEM);
+                            event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
+                            event.setType(EventTypes.EVENT_PROXY_STOP);
+                            event.setLevel(EventVO.LEVEL_ERROR);
+                            event.setStartId(startEventId);
+                            event.setDescription("Stopping console proxy failed due to negative answer from agent - " + proxy.getName());
+                            _eventDao.persist(event);
+                            return false;
+                        }
+                        completeStopCommand(proxy, Event.OperationSucceeded);
+
+                        SubscriptionMgr.getInstance().notifySubscribers(
+                                ConsoleProxyManager.ALERT_SUBJECT,
+                                this,
+                                new ConsoleProxyAlertEventArgs(ConsoleProxyAlertEventArgs.PROXY_DOWN, proxy.getDataCenterId(), proxy.getId(), proxy,
+                                        null));
+
+                        final EventVO event = new EventVO();
+                        event.setUserId(User.UID_SYSTEM);
+                        event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
+                        event.setType(EventTypes.EVENT_PROXY_STOP);
+                        event.setLevel(EventVO.LEVEL_INFO);
+                        event.setStartId(startEventId);
+                        event.setDescription("Console proxy stopped - " + proxy.getName());
+                        _eventDao.persist(event);
+                        return true;
+                    } catch (OperationTimedoutException e) {
+                        final EventVO event = new EventVO();
+                        event.setUserId(User.UID_SYSTEM);
+                        event.setAccountId(Account.ACCOUNT_ID_SYSTEM);
+                        event.setType(EventTypes.EVENT_PROXY_STOP);
+                        event.setLevel(EventVO.LEVEL_ERROR);
+                        event.setStartId(startEventId);
+                        event.setDescription("Stopping console proxy failed due to operation time out - " + proxy.getName());
+                        _eventDao.persist(event);
+                        throw new AgentUnavailableException(proxy.getHostId());
+                    }
+                } finally {
+                    proxyLock.unlock();
+                }
+            } else {
+                s_logger.debug("Unable to acquire console proxy lock : " + proxy.toString());
+                return false;
+            }
+        } finally {
+            proxyLock.releaseRef();
+        }
+    }
+
+    @Override
+    public boolean migrate(ConsoleProxyVO proxy, HostVO host) {
+        HostVO fromHost = _hostDao.findById(proxy.getId());
+
+        if (!_consoleProxyDao.updateIf(proxy, Event.MigrationRequested, proxy.getHostId())) {
+            s_logger.debug("State for " + proxy.toString() + " has changed so migration can not take place.");
+            return false;
+        }
+
+        MigrateCommand cmd = new MigrateCommand(proxy.getInstanceName(), host.getPrivateIpAddress(), false);
+        Answer answer = _agentMgr.easySend(fromHost.getId(), cmd);
+        if (answer == null) {
+            return false;
+        }
+
+        _storageMgr.unshare(proxy, fromHost);
+
+        return true;
+    }
+
+    @Override
+    public boolean completeMigration(ConsoleProxyVO proxy, HostVO host) throws AgentUnavailableException, OperationTimedoutException {
+
+        CheckVirtualMachineCommand cvm = new CheckVirtualMachineCommand(proxy.getInstanceName());
+        CheckVirtualMachineAnswer answer = (CheckVirtualMachineAnswer) _agentMgr.send(host.getId(), cvm);
+        if (!answer.getResult()) {
+            s_logger.debug("Unable to complete migration for " + proxy.getId());
+            _consoleProxyDao.updateIf(proxy, Event.AgentReportStopped, null);
+            return false;
+        }
+
+        State state = answer.getState();
+        if (state == State.Stopped) {
+            s_logger.warn("Unable to complete migration as we can not detect it on " + host.getId());
+            _consoleProxyDao.updateIf(proxy, Event.AgentReportStopped, null);
+            return false;
+        }
+
+        _consoleProxyDao.updateIf(proxy, Event.OperationSucceeded, host.getId());
+        return true;
+    }
+
+    @Override
+    public HostVO prepareForMigration(ConsoleProxyVO proxy) throws StorageUnavailableException {
+
+        VMTemplateVO template = _templateDao.findById(proxy.getTemplateId());
+        long routerId = proxy.getId();
+        boolean mirroredVols = proxy.isMirroredVols();
+        DataCenterVO dc = _dcDao.findById(proxy.getDataCenterId());
+        HostPodVO pod = _podDao.findById(proxy.getPodId());
+        List sps = _storageMgr.getStoragePoolsForVm(proxy.getId());
+        StoragePoolVO sp = sps.get(0); // FIXME
+
+        List vols = _volsDao.findCreatedByInstance(routerId);
+
+        String[] storageIps = new String[2];
+        VolumeVO vol = vols.get(0);
+        storageIps[0] = vol.getHostIp();
+        if (mirroredVols && (vols.size() == 2)) {
+            storageIps[1] = vols.get(1).getHostIp();
+        }
+
+        PrepareForMigrationCommand cmd = new PrepareForMigrationCommand(proxy.getName(), null, storageIps, vols, mirroredVols);
+
+        HostVO routingHost = null;
+        HashSet avoid = new HashSet();
+
+        HostVO fromHost = _hostDao.findById(proxy.getHostId());
+        if (fromHost.getClusterId() == null) {
+            s_logger.debug("The host is not in a cluster");
+            return null;
+        }
+        avoid.add(fromHost);
+
+        while ((routingHost = (HostVO) _agentMgr.findHost(Host.Type.Routing, dc, pod, sp, _serviceOffering, template, proxy, fromHost, avoid)) != null) {
+            avoid.add(routingHost);
+
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("Trying to migrate router to host " + routingHost.getName());
+            }
+
+            if (!_storageMgr.share(proxy, vols, routingHost, false)) {
+                s_logger.warn("Can not share " + proxy.getName());
+                throw new StorageUnavailableException(vol.getPoolId());
+            }
+
+            Answer answer = _agentMgr.easySend(routingHost.getId(), cmd);
+            if (answer != null && answer.getResult()) {
+                return routingHost;
+            }
+            _storageMgr.unshare(proxy, vols, routingHost);
+        }
+
+        return null;
+    }
+
+    private String getCapacityScanLockName() {
+        // to improve security, it may be better to return a unique mashed
+        // name(for example MD5 hashed)
+        return "consoleproxy.capacity.scan";
+    }
+
+    private String getAllocProxyLockName() {
+        // to improve security, it may be better to return a unique mashed
+        // name(for example MD5 hashed)
+        return "consoleproxy.alloc";
+    }
+
+    private String getProxyLockName(long id) {
+        return "consoleproxy." + id;
+    }
+
+    private Long saveStartedEvent(Long userId, Long accountId, String type, String description, long startEventId) {
+        EventVO event = new EventVO();
+        event.setUserId(userId);
+        event.setAccountId(accountId);
+        event.setType(type);
+        event.setState(EventState.Started);
+        event.setDescription(description);
+        event.setStartId(startEventId);
+        event = _eventDao.persist(event);
+        if (event != null)
+            return event.getId();
+        return null;
+    }
+
+    @Override
+    public boolean configure(String name, Map params) throws ConfigurationException {
+        if (s_logger.isInfoEnabled())
+            s_logger.info("Start configuring console proxy manager : " + name);
+
+        _name = name;
+
+        ComponentLocator locator = ComponentLocator.getCurrentLocator();
+        ConfigurationDao configDao = locator.getDao(ConfigurationDao.class);
+        if (configDao == null) {
+            throw new ConfigurationException("Unable to get the configuration dao.");
+        }
+
+        Map configs = configDao.getConfiguration("management-server", params);
+
+        _proxyRamSize = NumbersUtil.parseInt(configs.get("consoleproxy.ram.size"), DEFAULT_PROXY_VM_RAMSIZE);
+
+        String value = configs.get("start.retry");
+        _find_host_retry = NumbersUtil.parseInt(value, DEFAULT_FIND_HOST_RETRY_COUNT);
+
+        value = configs.get("consoleproxy.cmd.port");
+        _proxyCmdPort = NumbersUtil.parseInt(value, DEFAULT_PROXY_CMD_PORT);
+
+        value = configs.get("consoleproxy.sslEnabled");
+        if (value != null && value.equalsIgnoreCase("true"))
+            _sslEnabled = true;
+
+        value = configs.get("consoleproxy.capacityscan.interval");
+        _capacityScanInterval = NumbersUtil.parseLong(value, DEFAULT_CAPACITY_SCAN_INTERVAL);
+
+        _capacityPerProxy = NumbersUtil.parseInt(configs.get("consoleproxy.session.max"), DEFAULT_PROXY_CAPACITY);
+        _standbyCapacity = NumbersUtil.parseInt(configs.get("consoleproxy.capacity.standby"), DEFAULT_STANDBY_CAPACITY);
+        _proxySessionTimeoutValue = NumbersUtil.parseInt(configs.get("consoleproxy.session.timeout"), DEFAULT_PROXY_SESSION_TIMEOUT);
+
+        value = configs.get("consoleproxy.port");
+        if (value != null)
+            _consoleProxyPort = NumbersUtil.parseInt(value, ConsoleProxyManager.DEFAULT_PROXY_VNC_PORT);
+
+        value = configs.get("consoleproxy.url.port");
+        if (value != null)
+            _consoleProxyUrlPort = NumbersUtil.parseInt(value, ConsoleProxyManager.DEFAULT_PROXY_URL_PORT);
+
+        value = configs.get("system.vm.use.local.storage");
+        if (value != null && value.equalsIgnoreCase("true"))
+            _use_lvm = true;
+
+        value = configs.get("secondary.storage.vm");
+        if (value != null && value.equalsIgnoreCase("true"))
+            _use_storage_vm = true;
+
+        if (s_logger.isInfoEnabled()) {
+            s_logger.info("Console proxy max session soft limit : " + _capacityPerProxy);
+            s_logger.info("Console proxy standby capacity : " + _standbyCapacity);
+        }
+
+        _domain = configs.get("domain");
+        if (_domain == null) {
+            _domain = "foo.com";
+        }
+
+        _instance = configs.get("instance.name");
+        if (_instance == null) {
+            _instance = "DEFAULT";
+        }
+
+        value = (String) params.get("ssh.sleep");
+        _ssh_sleep = NumbersUtil.parseInt(value, 5) * 1000;
+
+        value = (String) params.get("ssh.retry");
+        _ssh_retry = NumbersUtil.parseInt(value, 3);
+
+        Map agentMgrConfigs = configDao.getConfiguration("AgentManager", params);
+        _mgmt_host = agentMgrConfigs.get("host");
+        if (_mgmt_host == null) {
+            s_logger.warn("Critical warning! Please configure your management server host address right after you have started your management server and then restart it, otherwise you won't be able to do console access");
+        }
+
+        value = agentMgrConfigs.get("port");
+        _mgmt_port = NumbersUtil.parseInt(value, 8250);
+
+        _consoleProxyAllocators = locator.getAdapters(ConsoleProxyAllocator.class);
+        if (_consoleProxyAllocators == null || !_consoleProxyAllocators.isSet()) {
+            throw new ConfigurationException("Unable to get proxy allocators");
+        }
+
+        _listener = new ConsoleProxyListener(this);
+        _agentMgr.registerForHostEvents(_listener, true, true, false);
+
+        Adapters ipAllocators = locator.getAdapters(IpAddrAllocator.class);
+        if (ipAllocators != null && ipAllocators.isSet()) {
+            Enumeration it = ipAllocators.enumeration();
+            _IpAllocator = it.nextElement();
+        }
+
+        HighAvailabilityManager haMgr = locator.getManager(HighAvailabilityManager.class);
+        if (haMgr != null) {
+            haMgr.registerHandler(VirtualMachine.Type.ConsoleProxy, this);
+        }
+
+        boolean useLocalStorage = Boolean.parseBoolean((String) params.get(Config.SystemVMUseLocalStorage.key()));
+        String networkRateStr = _configDao.getValue("network.throttling.rate");
+        String multicastRateStr = _configDao.getValue("multicast.throttling.rate");
+        _networkRate = ((networkRateStr == null) ? 200 : Integer.parseInt(networkRateStr));
+        _multicastRate = ((multicastRateStr == null) ? 10 : Integer.parseInt(multicastRateStr));
+        _serviceOffering = new ServiceOfferingVO("Fake Offering For DomP", 1, _proxyRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized,
+                useLocalStorage, true, null);
+        _serviceOffering.setUniqueName("Cloud.com-ConsoleProxy");
+        _serviceOffering = _offeringDao.persistSystemServiceOffering(_serviceOffering);
+        _template = _templateDao.findConsoleProxyTemplate();
+        if (_template == null) {
+            throw new ConfigurationException("Unable to find the template for console proxy VMs");
+        }
+        
+        _capacityScanScheduler.scheduleAtFixedRate(getCapacityScanTask(), STARTUP_DELAY, _capacityScanInterval, TimeUnit.MILLISECONDS);
+
+        if (s_logger.isInfoEnabled())
+            s_logger.info("Console Proxy Manager is configured.");
+        return true;
+    }
+
+    @Override
+    public boolean destroyConsoleProxy(DestroyConsoleProxyCmd cmd) throws ServerApiException{
+        Long proxyId = cmd.getId();
+        
+        // verify parameters
         ConsoleProxyVO proxy = _consoleProxyDao.findById(proxyId);
         if (proxy == null) {
-        	throw new ServerApiException (BaseCmd.PARAM_ERROR, "unable to find a console proxy with id " + proxyId);
+            throw new ServerApiException (BaseCmd.PARAM_ERROR, "unable to find a console proxy with id " + proxyId);
         }
         
         long eventId = EventUtils.saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_PROXY_DESTROY, "destroying console proxy with Id: "+proxyId);
         
         return destroyProxy(proxyId, eventId);
         
-	}
+    }
+
+    protected ConsoleProxyManagerImpl() {
+    }
 }
diff --git a/server/src/com/cloud/consoleproxy/StaticConsoleProxyManager.java b/server/src/com/cloud/consoleproxy/StaticConsoleProxyManager.java
index 780984ddf9d..549e0081048 100644
--- a/server/src/com/cloud/consoleproxy/StaticConsoleProxyManager.java
+++ b/server/src/com/cloud/consoleproxy/StaticConsoleProxyManager.java
@@ -26,6 +26,7 @@ import javax.naming.ConfigurationException;
 import com.cloud.configuration.dao.ConfigurationDao;
 import com.cloud.host.Host.Type;
 import com.cloud.host.HostVO;
+import com.cloud.info.ConsoleProxyInfo;
 import com.cloud.utils.component.ComponentLocator;
 import com.cloud.utils.component.Inject;
 import com.cloud.vm.ConsoleProxyVO;
@@ -46,31 +47,8 @@ public class StaticConsoleProxyManager extends AgentBasedConsoleProxyManager imp
     }
     
     @Override
-    public ConsoleProxyVO assignProxy(long dataCenterId, long userVmId) {
-        ConsoleProxyVO proxy =  new ConsoleProxyVO(1l, "EmbeddedProxy", null, null, null,
-                "02:02:02:02:02:02",
-                "127.0.0.1",
-                "255.255.255.0",
-                1l,
-                1l,
-                "03:03:03:03:03:03",
-                _ip,
-                "255.255.255.0",
-                null,
-                "untagged",
-                1l,
-                dataCenterId,
-                "0.0.0.0",
-                0L,
-                "dns1",
-                "dn2",
-                "domain",
-                0,
-                0);
-        
-		proxy.setPort(_consoleProxyUrlPort);
-		proxy.setSslEnabled(false);
-		return proxy;
+    public ConsoleProxyInfo assignProxy(long dataCenterId, long userVmId) {
+        return new ConsoleProxyInfo(false, _ip, _consoleProxyPort, _consoleProxyUrlPort);
     }
     
     @Override
diff --git a/utils/src/com/cloud/utils/db/DaoSearch.java b/server/src/com/cloud/deploy/PlannerBase.java
similarity index 81%
rename from utils/src/com/cloud/utils/db/DaoSearch.java
rename to server/src/com/cloud/deploy/PlannerBase.java
index bfe1536cdf3..970bf33d9b3 100644
--- a/utils/src/com/cloud/utils/db/DaoSearch.java
+++ b/server/src/com/cloud/deploy/PlannerBase.java
@@ -1,7 +1,7 @@
 /**
  *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
  * 
- * This software is licensed under the GNU General Public License v3 or later.  
+ * This software is licensed under the GNU General Public License v3 or later.
  * 
  * It is free software: you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -15,7 +15,10 @@
  * along with this program.  If not, see .
  * 
  */
-package com.cloud.utils.db;
+package com.cloud.deploy;
+
+import com.cloud.utils.component.AdapterBase;
+
+public abstract class PlannerBase extends AdapterBase implements DeploymentPlanner {
 
-public interface DaoSearch {
 }
diff --git a/server/src/com/cloud/deploy/SimplePlanner.java b/server/src/com/cloud/deploy/SimplePlanner.java
new file mode 100644
index 00000000000..c2f4c6674e8
--- /dev/null
+++ b/server/src/com/cloud/deploy/SimplePlanner.java
@@ -0,0 +1,69 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.deploy;
+
+import java.util.List;
+import java.util.Set;
+
+import javax.ejb.Local;
+
+import com.cloud.dc.DataCenterVO;
+import com.cloud.dc.Pod;
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.dc.dao.HostPodDao;
+import com.cloud.host.Host;
+import com.cloud.host.Host.Type;
+import com.cloud.host.HostVO;
+import com.cloud.host.dao.HostDao;
+import com.cloud.org.Cluster;
+import com.cloud.utils.component.Inject;
+import com.cloud.vm.VirtualMachineProfile;
+
+@Local(value=DeploymentPlanner.class)
+public class SimplePlanner extends PlannerBase implements DeploymentPlanner {
+    @Inject DataCenterDao _dcDao;
+    @Inject HostPodDao _podDao;
+    @Inject HostDao _hostDao;
+    @Inject ClusterDao _clusterDao;
+    
+    @Override
+    public DeployDestination plan(VirtualMachineProfile vm, DeploymentPlan plan, Set avoid) {
+        DataCenterVO dc = _dcDao.findById(plan.getDataCenterId());
+        List hosts = _hostDao.listBy(Type.Routing, plan.getDataCenterId());
+        
+        if (hosts.size() == 0) {
+            return null;
+        }
+        
+        Host host = hosts.get(0);
+        Pod pod = _podDao.findById(host.getPodId());
+        
+        Cluster cluster = null;
+        if (host.getClusterId() != null) {
+            cluster = _clusterDao.findById(host.getClusterId());
+        }
+        
+        return new DeployDestination(dc, pod, cluster, host);
+    }
+
+    
+    protected SimplePlanner() {
+        super();
+    }
+}
diff --git a/server/src/com/cloud/ha/HighAvailabilityManagerImpl.java b/server/src/com/cloud/ha/HighAvailabilityManagerImpl.java
index f04d8e6da8b..160fd660183 100644
--- a/server/src/com/cloud/ha/HighAvailabilityManagerImpl.java
+++ b/server/src/com/cloud/ha/HighAvailabilityManagerImpl.java
@@ -400,7 +400,7 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
             
             if (alive== null && !fenced) {
             	s_logger.debug("We were unable to fence off the VM " + vm.toString());
-                _alertMgr.sendAlert(alertType, vm.getDataCenterId(), vm.getPodId(), "Unable to restart " + vm.getName() + " which was running on host " + hostDesc, "Insufficient capcity to restart VM, name: " + vm.getName() + ", id: " + vmId + " which was running on host " + hostDesc);
+                _alertMgr.sendAlert(alertType, vm.getDataCenterId(), vm.getPodId(), "Unable to restart " + vm.getName() + " which was running on host " + hostDesc, "Insufficient capacity to restart VM, name: " + vm.getName() + ", id: " + vmId + " which was running on host " + hostDesc);
             	return (System.currentTimeMillis() >> 10) + _restartRetryInterval;
             }
 
@@ -453,7 +453,7 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
             return (System.currentTimeMillis() >> 10) + _restartRetryInterval;
         } catch (final InsufficientCapacityException e) {
         	s_logger.warn("Unable to restart " + vm.toString() + " due to " + e.getMessage());
-            _alertMgr.sendAlert(alertType, vm.getDataCenterId(), vm.getPodId(), "Unable to restart " + vm.getName() + " which was running on host " + hostDesc, "Insufficient capcity to restart VM, name: " + vm.getName() + ", id: " + vmId + " which was running on host " + hostDesc);
+            _alertMgr.sendAlert(alertType, vm.getDataCenterId(), vm.getPodId(), "Unable to restart " + vm.getName() + " which was running on host " + hostDesc, "Insufficient capacity to restart VM, name: " + vm.getName() + ", id: " + vmId + " which was running on host " + hostDesc);
             return null;
         } catch (final StorageUnavailableException e) {
         	s_logger.warn("Unable to restart " + vm.toString() + " due to " + e.getMessage());
@@ -722,7 +722,7 @@ public class HighAvailabilityManagerImpl implements HighAvailabilityManager {
                     _alertMgr.sendAlert(alertType, vm.getDataCenterId(), vm.getPodId(), "Unable to migrate vm " + vm.getName() + " from host " + fromHostName + " in zone " + dcVO.getName() + " and pod " + podVO.getName(), "Unable to find a suitable host");
                 }
             } catch(final InsufficientCapacityException e) {
-            	s_logger.warn("Unable to mgirate due to insufficient capcity " + vm.toString());
+            	s_logger.warn("Unable to mgirate due to insufficient capacity " + vm.toString());
                 _alertMgr.sendAlert(alertType, vm.getDataCenterId(), vm.getPodId(), "Unable to migrate vm " + vm.getName() + " from host " + fromHostName + " in zone " + dcVO.getName() + " and pod " + podVO.getName(), "Insufficient capacity");
             } catch(final StorageUnavailableException e) {
                 s_logger.warn("Storage is unavailable: " + vm.toString());
diff --git a/server/src/com/cloud/hypervisor/kvm/discoverer/KvmServerDiscoverer.java b/server/src/com/cloud/hypervisor/kvm/discoverer/KvmServerDiscoverer.java
new file mode 100644
index 00000000000..9b910ec0f3b
--- /dev/null
+++ b/server/src/com/cloud/hypervisor/kvm/discoverer/KvmServerDiscoverer.java
@@ -0,0 +1,285 @@
+package com.cloud.hypervisor.kvm.discoverer;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.net.InetAddress;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+
+import javax.ejb.Local;
+import javax.naming.ConfigurationException;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.Listener;
+import com.cloud.agent.api.AgentControlAnswer;
+import com.cloud.agent.api.AgentControlCommand;
+import com.cloud.agent.api.Answer;
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartupCommand;
+import com.cloud.configuration.dao.ConfigurationDao;
+import com.cloud.exception.DiscoveryException;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.Status.Event;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.kvm.resource.KvmDummyResourceBase;
+import com.cloud.hypervisor.xen.resource.CitrixResourceBase;
+import com.cloud.resource.Discoverer;
+import com.cloud.resource.DiscovererBase;
+import com.cloud.resource.ServerResource;
+import com.cloud.utils.component.ComponentLocator;
+import com.cloud.utils.component.Inject;
+import com.cloud.utils.script.Script;
+import com.trilead.ssh2.ChannelCondition;
+import com.trilead.ssh2.SCPClient;
+import com.trilead.ssh2.Session;
+
+@Local(value=Discoverer.class)
+public class KvmServerDiscoverer extends DiscovererBase implements Discoverer,
+		Listener {
+	 private static final Logger s_logger = Logger.getLogger(KvmServerDiscoverer.class);
+	 private String _setupAgentPath;
+	 private ConfigurationDao _configDao;
+	 private String _hostIp;
+	 private int _waitTime = 3; /*wait for 3 minutes*/
+	 @Inject HostDao _hostDao = null;
+	 
+	@Override
+	public boolean processAnswer(long agentId, long seq, Answer[] answers) {
+		// TODO Auto-generated method stub
+		return false;
+	}
+
+	@Override
+	public boolean processCommand(long agentId, long seq, Command[] commands) {
+		// TODO Auto-generated method stub
+		return false;
+	}
+
+	@Override
+	public AgentControlAnswer processControlCommand(long agentId,
+			AgentControlCommand cmd) {
+		// TODO Auto-generated method stub
+		return null;
+	}
+
+	@Override
+	public boolean processConnect(HostVO host, StartupCommand cmd) {
+		// TODO Auto-generated method stub
+		return false;
+	}
+
+	@Override
+	public boolean processDisconnect(long agentId, Status state) {
+		// TODO Auto-generated method stub
+		return false;
+	}
+
+	@Override
+	public boolean isRecurring() {
+		// TODO Auto-generated method stub
+		return false;
+	}
+
+	@Override
+	public int getTimeout() {
+		// TODO Auto-generated method stub
+		return 0;
+	}
+
+	@Override
+	public boolean processTimeout(long agentId, long seq) {
+		// TODO Auto-generated method stub
+		return false;
+	}
+	
+	private static boolean sshExecuteCmd(com.trilead.ssh2.Connection sshConnection, String cmd) {
+		s_logger.debug("Executing cmd: " + cmd);
+		Session sshSession = null;
+		try {
+			sshSession = sshConnection.openSession();
+			// There is a bug in Trilead library, wait a second before
+			// starting a shell and executing commands, from http://spci.st.ewi.tudelft.nl/chiron/xref/nl/tudelft/swerl/util/SSHConnection.html
+			Thread.sleep(1000);
+
+			if (sshSession == null) {
+				return false;
+			}
+			
+			sshSession.execCommand(cmd);
+			
+			InputStream stdout = sshSession.getStdout();
+			InputStream stderr = sshSession.getStderr();
+			
+	
+			byte[] buffer = new byte[8192];
+			while (true) {
+				if (stdout == null || stderr == null) {
+					return false;
+				}
+				
+				if ((stdout.available() == 0) && (stderr.available() == 0)) {
+					int conditions = sshSession.waitForCondition(
+							ChannelCondition.STDOUT_DATA
+							| ChannelCondition.STDERR_DATA
+							| ChannelCondition.EOF, 120000);
+					
+					if ((conditions & ChannelCondition.TIMEOUT) != 0) {
+						s_logger.info("Timeout while waiting for data from peer.");
+						break;
+					}
+
+					if ((conditions & ChannelCondition.EOF) != 0) {
+						if ((conditions & (ChannelCondition.STDOUT_DATA | ChannelCondition.STDERR_DATA)) == 0) {							
+							break;
+						}
+					}
+				}
+							
+				while (stdout.available() > 0) {
+					stdout.read(buffer);
+				}
+			
+				while (stderr.available() > 0) {
+					stderr.read(buffer);
+				}
+			}
+			
+			Thread.sleep(1000);
+			if (sshSession.getExitStatus() != 0) {
+				return false;
+			}
+			
+			return true;
+		} catch (IOException e) {
+			s_logger.debug("Executing cmd: " + cmd + " failed, due to: " + e.toString());
+			return false;
+		} catch (InterruptedException e) {
+			return false;
+		} catch (Exception e) {
+			return false;
+		}	finally {
+			if (sshSession != null)
+				sshSession.close();
+		}
+	}
+	
+	private static boolean sshExecuteCmd(com.trilead.ssh2.Connection sshConnection, String cmd, int nTimes) {
+		for (int i = 0; i < nTimes; i ++) {
+			if (sshExecuteCmd(sshConnection, cmd))
+				return true;
+		}
+		return false;
+	}
+
+	@Override
+	public Map> find(long dcId,
+			Long podId, Long clusterId, URI uri, String username,
+			String password) throws DiscoveryException {
+		 Map> resources = new HashMap>();
+		 Map details = new HashMap();
+		if (!uri.getScheme().equals("http")) {
+            String msg = "urlString is not kvm so we're not taking care of the discovery for this: " + uri;
+            s_logger.debug(msg);
+            return null;
+		}
+		com.trilead.ssh2.Connection sshConnection = null;
+		String agentIp = null;
+		try {
+			
+			String hostname = uri.getHost();
+			InetAddress ia = InetAddress.getByName(hostname);
+			agentIp = ia.getHostAddress();
+			String guid = UUID.nameUUIDFromBytes(agentIp.getBytes()).toString();
+			sshConnection = new com.trilead.ssh2.Connection(agentIp, 22);
+
+			sshConnection.connect(null, 60000, 60000);
+			if (!sshConnection.authenticateWithPassword(username, password)) {
+				throw new Exception("Unable to authenticate");
+			}
+			
+			if (!sshExecuteCmd(sshConnection, "lsmod|grep kvm >& /dev/null", 3)) {
+				s_logger.debug("It's not a KVM enabled machine");
+				return null;
+			}
+			
+			s_logger.debug("copying " + _setupAgentPath + " to host");
+			SCPClient scp = new SCPClient(sshConnection);
+			scp.put(_setupAgentPath, "/usr/bin", "0755");
+		
+			sshExecuteCmd(sshConnection, "/usr/bin/setup_agent.sh " + " -h " + _hostIp + " -z " + dcId + " -p " + podId + " -c " + clusterId + " -u " + guid + " 1>&2", 3);
+			
+			KvmDummyResourceBase kvmResource = new KvmDummyResourceBase();
+			Map params = new HashMap();
+			
+			params.put("zone", Long.toString(dcId));
+			params.put("pod", Long.toString(podId));
+			params.put("cluster",  Long.toString(clusterId));
+			params.put("guid", guid + "-LibvirtComputingResource"); /*tail added by agent.java*/
+			params.put("agentIp", agentIp);
+			kvmResource.configure("kvm agent", params);
+			kvmResource.setRemoteAgent(true);
+			resources.put(kvmResource, details);
+			return resources;
+		} catch (Exception e) {
+			String msg = " can't setup agent, due to " + e.toString() + " - " + e.getMessage();
+			s_logger.warn(msg);
+		} finally {
+			if (sshConnection != null)
+				sshConnection.close();
+		}
+		
+		return null;
+	}
+
+	@Override
+	public void postDiscovery(List hosts, long msId) throws DiscoveryException {
+		/*Wait for agent coming back*/
+		if (hosts.isEmpty()) {
+			return;
+		}
+		HostVO host = hosts.get(0);
+		for (int i = 0 ; i < _waitTime; i++) {
+			
+			if (host.getStatus() != Status.Up) {
+				s_logger.debug("Wait host comes back, try: " + i);
+				try {
+					Thread.sleep(60000);
+				} catch (InterruptedException e) {
+					s_logger.debug("Failed to sleep: " + e.toString());
+				}
+			} else {
+				return;
+			}
+		}
+		
+		
+		_hostDao.updateStatus(host, Event.AgentDisconnected, msId);
+		/*Timeout, throw warning msg to user*/
+		throw new DiscoveryException("Host " + host.getId() + ":" + host.getPrivateIpAddress() + " does not come back, It may connect to server later, if not, please check the agent log on this host");
+	}
+	
+	@Override
+    public boolean configure(String name, Map params) throws ConfigurationException {
+		ComponentLocator locator = ComponentLocator.getCurrentLocator();
+        _configDao = locator.getDao(ConfigurationDao.class);
+		_setupAgentPath = Script.findScript(getPatchPath(), "setup_agent.sh");
+		
+		if (_setupAgentPath == null) {
+			throw new ConfigurationException("Can't find setup_agent.sh");
+		}
+		_hostIp = _configDao.getValue("host");
+		if (_hostIp == null) {
+			throw new ConfigurationException("Can't get host IP");
+		}
+		return true;
+	}
+	
+	protected String getPatchPath() {
+        return "scripts/vm/hypervisor/kvm/";
+    }
+}
diff --git a/server/src/com/cloud/hypervisor/vmware/discoverer/VmwareServerDiscoverer.java b/server/src/com/cloud/hypervisor/vmware/discoverer/VmwareServerDiscoverer.java
deleted file mode 100644
index a7f186a19e0..00000000000
--- a/server/src/com/cloud/hypervisor/vmware/discoverer/VmwareServerDiscoverer.java
+++ /dev/null
@@ -1,40 +0,0 @@
-package com.cloud.hypervisor.vmware.discoverer;
-
-import java.net.URI;
-import java.util.List;
-import java.util.Map;
-
-import javax.ejb.Local;
-import javax.naming.ConfigurationException;
-
-import com.cloud.exception.DiscoveryException;
-import com.cloud.host.HostVO;
-import com.cloud.resource.Discoverer;
-import com.cloud.resource.DiscovererBase;
-import com.cloud.resource.ServerResource;
-
-@Local(value=Discoverer.class)
-public class VmwareServerDiscoverer extends DiscovererBase implements Discoverer {
-	
-    @Override
-    public Map> find(long dcId, Long podId, Long clusterId, URI url, 
-    	String username, String password) throws DiscoveryException {
-
-    	// ???
-    	return null;
-    }
-    
-    @Override
-    public void postDiscovery(List hosts, long msId) {
-        //do nothing
-    }
-
-    @Override
-    public boolean configure(String name, Map params) throws ConfigurationException {
-        super.configure(name, params);
-    
-        // TODO
-        return true;
-    }
-}
-
diff --git a/server/src/com/cloud/hypervisor/xen/discoverer/XcpServerDiscoverer.java b/server/src/com/cloud/hypervisor/xen/discoverer/XcpServerDiscoverer.java
index 5e4a55c29d5..e1d3005a80f 100644
--- a/server/src/com/cloud/hypervisor/xen/discoverer/XcpServerDiscoverer.java
+++ b/server/src/com/cloud/hypervisor/xen/discoverer/XcpServerDiscoverer.java
@@ -52,6 +52,7 @@ import com.cloud.host.Status;
 import com.cloud.host.dao.HostDao;
 import com.cloud.hypervisor.xen.resource.CitrixResourceBase;
 import com.cloud.hypervisor.xen.resource.XcpServerResource;
+import com.cloud.hypervisor.xen.resource.XenServerResource;
 import com.cloud.hypervisor.xen.resource.XenServerConnectionPool;
 import com.cloud.resource.Discoverer;
 import com.cloud.resource.DiscovererBase;
@@ -81,7 +82,6 @@ public class XcpServerDiscoverer extends DiscovererBase implements Discoverer, L
     protected String _privateNic;
     protected String _storageNic1;
     protected String _storageNic2;
-    protected boolean _formPoolsInPod;
     protected int _wait;
     protected XenServerConnectionPool _connPool;
     protected String _increase;
@@ -102,6 +102,7 @@ public class XcpServerDiscoverer extends DiscovererBase implements Discoverer, L
     public Map> find(long dcId, Long podId, Long clusterId, URI url, String username, String password) throws DiscoveryException {
         Map> resources = new HashMap>();
         Connection conn = null;
+        Connection slaveConn = null;
         if (!url.getScheme().equals("http")) {
             String msg = "urlString is not http so we're not taking care of the discovery for this: " + url;
             s_logger.debug(msg);
@@ -113,7 +114,8 @@ public class XcpServerDiscoverer extends DiscovererBase implements Discoverer, L
             InetAddress ia = InetAddress.getByName(hostname);
             String addr = ia.getHostAddress();
             
-            conn = _connPool.connect(addr, username, password, _wait);
+            conn = _connPool.masterConnect(addr, username, password);
+            
             if (conn == null) {
                 String msg = "Unable to get a connection to " + url;
                 s_logger.debug(msg);
@@ -132,47 +134,36 @@ public class XcpServerDiscoverer extends DiscovererBase implements Discoverer, L
             String cluster = null;
             if (clusterId != null) {
                 cluster = Long.toString(clusterId);
-            } else if (_formPoolsInPod) {
-                if (podId != null) {
-                    List clusters = _clusterDao.listByPodId(podId);
-                    if (clusters.size() > 1) {
-                        throw new DiscoveryException("There are more than one cluster in the pod and we don't know which to add to.");
-                    } else if (clusters.size() == 1) {
-                        clusterId = clusters.get(0).getId();
-                        cluster = Long.toString(clusterId);
-                    } else {
-                        Map pools = Pool.getAllRecords(conn);
-                        cluster = pools.values().iterator().next().uuid;
-                    }
-                }
             }
-            
+            Set pools = Pool.getAll(conn);
+            Pool pool = pools.iterator().next();
+            Pool.Record pr = pool.getRecord(conn);
+            String poolUuid = pr.uuid;
             Map hosts = Host.getAllRecords(conn);
-            
+            Host master = pr.master;
+           
             if (_checkHvm) {
-            for (Map.Entry entry : hosts.entrySet()) {
-                Host.Record record = entry.getValue();
-                
-                boolean support_hvm = false;
-                for ( String capability : record.capabilities ) {
-                    if(capability.contains("hvm")) {
-                       support_hvm = true;
-                       break;
+                for (Map.Entry entry : hosts.entrySet()) {
+                    Host.Record record = entry.getValue();
+                    
+                    boolean support_hvm = false;
+                    for ( String capability : record.capabilities ) {
+                        if(capability.contains("hvm")) {
+                           support_hvm = true;
+                           break;
+                        }
+                    } 
+                    if( !support_hvm ) {
+                        String msg = "Unable to add host " + record.address + " because it doesn't support hvm";
+                        _alertMgr.sendAlert(AlertManager.ALERT_TYPE_HOST, dcId, podId, msg, msg);
+                        s_logger.debug(msg);
+                        throw new RuntimeException(msg);
                     }
                 }
-                if( !support_hvm ) {
-                    String msg = "Unable to add host " + record.address + " because it doesn't support hvm";
-                    _alertMgr.sendAlert(AlertManager.ALERT_TYPE_HOST, dcId, podId, msg, msg);
-                    s_logger.debug(msg);
-                    throw new RuntimeException(msg);
-                    }
-                }
-
             }
+
             for (Map.Entry entry : hosts.entrySet()) {
                 Host.Record record = entry.getValue();
-                Host host = entry.getKey();
-                
                 String hostAddr = record.address;
                 
                 String prodVersion = record.softwareVersion.get("product_version");
@@ -184,20 +175,17 @@ public class XcpServerDiscoverer extends DiscovererBase implements Discoverer, L
                 if (_hostDao.findByGuid(record.uuid) != null) {
                     s_logger.debug("Skipping " + record.address + " because " + record.uuid + " is already in the database.");
                     continue;
-                }
-                
-                
-                if (!checkServer(conn, dcId, podId, host, record)) {
-                    continue;
-                }
-                    
+                }                
+
+                CitrixResourceBase resource = createServerResource(dcId, podId, record);
                 s_logger.info("Found host " + record.hostname + " ip=" + record.address + " product version=" + prodVersion);
-                CitrixResourceBase resource = createServerResource(record);
                             
                 Map details = new HashMap();
                 Map params = new HashMap();
                 details.put("url", hostAddr);
                 params.put("url", hostAddr);
+                details.put("pool", poolUuid);
+                params.put("pool", poolUuid);
                 details.put("username", username);
                 params.put("username", username);
                 details.put("password", password);
@@ -271,10 +259,13 @@ public class XcpServerDiscoverer extends DiscovererBase implements Discoverer, L
             return null;
         } finally {
             if (conn != null) {
-                XenServerConnectionPool.logout(conn);
+                try{
+                    Session.logout(conn);
+                } catch (Exception e ) {
+                }
+                conn.dispose();
             }
         }
-        
         return resources;
     }
     
@@ -300,7 +291,7 @@ public class XcpServerDiscoverer extends DiscovererBase implements Discoverer, L
         if ( resources.size() == 0 ) {
             return false;
         }
-        if (clusterId == null && !_formPoolsInPod) {
+        if (clusterId == null ) {
             if (resources.size() > 1) {
                 s_logger.warn("There's no cluster specified but we found a pool of xenservers " + resources.size());
                 throw new DiscoveryException("There's no cluster specified but we found a pool of xenservers " + resources.size());
@@ -351,20 +342,30 @@ public class XcpServerDiscoverer extends DiscovererBase implements Discoverer, L
             username = host.getDetail("username");
             password = host.getDetail("password");
             address = host.getDetail("url");
-            Connection hostConn = _connPool.connect(address, username, password, _wait);
+            Connection hostConn = _connPool.slaveConnect(address, username, password);
+            if (hostConn == null) {
+                continue;
+            }
             try {
-                if (hostConn == null) {
-                    continue;
-                }
                 Set pools = Pool.getAll(hostConn);
                 Pool pool = pools.iterator().next();
                 poolUuid1 = pool.getUuid(hostConn);
                 poolMaster = pool.getMaster(hostConn).getAddress(hostConn);
-                Session.logout(hostConn);
-            } finally {
-                hostConn.dispose();
+                break;
+
+            } catch (Exception e ) {
+                s_logger.warn("Can not get master ip address from host " + address);
+            }
+            finally {
+                try{
+                    Session.localLogout(hostConn);
+                } catch (Exception e ) {
+                }
+                hostConn.dispose();
+                hostConn = null;
+                poolMaster = null;
+                poolUuid1 = null;
             }
-            break;
         }
         
         if (poolMaster == null) {
@@ -393,22 +394,21 @@ public class XcpServerDiscoverer extends DiscovererBase implements Discoverer, L
         
         return true;
     }
-
-    protected boolean checkServer(Connection conn, long dcId, Long podId, Host host, Host.Record record) {
+    
+    protected CitrixResourceBase createServerResource(long dcId, Long podId, Host.Record record) {
         String prodBrand = record.softwareVersion.get("product_brand").trim();
         String prodVersion = record.softwareVersion.get("product_version").trim();
         
-        if(!prodBrand.equals("XenCloudPlatform") || !prodVersion.equals("0.1.1")) {
-            String msg = "Only support XCP 0.1.1, but this one is " + prodBrand + " " + prodVersion;
-            _alertMgr.sendAlert(AlertManager.ALERT_TYPE_HOST, dcId, podId, msg, msg);
-            s_logger.debug(msg);
-            throw new RuntimeException(msg);
-        }
-        return true;
-    }
-    
-    protected CitrixResourceBase createServerResource(Host.Record record) {
-        return new XcpServerResource();
+        if(prodBrand.equals("XenCloudPlatform") && prodVersion.equals("0.1.1")) 
+        	return new XcpServerResource();
+        
+        if(prodBrand.equals("XenServer") && prodVersion.equals("5.6.0")) 
+        	return new XenServerResource();
+        
+        String msg = "Only support XCP 0.1.1 and XenServer 5.6.0, but this one is " + prodBrand + " " + prodVersion;
+        _alertMgr.sendAlert(AlertManager.ALERT_TYPE_HOST, dcId, podId, msg, msg);
+        s_logger.debug(msg);
+        throw new RuntimeException(msg);
     }
     
     protected void serverConfig() {
@@ -439,9 +439,6 @@ public class XcpServerDiscoverer extends DiscovererBase implements Discoverer, L
         value = _params.get("xen.check.hvm");
         _checkHvm = value == null ? true : Boolean.parseBoolean(value);
         
-        value = _params.get(Config.CreatePoolsInPod.key());
-        _formPoolsInPod = Boolean.parseBoolean(value);
-        
         _connPool = XenServerConnectionPool.getInstance();
         
         _agentMgr.registerForHostEvents(this, true, false, true);
@@ -450,7 +447,7 @@ public class XcpServerDiscoverer extends DiscovererBase implements Discoverer, L
     }
 
     @Override
-    public void postDiscovery(List hosts, long msId) {
+    public void postDiscovery(List hosts, long msId)  throws DiscoveryException{
         //do nothing
     }
 
diff --git a/server/src/com/cloud/maid/StackMaidManagerImpl.java b/server/src/com/cloud/maid/StackMaidManagerImpl.java
index 3c68a86ecab..073244dcd32 100644
--- a/server/src/com/cloud/maid/StackMaidManagerImpl.java
+++ b/server/src/com/cloud/maid/StackMaidManagerImpl.java
@@ -52,7 +52,7 @@ public class StackMaidManagerImpl implements StackMaidManager {
 	private void cleanupLeftovers(List l) {
 		for(StackMaidVO maid : l) {
 			StackMaid.doCleanup(maid);
-			_maidDao.delete(maid.getId());
+			_maidDao.expunge(maid.getId());
 		}
 	}
 	
diff --git a/server/src/com/cloud/maint/UpgradeManager.java b/server/src/com/cloud/maint/UpgradeManager.java
index 490a325bb5b..10fce57bd18 100644
--- a/server/src/com/cloud/maint/UpgradeManager.java
+++ b/server/src/com/cloud/maint/UpgradeManager.java
@@ -43,7 +43,7 @@ public interface UpgradeManager extends Manager {
     /**
      * @return the URL to download the new agent.
      */
-    String getAgentUrl();
+//    String getAgentUrl();
     
 
 }
diff --git a/server/src/com/cloud/maint/UpgradeManagerImpl.java b/server/src/com/cloud/maint/UpgradeManagerImpl.java
index de3b2f0a5d7..06b4ddd84d2 100644
--- a/server/src/com/cloud/maint/UpgradeManagerImpl.java
+++ b/server/src/com/cloud/maint/UpgradeManagerImpl.java
@@ -59,7 +59,7 @@ public class UpgradeManagerImpl implements UpgradeManager {
     String _name;
     String _minimalVersion;
     String _recommendedVersion;
-    String _upgradeUrl;
+//    String _upgradeUrl;
     String _agentPath;
     long _checkInterval;
     
@@ -137,10 +137,10 @@ public class UpgradeManagerImpl implements UpgradeManager {
         return "File will be deployed.";
     }
     
-    @Override
-    public String getAgentUrl() {
-        return _upgradeUrl;
-    }
+//    @Override
+//    public String getAgentUrl() {
+//        return _upgradeUrl;
+//    }
 
     @Override
     public boolean configure(String name, Map params) throws ConfigurationException {
@@ -177,12 +177,12 @@ public class UpgradeManagerImpl implements UpgradeManager {
                 _recommendedVersion = _minimalVersion;
             }
 
-            _upgradeUrl = configs.get("upgrade.url");
+            //_upgradeUrl = configs.get("upgrade.url");
             
-			if (_upgradeUrl == null) {
-				s_logger.debug("There is no upgrade url found in configuration table");
-                // _upgradeUrl = "http://updates.vmops.com/releases/rss.xml";
-            }
+//			if (_upgradeUrl == null) {
+//				s_logger.debug("There is no upgrade url found in configuration table");
+//                // _upgradeUrl = "http://updates.vmops.com/releases/rss.xml";
+//            }
 
             return true;
         } catch (IOException ex) {
diff --git a/server/src/com/cloud/migration/Db20to21MigrationUtil.java b/server/src/com/cloud/migration/Db20to21MigrationUtil.java
index fe5c843d518..cee7727c302 100644
--- a/server/src/com/cloud/migration/Db20to21MigrationUtil.java
+++ b/server/src/com/cloud/migration/Db20to21MigrationUtil.java
@@ -38,7 +38,7 @@ import com.cloud.host.Host;
 import com.cloud.host.HostVO;
 import com.cloud.host.dao.HostDao;
 import com.cloud.migration.DiskOffering21VO.Type;
-import com.cloud.offering.ServiceOffering.GuestIpType;
+import com.cloud.offering.NetworkOffering;
 import com.cloud.storage.Storage.StoragePoolType;
 import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.Volume.VolumeType;
@@ -125,7 +125,7 @@ public class Db20to21MigrationUtil {
         sb.done();
         
         SearchCriteria sc = sb.create();
-        List results = _dcDao.searchAll(sc, (Filter)null);
+        List results = _dcDao.searchIncludingRemoved(sc, (Filter)null);
         if(results.size() > 0) {
         	System.out.println("We've found following zones are deployed in your database");
         	for(DcPod cols : results) {
@@ -152,7 +152,7 @@ public class Db20to21MigrationUtil {
 		SearchCriteria sc = sb.create();
 		sc.setParameters("zoneId", zoneId);
 		
-		List pods = _podDao.searchAll(sc, null, false, false);
+		List pods = _podDao.searchIncludingRemoved(sc, null, false, false);
 		if(pods.size() > 0) {
 			for(HostPodVO pod : pods) {
 				System.out.println("Migrating pod " + pod.getName() + " in zone " + zoneName + "...");
@@ -201,7 +201,7 @@ public class Db20to21MigrationUtil {
 		sc.setParameters("type", Host.Type.Routing);
 		
 		// join cluster for hosts in pod
-		List hostsInPod = _hostDao.searchAll(sc, null, false, false);
+		List hostsInPod = _hostDao.searchIncludingRemoved(sc, null, false, false);
 		if(hostsInPod.size() > 0) {
 			if(cluster == null) {
 				cluster = new ClusterVO(zoneId, podId, String.valueOf(podId));
@@ -227,7 +227,7 @@ public class Db20to21MigrationUtil {
 		scPool.setParameters("pod", podId);
 		scPool.setParameters("poolType", StoragePoolType.NetworkFilesystem.toString(), StoragePoolType.IscsiLUN.toString());
 		
-		List sPoolsInPod = _spDao.searchAll(scPool, null, false, false);
+		List sPoolsInPod = _spDao.searchIncludingRemoved(scPool, null, false, false);
 		if(sPoolsInPod.size() > 0) {
 			if(cluster == null) {
 				cluster = new ClusterVO(zoneId, podId, String.valueOf(podId));
@@ -264,7 +264,7 @@ public class Db20to21MigrationUtil {
 		System.out.println("Migrating domains...");
 
 		// we shouldn't have too many domains in the system, use a very dumb way to setup domain path
-		List domains = _domainDao.listAll();
+		List domains = _domainDao.listAllIncludingRemoved();
 		for(DomainVO domain : domains) {
 			StringBuilder path = new StringBuilder();
 			composeDomainPath(domain, path);
@@ -283,7 +283,7 @@ public class Db20to21MigrationUtil {
 		
 		long seq = getServiceOfferingStartSequence();
 
-		List oldServiceOfferings = _serviceOffering20Dao.listAll();
+		List oldServiceOfferings = _serviceOffering20Dao.listAllIncludingRemoved();
 		for(ServiceOffering20VO so20 : oldServiceOfferings) {
 			ServiceOffering21VO so21 = new ServiceOffering21VO(so20.getName(), so20.getCpu(), so20.getRamSize(), so20.getSpeed(), so20.getRateMbps(),
 				so20.getMulticastRateMbps(), so20.getOfferHA(), so20.getDisplayText(), so20.getGuestIpType(),
@@ -305,7 +305,7 @@ public class Db20to21MigrationUtil {
 			_configDao.getValue(Config.ConsoleProxyRamSize.key()), 
 			ConsoleProxyManager.DEFAULT_PROXY_VM_RAMSIZE);
 		ServiceOffering21VO soConsoleProxy = new ServiceOffering21VO("Fake Offering For DomP", 1,
-			proxyRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized,
+			proxyRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized,
 			useLocalStorage, true, null);
 		soConsoleProxy.setId(seq++);
 		soConsoleProxy.setUniqueName("Cloud.com-ConsoleProxy");
@@ -316,7 +316,7 @@ public class Db20to21MigrationUtil {
 			_configDao.getValue(Config.SecStorageVmRamSize.key()), 
 			SecondaryStorageVmManager.DEFAULT_SS_VM_RAMSIZE);
 		ServiceOffering21VO soSecondaryVm = new ServiceOffering21VO("Fake Offering For Secondary Storage VM", 1, 
-			secStorageVmRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized, useLocalStorage, true, null);
+			secStorageVmRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, useLocalStorage, true, null);
 		soSecondaryVm.setId(seq++);
 		soSecondaryVm.setUniqueName("Cloud.com-SecondaryStorage");
 		soSecondaryVm = _serviceOffering21Dao.persist(soSecondaryVm);
@@ -324,7 +324,7 @@ public class Db20to21MigrationUtil {
 		
         int routerRamSize = NumbersUtil.parseInt(_configDao.getValue("router.ram.size"), 128);
         ServiceOffering21VO soDomainRouter = new ServiceOffering21VO("Fake Offering For DomR", 1, 
-        	routerRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized, useLocalStorage, true, null);
+        	routerRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, useLocalStorage, true, null);
         soDomainRouter.setId(seq++);
         soDomainRouter.setUniqueName("Cloud.Com-SoftwareRouter");
         soDomainRouter = _serviceOffering21Dao.persist(soDomainRouter);
@@ -434,7 +434,7 @@ public class Db20to21MigrationUtil {
 	private void migrateDiskOfferings() {
 		System.out.println("Migrating disk offering...");
 		
-		List oldDiskOfferings = _diskOffering20Dao.listAll();
+		List oldDiskOfferings = _diskOffering20Dao.listAllIncludingRemoved();
 		long maxDiskOfferingId = _domRServiceOfferingId;
 		maxDiskOfferingId += 100;		
 		
@@ -459,7 +459,7 @@ public class Db20to21MigrationUtil {
 		System.out.println("Fixup NULL disk_offering_id references in volumes table ...");
 		
 		SearchCriteria scDiskOffering = _diskOffering21Dao.createSearchCriteria();
-		List offeringList = _diskOffering21Dao.searchAll(scDiskOffering, 
+		List offeringList = _diskOffering21Dao.searchIncludingRemoved(scDiskOffering, 
 			new Filter(DiskOffering21VO.class, "diskSize", true, null, null), false, false);
 		
 		for(DiskOffering21VO offering : offeringList) {
@@ -471,7 +471,7 @@ public class Db20to21MigrationUtil {
 		sb.done();
 		
 		SearchCriteria sc = sb.create();
-		List volumes = _volumeDao.searchAll(sc, null, false, false);
+		List volumes = _volumeDao.searchIncludingRemoved(sc, null, false, false);
 		
 		if(volumes.size() > 0) {
 			for(VolumeVO vol : volumes) {
@@ -557,7 +557,7 @@ public class Db20to21MigrationUtil {
 		SearchCriteria sc = sb.create();
 		sc.setParameters("zoneId", zoneId);
 		
-		List proxies =_consoleProxyDao.searchAll(sc, null, false, false);
+		List proxies =_consoleProxyDao.searchIncludingRemoved(sc, null, false, false);
 		for(ConsoleProxyVO proxy : proxies) {
 			String[] macAddresses = _dcDao.getNextAvailableMacAddressPair(zoneId, (1L << 31));
 			String guestMacAddress = macAddresses[0];
@@ -584,7 +584,7 @@ public class Db20to21MigrationUtil {
 		SearchCriteria sc2 = sb2.create();
 		sc2.setParameters("zoneId", zoneId);
 		
-		List secStorageVms =_secStorageVmDao.searchAll(sc2, null, false, false);
+		List secStorageVms =_secStorageVmDao.searchIncludingRemoved(sc2, null, false, false);
 		for(SecondaryStorageVmVO secStorageVm : secStorageVms) {
 			String[] macAddresses = _dcDao.getNextAvailableMacAddressPair(zoneId, (1L << 31));
 			String guestMacAddress = macAddresses[0];
@@ -611,7 +611,7 @@ public class Db20to21MigrationUtil {
 		
 		SearchCriteria sc3 = sb3.create();
 		sc3.setParameters("zoneId", zoneId);
-		List domRs = _routerDao.searchAll(sc3, null, false, false);
+		List domRs = _routerDao.searchIncludingRemoved(sc3, null, false, false);
 		for(DomainRouterVO router :  domRs) {
 			if(router.getState() == State.Running || router.getState() == State.Starting) {
 				router.setState(State.Stopping);
@@ -632,7 +632,7 @@ public class Db20to21MigrationUtil {
 		SearchCriteria sc = sb.create();
 		sc.setParameters("zoneId", zoneId);
 		sc.setParameters("podId", podId);
-		List vmInstances = _vmInstanceDao.searchAll(sc, null, false, false);
+		List vmInstances = _vmInstanceDao.searchIncludingRemoved(sc, null, false, false);
 		List podHosts = getHostsInPod(zoneId, podId);
 		for(VMInstanceVO vm : vmInstances) {
 			if(vm.getHostId() != null) {
@@ -659,13 +659,13 @@ public class Db20to21MigrationUtil {
 		sc.setParameters("podId", podId);
 		sc.setParameters("type", Host.Type.Routing.toString());
 		
-		return _hostDao.searchAll(sc, null, false, false);
+		return _hostDao.searchIncludingRemoved(sc, null, false, false);
 	}
 	
 	private void migrateVolumDeviceIds() {
 		System.out.println("Migrating device_id for volumes, this may take a while, please wait...");
 		SearchCriteria sc = _vmInstanceDao.createSearchCriteria();
-		List vmInstances = _vmInstanceDao.searchAll(sc, null, false, false);
+		List vmInstances = _vmInstanceDao.searchIncludingRemoved(sc, null, false, false);
 		
 		long deviceId = 1;
 		for(VMInstanceVO vm: vmInstances) {
@@ -676,7 +676,7 @@ public class Db20to21MigrationUtil {
 			SearchCriteria sc2 = sb.create();
 			sc2.setParameters("instanceId", vm.getId());
 			
-			List volumes = _volumeDao.searchAll(sc2, null, false, false);
+			List volumes = _volumeDao.searchIncludingRemoved(sc2, null, false, false);
 			deviceId = 1;	// reset for each VM iteration
 			for(VolumeVO vol : volumes) {
 				if(vol.getVolumeType() == VolumeType.ROOT) {
@@ -706,7 +706,7 @@ public class Db20to21MigrationUtil {
 		System.out.println("Migrating pool type for volumes...");
 		
 		SearchCriteria sc = _volumeDao.createSearchCriteria();
-		List volumes = _volumeDao.searchAll(sc, null, false, false);
+		List volumes = _volumeDao.searchIncludingRemoved(sc, null, false, false);
 		for(VolumeVO vol : volumes) {
 			if(vol.getPoolId() != null) {
 				StoragePoolVO pool = _poolDao.findById(vol.getPoolId());
diff --git a/server/src/com/cloud/migration/Db21to22MigrationUtil.java b/server/src/com/cloud/migration/Db21to22MigrationUtil.java
index 6aa06d2c926..f9e828f45cf 100644
--- a/server/src/com/cloud/migration/Db21to22MigrationUtil.java
+++ b/server/src/com/cloud/migration/Db21to22MigrationUtil.java
@@ -1,6 +1,8 @@
 package com.cloud.migration;
 
 import java.io.File;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
 import java.util.List;
 
 import org.apache.log4j.xml.DOMConfigurator;
@@ -16,16 +18,25 @@ import com.cloud.utils.PropertiesUtil;
 import com.cloud.utils.component.ComponentLocator;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
+import com.cloud.utils.db.Transaction;
+import com.cloud.vm.InstanceGroupVMMapVO;
+import com.cloud.vm.InstanceGroupVO;
+import com.cloud.vm.dao.InstanceGroupDao;
+import com.cloud.vm.dao.InstanceGroupVMMapDao;
 
 public class Db21to22MigrationUtil {
     private AccountDao _accountDao;
     private DomainDao _domainDao;
     private ResourceCountDao _resourceCountDao;
+    private InstanceGroupDao _vmGroupDao;
+    private InstanceGroupVMMapDao _groupVMMapDao;
 
     private void doMigration() {
         setupComponents();
 
         migrateResourceCounts();
+        
+        setupInstanceGroups();
 
         System.out.println("Migration done");
     }
@@ -56,13 +67,56 @@ public class Db21to22MigrationUtil {
     }
 
     private void setupComponents() {
-        ComponentLocator.getLocator("migration", "migration-components.xml", "log4j-cloud.xml");
-        ComponentLocator locator = ComponentLocator.getCurrentLocator();
+    	ComponentLocator locator = ComponentLocator.getLocator("migration", "migration-components.xml", "log4j-cloud.xml");
 
         _accountDao = locator.getDao(AccountDao.class);
         _domainDao = locator.getDao(DomainDao.class);
         _resourceCountDao = locator.getDao(ResourceCountDao.class);
+        _vmGroupDao = locator.getDao(InstanceGroupDao.class);
+        _groupVMMapDao = locator.getDao(InstanceGroupVMMapDao.class);
     }
+    
+    private void setupInstanceGroups() {
+    	System.out.println("setting up vm instance groups");
+    	
+    	//Search for all the vms that have not null groups
+    	Long vmId = 0L;
+    	Long accountId = 0L;
+    	String groupName;
+    	Transaction txn = Transaction.open(Transaction.CLOUD_DB);
+    	txn.start();
+		try {
+	    	String request = "SELECT vm.id, uservm.account_id, vm.group from vm_instance vm, user_vm uservm where vm.group is not null and vm.removed is null and vm.id=uservm.id order by id";
+	    	System.out.println(request);
+	    	PreparedStatement statement = txn.prepareStatement(request);
+	    	ResultSet result = statement.executeQuery();
+	    	while (result.next()) {
+	    		vmId = result.getLong(1);
+	    		accountId = result.getLong(2);
+	    		groupName = result.getString(3);
+		        InstanceGroupVO group = _vmGroupDao.findByAccountAndName(accountId, groupName);
+		    	//Create vm group if the group doesn't exist for this account
+		        if (group == null) {
+					group = new InstanceGroupVO(groupName, accountId);
+					group =  _vmGroupDao.persist(group);
+					System.out.println("Created new isntance group with name " + groupName + " for account id=" + accountId);
+		        }
+				
+				if (group != null) {
+					InstanceGroupVMMapVO groupVmMapVO = new InstanceGroupVMMapVO(group.getId(), vmId);
+					_groupVMMapDao.persist(groupVmMapVO);
+					System.out.println("Assigned vm id=" + vmId + " to group with name " + groupName + " for account id=" + accountId);
+				}
+	    	}
+			txn.commit();
+			statement.close();
+		} catch (Exception e) {
+			System.out.println("Unhandled exception: " + e);
+		} finally {
+			txn.close();
+		}
+    }
+
 
     public static void main(String[] args) {
         File file = PropertiesUtil.findConfigFile("log4j-cloud.xml");
diff --git a/server/src/com/cloud/migration/ServiceOffering20VO.java b/server/src/com/cloud/migration/ServiceOffering20VO.java
index bff56722ed3..acb355376f1 100644
--- a/server/src/com/cloud/migration/ServiceOffering20VO.java
+++ b/server/src/com/cloud/migration/ServiceOffering20VO.java
@@ -11,7 +11,8 @@ import javax.persistence.GenerationType;
 import javax.persistence.Id;
 import javax.persistence.Table;
 
-import com.cloud.offering.ServiceOffering.GuestIpType;
+import com.cloud.offering.NetworkOffering;
+import com.cloud.offering.NetworkOffering.GuestIpType;
 import com.cloud.utils.db.GenericDao;
 
 @Entity
@@ -51,7 +52,7 @@ public class ServiceOffering20VO {
     
     @Column(name="guest_ip_type")
     @Enumerated(EnumType.STRING)
-    private GuestIpType guestIpType = GuestIpType.Virtualized;
+    private NetworkOffering.GuestIpType guestIpType = NetworkOffering.GuestIpType.Virtualized;
     
     @Column(name="use_local_storage")
     private boolean useLocalStorage;
@@ -66,10 +67,10 @@ public class ServiceOffering20VO {
     }
 
     public ServiceOffering20VO(Long id, String name, int cpu, int ramSize, int speed, int rateMbps, int multicastRateMbps, boolean offerHA, String displayText, boolean localStorageRequired) {
-        this(id, name, cpu, ramSize, speed, rateMbps, multicastRateMbps, offerHA, displayText, GuestIpType.Virtualized, localStorageRequired);
+        this(id, name, cpu, ramSize, speed, rateMbps, multicastRateMbps, offerHA, displayText, NetworkOffering.GuestIpType.Virtualized, localStorageRequired);
     }
     
-    public ServiceOffering20VO(Long id, String name, int cpu, int ramSize, int speed, int rateMbps, int multicastRateMbps, boolean offerHA, String displayText, GuestIpType guestIpType, boolean useLocalStorage) {
+    public ServiceOffering20VO(Long id, String name, int cpu, int ramSize, int speed, int rateMbps, int multicastRateMbps, boolean offerHA, String displayText, NetworkOffering.GuestIpType guestIpType, boolean useLocalStorage) {
         this.id = id;
         this.name = name;
         this.cpu = cpu;
@@ -167,11 +168,11 @@ public class ServiceOffering20VO {
 		return multicastRateMbps;
 	}
 
-	public void setGuestIpType(GuestIpType guestIpType) {
+	public void setGuestIpType(NetworkOffering.GuestIpType guestIpType) {
 		this.guestIpType = guestIpType;
 	}
 
-	public GuestIpType getGuestIpType() {
+	public NetworkOffering.GuestIpType getGuestIpType() {
 		return guestIpType;
 	}
 	
diff --git a/server/src/com/cloud/migration/ServiceOffering21VO.java b/server/src/com/cloud/migration/ServiceOffering21VO.java
index 324e6143135..31ba534831c 100644
--- a/server/src/com/cloud/migration/ServiceOffering21VO.java
+++ b/server/src/com/cloud/migration/ServiceOffering21VO.java
@@ -9,6 +9,7 @@ import javax.persistence.PrimaryKeyJoinColumn;
 import javax.persistence.Table;
 import javax.persistence.Transient;
 
+import com.cloud.offering.NetworkOffering;
 import com.cloud.offering.ServiceOffering;
 
 @Entity
@@ -36,13 +37,13 @@ public class ServiceOffering21VO extends DiskOffering21VO implements ServiceOffe
     
     @Column(name="guest_ip_type")
     @Enumerated(EnumType.STRING)
-    private GuestIpType guestIpType;
+    private NetworkOffering.GuestIpType guestIpType;
     
     protected ServiceOffering21VO() {
         super();
     }
 
-    public ServiceOffering21VO(String name, int cpu, int ramSize, int speed, int rateMbps, int multicastRateMbps, boolean offerHA, String displayText, GuestIpType guestIpType, boolean useLocalStorage, boolean recreatable, String tags) {
+    public ServiceOffering21VO(String name, int cpu, int ramSize, int speed, int rateMbps, int multicastRateMbps, boolean offerHA, String displayText, NetworkOffering.GuestIpType guestIpType, boolean useLocalStorage, boolean recreatable, String tags) {
         super(name, displayText, false, tags, recreatable, useLocalStorage);
         this.cpu = cpu;
         this.ramSize = ramSize;
@@ -116,11 +117,11 @@ public class ServiceOffering21VO extends DiskOffering21VO implements ServiceOffe
 		return multicastRateMbps;
 	}
 
-	public void setGuestIpType(GuestIpType guestIpType) {
+	public void setGuestIpType(NetworkOffering.GuestIpType guestIpType) {
 		this.guestIpType = guestIpType;
 	}
 
-	public GuestIpType getGuestIpType() {
+	public NetworkOffering.GuestIpType getGuestIpType() {
 		return guestIpType;
 	}
 }
diff --git a/server/src/com/cloud/network/NetworkAccountDaoImpl.java b/server/src/com/cloud/network/NetworkAccountDaoImpl.java
new file mode 100644
index 00000000000..4898d2ed091
--- /dev/null
+++ b/server/src/com/cloud/network/NetworkAccountDaoImpl.java
@@ -0,0 +1,13 @@
+/**
+ * 
+ */
+package com.cloud.network;
+
+import com.cloud.utils.db.GenericDao;
+import com.cloud.utils.db.GenericDaoBase;
+
+public class NetworkAccountDaoImpl extends GenericDaoBase implements GenericDao {
+    public NetworkAccountDaoImpl() {
+        super();
+    }
+}
diff --git a/server/src/com/cloud/network/NetworkAccountVO.java b/server/src/com/cloud/network/NetworkAccountVO.java
new file mode 100644
index 00000000000..665b03c9691
--- /dev/null
+++ b/server/src/com/cloud/network/NetworkAccountVO.java
@@ -0,0 +1,45 @@
+/**
+ * 
+ */
+package com.cloud.network;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+
+import com.cloud.user.OwnedBy;
+
+@Entity
+@Table(name="account_network_ref")
+public class NetworkAccountVO implements OwnedBy {
+    @Id
+    @GeneratedValue(strategy=GenerationType.IDENTITY)
+    long id;
+    
+    @Column(name="account_id")
+    long accountId;
+    
+    @Column(name="network_configuration_id")
+    long networkConfigurationId;
+
+    protected NetworkAccountVO() {
+    }
+    
+    public NetworkAccountVO(long networkConfigurationId, long accountId) {
+        this.networkConfigurationId = networkConfigurationId;
+        this.accountId = accountId;
+    }
+    
+    @Override
+    public long getAccountId() {
+        return accountId;
+    }
+    
+    public long getNetworkConfigurationId() {
+        return networkConfigurationId;
+    }
+
+}
diff --git a/server/src/com/cloud/network/NetworkConfigurationVO.java b/server/src/com/cloud/network/NetworkConfigurationVO.java
new file mode 100644
index 00000000000..dd3ee6cc466
--- /dev/null
+++ b/server/src/com/cloud/network/NetworkConfigurationVO.java
@@ -0,0 +1,208 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.  
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.network;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.EnumType;
+import javax.persistence.Enumerated;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+
+import com.cloud.network.Network.BroadcastDomainType;
+import com.cloud.network.Network.Mode;
+import com.cloud.network.Network.TrafficType;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.net.NetUtils;
+
+/**
+ * NetworkProfileVO contains information about a specific network.
+ *
+ */
+@Entity
+@Table(name="network_configurations")
+public class NetworkConfigurationVO implements NetworkConfiguration {
+    @Id
+    @GeneratedValue(strategy=GenerationType.IDENTITY)
+    Long id;
+    
+    @Column(name="mode")
+    @Enumerated(value=EnumType.STRING)
+    Mode mode;
+    
+    @Column(name="broadcast_domain_type")
+    @Enumerated(value=EnumType.STRING)
+    BroadcastDomainType broadcastDomainType;
+    
+    @Column(name="traffic_type")
+    @Enumerated(value=EnumType.STRING)
+    TrafficType trafficType;
+    
+    @Column(name="broadcast_uri")
+    String broadcastUri; 
+    
+    @Column(name="gateway")
+    String gateway;
+    
+    @Column(name="cidr")
+    String cidr;
+    
+    @Column(name="network_offering_id")
+    long networkOfferingId;
+    
+    @Column(name="data_center_id")
+    long dataCenterId;
+    
+    @Column(name="guru_name")
+    String guruName;
+    
+    @Column(name="state")
+    @Enumerated(value=EnumType.STRING)
+    State state;
+    
+    public NetworkConfigurationVO() {
+    }
+    
+    public NetworkConfigurationVO(NetworkConfiguration that, long offeringId, long dataCenterId, String guruName) {
+        this(that.getTrafficType(), that.getMode(), that.getBroadcastDomainType(), offeringId, dataCenterId);
+        this.guruName = guruName;
+        this.state = that.getState();
+    }
+    
+    public NetworkConfigurationVO(TrafficType trafficType, Mode mode, BroadcastDomainType broadcastDomainType, long networkOfferingId, long dataCenterId) {
+        this.trafficType = trafficType;
+        this.mode = mode;
+        this.broadcastDomainType = broadcastDomainType;
+        this.networkOfferingId = networkOfferingId;
+        this.dataCenterId = dataCenterId;
+        this.state = State.Allocated;
+    }
+    
+    @Override
+    public State getState() {
+        return state;
+    }
+    
+    public void setState(State state) {
+        this.state = state;
+    }
+
+    @Override
+    public Long getId() {
+        return id;
+    }
+
+    @Override
+    public Mode getMode() {
+        return mode;
+    }
+    
+    @Override
+    public long getNetworkOfferingId() {
+        return networkOfferingId;
+    }
+
+    public void setMode(Mode mode) {
+        this.mode = mode;
+    }
+
+    @Override
+    public BroadcastDomainType getBroadcastDomainType() {
+        return broadcastDomainType;
+    }
+    
+    public String getGuruName() {
+        return guruName;
+    }
+    
+    public void setGuruName(String guruName) {
+        this.guruName = guruName;
+    }
+
+    public void setBroadcastDomainType(BroadcastDomainType broadcastDomainType) {
+        this.broadcastDomainType = broadcastDomainType;
+    }
+
+    @Override
+    public TrafficType getTrafficType() {
+        return trafficType;
+    }
+
+    public void setTrafficType(TrafficType trafficType) {
+        this.trafficType = trafficType;
+    }
+
+    @Override
+    public String getGateway() {
+        return gateway;
+    }
+
+    public void setGateway(String gateway) {
+        this.gateway = gateway;
+    }
+
+    @Override
+    public String getCidr() {
+        return cidr;
+    }
+
+    public void setCidr(String cidr) {
+        this.cidr = cidr;
+    }
+    
+    public String getBroadcastUri() {
+        return broadcastUri;
+    }
+
+    public void setBroadcastUri(String broadcastUri) {
+        this.broadcastUri = broadcastUri;
+    }
+    
+    @Override
+    public int hashCode() {
+        return NumbersUtil.hash(id);
+    }
+    
+    @Override
+    public long getDataCenterId() {
+        return dataCenterId;
+    }
+    
+    @Override
+    public boolean equals(Object obj) {
+        if (!(obj instanceof NetworkConfigurationVO)) {
+            return false;
+        }
+        NetworkConfigurationVO that = (NetworkConfigurationVO)obj;
+        if (this.trafficType != that.trafficType) {
+            return false;
+        }
+        
+        if ((this.cidr == null && that.cidr != null) || (this.cidr != null && that.cidr == null)) {
+            return false;
+        }
+        
+        if (this.cidr == null && that.cidr == null) {
+            return true;
+        }
+        
+        return NetUtils.isNetworkAWithinNetworkB(this.cidr, that.cidr);
+    }
+}
diff --git a/server/src/com/cloud/network/NetworkManager.java b/server/src/com/cloud/network/NetworkManager.java
index d7f8b839244..6333d7214df 100644
--- a/server/src/com/cloud/network/NetworkManager.java
+++ b/server/src/com/cloud/network/NetworkManager.java
@@ -20,6 +20,7 @@ package com.cloud.network;
 import java.util.List;
 import java.util.Map;
 
+import com.cloud.agent.api.to.NicTO;
 import com.cloud.api.commands.AssignToLoadBalancerRuleCmd;
 import com.cloud.api.commands.AssociateIPAddrCmd;
 import com.cloud.api.commands.CreateIPForwardingRuleCmd;
@@ -37,20 +38,29 @@ import com.cloud.api.commands.UpdateLoadBalancerRuleCmd;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
 import com.cloud.dc.VlanVO;
+import com.cloud.deploy.DeployDestination;
+import com.cloud.deploy.DeploymentPlan;
 import com.cloud.exception.ConcurrentOperationException;
 import com.cloud.exception.InsufficientAddressCapacityException;
 import com.cloud.exception.InsufficientCapacityException;
+import com.cloud.exception.InsufficientVirtualNetworkCapcityException;
 import com.cloud.exception.InternalErrorException;
 import com.cloud.exception.InvalidParameterValueException;
 import com.cloud.exception.NetworkRuleConflictException;
 import com.cloud.exception.PermissionDeniedException;
 import com.cloud.exception.ResourceAllocationException;
+import com.cloud.offerings.NetworkOfferingVO;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.user.AccountVO;
+import com.cloud.utils.Pair;
 import com.cloud.utils.component.Manager;
 import com.cloud.vm.DomainRouter;
 import com.cloud.vm.DomainRouterVO;
+import com.cloud.vm.NicProfile;
+import com.cloud.vm.NicVO;
 import com.cloud.vm.UserVmVO;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachineProfile;
 
 /**
  * NetworkManager manages the network for the different end users.
@@ -227,9 +237,10 @@ public interface NetworkManager extends Manager {
      * @param router router object to send the association to
      * @param ipAddrList list of public IP addresses
      * @param add true if associate, false if disassociate
+     * @param vmId
      * @return
      */
-    boolean associateIP(DomainRouterVO router, List ipAddrList, boolean add) throws ResourceAllocationException;
+    boolean associateIP(DomainRouterVO router, List ipAddrList, boolean add, long vmId) throws ResourceAllocationException;
     
     /**
      * Associates a public IP address for a router.
@@ -296,4 +307,17 @@ public interface NetworkManager extends Manager {
     
     public boolean deleteIpForwardingRule(DeleteIPForwardingRuleCmd cmd) throws PermissionDeniedException, InvalidParameterValueException;
 
+    NetworkConfigurationVO setupNetworkConfiguration(AccountVO owner, NetworkOfferingVO offering, DeploymentPlan plan);
+    NetworkConfigurationVO setupNetworkConfiguration(AccountVO owner, NetworkOfferingVO offering, NetworkConfiguration predefined, DeploymentPlan plan);
+    List setupNetworkConfigurations(AccountVO owner, List offerings, DeploymentPlan plan);
+    
+    List getSystemAccountNetworkOfferings(String... offeringNames);
+    
+    List allocate(VirtualMachineProfile vm, List> networks) throws InsufficientCapacityException;
+
+    NicTO[] prepare(VirtualMachineProfile profile, DeployDestination dest) throws InsufficientAddressCapacityException, InsufficientVirtualNetworkCapcityException;
+    
+     void create(K vm);
+    
+     List getNics(K vm);
 }
diff --git a/server/src/com/cloud/network/NetworkManagerImpl.java b/server/src/com/cloud/network/NetworkManagerImpl.java
old mode 100755
new mode 100644
index fdd8549f592..bbb952ad2ef
--- a/server/src/com/cloud/network/NetworkManagerImpl.java
+++ b/server/src/com/cloud/network/NetworkManagerImpl.java
@@ -54,6 +54,7 @@ import com.cloud.agent.api.routing.LoadBalancerCfgCommand;
 import com.cloud.agent.api.routing.SavePasswordCommand;
 import com.cloud.agent.api.routing.SetFirewallRuleCommand;
 import com.cloud.agent.api.routing.VmDataCommand;
+import com.cloud.agent.api.to.NicTO;
 import com.cloud.agent.manager.AgentManager;
 import com.cloud.alert.AlertManager;
 import com.cloud.api.BaseCmd;
@@ -91,6 +92,8 @@ import com.cloud.dc.dao.AccountVlanMapDao;
 import com.cloud.dc.dao.DataCenterDao;
 import com.cloud.dc.dao.HostPodDao;
 import com.cloud.dc.dao.VlanDao;
+import com.cloud.deploy.DeployDestination;
+import com.cloud.deploy.DeploymentPlan;
 import com.cloud.domain.DomainVO;
 import com.cloud.domain.dao.DomainDao;
 import com.cloud.event.EventState;
@@ -102,6 +105,7 @@ import com.cloud.exception.AgentUnavailableException;
 import com.cloud.exception.ConcurrentOperationException;
 import com.cloud.exception.InsufficientAddressCapacityException;
 import com.cloud.exception.InsufficientCapacityException;
+import com.cloud.exception.InsufficientVirtualNetworkCapcityException;
 import com.cloud.exception.InternalErrorException;
 import com.cloud.exception.InvalidParameterValueException;
 import com.cloud.exception.NetworkRuleConflictException;
@@ -114,14 +118,23 @@ import com.cloud.host.Host;
 import com.cloud.host.HostVO;
 import com.cloud.host.dao.HostDao;
 import com.cloud.hypervisor.Hypervisor;
+import com.cloud.network.Network.TrafficType;
+import com.cloud.network.configuration.NetworkGuru;
 import com.cloud.network.dao.FirewallRulesDao;
 import com.cloud.network.dao.IPAddressDao;
 import com.cloud.network.dao.LoadBalancerDao;
 import com.cloud.network.dao.LoadBalancerVMMapDao;
+import com.cloud.network.dao.NetworkConfigurationDao;
 import com.cloud.network.dao.NetworkRuleConfigDao;
 import com.cloud.network.dao.SecurityGroupDao;
 import com.cloud.network.dao.SecurityGroupVMMapDao;
-import com.cloud.offering.ServiceOffering.GuestIpType;
+import com.cloud.network.element.NetworkElement;
+import com.cloud.offering.NetworkOffering;
+import com.cloud.offering.NetworkOffering.GuestIpType;
+import com.cloud.offerings.NetworkOfferingVO;
+import com.cloud.offerings.dao.NetworkOfferingDao;
+import com.cloud.resource.Resource;
+import com.cloud.resource.Resource.ReservationStrategy;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.service.dao.ServiceOfferingDao;
 import com.cloud.storage.StorageManager;
@@ -146,10 +159,12 @@ import com.cloud.uservm.UserVm;
 import com.cloud.utils.NumbersUtil;
 import com.cloud.utils.Pair;
 import com.cloud.utils.StringUtils;
+import com.cloud.utils.component.Adapters;
 import com.cloud.utils.component.ComponentLocator;
 import com.cloud.utils.component.Inject;
 import com.cloud.utils.concurrency.NamedThreadFactory;
 import com.cloud.utils.db.DB;
+import com.cloud.utils.db.JoinBuilder;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
 import com.cloud.utils.db.Transaction;
@@ -159,13 +174,18 @@ import com.cloud.utils.net.NetUtils;
 import com.cloud.vm.DomainRouter;
 import com.cloud.vm.DomainRouter.Role;
 import com.cloud.vm.DomainRouterVO;
+import com.cloud.vm.NicProfile;
+import com.cloud.vm.NicVO;
 import com.cloud.vm.State;
 import com.cloud.vm.UserVmVO;
+import com.cloud.vm.VMInstanceVO;
 import com.cloud.vm.VirtualMachine;
 import com.cloud.vm.VirtualMachine.Event;
 import com.cloud.vm.VirtualMachineManager;
 import com.cloud.vm.VirtualMachineName;
+import com.cloud.vm.VirtualMachineProfile;
 import com.cloud.vm.dao.DomainRouterDao;
+import com.cloud.vm.dao.NicDao;
 import com.cloud.vm.dao.UserVmDao;
 
 /**
@@ -209,11 +229,20 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
     @Inject StoragePoolDao _storagePoolDao = null;
     @Inject SecurityGroupDao _securityGroupDao = null;
     @Inject ServiceOfferingDao _serviceOfferingDao = null;
-    @Inject UserStatisticsDao _statsDao;
     @Inject UserVmDao _userVmDao;
     @Inject FirewallRulesDao _firewallRulesDao;
     @Inject NetworkRuleConfigDao _networkRuleConfigDao;
     @Inject AccountVlanMapDao _accountVlanMapDao;
+    @Inject UserStatisticsDao _statsDao = null;
+    @Inject NetworkOfferingDao _networkOfferingDao = null;
+    @Inject NetworkConfigurationDao _networkProfileDao = null;
+    @Inject NicDao _nicDao;
+    
+    @Inject(adapter=NetworkGuru.class)
+    Adapters _networkGurus;
+    @Inject(adapter=NetworkElement.class)
+    Adapters _networkElements;
+
     long _routerTemplateId = -1;
     int _routerRamSize;
     // String _privateNetmask;
@@ -223,6 +252,10 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
     int _routerCleanupInterval = 3600;
     int _routerStatsInterval = 300;
     private ServiceOfferingVO _offering;
+    private int _networkRate;
+    private int _multicastRate;
+    private HashMap _systemNetworks = new HashMap(5);
+    
     private VMTemplateVO _template;
     
     ScheduledExecutorService _executor;
@@ -232,6 +265,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
 		return destroyRouter(router.getId());
 	}
 
+    @Override
     public boolean sendSshKeysToHost(Long hostId, String pubKey, String prvKey) {
     	ModifySshKeysCommand cmd = new ModifySshKeysCommand(pubKey, prvKey);
     	final Answer answer = _agentMgr.easySend(hostId, cmd);
@@ -244,7 +278,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
     
     @Override @DB
     public String assignSourceNatIpAddress(AccountVO account, final DataCenterVO dc, final String domain, final ServiceOfferingVO serviceOffering, long startEventId) throws ResourceAllocationException {
-    	if (serviceOffering.getGuestIpType() == GuestIpType.DirectDual || serviceOffering.getGuestIpType() == GuestIpType.DirectSingle) {
+    	if (serviceOffering.getGuestIpType() == NetworkOffering.GuestIpType.DirectDual || serviceOffering.getGuestIpType() == NetworkOffering.GuestIpType.DirectSingle) {
     		return null;
     	}
         final long dcId = dc.getId();
@@ -379,6 +413,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
         }
     }
 
+    @Override
     @DB
     public DomainRouterVO createDhcpServerForDirectlyAttachedGuests(long userId, long accountId, DataCenterVO dc, HostPodVO pod, Long candidateHost, VlanVO guestVlan) throws ConcurrentOperationException{
  
@@ -418,6 +453,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
 
             router =
                 new DomainRouterVO(id,
+                        _offering.getId(),
                         name,
                         mgmtMacAddress,
                         null,
@@ -461,7 +497,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
             List vols = _storageMgr.create(account, router, rtrTemplate, dc, pod, _offering, null,0);
             if (vols == null){
             	_ipAddressDao.unassignIpAddress(guestIp);
-            	_routerDao.delete(router.getId());
+            	_routerDao.expunge(router.getId());
             	if (s_logger.isDebugEnabled()) {
             		s_logger.debug("Unable to create dhcp server in storage host or pool in pod " + pod.getName() + " (id:" + pod.getId() + ")");
             	}
@@ -495,7 +531,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
             txn.rollback();
 
             if (router.getState() == State.Creating) {
-                _routerDao.delete(router.getId());
+                _routerDao.expunge(router.getId());
             }
             return null;
         } finally {
@@ -603,6 +639,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
                 }
 
                 router = new DomainRouterVO(id,
+                            _offering.getId(),
                             name,
                             privateMacAddress,
                             null,
@@ -613,7 +650,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
                             guestIpAddress,
                             guestNetmask,
                             accountId,
-                            account.getDomainId().longValue(),
+                            account.getDomainId(),
                             publicMacAddress,
                             publicIpAddress,
                             vlanNetmask,
@@ -637,7 +674,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
                     break;
                 }
 
-                _routerDao.delete(router.getId());
+                _routerDao.expunge(router.getId());
                 if (s_logger.isDebugEnabled()) {
                     s_logger.debug("Unable to find storage host or pool in pod " + pod.first().getName() + " (id:" + pod.first().getId() + "), checking other pods");
                 }
@@ -645,6 +682,9 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
             }
             
             if (!found) {
+                event.setDescription("failed to create Domain Router : " + name);
+                event.setLevel(EventVO.LEVEL_ERROR);
+                _eventDao.persist(event);
                 throw new ExecutionException("Unable to create DomainRouter");
             }
             _routerDao.updateIf(router, Event.OperationSucceeded, null);
@@ -669,7 +709,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
             txn.rollback();
 
             if (router != null && router.getState() == State.Creating) {
-                _routerDao.delete(router.getId());
+                _routerDao.expunge(router.getId());
             }
             return null;
         } finally {
@@ -837,6 +877,8 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
         	s_logger.debug("Lock on router " + routerId + " is acquired");
         
         boolean started = false;
+        String vnet = null;
+        boolean vnetAllocated = false;
         try {
 	        final State state = router.getState();
 	        if (state == State.Running) {
@@ -890,8 +932,6 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
 	            throw new ConcurrentOperationException("Someone else is starting the router: " + router.toString());
 	        }
 	
-	        String vnet = null;
-	        boolean vnetAllocated = false;
 	        final boolean mirroredVols = router.isMirroredVols();
 	        try {
 	            event = new EventVO();
@@ -905,6 +945,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
 	                for (final UserVmVO vm : vms) {
 	                    if (vm.getVnet() != null) {
 	                        vnet = vm.getVnet();
+	                        break;
 	                    }
 	                }
 	            }
@@ -923,7 +964,9 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
 	                    vnet = _dcDao.allocateVnet(router.getDataCenterId(), router.getAccountId());
 	                }
 	                vnetAllocated = true;
-	                routerMacAddress = getRouterMacForVnet(dc, vnet);
+	                if(vnet != null){
+	                    routerMacAddress = getRouterMacForVnet(dc, vnet);
+	                }
 	            } else if (router.getRole() == Role.DHCP_USERDATA) {
 	            	if (!Vlan.UNTAGGED.equals(router.getVlanId())) {
 	            		vnet = router.getVlanId().trim();
@@ -984,7 +1027,8 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
 
 	                try {
 	                    String[] storageIps = new String[2];
-	                    final StartRouterCommand cmdStartRouter = new StartRouterCommand(router, name, storageIps, vols, mirroredVols);
+	                    final StartRouterCommand cmdStartRouter = new StartRouterCommand(router, _networkRate,
+	                            _multicastRate, name, storageIps, vols, mirroredVols);
 	                    answer = _agentMgr.send(routingHost.getId(), cmdStartRouter);
 	                    if (answer != null && answer.getResult()) {
 	                        if (answer instanceof StartRouterAnswer){
@@ -1043,28 +1087,6 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
 	            return _routerDao.findById(routerId);
 	        } catch (final Throwable th) {
 	        	
-	        	Transaction txn = Transaction.currentTxn();
-        		if (!started) {
-		        	txn.start();
-		            if (vnetAllocated == true && vnet != null) {
-		                _dcDao.releaseVnet(vnet, router.getDataCenterId(), router.getAccountId());
-		            }
-		
-		            router.setVnet(null);
-		            String privateIpAddress = router.getPrivateIpAddress();
-		            
-		            router.setPrivateIpAddress(null);
-		            
-		            if (privateIpAddress != null) {
-		            	_dcDao.releasePrivateIpAddress(privateIpAddress, router.getDataCenterId(), router.getId());
-		            }
-		            
-		
-		            if (_routerDao.updateIf(router, Event.OperationFailed, null)) {
-			            txn.commit();
-		            }
-        		}
-		
 	            if (th instanceof ExecutionException) {
 	                s_logger.error("Error while starting router due to " + th.getMessage());
 	            } else if (th instanceof ConcurrentOperationException) {
@@ -1077,13 +1099,28 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
 	            return null;
 	        }
         } finally {
-        	if (router != null) {
-        		
-                if(s_logger.isDebugEnabled())
-                	s_logger.debug("Releasing lock on router " + routerId);
-        		_routerDao.release(routerId);
-        	}
-        	if (!started){
+            
+            if (!started){
+                Transaction txn = Transaction.currentTxn();
+                txn.start();
+                if (vnetAllocated == true && vnet != null) {
+                    _dcDao.releaseVnet(vnet, router.getDataCenterId(), router.getAccountId());
+                }
+
+                router.setVnet(null);
+                String privateIpAddress = router.getPrivateIpAddress();
+
+                router.setPrivateIpAddress(null);
+
+                if (privateIpAddress != null) {
+                    _dcDao.releasePrivateIpAddress(privateIpAddress, router.getDataCenterId(), router.getId());
+                }
+
+
+                if (_routerDao.updateIf(router, Event.OperationFailed, null)) {
+                    txn.commit();
+                }
+                
                 EventVO event = new EventVO();
                 event.setUserId(1L);
                 event.setAccountId(router.getAccountId());
@@ -1092,7 +1129,14 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
                 event.setLevel(EventVO.LEVEL_ERROR);
                 event.setStartId(startEventId);
                 _eventDao.persist(event);
+            }
+            
+        	if (router != null) {
+                if(s_logger.isDebugEnabled())
+                	s_logger.debug("Releasing lock on router " + routerId);
+        		_routerDao.release(routerId);
         	}
+
         }
     }
 
@@ -1123,7 +1167,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
 				ipAddrList.add(ipVO.getAddress());
 			}
 			if (!ipAddrList.isEmpty()) {
-				final boolean success = associateIP(router, ipAddrList, true);
+				final boolean success = associateIP(router, ipAddrList, true, 0);
 				if (!success) {
 					return false;
 				}
@@ -1398,7 +1442,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
     }
 
     @Override
-    public boolean associateIP(final DomainRouterVO router, final List ipAddrList, final boolean add) {
+    public boolean associateIP(final DomainRouterVO router, final List ipAddrList, final boolean add, long vmId) {
         final Command [] cmds = new Command[ipAddrList.size()];
         int i=0;
         boolean sourceNat = false;
@@ -1419,7 +1463,12 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
             	vifMacAddress = macAddresses[1];
 			}
 
-            cmds[i++] = new IPAssocCommand(router.getInstanceName(), router.getPrivateIpAddress(), ipAddress, add, firstIP, sourceNat, vlanId, vlanGateway, vlanNetmask, vifMacAddress);
+			String vmGuestAddress = null;
+			if(vmId!=0){
+				vmGuestAddress = _vmDao.findById(vmId).getGuestIpAddress();
+			}
+			
+            cmds[i++] = new IPAssocCommand(router.getInstanceName(), router.getPrivateIpAddress(), ipAddress, add, firstIP, sourceNat, vlanId, vlanGateway, vlanNetmask, vifMacAddress, vmGuestAddress);
             
             sourceNat = false;
         }
@@ -1543,7 +1592,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
             ipAddrs.add(ipAddress);
 
             if (router.getState() == State.Running) {
-                success = associateIP(router, ipAddrs, true);
+                success = associateIP(router, ipAddrs, true, 0L);
                 if (!success) {
                     errorMsg = "Unable to assign public IP address.";
                 }
@@ -1720,6 +1769,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
         }
     }
 
+    @Override
     public List  updatePortForwardingRules(final List fwRules, final DomainRouterVO router, Long hostId ){
         final List fwdRules = new ArrayList();
         final List result = new ArrayList();
@@ -1796,7 +1846,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
                 if (!_domainDao.isChildDomain(account.getDomainId(), userVM.getDomainId())) {
                     throw new PermissionDeniedException("Unable to create port forwarding rule, IP address " + ipAddress + " to virtual machine " + cmd.getVirtualMachineId() + ", permission denied.");
                 }
-            } else if (account.getId().longValue() != userVM.getAccountId()) {
+            } else if (account.getId() != userVM.getAccountId()) {
                 throw new PermissionDeniedException("Unable to create port forwarding rule, IP address " + ipAddress + " to virtual machine " + cmd.getVirtualMachineId() + ", permission denied.");
             }
         }
@@ -1903,7 +1953,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
             }
         } else {
             if (account != null) {
-                if ((ipAddressVO.getAccountId() == null) || (account.getId().longValue() != ipAddressVO.getAccountId().longValue())) {
+                if ((ipAddressVO.getAccountId() == null) || (account.getId() != ipAddressVO.getAccountId().longValue())) {
                     throw new PermissionDeniedException("Unable to list port forwarding rules for address " + ipAddress + ", permission denied for account " + account.getId());
                 }
             }
@@ -1943,7 +1993,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
         		if (!_domainDao.isChildDomain(account.getDomainId(), loadBalancer.getDomainId())) {
             		throw new PermissionDeniedException("Failed to assign to load balancer " + loadBalancerId + ", permission denied.");
         		}
-        	} else if (account.getId().longValue() != loadBalancer.getAccountId()) {
+        	} else if (account.getId() != loadBalancer.getAccountId()) {
         		throw new PermissionDeniedException("Failed to assign to load balancer " + loadBalancerId + ", permission denied.");
         	}
         }
@@ -2173,7 +2223,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
                 if (!_domainDao.isChildDomain(account.getDomainId(), ipAddr.getDomainId())) {
                     throw new PermissionDeniedException("Unable to create load balancer rule on IP address " + publicIp + ", permission denied.");
                 }
-            } else if (account.getId().longValue() != ipAddr.getAccountId().longValue()) {
+            } else if (account.getId() != ipAddr.getAccountId().longValue()) {
                 throw new PermissionDeniedException("Unable to create load balancer rule, account " + account.getAccountName() + " doesn't own ip address " + publicIp);
             }
         }
@@ -2367,7 +2417,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
                     s_logger.debug("Disassociate ip " + router.getName());
                 }
 
-                if (associateIP(router, ipAddrs, false)) {
+                if (associateIP(router, ipAddrs, false, 0)) {
                     _ipAddressDao.unassignIpAddress(ipAddress);
                 } else {
                 	if (s_logger.isDebugEnabled()) {
@@ -2420,6 +2470,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
         return _routerDao.listByHostId(hostId);
     }
 
+    @Override
     public boolean updateLoadBalancerRules(final List fwRules, final DomainRouterVO router, Long hostId) {
     	
     	for (FirewallRuleVO rule : fwRules) {
@@ -2450,10 +2501,9 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
         _executor = Executors.newScheduledThreadPool(1, new NamedThreadFactory("RouterMonitor"));
 
         final ComponentLocator locator = ComponentLocator.getCurrentLocator();
+        
         final Map configs = _configDao.getConfiguration("AgentManager", params);
 
-        _routerTemplateId = NumbersUtil.parseInt(configs.get("router.template.id"), 1);
-
         _routerRamSize = NumbersUtil.parseInt(configs.get("router.ram.size"), 128);
 
 //        String value = configs.get("guest.ip.network");
@@ -2492,19 +2542,41 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
         _haMgr.registerHandler(VirtualMachine.Type.DomainRouter, this);
 
         boolean useLocalStorage = Boolean.parseBoolean((String)params.get(Config.SystemVMUseLocalStorage.key()));
-        _offering = new ServiceOfferingVO("Fake Offering For DomR", 1, _routerRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized, useLocalStorage, true, null);
+        String networkRateStr = _configDao.getValue("network.throttling.rate");
+        String multicastRateStr = _configDao.getValue("multicast.throttling.rate");
+        _networkRate = ((networkRateStr == null) ? 200 : Integer.parseInt(networkRateStr));
+        _multicastRate = ((multicastRateStr == null) ? 10 : Integer.parseInt(multicastRateStr));
+        _offering = new ServiceOfferingVO("Fake Offering For DomR", 1, _routerRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, useLocalStorage, true, null);
         _offering.setUniqueName("Cloud.Com-SoftwareRouter");
         _offering = _serviceOfferingDao.persistSystemServiceOffering(_offering);
-        _template = _templateDao.findById(_routerTemplateId);
+        _template = _templateDao.findRoutingTemplate();
         if (_template == null) {
-            throw new ConfigurationException("Unable to find the template for the router: " + _routerTemplateId);
+        	s_logger.error("Unable to find system vm template.");
+        } else {
+        	_routerTemplateId = _template.getId();
         }
         
+        NetworkOfferingVO publicNetworkOffering = new NetworkOfferingVO(NetworkOfferingVO.SystemVmPublicNetwork, TrafficType.Public, null);
+        publicNetworkOffering = _networkOfferingDao.persistSystemNetworkOffering(publicNetworkOffering);
+        _systemNetworks.put(NetworkOfferingVO.SystemVmPublicNetwork, publicNetworkOffering);
+        NetworkOfferingVO managementNetworkOffering = new NetworkOfferingVO(NetworkOfferingVO.SystemVmManagementNetwork, TrafficType.Management, null);
+        managementNetworkOffering = _networkOfferingDao.persistSystemNetworkOffering(managementNetworkOffering);
+        _systemNetworks.put(NetworkOfferingVO.SystemVmManagementNetwork, managementNetworkOffering);
+        NetworkOfferingVO controlNetworkOffering = new NetworkOfferingVO(NetworkOfferingVO.SystemVmControlNetwork, TrafficType.Control, null);
+        controlNetworkOffering = _networkOfferingDao.persistSystemNetworkOffering(controlNetworkOffering);
+        _systemNetworks.put(NetworkOfferingVO.SystemVmControlNetwork, controlNetworkOffering);
+        NetworkOfferingVO guestNetworkOffering = new NetworkOfferingVO(NetworkOfferingVO.SystemVmGuestNetwork, TrafficType.Guest, GuestIpType.Virtualized);
+        guestNetworkOffering = _networkOfferingDao.persistSystemNetworkOffering(guestNetworkOffering);
+        _systemNetworks.put(NetworkOfferingVO.SystemVmGuestNetwork, guestNetworkOffering);
+        NetworkOfferingVO storageNetworkOffering = new NetworkOfferingVO(NetworkOfferingVO.SystemVmStorageNetwork, TrafficType.Storage, null);
+        storageNetworkOffering = _networkOfferingDao.persistSystemNetworkOffering(storageNetworkOffering);
+        _systemNetworks.put(NetworkOfferingVO.SystemVmGuestNetwork, storageNetworkOffering);
+        
         s_logger.info("Network Manager is configured.");
 
         return true;
     }
-
+    
     @Override
     public String getName() {
         return _name;
@@ -2819,6 +2891,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
         public RouterCleanupTask() {
         }
 
+        @Override
         public void run() {
             try {
                 final List ids = _routerDao.findLonelyRouters();
@@ -2966,7 +3039,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
 		
 		SearchBuilder virtualNetworkVlanSB = _vlanDao.createSearchBuilder();
     	virtualNetworkVlanSB.and("vlanType", virtualNetworkVlanSB.entity().getVlanType(), SearchCriteria.Op.EQ);
-    	ipAddressSB.join("virtualNetworkVlanSB", virtualNetworkVlanSB, ipAddressSB.entity().getVlanDbId(), virtualNetworkVlanSB.entity().getId());
+    	ipAddressSB.join("virtualNetworkVlanSB", virtualNetworkVlanSB, ipAddressSB.entity().getVlanDbId(), virtualNetworkVlanSB.entity().getId(), JoinBuilder.JoinType.INNER);
     	
     	SearchCriteria ipAddressSC = ipAddressSB.create();
     	ipAddressSC.setParameters("accountId", accountId);
@@ -2979,11 +3052,195 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
 		return _ipAddressDao.search(ipAddressSC, null);
     }
     
+    @Override
+    public NetworkConfigurationVO setupNetworkConfiguration(AccountVO owner, NetworkOfferingVO offering, DeploymentPlan plan) {
+        return setupNetworkConfiguration(owner, offering, null, plan);
+    }
+    
+    @Override
+    public NetworkConfigurationVO setupNetworkConfiguration(AccountVO owner, NetworkOfferingVO offering, NetworkConfiguration predefined, DeploymentPlan plan) {
+        List configs = _networkProfileDao.listBy(owner.getId(), offering.getId(), plan.getDataCenterId());
+        if (configs.size() > 0) {
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("Found existing network configuration for offering " + offering + ": " + configs.get(0));
+            }
+            return configs.get(0);
+        }
+        
+        for (NetworkGuru guru : _networkGurus) {
+            NetworkConfiguration config = guru.design(offering, plan, predefined, owner);
+            if (config == null) {
+                continue;
+            }
+            
+            if (config.getId() != null) {
+                if (config instanceof NetworkConfigurationVO) {
+                    return (NetworkConfigurationVO)config;
+                } else {
+                    return _networkProfileDao.findById(config.getId());
+                }
+            } 
+            
+            NetworkConfigurationVO vo = new NetworkConfigurationVO(config, offering.getId(), plan.getDataCenterId(), guru.getName());
+            return _networkProfileDao.persist(vo, owner.getId());
+        }
+
+        throw new CloudRuntimeException("Unable to convert network offering to network profile: " + offering.getId());
+    }
+
+    @Override
+    public List setupNetworkConfigurations(AccountVO owner, List offerings, DeploymentPlan plan) {
+        List profiles = new ArrayList(offerings.size());
+        for (NetworkOfferingVO offering : offerings) {
+            profiles.add(setupNetworkConfiguration(owner, offering, plan));
+        }
+        return profiles;
+    }
+    
+    @Override
+    public List getSystemAccountNetworkOfferings(String... offeringNames) {
+        List offerings = new ArrayList(offeringNames.length);
+        for (String offeringName : offeringNames) {
+            NetworkOfferingVO network = _systemNetworks.get(offeringName);
+            if (network == null) {
+                throw new CloudRuntimeException("Unable to find system network profile for " + offeringName);
+            }
+            offerings.add(network);
+        }
+        return offerings;
+    }
+    
+    public NetworkConfigurationVO createNetworkConfiguration(NetworkOfferingVO offering, DeploymentPlan plan, AccountVO owner) {
+        return null;
+    }
+
+
+    @Override @DB
+    public List allocate(VirtualMachineProfile vm, List> networks) throws InsufficientCapacityException {
+        List nicProfiles = new ArrayList(networks.size());
+        
+        Transaction txn = Transaction.currentTxn();
+        txn.start();
+        
+        int deviceId = 0;
+        
+        for (Pair network : networks) {
+            NetworkGuru concierge = _networkGurus.get(network.first().getGuruName());
+            NicProfile profile = concierge.allocate(network.first(), network.second(), vm);
+            if (profile == null) {
+                continue;
+            }
+            NicVO vo = new NicVO(concierge.getName(), vm.getId(), network.first().getId());
+            vo.setDeviceId(deviceId++);
+            vo.setMode(network.first().getMode());
+            if (profile.getIp4Address() != null) {
+                vo.setIp4Address(profile.getIp4Address());
+                vo.setState(NicVO.State.Reserved);
+            }
+            
+            if (profile.getMacAddress() != null) {
+                vo.setMacAddress(profile.getMacAddress());
+            }
+            
+            if (profile.getMode() != null) {
+                vo.setMode(profile.getMode());
+            }
+    
+            vo = _nicDao.persist(vo);
+            nicProfiles.add(new NicProfile(vo, network.first()));
+        }
+        txn.commit();
+        
+        return nicProfiles;
+    }
+    
+    protected NicTO toNicTO(NicVO nic, NetworkConfigurationVO config) {
+        NicTO to = new NicTO();
+        to.setDeviceId(nic.getDeviceId());
+        to.setBroadcastType(config.getBroadcastDomainType());
+        to.setType(config.getTrafficType());
+        to.setIp(nic.getIp4Address());
+        to.setNetmask(nic.getNetmask());
+        to.setMac(nic.getMacAddress());
+        
+        return to;
+    }
+    
+    @Override
+    public NicTO[] prepare(VirtualMachineProfile vmProfile, DeployDestination dest) throws InsufficientAddressCapacityException, InsufficientVirtualNetworkCapcityException {
+        List nics = _nicDao.listBy(vmProfile.getId());
+        NicTO[] nicTos = new NicTO[nics.size()];
+        int i = 0;
+        for (NicVO nic : nics) {
+            NetworkConfigurationVO config = _networkProfileDao.findById(nic.getNetworkConfigurationId());
+            if (nic.getReservationStrategy() == ReservationStrategy.Start) {
+                NetworkGuru concierge = _networkGurus.get(config.getGuruName());
+                nic.setState(Resource.State.Reserving);
+                _nicDao.update(nic.getId(), nic);
+                NicProfile profile = toNicProfile(nic);
+                String reservationId = concierge.reserve(profile, config, vmProfile, dest);
+                nic.setIp4Address(profile.getIp4Address());
+                nic.setIp6Address(profile.getIp6Address());
+                nic.setMacAddress(profile.getMacAddress());
+                nic.setIsolationUri(profile.getIsolationUri());
+                nic.setBroadcastUri(profile.getBroadCastUri());
+                nic.setReservationId(reservationId);
+                nic.setReserver(concierge.getName());
+                nic.setState(Resource.State.Reserved);
+                nic.setNetmask(profile.getNetmask());
+                nic.setGateway(profile.getGateway());
+                nic.setAddressFormat(profile.getFormat());
+                _nicDao.update(nic.getId(), nic);
+                for (NetworkElement element : _networkElements) {
+                    if (!element.prepare(config, profile, vmProfile, null)) {
+                        s_logger.warn("Unable to prepare " + nic + " for element " + element.getName());
+                        return null;
+                    }
+                }
+            }
+            
+            nicTos[i++] = toNicTO(nic, config);
+            
+        }
+        return nicTos;
+    }
+    
+    NicProfile toNicProfile(NicVO nic) {
+        NetworkConfiguration config = _networkProfileDao.findById(nic.getNetworkConfigurationId());
+        NicProfile profile = new NicProfile(nic, config);
+        return profile;
+    }
+    
+    public void release(long vmId) {
+        List nics = _nicDao.listBy(vmId);
+        
+        for (NicVO nic : nics) {
+            nic.setState(Resource.State.Releasing);
+            _nicDao.update(nic.getId(), nic);
+            NetworkGuru concierge = _networkGurus.get(nic.getReserver());
+            if (!concierge.release(nic.getReservationId())) {
+                s_logger.warn("Unable to release " + nic + " using " + concierge.getName());
+            }
+            nic.setState(Resource.State.Allocated);
+            _nicDao.update(nic.getId(), nic);
+        }
+    }
+    
+    @Override
+    public  void create(K vm) {
+    }
+    
+    @Override
+    public  List getNics(K vm) {
+        return _nicDao.listBy(vm.getId());
+    }
+    
     protected class NetworkUsageTask implements Runnable {
 
         public NetworkUsageTask() {
         }
 
+        @Override
         public void run() {
             final List routers = _routerDao.listUpByHostId(null);
             s_logger.debug("Found " + routers.size() + " running routers. ");
@@ -3056,7 +3313,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
         if (loadBalancer == null) {
             throw new ServerApiException(BaseCmd.PARAM_ERROR, "Unable to find load balancer rule with id " + loadBalancerId);
         } else if (account != null) {
-            if (!isAdmin(account.getType()) && (loadBalancer.getAccountId() != account.getId().longValue())) {
+            if (!isAdmin(account.getType()) && (loadBalancer.getAccountId() != account.getId())) {
                 throw new ServerApiException(BaseCmd.PARAM_ERROR, "Account " + account.getAccountName() + " does not own load balancer rule " + loadBalancer.getName() +
                         " (id:" + loadBalancer.getId() + ")");
             } else if (!_domainDao.isChildDomain(account.getDomainId(), loadBalancer.getDomainId())) {
@@ -3154,7 +3411,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
     	
     	if (account != null) {
             if (!isAdmin(account.getType())) {
-                if (loadBalancer.getAccountId() != account.getId().longValue()) {
+                if (loadBalancer.getAccountId() != account.getId()) {
                     throw new PermissionDeniedException("Account " + account.getAccountName() + " does not own load balancer rule " + loadBalancer.getName() + " (id:" + loadBalancerId + "), permission denied");
                 }
             } else if (!_domainDao.isChildDomain(account.getDomainId(), loadBalancer.getDomainId())) {
@@ -3284,7 +3541,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
         Long accountId = lbOwner.getId();
         if (account != null) {
             if (!isAdmin(account.getType())) {
-                if (account.getId().longValue() != accountId.longValue()) {
+                if (account.getId() != accountId.longValue()) {
                     throw new PermissionDeniedException("Unable to update load balancer rule, permission denied");
                 }
             } else if (!_domainDao.isChildDomain(account.getDomainId(), lbOwner.getDomainId())) {
@@ -3363,7 +3620,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
             SecurityGroupVO sg = _securityGroupDao.findById(netRule.getSecurityGroupId());
             if (account != null) {
                 if (!BaseCmd.isAdmin(account.getType())) {
-                    if ((sg.getAccountId() == null) || (sg.getAccountId().longValue() != account.getId().longValue())) {
+                    if (sg.getAccountId() != account.getId()) {
                         throw new PermissionDeniedException("Unable to delete port forwarding service rule " + netRuleId + "; account: " + account.getAccountName() + " is not the owner");
                     }
                 } else if (!_domainDao.isChildDomain(account.getDomainId(), sg.getDomainId())) {
@@ -3447,7 +3704,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
         Long accountId = accountByIp.getId();
         if (account != null) {
             if (!isAdmin(account.getType())) {
-                if (account.getId().longValue() != accountId.longValue()) {
+                if (account.getId() != accountId.longValue()) {
                     throw new ServerApiException(BaseCmd.PARAM_ERROR, "account " + account.getAccountName() + " doesn't own ip address " + ipAddress);
                 }
             } else if (!_domainDao.isChildDomain(account.getDomainId(), accountByIp.getDomainId())) {
@@ -3566,7 +3823,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
                 if (!_domainDao.isChildDomain(account.getDomainId(), ruleOwner.getDomainId())) {
                     throw new PermissionDeniedException("Unable to delete port forwarding rule " + ruleId + ", permission denied.");
                 }
-            } else if (account.getId().longValue() != ruleOwner.getId().longValue()) {
+            } else if (account.getId() != ruleOwner.getId()) {
                 throw new PermissionDeniedException("Unable to delete port forwarding rule " + ruleId + ", permission denied.");
             }
         }
@@ -3591,7 +3848,7 @@ public class NetworkManagerImpl implements NetworkManager, VirtualMachineManager
             } else if (fwdings.size() == 1) {
                 fwRule = fwdings.get(0);
                 if (fwRule.getPrivateIpAddress().equalsIgnoreCase(privateIp) && fwRule.getPrivatePort().equals(privatePort)) {
-                    _firewallRulesDao.delete(fwRule.getId());
+                    _firewallRulesDao.expunge(fwRule.getId());
                 } else {
                     throw new InvalidParameterValueException("No such rule");
                 }
diff --git a/server/src/com/cloud/network/NetworkProfileVO.java b/server/src/com/cloud/network/NetworkProfileVO.java
deleted file mode 100644
index 9956014b8e2..00000000000
--- a/server/src/com/cloud/network/NetworkProfileVO.java
+++ /dev/null
@@ -1,123 +0,0 @@
-/**
- *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
- * 
- * This software is licensed under the GNU General Public License v3 or later.  
- * 
- * It is free software: you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation, either version 3 of the License, or any later version.
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- * 
- * You should have received a copy of the GNU General Public License
- * along with this program.  If not, see .
- * 
- */
-package com.cloud.network;
-
-import javax.persistence.Column;
-import javax.persistence.Entity;
-import javax.persistence.EnumType;
-import javax.persistence.Enumerated;
-import javax.persistence.GeneratedValue;
-import javax.persistence.GenerationType;
-import javax.persistence.Id;
-import javax.persistence.Table;
-
-import com.cloud.network.Network.BroadcastDomainType;
-import com.cloud.network.Network.Mode;
-import com.cloud.network.Network.TrafficType;
-
-/**
- * NetworkProfileVO contains information about a specific network.
- *
- */
-@Entity
-@Table(name="network_profile")
-public class NetworkProfileVO {
-    @Id
-    @GeneratedValue(strategy=GenerationType.IDENTITY)
-    long id;
-    
-    @Column(name="mode")
-    @Enumerated(value=EnumType.STRING)
-    Mode mode;
-    
-    @Column(name="account_id")
-    long accountId;
-    
-    @Column(name="broadcast_domain_type")
-    @Enumerated(value=EnumType.STRING)
-    BroadcastDomainType broadcastDomainType;
-    
-    @Column(name="traffic_type")
-    @Enumerated(value=EnumType.STRING)
-    TrafficType trafficType;
-    
-    @Column(name="gateway")
-    String gateway;
-    
-    @Column(name="cidr")
-    String cidr;
-    
-    public NetworkProfileVO() {
-    }
-
-    public long getId() {
-        return id;
-    }
-
-    public void setId(long id) {
-        this.id = id;
-    }
-
-    public Mode getMode() {
-        return mode;
-    }
-
-    public void setMode(Mode mode) {
-        this.mode = mode;
-    }
-
-    public long getAccountId() {
-        return accountId;
-    }
-
-    public void setAccountId(long accountId) {
-        this.accountId = accountId;
-    }
-
-    public BroadcastDomainType getBroadcastDomainType() {
-        return broadcastDomainType;
-    }
-
-    public void setBroadcastDomainType(BroadcastDomainType broadcastDomainType) {
-        this.broadcastDomainType = broadcastDomainType;
-    }
-
-    public TrafficType getTrafficType() {
-        return trafficType;
-    }
-
-    public void setTrafficType(TrafficType trafficType) {
-        this.trafficType = trafficType;
-    }
-
-    public String getGateway() {
-        return gateway;
-    }
-
-    public void setGateway(String gateway) {
-        this.gateway = gateway;
-    }
-
-    public String getCidr() {
-        return cidr;
-    }
-
-    public void setCidr(String cidr) {
-        this.cidr = cidr;
-    }
-}
diff --git a/server/src/com/cloud/network/configuration/ControlNetworkGuru.java b/server/src/com/cloud/network/configuration/ControlNetworkGuru.java
new file mode 100644
index 00000000000..59bf7d3a83f
--- /dev/null
+++ b/server/src/com/cloud/network/configuration/ControlNetworkGuru.java
@@ -0,0 +1,124 @@
+/**
+ * 
+ */
+package com.cloud.network.configuration;
+
+import java.util.Map;
+
+import javax.ejb.Local;
+import javax.naming.ConfigurationException;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.configuration.Config;
+import com.cloud.configuration.dao.ConfigurationDao;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.deploy.DeployDestination;
+import com.cloud.deploy.DeploymentPlan;
+import com.cloud.exception.InsufficientAddressCapacityException;
+import com.cloud.exception.InsufficientVirtualNetworkCapcityException;
+import com.cloud.network.Network.AddressFormat;
+import com.cloud.network.Network.BroadcastDomainType;
+import com.cloud.network.Network.Mode;
+import com.cloud.network.Network.TrafficType;
+import com.cloud.network.NetworkConfiguration;
+import com.cloud.network.NetworkConfigurationVO;
+import com.cloud.offering.NetworkOffering;
+import com.cloud.resource.Resource.ReservationStrategy;
+import com.cloud.user.Account;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.component.ComponentLocator;
+import com.cloud.utils.component.Inject;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.vm.NicProfile;
+import com.cloud.vm.VirtualMachineProfile;
+
+@Local(value={NetworkGuru.class})
+public class ControlNetworkGuru extends AdapterBase implements NetworkGuru {
+    private static final Logger s_logger = Logger.getLogger(ControlNetworkGuru.class);
+    @Inject DataCenterDao _dcDao;
+    String _cidr;
+    String _gateway;
+
+    @Override
+    public NetworkConfiguration design(NetworkOffering offering, DeploymentPlan plan, NetworkConfiguration specifiedConfig, Account owner) {
+        if (offering.getTrafficType() != TrafficType.Control) {
+            return null;
+        }
+        
+        NetworkConfigurationVO config = new NetworkConfigurationVO(offering.getTrafficType(), Mode.Static, BroadcastDomainType.LinkLocal, offering.getId(), plan.getDataCenterId());
+        config.setCidr(_cidr);
+        config.setGateway(_gateway);
+        
+        return config;
+    }
+    
+    protected ControlNetworkGuru() {
+        super();
+    }
+    
+    @Override
+    public boolean configure(String name, Map params) throws ConfigurationException {
+        super.configure(name, params);
+        
+        ComponentLocator locator = ComponentLocator.getCurrentLocator();
+        
+        ConfigurationDao configDao = locator.getDao(ConfigurationDao.class);
+        Map dbParams = configDao.getConfiguration(params);
+        
+        _cidr = dbParams.get(Config.ControlCidr);
+        if (_cidr == null) {
+            _cidr = "169.254.0.0/16";
+        }
+        
+        _gateway = dbParams.get(Config.ControlGateway);
+        if (_gateway == null) {
+            _gateway = "169.254.0.1";
+        }
+        
+        s_logger.info("Control network setup: cidr=" + _cidr + "; gateway = " + _gateway);
+        
+        return true;
+    }
+
+    @Override
+    public NicProfile allocate(NetworkConfiguration config, NicProfile nic, VirtualMachineProfile vm) throws InsufficientVirtualNetworkCapcityException,
+            InsufficientAddressCapacityException {
+        if (config.getTrafficType() != TrafficType.Control) {
+            return null;
+        }
+        
+        if (nic != null) {
+            throw new CloudRuntimeException("Does not support nic specification at this time: " + nic);
+        }
+        
+        return new NicProfile(ReservationStrategy.Start, null, null, null, null);
+    }
+
+    @Override
+    public String reserve(NicProfile nic, NetworkConfiguration config, VirtualMachineProfile vm, DeployDestination dest) throws InsufficientVirtualNetworkCapcityException,
+            InsufficientAddressCapacityException {
+        String ip = _dcDao.allocateLinkLocalPrivateIpAddress(dest.getDataCenter().getId(), dest.getPod().getId(), vm.getId());
+        nic.setIp4Address(ip);
+        nic.setMacAddress("FE:FF:FF:FF:FF:FF");
+        nic.setNetmask("255.255.0.0");
+        nic.setFormat(AddressFormat.Ip4);
+        
+        return Long.toString(nic.getId());
+    }
+
+    @Override
+    public boolean release(String uniqueId) {
+        _dcDao.releaseLinkLocalPrivateIpAddress(Long.parseLong(uniqueId));
+        return true;
+    }
+
+    @Override
+    public NetworkConfiguration implement(NetworkConfiguration config, NetworkOffering offering, DeployDestination destination) {
+        return config;
+    }
+    
+    @Override
+    public void destroy(NetworkConfiguration config, NetworkOffering offering) {
+    }
+}
diff --git a/server/src/com/cloud/network/configuration/GuestNetworkGuru.java b/server/src/com/cloud/network/configuration/GuestNetworkGuru.java
new file mode 100644
index 00000000000..e15f3d8cc68
--- /dev/null
+++ b/server/src/com/cloud/network/configuration/GuestNetworkGuru.java
@@ -0,0 +1,94 @@
+/**
+ * 
+ */
+package com.cloud.network.configuration;
+
+import javax.ejb.Local;
+
+import com.cloud.dc.DataCenterVO;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.dc.dao.VlanDao;
+import com.cloud.deploy.DeployDestination;
+import com.cloud.deploy.DeploymentPlan;
+import com.cloud.exception.InsufficientAddressCapacityException;
+import com.cloud.exception.InsufficientVirtualNetworkCapcityException;
+import com.cloud.network.Network.BroadcastDomainType;
+import com.cloud.network.Network.Mode;
+import com.cloud.network.Network.TrafficType;
+import com.cloud.network.NetworkConfiguration;
+import com.cloud.network.NetworkConfigurationVO;
+import com.cloud.network.dao.NetworkConfigurationDao;
+import com.cloud.offering.NetworkOffering;
+import com.cloud.offering.NetworkOffering.GuestIpType;
+import com.cloud.user.Account;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.component.Inject;
+import com.cloud.vm.NicProfile;
+import com.cloud.vm.VirtualMachineProfile;
+
+@Local(value=NetworkGuru.class)
+public class GuestNetworkGuru extends AdapterBase implements NetworkGuru {
+    @Inject protected NetworkConfigurationDao _profileDao;
+    @Inject protected DataCenterDao _dcDao;
+    @Inject protected VlanDao _vlanDao; 
+    
+    protected GuestNetworkGuru() {
+        super();
+    } 
+    
+    @Override
+    public NetworkConfiguration design(NetworkOffering offering, DeploymentPlan plan, NetworkConfiguration userSpecified, Account owner) {
+        if (offering.getTrafficType() != TrafficType.Guest) {
+            return null;
+        }
+        
+        GuestIpType ipType = offering.getGuestIpType();
+        BroadcastDomainType broadcastType = null;
+        Mode mode = null;
+        if (ipType == GuestIpType.Virtualized) {
+            mode = Mode.Dhcp;
+            broadcastType = BroadcastDomainType.Vlan;
+        } else {
+            broadcastType = BroadcastDomainType.Native;
+            mode = Mode.Dhcp;
+        }
+        
+        NetworkConfigurationVO profile = new NetworkConfigurationVO(offering.getTrafficType(), mode, broadcastType, offering.getId(), plan.getDataCenterId());
+        DataCenterVO dc = _dcDao.findById(plan.getDataCenterId());
+        
+        return profile;
+    }
+
+    @Override
+    public NetworkConfiguration implement(NetworkConfiguration config, NetworkOffering offering, DeployDestination destination) {
+        // TODO Auto-generated method stub
+        return null;
+    }
+
+    @Override
+    public NicProfile allocate(NetworkConfiguration config, NicProfile nic, VirtualMachineProfile vm)
+            throws InsufficientVirtualNetworkCapcityException, InsufficientAddressCapacityException {
+        // TODO Auto-generated method stub
+        return null;
+    }
+
+    @Override
+    public String reserve(NicProfile nic, NetworkConfiguration config, VirtualMachineProfile vm, DeployDestination dest) throws InsufficientVirtualNetworkCapcityException,
+            InsufficientAddressCapacityException {
+        // TODO Auto-generated method stub
+        return null;
+    }
+
+    @Override
+    public boolean release(String uniqueId) {
+        // TODO Auto-generated method stub
+        return false;
+    }
+
+    @Override
+    public void destroy(NetworkConfiguration config, NetworkOffering offering) {
+        // TODO Auto-generated method stub
+        
+    }
+
+}
diff --git a/server/src/com/cloud/network/configuration/GuruUtils.java b/server/src/com/cloud/network/configuration/GuruUtils.java
new file mode 100644
index 00000000000..1b6c9c04bff
--- /dev/null
+++ b/server/src/com/cloud/network/configuration/GuruUtils.java
@@ -0,0 +1,9 @@
+/**
+ * 
+ */
+package com.cloud.network.configuration;
+
+public final class GuruUtils {
+    
+
+}
diff --git a/server/src/com/cloud/network/configuration/PodBasedNetworkGuru.java b/server/src/com/cloud/network/configuration/PodBasedNetworkGuru.java
new file mode 100644
index 00000000000..cf72ccd8aa7
--- /dev/null
+++ b/server/src/com/cloud/network/configuration/PodBasedNetworkGuru.java
@@ -0,0 +1,104 @@
+/**
+ * 
+ */
+package com.cloud.network.configuration;
+
+import javax.ejb.Local;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.dc.DataCenter;
+import com.cloud.dc.Pod;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.deploy.DeployDestination;
+import com.cloud.deploy.DeploymentPlan;
+import com.cloud.exception.InsufficientAddressCapacityException;
+import com.cloud.exception.InsufficientVirtualNetworkCapcityException;
+import com.cloud.network.Network.AddressFormat;
+import com.cloud.network.Network.BroadcastDomainType;
+import com.cloud.network.Network.Mode;
+import com.cloud.network.Network.TrafficType;
+import com.cloud.network.NetworkConfiguration;
+import com.cloud.network.NetworkConfigurationVO;
+import com.cloud.offering.NetworkOffering;
+import com.cloud.resource.Resource.ReservationStrategy;
+import com.cloud.user.Account;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.component.Inject;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.net.NetUtils;
+import com.cloud.vm.NicProfile;
+import com.cloud.vm.VirtualMachineProfile;
+
+@Local(value={NetworkGuru.class})
+public class PodBasedNetworkGuru extends AdapterBase implements NetworkGuru {
+    private static final Logger s_logger = Logger.getLogger(PodBasedNetworkGuru.class);
+    @Inject DataCenterDao _dcDao;
+
+    @Override
+    public NetworkConfiguration design(NetworkOffering offering, DeploymentPlan plan, NetworkConfiguration userSpecified, Account owner) {
+        TrafficType type = offering.getTrafficType();
+        
+        if (type != TrafficType.Management && type != TrafficType.Storage) {
+            return null;
+        }
+        
+        NetworkConfigurationVO config = new NetworkConfigurationVO(type, Mode.Static, BroadcastDomainType.Native, offering.getId(), plan.getDataCenterId());
+        
+        return config;
+    }
+    
+    protected PodBasedNetworkGuru() {
+        super();
+    }
+
+    @Override
+    public NicProfile allocate(NetworkConfiguration config, NicProfile nic, VirtualMachineProfile vm) throws InsufficientVirtualNetworkCapcityException,
+            InsufficientAddressCapacityException {
+        TrafficType trafficType = config.getTrafficType();
+        if (trafficType != TrafficType.Storage && trafficType != TrafficType.Management) {
+            return null;
+        }
+        
+        if (nic != null) {
+            throw new CloudRuntimeException("Does not support nic configuration");
+        }
+        
+        NicProfile profile = new NicProfile(ReservationStrategy.Start, null, null, null, null);
+        return profile;
+    }
+
+    @Override
+    public String reserve(NicProfile nic, NetworkConfiguration config, VirtualMachineProfile vm, DeployDestination dest) throws InsufficientVirtualNetworkCapcityException,
+            InsufficientAddressCapacityException {
+        DataCenter dc = dest.getDataCenter();
+        Pod pod = dest.getPod();
+        
+        String ip = _dcDao.allocatePrivateIpAddress(dest.getDataCenter().getId(), dest.getPod().getId(), nic.getId());
+        String[] macs = _dcDao.getNextAvailableMacAddressPair(dc.getId());
+        
+        nic.setIp4Address(ip);
+        nic.setGateway(pod.getGateway());
+        nic.setMacAddress(macs[0]);
+        nic.setFormat(AddressFormat.Ip4);
+        String netmask = NetUtils.getCidrSubNet(pod.getCidrAddress(), pod.getCidrSize());
+        nic.setNetmask(netmask);
+        
+        return Long.toString(nic.getId());
+    }
+
+    @Override
+    public boolean release(String uniqueId) {
+        _dcDao.releasePrivateIpAddress(Long.parseLong(uniqueId));
+        return true;
+    }
+
+    @Override
+    public NetworkConfiguration implement(NetworkConfiguration config, NetworkOffering offering, DeployDestination destination) {
+        return config;
+    }
+    
+    @Override
+    public void destroy(NetworkConfiguration config, NetworkOffering offering) {
+    }
+}
diff --git a/server/src/com/cloud/network/configuration/PublicNetworkGuru.java b/server/src/com/cloud/network/configuration/PublicNetworkGuru.java
new file mode 100644
index 00000000000..9e28c4c62fc
--- /dev/null
+++ b/server/src/com/cloud/network/configuration/PublicNetworkGuru.java
@@ -0,0 +1,110 @@
+/**
+ * 
+ */
+package com.cloud.network.configuration;
+
+import javax.ejb.Local;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.dc.Vlan.VlanType;
+import com.cloud.dc.VlanVO;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.dc.dao.VlanDao;
+import com.cloud.deploy.DeployDestination;
+import com.cloud.deploy.DeploymentPlan;
+import com.cloud.exception.InsufficientAddressCapacityException;
+import com.cloud.exception.InsufficientVirtualNetworkCapcityException;
+import com.cloud.network.Network.AddressFormat;
+import com.cloud.network.Network.BroadcastDomainType;
+import com.cloud.network.Network.IsolationType;
+import com.cloud.network.Network.Mode;
+import com.cloud.network.Network.TrafficType;
+import com.cloud.network.NetworkConfiguration;
+import com.cloud.network.NetworkConfigurationVO;
+import com.cloud.offering.NetworkOffering;
+import com.cloud.resource.Resource.ReservationStrategy;
+import com.cloud.user.Account;
+import com.cloud.utils.Pair;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.component.Inject;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.vm.NicProfile;
+import com.cloud.vm.VirtualMachineProfile;
+
+@Local(value={NetworkGuru.class})
+public class PublicNetworkGuru extends AdapterBase implements NetworkGuru {
+    private static final Logger s_logger = Logger.getLogger(PublicNetworkGuru.class);
+    
+    @Inject DataCenterDao _dcDao;
+    @Inject VlanDao _vlanDao;
+
+    @Override
+    public NetworkConfiguration design(NetworkOffering offering, DeploymentPlan plan, NetworkConfiguration config, Account owner) {
+        if (offering.getTrafficType() != TrafficType.Public) {
+            return null;
+        }
+        
+        return new NetworkConfigurationVO(offering.getTrafficType(), Mode.Static, BroadcastDomainType.Vlan, offering.getId(), plan.getDataCenterId());
+    }
+    
+    protected PublicNetworkGuru() {
+        super();
+    }
+
+    @Override
+    public NicProfile allocate(NetworkConfiguration config, NicProfile nic, VirtualMachineProfile vm) throws InsufficientVirtualNetworkCapcityException,
+            InsufficientAddressCapacityException {
+        if (config.getTrafficType() != TrafficType.Public) {
+            return null;
+        }
+        
+        if (nic != null) {
+            throw new CloudRuntimeException("Unsupported nic settings");
+        }
+
+        return new NicProfile(ReservationStrategy.Create, null, null, null, null);
+    }
+
+    @Override
+    public String reserve(NicProfile ch, NetworkConfiguration configuration, VirtualMachineProfile vm, DeployDestination dest) throws InsufficientVirtualNetworkCapcityException, InsufficientAddressCapacityException {
+        if (ch.getReservationId() != null) {
+            return ch.getReservationId();
+        }
+            
+        long dcId = dest.getDataCenter().getId();
+        
+        String[] macs = _dcDao.getNextAvailableMacAddressPair(dcId);
+
+        Pair ipAndVlan = _vlanDao.assignIpAddress(dcId, vm.getVm().getAccountId(), vm.getVm().getDomainId(), VlanType.VirtualNetwork, true);
+        if (ipAndVlan == null) {
+            throw new InsufficientVirtualNetworkCapcityException("Unable to get public ip address in " + dcId);
+        }
+        VlanVO vlan = ipAndVlan.second();
+        ch.setIp4Address(ipAndVlan.first());
+        ch.setGateway(vlan.getVlanGateway());
+        ch.setNetmask(vlan.getVlanNetmask());
+        ch.setIsolationUri(IsolationType.Vlan.toUri(vlan.getVlanId()));
+        ch.setBroadcastType(BroadcastDomainType.Vlan);
+        ch.setBroadcastUri(BroadcastDomainType.Vlan.toUri(vlan.getVlanId()));
+        ch.setMacAddress(macs[1]);
+        ch.setFormat(AddressFormat.Ip4);
+        ch.setReservationId(Long.toString(vlan.getId()));
+        
+        return Long.toString(vlan.getId());
+    }
+
+    @Override
+    public boolean release(String uniqueId) {
+        return _vlanDao.release(Long.parseLong(uniqueId));
+    }
+
+    @Override
+    public NetworkConfiguration implement(NetworkConfiguration config, NetworkOffering offering, DeployDestination destination) {
+        return config;
+    }
+    
+    @Override
+    public void destroy(NetworkConfiguration config, NetworkOffering offering) {
+    }
+}
diff --git a/server/src/com/cloud/network/dao/NetworkConfigurationDao.java b/server/src/com/cloud/network/dao/NetworkConfigurationDao.java
new file mode 100644
index 00000000000..a842a38d4f2
--- /dev/null
+++ b/server/src/com/cloud/network/dao/NetworkConfigurationDao.java
@@ -0,0 +1,30 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.  
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.network.dao;
+
+import java.util.List;
+
+import com.cloud.network.NetworkConfigurationVO;
+import com.cloud.utils.db.GenericDao;
+
+public interface NetworkConfigurationDao extends GenericDao {
+    List listBy(long accountId);
+    List listBy(long accountId, long offeringId, long dataCenterId);
+    NetworkConfigurationVO persist(NetworkConfigurationVO config, long accountId);
+    void addAccountToNetworkConfiguration(long configId, long accountId);
+}
diff --git a/server/src/com/cloud/network/dao/NetworkConfigurationDaoImpl.java b/server/src/com/cloud/network/dao/NetworkConfigurationDaoImpl.java
new file mode 100644
index 00000000000..8a5c0a03cfa
--- /dev/null
+++ b/server/src/com/cloud/network/dao/NetworkConfigurationDaoImpl.java
@@ -0,0 +1,118 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.  
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
+package com.cloud.network.dao;
+
+import java.util.List;
+
+import javax.ejb.Local;
+
+import com.cloud.network.Network.BroadcastDomainType;
+import com.cloud.network.Network.Mode;
+import com.cloud.network.Network.TrafficType;
+import com.cloud.network.NetworkAccountDaoImpl;
+import com.cloud.network.NetworkAccountVO;
+import com.cloud.network.NetworkConfigurationVO;
+import com.cloud.utils.db.DB;
+import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.JoinBuilder;
+import com.cloud.utils.db.SearchBuilder;
+import com.cloud.utils.db.SearchCriteria;
+import com.cloud.utils.db.Transaction;
+
+@Local(value=NetworkConfigurationDao.class) @DB(txn=false)
+public class NetworkConfigurationDaoImpl extends GenericDaoBase implements NetworkConfigurationDao {
+    final SearchBuilder ProfileSearch;
+    final SearchBuilder AccountSearch;
+    final SearchBuilder OfferingSearch;
+    
+    NetworkAccountDaoImpl _accountsDao = new NetworkAccountDaoImpl();
+    
+    protected NetworkConfigurationDaoImpl() {
+        super();
+        
+        ProfileSearch = createSearchBuilder();
+        ProfileSearch.and("trafficType", ProfileSearch.entity().getTrafficType(), SearchCriteria.Op.EQ);
+        ProfileSearch.and("cidr", ProfileSearch.entity().getCidr(), SearchCriteria.Op.EQ);
+        ProfileSearch.and("broadcastType", ProfileSearch.entity().getBroadcastDomainType(), SearchCriteria.Op.EQ);
+        ProfileSearch.and("offering", ProfileSearch.entity().getNetworkOfferingId(), SearchCriteria.Op.EQ);
+        ProfileSearch.and("datacenter", ProfileSearch.entity().getDataCenterId(), SearchCriteria.Op.EQ);
+        ProfileSearch.done();
+        
+        AccountSearch = createSearchBuilder();
+        AccountSearch.and("offering", AccountSearch.entity().getNetworkOfferingId(), SearchCriteria.Op.EQ);
+        SearchBuilder join = _accountsDao.createSearchBuilder();
+        join.and("account", join.entity().getAccountId(), SearchCriteria.Op.EQ);
+        AccountSearch.join("accounts", join, AccountSearch.entity().getId(), join.entity().getNetworkConfigurationId(), JoinBuilder.JoinType.INNER);
+        AccountSearch.and("datacenter", AccountSearch.entity().getDataCenterId(), SearchCriteria.Op.EQ);
+        AccountSearch.done();
+    
+        OfferingSearch = createSearchBuilder();
+        OfferingSearch.and("offering", OfferingSearch.entity().getNetworkOfferingId(), SearchCriteria.Op.EQ);
+        OfferingSearch.and("datacenter", OfferingSearch.entity().getDataCenterId(), SearchCriteria.Op.EQ);
+    }
+    
+    public List findBy(TrafficType trafficType, Mode mode, BroadcastDomainType broadcastType, long networkOfferingId, long dataCenterId) {
+        SearchCriteria sc = ProfileSearch.create();
+        sc.setParameters("trafficType", trafficType);
+        sc.setParameters("broadcastType", broadcastType);
+        sc.setParameters("offering", networkOfferingId);
+        sc.setParameters("datacenter", dataCenterId);
+        
+        return search(sc, null);
+    }
+    
+    @Override
+    public List listBy(long accountId) {
+        SearchCriteria sc = AccountSearch.create();
+        sc.setParameters("account", accountId);
+        sc.setJoinParameters("accounts", "account", accountId);
+        
+        return listBy(sc);
+    }
+    
+    @Override
+    public List listBy(long accountId, long offeringId, long dataCenterId) {
+        SearchCriteria sc = AccountSearch.create();
+        sc.setParameters("offering", offeringId);
+        sc.setJoinParameters("accounts", "account", accountId);
+        sc.setParameters("datacenter", dataCenterId);
+        
+        return listBy(sc);
+    }
+    
+    @Override
+    public NetworkConfigurationVO persist(NetworkConfigurationVO config) {
+        throw new UnsupportedOperationException("Use the persist for NetworkConfigurationDao");
+    }
+    
+    @Override @DB
+    public NetworkConfigurationVO persist(NetworkConfigurationVO config, long accountId) {
+        Transaction txn = Transaction.currentTxn();
+        txn.start();
+        config = super.persist(config);
+        addAccountToNetworkConfiguration(config.getId(), accountId);
+        txn.commit();
+        return config;
+    }
+    
+    @Override
+    public void addAccountToNetworkConfiguration(long configurationId, long accountId) {
+        NetworkAccountVO account = new NetworkAccountVO(configurationId, accountId);
+        _accountsDao.persist(account);
+    }
+}
diff --git a/server/src/com/cloud/network/router/DomainRouterManager.java b/server/src/com/cloud/network/router/DomainRouterManager.java
new file mode 100644
index 00000000000..e8ec0add3f3
--- /dev/null
+++ b/server/src/com/cloud/network/router/DomainRouterManager.java
@@ -0,0 +1,10 @@
+/**
+ * 
+ */
+package com.cloud.network.router;
+
+import com.cloud.utils.component.Manager;
+
+public interface DomainRouterManager extends Manager {
+
+}
diff --git a/server/src/com/cloud/network/router/DomainRouterManagerImpl.java b/server/src/com/cloud/network/router/DomainRouterManagerImpl.java
new file mode 100644
index 00000000000..6374652e535
--- /dev/null
+++ b/server/src/com/cloud/network/router/DomainRouterManagerImpl.java
@@ -0,0 +1,35 @@
+/**
+ * 
+ */
+package com.cloud.network.router;
+
+import java.util.Map;
+
+import javax.ejb.Local;
+import javax.naming.ConfigurationException;
+
+@Local(value=DomainRouterManager.class)
+public class DomainRouterManagerImpl implements DomainRouterManager {
+    String _name;
+
+    @Override
+    public boolean configure(String name, Map params) throws ConfigurationException {
+        _name = name;
+        return true;
+    }
+
+    @Override
+    public boolean start() {
+        return true;
+    }
+
+    @Override
+    public boolean stop() {
+        return true;
+    }
+
+    @Override
+    public String getName() {
+        return _name;
+    }
+}
diff --git a/server/src/com/cloud/network/security/NetworkGroupManagerImpl.java b/server/src/com/cloud/network/security/NetworkGroupManagerImpl.java
index 2d9af6bb15e..3cfac3acfa6 100644
--- a/server/src/com/cloud/network/security/NetworkGroupManagerImpl.java
+++ b/server/src/com/cloud/network/security/NetworkGroupManagerImpl.java
@@ -76,6 +76,7 @@ import com.cloud.utils.component.Inject;
 import com.cloud.utils.concurrency.NamedThreadFactory;
 import com.cloud.utils.db.DB;
 import com.cloud.utils.db.Filter;
+import com.cloud.utils.db.JoinBuilder;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
 import com.cloud.utils.db.Transaction;
@@ -116,7 +117,7 @@ public class NetworkGroupManagerImpl implements NetworkGroupManager {
 			Comparator {
 		@Override
 		public int compare(NetworkGroupVO o1, NetworkGroupVO o2) {
-			return o1.getId().compareTo(o2.getId());
+			return o1.getId() == o2.getId() ? 0 : o1.getId() < o2.getId() ? -1 : 1;
 		}
 	}
 
@@ -1050,6 +1051,9 @@ public class NetworkGroupManagerImpl implements NetworkGroupManager {
 	@Override
 	@DB
 	public boolean addInstanceToGroups(final Long userVmId, final List groups) {
+		if (!_enabled) {
+			return true;
+		}
 		if (groups != null) {
 			final Set uniqueGroups = new TreeSet(new NetworkGroupVOComparator());
 			uniqueGroups.addAll(groups);
@@ -1090,6 +1094,9 @@ public class NetworkGroupManagerImpl implements NetworkGroupManager {
 	@Override
 	@DB
 	public void removeInstanceFromGroups(Long userVmId) {
+		if (!_enabled) {
+			return;
+		}
 		final Transaction txn = Transaction.currentTxn();
 		txn.start();
 		UserVm userVm = _userVMDao.acquire(userVmId); //ensures that duplicate entries are not created in addInstance
@@ -1180,7 +1187,7 @@ public class NetworkGroupManagerImpl implements NetworkGroupManager {
 			txn.rollback();
 			throw new ResourceInUseException("Cannot delete group when there are ingress rules in this group");
 		}
-        _networkGroupDao.delete(groupId);
+        _networkGroupDao.expunge(groupId);
         txn.commit();
 	}
 
@@ -1234,7 +1241,7 @@ public class NetworkGroupManagerImpl implements NetworkGroupManager {
 
                 if (account != null) {
                     // check that the user is the owner of the VM (admin case was already verified
-                    if (account.getId().longValue() != userVM.getAccountId()) {
+                    if (account.getId() != userVM.getAccountId()) {
                         throw new PermissionDeniedException("Unable to list network groups for virtual machine instance " + instanceId + "; permission denied.");
                     }
                 }
@@ -1255,7 +1262,7 @@ public class NetworkGroupManagerImpl implements NetworkGroupManager {
         if ((accountId == null) && (instanceId == null) && (domainId != null) && Boolean.TRUE.equals(recursive)) {
             SearchBuilder domainSearch = _domainDao.createSearchBuilder();
             domainSearch.and("path", domainSearch.entity().getPath(), SearchCriteria.Op.LIKE);
-            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId());
+            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         }
 
         SearchCriteria sc = sb.create();
diff --git a/server/src/com/cloud/offerings/dao/NetworkOfferingDao.java b/server/src/com/cloud/offerings/dao/NetworkOfferingDao.java
new file mode 100644
index 00000000000..f244864c97c
--- /dev/null
+++ b/server/src/com/cloud/offerings/dao/NetworkOfferingDao.java
@@ -0,0 +1,31 @@
+/**
+ * 
+ */
+package com.cloud.offerings.dao;
+
+import com.cloud.offerings.NetworkOfferingVO;
+import com.cloud.utils.db.GenericDao;
+
+/**
+ * NetworkOfferingDao deals with searches and operations done on the
+ * network_offering table.
+ *
+ */
+public interface NetworkOfferingDao extends GenericDao {
+    /**
+     * Returns the network offering that matches the name.
+     * @param name name
+     * @return NetworkOfferingVO
+     */
+    NetworkOfferingVO findByName(String name);
+    
+    /**
+     * Persists the system network offering by checking the name.  If it
+     * is already there, then it returns the correct one in the database.
+     * If not, then it persists it into the database.
+     * 
+     * @param offering network offering to persist if not in the database.
+     * @return NetworkOfferingVO backed by a row in the database
+     */
+    NetworkOfferingVO persistSystemNetworkOffering(NetworkOfferingVO offering);
+}
diff --git a/server/src/com/cloud/offerings/dao/NetworkOfferingDaoImpl.java b/server/src/com/cloud/offerings/dao/NetworkOfferingDaoImpl.java
new file mode 100644
index 00000000000..dc63e8dd0a8
--- /dev/null
+++ b/server/src/com/cloud/offerings/dao/NetworkOfferingDaoImpl.java
@@ -0,0 +1,51 @@
+/**
+ * 
+ */
+package com.cloud.offerings.dao;
+
+import javax.ejb.Local;
+import javax.persistence.EntityExistsException;
+
+import com.cloud.offerings.NetworkOfferingVO;
+import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.SearchBuilder;
+import com.cloud.utils.db.SearchCriteria;
+
+@Local(value=NetworkOfferingDao.class)
+public class NetworkOfferingDaoImpl extends GenericDaoBase implements NetworkOfferingDao {
+    SearchBuilder NameSearch; 
+    
+    protected NetworkOfferingDaoImpl() {
+        super();
+        
+        NameSearch = createSearchBuilder();
+        NameSearch.and("name", NameSearch.entity().getName(), SearchCriteria.Op.EQ);
+        NameSearch.done();
+    }
+    
+    @Override
+    public NetworkOfferingVO findByName(String name) {
+        SearchCriteria sc = NameSearch.create();
+        
+        sc.setParameters("name", name);
+        
+        return findOneBy(sc);
+        
+    }
+    
+    @Override
+    public NetworkOfferingVO persistSystemNetworkOffering(NetworkOfferingVO offering) {
+        assert offering.getName() != null : "how are you going to find this later if you don't set it?";
+        NetworkOfferingVO vo = findByName(offering.getName());
+        if (vo != null) {
+            return vo;
+        }
+        try {
+            vo = persist(offering);
+            return vo;
+        } catch (EntityExistsException e) {
+            // Assume it's conflict on unique name from two different management servers.
+            return findByName(offering.getName());
+        }
+    }
+}
diff --git a/server/src/com/cloud/resource/Discoverer.java b/server/src/com/cloud/resource/Discoverer.java
index 1d82d9701f9..a162a417afd 100644
--- a/server/src/com/cloud/resource/Discoverer.java
+++ b/server/src/com/cloud/resource/Discoverer.java
@@ -40,5 +40,5 @@ public interface Discoverer extends Adapter {
      */
     Map> find(long dcId, Long podId, Long clusterId, URI uri, String username, String password) throws DiscoveryException;
 
-	void postDiscovery(List hosts, long msId);
+	void postDiscovery(List hosts, long msId)  throws DiscoveryException;
 }
diff --git a/server/src/com/cloud/server/ConfigurationServerImpl.java b/server/src/com/cloud/server/ConfigurationServerImpl.java
index b07169a1a62..24e70a54755 100644
--- a/server/src/com/cloud/server/ConfigurationServerImpl.java
+++ b/server/src/com/cloud/server/ConfigurationServerImpl.java
@@ -145,7 +145,7 @@ public class ConfigurationServerImpl implements ConfigurationServer {
 			_configMgr.createDiskOffering(DomainVO.ROOT_DOMAIN, "Small", "Small Disk, 5 GB", 5, null);
 			_configMgr.createDiskOffering(DomainVO.ROOT_DOMAIN, "Medium", "Medium Disk, 20 GB", 20, null);
 			_configMgr.createDiskOffering(DomainVO.ROOT_DOMAIN, "Large", "Large Disk, 100 GB", 100, null);
-			_configMgr.createDiskOffering(DomainVO.ROOT_DOMAIN, "Private", "Private Disk", 0, null);
+			//_configMgr.createDiskOffering(User.UID_SYSTEM, DomainVO.ROOT_DOMAIN, "Private", "Private Disk", 0, null);
 			
 			   //Add default manual snapshot policy
             SnapshotPolicyVO snapPolicy = new SnapshotPolicyVO(0L, "00", "GMT", (short)4, 0);
@@ -238,7 +238,7 @@ public class ConfigurationServerImpl implements ConfigurationServer {
 		
 		String[] defaultRouteList = defaultRoute.split("\\s+");
 		
-		if (defaultRouteList.length != 7) {
+		if (defaultRouteList.length < 5) {
 			return null;
 		}
 		
@@ -420,10 +420,12 @@ public class ConfigurationServerImpl implements ConfigurationServer {
 
             String homeDir = Script.runSimpleBashScript("echo ~");
             if (homeDir == "~") {
-                s_logger.warn("No home directory was detected.  Trouble with SSH keys ahead.");
-                return;
+                s_logger.error("No home directory was detected.  Set the HOME environment variable to point to your user profile or home directory.");
+                throw new RuntimeException("No home directory was detected.  Set the HOME environment variable to point to your user profile or home directory.");
             }
 
+            String keygenOutput = Script.runSimpleBashScript("if [ -f ~/.ssh/id_rsa ] ; then true ; else yes '' | ssh-keygen -t rsa -q ; fi");
+
             File privkeyfile = new File(homeDir + "/.ssh/id_rsa");
             File pubkeyfile  = new File(homeDir + "/.ssh/id_rsa.pub");
             byte[] arr1 = new byte[4094]; // configuration table column value size
@@ -431,8 +433,8 @@ public class ConfigurationServerImpl implements ConfigurationServer {
                 new DataInputStream(new FileInputStream(privkeyfile)).readFully(arr1);
             } catch (EOFException e) {
             } catch (Exception e) {
-                s_logger.warn("Cannot read the private key file",e);
-                return;
+                s_logger.error("Cannot read the private key file",e);
+                throw new RuntimeException("Cannot read the private key file");
             }
             String privateKey = new String(arr1).trim();
             byte[] arr2 = new byte[4094]; // configuration table column value size
@@ -441,7 +443,7 @@ public class ConfigurationServerImpl implements ConfigurationServer {
             } catch (EOFException e) {			    
             } catch (Exception e) {
                 s_logger.warn("Cannot read the public key file",e);
-                return;
+                throw new RuntimeException("Cannot read the public key file");
             }
             String publicKey  = new String(arr2).trim();
 
@@ -458,7 +460,8 @@ public class ConfigurationServerImpl implements ConfigurationServer {
                     s_logger.debug("Private key inserted into database");
                 }
             } catch (SQLException ex) {
-                s_logger.warn("SQL of the private key failed",ex);
+                s_logger.error("SQL of the private key failed",ex);
+                throw new RuntimeException("SQL of the private key failed");
             }
 
             try {
@@ -468,7 +471,8 @@ public class ConfigurationServerImpl implements ConfigurationServer {
                     s_logger.debug("Public key inserted into database");
                 }
             } catch (SQLException ex) {
-                s_logger.warn("SQL of the public key failed",ex);
+                s_logger.error("SQL of the public key failed",ex);
+                throw new RuntimeException("SQL of the public key failed");
             }
         }
     }
diff --git a/server/src/com/cloud/server/ManagementServer.java b/server/src/com/cloud/server/ManagementServer.java
index 0b381483d3b..7507b0e02db 100644
--- a/server/src/com/cloud/server/ManagementServer.java
+++ b/server/src/com/cloud/server/ManagementServer.java
@@ -17,6 +17,7 @@
  */
 package com.cloud.server;
 
+import java.net.URISyntaxException;
 import java.util.ArrayList;
 import java.util.Date;
 import java.util.List;
@@ -35,10 +36,12 @@ import com.cloud.api.commands.DeleteUserCmd;
 import com.cloud.api.commands.DeployVMCmd;
 import com.cloud.api.commands.EnableAccountCmd;
 import com.cloud.api.commands.EnableUserCmd;
+import com.cloud.api.commands.ExtractVolumeCmd;
 import com.cloud.api.commands.GetCloudIdentifierCmd;
 import com.cloud.api.commands.ListAccountsCmd;
 import com.cloud.api.commands.ListAlertsCmd;
 import com.cloud.api.commands.ListAsyncJobsCmd;
+import com.cloud.api.commands.ListCapabilitiesCmd;
 import com.cloud.api.commands.ListCapacityCmd;
 import com.cloud.api.commands.ListCfgsByCmd;
 import com.cloud.api.commands.ListClustersCmd;
@@ -67,6 +70,7 @@ import com.cloud.api.commands.ListSystemVMsCmd;
 import com.cloud.api.commands.ListTemplateOrIsoPermissionsCmd;
 import com.cloud.api.commands.ListTemplatesCmd;
 import com.cloud.api.commands.ListUsersCmd;
+import com.cloud.api.commands.ListVMGroupsCmd;
 import com.cloud.api.commands.ListVMsCmd;
 import com.cloud.api.commands.ListVlanIpRangesCmd;
 import com.cloud.api.commands.ListVolumesCmd;
@@ -86,6 +90,7 @@ import com.cloud.api.commands.UpdateIPForwardingRuleCmd;
 import com.cloud.api.commands.UpdateTemplateOrIsoCmd;
 import com.cloud.api.commands.UpdateTemplateOrIsoPermissionsCmd;
 import com.cloud.api.commands.UpdateUserCmd;
+import com.cloud.api.commands.UpdateVMGroupCmd;
 import com.cloud.async.AsyncJobResult;
 import com.cloud.async.AsyncJobVO;
 import com.cloud.capacity.CapacityVO;
@@ -136,8 +141,8 @@ import com.cloud.uservm.UserVm;
 import com.cloud.utils.Pair;
 import com.cloud.utils.exception.ExecutionException;
 import com.cloud.vm.ConsoleProxyVO;
-import com.cloud.vm.DomainRouter;
 import com.cloud.vm.DomainRouterVO;
+import com.cloud.vm.InstanceGroupVO;
 import com.cloud.vm.SecondaryStorageVmVO;
 import com.cloud.vm.UserVmVO;
 import com.cloud.vm.VMInstanceVO;
@@ -201,7 +206,6 @@ public interface ManagementServer {
      * @return true if disable was successful, false otherwise
      */
     boolean disableUser(long userId);
-    long disableUserAsync(long userId);
 
     /**
      * Disables an account by accountId
@@ -209,7 +213,6 @@ public interface ManagementServer {
      * @return true if disable was successful, false otherwise
      */
     boolean disableAccount(long accountId);
-    long disableAccountAsync(long accountId); 
     
     /**
      * Enables an account by accountId
@@ -383,45 +386,6 @@ public interface ManagementServer {
      */
     UserVm deployVirtualMachine(DeployVMCmd cmd) throws ResourceAllocationException, InvalidParameterValueException, InternalErrorException, InsufficientStorageCapacityException, PermissionDeniedException, ExecutionException, StorageUnavailableException, ConcurrentOperationException;
 
-    /**
-     * Destroys a Virtual Machine
-     * @param vmId
-     * @return true if destroyed, false otherwise
-     */
-    boolean destroyVirtualMachine(long userId, long vmId);
-    
-    /**
-     * 
-     * @param userId
-     * @param vmId
-     * @return the async-call job id
-     */
-    long destroyVirtualMachineAsync(long userId, long vmId);
-
-    /**
-     * Starts a Domain Router
-     * 
-     * @param routerId
-     * @return DomainRouter if successfully started, false otherwise
-     */
-	DomainRouter startRouter(long routerId, long startEventId) throws InternalErrorException;
-	
-	/**
-	 * Stops a Domain Router
-	 * 
-	 * @param routerId
-	 * @return true if successfully stopped, false otherwise
-	 */
-	boolean stopRouter(long routerId, long startEventId);
-	
-	/**
-	 * Reboots a Domain Router
-	 * 
-	 * @param routerId
-	 * @return true if successfully rebooted, false otherwise
-	 */
-	boolean rebootRouter(long routerId, long startEventId) throws InternalErrorException;
-
     /**
      * Finds a domain router by user and data center
      * @param userId
@@ -815,11 +779,8 @@ public interface ManagementServer {
 
 	ConsoleProxyInfo getConsoleProxy(long dataCenterId, long userVmId);
 	ConsoleProxyVO startConsoleProxy(long instanceId, long startEventId) throws InternalErrorException;
-	long startConsoleProxyAsync(long instanceId);
 	boolean stopConsoleProxy(long instanceId, long startEventId);
-	long stopConsoleProxyAsync(long instanceId);
 	boolean rebootConsoleProxy(long instanceId, long startEventId);
-	long rebootConsoleProxyAsync(long instanceId);
 	String getConsoleAccessUrlRoot(long vmId);
 	ConsoleProxyVO findConsoleProxyById(long instanceId);
 	VMInstanceVO findSystemVMById(long instanceId);
@@ -1087,8 +1048,6 @@ public interface ManagementServer {
      */
     void removeSecurityGroup(long userId, long securityGroupId, String publicIp, long vmId, long startEventId) throws InvalidParameterValueException, PermissionDeniedException;
     void removeSecurityGroup(RemovePortForwardingServiceCmd cmd) throws InvalidParameterValueException, PermissionDeniedException;
-    
-    long removeSecurityGroupAsync(Long userId, long securityGroupId, String publicIp, long vmId);
 
     /**
      * validate that the list of security groups can be applied to the instance
@@ -1210,21 +1169,6 @@ public interface ManagementServer {
      */
     NetworkGroupVO findNetworkGroupById(long networkGroupId);
 
-
-    /**
-	 * Revoke access to a network group.  Access could have been granted to a set of IP ranges, or to network groups belonging to other accounts.  Access
-	 * can be revoked in a similar manner (either from a set of IP ranges or from network groups belonging to other accounts).
-	 * @param accountId the account id of the owner of the given network group
-	 * @param groupName the name of the network group from which access is being revoked
-	 * @param protocol access had been granted on a port range (start port, end port) and network protocol, this protocol scopes the network protocol from which access is being revoked
-	 * @param startPort access had been granted on a port range (start port, end port) and network protocol, this start port scopes the start of a network port range from which access is being revoked
-	 * @param endPort access had been granted on a port range (start port, end port) and network protocol, this end port scopes the end of a network port range from which access is being revoked
-	 * @param cidrList the IP range from which access is being revoked
-	 * @param authorizedGroups the network groups (looked up by group name/account) from which access is being revoked
-	 * @return the job id if scheduled, 0 if the job was not scheduled
-	 */
-	long revokeNetworkGroupIngressAsync(Long accountId, String groupName, String protocol, int startPort, int endPort, String [] cidrList, List authorizedGroups);
-
 	/**
 	 * Is the hypervisor snapshot capable.
 	 * @return True if the hypervisor.type is XenServer
@@ -1239,4 +1183,35 @@ public interface ManagementServer {
 	boolean updateUser(UpdateUserCmd cmd) throws InvalidParameterValueException;
 	boolean updateTemplatePermissions(UpdateTemplateOrIsoPermissionsCmd cmd)throws InvalidParameterValueException, PermissionDeniedException,InternalErrorException;
 	String[] createApiKeyAndSecretKey(RegisterCmd cmd);
+
+	VolumeVO findVolumeByInstanceAndDeviceId(long instanceId, long deviceId);
+
+    InstanceGroupVO updateVmGroup(UpdateVMGroupCmd cmd);
+
+    List searchForVmGroups(ListVMGroupsCmd cmd);
+
+    InstanceGroupVO getGroupForVm(long vmId);
+
+    List searchForZoneWideVlans(long dcId, String vlanType,String vlanId);
+
+    /* 
+     * Fetches the version of cloud stack 
+     */
+    String getVersion();
+
+    Map listCapabilities(ListCapabilitiesCmd cmd);
+    GuestOSVO getGuestOs(Long guestOsId);
+    VolumeVO getRootVolume(Long instanceId);
+    long getPsMaintenanceCount(long podId);
+    boolean isPoolUp(long instanceId);
+    boolean checkIfMaintenable(long hostId);
+
+    /**
+     * Extracts the volume to a particular location.
+     * @param cmd the command specifying url (where the volume needs to be extracted to), zoneId (zone where the volume exists), id (the id of the volume)
+     * @throws URISyntaxException
+     * @throws InternalErrorException
+     *
+     */
+    void extractVolume(ExtractVolumeCmd cmd) throws URISyntaxException, InternalErrorException;
 }
diff --git a/server/src/com/cloud/server/ManagementServerImpl.java b/server/src/com/cloud/server/ManagementServerImpl.java
index e211050a9f7..6cc57bacb3e 100755
--- a/server/src/com/cloud/server/ManagementServerImpl.java
+++ b/server/src/com/cloud/server/ManagementServerImpl.java
@@ -18,7 +18,12 @@
 package com.cloud.server;
 
 import java.math.BigInteger;
+import java.net.Inet6Address;
+import java.net.InetAddress;
+import java.net.URI;
+import java.net.URISyntaxException;
 import java.net.URLEncoder;
+import java.net.UnknownHostException;
 import java.security.MessageDigest;
 import java.security.NoSuchAlgorithmException;
 import java.util.ArrayList;
@@ -48,6 +53,8 @@ import org.apache.log4j.Logger;
 
 import com.cloud.agent.api.GetVncPortAnswer;
 import com.cloud.agent.api.GetVncPortCommand;
+import com.cloud.agent.api.storage.CopyVolumeAnswer;
+import com.cloud.agent.api.storage.CopyVolumeCommand;
 import com.cloud.agent.manager.AgentManager;
 import com.cloud.alert.AlertManager;
 import com.cloud.alert.AlertVO;
@@ -66,10 +73,12 @@ import com.cloud.api.commands.DeleteUserCmd;
 import com.cloud.api.commands.DeployVMCmd;
 import com.cloud.api.commands.EnableAccountCmd;
 import com.cloud.api.commands.EnableUserCmd;
+import com.cloud.api.commands.ExtractVolumeCmd;
 import com.cloud.api.commands.GetCloudIdentifierCmd;
 import com.cloud.api.commands.ListAccountsCmd;
 import com.cloud.api.commands.ListAlertsCmd;
 import com.cloud.api.commands.ListAsyncJobsCmd;
+import com.cloud.api.commands.ListCapabilitiesCmd;
 import com.cloud.api.commands.ListCapacityCmd;
 import com.cloud.api.commands.ListCfgsByCmd;
 import com.cloud.api.commands.ListClustersCmd;
@@ -98,6 +107,7 @@ import com.cloud.api.commands.ListSystemVMsCmd;
 import com.cloud.api.commands.ListTemplateOrIsoPermissionsCmd;
 import com.cloud.api.commands.ListTemplatesCmd;
 import com.cloud.api.commands.ListUsersCmd;
+import com.cloud.api.commands.ListVMGroupsCmd;
 import com.cloud.api.commands.ListVMsCmd;
 import com.cloud.api.commands.ListVlanIpRangesCmd;
 import com.cloud.api.commands.ListVolumesCmd;
@@ -119,6 +129,7 @@ import com.cloud.api.commands.UpdateTemplateOrIsoCmd;
 import com.cloud.api.commands.UpdateTemplateOrIsoPermissionsCmd;
 import com.cloud.api.commands.UpdateTemplatePermissionsCmd;
 import com.cloud.api.commands.UpdateUserCmd;
+import com.cloud.api.commands.UpdateVMGroupCmd;
 import com.cloud.async.AsyncInstanceCreateStatus;
 import com.cloud.async.AsyncJobExecutor;
 import com.cloud.async.AsyncJobManager;
@@ -126,10 +137,7 @@ import com.cloud.async.AsyncJobResult;
 import com.cloud.async.AsyncJobVO;
 import com.cloud.async.BaseAsyncJobExecutor;
 import com.cloud.async.dao.AsyncJobDao;
-import com.cloud.async.executor.NetworkGroupIngressParam;
-import com.cloud.async.executor.SecurityGroupParam;
-import com.cloud.async.executor.VMOperationParam;
-import com.cloud.async.executor.VMOperationParam.VmOp;
+import com.cloud.async.executor.ExtractJobResultObject;
 import com.cloud.capacity.CapacityVO;
 import com.cloud.capacity.dao.CapacityDao;
 import com.cloud.configuration.ConfigurationManager;
@@ -172,6 +180,7 @@ import com.cloud.exception.ResourceAllocationException;
 import com.cloud.exception.StorageUnavailableException;
 import com.cloud.host.Host;
 import com.cloud.host.HostVO;
+import com.cloud.host.Status;
 import com.cloud.host.dao.HostDao;
 import com.cloud.hypervisor.Hypervisor;
 import com.cloud.info.ConsoleProxyInfo;
@@ -193,9 +202,9 @@ import com.cloud.network.dao.SecurityGroupVMMapDao;
 import com.cloud.network.security.NetworkGroupManager;
 import com.cloud.network.security.NetworkGroupVO;
 import com.cloud.network.security.dao.NetworkGroupDao;
+import com.cloud.offering.NetworkOffering;
+import com.cloud.offering.NetworkOffering.GuestIpType;
 import com.cloud.offering.ServiceOffering;
-import com.cloud.offering.ServiceOffering.GuestIpType;
-import com.cloud.serializer.GsonHelper;
 import com.cloud.server.auth.UserAuthenticator;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.service.dao.ServiceOfferingDao;
@@ -211,9 +220,13 @@ import com.cloud.storage.SnapshotVO;
 import com.cloud.storage.Storage;
 import com.cloud.storage.Storage.ImageFormat;
 import com.cloud.storage.StorageManager;
+import com.cloud.storage.StoragePoolHostVO;
 import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.StorageStats;
+import com.cloud.storage.Upload.Type;
+import com.cloud.storage.UploadVO;
 import com.cloud.storage.VMTemplateVO;
+import com.cloud.storage.Volume;
 import com.cloud.storage.Volume.VolumeType;
 import com.cloud.storage.VolumeStats;
 import com.cloud.storage.VolumeVO;
@@ -225,6 +238,8 @@ import com.cloud.storage.dao.LaunchPermissionDao;
 import com.cloud.storage.dao.SnapshotDao;
 import com.cloud.storage.dao.SnapshotPolicyDao;
 import com.cloud.storage.dao.StoragePoolDao;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.UploadDao;
 import com.cloud.storage.dao.VMTemplateDao;
 import com.cloud.storage.dao.VMTemplateDao.TemplateFilter;
 import com.cloud.storage.dao.VolumeDao;
@@ -232,6 +247,7 @@ import com.cloud.storage.preallocatedlun.PreallocatedLunVO;
 import com.cloud.storage.preallocatedlun.dao.PreallocatedLunDao;
 import com.cloud.storage.secondary.SecondaryStorageVmManager;
 import com.cloud.storage.snapshot.SnapshotManager;
+import com.cloud.storage.upload.UploadMonitor;
 import com.cloud.template.TemplateManager;
 import com.cloud.user.Account;
 import com.cloud.user.AccountManager;
@@ -260,6 +276,8 @@ import com.cloud.utils.concurrency.NamedThreadFactory;
 import com.cloud.utils.db.DB;
 import com.cloud.utils.db.Filter;
 import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.db.JoinBuilder;
+import com.cloud.utils.db.JoinBuilder.JoinType;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
 import com.cloud.utils.db.Transaction;
@@ -268,8 +286,9 @@ import com.cloud.utils.exception.ExecutionException;
 import com.cloud.utils.net.MacAddress;
 import com.cloud.utils.net.NetUtils;
 import com.cloud.vm.ConsoleProxyVO;
-import com.cloud.vm.DomainRouter;
 import com.cloud.vm.DomainRouterVO;
+import com.cloud.vm.InstanceGroupVMMapVO;
+import com.cloud.vm.InstanceGroupVO;
 import com.cloud.vm.SecondaryStorageVmVO;
 import com.cloud.vm.State;
 import com.cloud.vm.UserVmManager;
@@ -278,10 +297,11 @@ import com.cloud.vm.VMInstanceVO;
 import com.cloud.vm.VirtualMachine;
 import com.cloud.vm.dao.ConsoleProxyDao;
 import com.cloud.vm.dao.DomainRouterDao;
+import com.cloud.vm.dao.InstanceGroupDao;
+import com.cloud.vm.dao.InstanceGroupVMMapDao;
 import com.cloud.vm.dao.SecondaryStorageVmDao;
 import com.cloud.vm.dao.UserVmDao;
 import com.cloud.vm.dao.VMInstanceDao;
-import com.google.gson.Gson;
 
 public class ManagementServerImpl implements ManagementServer {	
     public static final Logger s_logger = Logger.getLogger(ManagementServerImpl.class.getName());
@@ -330,6 +350,7 @@ public class ManagementServerImpl implements ManagementServer {
     private final GuestOSDao _guestOSDao;
     private final GuestOSCategoryDao _guestOSCategoryDao;
     private final StoragePoolDao _poolDao;
+    private final StoragePoolHostDao _poolHostDao;
     private final StorageManager _storageMgr;
     private final UserVmDao _vmDao;
 
@@ -348,6 +369,10 @@ public class ManagementServerImpl implements ManagementServer {
     private final int _purgeDelay;
     private final boolean _directAttachNetworkExternalIpAllocator;
     private final PreallocatedLunDao _lunDao;
+    private final InstanceGroupDao _vmGroupDao;
+    private final InstanceGroupVMMapDao _groupVMMapDao;
+    private final UploadMonitor _uploadMonitor;
+    private final UploadDao _uploadDao;
 
     private final ScheduledExecutorService _executor = Executors.newScheduledThreadPool(1, new NamedThreadFactory("AccountChecker"));
     private final ScheduledExecutorService _eventExecutor = Executors.newScheduledThreadPool(1, new NamedThreadFactory("EventChecker"));
@@ -357,9 +382,6 @@ public class ManagementServerImpl implements ManagementServer {
     private final Map _configs;
 
     private String _domain;
-    private int _consoleProxyPort = ConsoleProxyManager.DEFAULT_PROXY_VNC_PORT;
-    // private int _consoleProxyUrlPort =
-    // ConsoleProxyManager.DEFAULT_PROXY_URL_PORT;
 
     private final int _routerRamSize;
     private final int _proxyRamSize;
@@ -422,7 +444,11 @@ public class ManagementServerImpl implements ManagementServer {
         _guestOSDao = locator.getDao(GuestOSDao.class);
         _guestOSCategoryDao = locator.getDao(GuestOSCategoryDao.class);
         _poolDao = locator.getDao(StoragePoolDao.class);
+        _poolHostDao = locator.getDao(StoragePoolHostDao.class);
         _vmDao = locator.getDao(UserVmDao.class);
+        _vmGroupDao = locator.getDao(InstanceGroupDao.class);
+        _groupVMMapDao = locator.getDao(InstanceGroupVMMapDao.class);
+        _uploadDao = locator.getDao(UploadDao.class);
 
         _configs = _configDao.getConfiguration();
         _userStatsDao = locator.getDao(UserStatisticsDao.class);
@@ -434,6 +460,7 @@ public class ManagementServerImpl implements ManagementServer {
         _tmpltMgr = locator.getManager(TemplateManager.class);
         _snapMgr = locator.getManager(SnapshotManager.class);
         _networkGroupMgr = locator.getManager(NetworkGroupManager.class);
+        _uploadMonitor = locator.getManager(UploadMonitor.class);                
         
         _userAuthenticators = locator.getAdapters(UserAuthenticator.class);
         if (_userAuthenticators == null || !_userAuthenticators.isSet()) {
@@ -448,21 +475,11 @@ public class ManagementServerImpl implements ManagementServer {
             _domain = "." + _domain;
         }
 
-        String value = _configs.get("consoleproxy.port");
-        if (value != null)
-            _consoleProxyPort = NumbersUtil.parseInt(value, ConsoleProxyManager.DEFAULT_PROXY_VNC_PORT);
-
-        // value = _configs.get("consoleproxy.url.port");
-        // if(value != null)
-        // _consoleProxyUrlPort = NumbersUtil.parseInt(value,
-        // ConsoleProxyManager.DEFAULT_PROXY_URL_PORT);
-
-        value = _configs.get("account.cleanup.interval");
+        String value = _configs.get("account.cleanup.interval");
         int cleanup = NumbersUtil.parseInt(value, 60 * 60 * 24); // 1 hour.
 
         // Parse the max number of UserVMs and public IPs from server-setup.xml,
         // and set them in the right places
-        
         _routerRamSize = NumbersUtil.parseInt(_configs.get("router.ram.size"),NetworkManager.DEFAULT_ROUTER_VM_RAMSIZE);
         _proxyRamSize = NumbersUtil.parseInt(_configs.get("consoleproxy.ram.size"), ConsoleProxyManager.DEFAULT_PROXY_VM_RAMSIZE);
         _ssRamSize = NumbersUtil.parseInt(_configs.get("secstorage.ram.size"), SecondaryStorageVmManager.DEFAULT_SS_VM_RAMSIZE);
@@ -694,6 +711,7 @@ public class ManagementServerImpl implements ManagementServer {
         return user;
     }
 
+    @Override
     public User getUser(long userId, boolean active) {
         if (s_logger.isDebugEnabled()) {
             s_logger.debug("Retrieiving user with id: " + userId + " and active = " + active);
@@ -815,6 +833,15 @@ public class ManagementServerImpl implements ManagementServer {
         boolean accountCleanupNeeded = false;
         
         try {
+        	//delete all vm groups belonging to accont
+        	List groups = _vmGroupDao.listByAccountId(accountId);
+            for (InstanceGroupVO group : groups) {
+                if (!_vmMgr.deleteVmGroup(group.getId())) {
+                    s_logger.error("Unable to delete group: " + group.getId());
+                    accountCleanupNeeded = true;
+                } 
+            }
+        	
             // Delete the snapshots dir for the account. Have to do this before destroying the VMs.
             boolean success = _snapMgr.deleteSnapshotDirsForAccount(accountId);
             if (success) {
@@ -838,6 +865,9 @@ public class ManagementServerImpl implements ManagementServer {
             // Mark the account's volumes as destroyed
             List volumes = _volumeDao.findDetachedByAccount(accountId);
             for (VolumeVO volume : volumes) {
+            	if(volume.getPoolId()==null){
+            		accountCleanupNeeded = true;
+            	}
             	_storageMgr.destroyVolume(volume);
             }
 
@@ -884,10 +914,10 @@ public class ManagementServerImpl implements ManagementServer {
                     // All vm instances have been destroyed, delete the security group -> instance_id mappings
                     SearchCriteria sc = _securityGroupVMMapDao.createSearchCriteria();
                     sc.addAnd("securityGroupId", SearchCriteria.Op.EQ, securityGroup.getId());
-                    _securityGroupVMMapDao.delete(sc);
+                    _securityGroupVMMapDao.expunge(sc);
 
                     // now clean the network rules and security groups themselves
-                    _networkRuleConfigDao.deleteBySecurityGroup(securityGroup.getId().longValue());
+                    _networkRuleConfigDao.deleteBySecurityGroup(securityGroup.getId());
                     _securityGroupDao.remove(securityGroup.getId());
                 }
             }
@@ -945,20 +975,6 @@ public class ManagementServerImpl implements ManagementServer {
         return doSetUserStatus(userId, Account.ACCOUNT_STATE_DISABLED);
     }
 
-    @Override
-    public long disableUserAsync(long userId) {
-        Long param = new Long(userId);
-        Gson gson = GsonHelper.getBuilder().create();
-
-        AsyncJobVO job = new AsyncJobVO();
-        job.setUserId(UserContext.current().getUserId());
-        job.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-        job.setCmd("DisableUser");
-        job.setCmdInfo(gson.toJson(param));
-        
-        return _asyncMgr.submitAsyncJob(job);
-    }
-
     @Override
     public boolean enableUser(EnableUserCmd cmd) throws InvalidParameterValueException, PermissionDeniedException{
     	Long userId = cmd.getId();
@@ -1070,20 +1086,6 @@ public class ManagementServerImpl implements ManagementServer {
         return success;
     }
 
-    @Override
-    public long disableAccountAsync(long accountId) {
-        Long param = new Long(accountId);
-        Gson gson = GsonHelper.getBuilder().create();
-
-        AsyncJobVO job = new AsyncJobVO();
-        job.setUserId(UserContext.current().getUserId());
-        job.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-        job.setCmd("DisableAccount");
-        job.setCmdInfo(gson.toJson(param));
-        
-        return _asyncMgr.submitAsyncJob(job);
-    }
-
     @Override
     public boolean updateAccount(UpdateAccountCmd cmd) throws InvalidParameterValueException, PermissionDeniedException{
     	Long domainId = cmd.getDomainId();
@@ -1104,7 +1106,7 @@ public class ManagementServerImpl implements ManagementServer {
         }
         
         //Don't allow to modify system account
-        if (account.getId().longValue() == Account.ACCOUNT_ID_SYSTEM) {
+        if (account.getId() == Account.ACCOUNT_ID_SYSTEM) {
     		throw new InvalidParameterValueException ("Can not modify system account");
     	}
         
@@ -1167,7 +1169,7 @@ public class ManagementServerImpl implements ManagementServer {
         }
         
         //Don't allow to modify system account
-        if (account.getId().longValue() == Account.ACCOUNT_ID_SYSTEM) {
+        if (account.getId() == Account.ACCOUNT_ID_SYSTEM) {
     		throw new InvalidParameterValueException ("Can not modify system account");
     	}
         
@@ -1221,8 +1223,7 @@ public class ManagementServerImpl implements ManagementServer {
             throw new ServerApiException(BaseCmd.PARAM_ERROR, "unable to find user by id");
         }
 
-        if((apiKey == null && secretKey != null) || (apiKey != null && secretKey == null))
-        {
+        if((apiKey == null && secretKey != null) || (apiKey != null && secretKey == null)) {
         	throw new ServerApiException(BaseCmd.PARAM_ERROR, "Please provide an api key/secret key pair");
         }
         
@@ -1555,12 +1556,12 @@ public class ManagementServerImpl implements ManagementServer {
             ArrayList a = new ArrayList(avoids.values());
             if (_directAttachNetworkExternalIpAllocator) {
             	try {
-            		created = _vmMgr.createDirectlyAttachedVMExternal(vmId, userId, account, dc, offering, template, diskOffering, displayName, group, userData, a, networkGroupVOs, startEventId, size);
+            		created = _vmMgr.createDirectlyAttachedVMExternal(vmId, userId, account, dc, offering, template, diskOffering, displayName, userData, a, networkGroupVOs, startEventId, size);
             	} catch (ResourceAllocationException rae) {
             		throw rae;
             	}
             } else {
-            	if (offering.getGuestIpType() == GuestIpType.Virtualized) {
+            	if (offering.getGuestIpType() == NetworkOffering.GuestIpType.Virtualized) {
             		try {
             			externalIp = _networkMgr.assignSourceNatIpAddress(account, dc, domain, offering, startEventId);
             		} catch (ResourceAllocationException rae) {
@@ -1576,19 +1577,32 @@ public class ManagementServerImpl implements ManagementServer {
             		}
 
             		try {
-            			created = _vmMgr.createVirtualMachine(vmId, userId, account, dc, offering, template, diskOffering, displayName, group, userData, a, startEventId, size);
+            			created = _vmMgr.createVirtualMachine(vmId, userId, account, dc, offering, template, diskOffering, displayName, userData, a, startEventId, size);
             		} catch (ResourceAllocationException rae) {
             			throw rae;
             		}
             	} else {
             		try {
-            			created = _vmMgr.createDirectlyAttachedVM(vmId, userId, account, dc, offering, template, diskOffering, displayName, group, userData, a, networkGroupVOs, startEventId, size);
+            			created = _vmMgr.createDirectlyAttachedVM(vmId, userId, account, dc, offering, template, diskOffering, displayName, userData, a, networkGroupVOs, startEventId, size);
             		} catch (ResourceAllocationException rae) {
             			throw rae;
             		}
             	}
             }
 
+            //assign vm to the group
+            try{
+            	if (group != null) {
+            	boolean addToGroup = _vmMgr.addInstanceToGroup(Long.valueOf(vmId), group);
+            	if (!addToGroup) {
+            		throw new InternalErrorException("Unable to assing Vm to the group " + group);
+            	}
+                }
+            } catch (Exception ex) {
+            	throw new InternalErrorException("Unable to assing Vm to the group " + group);
+            }
+            
+            
             if (created == null) {
                 throw new InternalErrorException("Unable to create VM for account (" + accountId + "): " + account.getAccountName());
             }
@@ -1778,7 +1792,7 @@ public class ManagementServerImpl implements ManagementServer {
             if (!BaseCmd.isAdmin(account.getType()) && ((templateOwner == null) || (templateOwner.longValue() != accountId))) {
                 // since the current account is not the owner of the template, check the launch permissions table to see if the
                 // account can launch a VM from this template
-                LaunchPermissionVO permission = _launchPermissionDao.findByTemplateAndAccount(templateId, account.getId().longValue());
+                LaunchPermissionVO permission = _launchPermissionDao.findByTemplateAndAccount(templateId, account.getId());
                 if (permission == null) {
                     throw new PermissionDeniedException("Account " + account.getAccountName() + " does not have permission to launch instances from template " + template.getName());
                 }
@@ -1799,12 +1813,12 @@ public class ManagementServerImpl implements ManagementServer {
         	}
 			
         }
-        if (offering.getGuestIpType() != GuestIpType.Virtualized) {
+        if (offering.getGuestIpType() != NetworkOffering.GuestIpType.Virtualized) {
         	_networkGroupMgr.createDefaultNetworkGroup(accountId);
     	}
         
         if (networkGroups != null) {
-        	if (offering.getGuestIpType() == GuestIpType.Virtualized) {
+        	if (offering.getGuestIpType() == NetworkOffering.GuestIpType.Virtualized) {
         		throw new InvalidParameterValueException("Network groups are not compatible with service offering " + offering.getName());
         	}
         	Set nameSet = new HashSet(); //handle duplicate names -- allowed
@@ -1816,7 +1830,7 @@ public class ManagementServerImpl implements ManagementServer {
         		throw new InvalidParameterValueException("Some network group names do not exist");
         	}
         } else { //create a default group if necessary
-        	if (offering.getGuestIpType() != GuestIpType.Virtualized && _networkGroupsEnabled) {
+        	if (offering.getGuestIpType() != NetworkOffering.GuestIpType.Virtualized && _networkGroupsEnabled) {
         		networkGroups = new String[]{NetworkGroupManager.DEFAULT_GROUP_NAME};
         	}
         }
@@ -1869,43 +1883,6 @@ public class ManagementServerImpl implements ManagementServer {
         }
     }
 
-    @Override
-    public boolean destroyVirtualMachine(long userId, long vmId) {
-        return _vmMgr.destroyVirtualMachine(userId, vmId);
-    }
-
-    @Override
-    public long destroyVirtualMachineAsync(long userId, long vmId) {
-        
-        UserVmVO userVm = _userVmDao.findById(vmId);
-
-        long eventId = EventUtils.saveScheduledEvent(userId, userVm.getAccountId(), EventTypes.EVENT_VM_DESTROY, "destroying Vm with Id: "+vmId);
-        VMOperationParam param = new VMOperationParam(userId, userVm.getAccountId(), vmId, null, eventId);
-        Gson gson = GsonHelper.getBuilder().create();
-
-        AsyncJobVO job = new AsyncJobVO();
-    	job.setUserId(UserContext.current().getUserId());
-    	job.setAccountId(userVm.getAccountId());
-        job.setCmd("DestroyVM");
-        job.setCmdInfo(gson.toJson(param));
-        return _asyncMgr.submitAsyncJob(job, true);
-    }
-
-    @Override
-    public DomainRouter startRouter(long routerId, long startEventId) throws InternalErrorException {
-        return _networkMgr.startRouter(routerId, startEventId);
-    }
-
-    @Override
-    public boolean stopRouter(long routerId, long startEventId) {
-        return _networkMgr.stopRouter(routerId, startEventId);
-    }
-
-    @Override
-    public boolean rebootRouter(long routerId, long startEventId) throws InternalErrorException {
-        return _networkMgr.rebootRouter(routerId, startEventId);
-    }
-
     @Override
     public DomainRouterVO findDomainRouterBy(long accountId, long dataCenterId) {
         return _routerDao.findBy(accountId, dataCenterId);
@@ -1918,7 +1895,7 @@ public class ManagementServerImpl implements ManagementServer {
 
     @Override
     public List listDataCenters(ListZonesByCmd cmd) {
-        List dcs = _dcDao.listAllActive();
+        List dcs = _dcDao.listAll();
 
         Account account = (Account)UserContext.current().getAccountObject();
         Boolean available = cmd.isAvailable();
@@ -1947,7 +1924,7 @@ public class ManagementServerImpl implements ManagementServer {
     public HostVO getHostBy(long hostId) {
         return _hostDao.findById(hostId);
     }
-    
+
     @Override
     public long getId() {
         return MacAddress.getMacAddress().toLong();
@@ -2005,7 +1982,7 @@ public class ManagementServerImpl implements ManagementServer {
             throw new ServerApiException(BaseCmd.PARAM_ERROR, "Unable to apply port forwarding services " + StringUtils.join(sgIdList, ",") + " to instance " + vmId + ".  Invalid list of port forwarding services for the given instance.");
         }
         if (account != null) {
-            if (!isAdmin(account.getType()) && (account.getId().longValue() != validatedAccountId.longValue())) {
+            if (!isAdmin(account.getType()) && (account.getId() != validatedAccountId.longValue())) {
                 throw new ServerApiException(BaseCmd.ACCOUNT_ERROR, "Permission denied applying port forwarding services " + StringUtils.join(sgIdList, ",") + " to instance " + vmId + ".");
             } else {
                 Account validatedAccount = findAccountById(validatedAccountId);
@@ -2039,7 +2016,7 @@ public class ManagementServerImpl implements ManagementServer {
 
             // sanity check that the vm can be applied to the load balancer
             ServiceOfferingVO offering = _offeringsDao.findById(userVm.getServiceOfferingId());
-            if ((offering == null) || !GuestIpType.Virtualized.equals(offering.getGuestIpType())) {
+            if ((offering == null) || !NetworkOffering.GuestIpType.Virtualized.equals(offering.getGuestIpType())) {
                 if (s_logger.isDebugEnabled()) {
                     s_logger.debug("Unable to apply port forwarding service to virtual machine " + userVm.toString() + ", bad network type (" + ((offering == null) ? "null" : offering.getGuestIpType()) + ")");
                 }
@@ -2274,7 +2251,7 @@ public class ManagementServerImpl implements ManagementServer {
         if (securityG == null) {
         	throw new ServerApiException(BaseCmd.PARAM_ERROR, "unable to find a port forwarding service with id " + securityGroupId);
         } else if (account != null) {
-            if (!isAdmin(account.getType()) && (account.getId().longValue() != securityG.getAccountId())) {
+            if (!isAdmin(account.getType()) && (account.getId() != securityG.getAccountId())) {
                 throw new ServerApiException(BaseCmd.PARAM_ERROR, "unable to find a port forwarding service with id " + securityGroupId + " for this account");
             } else if (!isChildDomain(account.getDomainId(), securityG.getDomainId())) {
                 throw new ServerApiException(BaseCmd.PARAM_ERROR, "Invalid port forwarding service id (" + securityGroupId + ") given, unable to remove port forwarding service.");
@@ -2286,7 +2263,7 @@ public class ManagementServerImpl implements ManagementServer {
         	throw new ServerApiException(BaseCmd.VM_INVALID_PARAM_ERROR, "unable to find a virtual machine with id " + vmId);
         }
         if (account != null) {
-            if (!isAdmin(account.getType()) && (account.getId().longValue() != vmInstance.getAccountId())) {
+            if (!isAdmin(account.getType()) && (account.getId() != vmInstance.getAccountId())) {
                 throw new ServerApiException(BaseCmd.VM_INVALID_PARAM_ERROR, "unable to find a virtual machine with id " + vmId + " for this account");
             } else if (!isChildDomain(account.getDomainId(), vmInstance.getDomainId())) {
                 throw new ServerApiException(BaseCmd.PARAM_ERROR, "Invalid virtual machine id (" + vmId + ") given, unable to remove port forwarding service.");
@@ -2304,7 +2281,7 @@ public class ManagementServerImpl implements ManagementServer {
 
         Long accountId = ipAddrAccount.getId();
         if ((account != null) && !isAdmin(account.getType())) {
-            if (account.getId().longValue() != accountId) {
+            if (account.getId() != accountId) {
                 throw new ServerApiException(BaseCmd.PARAM_ERROR, "account " + account.getAccountName() + " doesn't own ip address " + publicIp);
             }
         }
@@ -2468,21 +2445,6 @@ public class ManagementServerImpl implements ManagementServer {
         }
     }
 
-    @Override
-    public long removeSecurityGroupAsync(Long userId, long securityGroupId, String publicIp, long vmId) {
-        UserVm userVm = _userVmDao.findById(vmId);
-        long eventId = EventUtils.saveScheduledEvent(userId, userVm.getAccountId(), EventTypes.EVENT_PORT_FORWARDING_SERVICE_REMOVE, "removing port forwarding services for Vm with Id: "+vmId);
-        SecurityGroupParam param = new SecurityGroupParam(userId, securityGroupId, null, publicIp, vmId, eventId);
-        Gson gson = GsonHelper.getBuilder().create();
-
-        AsyncJobVO job = new AsyncJobVO();
-    	job.setUserId(UserContext.current().getUserId());
-    	job.setAccountId(userVm.getAccountId());
-        job.setCmd("RemoveSecurityGroup");
-        job.setCmdInfo(gson.toJson(param));
-        return _asyncMgr.submitAsyncJob(job);
-    }
-
     @Override
     public Long validateSecurityGroupsAndInstance(List securityGroupIds, Long instanceId) {
         if ((securityGroupIds == null) || securityGroupIds.isEmpty() || (instanceId == null)) {
@@ -2663,7 +2625,7 @@ public class ManagementServerImpl implements ManagementServer {
         if ((accountName == null) && (domainId != null)) {
             SearchBuilder domainSearch = _domainDao.createSearchBuilder();
             domainSearch.and("path", domainSearch.entity().getPath(), SearchCriteria.Op.LIKE);
-            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId());
+            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         }
 
         SearchCriteria sc = sb.create();
@@ -2734,7 +2696,7 @@ public class ManagementServerImpl implements ManagementServer {
                 throw new InvalidParameterValueException("unable to find a virtual machine with id " + vmId);
             }
             if ((account != null) && !isAdmin(account.getType())) {
-                if (account.getId().longValue() != vmInstance.getAccountId()) {
+                if (account.getId() != vmInstance.getAccountId()) {
                     throw new PermissionDeniedException("unable to find a virtual machine with id " + vmId + " for this account");
                 }
             }
@@ -2927,13 +2889,13 @@ public class ManagementServerImpl implements ManagementServer {
         if (accountId != null) {
         	SearchBuilder accountVlanMapSearch = _accountVlanMapDao.createSearchBuilder();
         	accountVlanMapSearch.and("accountId", accountVlanMapSearch.entity().getAccountId(), SearchCriteria.Op.EQ);
-        	sb.join("accountVlanMapSearch", accountVlanMapSearch, sb.entity().getId(), accountVlanMapSearch.entity().getVlanDbId());
+        	sb.join("accountVlanMapSearch", accountVlanMapSearch, sb.entity().getId(), accountVlanMapSearch.entity().getVlanDbId(), JoinBuilder.JoinType.INNER);
         }
         
         if (podId != null) {
         	SearchBuilder podVlanMapSearch = _podVlanMapDao.createSearchBuilder();
         	podVlanMapSearch.and("podId", podVlanMapSearch.entity().getPodId(), SearchCriteria.Op.EQ);
-        	sb.join("podVlanMapSearch", podVlanMapSearch, sb.entity().getId(), podVlanMapSearch.entity().getVlanDbId());
+        	sb.join("podVlanMapSearch", podVlanMapSearch, sb.entity().getId(), podVlanMapSearch.entity().getVlanDbId(), JoinBuilder.JoinType.INNER);
         }
         
         SearchCriteria sc = sb.create();
@@ -3253,6 +3215,7 @@ public class ManagementServerImpl implements ManagementServer {
         SearchBuilder sb = _accountDao.createSearchBuilder();
         sb.and("accountName", sb.entity().getAccountName(), SearchCriteria.Op.LIKE);
         sb.and("id", sb.entity().getId(), SearchCriteria.Op.EQ);
+        sb.and("nid", sb.entity().getId(), SearchCriteria.Op.NEQ);
         sb.and("type", sb.entity().getType(), SearchCriteria.Op.EQ);
         sb.and("state", sb.entity().getState(), SearchCriteria.Op.EQ);
         sb.and("needsCleanup", sb.entity().getNeedsCleanup(), SearchCriteria.Op.EQ);
@@ -3261,7 +3224,7 @@ public class ManagementServerImpl implements ManagementServer {
             // if accountId isn't specified, we can do a domain match for the admin case
             SearchBuilder domainSearch = _domainDao.createSearchBuilder();
             domainSearch.and("path", domainSearch.entity().getPath(), SearchCriteria.Op.LIKE);
-            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId());
+            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         }
 
         SearchCriteria sc = sb.create();
@@ -3284,6 +3247,9 @@ public class ManagementServerImpl implements ManagementServer {
 
             // I want to join on user_vm.domain_id = domain.id where domain.path like 'foo%'
             sc.setJoinParameters("domainSearch", "path", domain.getPath() + "%");
+            sc.setParameters("nid", 1L);
+        } else {
+        	sc.setParameters("nid", 1L);
         }
 
         if (type != null) {
@@ -3316,7 +3282,7 @@ public class ManagementServerImpl implements ManagementServer {
         if (limitId == null)
             return false;
 
-        return _resourceLimitDao.delete(limitId);
+        return _resourceLimitDao.expunge(limitId);
     }
 
     @Override
@@ -3379,12 +3345,12 @@ public class ManagementServerImpl implements ManagementServer {
 
     @Override
     public List listAllServiceOfferings() {
-        return _offeringsDao.listAll();
+        return _offeringsDao.listAllIncludingRemoved();
     }
 
     @Override
     public List listAllActiveHosts() {
-        return _hostDao.listAllActive();
+        return _hostDao.listAll();
     }
 
     @Override
@@ -3418,7 +3384,7 @@ public class ManagementServerImpl implements ManagementServer {
         if (account != null) {
             Long templateOwner = template.getAccountId();
             if (!BaseCmd.isAdmin(account.getType())) {
-                if ((templateOwner == null) || (account.getId().longValue() != templateOwner.longValue())) {
+                if ((templateOwner == null) || (account.getId() != templateOwner.longValue())) {
                     throw new PermissionDeniedException("Unable to modify template/iso with id " + id + ", permission denied.");
                 }
             } else if (account.getType() != Account.ACCOUNT_TYPE_ADMIN) {
@@ -3547,12 +3513,13 @@ public class ManagementServerImpl implements ManagementServer {
     @Override
     public List searchForUserVMs(Criteria c) {
         Filter searchFilter = new Filter(UserVmVO.class, c.getOrderBy(), c.getAscending(), c.getOffset(), c.getLimit());
+        
         SearchBuilder sb = _userVmDao.createSearchBuilder();
-
+       
         // some criteria matter for generating the join condition
         Object[] accountIds = (Object[]) c.getCriteria(Criteria.ACCOUNTID);
         Object domainId = c.getCriteria(Criteria.DOMAINID);
-
+        
         // get the rest of the criteria
         Object id = c.getCriteria(Criteria.ID);
         Object name = c.getCriteria(Criteria.NAME);
@@ -3565,7 +3532,8 @@ public class ManagementServerImpl implements ManagementServer {
         Object keyword = c.getCriteria(Criteria.KEYWORD);
         Object isAdmin = c.getCriteria(Criteria.ISADMIN);
         Object ipAddress = c.getCriteria(Criteria.IPADDRESS);
-
+        Object groupId = c.getCriteria(Criteria.GROUPID);
+        
         sb.and("displayName", sb.entity().getDisplayName(), SearchCriteria.Op.LIKE);
         sb.and("id", sb.entity().getId(), SearchCriteria.Op.EQ);
         sb.and("accountIdEQ", sb.entity().getAccountId(), SearchCriteria.Op.EQ);
@@ -3579,22 +3547,39 @@ public class ManagementServerImpl implements ManagementServer {
         sb.and("hostIdEQ", sb.entity().getHostId(), SearchCriteria.Op.EQ);
         sb.and("hostIdIN", sb.entity().getHostId(), SearchCriteria.Op.IN);
         sb.and("guestIP", sb.entity().getGuestIpAddress(), SearchCriteria.Op.EQ);
-
+        
         if ((accountIds == null) && (domainId != null)) {
             // if accountId isn't specified, we can do a domain match for the admin case
             SearchBuilder domainSearch = _domainDao.createSearchBuilder();
             domainSearch.and("path", domainSearch.entity().getPath(), SearchCriteria.Op.LIKE);
-            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId());
+            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId(), JoinBuilder.JoinType.INNER);
+        }
+        
+        if (groupId != null && (Long)groupId == -1) {
+        	SearchBuilder vmSearch = _groupVMMapDao.createSearchBuilder();
+        	vmSearch.and("instanceId", vmSearch.entity().getInstanceId(), SearchCriteria.Op.EQ);
+            sb.join("vmSearch", vmSearch, sb.entity().getId(), vmSearch.entity().getInstanceId(), JoinBuilder.JoinType.LEFTOUTER);
+        }
+        else if (groupId != null) {
+        	SearchBuilder groupSearch = _groupVMMapDao.createSearchBuilder();
+        	groupSearch.and("groupId", groupSearch.entity().getGroupId(), SearchCriteria.Op.EQ);
+            sb.join("groupSearch", groupSearch, sb.entity().getId(), groupSearch.entity().getInstanceId(), JoinBuilder.JoinType.INNER);
         }
 
         // populate the search criteria with the values passed in
         SearchCriteria sc = sb.create();
+        
+        if (groupId != null && (Long)groupId == -1){
+        	sc.setJoinParameters("vmSearch", "instanceId", (Object)null);
+        }
+        else if (groupId != null ) {
+        	sc.setJoinParameters("groupSearch", "groupId", groupId);
+        }
 
         if (keyword != null) {
             SearchCriteria ssc = _userVmDao.createSearchCriteria();
             ssc.addOr("displayName", SearchCriteria.Op.LIKE, "%" + keyword + "%");
             ssc.addOr("name", SearchCriteria.Op.LIKE, "%" + keyword + "%");
-            ssc.addOr("group", SearchCriteria.Op.LIKE, "%" + keyword + "%");
             ssc.addOr("instanceName", SearchCriteria.Op.LIKE, "%" + keyword + "%");
             ssc.addOr("state", SearchCriteria.Op.LIKE, "%" + keyword + "%");
 
@@ -3668,7 +3653,7 @@ public class ManagementServerImpl implements ManagementServer {
         if (ipAddress != null) {
             sc.setParameters("guestIP", ipAddress);
         }
-
+        
         return _userVmDao.search(sc, searchFilter);
     }
 
@@ -3735,7 +3720,7 @@ public class ManagementServerImpl implements ManagementServer {
                 if (!_domainDao.isChildDomain(account.getDomainId(), ipAddressVO.getDomainId())) {
                     throw new PermissionDeniedException("Unable to update port forwarding rule on IP address " + publicIp + ", permission denied.");
                 }
-            } else if (account.getId().longValue() != ipAddressVO.getAccountId()) {
+            } else if (account.getId() != ipAddressVO.getAccountId()) {
                 throw new PermissionDeniedException("Unable to update port forwarding rule on IP address " + publicIp + ", permission denied.");
             }
         }
@@ -3793,7 +3778,7 @@ public class ManagementServerImpl implements ManagementServer {
         if ((groupId != null) && (accountId != null)) {
             SecurityGroupVO sg = _securityGroupDao.findById(groupId);
             if (sg != null) {
-                if ((sg.getAccountId() != null) && sg.getAccountId().longValue() != accountId.longValue()) {
+                if (sg.getAccountId() != accountId.longValue()) {
                     throw new PermissionDeniedException("Unable to list port forwarding service rules, account " + accountId + " does not own port forwarding service " + groupId);
                 }
             } else {
@@ -3819,7 +3804,7 @@ public class ManagementServerImpl implements ManagementServer {
             // join with securityGroup table to make sure the account is the owner of the network rule
             SearchBuilder securityGroupSearch = _securityGroupDao.createSearchBuilder();
             securityGroupSearch.and("accountId", securityGroupSearch.entity().getAccountId(), SearchCriteria.Op.EQ);
-            sb.join("groupId", securityGroupSearch, securityGroupSearch.entity().getId(), sb.entity().getSecurityGroupId());
+            sb.join("groupId", securityGroupSearch, securityGroupSearch.entity().getId(), sb.entity().getSecurityGroupId(), JoinBuilder.JoinType.INNER);
         }
 
         SearchCriteria sc = sb.create();
@@ -3902,7 +3887,7 @@ public class ManagementServerImpl implements ManagementServer {
             // if accountId isn't specified, we can do a domain match for the admin case
             SearchBuilder domainSearch = _domainDao.createSearchBuilder();
             domainSearch.and("path", domainSearch.entity().getPath(), SearchCriteria.Op.LIKE);
-            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId());
+            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         }
 
         SearchCriteria sc = sb.create();
@@ -3957,7 +3942,7 @@ public class ManagementServerImpl implements ManagementServer {
 
     @Override
     public List listAllActiveRouters() {
-        return _routerDao.listAllActive();
+        return _routerDao.listAll();
     }
 
     @Override
@@ -4005,13 +3990,12 @@ public class ManagementServerImpl implements ManagementServer {
             // if accountId isn't specified, we can do a domain match for the admin case
             SearchBuilder domainSearch = _domainDao.createSearchBuilder();
             domainSearch.and("path", domainSearch.entity().getPath(), SearchCriteria.Op.LIKE);
-            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId());
+            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         }
 
         SearchCriteria sc = sb.create();
         if (keyword != null) {
             SearchCriteria ssc = _routerDao.createSearchCriteria();
-            ssc.addOr("displayName", SearchCriteria.Op.LIKE, "%" + keyword + "%");
             ssc.addOr("name", SearchCriteria.Op.LIKE, "%" + keyword + "%");
             ssc.addOr("instanceName", SearchCriteria.Op.LIKE, "%" + keyword + "%");
             ssc.addOr("state", SearchCriteria.Op.LIKE, "%" + keyword + "%");
@@ -4168,7 +4152,7 @@ public class ManagementServerImpl implements ManagementServer {
             // if accountId isn't specified, we can do a domain match for the admin case
             SearchBuilder domainSearch = _domainDao.createSearchBuilder();
             domainSearch.and("path", domainSearch.entity().getPath(), SearchCriteria.Op.LIKE);
-            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId());
+            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         }
 
         // now set the SC criteria...
@@ -4210,14 +4194,42 @@ public class ManagementServerImpl implements ManagementServer {
         }
         
         // Don't return DomR and ConsoleProxy volumes
+        /*
         sc.setParameters("domRNameLabel", "r-%");
         sc.setParameters("domPNameLabel", "v-%");
         sc.setParameters("domSNameLabel", "s-%");
-
+		*/
+        
         // Only return volumes that are not destroyed
         sc.setParameters("destroyed", false);
 
-        return _volumeDao.search(sc, searchFilter);
+        List allVolumes = _volumeDao.search(sc, searchFilter);
+        List returnableVolumes = new ArrayList(); //these are ones without domr and console proxy
+        
+        for(VolumeVO v:allVolumes)
+        {
+        	VMTemplateVO template = _templateDao.findById(v.getTemplateId());
+        	if(template!=null && template.getUniqueName().equalsIgnoreCase("routing"))
+        	{
+        		//do nothing
+        	}
+        	else
+        	{
+        		returnableVolumes.add(v);
+        	}
+        }
+        
+        return returnableVolumes;
+    }
+
+    @Override
+    public VolumeVO findVolumeByInstanceAndDeviceId(long instanceId, long deviceId) {
+         VolumeVO volume = _volumeDao.findByInstanceAndDeviceId(instanceId, deviceId).get(0);
+         if (volume != null && !volume.getDestroyed() && volume.getRemoved() == null) {
+             return volume;
+         } else {
+             return null;
+         }
     }
 
     @Override
@@ -4282,13 +4294,13 @@ public class ManagementServerImpl implements ManagementServer {
             // if accountId isn't specified, we can do a domain match for the admin case
             SearchBuilder domainSearch = _domainDao.createSearchBuilder();
             domainSearch.and("path", domainSearch.entity().getPath(), SearchCriteria.Op.LIKE);
-            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId());
+            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         }
         
         if (forVirtualNetwork != null) {
         	SearchBuilder vlanSearch = _vlanDao.createSearchBuilder();
         	vlanSearch.and("vlanType", vlanSearch.entity().getVlanType(), SearchCriteria.Op.EQ);
-        	sb.join("vlanSearch", vlanSearch, sb.entity().getVlanDbId(), vlanSearch.entity().getId());
+        	sb.join("vlanSearch", vlanSearch, sb.entity().getVlanDbId(), vlanSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         }
 
         if ((isAllocated != null) && (isAllocated == true)) {
@@ -4329,7 +4341,7 @@ public class ManagementServerImpl implements ManagementServer {
 
     @Override
     public List listAllActiveDiskTemplates() {
-        return _diskTemplateDao.listAllActive();
+        return _diskTemplateDao.listAll();
     }
 
     @Override
@@ -4486,7 +4498,7 @@ public class ManagementServerImpl implements ManagementServer {
                     if (!_domainDao.isChildDomain(account.getDomainId(), sg.getDomainId())) {
                         throw new PermissionDeniedException("Unable to find rules for port forwarding service id = " + securityGroupId + ", permission denied.");
                     }
-                } else if (account.getId().longValue() != sg.getAccountId().longValue()) {
+                } else if (account.getId() != sg.getAccountId()) {
                     throw new PermissionDeniedException("Invalid port forwarding service (" + securityGroupId + ") given, unable to create rule.");
                 }
             }
@@ -4549,8 +4561,9 @@ public class ManagementServerImpl implements ManagementServer {
         return netRule;
     }
 
+    @Override
     public List listAllTemplates() {
-        return _templateDao.listAll();
+        return _templateDao.listAllIncludingRemoved();
     }
 
     @Override
@@ -4593,76 +4606,32 @@ public class ManagementServerImpl implements ManagementServer {
     	return _guestOSCategoryDao.search(sc, searchFilter);
     }
     
+    @Override
     public String getConfigurationValue(String name) {
     	return _configDao.getValue(name);
     }
 
+    @Override
     public ConsoleProxyInfo getConsoleProxy(long dataCenterId, long userVmId) {
-        ConsoleProxyVO proxy = _consoleProxyMgr.assignProxy(dataCenterId, userVmId);
-        if (proxy == null)
-            return null;
-        
-        return new ConsoleProxyInfo(proxy.isSslEnabled(), proxy.getPublicIpAddress(), _consoleProxyPort, proxy.getPort());
+        return _consoleProxyMgr.assignProxy(dataCenterId, userVmId);
     }
 
+    @Override
     public ConsoleProxyVO startConsoleProxy(long instanceId, long startEventId) throws InternalErrorException {
         return _consoleProxyMgr.startProxy(instanceId, startEventId);
     }
 
+    @Override
     public boolean stopConsoleProxy(long instanceId, long startEventId) {
         return _consoleProxyMgr.stopProxy(instanceId, startEventId);
     }
 
+    @Override
     public boolean rebootConsoleProxy(long instanceId, long startEventId) {
         return _consoleProxyMgr.rebootProxy(instanceId, startEventId);
     }
 
-    public long startConsoleProxyAsync(long instanceId) {
-        long eventId = EventUtils.saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_PROXY_START, "starting console proxy with Id: "+instanceId);
-        VMOperationParam param = new VMOperationParam(0, instanceId, null, eventId);
-        Gson gson = GsonHelper.getBuilder().create();
-
-        AsyncJobVO job = new AsyncJobVO();
-        job.setUserId(UserContext.current().getUserId());
-        job.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-        job.setCmd("StartConsoleProxy");
-        job.setCmdInfo(gson.toJson(param));
-        job.setCmdOriginator(StartSystemVMCmd.getResultObjectName());
-        return _asyncMgr.submitAsyncJob(job, true);
-    }
-
-    public long stopConsoleProxyAsync(long instanceId) {
-        long eventId = EventUtils.saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_PROXY_STOP, "stopping console proxy with Id: "+instanceId);
-        VMOperationParam param = new VMOperationParam(0, instanceId, null, eventId);
-        Gson gson = GsonHelper.getBuilder().create();
-
-        AsyncJobVO job = new AsyncJobVO();
-        job.setUserId(UserContext.current().getUserId());
-        job.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-        job.setCmd("StopConsoleProxy");
-        job.setCmdInfo(gson.toJson(param));
-        // use the same result object name as StartConsoleProxyCmd
-        job.setCmdOriginator(StartSystemVMCmd.getResultObjectName());
-        
-        return _asyncMgr.submitAsyncJob(job, true);
-    }
-
-    public long rebootConsoleProxyAsync(long instanceId) {
-        long eventId = EventUtils.saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_PROXY_REBOOT, "rebooting console proxy with Id: "+instanceId);
-        VMOperationParam param = new VMOperationParam(0, instanceId, null, eventId);
-        Gson gson = GsonHelper.getBuilder().create();
-
-        AsyncJobVO job = new AsyncJobVO();
-        job.setUserId(UserContext.current().getUserId());
-        job.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-        job.setCmd("RebootConsoleProxy");
-        job.setCmdInfo(gson.toJson(param));
-        // use the same result object name as StartConsoleProxyCmd
-        job.setCmdOriginator(StartSystemVMCmd.getResultObjectName());
-        
-        return _asyncMgr.submitAsyncJob(job, true);
-    }
-
+    @Override
     public String getConsoleAccessUrlRoot(long vmId) {
         VMInstanceVO vm = this.findVMInstanceById(vmId);
         if (vm != null) {
@@ -4673,6 +4642,7 @@ public class ManagementServerImpl implements ManagementServer {
         return null;
     }
 
+    @Override
     public int getVncPort(VirtualMachine vm) {
         if (vm.getHostId() == null) {
         	s_logger.warn("VM " + vm.getName() + " does not have host, return -1 for its VNC port");
@@ -4691,6 +4661,7 @@ public class ManagementServerImpl implements ManagementServer {
         return port;
     }
 
+    @Override
     public ConsoleProxyVO findConsoleProxyById(long instanceId) {
         return _consoleProxyDao.findById(instanceId);
     }
@@ -4844,7 +4815,7 @@ public class ManagementServerImpl implements ManagementServer {
         try {
             DomainVO domain = _domainDao.findById(domainId);
             if (domain != null) {
-                long ownerId = domain.getOwner();
+                long ownerId = domain.getAccountId();
                 if ((cleanup != null) && cleanup.booleanValue()) {
                     boolean success = cleanupDomain(domainId, ownerId);
                     if (!success) {
@@ -4880,7 +4851,7 @@ public class ManagementServerImpl implements ManagementServer {
 
             // cleanup sub-domains first
             for (DomainVO domain : domains) {
-                success = (success && cleanupDomain(domain.getId(), domain.getOwner()));
+                success = (success && cleanupDomain(domain.getId(), domain.getAccountId()));
             }
         }
 
@@ -4910,6 +4881,7 @@ public class ManagementServerImpl implements ManagementServer {
         return success && deleteDomainSuccess;
     }
 
+    @Override
     public boolean updateDomain(UpdateDomainCmd cmd) throws InvalidParameterValueException, PermissionDeniedException{
     	Long domainId = cmd.getId();
     	String domainName = cmd.getName();
@@ -4939,15 +4911,16 @@ public class ManagementServerImpl implements ManagementServer {
         if ((domains == null) || domains.isEmpty()) {
             _domainDao.update(domainId, domainName);
             domain = _domainDao.findById(domainId);
-            EventUtils.saveEvent(new Long(1), domain.getOwner(), EventVO.LEVEL_INFO, EventTypes.EVENT_DOMAIN_UPDATE, "Domain, " + domainName + " was updated");
+            EventUtils.saveEvent(new Long(1), domain.getAccountId(), EventVO.LEVEL_INFO, EventTypes.EVENT_DOMAIN_UPDATE, "Domain, " + domainName + " was updated");
             return true;
         } else {
             domain = _domainDao.findById(domainId);
-            EventUtils.saveEvent(new Long(1), domain.getOwner(), EventVO.LEVEL_ERROR, EventTypes.EVENT_DOMAIN_UPDATE, "Failed to update domain " + domain.getName() + " with name " + domainName + ", name in use.");
+            EventUtils.saveEvent(new Long(1), domain.getAccountId(), EventVO.LEVEL_ERROR, EventTypes.EVENT_DOMAIN_UPDATE, "Failed to update domain " + domain.getName() + " with name " + domainName + ", name in use.");
             return false;
         }
     }
 
+    @Override
     public Long findDomainIdByAccountId(Long accountId) {
         if (accountId == null) {
             return null;
@@ -5024,6 +4997,7 @@ public class ManagementServerImpl implements ManagementServer {
         return _capacityDao.search(sc, searchFilter);
     }
     
+    @Override
     public long getMemoryUsagebyHost(Long hostId) {
         long mem = 0;
         List vms = _vmInstanceDao.listUpByHostIdTypes(hostId, VirtualMachine.Type.DomainRouter);
@@ -5276,12 +5250,12 @@ public class ManagementServerImpl implements ManagementServer {
             for (String accountName : accountNames) {
                 Account permittedAccount = _accountDao.findActiveAccount(accountName, domainId);
                 if (permittedAccount != null) {
-                    if (permittedAccount.getId().longValue() == account.getId().longValue()) {
+                    if (permittedAccount.getId() == account.getId()) {
                         continue; // don't grant permission to the template owner, they implicitly have permission
                     }
-                    LaunchPermissionVO existingPermission = _launchPermissionDao.findByTemplateAndAccount(id, permittedAccount.getId().longValue());
+                    LaunchPermissionVO existingPermission = _launchPermissionDao.findByTemplateAndAccount(id, permittedAccount.getId());
                     if (existingPermission == null) {
-                        LaunchPermissionVO launchPermission = new LaunchPermissionVO(id, permittedAccount.getId().longValue());
+                        LaunchPermissionVO launchPermission = new LaunchPermissionVO(id, permittedAccount.getId());
                         _launchPermissionDao.persist(launchPermission);
                     }
                 } else {
@@ -5541,7 +5515,7 @@ public class ManagementServerImpl implements ManagementServer {
 
         if (account != null) {
             if (!isAdmin(account.getType())) {
-                if (account.getId().longValue() != securityGroup.getAccountId()) {
+                if (account.getId() != securityGroup.getAccountId()) {
                     throw new PermissionDeniedException("unable to find port forwarding service with id " + securityGroupId + " for this account, permission denied");
                 }
             } else if (!isChildDomain(account.getDomainId(), securityGroup.getDomainId())) {
@@ -5637,7 +5611,7 @@ public class ManagementServerImpl implements ManagementServer {
             // if accountId isn't specified, we can do a domain match for the admin case
             SearchBuilder domainSearch = _domainDao.createSearchBuilder();
             domainSearch.and("path", domainSearch.entity().getPath(), SearchCriteria.Op.LIKE);
-            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId());
+            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         }
 
         SearchCriteria sc = sb.create();
@@ -5796,7 +5770,7 @@ public class ManagementServerImpl implements ManagementServer {
                 if (!_domainDao.isChildDomain(account.getDomainId(), userAccount.getDomainId())) {
                     throw new PermissionDeniedException("Invalid load balancer rule id (" + loadBalancerId + ") given, unable to list load balancer instances.");
                 }
-            } else if (account.getId().longValue() != lbAcctId) {
+            } else if (account.getId() != lbAcctId) {
                 throw new PermissionDeniedException("Unable to list load balancer instances, account " + account.getAccountName() + " does not own load balancer rule " + loadBalancer.getName());
             }
         }
@@ -5907,13 +5881,13 @@ public class ManagementServerImpl implements ManagementServer {
             // if accountId isn't specified, we can do a domain match for the admin case
             SearchBuilder domainSearch = _domainDao.createSearchBuilder();
             domainSearch.and("path", domainSearch.entity().getPath(), SearchCriteria.Op.LIKE);
-            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId());
+            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         }
 
         if (instanceId != null) {
             SearchBuilder lbVMSearch = _loadBalancerVMMapDao.createSearchBuilder();
             lbVMSearch.and("instanceId", lbVMSearch.entity().getInstanceId(), SearchCriteria.Op.EQ);
-            sb.join("lbVMSearch", lbVMSearch, sb.entity().getId(), lbVMSearch.entity().getLoadBalancerId());
+            sb.join("lbVMSearch", lbVMSearch, sb.entity().getId(), lbVMSearch.entity().getLoadBalancerId(), JoinBuilder.JoinType.INNER);
         }
 
         SearchCriteria sc = sb.create();
@@ -6020,7 +5994,7 @@ public class ManagementServerImpl implements ManagementServer {
                     List oldEvents = _eventDao.listOlderEvents(purgeTime);
                     s_logger.debug("Found "+oldEvents.size()+" events to be purged");
                     for (EventVO event : oldEvents){
-                        _eventDao.delete(event.getId());
+                        _eventDao.expunge(event.getId());
                     }
                 } catch (Exception e) {
                     s_logger.error("Exception ", e);
@@ -6164,9 +6138,9 @@ public class ManagementServerImpl implements ManagementServer {
                 domainSearch.and("path", domainSearch.entity().getPath(), SearchCriteria.Op.LIKE);
 
                 SearchBuilder accountSearch = _accountDao.createSearchBuilder();
-                accountSearch.join("domainSearch", domainSearch, accountSearch.entity().getDomainId(), domainSearch.entity().getId());
+                accountSearch.join("domainSearch", domainSearch, accountSearch.entity().getDomainId(), domainSearch.entity().getId(), JoinType.INNER);
 
-                sb.join("accountSearch", accountSearch, sb.entity().getAccountId(), accountSearch.entity().getId());
+                sb.join("accountSearch", accountSearch, sb.entity().getAccountId(), accountSearch.entity().getId(), JoinType.INNER);
             }
         } else {
             accountId = account.getId();
@@ -6204,7 +6178,7 @@ public class ManagementServerImpl implements ManagementServer {
     public boolean isChildDomain(Long parentId, Long childId) {
         return _domainDao.isChildDomain(parentId, childId);
     }
-    
+
     public SecondaryStorageVmVO startSecondaryStorageVm(long instanceId, long startEventId) throws InternalErrorException {
         return _secStorageVmMgr.startSecStorageVm(instanceId, startEventId);
     }
@@ -6221,66 +6195,6 @@ public class ManagementServerImpl implements ManagementServer {
         return _secStorageVmMgr.destroySecStorageVm(instanceId, startEventId);
     }
 
-    public long startSecondaryStorageVmAsync(long instanceId) {
-        long eventId = EventUtils.saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_SSVM_START, "starting secondary storage Vm Id: "+instanceId);
-        VMOperationParam param = new VMOperationParam(0, instanceId, null, eventId);
-        param.setOperation(VmOp.Start);
-        Gson gson = GsonHelper.getBuilder().create();
-
-        AsyncJobVO job = new AsyncJobVO();
-        job.setUserId(UserContext.current().getUserId());
-        job.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-        job.setCmd("SystemVmCmd");
-        job.setCmdInfo(gson.toJson(param));
-        job.setCmdOriginator(StartSystemVMCmd.getResultObjectName());
-        return _asyncMgr.submitAsyncJob(job, true);
-    }
-
-    public long stopSecondaryStorageVmAsync(long instanceId) {
-        long eventId = EventUtils.saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_SSVM_STOP, "stopping secondary storage Vm Id: "+instanceId);
-        VMOperationParam param = new VMOperationParam(0, instanceId, null, eventId);
-        Gson gson = GsonHelper.getBuilder().create();
-        param.setOperation(VmOp.Stop);
-
-        AsyncJobVO job = new AsyncJobVO();
-        job.setUserId(UserContext.current().getUserId());
-        job.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-        job.setCmd("SystemVmCmd");
-        job.setCmdInfo(gson.toJson(param));
-        job.setCmdOriginator(StartSystemVMCmd.getResultObjectName());
-        return _asyncMgr.submitAsyncJob(job, true);
-    }
-
-    public long rebootSecondaryStorageVmAsync(long instanceId) {
-        long eventId = EventUtils.saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_SSVM_REBOOT, "rebooting secondary storage Vm Id: "+instanceId);
-        VMOperationParam param = new VMOperationParam(0, instanceId, null, eventId);
-        Gson gson = GsonHelper.getBuilder().create();
-        param.setOperation(VmOp.Reboot);
-
-        AsyncJobVO job = new AsyncJobVO();
-        job.setUserId(UserContext.current().getUserId());
-        job.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-        job.setCmd("SystemVmCmd");
-        job.setCmdInfo(gson.toJson(param));
-        job.setCmdOriginator(StartSystemVMCmd.getResultObjectName());
-        return _asyncMgr.submitAsyncJob(job, true);
-    }
-
-    public long destroySecondaryStorageVmAsync(long instanceId) {
-        long eventId = EventUtils.saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_SSVM_DESTROY, "destroying secondary storage Vm Id: "+instanceId);
-        VMOperationParam param = new VMOperationParam(0, instanceId, null, eventId);
-        Gson gson = GsonHelper.getBuilder().create();
-        param.setOperation(VmOp.Destroy);
-
-        AsyncJobVO job = new AsyncJobVO();
-        job.setUserId(UserContext.current().getUserId());
-        job.setAccountId(Account.ACCOUNT_ID_SYSTEM);
-        job.setCmd("SystemVmCmd");
-        job.setCmdInfo(gson.toJson(param));
-        return _asyncMgr.submitAsyncJob(job);
-
-    }
-
 	@Override
     public List searchForSecondaryStorageVm(Criteria c) {
         Filter searchFilter = new Filter(SecondaryStorageVmVO.class, c.getOrderBy(), c.getAscending(), c.getOffset(), c.getLimit());
@@ -6442,7 +6356,7 @@ public class ManagementServerImpl implements ManagementServer {
 		}
 		return null;
 	}
-        
+
     public ArrayList getCloudIdentifierResponse(GetCloudIdentifierCmd cmd) throws InvalidParameterValueException{
     	Long userId = cmd.getUserId();
     	
@@ -6486,24 +6400,6 @@ public class ManagementServerImpl implements ManagementServer {
         return groupVO;
     }
 
-	@Override
-	public long revokeNetworkGroupIngressAsync(Long accountId, String groupName, String protocol, int startPort, int endPort, String [] cidrList, List authorizedGroups) {
-		AccountVO account = (AccountVO)findAccountById(accountId);
-		if (account == null) {
-			s_logger.warn("Unable to revoke network group ingress on group: " + groupName + " for account " + accountId + " -- account not found.");
-			return 0;
-		}
-
-		NetworkGroupIngressParam param = new NetworkGroupIngressParam(account, groupName, protocol, startPort, endPort, cidrList, authorizedGroups);
-        Gson gson = GsonHelper.getBuilder().create();
-        AsyncJobVO job = new AsyncJobVO();
-        job.setUserId(UserContext.current().getUserId());
-        job.setAccountId(accountId);
-        job.setCmd("RevokeNetworkGroupIngress");
-        job.setCmdInfo(gson.toJson(param));
-        return _asyncMgr.submitAsyncJob(job);
-	}
-
     /**
      * {@inheritDoc}
      */
@@ -6573,7 +6469,6 @@ public class ManagementServerImpl implements ManagementServer {
 
 	@Override
 	public boolean lockAccount(LockAccountCmd cmd) {
-
         Account adminAccount = (Account)UserContext.current().getAccountObject();
         Long domainId = cmd.getDomainId();
         String accountName = UserContext.current().getAccountName();
@@ -6588,7 +6483,7 @@ public class ManagementServerImpl implements ManagementServer {
         }
 
         // don't allow modify system account
-        if (account.getId().longValue() == Account.ACCOUNT_ID_SYSTEM) {
+        if (account.getId() == Account.ACCOUNT_ID_SYSTEM) {
             throw new ServerApiException(BaseCmd.INTERNAL_ERROR, "can not lock system account");
         }
 
@@ -6614,13 +6509,302 @@ public class ManagementServerImpl implements ManagementServer {
         return deleteUserInternal(userId);
 	}
 
+	@Override
+	public boolean checkIfMaintenable(long hostId) {
+
+		//get the poolhostref record
+		List poolHostRecordSet = _poolHostDao.listByHostId(hostId);
+		
+		if(poolHostRecordSet!=null)
+		{
+			//the above list has only 1 record
+			StoragePoolHostVO poolHostRecord = poolHostRecordSet.get(0);
+			
+			//get the poolId and get hosts associated in that pool
+			List hostsInPool = _poolHostDao.listByPoolId(poolHostRecord.getPoolId());
+			
+			if(hostsInPool!=null && hostsInPool.size()>1)
+			{
+				return true; //since there are other hosts to take over as master in this pool
+			}
+		}
+		return false;
+	}
+
+    @Override
+    public Map listCapabilities(ListCapabilitiesCmd cmd) {
+        Map capabilities = new HashMap();
+        
+        String networkGroupsEnabled = _configs.get("direct.attach.network.groups.enabled");
+        if(networkGroupsEnabled == null) 
+            networkGroupsEnabled = "false";             
+
+        capabilities.put("networkGroupsEnabled", networkGroupsEnabled);        
+        capabilities.put("cloudStackVersion", getVersion());
+        return capabilities;
+    }
+
+    @Override
+    public GuestOSVO getGuestOs(Long guestOsId)
+    {
+    	return _guestOSDao.findById(guestOsId);
+    }
+    
+    @Override
+    public VolumeVO getRootVolume(Long instanceId)
+    {
+    	return _volumeDao.findByInstanceAndType(instanceId, Volume.VolumeType.ROOT).get(0);
+    }
+    
+    @Override
+    public long getPsMaintenanceCount(long podId){
+    	List poolsInTransition = new ArrayList();
+    	poolsInTransition.addAll(_poolDao.listPoolsByStatus(Status.Maintenance));
+    	poolsInTransition.addAll(_poolDao.listPoolsByStatus(Status.PrepareForMaintenance));
+    	poolsInTransition.addAll(_poolDao.listPoolsByStatus(Status.ErrorInMaintenance));
+
+    	return poolsInTransition.size();
+    }
+    
+    @Override
+    public boolean isPoolUp(long instanceId){
+		VolumeVO rootVolume = _volumeDao.findByInstance(instanceId).get(0);
+		
+		if(rootVolume!=null){
+			Status poolStatus = _poolDao.findById(rootVolume.getPoolId()).getStatus();
+    	
+			if(!poolStatus.equals(Status.Up))
+				return false;
+			else
+				return true;
+		}
+		
+		return false;
+    }
+
+    @Override
+    public void extractVolume(ExtractVolumeCmd cmd) throws URISyntaxException, InternalErrorException {
+        Long volumeId = cmd.getId();
+        String url = cmd.getUrl();
+        Long zoneId = cmd.getZoneId();
+        AsyncJobVO job = cmd.getJob();
+          
+        VolumeVO volume = _volumeDao.findById(volumeId);        
+        if (volume == null) {
+            throw new ServerApiException(BaseCmd.INTERNAL_ERROR, "Unable to find volume with id " + volumeId);
+        }
+
+        URI uri = new URI(url);
+        if ( (uri.getScheme() == null) || (!uri.getScheme().equalsIgnoreCase("ftp") )) {
+           throw new IllegalArgumentException("Unsupported scheme for url: " + url);
+        }
+
+        String host = uri.getHost();
+        try {
+            InetAddress hostAddr = InetAddress.getByName(host);
+            if (hostAddr.isAnyLocalAddress() || hostAddr.isLinkLocalAddress() || hostAddr.isLoopbackAddress() || hostAddr.isMulticastAddress() ) {
+                throw new IllegalArgumentException("Illegal host specified in url");
+            }
+            if (hostAddr instanceof Inet6Address) {
+                throw new IllegalArgumentException("IPV6 addresses not supported (" + hostAddr.getHostAddress() + ")");
+            }
+        } catch (UnknownHostException uhe) {
+            throw new IllegalArgumentException("Unable to resolve " + host);
+        }
+        
+        if (_dcDao.findById(zoneId) == null) {
+            throw new IllegalArgumentException("Please specify a valid zone.");
+        }
+        
+        if ( _uploadMonitor.isTypeUploadInProgress(volumeId, Type.VOLUME) ){
+            throw new IllegalArgumentException(volume.getName() + " upload is in progress. Please wait for some time to schedule another upload for the same");
+        }
+        
+        long userId = UserContext.current().getUserId();
+        long accountId = volume.getAccountId();        
+        long eventId = EventUtils.saveScheduledEvent(userId, accountId, EventTypes.EVENT_VOLUME_UPLOAD, "Extraction job");
+
+        String secondaryStorageURL = _storageMgr.getSecondaryStorageURL(zoneId); 
+        StoragePoolVO srcPool = _poolDao.findById(volume.getPoolId());
+        Long sourceHostId = _storageMgr.findHostIdForStoragePool(srcPool);
+        List storageServers = _hostDao.listByTypeDataCenter(Host.Type.SecondaryStorage, zoneId);
+        HostVO sserver = storageServers.get(0);
+
+// FIXME:  fix the saving of scheduled v. started events
+//        EventUtils.saveStartedEvent(1L, volume.getAccountId(), EventTypes.EVENT_VOLUME_UPLOAD, "Starting upload of " +volume.getName()+ " to " +url, eventId);        
+        UploadVO uploadJob = _uploadMonitor.createNewUploadEntry(sserver.getId(), volumeId, UploadVO.Status.COPY_IN_PROGRESS, 0, Type.VOLUME, null, null, url);
+        uploadJob = _uploadDao.createForUpdate(uploadJob.getId());
+        
+        // Update the async Job
+        ExtractJobResultObject resultObj = new ExtractJobResultObject(volume.getAccountId(), volume.getName(), UploadVO.Status.COPY_IN_PROGRESS.toString(), 0, uploadJob.getId());
+        _asyncMgr.updateAsyncJobAttachment(job.getId(), Type.VOLUME.toString(), volumeId);
+        _asyncMgr.updateAsyncJobStatus(job.getId(), AsyncJobResult.STATUS_IN_PROGRESS, resultObj);
+        
+        
+     // Copy the volume from the source storage pool to secondary storage
+        CopyVolumeCommand cvCmd = new CopyVolumeCommand(volume.getId(), volume.getPath(), srcPool, secondaryStorageURL, true);
+        CopyVolumeAnswer cvAnswer = (CopyVolumeAnswer) _agentMgr.easySend(sourceHostId, cvCmd);
+                
+        if (cvAnswer == null || !cvAnswer.getResult()) {
+            
+            String errorString = "Failed to copy the volume from the source primary storage pool to secondary storage.";
+            
+            resultObj.setResult_string(errorString);
+            resultObj.setUploadStatus(UploadVO.Status.COPY_ERROR.toString());
+            _asyncMgr.completeAsyncJob(job.getId(), AsyncJobResult.STATUS_FAILED, 0, resultObj);
+            
+            uploadJob.setUploadState(UploadVO.Status.COPY_ERROR);            
+            uploadJob.setErrorString(errorString);
+            uploadJob.setLastUpdated(new Date());
+            _uploadDao.update(uploadJob.getId(), uploadJob);
+            
+            EventUtils.saveEvent(1L, volume.getAccountId(), EventTypes.EVENT_VOLUME_UPLOAD, errorString);
+            
+            throw new InternalErrorException(errorString);            
+        }
+        String volumeLocalPath = "volumes/"+volume.getId()+"/"+cvAnswer.getVolumePath()+".vhd"; 
+        uploadJob.setUploadState(UploadVO.Status.COPY_COMPLETE);        
+        uploadJob.setLastUpdated(new Date());
+        _uploadDao.update(uploadJob.getId(), uploadJob);
+        
+        _uploadMonitor.extractVolume(uploadJob, sserver, volume, url, zoneId, volumeLocalPath, eventId, job.getId(), _asyncMgr);
+    }
+
+    @Override
+    public InstanceGroupVO updateVmGroup(UpdateVMGroupCmd cmd) {
+        Account account = (Account)UserContext.current().getAccountObject();
+        Long groupId = cmd.getId();
+        String groupName = cmd.getGroupName();
+
+        // Verify input parameters
+        InstanceGroupVO group = _vmGroupDao.findById(groupId.longValue());
+        if (group == null) {
+            throw new ServerApiException(BaseCmd.PARAM_ERROR, "unable to find a vm group with id " + groupId);
+        }
+
+        if (account != null) {
+            Account tempAccount = _accountDao.findById(group.getAccountId());
+            if (!isAdmin(account.getType()) && (account.getId() != group.getAccountId())) {
+                throw new ServerApiException(BaseCmd.ACCOUNT_ERROR, "unable to find a group with id " + groupId + " for this account");
+            } else if (!_domainDao.isChildDomain(account.getDomainId(), tempAccount.getDomainId())) {
+                throw new ServerApiException(BaseCmd.ACCOUNT_ERROR, "Invalid group id (" + groupId + ") given, unable to update the group.");
+            }
+        }
+
+        //Check if name is already in use by this account (exclude this group)
+        boolean isNameInUse = _vmGroupDao.isNameInUse(group.getAccountId(), groupName);
+
+        if (isNameInUse && !group.getName().equals(groupName)) {
+            throw new ServerApiException(BaseCmd.PARAM_ERROR, "Unable to update vm group, a group with name " + groupName + " already exisits for account");
+        }
+
+        if (groupName != null) {
+        	_vmGroupDao.updateVmGroup(groupId, groupName);
+        }
+        InstanceGroupVO vmGroup = _vmGroupDao.findById(groupId);
+        return vmGroup;
+    }
+
+    @Override
+    public List searchForVmGroups(ListVMGroupsCmd cmd) {
+        Account account = (Account)UserContext.current().getAccountObject();
+        Long domainId = cmd.getDomainId();
+        String accountName = cmd.getAccountName();
+        Long accountId = null;
+        if ((account == null) || isAdmin(account.getType())) {
+            if (domainId != null) {
+                if ((account != null) && !_domainDao.isChildDomain(account.getDomainId(), domainId)) {
+                    throw new ServerApiException(BaseCmd.PARAM_ERROR, "Invalid domain id (" + domainId + ") given, unable to list vm groups.");
+                }
+
+                if (accountName != null) {
+                    account = _accountDao.findActiveAccount(accountName, domainId);
+                    if (account == null) {
+                        throw new ServerApiException(BaseCmd.ACCOUNT_ERROR, "Unable to find account " + accountName + " in domain " + domainId);
+                    }
+                    accountId = account.getId();
+                }
+            } else {
+                domainId = ((account == null) ? DomainVO.ROOT_DOMAIN : account.getDomainId());
+            }
+        } else {
+            accountName = account.getAccountName();
+            accountId = account.getId();
+            domainId = account.getDomainId();
+        }
+
+        Filter searchFilter = new Filter(InstanceGroupVO.class, "id", true, cmd.getStartIndex(), cmd.getPageSizeVal());
+
+        Object id = cmd.getId();
+        Object name = cmd.getGroupName();
+        Object keyword = cmd.getKeyword();
+
+        SearchBuilder sb = _vmGroupDao.createSearchBuilder();
+        sb.and("id", sb.entity().getId(), SearchCriteria.Op.EQ);
+        sb.and("name", sb.entity().getName(), SearchCriteria.Op.LIKE);
+        sb.and("accountId", sb.entity().getAccountId(), SearchCriteria.Op.EQ);
+
+        if ((accountId == null) && (domainId != null)) {
+            // if accountId isn't specified, we can do a domain match for the admin case
+            SearchBuilder domainSearch = _domainDao.createSearchBuilder();
+            domainSearch.and("path", domainSearch.entity().getPath(), SearchCriteria.Op.LIKE);
+            sb.join("domainSearch", domainSearch, sb.entity().getDomainId(), domainSearch.entity().getId(), JoinBuilder.JoinType.INNER);
+        }
+
+        SearchCriteria sc = sb.create();
+        if (keyword != null) {
+            SearchCriteria ssc = _vmGroupDao.createSearchCriteria();
+            ssc.addOr("name", SearchCriteria.Op.LIKE, "%" + keyword + "%");
+        }
+        
+    	if (id != null) {
+        	sc.setParameters("id", id);
+    	}
+        
+        if (name != null) {
+            sc.setParameters("name", "%" + name + "%");
+        }
+
+        if (accountId != null) {
+            sc.setParameters("accountId", accountId);
+        } else if (domainId != null) {
+            DomainVO domain = _domainDao.findById((Long)domainId);
+            if (domain != null){
+            	sc.setJoinParameters("domainSearch", "path", domain.getPath() + "%");
+            }   
+        }
+
+        return _vmGroupDao.search(sc, searchFilter);
+    }
+
+	@Override
+	public InstanceGroupVO getGroupForVm(long vmId){
+		return _vmMgr.getGroupForVm(vmId);
+	}
+
+    @Override
+    public List searchForZoneWideVlans(long dcId, String vlanType, String vlanId){
+    	return _vlanDao.searchForZoneWideVlans(dcId, vlanType, vlanId);
+    }
+
+    @Override
+    public String getVersion(){
+        final Class c = ManagementServer.class;
+        String fullVersion = c.getPackage().getImplementationVersion();
+        String version = "unknown"; 
+        if(fullVersion.length() > 0){
+            version = fullVersion.substring(0,fullVersion.lastIndexOf("."));
+        }
+        return version;
+    }
+
 	private Long checkAccountPermissions(long targetAccountId, long targetDomainId, String targetDesc, long targetId) throws ServerApiException {
         Long accountId = null;
 
         Account account = (Account)UserContext.current().getAccountObject();
         if (account != null) {
             if (!isAdmin(account.getType())) {
-                if (account.getId().longValue() != targetAccountId) {
+                if (account.getId() != targetAccountId) {
                     throw new ServerApiException(BaseCmd.PARAM_ERROR, "Unable to find a " + targetDesc + " with id " + targetId + " for this account");
                 }
             } else if (!_domainDao.isChildDomain(account.getDomainId(), targetDomainId)) {
diff --git a/server/src/com/cloud/server/StatsCollector.java b/server/src/com/cloud/server/StatsCollector.java
index 861a686fd71..5e10967bd30 100644
--- a/server/src/com/cloud/server/StatsCollector.java
+++ b/server/src/com/cloud/server/StatsCollector.java
@@ -61,6 +61,7 @@ import com.cloud.utils.component.ComponentLocator;
 import com.cloud.utils.concurrency.NamedThreadFactory;
 import com.cloud.utils.db.GlobalLock;
 import com.cloud.utils.db.SearchCriteria;
+import com.cloud.utils.db.Transaction;
 import com.cloud.vm.UserVmManager;
 import com.cloud.vm.UserVmVO;
 import com.cloud.vm.VmStats;
@@ -99,7 +100,7 @@ public class StatsCollector {
 	long storageStatsInterval = -1L;
 	long volumeStatsInterval = -1L;
 
-	private final GlobalLock m_capacityCheckLock = GlobalLock.getInternLock("capacity.check");
+	//private final GlobalLock m_capacityCheckLock = GlobalLock.getInternLock("capacity.check");
 
     public static StatsCollector getInstance() {
         return s_instance;
@@ -283,7 +284,7 @@ public class StatsCollector {
 				
 				ConcurrentHashMap storagePoolStats = new ConcurrentHashMap();
 
-				List storagePools = _storagePoolDao.listAllActive();
+				List storagePools = _storagePoolDao.listAll();
 				for (StoragePoolVO pool: storagePools) {
 					GetStorageStatsCommand command = new GetStorageStatsCommand(pool.getUuid(), pool.getPoolType(), pool.getPath());
 					Answer answer = _storageManager.sendToPool(pool, command);
@@ -292,66 +293,65 @@ public class StatsCollector {
 					}
 				}
 				_storagePoolStats = storagePoolStats;
+
+				// a list to store the new capacity entries that will be committed once everything is calculated
+				List newCapacities = new ArrayList();
 
-		        if (m_capacityCheckLock.lock(5)) { // 5 second timeout
-		            if (s_logger.isTraceEnabled()) {
-		                s_logger.trace("recalculating system storage capacity");
-		            }
-		            try {
-		                // now update the capacity table with the new stats
-		                // FIXME: the right way to do this is to register a listener (see RouterStatsListener)
-		                //        for the host stats, send the WatchCommand at a regular interval
-		                //        to collect the stats from an agent and update the database as needed.  The
-		                //        listener model has connects/disconnects to keep things in sync much better
-		                //        than this model right now
-		                _capacityDao.clearStorageCapacities();
-
-		                // create new entries
-		                for (Long hostId : storageStats.keySet()) {
-		                    StorageStats stats = storageStats.get(hostId);
-		                    HostVO host = _hostDao.findById(hostId);
-		                    host.setTotalSize(stats.getCapacityBytes());
-		                    _hostDao.update(host.getId(), host);
-
-		                    if (Host.Type.SecondaryStorage.equals(host.getType())) {
-	                            CapacityVO capacity = new CapacityVO(host.getId(), host.getDataCenterId(), host.getPodId(), stats.getByteUsed(), stats.getCapacityBytes(), CapacityVO.CAPACITY_TYPE_SECONDARY_STORAGE);
-	                            _capacityDao.persist(capacity);
-		                    } else if (Host.Type.Storage.equals(host.getType())) {
-	                            CapacityVO capacity = new CapacityVO(host.getId(), host.getDataCenterId(), host.getPodId(), stats.getByteUsed(), stats.getCapacityBytes(), CapacityVO.CAPACITY_TYPE_STORAGE);
-	                            _capacityDao.persist(capacity);
-		                    }
-		                }
-
-		                for (Long poolId : storagePoolStats.keySet()) {
-		                    StorageStats stats = storagePoolStats.get(poolId);
-		                    StoragePoolVO pool = _storagePoolDao.findById(poolId);
-		                    
-		                    if (pool == null) {
-		                    	continue;
-		                    }
-		                    
-		                    pool.setCapacityBytes(stats.getCapacityBytes());
-		                    long available = stats.getCapacityBytes() - stats.getByteUsed();
-		                    if( available < 0 ) {
-		                        available = 0;
-		                    }
-		                    pool.setAvailableBytes(available);
-		                    _storagePoolDao.update(pool.getId(), pool);		                    
-		                    
-		                    CapacityVO capacity = new CapacityVO(poolId, pool.getDataCenterId(), pool.getPodId(), stats.getByteUsed(), stats.getCapacityBytes(), CapacityVO.CAPACITY_TYPE_STORAGE);
-		                    _capacityDao.persist(capacity);
-		                }
-		            } finally {
-		                m_capacityCheckLock.unlock();
-		            }
-                    if (s_logger.isTraceEnabled()) {
-                        s_logger.trace("done recalculating system storage capacity");
-                    }
-		        } else {
-                    if (s_logger.isTraceEnabled()) {
-                        s_logger.trace("not recalculating system storage capacity, unable to lock capacity table");
-                    }
-		        }
+                // create new entries
+                for (Long hostId : storageStats.keySet()) {
+                    StorageStats stats = storageStats.get(hostId);
+                    HostVO host = _hostDao.findById(hostId);
+                    host.setTotalSize(stats.getCapacityBytes());
+                    _hostDao.update(host.getId(), host);
+
+                    if (Host.Type.SecondaryStorage.equals(host.getType())) {
+                        CapacityVO capacity = new CapacityVO(host.getId(), host.getDataCenterId(), host.getPodId(), stats.getByteUsed(), stats.getCapacityBytes(), CapacityVO.CAPACITY_TYPE_SECONDARY_STORAGE);
+                        newCapacities.add(capacity);
+//                        _capacityDao.persist(capacity);
+                    } else if (Host.Type.Storage.equals(host.getType())) {
+                        CapacityVO capacity = new CapacityVO(host.getId(), host.getDataCenterId(), host.getPodId(), stats.getByteUsed(), stats.getCapacityBytes(), CapacityVO.CAPACITY_TYPE_STORAGE);
+                        newCapacities.add(capacity);
+//                        _capacityDao.persist(capacity);
+                    }
+                }
+
+                for (Long poolId : storagePoolStats.keySet()) {
+                    StorageStats stats = storagePoolStats.get(poolId);
+                    StoragePoolVO pool = _storagePoolDao.findById(poolId);
+                    
+                    if (pool == null) {
+                        continue;
+                    }
+                    
+                    pool.setCapacityBytes(stats.getCapacityBytes());
+                    long available = stats.getCapacityBytes() - stats.getByteUsed();
+                    if( available < 0 ) {
+                        available = 0;
+                    }
+                    pool.setAvailableBytes(available);
+                    _storagePoolDao.update(pool.getId(), pool);                         
+                    
+                    _storageManager.createCapacityEntry(pool, 0L);
+                }
+
+                Transaction txn = Transaction.open(Transaction.CLOUD_DB);
+                try {
+                	if (s_logger.isTraceEnabled()) {
+		                s_logger.trace("recalculating system storage capacity");
+		            }
+                	txn.start();
+	                for (CapacityVO newCapacity : newCapacities) {
+	                	s_logger.trace("Executing capacity update");
+	                    _capacityDao.persist(newCapacity);
+	                    s_logger.trace("Done with capacity update");
+	                }
+		            txn.commit();
+                } catch (Exception ex) {
+                	txn.rollback();
+                	s_logger.error("Unable to start transaction for storage capacity update");
+                }finally {
+                	txn.close();
+                }
 			} catch (Throwable t) {
 				s_logger.error("Error trying to retrieve storage stats", t);
 			}
@@ -373,7 +373,7 @@ public class StatsCollector {
 	class VolumeCollector implements Runnable {
 		public void run() {
 			try {
-				List volumes = _volsDao.listAllActive();
+				List volumes = _volsDao.listAll();
 				Map> commandsByPool = new HashMap>();
 				
 				for (VolumeVO volume : volumes) {
diff --git a/server/src/com/cloud/servlet/ConsoleProxyServlet.java b/server/src/com/cloud/servlet/ConsoleProxyServlet.java
index 7efde54e29d..ad28f1b7522 100644
--- a/server/src/com/cloud/servlet/ConsoleProxyServlet.java
+++ b/server/src/com/cloud/servlet/ConsoleProxyServlet.java
@@ -19,7 +19,15 @@
 package com.cloud.servlet;
 
 import java.io.IOException;
+import java.net.URLEncoder;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
 
+import javax.crypto.Mac;
+import javax.crypto.spec.SecretKeySpec;
 import javax.servlet.http.HttpServlet;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
@@ -31,7 +39,10 @@ import com.cloud.host.HostVO;
 import com.cloud.server.ManagementServer;
 import com.cloud.user.Account;
 import com.cloud.user.User;
+import com.cloud.utils.Pair;
 import com.cloud.utils.component.ComponentLocator;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.encoding.Base64;
 import com.cloud.vm.UserVmVO;
 import com.cloud.vm.VMInstanceVO;
 
@@ -56,24 +67,32 @@ public class ConsoleProxyServlet extends HttpServlet {
 	@Override
     protected void doGet(HttpServletRequest req, HttpServletResponse resp) {
 		try {
+            String userId = null;
+            String account = null;
+            Account accountObj = null;
+			
+            Map params = new HashMap();
+            params.putAll(req.getParameterMap());
+            
             HttpSession session = req.getSession(false);
             if(session == null) {
-				s_logger.info("Invalid web session, reject console/thumbnail access");
-				sendResponse(resp, "Access denied. You haven't logged in or your web session has timed out");
-				return;
-            }
-
-            // FIXME:  are these set up correctly from ApiServer?
-            String userIdStr = (String)session.getAttribute("userid");
-            String account = (String)session.getAttribute("account");
-            Object accountObj = session.getAttribute("accountobj");
-            Long userId = null;
-            if (userIdStr != null) {
-                userId = Long.parseLong(userIdStr);
+            	if(verifyRequest(params)) {
+                    userId = (String)params.get("userid")[0];
+                    account = (String)params.get("account")[0];
+                    accountObj = (Account)params.get("accountobj")[0];
+            	} else {
+					s_logger.info("Invalid web session or API key in request, reject console/thumbnail access");
+					sendResponse(resp, "Access denied. Invalid web session or API key in request");
+					return;
+            	}
+            } else {
+                userId = (String)session.getAttribute("userid");
+                account = (String)session.getAttribute("account");
+                accountObj = (Account)session.getAttribute("accountobj");
             }
 
             // Do a sanity check here to make sure the user hasn't already been deleted
-            if ((userId == null) || (account == null) || (accountObj == null) || !verifyUser(userId)) {
+            if ((userId == null) || (account == null) || (accountObj == null) || !verifyUser(Long.valueOf(userId))) {
 				s_logger.info("Invalid user/account, reject console/thumbnail access");
 				sendResponse(resp, "Access denied. Invalid or inconsistent account is found");
 				return;
@@ -96,7 +115,7 @@ public class ConsoleProxyServlet extends HttpServlet {
 				return;
 			}
 			
-			if(!checkSessionPermision(req, vmId)) {
+			if(!checkSessionPermision(req, vmId, accountObj)) {
 				sendResponse(resp, "Permission denied");
 				return;
 			}
@@ -110,7 +129,7 @@ public class ConsoleProxyServlet extends HttpServlet {
 			
 		} catch (Throwable e) {
 			s_logger.error("Unexepected exception in ConsoleProxyServlet", e);
-			sendResponse(resp, "");
+			sendResponse(resp, "Server Internal Error");
 		}
 	}
 	
@@ -261,7 +280,8 @@ public class ConsoleProxyServlet extends HttpServlet {
 	}
 	
 	private void sendResponse(HttpServletResponse resp, String content) {
-		try {
+		try {
+			resp.setContentType("text/html");
 			resp.getWriter().print(content);
 		} catch(IOException e) {
 			if(s_logger.isInfoEnabled())
@@ -269,18 +289,15 @@ public class ConsoleProxyServlet extends HttpServlet {
 		}
 	}
 	
-	private boolean checkSessionPermision(HttpServletRequest req, long vmId) {
+	private boolean checkSessionPermision(HttpServletRequest req, long vmId, Account accountObj) {
 
-        HttpSession session = req.getSession(false);
-        Account accountObj = (Account)session.getAttribute("accountobj");
-        
         VMInstanceVO vm = _ms.findVMInstanceById(vmId);
         UserVmVO userVm;
         switch(vm.getType())
         {
         case User :
         	userVm = _ms.findUserVMInstanceById(vmId);
-        	if(userVm.getAccountId() != accountObj.getId().longValue() && accountObj.getType() != Account.ACCOUNT_TYPE_ADMIN) {
+        	if(userVm.getAccountId() != accountObj.getId() && accountObj.getType() != Account.ACCOUNT_TYPE_ADMIN) {
         		if(s_logger.isDebugEnabled())
 	        		s_logger.debug("VM access is denied. VM owner account " + userVm.getAccountId() 
 	        			+ " does not match the account id in session " + accountObj.getId());
@@ -325,4 +342,95 @@ public class ConsoleProxyServlet extends HttpServlet {
     	}
     	return true;
     }
+    
+	// copied and modified from ApiServer.java.
+    // TODO need to replace the whole servlet with a API command
+    private boolean verifyRequest(Map requestParameters) {
+        try {
+            String apiKey = null;
+            String secretKey = null;
+            String signature = null;
+            String unsignedRequest = null;
+
+            // - build a request string with sorted params, make sure it's all lowercase
+            // - sign the request, verify the signature is the same
+            List parameterNames = new ArrayList();
+
+            for (Object paramNameObj : requestParameters.keySet()) {
+                parameterNames.add((String)paramNameObj); // put the name in a list that we'll sort later
+            }
+
+            Collections.sort(parameterNames);
+
+            for (String paramName : parameterNames) {
+                // parameters come as name/value pairs in the form String/String[]
+                String paramValue = ((String[])requestParameters.get(paramName))[0];
+                
+                if ("signature".equalsIgnoreCase(paramName)) {
+                    signature = paramValue;
+                } else {
+                    if ("apikey".equalsIgnoreCase(paramName)) {
+                        apiKey = paramValue;
+                    }
+
+                    if (unsignedRequest == null) {
+                        unsignedRequest = paramName + "=" + URLEncoder.encode(paramValue, "UTF-8").replaceAll("\\+", "%20");
+                    } else {
+                        unsignedRequest = unsignedRequest + "&" + paramName + "=" + URLEncoder.encode(paramValue, "UTF-8").replaceAll("\\+", "%20");
+                    }
+                }
+            }
+            
+
+            // if api/secret key are passed to the parameters
+            if ((signature == null) || (apiKey == null)) {
+                if (s_logger.isDebugEnabled()) {
+                    s_logger.info("expired session, missing signature, or missing apiKey -- ignoring request...sig: " + signature + ", apiKey: " + apiKey);
+                }
+                return false; // no signature, bad request
+            }
+
+            Transaction txn = Transaction.open(Transaction.CLOUD_DB);
+            txn.close();
+            User user = null;
+            // verify there is a user with this api key
+            Pair userAcctPair = _ms.findUserByApiKey(apiKey);
+            if (userAcctPair == null) {
+                s_logger.info("apiKey does not map to a valid user -- ignoring request, apiKey: " + apiKey);
+                return false;
+            }
+
+            user = userAcctPair.first();
+            Account account = userAcctPair.second();
+
+            if (!user.getState().equals(Account.ACCOUNT_STATE_ENABLED) || !account.getState().equals(Account.ACCOUNT_STATE_ENABLED)) {
+                s_logger.info("disabled or locked user accessing the api, userid = " + user.getId() + "; name = " + user.getUsername() + "; state: " + user.getState() + "; accountState: " + account.getState());
+                return false;
+            }     
+
+            // verify secret key exists
+            secretKey = user.getSecretKey();
+            if (secretKey == null) {
+                s_logger.info("User does not have a secret key associated with the account -- ignoring request, username: " + user.getUsername());
+                return false;
+            }
+
+            unsignedRequest = unsignedRequest.toLowerCase();
+
+            Mac mac = Mac.getInstance("HmacSHA1");
+            SecretKeySpec keySpec = new SecretKeySpec(secretKey.getBytes(), "HmacSHA1");
+            mac.init(keySpec);
+            mac.update(unsignedRequest.getBytes());
+            byte[] encryptedBytes = mac.doFinal();
+            String computedSignature = Base64.encodeBytes(encryptedBytes);
+            boolean equalSig = signature.equals(computedSignature);
+            if (!equalSig) {
+            	s_logger.info("User signature: " + signature + " is not equaled to computed signature: " + computedSignature);
+            }
+            return equalSig;
+        } catch (Exception ex) {
+            s_logger.error("unable to verifty request signature", ex);
+        }
+        return false;
+    }
 }
diff --git a/server/src/com/cloud/storage/LocalStoragePoolListener.java b/server/src/com/cloud/storage/LocalStoragePoolListener.java
index 626a7cad7d8..2c82edfe064 100644
--- a/server/src/com/cloud/storage/LocalStoragePoolListener.java
+++ b/server/src/com/cloud/storage/LocalStoragePoolListener.java
@@ -29,7 +29,7 @@ import com.cloud.agent.api.StartupStorageCommand;
 import com.cloud.agent.api.StoragePoolInfo;
 import com.cloud.host.HostVO;
 import com.cloud.host.Status;
-import com.cloud.storage.Volume.StorageResourceType;
+import com.cloud.storage.Storage.StorageResourceType;
 import com.cloud.storage.dao.StoragePoolDao;
 import com.cloud.storage.dao.StoragePoolHostDao;
 import com.cloud.utils.component.Inject;
@@ -72,7 +72,7 @@ public class LocalStoragePoolListener implements Listener {
         
         StartupStorageCommand ssCmd = (StartupStorageCommand)cmd;
         
-        if (ssCmd.getResourceType() != StorageResourceType.STORAGE_POOL) {
+        if (ssCmd.getResourceType() != Storage.StorageResourceType.STORAGE_POOL) {
             return true;
         }
         
diff --git a/server/src/com/cloud/storage/StorageManager.java b/server/src/com/cloud/storage/StorageManager.java
old mode 100644
new mode 100755
index fe5e0594de6..7c12ae904b9
--- a/server/src/com/cloud/storage/StorageManager.java
+++ b/server/src/com/cloud/storage/StorageManager.java
@@ -29,8 +29,10 @@ import com.cloud.api.commands.DeletePoolCmd;
 import com.cloud.api.commands.DeleteVolumeCmd;
 import com.cloud.api.commands.PreparePrimaryStorageForMaintenanceCmd;
 import com.cloud.api.commands.UpdateStoragePoolCmd;
+import com.cloud.agent.api.to.VolumeTO;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
+import com.cloud.deploy.DeployDestination;
 import com.cloud.exception.InternalErrorException;
 import com.cloud.exception.InvalidParameterValueException;
 import com.cloud.exception.PermissionDeniedException;
@@ -40,13 +42,21 @@ import com.cloud.exception.StorageUnavailableException;
 import com.cloud.host.Host;
 import com.cloud.host.HostVO;
 import com.cloud.service.ServiceOfferingVO;
+import com.cloud.storage.Volume.VolumeType;
 import com.cloud.user.Account;
+import com.cloud.user.AccountVO;
 import com.cloud.utils.Pair;
 import com.cloud.utils.component.Manager;
 import com.cloud.utils.exception.ExecutionException;
+import com.cloud.vm.DiskProfile;
 import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachineProfile;
 
 public interface StorageManager extends Manager {
+    
+    
+    VolumeVO allocateIsoInstalledVm(VMInstanceVO vm, VMTemplateVO template, DiskOfferingVO rootOffering, Long size, DataCenterVO dc, AccountVO account);
+    
 	/**
 	 * Calls the storage agent and makes the volumes sharable with this host.
 	 * 
@@ -294,4 +304,26 @@ public interface StorageManager extends Manager {
     public StoragePoolVO cancelPrimaryStorageForMaintenance(CancelPrimaryStorageMaintenanceCmd cmd) throws InvalidParameterValueException;
 
 	public StoragePoolVO updateStoragePool(UpdateStoragePoolCmd cmd) throws IllegalArgumentException;
+    
+    /**
+     * Allocates one volume.
+     * @param 
+     * @param type
+     * @param offering
+     * @param name
+     * @param size
+     * @param template
+     * @param vm
+     * @param account
+     * @return VolumeVO a persisted volume.
+     */
+     DiskProfile allocateRawVolume(VolumeType type, String name, DiskOfferingVO offering, Long size, T vm, AccountVO owner);
+     DiskProfile allocateTemplatedVolume(VolumeType type, String name, DiskOfferingVO offering, VMTemplateVO template, T vm, AccountVO owner);
+    
+     void create(T vm);
+    Long findHostIdForStoragePool(StoragePoolVO pool);
+	void createCapacityEntry(StoragePoolVO storagePool, long allocated);
+
+    
+    VolumeTO[] prepare(VirtualMachineProfile vm, DeployDestination dest) throws StorageUnavailableException;
 }
diff --git a/server/src/com/cloud/storage/StorageManagerImpl.java b/server/src/com/cloud/storage/StorageManagerImpl.java
old mode 100644
new mode 100755
index dd303e663dc..9c3adbc25e8
--- a/server/src/com/cloud/storage/StorageManagerImpl.java
+++ b/server/src/com/cloud/storage/StorageManagerImpl.java
@@ -57,7 +57,6 @@ import com.cloud.agent.api.storage.CreateAnswer;
 import com.cloud.agent.api.storage.CreateCommand;
 import com.cloud.agent.api.storage.DeleteTemplateCommand;
 import com.cloud.agent.api.storage.DestroyCommand;
-import com.cloud.agent.api.to.DiskCharacteristicsTO;
 import com.cloud.agent.api.to.VolumeTO;
 import com.cloud.agent.manager.AgentManager;
 import com.cloud.alert.AlertManager;
@@ -83,6 +82,7 @@ import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
 import com.cloud.dc.dao.DataCenterDao;
 import com.cloud.dc.dao.HostPodDao;
+import com.cloud.deploy.DeployDestination;
 import com.cloud.domain.dao.DomainDao;
 import com.cloud.event.EventState;
 import com.cloud.event.EventTypes;
@@ -112,8 +112,9 @@ import com.cloud.service.ServiceOfferingVO;
 import com.cloud.service.dao.ServiceOfferingDao;
 import com.cloud.storage.Storage.ImageFormat;
 import com.cloud.storage.Storage.StoragePoolType;
+import com.cloud.storage.Storage.StorageResourceType;
 import com.cloud.storage.Volume.MirrorState;
-import com.cloud.storage.Volume.StorageResourceType;
+import com.cloud.storage.Volume.SourceType;
 import com.cloud.storage.Volume.VolumeType;
 import com.cloud.storage.allocator.StoragePoolAllocator;
 import com.cloud.storage.dao.DiskOfferingDao;
@@ -135,6 +136,7 @@ import com.cloud.user.AccountVO;
 import com.cloud.user.User;
 import com.cloud.user.UserContext;
 import com.cloud.user.dao.AccountDao;
+import com.cloud.user.dao.UserDao;
 import com.cloud.uservm.UserVm;
 import com.cloud.utils.NumbersUtil;
 import com.cloud.utils.Pair;
@@ -144,16 +146,19 @@ import com.cloud.utils.component.Inject;
 import com.cloud.utils.concurrency.NamedThreadFactory;
 import com.cloud.utils.db.DB;
 import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.db.JoinBuilder;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
 import com.cloud.utils.db.Transaction;
 import com.cloud.utils.exception.CloudRuntimeException;
 import com.cloud.utils.exception.ExecutionException;
+import com.cloud.vm.DiskProfile;
 import com.cloud.vm.State;
 import com.cloud.vm.UserVmManager;
 import com.cloud.vm.UserVmVO;
 import com.cloud.vm.VMInstanceVO;
 import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VirtualMachineProfile;
 import com.cloud.vm.dao.ConsoleProxyDao;
 import com.cloud.vm.dao.UserVmDao;
 import com.cloud.vm.dao.VMInstanceDao;
@@ -179,7 +184,9 @@ public class StorageManagerImpl implements StorageManager {
     @Inject protected ConsoleProxyDao _consoleProxyDao;
     @Inject protected DetailsDao _detailsDao;
     @Inject protected SnapshotDao _snapshotDao;
+    @Inject(adapter=StoragePoolAllocator.class)
     protected Adapters _storagePoolAllocators;
+    @Inject(adapter=StoragePoolDiscoverer.class)
     protected Adapters _discoverers;
     @Inject protected StoragePoolHostDao _storagePoolHostDao;
     @Inject protected AlertManager _alertMgr;
@@ -200,6 +207,7 @@ public class StorageManagerImpl implements StorageManager {
     @Inject protected VMTemplateHostDao _templateHostDao;
     @Inject protected ServiceOfferingDao _offeringDao;
     @Inject protected DomainDao _domainDao;
+    @Inject protected UserDao _userDao;
     
     protected SearchBuilder HostTemplateStatesSearch;
     protected SearchBuilder PoolsUsedByVmSearch;
@@ -239,6 +247,127 @@ public class StorageManagerImpl implements StorageManager {
         return true;
     }
     
+    @DB
+    public List allocate(DiskProfile rootDisk, List dataDisks, VMInstanceVO vm, DataCenterVO dc, AccountVO account) {
+        ArrayList vols = new ArrayList(dataDisks.size() + 1);
+        VolumeVO dataVol = null;
+        VolumeVO rootVol = null;
+        long deviceId = 0;
+        Transaction txn = Transaction.currentTxn();
+        txn.start();
+        rootVol = new VolumeVO(VolumeType.ROOT, rootDisk.getName(), dc.getId(), account.getDomainId(), account.getId(), rootDisk.getDiskOfferingId(), rootDisk.getSize());
+        if (rootDisk.getTemplateId() != null) {
+            rootVol.setTemplateId(rootDisk.getTemplateId());
+        }
+        rootVol.setInstanceId(vm.getId());
+        rootVol.setDeviceId(deviceId++);
+        rootVol = _volsDao.persist(rootVol);
+        vols.add(rootVol);
+        for (DiskProfile dataDisk : dataDisks) {
+            dataVol = new VolumeVO(VolumeType.DATADISK, dataDisk.getName(), dc.getId(), account.getDomainId(), account.getId(), dataDisk.getDiskOfferingId(), dataDisk.getSize());
+            dataVol.setDeviceId(deviceId++);
+            dataVol.setInstanceId(vm.getId());
+            dataVol = _volsDao.persist(dataVol);
+            vols.add(dataVol);
+        }
+        txn.commit();
+        
+        return vols;
+    }
+    
+    @Override
+    public VolumeVO allocateIsoInstalledVm(VMInstanceVO vm, VMTemplateVO template, DiskOfferingVO rootOffering, Long size, DataCenterVO dc, AccountVO account) {
+        assert (template.getFormat() == ImageFormat.ISO) : "The template has to be ISO";
+        
+        long rootId = _volsDao.getNextInSequence(Long.class, "volume_seq");
+        DiskProfile rootDisk = new DiskProfile(rootId, VolumeType.ROOT, "ROOT-" + vm.getId() + "-" + rootId, rootOffering.getId(), size != null ? size : rootOffering.getDiskSizeInBytes(), rootOffering.getTagsArray(), rootOffering.getUseLocalStorage(), rootOffering.isRecreatable(), null);
+        List vols = allocate(rootDisk, null, vm, dc, account);
+        return vols.get(0);
+    }
+    
+    @Override
+    public VolumeTO[] prepare(VirtualMachineProfile vm, DeployDestination dest) {
+        List vols = _volsDao.findCreatedByInstance(vm.getId());
+        List recreateVols = new ArrayList(vols.size());
+        Host host = dest.getHost();
+        
+        VolumeTO[] disks = new VolumeTO[vols.size()];
+        int i = 0;
+        Iterator it = vols.iterator();
+        while (it.hasNext()) {
+            VolumeVO vol = it.next();
+            if (vol.isRecreatable()) {
+                it.remove();
+                //if we have a system vm
+                //get the storage pool
+                //if pool is in maintenance
+                //add to recreate vols, and continue
+                if(vm.getType().equals(VirtualMachine.Type.ConsoleProxy) || vm.getType().equals(VirtualMachine.Type.DomainRouter) || vm.getType().equals(VirtualMachine.Type.SecondaryStorageVm))
+                {
+                    StoragePoolVO sp = _storagePoolDao.findById(vol.getPoolId());
+                    
+                    if(sp!=null && sp.getStatus().equals(Status.PrepareForMaintenance))
+                    {
+                        recreateVols.add(vol);
+                        continue;
+                    }
+                }
+                
+                StoragePoolHostVO ph = _storagePoolHostDao.findByPoolHost(vol.getPoolId(), host.getId());
+                if (ph == null) {
+                    if (s_logger.isDebugEnabled()) {
+                        s_logger.debug("Must recreate " + vol + " since " + vol.getPoolId() + " has is not hooked up with host " + host.getId());
+                    }
+                    recreateVols.add(vol);
+                }
+            }
+        }
+        
+        for (VolumeVO vol : recreateVols) {
+            VolumeVO create = allocateDuplicateVolume(vol);
+            vols.add(vol);
+        }
+        
+        /*
+            create.setDiskOfferingId(vol.getDiskOfferingId());
+            create.setDeviceId(vol.getDeviceId());
+            create = _volsDao.persist(create);
+            VMTemplateVO template = _templateDao.findById(create.getTemplateId());
+            DataCenterVO dc = _dcDao.findById(create.getDataCenterId());
+            HostPodVO pod = _podDao.findById(host.getPodId());
+            DiskOfferingVO diskOffering = null;
+            diskOffering = _diskOfferingDao.findById(vol.getDiskOfferingId());
+            ServiceOfferingVO offering;
+            if (vm instanceof UserVmVO) {
+                offering = _offeringDao.findById(((UserVmVO)vm).getServiceOfferingId());
+            } else {
+                offering = _offeringDao.findById(vol.getDiskOfferingId());
+            }
+            VolumeVO created = createVolume(create, vm, template, dc, pod, host.getClusterId(), offering, diskOffering, new ArrayList(),0);
+            if (created == null) {
+                break;
+            }
+            createds.add(created);
+
+            for (VolumeVO vol : recreateVols) {
+                _volsDao.remove(vol.getId());
+            }
+            disks[i++] = null;
+        }
+        */
+        return disks;
+    }
+    
+    VolumeVO allocateDuplicateVolume(VolumeVO oldVol) {
+        VolumeVO newVol = new VolumeVO(oldVol.getVolumeType(), oldVol.getName(), oldVol.getDataCenterId(), oldVol.getDomainId(), oldVol.getAccountId(), oldVol.getDiskOfferingId(), oldVol.getSize());
+        newVol.setTemplateId(oldVol.getTemplateId());
+        newVol.setDeviceId(oldVol.getDeviceId());
+        newVol.setInstanceId(oldVol.getInstanceId());
+        return _volsDao.persist(newVol);
+    }
+    
+    
+    @Override
     public List prepare(VMInstanceVO vm, HostVO host) {
         List vols = _volsDao.findCreatedByInstance(vm.getId());
         List recreateVols = new ArrayList(vols.size());
@@ -247,6 +376,21 @@ public class StorageManagerImpl implements StorageManager {
                 return vols;
             }
             
+            //if we have a system vm
+            //get the storage pool
+            //if pool is in maintenance
+            //add to recreate vols, and continue
+            if(vm.getType().equals(VirtualMachine.Type.ConsoleProxy) || vm.getType().equals(VirtualMachine.Type.DomainRouter) || vm.getType().equals(VirtualMachine.Type.SecondaryStorageVm))
+            {
+            	StoragePoolVO sp = _storagePoolDao.findById(vol.getPoolId());
+            	
+            	if(sp!=null && sp.getStatus().equals(Status.PrepareForMaintenance))
+            	{
+            		recreateVols.add(vol);
+            		continue;
+            	}
+            }
+            
             StoragePoolHostVO ph = _storagePoolHostDao.findByPoolHost(vol.getPoolId(), host.getId());
             if (ph == null) {
                 if (s_logger.isDebugEnabled()) {
@@ -271,9 +415,7 @@ public class StorageManagerImpl implements StorageManager {
             DataCenterVO dc = _dcDao.findById(create.getDataCenterId());
             HostPodVO pod = _podDao.findById(host.getPodId());
             DiskOfferingVO diskOffering = null;
-            if (vol.getDiskOfferingId() != null) {
-                diskOffering = _diskOfferingDao.findById(vol.getDiskOfferingId());
-            }
+            diskOffering = _diskOfferingDao.findById(vol.getDiskOfferingId());
             ServiceOfferingVO offering;
             if (vm instanceof UserVmVO) {
                 offering = _offeringDao.findById(((UserVmVO)vm).getServiceOfferingId());
@@ -319,7 +461,7 @@ public class StorageManagerImpl implements StorageManager {
     			
     			SearchBuilder activeVmSB = _vmInstanceDao.createSearchBuilder();
 	        	activeVmSB.and("state", activeVmSB.entity().getState(), SearchCriteria.Op.IN);
-	        	volumeSB.join("activeVmSB", activeVmSB, volumeSB.entity().getInstanceId(), activeVmSB.entity().getId());
+	        	volumeSB.join("activeVmSB", activeVmSB, volumeSB.entity().getInstanceId(), activeVmSB.entity().getId(), JoinBuilder.JoinType.INNER);
     			
 	        	SearchCriteria volumeSC = volumeSB.create();
 	        	volumeSC.setParameters("poolId", storagePool.getId());
@@ -345,11 +487,11 @@ public class StorageManagerImpl implements StorageManager {
         return unshare(vm, vols, host) ? vols : null;
     }
 
-    protected StoragePoolVO findStoragePool(DiskCharacteristicsTO dskCh, final DataCenterVO dc, HostPodVO pod, Long clusterId, final ServiceOffering offering, final VMInstanceVO vm, final VMTemplateVO template, final Set avoid) {
+    protected StoragePoolVO findStoragePool(DiskProfile dskCh, final DataCenterVO dc, HostPodVO pod, Long clusterId, final ServiceOffering offering, final VMInstanceVO vm, final VMTemplateVO template, final Set avoid) {
         Enumeration en = _storagePoolAllocators.enumeration();
         while (en.hasMoreElements()) {
             final StoragePoolAllocator allocator = en.nextElement();
-            final StoragePool pool = allocator.allocateToPool(dskCh, offering, dc, pod, clusterId, vm, template, avoid);
+            final StoragePool pool = allocator.allocateToPool(dskCh, dc, pod, clusterId, vm, template, avoid);
             if (pool != null) {
                 return (StoragePoolVO) pool;
             }
@@ -357,7 +499,8 @@ public class StorageManagerImpl implements StorageManager {
         return null;
     }
 
-    protected Long findHostIdForStoragePool(StoragePoolVO pool) {
+    @Override
+    public Long findHostIdForStoragePool(StoragePoolVO pool) {
         List poolHosts = _poolHostDao.listByHostStatus(pool.getId(), Status.Up);
 
         if (poolHosts.size() == 0) {
@@ -374,7 +517,9 @@ public class StorageManagerImpl implements StorageManager {
         
         for (StoragePoolHostVO poolHost: poolHosts) {
             try {
-                return _agentMgr.send(poolHost.getHostId(), cmds, stopOnError);
+                Answer[] answerRet = _agentMgr.send(poolHost.getHostId(), cmds, stopOnError);
+                return answerRet;
+                
             } catch (AgentUnavailableException e) {
                 s_logger.debug("Moving on because unable to send to " + poolHost.getHostId() + " due to " + e.getMessage());
             } catch (OperationTimedoutException e) {
@@ -397,22 +542,22 @@ public class StorageManagerImpl implements StorageManager {
         return answers[0];
     }
     
-    private DiskCharacteristicsTO createDiskCharacteristics(VolumeVO volume, VMTemplateVO template, long dataCenterId, DiskOfferingVO diskOffering) {
+    protected DiskProfile createDiskCharacteristics(VolumeVO volume, VMTemplateVO template, DataCenterVO dc, DiskOfferingVO diskOffering) {
         if (volume.getVolumeType() == VolumeType.ROOT && Storage.ImageFormat.ISO != template.getFormat()) {
             SearchCriteria sc = HostTemplateStatesSearch.create();
             sc.setParameters("id", template.getId());
             sc.setParameters("state", com.cloud.storage.VMTemplateStorageResourceAssoc.Status.DOWNLOADED);
-            sc.setJoinParameters("host", "dcId", dataCenterId);
+            sc.setJoinParameters("host", "dcId", dc.getId());
         
             List sss = _vmTemplateHostDao.search(sc, null);
             if (sss.size() == 0) {
-                throw new CloudRuntimeException("Template " + template.getName() + " has not been completely downloaded to zone " + dataCenterId);
+                throw new CloudRuntimeException("Template " + template.getName() + " has not been completely downloaded to zone " + dc.getId());
             }
             VMTemplateHostVO ss = sss.get(0);
         
-            return new DiskCharacteristicsTO(volume.getVolumeType(), volume.getName(), diskOffering, ss.getSize());
+            return new DiskProfile(volume.getId(), volume.getVolumeType(), volume.getName(), diskOffering.getId(), ss.getSize(), diskOffering.getTagsArray(), diskOffering.getUseLocalStorage(), diskOffering.isRecreatable(), Storage.ImageFormat.ISO != template.getFormat() ? template.getId() : null);
         } else {
-            return new DiskCharacteristicsTO(volume.getVolumeType(), volume.getName(), diskOffering);
+            return new DiskProfile(volume.getId(), volume.getVolumeType(), volume.getName(), diskOffering.getId(), diskOffering.getDiskSizeInBytes(), diskOffering.getTagsArray(), diskOffering.getUseLocalStorage(), diskOffering.isRecreatable(), null);
         }
     }
     
@@ -436,7 +581,7 @@ public class StorageManagerImpl implements StorageManager {
         
         // Create the Volume object and save it so that we can return it to the user
         Account account = _accountDao.findById(volume.getAccountId());
-        
+
         final HashSet poolsToAvoid = new HashSet();
         StoragePoolVO pool = null;
         boolean success = false;
@@ -444,9 +589,10 @@ public class StorageManagerImpl implements StorageManager {
         Pair pod = null;
         String volumeUUID = null;
         String details = null;
+        
         DiskOfferingVO diskOffering = _diskOfferingDao.findById(volume.getDiskOfferingId());
         DataCenterVO dc = _dcDao.findById(volume.getDataCenterId());
-        DiskCharacteristicsTO dskCh = createDiskCharacteristics(volume, template, dc.getId(), diskOffering);
+        DiskProfile dskCh = createDiskCharacteristics(volume, template, dc, diskOffering);
 
         // Determine what pod to store the volume in
         while ((pod = _agentMgr.findPod(null, null, dc, account.getId(), podsToAvoid)) != null) {
@@ -496,7 +642,7 @@ public class StorageManagerImpl implements StorageManager {
             createdVolume.setPoolType(pool.getPoolType());
             createdVolume.setFolder(volumeFolder);
             createdVolume.setPath(volumeUUID);
-            createdVolume.setDomainId(account.getDomainId().longValue());
+            createdVolume.setDomainId(account.getDomainId());
         } else {
             createdVolume.setStatus(AsyncInstanceCreateStatus.Corrupted);
             createdVolume.setDestroyed(true);
@@ -597,9 +743,7 @@ public class StorageManagerImpl implements StorageManager {
         if(originalVolume.getTemplateId() != null){
             templateId = originalVolume.getTemplateId();
         }
-        if(originalVolume.getDiskOfferingId() != null){
-            diskOfferingId = originalVolume.getDiskOfferingId();
-        }
+        diskOfferingId = originalVolume.getDiskOfferingId();
         long sizeMB = createdVolume.getSize()/(1024*1024);
 
         String poolName = _storagePoolDao.findById(createdVolume.getPoolId()).getName();
@@ -669,12 +813,12 @@ public class StorageManagerImpl implements StorageManager {
                                     ServiceOfferingVO offering, DiskOfferingVO diskOffering, List avoids, long size) {
         StoragePoolVO pool = null;
         final HashSet avoidPools = new HashSet(avoids);
-
-        DiskCharacteristicsTO dskCh = null;
+       
+        DiskProfile dskCh = null;
         if (volume.getVolumeType() == VolumeType.ROOT && Storage.ImageFormat.ISO != template.getFormat()) {
-            dskCh = createDiskCharacteristics(volume, template, dc.getId(), offering);
+            dskCh = createDiskCharacteristics(volume, template, dc, offering);
         } else {
-            dskCh = createDiskCharacteristics(volume, template, dc.getId(), diskOffering);
+            dskCh = createDiskCharacteristics(volume, template, dc, diskOffering);
         }
 
         Transaction txn = Transaction.currentTxn();
@@ -774,18 +918,24 @@ public class StorageManagerImpl implements StorageManager {
         if (Storage.ImageFormat.ISO == template.getFormat()) {
             rootVol = new VolumeVO(VolumeType.ROOT, vm.getId(), vm.getInstanceName() + "-ROOT", dc.getId(), pod.getId(), account.getId(), account.getDomainId(),(size>0)? size : diskOffering.getDiskSizeInBytes());
             rootVol.setDiskOfferingId(diskOffering.getId());
+            rootVol.setSourceType(SourceType.Template);
+            rootVol.setSourceId(template.getId());
             rootVol.setDeviceId(0l);
             rootVol = _volsDao.persist(rootVol);
         } else {
             rootVol = new VolumeVO(VolumeType.ROOT, vm.getId(), template.getId(), vm.getInstanceName() + "-ROOT", dc.getId(), pod.getId(), account.getId(), account.getDomainId(), offering.isRecreatable());
             rootVol.setDiskOfferingId(offering.getId());
             rootVol.setTemplateId(template.getId());
+            rootVol.setSourceId(template.getId());
+            rootVol.setSourceType(SourceType.Template);
             rootVol.setDeviceId(0l);
             rootVol = _volsDao.persist(rootVol);
             
             if (diskOffering != null && diskOffering.getDiskSizeInBytes() > 0) {
                 dataVol = new VolumeVO(VolumeType.DATADISK, vm.getId(), vm.getInstanceName() + "-DATA", dc.getId(), pod.getId(), account.getId(), account.getDomainId(), (size>0)? size : diskOffering.getDiskSizeInBytes());
                 dataVol.setDiskOfferingId(diskOffering.getId());
+                dataVol.setSourceType(SourceType.DiskOffering);
+                dataVol.setSourceId(diskOffering.getId());
                 dataVol.setDeviceId(1l);
                 dataVol = _volsDao.persist(dataVol);
             }
@@ -804,7 +954,7 @@ public class StorageManagerImpl implements StorageManager {
             
             if (dataVol != null) {
                 StoragePoolVO pool = _storagePoolDao.findById(rootCreated.getPoolId());
-                dataCreated = createVolume(dataVol, vm, template, dc, pod, pool.getClusterId(), offering, diskOffering, avoids,size);
+                dataCreated = createVolume(dataVol, vm, null, dc, pod, pool.getClusterId(), offering, diskOffering, avoids,size);
                 if (dataCreated == null) {
                     throw new CloudRuntimeException("Unable to create " + dataVol);
                 }
@@ -813,8 +963,9 @@ public class StorageManagerImpl implements StorageManager {
             
             return vols;
         } catch (Exception e) {
-        	s_logger.error("Unexpected exception ", e);
-        	
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug(e.getMessage());
+            }
             if (rootCreated != null) {
                 destroyVolume(rootCreated);
             }
@@ -827,13 +978,31 @@ public class StorageManagerImpl implements StorageManager {
     public long createUserVM(Account account, VMInstanceVO vm, VMTemplateVO template, DataCenterVO dc, HostPodVO pod, ServiceOfferingVO offering, DiskOfferingVO diskOffering,
             List avoids, long size) {
         List volumes = create(account, vm, template, dc, pod, offering, diskOffering, avoids, size);
+        if( volumes == null || volumes.size() == 0) {
+            throw new CloudRuntimeException("Unable to create volume for " + vm.getName());
+        }
         
         for (VolumeVO v : volumes) {
+        	
+        	//when the user vm is created, the volume is attached upon creation
+        	//set the attached datetime
+        	try{
+        		v.setAttached(new Date());
+        		_volsDao.update(v.getId(), v);
+        	}catch(Exception e)
+        	{
+        		s_logger.warn("Error updating the attached value for volume "+v.getId()+":"+e);
+        	}
+        	
+            long templateId = -1;
+        	if(v.getVolumeType() == VolumeType.ROOT && Storage.ImageFormat.ISO != template.getFormat()){
+        	        templateId = template.getId();
+        	}
+        	
         	long volumeId = v.getId();
         	// Create an event
-        	long sizeMB = v.getSize() / (1024 * 1024);
-        	String diskOfferingIdentifier = (diskOffering != null) ? String.valueOf(diskOffering.getId()) : "-1";
-        	String eventParams = "id=" + volumeId + "\ndoId=" + diskOfferingIdentifier + "\ntId=" + template.getId() + "\ndcId=" + dc.getId() + "\nsize=" + sizeMB;
+            long sizeMB = v.getSize() / (1024 * 1024);
+        	String eventParams = "id=" + volumeId + "\ndoId=" + v.getDiskOfferingId() + "\ntId=" + templateId + "\ndcId=" + dc.getId() + "\nsize=" + sizeMB;
         	EventVO event = new EventVO();
         	event.setAccountId(account.getId());
         	event.setUserId(1L);
@@ -851,10 +1020,12 @@ public class StorageManagerImpl implements StorageManager {
     		if (vmId != null) {
     			VMInstanceVO vmInstance = _vmInstanceDao.findById(vmId);
     			if (vmInstance != null) {
-    				return vmInstance.getHostId();
+    				Long hostId = vmInstance.getHostId();
+    				if (hostId != null && !avoidHosts.contains(vmInstance.getHostId()))
+    					return hostId;
     			}
     		}
-    		return null;
+    		/*Can't find the vm where host resides on(vm is destroyed? or volume is detached from vm), randomly choose a host to send the cmd */
     	}
         List poolHosts = _poolHostDao.listByHostStatus(poolVO.getId(), Status.Up);
         Collections.shuffle(poolHosts);
@@ -892,6 +1063,7 @@ public class StorageManagerImpl implements StorageManager {
         return true;
     }
 
+    @Override
     public void destroy(VMInstanceVO vm, List vols) {
         if (s_logger.isDebugEnabled() && vm != null) {
             s_logger.debug("Destroying volumes of " + vm.toString());
@@ -948,20 +1120,14 @@ public class StorageManagerImpl implements StorageManager {
             return false;
         }
 
-        _storagePoolAllocators = locator.getAdapters(StoragePoolAllocator.class);
-        if (!_storagePoolAllocators.isSet()) {
-            throw new ConfigurationException("Unable to get any storage pool allocators.");
-        }
+        Map configs = configDao.getConfiguration("management-server", params);
         
-        _discoverers = locator.getAdapters(StoragePoolDiscoverer.class);
 
-        String overProvisioningFactorStr = (String) params.get("storage.overprovisioning.factor");
+        String overProvisioningFactorStr = configs.get("storage.overprovisioning.factor");
         if (overProvisioningFactorStr != null) {
             _overProvisioningFactor = Integer.parseInt(overProvisioningFactorStr);
         }
 
-        Map configs = configDao.getConfiguration("management-server", params);
-
         _retry = NumbersUtil.parseInt(configs.get(Config.StartRetry.key()), 2);
         _pingInterval = NumbersUtil.parseInt(configs.get("ping.interval"), 60);
         _hostRetry = NumbersUtil.parseInt(configs.get("host.retry"), 2);
@@ -974,6 +1140,8 @@ public class StorageManagerImpl implements StorageManager {
         String hypervisoType = configDao.getValue("hypervisor.type");
         if (hypervisoType.equalsIgnoreCase("KVM")) {
         	_hypervisorType = Hypervisor.Type.KVM;
+        } else if(hypervisoType.equalsIgnoreCase("vmware")) {
+        	_hypervisorType = Hypervisor.Type.VmWare;
         }
         _agentMgr.registerForHostEvents(new StoragePoolMonitor(this, _hostDao, _storagePoolDao), true, false, true);
 
@@ -994,7 +1162,7 @@ public class StorageManagerImpl implements StorageManager {
         
         PoolsUsedByVmSearch = _storagePoolDao.createSearchBuilder();
         SearchBuilder volSearch = _volsDao.createSearchBuilder();
-        PoolsUsedByVmSearch.join("volumes", volSearch, volSearch.entity().getPoolId(), PoolsUsedByVmSearch.entity().getId());
+        PoolsUsedByVmSearch.join("volumes", volSearch, volSearch.entity().getPoolId(), PoolsUsedByVmSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         volSearch.and("vm", volSearch.entity().getInstanceId(), SearchCriteria.Op.EQ);
         volSearch.done();
         PoolsUsedByVmSearch.done();
@@ -1006,7 +1174,7 @@ public class StorageManagerImpl implements StorageManager {
         SearchBuilder HostSearch = _hostDao.createSearchBuilder();
         HostSearch.and("dcId", HostSearch.entity().getDataCenterId(), SearchCriteria.Op.EQ);
         
-        HostTemplateStatesSearch.join("host", HostSearch, HostSearch.entity().getId(), HostTemplateStatesSearch.entity().getHostId());
+        HostTemplateStatesSearch.join("host", HostSearch, HostSearch.entity().getId(), HostTemplateStatesSearch.entity().getHostId(), JoinBuilder.JoinType.INNER);
         HostSearch.done();
         HostTemplateStatesSearch.done();
         
@@ -1024,6 +1192,7 @@ public class StorageManagerImpl implements StorageManager {
         return UUID.randomUUID().toString();
     }
 
+    @Override
     public boolean volumeOnSharedStoragePool(VolumeVO volume) {
         Long poolId = volume.getPoolId();
         if (poolId == null) {
@@ -1039,6 +1208,7 @@ public class StorageManagerImpl implements StorageManager {
         }
     }
 
+    @Override
     public boolean volumeInactive(VolumeVO volume) {
         Long vmId = volume.getInstanceId();
         if (vmId != null) {
@@ -1056,6 +1226,7 @@ public class StorageManagerImpl implements StorageManager {
         return true;
     }
     
+    @Override
     public String getVmNameOnVolume(VolumeVO volume) {
     	 Long vmId = volume.getInstanceId();
          if (vmId != null) {
@@ -1069,6 +1240,7 @@ public class StorageManagerImpl implements StorageManager {
          return null;
     }
 
+    @Override
     public String getAbsoluteIsoPath(long templateId, long dataCenterId) {
         String isoPath = null;
 
@@ -1086,6 +1258,7 @@ public class StorageManagerImpl implements StorageManager {
         return isoPath;
     }
 
+    @Override
     public String getSecondaryStorageURL(long zoneId) {
         // Determine the secondary storage URL
         HostVO secondaryStorageHost = _hostDao.findSecondaryStorageHost(zoneId);
@@ -1097,6 +1270,7 @@ public class StorageManagerImpl implements StorageManager {
         return secondaryStorageHost.getStorageUrl();
     }
 
+    @Override
     public HostVO getSecondaryStorageHost(long zoneId) {
         return _hostDao.findSecondaryStorageHost(zoneId);
     }
@@ -1211,6 +1385,8 @@ public class StorageManagerImpl implements StorageManager {
         if (hypervisorType == null) {
         	if (_hypervisorType == Hypervisor.Type.KVM) {
         		hypervisorType = Hypervisor.Type.KVM;
+        	} else if(_hypervisorType == Hypervisor.Type.VmWare) {
+        		hypervisorType = Hypervisor.Type.VmWare;
         	} else {
         	    if (s_logger.isDebugEnabled()) {
                     s_logger.debug("Couldn't find a host to serve in the server pool");
@@ -1299,10 +1475,20 @@ public class StorageManagerImpl implements StorageManager {
             throw new ResourceAllocationException("No host exists to associate a storage pool with");
         }
         long poolId = _storagePoolDao.getNextInSequence(Long.class, "id");
-        String uuid = UUID.nameUUIDFromBytes(new String(storageHost + hostPath + System.currentTimeMillis()).getBytes()).toString();
-        if (s_logger.isDebugEnabled()) {
-            s_logger.debug("In createPool Setting poolId - " +poolId+ " uuid - " +uuid+ " zoneId - " +zoneId+ " podId - " +podId+ " poolName - " +cmd.getStoragePoolName());
+        String uuid = UUID.nameUUIDFromBytes(new String(storageHost + hostPath).getBytes()).toString();
+        
+        List spHandles = _storagePoolDao.findIfDuplicatePoolsExistByUUID(uuid);
+        if ((spHandles != null) && (spHandles.size() > 0)) {
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("Another active pool with the same uuid already exists");
+            }
+        	throw new ResourceInUseException("Another active pool with the same uuid already exists");
         }
+
+        if (s_logger.isDebugEnabled()) {
+            s_logger.debug("In createPool Setting poolId - " +poolId+ " uuid - " +uuid+ " zoneId - " +zoneId+ " podId - " +podId+ " poolName - " + cmd.getStoragePoolName());
+        }
+
         pool.setId(poolId);
         pool.setUuid(uuid);
         pool.setDataCenterId(cmd.getZoneId());
@@ -1324,7 +1510,7 @@ public class StorageManagerImpl implements StorageManager {
         }
 
         if (poolHosts.isEmpty()) {
-            _storagePoolDao.delete(pool.getId());
+            _storagePoolDao.expunge(pool.getId());
             pool = null;
         } else {
             createCapacityEntry(pool);
@@ -1360,6 +1546,7 @@ public class StorageManagerImpl implements StorageManager {
     	return pool;
     }
 
+    @Override
     @DB
     public boolean deletePool(DeletePoolCmd command) throws InvalidParameterValueException{
     	Long id = command.getId();
@@ -1475,10 +1662,11 @@ public class StorageManagerImpl implements StorageManager {
         return false;
     }
 
+    @Override
     public VolumeVO moveVolume(VolumeVO volume, long destPoolDcId, Long destPoolPodId, Long destPoolClusterId) throws InternalErrorException {
     	// Find a destination storage pool with the specified criteria
     	DiskOfferingVO diskOffering = _diskOfferingDao.findById(volume.getDiskOfferingId());
-    	DiskCharacteristicsTO dskCh = new DiskCharacteristicsTO(volume.getVolumeType(), volume.getName(), diskOffering);
+    	DiskProfile dskCh = new DiskProfile(volume.getId(), volume.getVolumeType(), volume.getName(), diskOffering.getId(), diskOffering.getDiskSizeInBytes(), diskOffering.getTagsArray(), diskOffering.getUseLocalStorage(), diskOffering.isRecreatable(), null);
     	DataCenterVO destPoolDataCenter = _dcDao.findById(destPoolDcId);
     	HostPodVO destPoolPod = _podDao.findById(destPoolPodId);
         StoragePoolVO destPool = findStoragePool(dskCh, destPoolDataCenter, destPoolPod, destPoolClusterId, null, null, null, new HashSet());
@@ -1616,7 +1804,7 @@ public class StorageManagerImpl implements StorageManager {
                     if (!_domainDao.isChildDomain(account.getDomainId(), snapshotOwner.getDomainId())) {
                         throw new ServerApiException(BaseCmd.ACCOUNT_ERROR, "Unable to create volume from snapshot with id " + snapshotId + ", permission denied.");
                     }
-                } else if (account.getId().longValue() != snapshotCheck.getAccountId()) {
+                } else if (account.getId() != snapshotCheck.getAccountId()) {
                     throw new ServerApiException(BaseCmd.SNAPSHOT_INVALID_PARAM_ERROR, "unable to find a snapshot with id " + snapshotId + " for this account");
                 }
             }
@@ -1653,19 +1841,19 @@ public class StorageManagerImpl implements StorageManager {
             userSpecifiedName = getRandomVolumeName();
         }
 
-        VolumeVO volume = new VolumeVO(null, userSpecifiedName, -1, -1, -1, -1, new Long(-1), null, null, 0, Volume.VolumeType.DATADISK);
+        VolumeVO volume = new VolumeVO(userSpecifiedName, -1, -1, -1, -1, new Long(-1), null, null, 0, Volume.VolumeType.DATADISK);
         volume.setPoolId(null);
         volume.setDataCenterId(zoneId);
         volume.setPodId(null);
         volume.setAccountId(targetAccount.getId());
-        volume.setDomainId(account.getDomainId().longValue());
+        volume.setDomainId(account.getDomainId());
         volume.setMirrorState(MirrorState.NOT_MIRRORED);
         volume.setDiskOfferingId(cmd.getDiskOfferingId());
         volume.setStorageResourceType(StorageResourceType.STORAGE_POOL);
         volume.setInstanceId(null);
         volume.setUpdated(new Date());
         volume.setStatus(AsyncInstanceCreateStatus.Creating);
-        volume.setDomainId(account.getDomainId().longValue());
+        volume.setDomainId(account.getDomainId());
         volume = _volsDao.persist(volume);
 
         return volume;
@@ -1764,9 +1952,14 @@ public class StorageManagerImpl implements StorageManager {
                 
         txn.commit();
     }
-
+    
     @Override
     public void createCapacityEntry(StoragePoolVO storagePool) {
+    	createCapacityEntry(storagePool, 0);
+    }
+
+    @Override
+    public void createCapacityEntry(StoragePoolVO storagePool, long allocated) {
         SearchCriteria capacitySC = _capacityDao.createSearchCriteria();
         capacitySC.addAnd("hostOrPoolId", SearchCriteria.Op.EQ, storagePool.getId());
         capacitySC.addAnd("dataCenterId", SearchCriteria.Op.EQ, storagePool.getDataCenterId());
@@ -1775,15 +1968,18 @@ public class StorageManagerImpl implements StorageManager {
         List capacities = _capacityDao.search(capacitySC, null);
 
         if (capacities.size() == 0) {
-            CapacityVO capacity = new CapacityVO(storagePool.getId(), storagePool.getDataCenterId(), storagePool.getPodId(), 0L, storagePool.getCapacityBytes(),
+            CapacityVO capacity = new CapacityVO(storagePool.getId(), storagePool.getDataCenterId(), storagePool.getPodId(), storagePool.getAvailableBytes(), storagePool.getCapacityBytes(),
                     CapacityVO.CAPACITY_TYPE_STORAGE);
             _capacityDao.persist(capacity);
         } else {
             CapacityVO capacity = capacities.get(0);
-            if (capacity.getTotalCapacity() != storagePool.getCapacityBytes()) {
-                capacity.setTotalCapacity(storagePool.getCapacityBytes());
-                _capacityDao.update(capacity.getId(), capacity);
+            capacity.setTotalCapacity(storagePool.getCapacityBytes());
+            long used = storagePool.getCapacityBytes() - storagePool.getAvailableBytes();
+            if( used <= 0 ) {
+            	used = 0;
             }
+            capacity.setUsedCapacity(used);
+            _capacityDao.update(capacity.getId(), capacity);
         }
         s_logger.debug("Successfully set Capacity - " +storagePool.getCapacityBytes()+ " for CAPACITY_TYPE_STORAGE, DataCenterId - " +storagePool.getDataCenterId()+ ", HostOrPoolId - " +storagePool.getId()+ ", PodId " +storagePool.getPodId());
         capacitySC = _capacityDao.createSearchCriteria();
@@ -1793,21 +1989,28 @@ public class StorageManagerImpl implements StorageManager {
 
         capacities = _capacityDao.search(capacitySC, null);
 
+        int provFactor = 1;
+        if( storagePool.getPoolType() == StoragePoolType.NetworkFilesystem ) {
+            provFactor = _overProvisioningFactor;
+        }
         if (capacities.size() == 0) {
-            int provFactor = 1;
-            if( storagePool.getPoolType() == StoragePoolType.NetworkFilesystem ) {
-                provFactor = _overProvisioningFactor;
-            }
-
-            CapacityVO capacity = new CapacityVO(storagePool.getId(), storagePool.getDataCenterId(), storagePool.getPodId(), 0L, storagePool.getCapacityBytes()
+            CapacityVO capacity = new CapacityVO(storagePool.getId(), storagePool.getDataCenterId(), storagePool.getPodId(), allocated, storagePool.getCapacityBytes()
                     * provFactor, CapacityVO.CAPACITY_TYPE_STORAGE_ALLOCATED);
             _capacityDao.persist(capacity);
         } else {
             CapacityVO capacity = capacities.get(0);
-            long currCapacity = _overProvisioningFactor * storagePool.getCapacityBytes();
+            long currCapacity = provFactor * storagePool.getCapacityBytes();
+        	boolean update = false;
             if (capacity.getTotalCapacity() != currCapacity) {
                 capacity.setTotalCapacity(currCapacity);
-                _capacityDao.update(capacity.getId(), capacity);
+                update = true;
+            }
+            if ( allocated != 0 ) {
+                capacity.setUsedCapacity(allocated);
+                update = true;
+            }
+            if ( update ) {
+            	_capacityDao.update(capacity.getId(), capacity);
             }
         }
         s_logger.debug("Successfully set Capacity - " +storagePool.getCapacityBytes()* _overProvisioningFactor+ " for CAPACITY_TYPE_STORAGE_ALLOCATED, DataCenterId - " +storagePool.getDataCenterId()+ ", HostOrPoolId - " +storagePool.getId()+ ", PodId " +storagePool.getPodId());
@@ -1852,7 +2055,8 @@ public class StorageManagerImpl implements StorageManager {
                     }
                 }
                 s_logger.debug("Trying to execute Command: " + cmd + " on host: " + hostId + " try: " + tryCount);
-                answer = _agentMgr.send(hostId, cmd);
+                // set 120 min timeout for storage related command
+                answer = _agentMgr.send(hostId, cmd, 120*60*1000);
                 
                 if (answer != null && answer.getResult()) {
                     return answer;
@@ -1878,6 +2082,7 @@ public class StorageManagerImpl implements StorageManager {
         public StorageGarbageCollector() {
         }
 
+        @Override
         public void run() {
             try {
                 s_logger.info("Storage Garbage Collection Thread is running.");
@@ -1901,10 +2106,11 @@ public class StorageManagerImpl implements StorageManager {
         }
     }
 
+    @Override
     public void cleanupStorage(boolean recurring) {
 
         // Cleanup primary storage pools
-        List storagePools = _storagePoolDao.listAllActive();
+        List storagePools = _storagePoolDao.listAll();
         for (StoragePoolVO pool : storagePools) {
             try {
                 if (recurring && pool.isLocal()) {
@@ -1987,6 +2193,7 @@ public class StorageManagerImpl implements StorageManager {
 
     }
     
+    @Override
     public List getStoragePoolsForVm(long vmId) {
         SearchCriteria sc = PoolsUsedByVmSearch.create();
         sc.setJoinParameters("volumes", "vm", vmId);
@@ -1994,6 +2201,7 @@ public class StorageManagerImpl implements StorageManager {
         return _storagePoolDao.search(sc, null);
     }
     
+    @Override
     public String getPrimaryStorageNameLabel(VolumeVO volume) {
         Long poolId     = volume.getPoolId();
         
@@ -2004,16 +2212,11 @@ public class StorageManagerImpl implements StorageManager {
         return storagePoolVO.getUuid();
     }
     
-    @Override
-    @DB
+    @Override @DB
     public boolean preparePrimaryStorageForMaintenance(PreparePrimaryStorageForMaintenanceCmd cmd) {
     	Long primaryStorageId = cmd.getId();
     	Long userId = UserContext.current().getUserId();
-    	
-        boolean destroyVolumes = false;
-        long count = 1;
-        long consoleProxyId = 0;
-        long ssvmId = 0;
+        boolean restart = true;
         try 
         {
         	//1. Get the primary storage record and perform validation check
@@ -2030,34 +2233,31 @@ public class StorageManagerImpl implements StorageManager {
         	}
         	
         	//check to see if other ps exist
-        	//if they do, then we can migrate over the system vms to them, destroy volumes for sys vms
-        	//if they dont, then do NOT destroy the volumes on this one
-        	count = _storagePoolDao.countBy(primaryStorage.getId(), Status.Up);
-        	if(count>1)
-        	{
-        		destroyVolumes = true;
-        	}
+        	//if they do, then we can migrate over the system vms to them
+        	//if they dont, then just stop all vms on this one
+        	List upPools = _storagePoolDao.listPoolsByStatus(Status.Up);
         	
+        	if(upPools==null || upPools.size()==0)
+        		restart = false;
+        		
         	//2. Get a list of all the volumes within this storage pool
         	List allVolumes = _volsDao.findByPoolId(primaryStorageId);
-        	List markedVolumes = new ArrayList();
         	
         	//3. Each volume has an instance associated with it, stop the instance if running
         	for(VolumeVO volume : allVolumes)
         	{
         		VMInstanceVO vmInstance = _vmInstanceDao.findById(volume.getInstanceId());
         		
+        		if(vmInstance == null)
+        			continue;
+        		
         		//shut down the running vms
         		if(vmInstance.getState().equals(State.Running) || vmInstance.getState().equals(State.Stopped) || vmInstance.getState().equals(State.Stopping) || vmInstance.getState().equals(State.Starting))
         		{
         			
         			//if the instance is of type consoleproxy, call the console proxy
         			if(vmInstance.getType().equals(VirtualMachine.Type.ConsoleProxy))
-        			{
-        				//add this volume to be removed if flag=true
-        				if(destroyVolumes)
-        					markedVolumes.add(volume);
-        				
+        			{        				
         				//make sure it is not restarted again, update config to set flag to false
         				_configMgr.updateConfiguration(userId, "consoleproxy.restart", "false");
         				
@@ -2072,14 +2272,21 @@ public class StorageManagerImpl implements StorageManager {
                     		_storagePoolDao.persist(primaryStorage);
                     		return false;
             			}
-        				else
+        				else if(restart)
         				{
-        					if(destroyVolumes)
-        					{
-        						//proxy vm is stopped, and we have another ps available 
-        						//get the id for restart
-        						consoleProxyId = vmInstance.getId();        						
-        					}
+    						//create a dummy event
+    						long eventId1 = saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_PROXY_START, "starting console proxy with Id: "+vmInstance.getId());
+    						
+    						//Restore config val for consoleproxy.restart to true
+    						_configMgr.updateConfiguration(userId, "consoleproxy.restart", "true");
+    						
+    						if(_consoleProxyMgr.startProxy(vmInstance.getId(), eventId1)==null)
+    						{
+    							s_logger.warn("There was an error starting the console proxy id: "+vmInstance.getId()+" on another storage pool, cannot enable primary storage maintenance");
+    			            	primaryStorage.setStatus(Status.ErrorInMaintenance);
+    			        		_storagePoolDao.persist(primaryStorage);
+    							return false;				
+    						}	  						
         				}
         			}
         			
@@ -2101,10 +2308,6 @@ public class StorageManagerImpl implements StorageManager {
         			//if the instance is of type secondary storage vm, call the secondary storage vm manager
         			if(vmInstance.getType().equals(VirtualMachine.Type.SecondaryStorageVm))
         			{           				
-        				//add this volume to be removed if flag=true
-        				if(destroyVolumes)
-        					markedVolumes.add(volume);
-        				
         				//create a dummy event
         				long eventId1 = saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_SSVM_STOP, "stopping ssvm with Id: "+vmInstance.getId());
 
@@ -2115,25 +2318,23 @@ public class StorageManagerImpl implements StorageManager {
         	        		_storagePoolDao.persist(primaryStorage);
         					return false;
         				}
-        				else
+        				else if(restart)
         				{
-        					if(destroyVolumes)
-        					{
-        						//ss vm is stopped, and we have another ps available 				
-        						//get the id for restart
-        						ssvmId = vmInstance.getId();
-        					}
+    						//create a dummy event and restart the ssvm immediately
+    						long eventId = saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_SSVM_START, "starting ssvm with Id: "+vmInstance.getId());
+    						if(_secStorageMgr.startSecStorageVm(vmInstance.getId(), eventId)==null)
+    						{
+    							s_logger.warn("There was an error starting the ssvm id: "+vmInstance.getId()+" on another storage pool, cannot enable primary storage maintenance");
+    			            	primaryStorage.setStatus(Status.ErrorInMaintenance);
+    			        		_storagePoolDao.persist(primaryStorage);
+    							return false;
+    						}
         				}
-
         			}
 
            			//if the instance is of type domain router vm, call the network manager
         			if(vmInstance.getType().equals(VirtualMachine.Type.DomainRouter))
         			{   
-        				//add this volume to be removed if flag=true
-        				if(destroyVolumes)
-        					markedVolumes.add(volume);
-        				
         				//create a dummy event
         				long eventId2 = saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_ROUTER_STOP, "stopping domain router with Id: "+vmInstance.getId());
 
@@ -2144,45 +2345,23 @@ public class StorageManagerImpl implements StorageManager {
         	        		_storagePoolDao.persist(primaryStorage);
         					return false;
         				}
+           				else if(restart)
+        				{
+    						//create a dummy event and restart the domr immediately
+    						long eventId = saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_PROXY_START, "starting domr with Id: "+vmInstance.getId());
+    						if(_networkMgr.startRouter(vmInstance.getId(), eventId)==null)
+    						{
+    							s_logger.warn("There was an error starting the omr id: "+vmInstance.getId()+" on another storage pool, cannot enable primary storage maintenance");
+    			            	primaryStorage.setStatus(Status.ErrorInMaintenance);
+    			        		_storagePoolDao.persist(primaryStorage);
+    							return false;
+    						}
+        				}
         			}
-
         		}	
         	}
         	
-        	//4. Mark the volumes as removed
-        	for(VolumeVO vol : markedVolumes)
-        	{
-        		_volsDao.remove(vol.getId());
-        	}
-        	
-        	//5. Restart all the system vms conditionally
-        	if(destroyVolumes) //this means we have another ps. Ok to restart
-        	{
-				//create a dummy event
-				long eventId = saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_SSVM_START, "starting ssvm with Id: "+ssvmId);
-				if(_secStorageMgr.startSecStorageVm(ssvmId, eventId)==null)
-				{
-					s_logger.warn("There was an error starting the ssvm id: "+ssvmId+" on another storage pool, cannot enable primary storage maintenance");
-	            	primaryStorage.setStatus(Status.ErrorInMaintenance);
-	        		_storagePoolDao.persist(primaryStorage);
-					return false;
-				}
-				
-				//create a dummy event
-				long eventId1 = saveScheduledEvent(User.UID_SYSTEM, Account.ACCOUNT_ID_SYSTEM, EventTypes.EVENT_PROXY_START, "starting console proxy with Id: "+consoleProxyId);
-				
-				//Restore config val for consoleproxy.restart to true
-				_configMgr.updateConfiguration(userId, "consoleproxy.restart", "true");
-				
-				if(_consoleProxyMgr.startProxy(consoleProxyId, eventId1)==null)
-				{
-					s_logger.warn("There was an error starting the console proxy id: "+consoleProxyId+" on another storage pool, cannot enable primary storage maintenance");
-	            	primaryStorage.setStatus(Status.ErrorInMaintenance);
-	        		_storagePoolDao.persist(primaryStorage);
-					return false;				}
-        	}
-        	
-        	//6. Update the status
+        	//5. Update the status
         	primaryStorage.setStatus(Status.Maintenance);
         	_storagePoolDao.persist(primaryStorage);
         	
@@ -2418,4 +2597,156 @@ public class StorageManagerImpl implements StorageManager {
 
 	    return true;
 	}
-}
\ No newline at end of file
+	
+	protected DiskProfile toDiskProfile(VolumeVO vol, DiskOfferingVO offering) {
+	    return new DiskProfile(vol.getId(), vol.getVolumeType(), vol.getName(), offering.getId(), vol.getSize(), offering.getTagsArray(), offering.getUseLocalStorage(), offering.isRecreatable(), vol.getTemplateId());
+	}
+	
+    @Override
+    public  DiskProfile allocateRawVolume(VolumeType type, String name, DiskOfferingVO offering, Long size, T vm, AccountVO owner) {
+        if (size == null) {
+            size = offering.getDiskSizeInBytes();
+        }
+        VolumeVO vol = new VolumeVO(type, name, vm.getDataCenterId(), owner.getDomainId(), owner.getId(), offering.getId(), size);
+        if (vm != null) {
+            vol.setInstanceId(vm.getId());
+        }
+        
+        vol = _volsDao.persist(vol);
+        
+        return toDiskProfile(vol, offering);
+    }
+    
+    @Override 
+    public  DiskProfile allocateTemplatedVolume(VolumeType type, String name, DiskOfferingVO offering, VMTemplateVO template, T vm, AccountVO owner) {
+        assert (template.getFormat() != ImageFormat.ISO) : "ISO is not a template really....";
+        
+        SearchCriteria sc = HostTemplateStatesSearch.create();
+        sc.setParameters("id", template.getId());
+        sc.setParameters("state", com.cloud.storage.VMTemplateStorageResourceAssoc.Status.DOWNLOADED);
+        sc.setJoinParameters("host", "dcId", vm.getDataCenterId());
+        
+        List sss = _vmTemplateHostDao.search(sc, null);
+        if (sss.size() == 0) {
+            throw new CloudRuntimeException("Template " + template.getName() + " has not been completely downloaded to zone " + vm.getDataCenterId());
+        }
+        VMTemplateHostVO ss = sss.get(0);
+        
+        VolumeVO vol = new VolumeVO(type, name, vm.getDataCenterId(), owner.getDomainId(), owner.getId(), offering.getId(), ss.getSize());
+        if (vm != null) {
+            vol.setInstanceId(vm.getId());
+        }
+        vol.setTemplateId(template.getId());
+        
+        vol = _volsDao.persist(vol);
+        
+        return toDiskProfile(vol, offering);
+    }
+    
+    final protected DiskProfile createDiskCharacteristics(VolumeVO volume, DiskOfferingVO offering) {
+        return new DiskProfile(volume.getId(), volume.getVolumeType(), volume.getName(), offering.getId(), volume.getSize(), offering.getTagsArray(), offering.getUseLocalStorage(), offering.isRecreatable(), volume.getTemplateId());
+    }
+    
+    final protected DiskProfile createDiskCharacteristics(VolumeVO volume) {
+        DiskOfferingVO offering = _diskOfferingDao.findById(volume.getDiskOfferingId());
+        return createDiskCharacteristics(volume, offering);
+    }
+    
+    @Override
+    public  void create(T vm) {
+        List vols = _volsDao.findByInstance(vm.getId());
+        assert vols.size() >= 1 : "Come on, what's with the zero volumes for " + vm;
+        for (VolumeVO vol : vols) {
+            DiskProfile dskCh = createDiskCharacteristics(vol);
+            int retry = _retry;
+            while (--retry >= 0) {
+                
+            }
+        }
+        /*
+StoragePoolVO pool = null;
+final HashSet avoidPools = new HashSet(avoids);
+
+VolumeType volType = volume.getVolumeType();
+
+VolumeTO created = null;
+int retry = _retry;
+while (--retry >= 0) {
+created = null;
+
+txn.start();
+
+long podId = pod.getId();
+pod = _podDao.lock(podId, true);
+if (pod == null) {
+txn.rollback();
+volume.setStatus(AsyncInstanceCreateStatus.Failed);
+volume.setDestroyed(true);
+_volsDao.persist(volume);
+throw new CloudRuntimeException("Unable to acquire lock on the pod " + podId);
+}
+
+pool = findStoragePool(dskCh, dc, pod, clusterId, offering, vm, template, avoidPools);
+if (pool == null) {
+txn.rollback();
+break;
+}
+
+avoidPools.add(pool);
+if (s_logger.isDebugEnabled()) {
+s_logger.debug("Trying to create " + volume + " on " + pool);
+}
+
+volume.setPoolId(pool.getId());
+_volsDao.persist(volume);
+
+txn.commit();
+
+CreateCommand cmd = null;
+VMTemplateStoragePoolVO tmpltStoredOn = null;
+if (volume.getVolumeType() == VolumeType.ROOT && Storage.ImageFormat.ISO != template.getFormat()) {
+tmpltStoredOn = _tmpltMgr.prepareTemplateForCreate(template, pool);
+if (tmpltStoredOn == null) {
+continue;
+}
+cmd = new CreateCommand(volume, vm, dskCh, tmpltStoredOn.getLocalDownloadPath(), pool);
+} else {
+cmd = new CreateCommand(volume, vm, dskCh, pool, size);
+}
+
+Answer answer = sendToPool(pool, cmd);
+if (answer != null && answer.getResult()) {
+created = ((CreateAnswer)answer).getVolume();
+break;
+}
+
+volume.setPoolId(null);
+_volsDao.persist(volume);
+
+s_logger.debug("Retrying the create because it failed on pool " + pool);
+}
+
+if (created == null) {
+if (s_logger.isDebugEnabled()) {
+s_logger.debug("Unable to create a volume for " + volume);
+}
+volume.setStatus(AsyncInstanceCreateStatus.Failed);
+volume.setDestroyed(true);
+_volsDao.persist(volume);
+
+return null;
+}
+
+volume.setStatus(AsyncInstanceCreateStatus.Created);
+volume.setFolder(pool.getPath());
+volume.setPath(created.getPath());
+volume.setSize(created.getSize());
+volume.setPoolType(pool.getPoolType());
+volume.setPodId(pod.getId());
+_volsDao.persist(volume);
+return volume;
+    */
+        
+    }
+}
+
diff --git a/server/src/com/cloud/storage/allocator/AbstractStoragePoolAllocator.java b/server/src/com/cloud/storage/allocator/AbstractStoragePoolAllocator.java
index 9b9cea7de5b..72e719de9cc 100644
--- a/server/src/com/cloud/storage/allocator/AbstractStoragePoolAllocator.java
+++ b/server/src/com/cloud/storage/allocator/AbstractStoragePoolAllocator.java
@@ -27,11 +27,10 @@ import javax.naming.ConfigurationException;
 
 import org.apache.log4j.Logger;
 
-import com.cloud.agent.api.to.DiskCharacteristicsTO;
 import com.cloud.configuration.dao.ConfigurationDao;
+import com.cloud.deploy.DeployDestination;
 import com.cloud.host.Host;
 import com.cloud.host.HostVO;
-import com.cloud.offering.ServiceOffering;
 import com.cloud.server.StatsCollector;
 import com.cloud.storage.Storage.StoragePoolType;
 import com.cloud.storage.StorageManager;
@@ -43,6 +42,7 @@ import com.cloud.storage.VMTemplateStoragePoolVO;
 import com.cloud.storage.VMTemplateStorageResourceAssoc;
 import com.cloud.storage.VMTemplateStorageResourceAssoc.Status;
 import com.cloud.storage.VMTemplateVO;
+import com.cloud.storage.Volume;
 import com.cloud.storage.Volume.VolumeType;
 import com.cloud.storage.dao.StoragePoolDao;
 import com.cloud.storage.dao.StoragePoolHostDao;
@@ -54,10 +54,11 @@ import com.cloud.template.TemplateManager;
 import com.cloud.utils.NumbersUtil;
 import com.cloud.utils.Pair;
 import com.cloud.utils.component.AdapterBase;
-import com.cloud.utils.component.ComponentLocator;
 import com.cloud.utils.component.Inject;
+import com.cloud.vm.DiskProfile;
 import com.cloud.vm.VMInstanceVO;
 import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VirtualMachineProfile;
 
 public abstract class AbstractStoragePoolAllocator extends AdapterBase implements StoragePoolAllocator {
 	private static final Logger s_logger = Logger.getLogger(FirstFitStoragePoolAllocator.class);
@@ -79,7 +80,6 @@ public abstract class AbstractStoragePoolAllocator extends AdapterBase implement
     @Override
     public boolean configure(String name, Map params) throws ConfigurationException {
         super.configure(name, params);
-        ComponentLocator locator = ComponentLocator.getCurrentLocator();
         
         Map configs = _configDao.getConfiguration(null, params);
         
@@ -100,7 +100,7 @@ public abstract class AbstractStoragePoolAllocator extends AdapterBase implement
         return true;
     }
     
-    abstract boolean allocatorIsCorrectType(DiskCharacteristicsTO dskCh, VMInstanceVO vm, ServiceOffering offering);
+    abstract boolean allocatorIsCorrectType(DiskProfile dskCh, VMInstanceVO vm);
     
 	protected boolean templateAvailable(long templateId, long poolId) {
     	VMTemplateStorageResourceAssoc thvo = _templatePoolDao.findByPoolTemplate(poolId, templateId);
@@ -114,26 +114,16 @@ public abstract class AbstractStoragePoolAllocator extends AdapterBase implement
     	}
     }
 	
-	protected boolean localStorageAllocationNeeded(DiskCharacteristicsTO dskCh, VMInstanceVO vm, ServiceOffering offering) {
-		if (vm == null) {
-    		// We are finding a pool for a volume, so we need a shared storage allocator
-    		return false;
-    	} else if (vm.getType() == VirtualMachine.Type.User) {
-    		// We are finding a pool for a UserVM, so check the service offering to see if we should use local storage
-    		return offering.getUseLocalStorage();
-    	} else {
-    		// We are finding a pool for a DomR or ConsoleProxy, so check the configuration table to see if we should use local storage
-    		String configValue = _configDao.getValue("system.vm.use.local.storage");
-    		return Boolean.parseBoolean(configValue);
-    	}
+	protected boolean localStorageAllocationNeeded(DiskProfile dskCh, VMInstanceVO vm) {
+	    return dskCh.useLocalStorage();
 	}
 	
-	protected boolean poolIsCorrectType(DiskCharacteristicsTO dskCh, StoragePool pool, VMInstanceVO vm, ServiceOffering offering) {
-		boolean localStorageAllocationNeeded = localStorageAllocationNeeded(dskCh, vm, offering);
+	protected boolean poolIsCorrectType(DiskProfile dskCh, StoragePool pool, VMInstanceVO vm) {
+		boolean localStorageAllocationNeeded = localStorageAllocationNeeded(dskCh, vm);
 		return ((!localStorageAllocationNeeded && pool.getPoolType().isShared()) || (localStorageAllocationNeeded && !pool.getPoolType().isShared()));
 	}
 	
-	protected boolean checkPool(Set avoid, StoragePoolVO pool, DiskCharacteristicsTO dskCh, VMTemplateVO template, List templatesInPool, ServiceOffering offering,
+	protected boolean checkPool(Set avoid, StoragePoolVO pool, DiskProfile dskCh, VMTemplateVO template, List templatesInPool, 
 			VMInstanceVO vm, StatsCollector sc) {
 		if (avoid.contains(pool)) {
 			return false;
@@ -144,7 +134,7 @@ public abstract class AbstractStoragePoolAllocator extends AdapterBase implement
             
 		
 		// Check that the pool type is correct
-		if (!poolIsCorrectType(dskCh, pool, vm, offering)) {
+		if (!poolIsCorrectType(dskCh, pool, vm)) {
 			return false;
 		}
 
@@ -166,7 +156,7 @@ public abstract class AbstractStoragePoolAllocator extends AdapterBase implement
 
 		Pair sizes = _volumeDao.getCountAndTotalByPool(pool.getId());
 		
-		long totalAllocatedSize = sizes.second() + (long)sizes.first() * _extraBytesPerVolume;
+		long totalAllocatedSize = sizes.second() + sizes.first() * _extraBytesPerVolume;
 
 		// Iterate through all templates on this storage pool
 		boolean tmpinstalled = false;
@@ -179,11 +169,10 @@ public abstract class AbstractStoragePoolAllocator extends AdapterBase implement
 
 		for (VMTemplateStoragePoolVO templatePoolVO : templatePoolVOs) {
 			VMTemplateVO templateInPool = _templateDao.findById(templatePoolVO.getTemplateId());
-			int templateSizeMultiplier = 2;
+			int templateSizeMultiplier = pool.getPoolType() == StoragePoolType.NetworkFilesystem ? 1 : 2;
 
 			if ((template != null) && !tmpinstalled && (templateInPool.getId() == template.getId())) {
 				tmpinstalled = true;
-				templateSizeMultiplier = 3;
 			}
 			
 			s_logger.debug("For template: " + templateInPool.getName() + ", using template size multiplier: " + templateSizeMultiplier);
@@ -203,9 +192,9 @@ public abstract class AbstractStoragePoolAllocator extends AdapterBase implement
 				if (templateHostVO == null) {
 					return false;
 				} else {
-					s_logger.debug("For template: " + template.getName() + ", using template size multiplier: " + 3);
+					s_logger.debug("For template: " + template.getName() + ", using template size multiplier: " + 2);
 					long templateSize = templateHostVO.getSize();
-					totalAllocatedSize += 3 * (templateSize + _extraBytesPerVolume);
+					totalAllocatedSize += 2 * (templateSize + _extraBytesPerVolume);
 				}
 			}
 		}
@@ -236,4 +225,10 @@ public abstract class AbstractStoragePoolAllocator extends AdapterBase implement
 	public String chooseStorageIp(VirtualMachine vm, Host host, Host storage) {
 		return storage.getStorageIpAddress();
 	}
+	
+	@Override
+	public StoragePool allocateTo(DiskProfile dskCh, VirtualMachineProfile vm, DeployDestination dest, List disks, Set avoids) {
+	    
+	    return null;
+	}
 }
diff --git a/server/src/com/cloud/storage/allocator/FirstFitStoragePoolAllocator.java b/server/src/com/cloud/storage/allocator/FirstFitStoragePoolAllocator.java
index ae9bf16f7db..44002a4e32a 100644
--- a/server/src/com/cloud/storage/allocator/FirstFitStoragePoolAllocator.java
+++ b/server/src/com/cloud/storage/allocator/FirstFitStoragePoolAllocator.java
@@ -25,14 +25,13 @@ import javax.ejb.Local;
 
 import org.apache.log4j.Logger;
 
-import com.cloud.agent.api.to.DiskCharacteristicsTO;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
-import com.cloud.offering.ServiceOffering;
 import com.cloud.server.StatsCollector;
 import com.cloud.storage.StoragePool;
 import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.VMTemplateVO;
+import com.cloud.vm.DiskProfile;
 import com.cloud.vm.VMInstanceVO;
 
 @Local(value=StoragePoolAllocator.class)
@@ -40,15 +39,15 @@ public class FirstFitStoragePoolAllocator extends AbstractStoragePoolAllocator {
     private static final Logger s_logger = Logger.getLogger(FirstFitStoragePoolAllocator.class);
 
     @Override
-    public boolean allocatorIsCorrectType(DiskCharacteristicsTO dskCh, VMInstanceVO vm, ServiceOffering offering) {
-    	return !localStorageAllocationNeeded(dskCh, vm, offering);
+    public boolean allocatorIsCorrectType(DiskProfile dskCh, VMInstanceVO vm) {
+    	return !localStorageAllocationNeeded(dskCh, vm);
     }
 
     @Override
-	public StoragePool allocateToPool(DiskCharacteristicsTO dskCh, ServiceOffering offering, DataCenterVO dc, HostPodVO pod, Long clusterId,
+	public StoragePool allocateToPool(DiskProfile dskCh, DataCenterVO dc, HostPodVO pod, Long clusterId,
 									  VMInstanceVO vm, VMTemplateVO template, Set avoid) {
 		// Check that the allocator type is correct
-        if (!allocatorIsCorrectType(dskCh, vm, offering)) {
+        if (!allocatorIsCorrectType(dskCh, vm)) {
         	return null;
         }
 
@@ -65,7 +64,7 @@ public class FirstFitStoragePoolAllocator extends AbstractStoragePoolAllocator {
         Collections.shuffle(pools);
         
         for (StoragePoolVO pool: pools) {
-        	if (checkPool(avoid, pool, dskCh, template, null, offering, vm, sc)) {
+        	if (checkPool(avoid, pool, dskCh, template, null, vm, sc)) {
         		return pool;
         	}
         }
diff --git a/server/src/com/cloud/storage/allocator/GarbageCollectingStoragePoolAllocator.java b/server/src/com/cloud/storage/allocator/GarbageCollectingStoragePoolAllocator.java
index 9bb42ce5c90..7de9b6614e7 100644
--- a/server/src/com/cloud/storage/allocator/GarbageCollectingStoragePoolAllocator.java
+++ b/server/src/com/cloud/storage/allocator/GarbageCollectingStoragePoolAllocator.java
@@ -26,15 +26,14 @@ import javax.naming.ConfigurationException;
 
 import org.apache.log4j.Logger;
 
-import com.cloud.agent.api.to.DiskCharacteristicsTO;
 import com.cloud.configuration.dao.ConfigurationDao;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
-import com.cloud.offering.ServiceOffering;
 import com.cloud.storage.StorageManager;
 import com.cloud.storage.StoragePool;
 import com.cloud.storage.VMTemplateVO;
 import com.cloud.utils.component.ComponentLocator;
+import com.cloud.vm.DiskProfile;
 import com.cloud.vm.VMInstanceVO;
 
 @Local(value=StoragePoolAllocator.class)
@@ -48,7 +47,7 @@ public class GarbageCollectingStoragePoolAllocator extends AbstractStoragePoolAl
     boolean _storagePoolCleanupEnabled;
     
     @Override
-    public boolean allocatorIsCorrectType(DiskCharacteristicsTO dskCh, VMInstanceVO vm, ServiceOffering offering) {
+    public boolean allocatorIsCorrectType(DiskProfile dskCh, VMInstanceVO vm) {
     	return true;
     }
     
@@ -61,8 +60,7 @@ public class GarbageCollectingStoragePoolAllocator extends AbstractStoragePoolAl
     }
     
     @Override
-    public StoragePool allocateToPool(DiskCharacteristicsTO dskCh,
-                                      ServiceOffering offering,
+    public StoragePool allocateToPool(DiskProfile dskCh,
                                       DataCenterVO dc,
                                       HostPodVO pod,
                                       Long clusterId,
@@ -80,7 +78,7 @@ public class GarbageCollectingStoragePoolAllocator extends AbstractStoragePoolAl
     	
     	// Determine what allocator to use
     	StoragePoolAllocator allocator;
-    	if (localStorageAllocationNeeded(dskCh, vm, offering)) {
+    	if (localStorageAllocationNeeded(dskCh, vm)) {
     		allocator = _localStoragePoolAllocator;
     	} else {
     		allocator = _firstFitStoragePoolAllocator;
@@ -92,7 +90,7 @@ public class GarbageCollectingStoragePoolAllocator extends AbstractStoragePoolAl
             myAvoids.add(pool);
         }
         
-        return allocator.allocateToPool(dskCh, offering, dc, pod, clusterId, vm, template, myAvoids);
+        return allocator.allocateToPool(dskCh, dc, pod, clusterId, vm, template, myAvoids);
     }
 
     @Override
diff --git a/server/src/com/cloud/storage/allocator/LocalStoragePoolAllocator.java b/server/src/com/cloud/storage/allocator/LocalStoragePoolAllocator.java
index 6c77e946c2a..ed0b313c223 100644
--- a/server/src/com/cloud/storage/allocator/LocalStoragePoolAllocator.java
+++ b/server/src/com/cloud/storage/allocator/LocalStoragePoolAllocator.java
@@ -27,14 +27,13 @@ import javax.naming.ConfigurationException;
 
 import org.apache.log4j.Logger;
 
-import com.cloud.agent.api.to.DiskCharacteristicsTO;
 import com.cloud.capacity.CapacityVO;
 import com.cloud.capacity.dao.CapacityDao;
 import com.cloud.configuration.dao.ConfigurationDao;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
+import com.cloud.offering.NetworkOffering;
 import com.cloud.offering.ServiceOffering;
-import com.cloud.offering.ServiceOffering.GuestIpType;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.service.dao.ServiceOfferingDao;
 import com.cloud.storage.StoragePool;
@@ -46,14 +45,16 @@ import com.cloud.utils.DateUtil;
 import com.cloud.utils.NumbersUtil;
 import com.cloud.utils.component.Inject;
 import com.cloud.utils.db.GenericSearchBuilder;
+import com.cloud.utils.db.JoinBuilder;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
 import com.cloud.utils.db.SearchCriteria.Func;
+import com.cloud.vm.DiskProfile;
 import com.cloud.vm.State;
 import com.cloud.vm.UserVmVO;
 import com.cloud.vm.VMInstanceVO;
 import com.cloud.vm.VirtualMachine;
-import com.cloud.vm.VmCharacteristics;
+import com.cloud.vm.VirtualMachineProfile;
 import com.cloud.vm.dao.UserVmDao;
 import com.cloud.vm.dao.VMInstanceDao;
 
@@ -83,13 +84,12 @@ public class LocalStoragePoolAllocator extends FirstFitStoragePoolAllocator {
     
     
     @Override
-    public boolean allocatorIsCorrectType(DiskCharacteristicsTO dskCh, VMInstanceVO vm, ServiceOffering offering) {
-    	return localStorageAllocationNeeded(dskCh, vm, offering);
+    public boolean allocatorIsCorrectType(DiskProfile dskCh, VMInstanceVO vm) {
+    	return localStorageAllocationNeeded(dskCh, vm);
     }
     
     @Override
-    public StoragePool allocateToPool(DiskCharacteristicsTO dskCh,
-                                      ServiceOffering offering,
+    public StoragePool allocateToPool(DiskProfile dskCh,
                                       DataCenterVO dc,
                                       HostPodVO pod,
                                       Long clusterId,
@@ -98,14 +98,14 @@ public class LocalStoragePoolAllocator extends FirstFitStoragePoolAllocator {
                                       Set avoid) {
     	
     	// Check that the allocator type is correct
-        if (!allocatorIsCorrectType(dskCh, vm, offering)) {
+        if (!allocatorIsCorrectType(dskCh, vm)) {
         	return null;
         }
 
         Set myAvoids = new HashSet(avoid);
-        VmCharacteristics vmc = new VmCharacteristics(vm.getType());
+        VirtualMachineProfile vmc = new VirtualMachineProfile(vm.getType());
         StoragePool pool = null;
-        while ((pool = super.allocateToPool(dskCh, offering, dc, pod, clusterId, vm, template, myAvoids)) != null) {
+        while ((pool = super.allocateToPool(dskCh, dc, pod, clusterId, vm, template, myAvoids)) != null) {
             myAvoids.add(pool);
             if (pool.getPoolType().isShared()) {
                 return pool;
@@ -119,7 +119,7 @@ public class LocalStoragePoolAllocator extends FirstFitStoragePoolAllocator {
         	SearchCriteria sc = VmsOnPoolSearch.create();
         	sc.setJoinParameters("volumeJoin", "poolId", pool.getId());
         	sc.setParameters("state", State.Expunging);
-        	List vmsOnHost = _vmInstanceDao.searchAll(sc, null);
+        	List vmsOnHost = _vmInstanceDao.searchIncludingRemoved(sc, null);
             
         	if(s_logger.isDebugEnabled()) {
         		s_logger.debug("Found " + vmsOnHost.size() + " VM instances are alloacated at host " + spHost.getHostId() + " with local storage pool " + pool.getName());
@@ -146,14 +146,14 @@ public class LocalStoragePoolAllocator extends FirstFitStoragePoolAllocator {
     			so = _offeringDao.findById(userVm.getServiceOfferingId());
     	} else if(vm.getType() == VirtualMachine.Type.ConsoleProxy) {
     		so = new ServiceOfferingVO("Fake Offering For DomP", 1,
-				_proxyRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized, false, true, null);
+				_proxyRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, false, true, null);
     	} else if(vm.getType() == VirtualMachine.Type.SecondaryStorageVm) {
-    		so = new ServiceOfferingVO("Fake Offering For Secondary Storage VM", 1, _secStorageVmRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized, false, true, null);
+    		so = new ServiceOfferingVO("Fake Offering For Secondary Storage VM", 1, _secStorageVmRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, false, true, null);
     	} else if(vm.getType() == VirtualMachine.Type.DomainRouter) {
-            so = new ServiceOfferingVO("Fake Offering For DomR", 1, _routerRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized, false, true, null);
+            so = new ServiceOfferingVO("Fake Offering For DomR", 1, _routerRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, false, true, null);
     	} else {
     		assert(false) : "Unsupported system vm type";
-            so = new ServiceOfferingVO("Fake Offering For unknow system VM", 1, 128, 0, 0, 0, false, null, GuestIpType.Virtualized, false, true, null);
+            so = new ServiceOfferingVO("Fake Offering For unknow system VM", 1, 128, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, false, true, null);
     	}
     	
     	long usedMemory = calcHostAllocatedCpuMemoryCapacity(vmOnHost, CapacityVO.CAPACITY_TYPE_MEMORY);
@@ -243,14 +243,14 @@ public class LocalStoragePoolAllocator extends FirstFitStoragePoolAllocator {
         		so = _offeringDao.findById(userVm.getServiceOfferingId());
         	} else if(vm.getType() == VirtualMachine.Type.ConsoleProxy) {
         		so = new ServiceOfferingVO("Fake Offering For DomP", 1,
-    				_proxyRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized, false, true, null);
+    				_proxyRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, false, true, null);
         	} else if(vm.getType() == VirtualMachine.Type.SecondaryStorageVm) {
-        		so = new ServiceOfferingVO("Fake Offering For Secondary Storage VM", 1, _secStorageVmRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized, false, true, null);
+        		so = new ServiceOfferingVO("Fake Offering For Secondary Storage VM", 1, _secStorageVmRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, false, true, null);
         	} else if(vm.getType() == VirtualMachine.Type.DomainRouter) {
-                so = new ServiceOfferingVO("Fake Offering For DomR", 1, _routerRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized, false, true, null);
+                so = new ServiceOfferingVO("Fake Offering For DomR", 1, _routerRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, false, true, null);
         	} else {
         		assert(false) : "Unsupported system vm type";
-                so = new ServiceOfferingVO("Fake Offering For unknow system VM", 1, 128, 0, 0, 0, false, null, GuestIpType.Virtualized, false, true, null);
+                so = new ServiceOfferingVO("Fake Offering For unknow system VM", 1, 128, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, false, true, null);
         	}
             
             if(capacityType == CapacityVO.CAPACITY_TYPE_MEMORY) {
@@ -287,7 +287,7 @@ public class LocalStoragePoolAllocator extends FirstFitStoragePoolAllocator {
         SearchBuilder sbVolume = _volumeDao.createSearchBuilder();
         sbVolume.and("poolId", sbVolume.entity().getPoolId(), SearchCriteria.Op.EQ);
         
-        VmsOnPoolSearch.join("volumeJoin", sbVolume, VmsOnPoolSearch.entity().getId(), sbVolume.entity().getInstanceId());
+        VmsOnPoolSearch.join("volumeJoin", sbVolume, VmsOnPoolSearch.entity().getId(), sbVolume.entity().getInstanceId(), JoinBuilder.JoinType.INNER);
         
         sbVolume.done();
         VmsOnPoolSearch.done();
diff --git a/server/src/com/cloud/storage/allocator/RandomStoragePoolAllocator.java b/server/src/com/cloud/storage/allocator/RandomStoragePoolAllocator.java
index ee0b6252911..3501eba9e64 100644
--- a/server/src/com/cloud/storage/allocator/RandomStoragePoolAllocator.java
+++ b/server/src/com/cloud/storage/allocator/RandomStoragePoolAllocator.java
@@ -25,14 +25,13 @@ import javax.ejb.Local;
 
 import org.apache.log4j.Logger;
 
-import com.cloud.agent.api.to.DiskCharacteristicsTO;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
-import com.cloud.offering.ServiceOffering;
 import com.cloud.server.StatsCollector;
 import com.cloud.storage.StoragePool;
 import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.VMTemplateVO;
+import com.cloud.vm.DiskProfile;
 import com.cloud.vm.VMInstanceVO;
 
 @Local(value=StoragePoolAllocator.class)
@@ -40,17 +39,17 @@ public class RandomStoragePoolAllocator extends AbstractStoragePoolAllocator {
     private static final Logger s_logger = Logger.getLogger(RandomStoragePoolAllocator.class);
     
     @Override
-    public boolean allocatorIsCorrectType(DiskCharacteristicsTO dskCh, VMInstanceVO vm, ServiceOffering offering) {
+    public boolean allocatorIsCorrectType(DiskProfile dskCh, VMInstanceVO vm) {
     	return true;
     }
     
     @Override
-    public StoragePool allocateToPool(DiskCharacteristicsTO dskCh, ServiceOffering offering,
+    public StoragePool allocateToPool(DiskProfile dskCh,
 			 DataCenterVO dc, HostPodVO pod, Long clusterId, VMInstanceVO vm,
 			VMTemplateVO template, Set avoid) {
     	
     	// Check that the allocator type is correct
-        if (!allocatorIsCorrectType(dskCh, vm, offering)) {
+        if (!allocatorIsCorrectType(dskCh, vm)) {
         	return null;
         }
     	
@@ -67,7 +66,7 @@ public class RandomStoragePoolAllocator extends AbstractStoragePoolAllocator {
         Collections.shuffle(pools);
 
         for (StoragePoolVO pool: pools) {
-        	if (checkPool(avoid, pool, dskCh, template, null, offering, vm, sc)) {
+        	if (checkPool(avoid, pool, dskCh, template, null, vm, sc)) {
         		return pool;
         	}
         }
diff --git a/server/src/com/cloud/storage/allocator/StoragePoolAllocator.java b/server/src/com/cloud/storage/allocator/StoragePoolAllocator.java
index 8d592ec4bfb..deec2ac7049 100644
--- a/server/src/com/cloud/storage/allocator/StoragePoolAllocator.java
+++ b/server/src/com/cloud/storage/allocator/StoragePoolAllocator.java
@@ -17,18 +17,21 @@
  */
 package com.cloud.storage.allocator;
 
+import java.util.List;
 import java.util.Set;
 
-import com.cloud.agent.api.to.DiskCharacteristicsTO;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
+import com.cloud.deploy.DeployDestination;
 import com.cloud.host.Host;
-import com.cloud.offering.ServiceOffering;
 import com.cloud.storage.StoragePool;
 import com.cloud.storage.VMTemplateVO;
+import com.cloud.storage.Volume;
 import com.cloud.utils.component.Adapter;
+import com.cloud.vm.DiskProfile;
 import com.cloud.vm.VMInstanceVO;
 import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VirtualMachineProfile;
 
 /**
  * Allocator for a disk.  This determines which StoragePool should
@@ -36,7 +39,9 @@ import com.cloud.vm.VirtualMachine;
  */
 public interface StoragePoolAllocator extends Adapter {
 	
-	StoragePool allocateToPool(DiskCharacteristicsTO dskCh, ServiceOffering offering, DataCenterVO dc, HostPodVO pod, Long cluster, VMInstanceVO vm, VMTemplateVO template, Set avoids);
+	StoragePool allocateToPool(DiskProfile dskCh, DataCenterVO dc, HostPodVO pod, Long cluster, VMInstanceVO vm, VMTemplateVO template, Set avoids);
 	
 	String chooseStorageIp(VirtualMachine vm, Host host, Host storage);
+	
+	StoragePool allocateTo(DiskProfile dskCh, VirtualMachineProfile vm, DeployDestination dest, List disks, Set avoids);
 }
diff --git a/server/src/com/cloud/storage/allocator/UseLocalForRootAllocator.java b/server/src/com/cloud/storage/allocator/UseLocalForRootAllocator.java
index 1423b214349..2875e1dd10e 100644
--- a/server/src/com/cloud/storage/allocator/UseLocalForRootAllocator.java
+++ b/server/src/com/cloud/storage/allocator/UseLocalForRootAllocator.java
@@ -23,17 +23,16 @@ import java.util.Set;
 import javax.ejb.Local;
 import javax.naming.ConfigurationException;
 
-import com.cloud.agent.api.to.DiskCharacteristicsTO;
 import com.cloud.configuration.Config;
 import com.cloud.configuration.dao.ConfigurationDao;
 import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
 import com.cloud.host.Host;
-import com.cloud.offering.ServiceOffering;
 import com.cloud.storage.StoragePool;
 import com.cloud.storage.VMTemplateVO;
 import com.cloud.storage.Volume.VolumeType;
 import com.cloud.utils.component.ComponentLocator;
+import com.cloud.vm.DiskProfile;
 import com.cloud.vm.VMInstanceVO;
 import com.cloud.vm.VirtualMachine;
 
@@ -42,12 +41,12 @@ public class UseLocalForRootAllocator extends LocalStoragePoolAllocator implemen
     boolean _useLocalStorage;
 
     @Override
-    public StoragePool allocateToPool(DiskCharacteristicsTO dskCh, ServiceOffering offering, DataCenterVO dc, HostPodVO pod, Long clusterId, VMInstanceVO vm, VMTemplateVO template, Set avoids) {
+    public StoragePool allocateToPool(DiskProfile dskCh, DataCenterVO dc, HostPodVO pod, Long clusterId, VMInstanceVO vm, VMTemplateVO template, Set avoids) {
         if (!_useLocalStorage) {
             return null;
         }
         
-        return super.allocateToPool(dskCh, offering, dc, pod, clusterId, vm, template, avoids);
+        return super.allocateToPool(dskCh, dc, pod, clusterId, vm, template, avoids);
     }
 
     @Override
@@ -69,13 +68,13 @@ public class UseLocalForRootAllocator extends LocalStoragePoolAllocator implemen
     }
     
     @Override
-    protected boolean localStorageAllocationNeeded(DiskCharacteristicsTO dskCh, VMInstanceVO vm, ServiceOffering offering) {
+    protected boolean localStorageAllocationNeeded(DiskProfile dskCh, VMInstanceVO vm) {
         if (dskCh.getType() == VolumeType.ROOT) {
             return true;
         } else if (dskCh.getType() == VolumeType.DATADISK) {
             return false;
         } else {
-            return super.localStorageAllocationNeeded(dskCh, vm, offering);
+            return super.localStorageAllocationNeeded(dskCh, vm);
         }
     }
     
diff --git a/server/src/com/cloud/storage/download/DownloadListener.java b/server/src/com/cloud/storage/download/DownloadListener.java
index 31876472f2c..b484f56ed44 100644
--- a/server/src/com/cloud/storage/download/DownloadListener.java
+++ b/server/src/com/cloud/storage/download/DownloadListener.java
@@ -40,10 +40,10 @@ import com.cloud.agent.api.storage.DownloadProgressCommand.RequestType;
 import com.cloud.event.EventTypes;
 import com.cloud.event.EventVO;
 import com.cloud.host.HostVO;
+import com.cloud.storage.Storage;
 import com.cloud.storage.VMTemplateHostVO;
 import com.cloud.storage.VMTemplateStorageResourceAssoc.Status;
 import com.cloud.storage.VMTemplateVO;
-import com.cloud.storage.Volume;
 import com.cloud.storage.dao.VMTemplateHostDao;
 import com.cloud.storage.download.DownloadState.DownloadEvent;
 import com.cloud.utils.exception.CloudRuntimeException;
@@ -286,8 +286,8 @@ public class DownloadListener implements Listener {
 	    long agentId = agent.getId();
 	    
 	    StartupStorageCommand storage = (StartupStorageCommand)cmd;
-	    if (storage.getResourceType() == Volume.StorageResourceType.STORAGE_HOST ||
-	    storage.getResourceType() == Volume.StorageResourceType.SECONDARY_STORAGE )
+	    if (storage.getResourceType() == Storage.StorageResourceType.STORAGE_HOST ||
+	    storage.getResourceType() == Storage.StorageResourceType.SECONDARY_STORAGE )
 	    {
 	    	downloadMonitor.handleTemplateSync(agentId, storage.getTemplateInfo());
 	    } else {
diff --git a/server/src/com/cloud/storage/download/DownloadMonitorImpl.java b/server/src/com/cloud/storage/download/DownloadMonitorImpl.java
old mode 100644
new mode 100755
index a61f538ee3e..824169d0288
--- a/server/src/com/cloud/storage/download/DownloadMonitorImpl.java
+++ b/server/src/com/cloud/storage/download/DownloadMonitorImpl.java
@@ -303,7 +303,7 @@ public class DownloadMonitorImpl implements  DownloadMonitor {
 			List dcs = new ArrayList();
 			
 			if (zoneId == null) {
-				dcs.addAll(_dcDao.listAll());
+				dcs.addAll(_dcDao.listAllIncludingRemoved());
 			} else {
 				dcs.add(_dcDao.findById(zoneId));
 			}
@@ -446,12 +446,13 @@ public class DownloadMonitorImpl implements  DownloadMonitor {
                     tmpltHost.setDownloadPercent(100);
                     tmpltHost.setDownloadState(Status.DOWNLOADED);
                     tmpltHost.setInstallPath(templateInfo.get(uniqueName).getInstallPath());
+                    tmpltHost.setSize(templateInfo.get(uniqueName).getSize());
                     tmpltHost.setLastUpdated(new Date());
 					_vmTemplateHostDao.update(tmpltHost.getId(), tmpltHost);
 				} else {
-					VMTemplateHostVO templtHost = new VMTemplateHostVO(sserverId, tmplt.getId(), new Date(), 100, Status.DOWNLOADED, null, null, null, templateInfo.get(uniqueName).getInstallPath(), tmplt.getUrl());
-					templtHost.setSize(templateInfo.get(uniqueName).getSize());
-					_vmTemplateHostDao.persist(templtHost);
+				    tmpltHost = new VMTemplateHostVO(sserverId, tmplt.getId(), new Date(), 100, Status.DOWNLOADED, null, null, null, templateInfo.get(uniqueName).getInstallPath(), tmplt.getUrl());
+					tmpltHost.setSize(templateInfo.get(uniqueName).getSize());
+					_vmTemplateHostDao.persist(tmpltHost);
 				}
 				templateInfo.remove(uniqueName);
 				continue;
@@ -482,7 +483,15 @@ public class DownloadMonitorImpl implements  DownloadMonitor {
 			TemplateInfo tInfo = templateInfo.get(uniqueName);
 			DeleteTemplateCommand dtCommand = new DeleteTemplateCommand(tInfo.getInstallPath());
 			long result = send(sserverId, dtCommand, null);
-			s_logger.info("Deleted template " + tInfo.getTemplateName() + " on secondary storage " + sserverId + " since it isn't in the database, result=" + result);
+			if (result == -1 ){
+			    String description = "Failed to delete " + tInfo.getTemplateName() + " on secondary storage " + sserverId + " which isn't in the database";
+			    logEvent(1L, EventTypes.EVENT_TEMPLATE_DELETE, description , EventVO.LEVEL_ERROR);
+			    s_logger.error(description);
+			    return;
+			}
+			String description = "Deleted template " + tInfo.getTemplateName() + " on secondary storage " + sserverId + " since it isn't in the database, result=" + result;
+			logEvent(1L, EventTypes.EVENT_TEMPLATE_DELETE, description, EventVO.LEVEL_INFO);
+			s_logger.info(description);
 
 		}
 
diff --git a/server/src/com/cloud/storage/listener/StoragePoolMonitor.java b/server/src/com/cloud/storage/listener/StoragePoolMonitor.java
index aede905b511..eead217a944 100755
--- a/server/src/com/cloud/storage/listener/StoragePoolMonitor.java
+++ b/server/src/com/cloud/storage/listener/StoragePoolMonitor.java
@@ -69,7 +69,8 @@ public class StoragePoolMonitor implements Listener {
     public boolean processConnect(HostVO host, StartupCommand cmd) {
     	if (cmd instanceof StartupRoutingCommand) {
     		StartupRoutingCommand scCmd = (StartupRoutingCommand)cmd;
-    		if (scCmd.getHypervisorType() == Hypervisor.Type.XenServer || scCmd.getHypervisorType() ==  Hypervisor.Type.KVM) {
+    		if (scCmd.getHypervisorType() == Hypervisor.Type.XenServer || scCmd.getHypervisorType() ==  Hypervisor.Type.KVM ||
+				scCmd.getHypervisorType() == Hypervisor.Type.VmWare) {
     			List pools = _poolDao.listBy(host.getDataCenterId(), host.getPodId(), host.getClusterId());
     			for (StoragePoolVO pool : pools) {
     				Long hostId = host.getId();
diff --git a/server/src/com/cloud/storage/preallocatedlun/dao/PreallocatedLunDaoImpl.java b/server/src/com/cloud/storage/preallocatedlun/dao/PreallocatedLunDaoImpl.java
index c00b0ad745d..2e37fd7cf4b 100644
--- a/server/src/com/cloud/storage/preallocatedlun/dao/PreallocatedLunDaoImpl.java
+++ b/server/src/com/cloud/storage/preallocatedlun/dao/PreallocatedLunDaoImpl.java
@@ -36,6 +36,7 @@ import com.cloud.utils.component.ComponentLocator;
 import com.cloud.utils.db.DB;
 import com.cloud.utils.db.GenericDaoBase;
 import com.cloud.utils.db.GenericSearchBuilder;
+import com.cloud.utils.db.JoinBuilder;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
 import com.cloud.utils.db.SearchCriteria.Func;
@@ -73,7 +74,7 @@ public class PreallocatedLunDaoImpl extends GenericDaoBase targetSearch = createSearchBuilder();
         targetSearch.and("targetiqn", targetSearch.entity().getTargetIqn(), SearchCriteria.Op.EQ);
-        DetailsSearch.join("target", targetSearch, targetSearch.entity().getId(), DetailsSearch.entity().getLunId());
+        DetailsSearch.join("target", targetSearch, targetSearch.entity().getId(), DetailsSearch.entity().getLunId(), JoinBuilder.JoinType.INNER);
         DetailsSearch.select(null, Func.DISTINCT, DetailsSearch.entity().getTag());
         DetailsSearch.done();
         targetSearch.done();
@@ -100,7 +101,7 @@ public class PreallocatedLunDaoImpl extends GenericDaoBase sc = DeleteSearch.create();
     	sc.setParameters("id", id);
     	
-    	return delete(sc) > 0;
+    	return expunge(sc) > 0;
     }
     
     @Override
@@ -184,7 +185,7 @@ public class PreallocatedLunDaoImpl extends GenericDaoBase sc = TotalSizeSearch.create();
         sc.setParameters("target", targetIqn);
         
-        List results = searchAll(sc, null);
+        List results = searchIncludingRemoved(sc, null);
         if (results.size() == 0) {
             return 0;
         }
@@ -197,7 +198,7 @@ public class PreallocatedLunDaoImpl extends GenericDaoBase sc = UsedSizeSearch.create();
         sc.setParameters("target", targetIqn);
         
-        List results = searchAll(sc, null);
+        List results = searchIncludingRemoved(sc, null);
         if (results.size() == 0) {
             return 0;
         }
@@ -209,7 +210,7 @@ public class PreallocatedLunDaoImpl extends GenericDaoBase findDistinctTagsForTarget(String targetIqn) {
         SearchCriteria sc = DetailsSearch.create();
         sc.setJoinParameters("target", "targetiqn", targetIqn);
-        return _detailsDao.searchAll(sc, null);
+        return _detailsDao.searchIncludingRemoved(sc, null);
     }
 
     @Override @DB
diff --git a/server/src/com/cloud/storage/secondary/LocalSecondaryStorageResource.java b/server/src/com/cloud/storage/secondary/LocalSecondaryStorageResource.java
index 10514bcbd4b..7a73d62007d 100644
--- a/server/src/com/cloud/storage/secondary/LocalSecondaryStorageResource.java
+++ b/server/src/com/cloud/storage/secondary/LocalSecondaryStorageResource.java
@@ -40,9 +40,9 @@ import com.cloud.host.Host;
 import com.cloud.host.Host.Type;
 import com.cloud.resource.ServerResource;
 import com.cloud.resource.ServerResourceBase;
+import com.cloud.storage.Storage;
 import com.cloud.storage.Storage.StoragePoolType;
 import com.cloud.storage.StorageLayer;
-import com.cloud.storage.Volume;
 import com.cloud.storage.template.DownloadManager;
 import com.cloud.storage.template.DownloadManagerImpl;
 import com.cloud.storage.template.TemplateInfo;
@@ -160,7 +160,7 @@ public class LocalSecondaryStorageResource extends ServerResourceBase implements
     public StartupCommand[] initialize() {
         
         final StartupStorageCommand cmd = new StartupStorageCommand(_parent, StoragePoolType.Filesystem, 1024l*1024l*1024l*1024l, _dlMgr.gatherTemplateInfo());
-        cmd.setResourceType(Volume.StorageResourceType.SECONDARY_STORAGE);
+        cmd.setResourceType(Storage.StorageResourceType.SECONDARY_STORAGE);
         cmd.setIqn(null);
         fillNetworkInformation(cmd);
         cmd.setDataCenter(_dc);
diff --git a/server/src/com/cloud/storage/secondary/SecondaryStorageDiscoverer.java b/server/src/com/cloud/storage/secondary/SecondaryStorageDiscoverer.java
index aecc77ccf92..6aa0138ae50 100644
--- a/server/src/com/cloud/storage/secondary/SecondaryStorageDiscoverer.java
+++ b/server/src/com/cloud/storage/secondary/SecondaryStorageDiscoverer.java
@@ -271,7 +271,7 @@ public class SecondaryStorageDiscoverer extends DiscovererBase implements Discov
     		_vmTemplateZoneDao.persist(vmTemplateZone);
     	}
 
-    	List allTemplates = _vmTemplateDao.listAllActive();
+    	List allTemplates = _vmTemplateDao.listAll();
     	for (VMTemplateVO vt: allTemplates){
     		if (vt.isCrossZones()){
     			tmpltZone = _vmTemplateZoneDao.findByZoneTemplate(dcId, vt.getId());
diff --git a/server/src/com/cloud/storage/secondary/SecondaryStorageListener.java b/server/src/com/cloud/storage/secondary/SecondaryStorageListener.java
index 3c91763f102..72677dbfe9c 100644
--- a/server/src/com/cloud/storage/secondary/SecondaryStorageListener.java
+++ b/server/src/com/cloud/storage/secondary/SecondaryStorageListener.java
@@ -28,7 +28,8 @@ import com.cloud.agent.api.StartupCommand;
 import com.cloud.agent.api.StartupStorageCommand;
 import com.cloud.host.HostVO;
 import com.cloud.host.Status;
-import com.cloud.storage.Volume.StorageResourceType;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.StorageResourceType;
 
 public class SecondaryStorageListener implements Listener {
     private final static Logger s_logger = Logger.getLogger(SecondaryStorageListener.class);
@@ -82,7 +83,7 @@ public class SecondaryStorageListener implements Listener {
         if (cmd instanceof StartupStorageCommand) {
             
             StartupStorageCommand ss = (StartupStorageCommand)cmd;
-            if (ss.getResourceType() == StorageResourceType.SECONDARY_STORAGE) {
+            if (ss.getResourceType() == Storage.StorageResourceType.SECONDARY_STORAGE) {
             	_ssVmMgr.onAgentConnect(agent.getDataCenterId(), cmd);
             	_ssVmMgr.generateFirewallConfiguration(agent.getId());
             	_ssVmMgr.generateSetupCommand(agent.getDataCenterId());
diff --git a/server/src/com/cloud/storage/secondary/SecondaryStorageManagerImpl.java b/server/src/com/cloud/storage/secondary/SecondaryStorageManagerImpl.java
index b8434160329..f81f4fb7fe4 100644
--- a/server/src/com/cloud/storage/secondary/SecondaryStorageManagerImpl.java
+++ b/server/src/com/cloud/storage/secondary/SecondaryStorageManagerImpl.java
@@ -86,7 +86,7 @@ import com.cloud.network.IpAddrAllocator;
 import com.cloud.network.IpAddrAllocator.networkInfo;
 import com.cloud.network.NetworkManager;
 import com.cloud.network.dao.IPAddressDao;
-import com.cloud.offering.ServiceOffering.GuestIpType;
+import com.cloud.offering.NetworkOffering;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.service.dao.ServiceOfferingDao;
 import com.cloud.storage.StorageManager;
@@ -102,6 +102,7 @@ import com.cloud.storage.dao.VMTemplateHostDao;
 import com.cloud.storage.dao.VolumeDao;
 import com.cloud.storage.template.TemplateConstants;
 import com.cloud.user.Account;
+import com.cloud.user.AccountManager;
 import com.cloud.user.AccountVO;
 import com.cloud.user.User;
 import com.cloud.user.dao.AccountDao;
@@ -197,10 +198,13 @@ public class SecondaryStorageManagerImpl implements SecondaryStorageVmManager, V
 	private SecondaryStorageListener _listener;
 	
     private ServiceOfferingVO _serviceOffering;
+    private int _networkRate;
+    private int _multicastRate;
     private VMTemplateVO _template;
     @Inject private ConfigurationDao _configDao;
     @Inject private EventDao _eventDao;
     @Inject private ServiceOfferingDao _offeringDao;
+    @Inject private AccountManager _accountMgr;
     
     private IpAddrAllocator _IpAllocator;
     
@@ -317,8 +321,9 @@ public class SecondaryStorageManagerImpl implements SecondaryStorageVmManager, V
 
 			if (routingHost == null) {
 				if (s_logger.isDebugEnabled()) {
-					s_logger.debug("Unable to find a routing host for " + secStorageVm.toString());
-					continue;
+                    String msg = "Unable to find a routing host for " + secStorageVm.toString() + " in pod " + pod.getId();
+                    s_logger.debug(msg);
+                    throw new CloudRuntimeException(msg);
 				}
 			}
 			// to ensure atomic state transition to Starting state
@@ -349,9 +354,9 @@ public class SecondaryStorageManagerImpl implements SecondaryStorageVmManager, V
 							secStorageVm.getDataCenterId(), routingHost.getPodId(),
 							secStorageVm.getId(), secStorageVm.getPrivateMacAddress());
 					if (privateIpAddress == null && (_IpAllocator != null && !_IpAllocator.exteralIpAddressAllocatorEnabled())) {
-						s_logger.debug("Not enough ip addresses in " + routingHost.getPodId());
-						avoid.add(routingHost);
-						continue;
+                        String msg = "Unable to allocate private ip addresses for  " + secStorageVm.getName() + " in pod " + pod.getId();
+                        s_logger.debug(msg);
+                        throw new CloudRuntimeException(msg);
 					}
 
 					secStorageVm.setPrivateIpAddress(privateIpAddress);
@@ -362,17 +367,17 @@ public class SecondaryStorageManagerImpl implements SecondaryStorageVmManager, V
 
 					List vols = _storageMgr.prepare(secStorageVm, routingHost);
 					if (vols == null || vols.size() == 0) {
-                        s_logger.warn("Can not share " + secStorageVm.getName());
-                        avoid.add(routingHost);
-                        continue;
+                        String msg = "Unable to prepare storage for " + secStorageVm.getName() + " in pod " + pod.getId();
+                        s_logger.debug(msg);
+                        throw new CloudRuntimeException(msg);
 					}
 		            VolumeVO vol = vols.get(0);
 		            
 					// carry the secondary storage vm port info over so that we don't
 					// need to configure agent on this
-					StartSecStorageVmCommand cmdStart = new StartSecStorageVmCommand(
-							_secStorageVmCmdPort, secStorageVm, secStorageVm.getName(), "",
-							vols, _mgmt_host, _mgmt_port, _useSSlCopy);
+					StartSecStorageVmCommand cmdStart = new StartSecStorageVmCommand(_networkRate, 
+					        _multicastRate, _secStorageVmCmdPort, secStorageVm, 
+					        secStorageVm.getName(), "", vols, _mgmt_host, _mgmt_port, _useSSlCopy);
 
 					if (s_logger.isDebugEnabled())
 						s_logger.debug("Sending start command for secondary storage vm "
@@ -521,12 +526,13 @@ public class SecondaryStorageManagerImpl implements SecondaryStorageVmManager, V
 		return null;
 	}
 	
-	public boolean  generateFirewallConfiguration(Long hostId){
+	@Override
+    public boolean  generateFirewallConfiguration(Long hostId){
 		if (hostId == null) {
 			return true;
 		}
 		boolean success = true;
-		List allZones = _dcDao.listAllActive();
+		List allZones = _dcDao.listAll();
 		for (DataCenterVO zone: allZones){
 			success = success && generateFirewallConfigurationForZone( zone.getId());
 		}
@@ -715,10 +721,9 @@ public class SecondaryStorageManagerImpl implements SecondaryStorageVmManager, V
 			}
 			
 			if (pod == null || publicIpAndVlan == null) {
-				s_logger.warn("Unable to allocate pod for secondary storage vm in data center : " + dataCenterId);
-
-				context.put("secStorageVmId", (long) 0);
-				return context;
+                String msg = "Unable to allocate pod for secondary storage vm in data center : " + dataCenterId;
+                s_logger.warn(msg);
+                throw new CloudRuntimeException(msg);
 			}
 			
 			long id = _secStorageVmDao.getNextInSequence(Long.class, "id");
@@ -737,14 +742,15 @@ public class SecondaryStorageManagerImpl implements SecondaryStorageVmManager, V
             String vlanGateway = publicIpAndVlan._gateWay;
             String vlanNetmask = publicIpAndVlan._netMask;
 
+            AccountVO systemAcct = _accountMgr.getSystemAccount();
 			txn.start();
 			SecondaryStorageVmVO secStorageVm;
 			String name = VirtualMachineName.getSystemVmName(id, _instance, "s").intern();
 	        
-			secStorageVm = new SecondaryStorageVmVO(id, name, guestMacAddress, null, NetUtils.getLinkLocalNetMask(),
+			secStorageVm = new SecondaryStorageVmVO(id, _serviceOffering.getId(), name, guestMacAddress, null, NetUtils.getLinkLocalNetMask(),
 					privateMacAddress, null, cidrNetmask, _template.getId(), _template.getGuestOSId(),
 					publicMacAddress, publicIpAddress, vlanNetmask, publicIpAndVlan._vlanDbId, publicIpAndVlan._vlanid,
-					pod.first().getId(), dataCenterId, vlanGateway, null,
+					pod.first().getId(), dataCenterId, systemAcct.getDomainId(), systemAcct.getId(), vlanGateway, null,
 					dc.getInternalDns1(), dc.getInternalDns2(), _domain, _secStorageVmRamSize, secHost.getGuid(), secHost.getStorageUrl());
 
 			secStorageVm.setLastHostId(pod.second());
@@ -952,7 +958,7 @@ public class SecondaryStorageManagerImpl implements SecondaryStorageVmManager, V
 					try {
 						checkPendingSecStorageVMs();
 						
-						List datacenters = _dcDao.listAll();
+						List datacenters = _dcDao.listAllIncludingRemoved();
 
 
 						for (DataCenterVO dc: datacenters){
@@ -1365,11 +1371,15 @@ public class SecondaryStorageManagerImpl implements SecondaryStorageVmManager, V
 		}
 		
 		boolean useLocalStorage = Boolean.parseBoolean((String)params.get(Config.SystemVMUseLocalStorage.key()));
-		_serviceOffering = new ServiceOfferingVO("Fake Offering For Secondary Storage VM", 1, _secStorageVmRamSize, 0, 0, 0, false, null, GuestIpType.Virtualized, useLocalStorage, true, null);
+        String networkRateStr = _configDao.getValue("network.throttling.rate");
+        String multicastRateStr = _configDao.getValue("multicast.throttling.rate");
+        _networkRate = ((networkRateStr == null) ? 200 : Integer.parseInt(networkRateStr));
+        _multicastRate = ((multicastRateStr == null) ? 10 : Integer.parseInt(multicastRateStr));
+		_serviceOffering = new ServiceOfferingVO("Fake Offering For Secondary Storage VM", 1, _secStorageVmRamSize, 0, 0, 0, false, null, NetworkOffering.GuestIpType.Virtualized, useLocalStorage, true, null);
 		_serviceOffering.setUniqueName("Cloud.com-SecondaryStorage");
 		_serviceOffering = _offeringDao.persistSystemServiceOffering(_serviceOffering);
         _template = _templateDao.findConsoleProxyTemplate();
-        if (_template == null) {
+        if (_template == null && _useServiceVM) {
             throw new ConfigurationException("Unable to find the template for secondary storage vm VMs");
         }
  
diff --git a/server/src/com/cloud/storage/snapshot/SnapshotManager.java b/server/src/com/cloud/storage/snapshot/SnapshotManager.java
index b6c239c5e48..54e238a6d65 100644
--- a/server/src/com/cloud/storage/snapshot/SnapshotManager.java
+++ b/server/src/com/cloud/storage/snapshot/SnapshotManager.java
@@ -32,6 +32,7 @@ import com.cloud.exception.ResourceAllocationException;
 import com.cloud.storage.SnapshotPolicyVO;
 import com.cloud.storage.SnapshotScheduleVO;
 import com.cloud.storage.SnapshotVO;
+import com.cloud.storage.Storage.ImageFormat;
 import com.cloud.utils.component.Manager;
 import com.cloud.utils.db.Filter;
 
@@ -148,4 +149,5 @@ public interface SnapshotManager extends Manager {
     void validateSnapshot(Long userId, SnapshotVO snapshot);
 
     boolean deleteSnapshotPolicies(DeleteSnapshotPoliciesCmd cmd) throws InvalidParameterValueException;
+	ImageFormat getImageFormat(Long volumeId);
 }
diff --git a/server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java b/server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java
index 9644d8305bb..4c939e4ef52 100644
--- a/server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java
+++ b/server/src/com/cloud/storage/snapshot/SnapshotManagerImpl.java
@@ -105,6 +105,7 @@ import com.cloud.utils.component.Inject;
 import com.cloud.utils.db.DB;
 import com.cloud.utils.db.Filter;
 import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.JoinBuilder;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
 import com.cloud.utils.db.Transaction;
@@ -194,7 +195,8 @@ public class SnapshotManagerImpl implements SnapshotManager {
         return runSnap;
     }
     
-    private ImageFormat getImageFormat(Long volumeId) {
+    @Override
+    public ImageFormat getImageFormat(Long volumeId) {
         ImageFormat format = null;
         VolumeVO volume = _volsDao.findById(volumeId);
         Long templateId = volume.getTemplateId();
@@ -319,7 +321,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
         }
         txn.commit();
 
-        VolumeVO volume = _volsDao.findById(volumeId);
+        VolumeVO volume = _volsDao.lock(volumeId, true);
         
         if (!shouldRunSnapshot(userId, volume, policyIds)) {
             // A null snapshot is interpreted as snapshot creation failed which is what we want to indicate
@@ -399,7 +401,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
             // The snapshot was not successfully created
             createdSnapshot = _snapshotDao.findById(id);
             // delete from the snapshots table
-            _snapshotDao.delete(id);
+            _snapshotDao.expunge(id);
             
         }
 
@@ -486,7 +488,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
                 if (answer.getResult()) {
                     // The expected snapshot details on the primary is the same as it would be if this snapshot was never taken at all
                     // Just delete the entry in the table
-                    _snapshotDao.delete(id);
+                    _snapshotDao.expunge(id);
                     // Create an event saying the snapshot failed. ??
                     // An event is not generated when validatePreviousSnapshotBackup fails. So not generating it here too.
                 }
@@ -533,7 +535,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
         _snapshotDao.update(snapshot.getId(), snapshot);
         
         long volumeId   = snapshot.getVolumeId();
-        VolumeVO volume = _volsDao.findById(volumeId);
+        VolumeVO volume = _volsDao.lock(volumeId, true);
         
         String primaryStoragePoolNameLabel = _storageMgr.getPrimaryStorageNameLabel(volume);
         Long dcId                          = volume.getDataCenterId();
@@ -621,7 +623,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
                     long pprevSnapshotId = prevSnapshot.getPrevSnapshotId();
                     snapshot.setPrevSnapshotId(pprevSnapshotId);
                     _snapshotDao.update(snapshot.getId(), snapshot);
-                    _snapshotDao.delete(prevSnapshot.getId());
+                    _snapshotDao.expunge(prevSnapshot.getId());
                     
                     EventVO event = new EventVO();
                     String eventParams = "id=" + prevSnapshot.getId() + "\nssName=" + prevSnapshot.getName();
@@ -750,7 +752,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
                 SnapshotScheduleVO snapshotSchedule = _snapshotScheduleDao.getCurrentSchedule(volumeId, policyId, true);
                 if (snapshotSchedule != null) {
                     // We should lock the row before deleting it as it is also being deleted by the scheduler.
-                    _snapshotScheduleDao.delete(snapshotSchedule.getId());
+                    _snapshotScheduleDao.expunge(snapshotSchedule.getId());
                 }
             }
         }
@@ -830,7 +832,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
     	Account account = (Account)UserContext.current().getAccountObject();
     	if (account != null) {
     		if (!isAdmin(account.getType())) {
-    			if (account.getId().longValue() != targetAccountId) {
+    			if (account.getId() != targetAccountId) {
     				throw new ServerApiException(BaseCmd.PARAM_ERROR, "Unable to find a " + targetDesc + " with id " + targetId + " for this account");
     			}
     		} else if (!_domainDao.isChildDomain(account.getDomainId(), targetDomainId)) {
@@ -1062,7 +1064,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
         if(isLastSnap){
             _snapshotDao.remove(snapshotId);
         } else {
-            _snapshotDao.delete(snapshotId);
+            _snapshotDao.expunge(snapshotId);
             // In the snapshots table,
             // the last_snapshot_id field of the next snapshot becomes the last_snapshot_id of the deleted snapshot
             long prevSnapshotId = snapshot.getPrevSnapshotId();
@@ -1092,6 +1094,9 @@ public class SnapshotManagerImpl implements SnapshotManager {
         // i.e Call them before the VMs for those volumes are destroyed.
         boolean success = true;
         for (VolumeVO volume : volumes) {
+        	if(volume.getPoolId()==null){
+        		continue;
+        	}
         	Long volumeId = volume.getId();
         	Long dcId = volume.getDataCenterId();
         	String secondaryStoragePoolURL = _storageMgr.getSecondaryStorageURL(dcId);
@@ -1135,7 +1140,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
         	// Either way delete the snapshots for this volume.
         	List snapshots = listSnapsforVolume(volumeId);
         	for (SnapshotVO snapshot: snapshots) {
-        	    if(_snapshotDao.delete(snapshot.getId())){
+        	    if(_snapshotDao.expunge(snapshot.getId())){
         	    	_accountMgr.decrementResourceCount(accountId, ResourceType.snapshot);
         	    	
         	        //Log event after successful deletion
@@ -1352,7 +1357,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
     	// We can only delete the schedules in the future, not the ones which are already executing.
     	SnapshotScheduleVO snapshotSchedule = _snapshotScheduleDao.getCurrentSchedule(volumeId, Snapshot.MANUAL_POLICY_ID, false);
     	if (snapshotSchedule != null) {
-    	    _snapshotScheduleDao.delete(snapshotSchedule.getId());
+    	    _snapshotScheduleDao.expunge(snapshotSchedule.getId());
     	}
 	}
     
@@ -1411,7 +1416,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
             }
         }
         if(success){
-            _snapshotDao.delete(snapshot.getId());
+            _snapshotDao.expunge(snapshot.getId());
         }
         return success;
     }
@@ -1438,7 +1443,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
                 if (!_domainDao.isChildDomain(account.getDomainId(), userAccount.getDomainId())) {
                     throw new PermissionDeniedException("Unable to list snapshot schedule for volume " + volumeId + ", permission denied.");
                 }
-            } else if (account.getId().longValue() != volAcctId) {
+            } else if (account.getId() != volAcctId) {
                 throw new PermissionDeniedException("Unable to list snapshot schedule, account " + account.getAccountName() + " does not own volume id " + volAcctId);
             }
         }
@@ -1485,7 +1490,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
         policySearch.and("policyId", policySearch.entity().getPolicyId(), SearchCriteria.Op.EQ);
         
         PolicySnapshotSearch = _snapshotDao.createSearchBuilder();
-        PolicySnapshotSearch.join("policy", policySearch, policySearch.entity().getSnapshotId(), PolicySnapshotSearch.entity().getId());
+        PolicySnapshotSearch.join("policy", policySearch, policySearch.entity().getSnapshotId(), PolicySnapshotSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         policySearch.done();
         PolicySnapshotSearch.done();
         
@@ -1494,7 +1499,7 @@ public class SnapshotManagerImpl implements SnapshotManager {
         SearchBuilder policyRefSearch = _snapPolicyRefDao.createSearchBuilder();
         policyRefSearch.and("snapshotId", policyRefSearch.entity().getSnapshotId(), SearchCriteria.Op.EQ);
         
-        PoliciesForSnapSearch.join("policyRef", policyRefSearch, policyRefSearch.entity().getPolicyId(), PoliciesForSnapSearch.entity().getId());
+        PoliciesForSnapSearch.join("policyRef", policyRefSearch, policyRefSearch.entity().getPolicyId(), PoliciesForSnapSearch.entity().getId(), JoinBuilder.JoinType.INNER);
         policyRefSearch.done();
         PoliciesForSnapSearch.done();
 
diff --git a/server/src/com/cloud/storage/snapshot/SnapshotSchedulerImpl.java b/server/src/com/cloud/storage/snapshot/SnapshotSchedulerImpl.java
index 4a3a4717fa5..ca8c43cbc48 100644
--- a/server/src/com/cloud/storage/snapshot/SnapshotSchedulerImpl.java
+++ b/server/src/com/cloud/storage/snapshot/SnapshotSchedulerImpl.java
@@ -326,7 +326,7 @@ public class SnapshotSchedulerImpl implements SnapshotScheduler {
         if (_snapshotScheduleDao.findById(expectedId) != null) {
             // We need to acquire a lock and delete it, then release the lock.
             // But I don't know how to.
-            _snapshotScheduleDao.delete(expectedId);
+            _snapshotScheduleDao.expunge(expectedId);
         }
         if (policyId.longValue() == Snapshot.MANUAL_POLICY_ID) {
             // Don't need to schedule the next job for this.
@@ -351,7 +351,7 @@ public class SnapshotSchedulerImpl implements SnapshotScheduler {
         SnapshotScheduleVO schedule = _snapshotScheduleDao.getCurrentSchedule(volumeId, policyId, false);
         boolean success = true;
         if (schedule != null) {
-            success = _snapshotScheduleDao.delete(schedule.getId());
+            success = _snapshotScheduleDao.expunge(schedule.getId());
         }
         if(!success){
             s_logger.debug("Error while deleting Snapshot schedule with Id: "+schedule.getId());
diff --git a/server/src/com/cloud/storage/upload/NotUploadedState.java b/server/src/com/cloud/storage/upload/NotUploadedState.java
new file mode 100644
index 00000000000..76f3d8fab72
--- /dev/null
+++ b/server/src/com/cloud/storage/upload/NotUploadedState.java
@@ -0,0 +1,23 @@
+package com.cloud.storage.upload;
+
+import com.cloud.agent.api.storage.UploadProgressCommand.RequestType;
+import com.cloud.storage.VMTemplateStorageResourceAssoc.Status;
+
+public class NotUploadedState extends UploadActiveState {
+	
+	public NotUploadedState(UploadListener uploadListener) {
+		super(uploadListener);
+	}	
+
+	@Override
+	public String getName() {
+		return Status.NOT_UPLOADED.toString();
+	}
+
+	@Override
+	public void onEntry(String prevState, UploadEvent event, Object evtObj) {
+		super.onEntry(prevState, event, evtObj);
+		getUploadListener().scheduleStatusCheck(RequestType.GET_STATUS);
+	}
+	
+}
diff --git a/server/src/com/cloud/storage/upload/UploadAbandonedState.java b/server/src/com/cloud/storage/upload/UploadAbandonedState.java
new file mode 100644
index 00000000000..ac3fa03ad64
--- /dev/null
+++ b/server/src/com/cloud/storage/upload/UploadAbandonedState.java
@@ -0,0 +1,27 @@
+package com.cloud.storage.upload;
+
+import com.cloud.agent.api.storage.UploadProgressCommand.RequestType;
+import com.cloud.storage.Upload.Status;
+
+public class UploadAbandonedState extends UploadInactiveState {
+
+	public UploadAbandonedState(UploadListener dl) {
+		super(dl);
+	}
+
+	@Override
+	public String getName() {
+		return Status.ABANDONED.toString();
+	}
+
+	@Override
+	public void onEntry(String prevState, UploadEvent event, Object evtObj) {
+		super.onEntry(prevState, event, evtObj);
+		if (!prevState.equalsIgnoreCase(getName())){
+			getUploadListener().updateDatabase(Status.ABANDONED, "Upload canceled");
+			getUploadListener().cancelStatusTask();
+			getUploadListener().cancelTimeoutTask();
+			getUploadListener().sendCommand(RequestType.ABORT);
+		}
+	}
+}
diff --git a/server/src/com/cloud/storage/upload/UploadActiveState.java b/server/src/com/cloud/storage/upload/UploadActiveState.java
new file mode 100644
index 00000000000..a249906a9ec
--- /dev/null
+++ b/server/src/com/cloud/storage/upload/UploadActiveState.java
@@ -0,0 +1,95 @@
+package com.cloud.storage.upload;
+
+import org.apache.log4j.Level;
+
+import com.cloud.agent.api.storage.DownloadAnswer;
+import com.cloud.agent.api.storage.UploadAnswer;
+import com.cloud.agent.api.storage.UploadAnswer;
+import com.cloud.agent.api.storage.UploadProgressCommand.RequestType;
+import com.cloud.storage.VMTemplateStorageResourceAssoc.Status;
+import com.cloud.storage.download.DownloadState.DownloadEvent;
+
+public abstract class UploadActiveState extends UploadState {
+	
+	public UploadActiveState(UploadListener ul) {
+		super(ul);
+	}
+	
+	@Override
+	public  String handleAbort(){
+		return Status.ABANDONED.toString();
+	}
+	
+	@Override
+	public  String handleDisconnect(){
+
+		return Status.UPLOAD_ERROR.toString();
+	}
+
+
+	@Override
+	public String handleAnswer(UploadAnswer answer) {
+		if (s_logger.isDebugEnabled()) {
+			s_logger.debug("handleAnswer, answer status=" + answer.getUploadStatus() + ", curr state=" + getName());
+		}
+		switch (answer.getUploadStatus()) {
+		case UPLOAD_IN_PROGRESS:
+			getUploadListener().scheduleStatusCheck(RequestType.GET_STATUS);
+			return Status.UPLOAD_IN_PROGRESS.toString();
+		case UPLOADED:
+			getUploadListener().scheduleImmediateStatusCheck(RequestType.PURGE);
+			getUploadListener().cancelTimeoutTask();
+			return Status.UPLOADED.toString();
+		case NOT_UPLOADED:
+			getUploadListener().scheduleStatusCheck(RequestType.GET_STATUS);
+			return Status.NOT_UPLOADED.toString();
+		case UPLOAD_ERROR:
+			getUploadListener().cancelStatusTask();
+			getUploadListener().cancelTimeoutTask();
+			return Status.UPLOAD_ERROR.toString();
+		case UNKNOWN:
+			getUploadListener().cancelStatusTask();
+			getUploadListener().cancelTimeoutTask();
+			return Status.UPLOAD_ERROR.toString();
+		default:
+			return null;
+		}
+	}
+
+	@Override
+	public String handleTimeout(long updateMs) {
+		if (s_logger.isTraceEnabled()) {
+			getUploadListener().log("handleTimeout, updateMs=" + updateMs + ", curr state= " + getName(), Level.TRACE);
+		}
+		String newState = this.getName();
+		if (updateMs > 5*UploadListener.STATUS_POLL_INTERVAL){
+			newState=Status.UPLOAD_ERROR.toString();
+			getUploadListener().log("timeout: transitioning to upload error state, currstate=" + getName(), Level.DEBUG );
+		} else if (updateMs > 3*UploadListener.STATUS_POLL_INTERVAL) {
+			getUploadListener().cancelStatusTask();
+			getUploadListener().scheduleImmediateStatusCheck(RequestType.GET_STATUS);
+			getUploadListener().scheduleTimeoutTask(3*UploadListener.STATUS_POLL_INTERVAL);
+			getUploadListener().log(getName() + " first timeout: checking again ", Level.DEBUG );
+		} else {
+			getUploadListener().scheduleTimeoutTask(3*UploadListener.STATUS_POLL_INTERVAL);
+		}
+		return newState;
+	}
+	
+	@Override
+	public  void onEntry(String prevState, UploadEvent event, Object evtObj) {
+		if (s_logger.isTraceEnabled()) {
+			getUploadListener().log("onEntry, prev state= " + prevState + ", curr state=" + getName() + ", event=" + event, Level.TRACE);
+		}
+		
+		if (event == UploadEvent.UPLOAD_ANSWER) {
+			getUploadListener().updateDatabase((UploadAnswer)evtObj);
+			getUploadListener().setLastUpdated();
+		}
+		
+	}
+	
+	@Override
+	public  void onExit() {
+	}
+}
diff --git a/server/src/com/cloud/storage/upload/UploadCompleteState.java b/server/src/com/cloud/storage/upload/UploadCompleteState.java
new file mode 100644
index 00000000000..4973e79187d
--- /dev/null
+++ b/server/src/com/cloud/storage/upload/UploadCompleteState.java
@@ -0,0 +1,30 @@
+package com.cloud.storage.upload;
+
+import com.cloud.agent.api.storage.UploadProgressCommand.RequestType;
+import com.cloud.storage.Upload.Status;
+
+public class UploadCompleteState extends UploadInactiveState {
+
+	public UploadCompleteState(UploadListener ul) {
+		super(ul);
+	}
+
+	@Override
+	public String getName() {
+		return Status.UPLOADED.toString();
+
+	}
+
+
+	@Override
+	public void onEntry(String prevState, UploadEvent event, Object evtObj) {
+		super.onEntry(prevState, event, evtObj);
+		if (! prevState.equals(getName())) {
+			if (event == UploadEvent.UPLOAD_ANSWER){
+				getUploadListener().scheduleImmediateStatusCheck(RequestType.PURGE);
+			}
+			getUploadListener().setUploadInactive(Status.UPLOADED);
+		}
+		
+	}
+}
diff --git a/server/src/com/cloud/storage/upload/UploadErrorState.java b/server/src/com/cloud/storage/upload/UploadErrorState.java
new file mode 100644
index 00000000000..5fd872e6580
--- /dev/null
+++ b/server/src/com/cloud/storage/upload/UploadErrorState.java
@@ -0,0 +1,73 @@
+package com.cloud.storage.upload;
+
+import org.apache.log4j.Level;
+
+import com.cloud.agent.api.storage.UploadAnswer;
+import com.cloud.agent.api.storage.UploadProgressCommand.RequestType;
+import com.cloud.storage.Upload.Status;
+
+public class UploadErrorState extends UploadInactiveState {
+
+	public UploadErrorState(UploadListener ul) {
+		super(ul);
+	}
+
+	@Override
+	public String handleAnswer(UploadAnswer answer) {
+		switch (answer.getUploadStatus()) {
+		case UPLOAD_IN_PROGRESS:
+			getUploadListener().scheduleStatusCheck(RequestType.GET_STATUS);
+			return Status.UPLOAD_IN_PROGRESS.toString();
+		case UPLOADED:
+			getUploadListener().scheduleImmediateStatusCheck(RequestType.PURGE);
+			getUploadListener().cancelTimeoutTask();
+			return Status.UPLOADED.toString();
+		case NOT_UPLOADED:
+			getUploadListener().scheduleStatusCheck(RequestType.GET_STATUS);
+			return Status.NOT_UPLOADED.toString();
+		case UPLOAD_ERROR:
+			getUploadListener().cancelStatusTask();
+			getUploadListener().cancelTimeoutTask();
+			return Status.UPLOAD_ERROR.toString();
+		case UNKNOWN:
+			getUploadListener().cancelStatusTask();
+			getUploadListener().cancelTimeoutTask();
+			return Status.UPLOAD_ERROR.toString();
+		default:
+			return null;
+		}
+	}
+
+
+
+	@Override
+	public String handleAbort() {
+		return Status.ABANDONED.toString();
+	}
+
+
+	@Override
+	public String getName() {
+		return Status.UPLOAD_ERROR.toString();
+	}
+
+
+	@Override
+	public void onEntry(String prevState, UploadEvent event, Object evtObj) {
+		super.onEntry(prevState, event, evtObj);
+		if (event==UploadEvent.DISCONNECT){
+			getUploadListener().logDisconnect();
+			getUploadListener().cancelStatusTask();
+			getUploadListener().cancelTimeoutTask();
+			getUploadListener().updateDatabase(Status.UPLOAD_ERROR, "Storage agent or storage VM disconnected");  
+			getUploadListener().log("Entering upload error state because the storage host disconnected", Level.WARN);
+		} else if (event==UploadEvent.TIMEOUT_CHECK){
+			getUploadListener().updateDatabase(Status.UPLOAD_ERROR, "Timeout waiting for response from storage host");
+			getUploadListener().log("Entering upload error state: timeout waiting for response from storage host", Level.WARN);
+		}
+		getUploadListener().setUploadInactive(Status.UPLOAD_ERROR);
+	}
+
+
+
+}
diff --git a/server/src/com/cloud/storage/upload/UploadInProgressState.java b/server/src/com/cloud/storage/upload/UploadInProgressState.java
new file mode 100644
index 00000000000..e72f3db42cf
--- /dev/null
+++ b/server/src/com/cloud/storage/upload/UploadInProgressState.java
@@ -0,0 +1,23 @@
+package com.cloud.storage.upload;
+
+import com.cloud.storage.VMTemplateStorageResourceAssoc.Status;
+
+public class UploadInProgressState extends UploadActiveState {
+
+	public UploadInProgressState(UploadListener dl) {
+		super(dl);
+	}
+
+	@Override
+	public String getName() {
+		return Status.UPLOAD_IN_PROGRESS.toString();
+	}
+
+	@Override
+	public void onEntry(String prevState, UploadEvent event, Object evtObj) {
+		super.onEntry(prevState, event, evtObj);
+		if (!prevState.equals(getName()))
+			getUploadListener().logUploadStart();
+	}
+
+}
diff --git a/server/src/com/cloud/storage/upload/UploadInactiveState.java b/server/src/com/cloud/storage/upload/UploadInactiveState.java
new file mode 100644
index 00000000000..f70f3d8d0e3
--- /dev/null
+++ b/server/src/com/cloud/storage/upload/UploadInactiveState.java
@@ -0,0 +1,35 @@
+package com.cloud.storage.upload;
+
+import com.cloud.agent.api.storage.DownloadAnswer;
+import com.cloud.agent.api.storage.UploadAnswer;
+
+public abstract class UploadInactiveState extends UploadState {
+
+	public UploadInactiveState(UploadListener ul) {
+		super(ul);
+	}
+
+	@Override
+	public String handleAnswer(UploadAnswer answer) {
+		// ignore and stay put
+		return getName();
+	}
+
+	@Override
+	public String handleAbort() {
+		// ignore and stay put
+		return getName();
+	}
+
+	@Override
+	public String handleDisconnect() {
+		//ignore and stay put
+		return getName();
+	}
+
+	@Override
+	public String handleTimeout(long updateMs) {
+		// ignore and stay put
+		return getName();
+	}
+}
diff --git a/server/src/com/cloud/storage/upload/UploadListener.java b/server/src/com/cloud/storage/upload/UploadListener.java
new file mode 100755
index 00000000000..b956074df66
--- /dev/null
+++ b/server/src/com/cloud/storage/upload/UploadListener.java
@@ -0,0 +1,423 @@
+package com.cloud.storage.upload;
+
+
+import java.util.Date;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Timer;
+import java.util.TimerTask;
+
+import org.apache.log4j.Level;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.Listener;
+import com.cloud.agent.api.AgentControlAnswer;
+import com.cloud.agent.api.AgentControlCommand;
+import com.cloud.agent.api.Answer;
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartupCommand;
+import com.cloud.agent.api.StartupStorageCommand;
+import com.cloud.agent.api.storage.UploadAnswer;
+import com.cloud.agent.api.storage.UploadCommand;
+import com.cloud.agent.api.storage.UploadProgressCommand;
+import com.cloud.agent.api.storage.UploadProgressCommand.RequestType;
+import com.cloud.async.AsyncJobManager;
+import com.cloud.async.AsyncJobResult;
+import com.cloud.async.executor.ExtractJobResultObject;
+import com.cloud.event.EventTypes;
+import com.cloud.event.EventVO;
+import com.cloud.host.HostVO;
+import com.cloud.storage.Storage;
+import com.cloud.storage.UploadVO;
+import com.cloud.storage.dao.UploadDao;
+import com.cloud.storage.Upload.Status;
+import com.cloud.storage.Upload.Type;
+import com.cloud.storage.upload.UploadMonitorImpl;
+import com.cloud.storage.upload.UploadState.UploadEvent;
+import com.cloud.utils.exception.CloudRuntimeException;
+
+public class UploadListener implements Listener {
+	
+
+	private static final class StatusTask extends TimerTask {
+		private final UploadListener ul;
+		private final RequestType reqType;
+		
+		public StatusTask( UploadListener ul,  RequestType req) {
+			this.reqType = req;
+			this.ul = ul;
+		}
+
+		@Override
+		public void run() {
+		  ul.sendCommand(reqType);
+
+		}
+	}
+	
+	private static final class TimeoutTask extends TimerTask {
+		private final UploadListener ul;
+		
+		public TimeoutTask( UploadListener ul) {
+			this.ul = ul;
+		}
+
+		@Override
+		public void run() {
+		  ul.checkProgress();
+		}
+	}
+
+	public static final Logger s_logger = Logger.getLogger(UploadListener.class.getName());
+	public static final int SMALL_DELAY = 100;
+	public static final long STATUS_POLL_INTERVAL = 10000L;
+	
+	public static final String UPLOADED=Status.UPLOADED.toString();
+	public static final String NOT_UPLOADED=Status.NOT_UPLOADED.toString();
+	public static final String UPLOAD_ERROR=Status.UPLOAD_ERROR.toString();
+	public static final String UPLOAD_IN_PROGRESS=Status.UPLOAD_IN_PROGRESS.toString();
+	public static final String UPLOAD_ABANDONED=Status.ABANDONED.toString();
+
+
+	private HostVO sserver;	
+	
+	private boolean uploadActive = true;
+	
+	private UploadDao uploadDao;
+	
+	private final UploadMonitorImpl uploadMonitor;
+	
+	private UploadState currState;
+	
+	private UploadCommand cmd;
+
+	private Timer timer;
+
+	private StatusTask statusTask;
+	private TimeoutTask timeoutTask;
+	private Date lastUpdated = new Date();
+	private String jobId;
+	private Long accountId;
+	private String typeName;
+	private Type type;
+	private long asyncJobId;
+	private long eventId;
+	private AsyncJobManager asyncMgr;
+	private ExtractJobResultObject resultObj;
+	
+	public AsyncJobManager getAsyncMgr() {
+		return asyncMgr;
+	}
+
+	public void setAsyncMgr(AsyncJobManager asyncMgr) {
+		this.asyncMgr = asyncMgr;
+	}
+
+	public long getAsyncJobId() {
+		return asyncJobId;
+	}
+
+	public void setAsyncJobId(long asyncJobId) {
+		this.asyncJobId = asyncJobId;
+	}
+
+	public long getEventId() {
+		return eventId;
+	}
+
+	public void setEventId(long eventId) {
+		this.eventId = eventId;
+	}
+
+	private final Map stateMap = new HashMap();
+	private Long uploadId;	
+	
+	public UploadListener(HostVO host, Timer _timer, UploadDao uploadDao,
+			Long uploadId, UploadMonitorImpl uploadMonitor, UploadCommand cmd,
+			Long accountId, String typeName, Type type, long eventId, long asyncJobId, AsyncJobManager asyncMgr) {
+		this.sserver = host;				
+		this.uploadDao = uploadDao;
+		this.uploadMonitor = uploadMonitor;
+		this.cmd = cmd;
+		this.uploadId = uploadId;
+		this.accountId = accountId;
+		this.typeName = typeName;
+		this.type = type;
+		initStateMachine();
+		this.currState = getState(Status.NOT_UPLOADED.toString());
+		this.timer = _timer;
+		this.timeoutTask = new TimeoutTask(this);
+		this.timer.schedule(timeoutTask, 3*STATUS_POLL_INTERVAL);
+		this.eventId = eventId;
+		this.asyncJobId = asyncJobId;
+		this.asyncMgr = asyncMgr;
+		this.resultObj = new ExtractJobResultObject(accountId, typeName, Status.NOT_UPLOADED.toString(), 0, uploadId);
+		updateDatabase(Status.NOT_UPLOADED, cmd.getUrl(),"");
+	}
+	
+	public UploadListener(UploadMonitorImpl monitor) {
+	    uploadMonitor = monitor;
+	}	
+	
+	public void checkProgress() {
+		transition(UploadEvent.TIMEOUT_CHECK, null);
+	}
+
+	@Override
+	public int getTimeout() {
+		return -1;
+	}
+
+	@Override
+	public boolean isRecurring() {
+		return false;
+	}
+
+	public void setCommand(UploadCommand _cmd) {
+		this.cmd = _cmd;
+	}
+	
+	public void setJobId(String _jobId) {
+		this.jobId = _jobId;
+	}
+	
+	public String getJobId() {
+		return jobId;
+	}
+	
+	@Override
+	public boolean processAnswer(long agentId, long seq, Answer[] answers) {
+		boolean processed = false;
+    	if(answers != null & answers.length > 0) {
+    		if(answers[0] instanceof UploadAnswer) {
+    			final UploadAnswer answer = (UploadAnswer)answers[0];
+    			if (getJobId() == null) {
+    				setJobId(answer.getJobId());
+    			} else if (!getJobId().equalsIgnoreCase(answer.getJobId())){
+    				return false;//TODO
+    			}
+    			transition(UploadEvent.UPLOAD_ANSWER, answer);
+    			processed = true;
+    		}
+    	}
+        return processed;
+	}
+	
+
+	@Override
+	public boolean processCommand(long agentId, long seq, Command[] commands) {
+		return false;
+	}
+
+	@Override
+	public boolean processConnect(HostVO agent, StartupCommand cmd) {		
+		if (!(cmd instanceof StartupStorageCommand)) {
+	        return true;
+	    }
+	   
+	    long agentId = agent.getId();
+	    
+	    StartupStorageCommand storage = (StartupStorageCommand)cmd;
+	    if (storage.getResourceType() == Storage.StorageResourceType.STORAGE_HOST ||
+	    storage.getResourceType() == Storage.StorageResourceType.SECONDARY_STORAGE )
+	    {
+	    	uploadMonitor.handleUploadSync(agentId);
+	    }
+		return true;
+	}
+
+	@Override
+	public AgentControlAnswer processControlCommand(long agentId,
+			AgentControlCommand cmd) {
+		return null;
+	}
+	
+	public void setUploadInactive(Status reason) {
+		uploadActive=false;
+		uploadMonitor.handleUploadEvent(sserver, accountId, typeName, type, uploadId, reason, eventId);
+	}
+	
+	public void logUploadStart() {
+		String event = uploadMonitor.getEvent(type);
+		//uploadMonitor.logEvent(accountId, event, "Storage server " + sserver.getName() + " started upload of " +type.toString() + " " + typeName, EventVO.LEVEL_INFO, eventId);
+	}
+	
+	public void cancelTimeoutTask() {
+		if (timeoutTask != null) timeoutTask.cancel();
+	}
+	
+	public void cancelStatusTask() {
+		if (statusTask != null) statusTask.cancel();
+	}
+
+	@Override
+	public boolean processDisconnect(long agentId, com.cloud.host.Status state) {	
+		setDisconnected();
+		return true;
+	}
+
+	@Override
+	public boolean processTimeout(long agentId, long seq) {		
+		return true;
+	}
+	
+	private void initStateMachine() {
+		stateMap.put(Status.NOT_UPLOADED.toString(), new NotUploadedState(this));
+		stateMap.put(Status.UPLOADED.toString(), new UploadCompleteState(this));
+		stateMap.put(Status.UPLOAD_ERROR.toString(), new UploadErrorState(this));
+		stateMap.put(Status.UPLOAD_IN_PROGRESS.toString(), new UploadInProgressState(this));
+		stateMap.put(Status.ABANDONED.toString(), new UploadAbandonedState(this));
+	}
+	
+	private UploadState getState(String stateName) {
+		return stateMap.get(stateName);
+	}
+
+	private synchronized void transition(UploadEvent event, Object evtObj) {
+	    if (currState == null) {
+	        return;
+	    }
+		String prevName = currState.getName();
+		String nextState = currState.handleEvent(event, evtObj);
+		if (nextState != null) {
+			currState = getState(nextState);
+			if (currState != null) {
+				currState.onEntry(prevName, event, evtObj);
+			} else {
+				throw new CloudRuntimeException("Invalid next state: currState="+prevName+", evt="+event + ", next=" + nextState);
+			}
+		} else {
+			throw new CloudRuntimeException("Unhandled event transition: currState="+prevName+", evt="+event);
+		}
+	}
+	
+	public Date getLastUpdated() {
+		return lastUpdated;
+	}
+	
+	public void setLastUpdated() {
+		lastUpdated  = new Date();
+	}
+	
+	public void log(String message, Level level) {
+		s_logger.log(level, message + ", " + type.toString() + " = " + typeName + " at host " + sserver.getName());
+	}
+
+	public void setDisconnected() {
+		transition(UploadEvent.DISCONNECT, null);
+	}
+	
+	public void scheduleStatusCheck(com.cloud.agent.api.storage.UploadProgressCommand.RequestType getStatus) {
+		if (statusTask != null) statusTask.cancel();
+
+		statusTask = new StatusTask(this, getStatus);
+		timer.schedule(statusTask, STATUS_POLL_INTERVAL);
+	}
+
+	public void scheduleTimeoutTask(long delay) {
+		if (timeoutTask != null) timeoutTask.cancel();
+
+		timeoutTask = new TimeoutTask(this);
+		timer.schedule(timeoutTask, delay);
+		if (s_logger.isDebugEnabled()) {
+			log("Scheduling timeout at " + delay + " ms", Level.DEBUG);
+		}
+	}
+	
+	public void updateDatabase(Status state, String uploadErrorString) {
+		resultObj.setResult_string(uploadErrorString);
+		resultObj.setState(state.toString());
+		asyncMgr.updateAsyncJobAttachment(asyncJobId, type.toString(), 1L);
+		asyncMgr.updateAsyncJobStatus(asyncJobId, AsyncJobResult.STATUS_IN_PROGRESS, resultObj);
+		
+		UploadVO vo = uploadDao.createForUpdate();
+		vo.setUploadState(state);
+		vo.setLastUpdated(new Date());
+		vo.setErrorString(uploadErrorString);
+		uploadDao.update(getUploadId(), vo);
+	}
+	
+	public void updateDatabase(Status state, String uploadUrl,String uploadErrorString) {
+		resultObj.setResult_string(uploadErrorString);
+		resultObj.setState(state.toString());
+		asyncMgr.updateAsyncJobAttachment(asyncJobId, type.toString(), 1L);
+		asyncMgr.updateAsyncJobStatus(asyncJobId, AsyncJobResult.STATUS_IN_PROGRESS, resultObj);
+		
+		
+		UploadVO vo = uploadDao.createForUpdate();
+		vo.setUploadState(state);
+		vo.setLastUpdated(new Date());
+		vo.setUploadUrl(uploadUrl);
+		vo.setJobId(null);
+		vo.setUploadPercent(0);
+		vo.setErrorString(uploadErrorString);
+		
+		uploadDao.update(getUploadId(), vo);
+	}
+	
+	private Long getUploadId() {
+		return uploadId;
+	}
+
+	public synchronized void updateDatabase(UploadAnswer answer) {		
+		
+	    if(answer.getErrorString().startsWith("553")){
+	        answer.setErrorString(answer.getErrorString().concat(". Please check if the file name already exists."));
+	    }
+		resultObj.setResult_string(answer.getErrorString());
+		resultObj.setState(answer.getUploadStatus().toString());
+		resultObj.setUploadPercent(answer.getUploadPct());
+		
+		if (answer.getUploadStatus() == Status.UPLOAD_IN_PROGRESS){
+			asyncMgr.updateAsyncJobAttachment(asyncJobId, type.toString(), 1L);
+			asyncMgr.updateAsyncJobStatus(asyncJobId, AsyncJobResult.STATUS_IN_PROGRESS, resultObj);
+		}else if(answer.getUploadStatus() == Status.UPLOADED){
+			asyncMgr.completeAsyncJob(asyncJobId, AsyncJobResult.STATUS_SUCCEEDED, 0, resultObj);
+		}else{
+			asyncMgr.completeAsyncJob(asyncJobId, AsyncJobResult.STATUS_FAILED, 0, resultObj);
+		}
+        UploadVO updateBuilder = uploadDao.createForUpdate();
+		updateBuilder.setUploadPercent(answer.getUploadPct());
+		updateBuilder.setUploadState(answer.getUploadStatus());
+		updateBuilder.setLastUpdated(new Date());
+		updateBuilder.setErrorString(answer.getErrorString());
+		updateBuilder.setJobId(answer.getJobId());
+		
+		uploadDao.update(getUploadId(), updateBuilder);
+	}
+
+	public void sendCommand(RequestType reqType) {
+		if (getJobId() != null) {
+			if (s_logger.isTraceEnabled()) {
+				log("Sending progress command ", Level.TRACE);
+			}
+			long sent = uploadMonitor.send(sserver.getId(), new UploadProgressCommand(getCommand(), getJobId(), reqType), this);
+			if (sent == -1) {
+				setDisconnected();
+			}
+		}
+		
+	}
+	
+	private UploadCommand getCommand() {
+		return cmd;
+	}
+
+	public void logDisconnect() {
+		s_logger.warn("Unable to monitor upload progress of " + typeName + " at host " + sserver.getName());
+		String event;
+		event = uploadMonitor.getEvent(type);
+		uploadMonitor.logEvent(accountId, event, "Storage server " + sserver.getName() + " disconnected during upload of " + typeName, EventVO.LEVEL_WARN, eventId);
+	}
+	
+	public void scheduleImmediateStatusCheck(RequestType request) {
+		if (statusTask != null) statusTask.cancel();
+		statusTask = new StatusTask(this, request);
+		timer.schedule(statusTask, SMALL_DELAY);
+	}
+
+	public void setCurrState(Status uploadState) {
+		this.currState = getState(currState.toString());		
+	}
+	
+}
diff --git a/server/src/com/cloud/storage/upload/UploadMonitor.java b/server/src/com/cloud/storage/upload/UploadMonitor.java
new file mode 100755
index 00000000000..be5d8c7769d
--- /dev/null
+++ b/server/src/com/cloud/storage/upload/UploadMonitor.java
@@ -0,0 +1,58 @@
+/**
+ *  Copyright (C) 2010 Cloud.com, Inc.  All rights reserved.
+ * 
+ * This software is licensed under the GNU General Public License v3 or later.
+ * 
+ * It is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ * 
+ */
+
+package com.cloud.storage.upload;
+
+import java.util.Map;
+
+import com.cloud.async.AsyncJobManager;
+import com.cloud.host.HostVO;
+import com.cloud.storage.UploadVO;
+import com.cloud.storage.VMTemplateHostVO;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.Upload.Status;
+import com.cloud.storage.Upload.Type;
+import com.cloud.storage.template.TemplateInfo;
+import com.cloud.utils.component.Manager;
+
+/**
+ * Monitor upload progress of all entities.
+ * @author nitin
+ *
+ */
+public interface UploadMonitor extends Manager{		
+	
+	public void cancelAllUploads(Long templateId);
+
+	public void extractTemplate(VMTemplateVO template, String url,
+			VMTemplateHostVO tmpltHostRef,Long dataCenterId, long eventId, long asyncJobId, AsyncJobManager asyncMgr);
+
+    boolean isTypeUploadInProgress(Long typeId, Type type);
+
+    void handleUploadSync(long sserverId);
+
+    UploadVO createNewUploadEntry(Long hostId, Long typeId, Status uploadState,
+            int uploadPercent, Type type, String errorString, String jobId,
+            String uploadUrl);
+
+    void extractVolume(UploadVO uploadVolumeObj, HostVO sserver, VolumeVO volume, String url,
+            Long dataCenterId, String installPath, long eventId,
+            long asyncJobId, AsyncJobManager asyncMgr);
+
+}
\ No newline at end of file
diff --git a/server/src/com/cloud/storage/upload/UploadMonitorImpl.java b/server/src/com/cloud/storage/upload/UploadMonitorImpl.java
new file mode 100755
index 00000000000..a1424dead92
--- /dev/null
+++ b/server/src/com/cloud/storage/upload/UploadMonitorImpl.java
@@ -0,0 +1,271 @@
+package com.cloud.storage.upload;
+
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.List;
+import java.util.Map;
+import java.util.Timer;
+import java.util.concurrent.ConcurrentHashMap;
+
+import javax.ejb.Local;
+import javax.naming.ConfigurationException;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.Listener;
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.storage.UploadCommand;
+import com.cloud.agent.api.storage.UploadProgressCommand.RequestType;
+import com.cloud.agent.manager.AgentManager;
+import com.cloud.async.AsyncJobManager;
+import com.cloud.configuration.dao.ConfigurationDao;
+import com.cloud.event.EventTypes;
+import com.cloud.event.EventVO;
+import com.cloud.event.dao.EventDao;
+import com.cloud.host.Host;
+import com.cloud.host.HostVO;
+import com.cloud.host.dao.HostDao;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.Upload;
+import com.cloud.storage.Upload.Type;
+import com.cloud.storage.UploadVO;
+import com.cloud.storage.VMTemplateHostVO;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.UploadDao;
+import com.cloud.storage.dao.VMTemplateDao;
+import com.cloud.storage.dao.VMTemplateHostDao;
+import com.cloud.utils.component.Inject;
+import com.cloud.vm.dao.SecondaryStorageVmDao;
+
+/**
+ * @author nitin
+ * Monitors the progress of upload.
+ */
+@Local(value={UploadMonitor.class})
+public class UploadMonitorImpl implements UploadMonitor {
+
+	static final Logger s_logger = Logger.getLogger(UploadMonitorImpl.class);
+	
+	private String _hyperVisorType;
+    @Inject 
+    VMTemplateHostDao _vmTemplateHostDao;
+    @Inject 
+    UploadDao _uploadDao;
+    @Inject
+    SecondaryStorageVmDao _secStorageVmDao;
+
+    
+    @Inject
+    HostDao _serverDao = null;    
+    @Inject
+    VMTemplateDao _templateDao =  null;
+    @Inject
+	private final EventDao _eventDao = null;
+    @Inject
+	private AgentManager _agentMgr;
+    @Inject
+    ConfigurationDao _configDao;
+
+	private String _name;
+	private Boolean _sslCopy = new Boolean(false);
+	private String _copyAuthPasswd;
+
+
+	Timer _timer;
+
+	final Map _listenerMap = new ConcurrentHashMap();
+
+	
+	@Override
+	public void cancelAllUploads(Long templateId) {
+		// TODO
+
+	}	
+	
+	@Override
+	public boolean isTypeUploadInProgress(Long typeId, Type type) {
+		List uploadsInProgress =
+			_uploadDao.listByTypeUploadStatus(typeId, type, UploadVO.Status.UPLOAD_IN_PROGRESS);
+		
+		if(uploadsInProgress.size() > 0)
+		    return true;
+		else if (type == Type.VOLUME && _uploadDao.listByTypeUploadStatus(typeId, type, UploadVO.Status.COPY_IN_PROGRESS).size() > 0){
+		    return true;
+		}
+		return false;
+		
+	}
+	
+	@Override
+	public UploadVO createNewUploadEntry(Long hostId, Long typeId, UploadVO.Status  uploadState,
+	                                        int uploadPercent, Type  type, 
+	                                        String  errorString, String  jobId, String  uploadUrl){
+	       
+        UploadVO uploadObj = new UploadVO(hostId, typeId, new Date(), 
+                            uploadState, 0, type, null, "jobid0000", uploadUrl);
+        _uploadDao.persist(uploadObj);
+        
+        return uploadObj;
+	    
+	}
+	
+	@Override
+	public void extractVolume(UploadVO uploadVolumeObj, HostVO sserver, VolumeVO volume, String url, Long dataCenterId, String installPath, long eventId, long asyncJobId, AsyncJobManager asyncMgr){				
+						
+		uploadVolumeObj.setUploadState(Upload.Status.NOT_UPLOADED);
+		_uploadDao.update(uploadVolumeObj.getId(), uploadVolumeObj);
+				
+	    start();		
+		UploadCommand ucmd = new UploadCommand(url, volume.getId(), volume.getSize(), installPath, Type.VOLUME);
+		UploadListener ul = new UploadListener(sserver, _timer, _uploadDao, uploadVolumeObj.getId(), this, ucmd, volume.getAccountId(), volume.getName(), Type.VOLUME, eventId, asyncJobId, asyncMgr);
+		_listenerMap.put(uploadVolumeObj, ul);
+
+		long result = send(sserver.getId(), ucmd, ul);	
+		if (result == -1) {
+			s_logger.warn("Unable to start upload of volume " + volume.getName() + " from " + sserver.getName() + " to " +url);
+			ul.setDisconnected();
+			ul.scheduleStatusCheck(RequestType.GET_OR_RESTART);
+		}				
+		
+	}
+
+	@Override
+	public void extractTemplate( VMTemplateVO template, String url,
+			VMTemplateHostVO vmTemplateHost,Long dataCenterId, long eventId, long asyncJobId, AsyncJobManager asyncMgr){
+
+		Type type = (template.getFormat() == ImageFormat.ISO) ? Type.ISO : Type.TEMPLATE ;
+				
+		List storageServers = _serverDao.listByTypeDataCenter(Host.Type.SecondaryStorage, dataCenterId);
+		HostVO sserver = storageServers.get(0);			
+		
+		UploadVO uploadTemplateObj = new UploadVO(sserver.getId(), template.getId(), new Date(), 
+													Upload.Status.NOT_UPLOADED, 0, type, 
+													null, "jobid0000", url);
+		_uploadDao.persist(uploadTemplateObj);        		               
+        		
+		if(vmTemplateHost != null) {
+		    start();
+			UploadCommand ucmd = new UploadCommand(template, url, vmTemplateHost);	
+			UploadListener ul = new UploadListener(sserver, _timer, _uploadDao, uploadTemplateObj.getId(), this, ucmd, template.getAccountId(), template.getName(), type, eventId, asyncJobId, asyncMgr);			
+			_listenerMap.put(uploadTemplateObj, ul);
+
+			long result = send(sserver.getId(), ucmd, ul);	
+			if (result == -1) {
+				s_logger.warn("Unable to start upload of " + template.getUniqueName() + " from " + sserver.getName() + " to " +url);
+				ul.setDisconnected();
+				ul.scheduleStatusCheck(RequestType.GET_OR_RESTART);
+			}
+		}
+		
+	}
+
+
+	public long send(Long hostId, Command cmd, Listener listener) {
+		return _agentMgr.gatherStats(hostId, cmd, listener);
+	}
+
+	@Override
+	public boolean configure(String name, Map params)
+			throws ConfigurationException {
+		_name = name;
+        final Map configs = _configDao.getConfiguration("ManagementServer", params);
+        _sslCopy = Boolean.parseBoolean(configs.get("secstorage.encrypt.copy"));
+        
+        String cert = configs.get("secstorage.secure.copy.cert");
+        if ("realhostip.com".equalsIgnoreCase(cert)) {
+        	s_logger.warn("Only realhostip.com ssl cert is supported, ignoring self-signed and other certs");
+        }
+        
+        _hyperVisorType = _configDao.getValue("hypervisor.type");
+        
+        _copyAuthPasswd = configs.get("secstorage.copy.password");
+        
+        _agentMgr.registerForHostEvents(new UploadListener(this), true, false, false);
+		return true;
+	}
+
+	@Override
+	public String getName() {
+		return _name;
+	}
+
+	@Override
+	public boolean start() {
+		_timer = new Timer();
+		return true;
+	}
+
+	@Override
+	public boolean stop() {		
+		return true;
+	}
+	
+	public String getEvent(Type type){
+					
+		if(type == Type.TEMPLATE) return EventTypes.EVENT_TEMPLATE_UPLOAD;
+		if(type == Type.ISO) return EventTypes.EVENT_ISO_UPLOAD;
+		if(type == Type.VOLUME) return EventTypes.EVENT_VOLUME_UPLOAD;			
+		
+		return null;
+	}
+	
+	public void handleUploadEvent(HostVO host, Long accountId, String typeName, Type type, Long uploadId, com.cloud.storage.Upload.Status reason, long eventId) {
+		
+		if ((reason == Upload.Status.UPLOADED) || (reason==Upload.Status.ABANDONED)){
+			UploadVO uploadObj = new UploadVO(uploadId);
+			UploadListener oldListener = _listenerMap.get(uploadObj);
+			if (oldListener != null) {
+				_listenerMap.remove(uploadObj);
+			}
+		}
+		if (reason == Upload.Status.UPLOADED) {
+			logEvent(accountId, getEvent(type), typeName + " successfully uploaded from storage server " + host.getName(), EventVO.LEVEL_INFO, eventId);
+		}
+		if (reason == Upload.Status.UPLOAD_ERROR) {
+			logEvent(accountId, getEvent(type), typeName + " failed to upload from storage server " + host.getName(), EventVO.LEVEL_ERROR, eventId);
+		}
+		if (reason == Upload.Status.ABANDONED) {
+			logEvent(accountId, getEvent(type), typeName + " :aborted upload from storage server " + host.getName(), EventVO.LEVEL_WARN, eventId);
+		}			
+
+	}
+	
+	public void logEvent(long accountId, String evtType, String description, String level, long eventId) {
+		EventVO event = new EventVO();
+		event.setUserId(1);
+		event.setAccountId(accountId);
+		event.setType(evtType);
+		event.setDescription(description);
+		event.setLevel(level);
+		event.setStartId(eventId);
+		_eventDao.persist(event);
+		
+	}
+
+	@Override
+	public void handleUploadSync(long sserverId) {
+	    
+	    HostVO storageHost = _serverDao.findById(sserverId);
+        if (storageHost == null) {
+            s_logger.warn("Huh? Agent id " + sserverId + " does not correspond to a row in hosts table?");
+            return;
+        }
+        s_logger.debug("Handling upload sserverId " +sserverId);
+        List uploadsInProgress = new ArrayList();
+        uploadsInProgress.addAll(_uploadDao.listByHostAndUploadStatus(sserverId, UploadVO.Status.UPLOAD_IN_PROGRESS));
+        uploadsInProgress.addAll(_uploadDao.listByHostAndUploadStatus(sserverId, UploadVO.Status.COPY_IN_PROGRESS));
+        if (uploadsInProgress.size() > 0){
+            for (UploadVO uploadJob : uploadsInProgress){
+                uploadJob.setUploadState(UploadVO.Status.UPLOAD_ERROR);
+                uploadJob.setErrorString("Could not complete the upload.");
+                uploadJob.setLastUpdated(new Date());
+                _uploadDao.update(uploadJob.getId(), uploadJob);
+            }
+            
+        }
+	        
+
+	}				
+		
+}
diff --git a/server/src/com/cloud/storage/upload/UploadState.java b/server/src/com/cloud/storage/upload/UploadState.java
new file mode 100644
index 00000000000..6bccbf06e2c
--- /dev/null
+++ b/server/src/com/cloud/storage/upload/UploadState.java
@@ -0,0 +1,70 @@
+package com.cloud.storage.upload;
+
+import java.util.Date;
+
+import org.apache.log4j.Level;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.storage.UploadAnswer;
+import com.cloud.storage.upload.UploadState.UploadEvent;
+
+public abstract class UploadState {
+
+	public static enum UploadEvent {UPLOAD_ANSWER, ABANDON_UPLOAD, TIMEOUT_CHECK, DISCONNECT};
+	protected static final Logger s_logger = Logger.getLogger(UploadListener.class.getName());
+
+	private UploadListener ul;
+	
+	public UploadState(UploadListener ul) {
+		this.ul = ul;
+	}
+	
+	protected UploadListener getUploadListener() {
+		return ul;
+	}
+	
+	public String handleEvent(UploadEvent event, Object eventObj){
+		if (s_logger.isTraceEnabled()) {
+			getUploadListener().log("handleEvent, event type=" + event + ", curr state=" + getName(), Level.TRACE);
+		}
+		switch (event) {
+		case UPLOAD_ANSWER:
+			UploadAnswer answer=(UploadAnswer)eventObj;
+			return handleAnswer(answer);
+		case ABANDON_UPLOAD:
+			return handleAbort();
+		case TIMEOUT_CHECK:
+			Date now = new Date();
+			long update = now.getTime() - ul.getLastUpdated().getTime();
+			return handleTimeout(update);
+		case DISCONNECT:
+			return handleDisconnect();
+		}
+		return null;
+	}
+	
+	public   void onEntry(String prevState, UploadEvent event, Object evtObj){
+		if (s_logger.isTraceEnabled()) {
+			getUploadListener().log("onEntry, event type=" + event + ", curr state=" + getName(), Level.TRACE);
+		}
+		if (event == UploadEvent.UPLOAD_ANSWER) {
+			getUploadListener().updateDatabase((UploadAnswer)evtObj);
+		}
+	}
+	
+	public  void onExit() {
+		
+	}
+	
+	public abstract String handleTimeout(long updateMs) ;
+	
+	public abstract String handleAbort();
+	
+	public abstract  String handleDisconnect();
+
+	public abstract String handleAnswer(UploadAnswer answer) ;
+	
+	public abstract String getName();
+
+
+}
diff --git a/server/src/com/cloud/template/TemplateManager.java b/server/src/com/cloud/template/TemplateManager.java
old mode 100644
new mode 100755
index 4246260f1a2..9242f9f3eb9
--- a/server/src/com/cloud/template/TemplateManager.java
+++ b/server/src/com/cloud/template/TemplateManager.java
@@ -27,8 +27,11 @@ import com.cloud.api.commands.CopyTemplateCmd;
 import com.cloud.api.commands.DeleteIsoCmd;
 import com.cloud.api.commands.DeleteTemplateCmd;
 import com.cloud.api.commands.DetachIsoCmd;
+import com.cloud.api.commands.ExtractIsoCmd;
+import com.cloud.api.commands.ExtractTemplateCmd;
 import com.cloud.api.commands.RegisterIsoCmd;
 import com.cloud.api.commands.RegisterTemplateCmd;
+import com.cloud.async.AsyncJobManager;
 import com.cloud.exception.InternalErrorException;
 import com.cloud.exception.InvalidParameterValueException;
 import com.cloud.exception.PermissionDeniedException;
@@ -108,7 +111,34 @@ public interface TemplateManager extends Manager {
      * @param sourceZoneId
      * @param destZoneId
      * @return true if success
-     * @throws InternalErrorException
+     * @throws InternalErrorException        URI uri = new URI(url);
+        if ( (uri.getScheme() == null) || (!uri.getScheme().equalsIgnoreCase("ftp") )) {
+           throw new IllegalArgumentException("Unsupported scheme for url: " + url);
+        }
+        String host = uri.getHost();
+        
+        try {
+        	InetAddress hostAddr = InetAddress.getByName(host);
+        	if (hostAddr.isAnyLocalAddress() || hostAddr.isLinkLocalAddress() || hostAddr.isLoopbackAddress() || hostAddr.isMulticastAddress() ) {
+        		throw new IllegalArgumentException("Illegal host specified in url");
+        	}
+        	if (hostAddr instanceof Inet6Address) {
+        		throw new IllegalArgumentException("IPV6 addresses not supported (" + hostAddr.getHostAddress() + ")");
+        	}
+        } catch (UnknownHostException uhe) {
+        	throw new IllegalArgumentException("Unable to resolve " + host);
+        }
+        
+    	if (_dcDao.findById(zoneId) == null) {
+    		throw new IllegalArgumentException("Please specify a valid zone.");
+    	}
+        
+        VMTemplateVO template = findTemplateById(templateId);
+        
+        VMTemplateHostVO tmpltHostRef = findTemplateHostRef(templateId, zoneId);
+        if (tmpltHostRef != null && tmpltHostRef.getDownloadState() != com.cloud.storage.VMTemplateStorageResourceAssoc.Status.DOWNLOADED){
+        	throw new IllegalArgumentException("The template hasnt been downloaded ");
+        }
      * @throws StorageUnavailableException 
      * @throws InvalidParameterValueException 
      */
@@ -157,5 +187,8 @@ public interface TemplateManager extends Manager {
      * @throws InvalidParameterValueException, InternalErrorException, PermissionDeniedException
      */
     boolean deleteIso(DeleteIsoCmd cmd) throws InvalidParameterValueException, InternalErrorException, PermissionDeniedException;
-    
+
+	void extract(VMTemplateVO template, String url, VMTemplateHostVO tmpltHostRef, Long zoneId, long eventId, long asyncJobId, AsyncJobManager asyncMgr);
+    void extract(ExtractIsoCmd cmd) throws InvalidParameterValueException, PermissionDeniedException;
+    void extract(ExtractTemplateCmd cmd) throws InvalidParameterValueException, PermissionDeniedException;
 }
diff --git a/server/src/com/cloud/template/TemplateManagerImpl.java b/server/src/com/cloud/template/TemplateManagerImpl.java
old mode 100644
new mode 100755
index fabd218f603..6edbfc0fd48
--- a/server/src/com/cloud/template/TemplateManagerImpl.java
+++ b/server/src/com/cloud/template/TemplateManagerImpl.java
@@ -44,8 +44,12 @@ import com.cloud.api.commands.CopyTemplateCmd;
 import com.cloud.api.commands.DeleteIsoCmd;
 import com.cloud.api.commands.DeleteTemplateCmd;
 import com.cloud.api.commands.DetachIsoCmd;
+import com.cloud.api.commands.ExtractIsoCmd;
+import com.cloud.api.commands.ExtractTemplateCmd;
 import com.cloud.api.commands.RegisterIsoCmd;
 import com.cloud.api.commands.RegisterTemplateCmd;
+import com.cloud.async.AsyncJobManager;
+import com.cloud.async.AsyncJobVO;
 import com.cloud.configuration.ResourceCount.ResourceType;
 import com.cloud.configuration.dao.ConfigurationDao;
 import com.cloud.dc.DataCenterVO;
@@ -70,6 +74,7 @@ import com.cloud.storage.Storage.ImageFormat;
 import com.cloud.storage.StorageManager;
 import com.cloud.storage.StoragePoolHostVO;
 import com.cloud.storage.StoragePoolVO;
+import com.cloud.storage.Upload.Type;
 import com.cloud.storage.VMTemplateHostVO;
 import com.cloud.storage.VMTemplateStoragePoolVO;
 import com.cloud.storage.VMTemplateStorageResourceAssoc;
@@ -86,6 +91,7 @@ import com.cloud.storage.dao.VMTemplatePoolDao;
 import com.cloud.storage.dao.VMTemplateZoneDao;
 import com.cloud.storage.dao.VolumeDao;
 import com.cloud.storage.download.DownloadMonitor;
+import com.cloud.storage.upload.UploadMonitor;
 import com.cloud.user.Account;
 import com.cloud.user.AccountManager;
 import com.cloud.user.AccountVO;
@@ -101,6 +107,7 @@ import com.cloud.utils.NumbersUtil;
 import com.cloud.utils.component.ComponentLocator;
 import com.cloud.utils.component.Inject;
 import com.cloud.utils.db.DB;
+import com.cloud.utils.db.JoinBuilder;
 import com.cloud.utils.db.SearchBuilder;
 import com.cloud.utils.db.SearchCriteria;
 import com.cloud.utils.db.Transaction;
@@ -125,6 +132,7 @@ public class TemplateManagerImpl implements TemplateManager {
     @Inject StoragePoolHostDao _poolHostDao;
     @Inject EventDao _eventDao;
     @Inject DownloadMonitor _downloadMonitor;
+    @Inject UploadMonitor _uploadMonitor;
     @Inject UserAccountDao _userAccountDao;
     @Inject AccountDao _accountDao;
     @Inject UserDao _userDao;
@@ -165,7 +173,7 @@ public class TemplateManagerImpl implements TemplateManager {
         
         long accountId = 1L; // default to system account
         if (account != null) {
-            accountId = account.getId().longValue();
+            accountId = account.getId();
         }
         
         Account accountObj;
@@ -249,7 +257,7 @@ public class TemplateManagerImpl implements TemplateManager {
                 
         long accountId = 1L; // default to system account
         if (account != null) {
-            accountId = account.getId().longValue();
+            accountId = account.getId();
         }
         
         Account accountObj;
@@ -378,7 +386,7 @@ public class TemplateManagerImpl implements TemplateManager {
         
         VMTemplateVO template = new VMTemplateVO(id, name, format, isPublic, featured, fs, url.toString(), requiresHvm, bits, accountId, chksum, displayText, enablePassword, guestOSId, bootable);
         if (zoneId == null) {
-            List dcs = _dcDao.listAll();
+            List dcs = _dcDao.listAllIncludingRemoved();
 
         	for (DataCenterVO dc: dcs) {
     			_tmpltDao.addTemplateToZone(template, dc.getId());
@@ -397,6 +405,123 @@ public class TemplateManagerImpl implements TemplateManager {
         
         return id;
     }
+
+    @Override
+    public void extract(ExtractIsoCmd cmd) throws InvalidParameterValueException, PermissionDeniedException {
+        Account account = (Account)UserContext.current().getAccountObject();
+        Long templateId = cmd.getId();
+        Long zoneId = cmd.getZoneId();
+        String url = cmd.getUrl();
+
+        extract(account, templateId, url, zoneId, true, cmd.getJob(), cmd.getAsyncJobManager());
+    }
+
+    @Override
+    public void extract(ExtractTemplateCmd cmd) throws InvalidParameterValueException, PermissionDeniedException {
+        Account account = (Account)UserContext.current().getAccountObject();
+        Long templateId = cmd.getId();
+        Long zoneId = cmd.getZoneId();
+        String url = cmd.getUrl();
+
+        extract(account, templateId, url, zoneId, true, cmd.getJob(), cmd.getAsyncJobManager());
+    }
+
+    private void extract(Account account, Long templateId, String url, Long zoneId, boolean isISO, AsyncJobVO job, AsyncJobManager mgr) throws InvalidParameterValueException, PermissionDeniedException {
+        String desc = "template";
+        if (isISO) {
+            desc = "ISO";
+        }
+
+        VMTemplateVO template = _tmpltDao.findById(templateId);
+        if (template == null) {
+            throw new InvalidParameterValueException("Unable to find template with id " + templateId);
+        }
+        if (template.getName().startsWith("xs-tools") ){
+            throw new InvalidParameterValueException("Unable to extract the " + desc + " " + template.getName() + " It is not allowed");
+        }
+        if (isISO) {
+            if (template.getFormat() != ImageFormat.ISO ){
+                throw new InvalidParameterValueException("Unsupported format, could not extract the ISO");
+            }
+        } else {
+            if (template.getFormat() == ImageFormat.ISO ){
+                throw new InvalidParameterValueException("Unsupported format, could not extract the template");
+            }
+        }
+
+        if (_dcDao.findById(zoneId) == null) {
+            throw new IllegalArgumentException("Please specify a valid zone.");
+        }
+
+        if (url.toLowerCase().contains("file://")){
+            throw new InvalidParameterValueException("file:// type urls are currently unsupported");
+        }
+
+        URI uri = null;
+        try {
+            uri = new URI(url);
+            if ((uri.getScheme() == null) || (!uri.getScheme().equalsIgnoreCase("ftp") )) {
+               throw new InvalidParameterValueException("Unsupported scheme for url: " + url);
+            }
+        } catch (URISyntaxException ex) {
+            throw new InvalidParameterValueException("Invalid url given: " + url);
+        }
+
+        String host = uri.getHost();
+        try {
+            InetAddress hostAddr = InetAddress.getByName(host);
+            if (hostAddr.isAnyLocalAddress() || hostAddr.isLinkLocalAddress() || hostAddr.isLoopbackAddress() || hostAddr.isMulticastAddress() ) {
+                throw new InvalidParameterValueException("Illegal host specified in url");
+            }
+            if (hostAddr instanceof Inet6Address) {
+                throw new InvalidParameterValueException("IPV6 addresses not supported (" + hostAddr.getHostAddress() + ")");
+            }
+        } catch (UnknownHostException uhe) {
+            throw new InvalidParameterValueException("Unable to resolve " + host);
+        }
+                
+        if (account != null) {                  
+            if(!isAdmin(account.getType())){
+                if (template.getAccountId() != account.getId()){
+                    throw new PermissionDeniedException("Unable to find " + desc + " with ID: " + templateId + " for account: " + account.getAccountName());
+                }
+            } else {
+                Account userAccount = _accountDao.findById(template.getAccountId());
+                if((userAccount == null) || !_domainDao.isChildDomain(account.getDomainId(), userAccount.getDomainId())) {
+                    throw new PermissionDeniedException("Unable to extract " + desc + " " + templateId + " to " + url + ", permission denied.");
+                }
+            }
+        }
+
+        HostVO secondaryStorageHost = _storageMgr.getSecondaryStorageHost(zoneId);
+        VMTemplateHostVO tmpltHostRef = null;
+        if (secondaryStorageHost != null) {
+            tmpltHostRef = _tmpltHostDao.findByHostTemplate(secondaryStorageHost.getId(), templateId);
+            if (tmpltHostRef != null && tmpltHostRef.getDownloadState() != com.cloud.storage.VMTemplateStorageResourceAssoc.Status.DOWNLOADED) {
+                throw new InvalidParameterValueException("The " + desc + " has not been downloaded ");
+            }
+        }
+
+        if ( _uploadMonitor.isTypeUploadInProgress(templateId, isISO ? Type.ISO : Type.TEMPLATE) ){
+            throw new IllegalArgumentException(template.getName() + " upload is in progress. Please wait for some time to schedule another upload for the same"); 
+        }
+
+        long userId = UserContext.current().getUserId();
+        long accountId = template.getAccountId();
+        String event = isISO ? EventTypes.EVENT_ISO_UPLOAD : EventTypes.EVENT_TEMPLATE_UPLOAD;
+        long eventId = EventUtils.saveScheduledEvent(userId, accountId, event, "Extraction job");
+  
+// FIXME:  scheduled event should've already been saved, we should be saving this started event here...
+//        String event = template.getFormat() == ImageFormat.ISO ? EventTypes.EVENT_ISO_UPLOAD : EventTypes.EVENT_TEMPLATE_UPLOAD;
+//        EventUtils.saveStartedEvent(template.getAccountId(), template.getAccountId(), event, "Starting upload of " +template.getName()+ " to " +url, cmd.getStartEventId());
+
+        extract(template, url, tmpltHostRef, zoneId, eventId, job.getId(), mgr);
+    }
+
+    @Override
+    public void extract(VMTemplateVO template, String url, VMTemplateHostVO tmpltHostRef, Long zoneId, long eventId, long asyncJobId, AsyncJobManager asyncMgr){
+        _uploadMonitor.extractTemplate(template, url, tmpltHostRef, zoneId, eventId, asyncJobId, asyncMgr);
+    }
     
     @Override @DB
     public VMTemplateStoragePoolVO prepareTemplateForCreate(VMTemplateVO template, StoragePoolVO pool) {
@@ -476,14 +601,21 @@ public class TemplateManagerImpl implements TemplateManager {
                 return templateStoragePoolRef;
             }
             String url = origUrl + "/" + templateHostRef.getInstallPath();
-            PrimaryStorageDownloadCommand dcmd = new PrimaryStorageDownloadCommand(template.getUniqueName(), url, template.getFormat(), template.getAccountId(), pool.getId(), pool.getUuid());
+            PrimaryStorageDownloadCommand dcmd = new PrimaryStorageDownloadCommand(template.getUniqueName(), url, template.getFormat(), 
+            	template.getAccountId(), pool.getId(), pool.getUuid());
+            HostVO secondaryStorageHost = _hostDao.findSecondaryStorageHost(pool.getDataCenterId());
+            assert(secondaryStorageHost != null);
+            dcmd.setSecondaryStorageUrl(secondaryStorageHost.getStorageUrl());
+            // TODO temporary hacking, hard-coded to NFS primary data store
+            dcmd.setPrimaryStorageUrl("nfs://" + pool.getHostAddress() + pool.getPath());
             
             for (StoragePoolHostVO vo : vos) {
                 if (s_logger.isDebugEnabled()) {
                     s_logger.debug("Downloading " + templateId + " via " + vo.getHostId());
                 }
             	dcmd.setLocalPath(vo.getLocalPath());
-                DownloadAnswer answer = (DownloadAnswer)_agentMgr.easySend(vo.getHostId(), dcmd);
+            	// set 120 min timeout for this command
+                DownloadAnswer answer = (DownloadAnswer)_agentMgr.easySend(vo.getHostId(), dcmd, 120*60*1000);
                 if (answer != null) {
             		templateStoragePoolRef.setDownloadPercent(templateStoragePoolRef.getDownloadPercent());
             		templateStoragePoolRef.setDownloadState(answer.getDownloadStatus());
@@ -905,7 +1037,7 @@ public class TemplateManagerImpl implements TemplateManager {
         SearchBuilder HostSearch = _hostDao.createSearchBuilder();
         HostSearch.and("dcId", HostSearch.entity().getDataCenterId(), SearchCriteria.Op.EQ);
         
-        HostTemplateStatesSearch.join("host", HostSearch, HostSearch.entity().getId(), HostTemplateStatesSearch.entity().getHostId());
+        HostTemplateStatesSearch.join("host", HostSearch, HostSearch.entity().getId(), HostTemplateStatesSearch.entity().getHostId(), JoinBuilder.JoinType.INNER);
         HostSearch.done();
         HostTemplateStatesSearch.done();
         
@@ -1070,11 +1202,11 @@ public class TemplateManagerImpl implements TemplateManager {
 		
     	if (account != null) {
     	    if (!isAdmin(account.getType())) {
-				if ((vmInstanceCheck != null) && (account.getId().longValue() != vmInstanceCheck.getAccountId())) {
+				if ((vmInstanceCheck != null) && (account.getId() != vmInstanceCheck.getAccountId())) {
 		            throw new PermissionDeniedException(msg + ". Permission denied.");
 		        }
 
-	    		if ((template != null) && (!template.isPublicTemplate() && (account.getId().longValue() != template.getAccountId()) && (!template.getName().startsWith("xs-tools")))) {
+	    		if ((template != null) && (!template.isPublicTemplate() && (account.getId() != template.getAccountId()) && (!template.getName().startsWith("xs-tools")))) {
                     throw new PermissionDeniedException(msg + ". Permission denied.");
                 }
                 
diff --git a/server/src/com/cloud/test/DatabaseConfig.java b/server/src/com/cloud/test/DatabaseConfig.java
index 2be08d86823..b51a99f3af4 100644
--- a/server/src/com/cloud/test/DatabaseConfig.java
+++ b/server/src/com/cloud/test/DatabaseConfig.java
@@ -49,7 +49,9 @@ import org.xml.sax.Attributes;
 import org.xml.sax.SAXException;
 import org.xml.sax.helpers.DefaultHandler;
 
-import com.cloud.offering.ServiceOffering.GuestIpType;
+import com.cloud.host.Status;
+import com.cloud.offering.NetworkOffering;
+import com.cloud.offering.NetworkOffering.GuestIpType;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.service.dao.ServiceOfferingDaoImpl;
 import com.cloud.storage.DiskOfferingVO;
@@ -528,7 +530,7 @@ public class DatabaseConfig {
          String storageType = _currentObjectParams.get("storageType");
          String uuid = UUID.nameUUIDFromBytes(new String(hostAddress+hostPath).getBytes()).toString();
  
-         String insertSql1 = "INSERT INTO `storage_pool` (`id`, `name`, `uuid` , `pool_type` , `port`, `data_center_id` ,`available_bytes` , `capacity_bytes` ,`host_address`, `path`, `created`, `pod_id` ) VALUES (?,?,?,?,?,?,?,?,?,?,?,?)";
+         String insertSql1 = "INSERT INTO `storage_pool` (`id`, `name`, `uuid` , `pool_type` , `port`, `data_center_id` ,`available_bytes` , `capacity_bytes` ,`host_address`, `path`, `created`, `pod_id`,`status` ) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)";
          // String insertSql2 = "INSERT INTO `netfs_storage_pool` VALUES (?,?,?)";
          
          Transaction txn = Transaction.currentTxn();
@@ -549,6 +551,7 @@ public class DatabaseConfig {
              stmt.setString(10, hostPath);
              stmt.setDate(11, new Date(new java.util.Date().getTime()));
              stmt.setLong(12, podId);
+             stmt.setString(13, Status.Up.toString());
              stmt.executeUpdate();
              // stmt = txn.prepareAutoCloseStatement(insertSql2);
              // stmt.setLong(1, 2);
@@ -720,11 +723,11 @@ public class DatabaseConfig {
         boolean ha = Boolean.parseBoolean(_currentObjectParams.get("enableHA"));
         boolean mirroring = Boolean.parseBoolean(_currentObjectParams.get("mirrored"));
         String guestIpType = _currentObjectParams.get("guestIpType");
-        GuestIpType type = null;
+        NetworkOffering.GuestIpType type = null;
         if (guestIpType == null) {
-            type = GuestIpType.Virtualized;
+            type = NetworkOffering.GuestIpType.Virtualized;
         } else {
-            type = GuestIpType.valueOf(guestIpType);
+            type = NetworkOffering.GuestIpType.valueOf(guestIpType);
         }
         
         boolean useLocalStorage;
@@ -770,6 +773,11 @@ public class DatabaseConfig {
         int diskSpace = Integer.parseInt(_currentObjectParams.get("diskSpace"));
 //        boolean mirroring = Boolean.parseBoolean(_currentObjectParams.get("mirrored"));
         String tags = _currentObjectParams.get("tags");
+        String useLocal = _currentObjectParams.get("useLocal");
+        boolean local = false;
+        if (useLocal != null) {
+        	local = Boolean.parseBoolean(useLocal);
+        }
         
         if (tags != null && tags.length() > 0) {
             String[] tokens = tags.split(",");
@@ -781,6 +789,7 @@ public class DatabaseConfig {
             tags = newTags.toString();
         }
         DiskOfferingVO diskOffering = new DiskOfferingVO(domainId, name, displayText, diskSpace, tags);
+        diskOffering.setUseLocalStorage(local);
         DiskOfferingDaoImpl offering = ComponentLocator.inject(DiskOfferingDaoImpl.class);
         try {
             offering.persist(diskOffering);
diff --git a/server/src/com/cloud/user/AccountManager.java b/server/src/com/cloud/user/AccountManager.java
index e87dd91b48d..71a570a0ebf 100644
--- a/server/src/com/cloud/user/AccountManager.java
+++ b/server/src/com/cloud/user/AccountManager.java
@@ -113,4 +113,6 @@ public interface AccountManager extends Manager {
      * @throws InvalidParameterValueException
      */
 	ResourceLimitVO updateResourceLimit(UpdateResourceLimitCmd cmd) throws InvalidParameterValueException;
+	
+	AccountVO getSystemAccount();
 }
diff --git a/server/src/com/cloud/user/AccountManagerImpl.java b/server/src/com/cloud/user/AccountManagerImpl.java
index ba8eed1d71c..4875b4f2bbe 100644
--- a/server/src/com/cloud/user/AccountManagerImpl.java
+++ b/server/src/com/cloud/user/AccountManagerImpl.java
@@ -40,10 +40,8 @@ import com.cloud.domain.dao.DomainDao;
 import com.cloud.exception.InvalidParameterValueException;
 import com.cloud.exception.PermissionDeniedException;
 import com.cloud.server.Criteria;
-import com.cloud.storage.dao.VMTemplateDao;
 import com.cloud.user.dao.AccountDao;
-import com.cloud.user.dao.UserDao;
-import com.cloud.utils.component.ComponentLocator;
+import com.cloud.utils.component.Inject;
 import com.cloud.utils.db.Filter;
 import com.cloud.utils.db.GlobalLock;
 import com.cloud.utils.db.SearchBuilder;
@@ -54,49 +52,22 @@ public class AccountManagerImpl implements AccountManager {
 	public static final Logger s_logger = Logger.getLogger(AccountManagerImpl.class.getName());
 	
 	private String _name;
-	private AccountDao _accountDao;
-	private DomainDao _domainDao;
-	private UserDao _userDao;
-	private VMTemplateDao _templateDao;
-	private ResourceLimitDao _resourceLimitDao;
-	private ResourceCountDao _resourceCountDao;
+	@Inject private AccountDao _accountDao;
+	@Inject private DomainDao _domainDao;
+	@Inject private ResourceLimitDao _resourceLimitDao;
+	@Inject private ResourceCountDao _resourceCountDao;
 	private final GlobalLock m_resourceCountLock = GlobalLock.getInternLock("resource.count");
 	
+	AccountVO _systemAccount;
+	
 	@Override
     public boolean configure(final String name, final Map params) throws ConfigurationException {
     	_name = name;
-    	final ComponentLocator locator = ComponentLocator.getCurrentLocator();
-    	
-    	_accountDao = locator.getDao(AccountDao.class);
-        if (_accountDao == null) {
-            throw new ConfigurationException("Unable to get the account dao.");
-        }
-        
-        _domainDao = locator.getDao(DomainDao.class);
-        if (_domainDao == null) {
-        	throw new ConfigurationException("Unable to get the domain dao.");
-        }
-        
-        _userDao = locator.getDao(UserDao.class);
-        if (_userDao == null) {
-            throw new ConfigurationException("Unable to get the user dao.");
-        }
-        
-        _templateDao = locator.getDao(VMTemplateDao.class);
-        if (_templateDao == null) {
-        	throw new ConfigurationException("Unable to get the template dao.");
-        }
-        
-        _resourceLimitDao = locator.getDao(ResourceLimitDao.class);
-        if (_resourceLimitDao == null) {
-            throw new ConfigurationException("Unable to get " + ResourceLimitDao.class.getName());
-        }
-        
-        _resourceCountDao = locator.getDao(ResourceCountDao.class);
-        if (_resourceCountDao == null) {
-            throw new ConfigurationException("Unable to get " + ResourceCountDao.class.getName());
-        }
     	
+    	_systemAccount = _accountDao.findById(AccountVO.ACCOUNT_ID_SYSTEM);
+    	if (_systemAccount == null) {
+    	    throw new ConfigurationException("Unable to find the system account using " + AccountVO.ACCOUNT_ID_SYSTEM);
+    	}
     	return true;
     }
 	
@@ -584,4 +555,9 @@ public class AccountManagerImpl implements AccountManager {
             return _resourceLimitDao.persist(new ResourceLimitVO(domainId, accountId, resourceType, max));
         }
     }
+
+    @Override
+    public AccountVO getSystemAccount() {
+        return _systemAccount;
+    }
 }
diff --git a/server/src/com/cloud/vm/MauriceMoss.java b/server/src/com/cloud/vm/MauriceMoss.java
index 1223b63012e..a12c6c47733 100644
--- a/server/src/com/cloud/vm/MauriceMoss.java
+++ b/server/src/com/cloud/vm/MauriceMoss.java
@@ -17,51 +17,270 @@
  */
 package com.cloud.vm;
 
-import com.cloud.offerings.NetworkOfferingVO;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import javax.ejb.Local;
+import javax.naming.ConfigurationException;
+
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.agent.manager.AgentManager;
+import com.cloud.configuration.Config;
+import com.cloud.configuration.dao.ConfigurationDao;
+import com.cloud.deploy.DeployDestination;
+import com.cloud.deploy.DeploymentPlan;
+import com.cloud.deploy.DeploymentPlanner;
+import com.cloud.exception.AgentUnavailableException;
+import com.cloud.exception.InsufficientCapacityException;
+import com.cloud.network.NetworkConfigurationVO;
+import com.cloud.network.NetworkManager;
+import com.cloud.offering.ServiceOffering;
 import com.cloud.service.ServiceOfferingVO;
+import com.cloud.service.dao.ServiceOfferingDao;
 import com.cloud.storage.DiskOfferingVO;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.StorageManager;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.storage.Volume.VolumeType;
+import com.cloud.user.AccountVO;
+import com.cloud.utils.Journal;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.component.Adapters;
+import com.cloud.utils.component.ComponentLocator;
+import com.cloud.utils.component.Inject;
+import com.cloud.utils.db.DB;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.vm.VirtualMachine.Event;
+import com.cloud.vm.dao.VMInstanceDao;
 
+@Local(value=VmManager.class)
 public class MauriceMoss implements VmManager {
+    private static final Logger s_logger = Logger.getLogger(MauriceMoss.class);
+    
+    String _name;
+    @Inject private StorageManager _storageMgr;
+    @Inject private NetworkManager _networkMgr;
+    @Inject private AgentManager _agentMgr;
+    @Inject private VMInstanceDao _vmDao;
+    @Inject private ServiceOfferingDao _offeringDao;
+    
+    @Inject(adapter=DeploymentPlanner.class)
+    private Adapters _planners;
+    
+    private int _retry;
 
+    @Override @DB
+    public  VirtualMachineProfile allocate(T vm,
+            VMTemplateVO template,
+            ServiceOfferingVO serviceOffering,
+            Pair rootDiskOffering,
+            List> dataDiskOfferings,
+            List> networks, 
+            DeploymentPlan plan,
+            AccountVO owner) throws InsufficientCapacityException {
+        if (s_logger.isDebugEnabled()) {
+            s_logger.debug("Allocating entries for VM: " + vm);
+        }
+        //VMInstanceVO vm = _vmDao.findById(vm.getId());
+        VirtualMachineProfile vmProfile = new VirtualMachineProfile(vm, serviceOffering);
+        
+        Transaction txn = Transaction.currentTxn();
+        txn.start();
+        vm.setDataCenterId(plan.getDataCenterId());
+        _vmDao.update(vm.getId(), vm);
+        
+        List nics = _networkMgr.allocate(vmProfile, networks);
+        vmProfile.setNics(nics);
+
+        if (dataDiskOfferings == null) {
+            dataDiskOfferings = new ArrayList>(0);
+        }
+        
+        List disks = new ArrayList(dataDiskOfferings.size() + 1);
+        if (template.getFormat() == ImageFormat.ISO) {
+            disks.add(_storageMgr.allocateRawVolume(VolumeType.ROOT, "ROOT-" + vm.getId(), rootDiskOffering.first(), rootDiskOffering.second(), vm, owner));
+        } else {
+            disks.add(_storageMgr.allocateTemplatedVolume(VolumeType.ROOT, "ROOT-" + vm.getId(), rootDiskOffering.first(), template, vm, owner));
+        }
+        for (Pair offering : dataDiskOfferings) {
+            disks.add(_storageMgr.allocateRawVolume(VolumeType.DATADISK, "DATA-" + vm.getId(), offering.first(), offering.second(), vm, owner));
+        }
+        vmProfile.setDisks(disks);
+
+        _vmDao.updateIf(vm, Event.OperationSucceeded, null);
+        txn.commit();
+        if (s_logger.isDebugEnabled()) {
+            s_logger.debug("Allocation completed for VM: " + vm);
+        }
+        
+        return vmProfile;
+    }
+    
     @Override
-    public VMInstanceVO allocate(VMInstanceVO vm, ServiceOfferingVO serviceOffering, NetworkOfferingVO[] networkOfferings, DiskOfferingVO[] diskOffering) {
-        return null;
+    public  VirtualMachineProfile allocate(T vm,
+            VMTemplateVO template,
+            ServiceOfferingVO serviceOffering,
+            Long rootSize,
+            Pair dataDiskOffering,
+            List> networks,
+            DeploymentPlan plan,
+            AccountVO owner) throws InsufficientCapacityException {
+        List> diskOfferings = new ArrayList>(1);
+        if (dataDiskOffering != null) {
+            diskOfferings.add(dataDiskOffering);
+        }
+        return allocate(vm, template, serviceOffering, new Pair(serviceOffering, rootSize), diskOfferings, networks, plan, owner);
+    }
+    
+    @Override
+    public  VirtualMachineProfile allocate(T vm,
+            VMTemplateVO template,
+            ServiceOfferingVO serviceOffering,
+            List networkProfiles,
+            DeploymentPlan plan, 
+            AccountVO owner) throws InsufficientCapacityException {
+        List> networks = new ArrayList>(networkProfiles.size());
+        for (NetworkConfigurationVO profile : networkProfiles) {
+            networks.add(new Pair(profile, null));
+        }
+        return allocate(vm, template, serviceOffering, new Pair(serviceOffering, null), null, networks, plan, owner);
+    }
+    
+    protected VirtualMachineProfile create(VirtualMachineProfile vmProfile, DeploymentPlan plan) throws InsufficientCapacityException {
+        if (s_logger.isDebugEnabled()) {
+            s_logger.debug("Creating actual resources for VM " + vmProfile);
+        }
+        
+        Journal journal = new Journal.LogJournal("Creating " + vmProfile, s_logger);
+
+        Set avoids = new HashSet();
+        int retry = _retry;
+        while (_retry-- > 0) {
+            DeployDestination context = null;
+            for (DeploymentPlanner dispatcher : _planners) {
+                context = dispatcher.plan(vmProfile, plan, avoids);
+                if (context != null) {
+                    journal.record("Deployment found ", vmProfile, context);
+                    break;
+                }
+            }
+            
+            if (context == null) {
+                throw new CloudRuntimeException("Unable to create a deployment for " + vmProfile);
+            }
+            
+            VMInstanceVO vm = _vmDao.findById(vmProfile.getId());
+
+            vm.setDataCenterId(context.getDataCenter().getId());
+            vm.setPodId(context.getPod().getId());
+            vm.setHostId(context.getHost().getId());
+            _vmDao.update(vm.getId(), vm);
+            
+            _networkMgr.create(vm);
+            _storageMgr.create(vm);
+        }
+        
+        if (s_logger.isDebugEnabled()) {
+            s_logger.debug("Creation complete for VM " + vmProfile);
+        }
+        
+        return vmProfile;
     }
 
-    /* (non-Javadoc)
-     * @see com.cloud.vm.VmManager#create(com.cloud.vm.VMInstanceVO)
-     */
-    @Override
-    public void create(VMInstanceVO vm) {
-        // TODO Auto-generated method stub
-
-    }
-
-    /* (non-Javadoc)
-     * @see com.cloud.vm.VmManager#destroy()
-     */
     @Override
     public void destroy() {
         // TODO Auto-generated method stub
 
     }
 
-    /* (non-Javadoc)
-     * @see com.cloud.vm.VmManager#start()
-     */
     @Override
-    public void start() {
-        // TODO Auto-generated method stub
-
+    public boolean start() {
+        return true;
     }
 
-    /* (non-Javadoc)
-     * @see com.cloud.vm.VmManager#stop()
-     */
     @Override
-    public void stop() {
-        // TODO Auto-generated method stub
-
+    public boolean stop() {
+        return true;
+    }
+    
+    @Override
+    public boolean configure(String name, Map xmlParams) throws ConfigurationException {
+        _name = name;
+        
+        ComponentLocator locator = ComponentLocator.getCurrentLocator();
+        ConfigurationDao configDao = locator.getDao(ConfigurationDao.class);
+        Map params = configDao.getConfiguration(xmlParams);
+        
+        _planners = locator.getAdapters(DeploymentPlanner.class);
+        
+        _retry = NumbersUtil.parseInt(params.get(Config.StartRetry.key()), 2);
+        return true;
+    }
+    
+    @Override
+    public String getName() {
+        return _name;
+    }
+    
+    protected MauriceMoss() {
     }
 
+    @Override
+    public  T start(T vm, DeploymentPlan plan) throws InsufficientCapacityException {
+        if (s_logger.isDebugEnabled()) {
+            s_logger.debug("Creating actual resources for VM " + vm);
+        }
+        
+        Journal journal = new Journal.LogJournal("Creating " + vm, s_logger);
+        
+        ServiceOffering offering = _offeringDao.findById(vm.getServiceOfferingId());
+        
+        VirtualMachineProfile vmProfile = new VirtualMachineProfile(vm, offering);
+
+        Set avoids = new HashSet();
+        int retry = _retry;
+        while (retry-- > 0) {
+            DeployDestination dest = null;
+            for (DeploymentPlanner dispatcher : _planners) {
+                dest = dispatcher.plan(vmProfile, plan, avoids);
+                if (dest != null) {
+                    avoids.add(dest);
+                    journal.record("Deployment found ", vmProfile, dest);
+                    break;
+                }
+            }
+            
+            if (dest == null) {
+                throw new CloudRuntimeException("Unable to create a deployment for " + vmProfile);
+            }
+            
+            vm.setDataCenterId(dest.getDataCenter().getId());
+            vm.setPodId(dest.getPod().getId());
+            _vmDao.updateIf(vm, Event.StartRequested, dest.getHost().getId());
+            
+            VirtualMachineTO vmTO = new VirtualMachineTO();
+            _networkMgr.prepare(vmProfile, dest);
+//            _storageMgr.prepare(vm, dest);
+        }
+        
+        if (s_logger.isDebugEnabled()) {
+            s_logger.debug("Creation complete for VM " + vmProfile);
+        }
+        
+        return null;
+    }
+
+    @Override
+    public  T stop(T vm) throws AgentUnavailableException {
+        // TODO Auto-generated method stub
+        return null;
+    }
+    
 }
diff --git a/server/src/com/cloud/vm/NicVO.java b/server/src/com/cloud/vm/NicVO.java
index 8d6c69a2569..fdc9f3a730f 100644
--- a/server/src/com/cloud/vm/NicVO.java
+++ b/server/src/com/cloud/vm/NicVO.java
@@ -17,6 +17,9 @@
  */
 package com.cloud.vm;
 
+import java.net.URI;
+import java.util.Date;
+
 import javax.persistence.Column;
 import javax.persistence.Entity;
 import javax.persistence.EnumType;
@@ -26,11 +29,11 @@ import javax.persistence.GenerationType;
 import javax.persistence.Id;
 import javax.persistence.Table;
 
+import com.cloud.network.Network.AddressFormat;
 import com.cloud.network.Network.Mode;
-import com.cloud.network.Network.TrafficType;
 
 @Entity
-@Table(name="network")
+@Table(name="nics")
 public class NicVO implements Nic {
     protected NicVO() {
     }
@@ -43,33 +46,60 @@ public class NicVO implements Nic {
     @Column(name="instance_id")
     long instanceId;
     
-    @Column(name="type")
-    @Enumerated(value=EnumType.STRING)
-    TrafficType trafficType;
-    
     @Column(name="ip4_address")
     String ip4Address;
     
-    @Column(name="mac_address")
-    String macAddress;
+    @Column(name="ip6_address")
+    String ip6Address;
     
     @Column(name="netmask")
-    String netMask;
+    String netmask;
+    
+    @Column(name="isolation_uri")
+    URI isolationUri;
+    
+    @Column(name="ip_type")
+    AddressFormat addressFormat;
+    
+    @Column(name="broadcast_uri")
+    URI broadcastUri;
+    
+    @Column(name="gateway")
+    String gateway;
+    
+    @Column(name="mac_address")
+    String macAddress;
     
     @Column(name="mode")
     @Enumerated(value=EnumType.STRING)
     Mode mode;
     
-    @Column(name="network_profile_id")
-    long networkProfileId;
-    
-    @Column(name="String")
-    String vlan;
+    @Column(name="network_configuration_id")
+    long networkConfigurationId;
     
     @Column(name="state")
     @Enumerated(value=EnumType.STRING)
     State state;
+    
+    @Column(name="reserver_name")
+    String reserver;
+    
+    @Column(name="reservation_id")
+    String reservationId;
+    
+    @Column(name="device_id")
+    int deviceId;
+    
+    @Column(name="update_time")
+    Date updateTime;
 
+    public NicVO(String reserver, long instanceId, long configurationId) {
+        this.reserver = reserver;
+        this.instanceId = instanceId;
+        this.networkConfigurationId = configurationId;
+        this.state = State.Allocated;
+    }
+    
     @Override
     public String getIp4Address() {
         return ip4Address;
@@ -83,11 +113,79 @@ public class NicVO implements Nic {
     public String getMacAddress() {
         return macAddress;
     }
+    
+    public void setMacAddress(String macAddress) {
+        this.macAddress = macAddress;
+    }
 
     @Override
     public State getState() {
         return state;
     }
+    
+    public String getIp6Address() {
+        return ip6Address;
+    }
+
+    public void setIp6Address(String ip6Address) {
+        this.ip6Address = ip6Address;
+    }
+
+    public String getNetmask() {
+        return netmask;
+    }
+    
+    public String getGateway() {
+        return gateway;
+    }
+    
+    public void setGateway(String gateway) {
+        this.gateway = gateway;
+    }
+    
+    public AddressFormat getAddressFormat() {
+        return addressFormat;
+    }
+    
+    public void setAddressFormat(AddressFormat format) {
+        this.addressFormat = format;
+    }
+
+    public void setNetmask(String netmask) {
+        this.netmask = netmask;
+    }
+
+    public URI getIsolationUri() {
+        return isolationUri;
+    }
+
+    public void setIsolationUri(URI isolationUri) {
+        this.isolationUri = isolationUri;
+    }
+
+    public URI getBroadcastUri() {
+        return broadcastUri;
+    }
+
+    public void setBroadcastUri(URI broadcastUri) {
+        this.broadcastUri = broadcastUri;
+    }
+
+    public void setInstanceId(long instanceId) {
+        this.instanceId = instanceId;
+    }
+
+    public void setNetworkConfigurationId(long networkConfigurationId) {
+        this.networkConfigurationId = networkConfigurationId;
+    }
+
+    public void setUpdateTime(Date updateTime) {
+        this.updateTime = updateTime;
+    }
+
+    public void setState(State state) {
+        this.state = state;
+    }
 
     @Override
     public long getId() {
@@ -100,7 +198,63 @@ public class NicVO implements Nic {
     }
 
     @Override
-    public long getNetworkProfileId() {
-        return networkProfileId;
+    public long getNetworkConfigurationId() {
+        return networkConfigurationId;
+    }
+    
+    @Override
+    public int getDeviceId() {
+        return deviceId;
+    }
+    
+    @Override
+    public String getReservationId() {
+        return reservationId;
+    }
+    
+    public void setReservationId(String id) {
+        this.reservationId = id;
+    }
+    
+    public void setDeviceId(int deviceId) {
+        this.deviceId = deviceId;
+    }
+
+    @Override
+    public Mode getMode() {
+        return mode;
+    }
+    
+    public void setMode(Mode mode) {
+        this.mode = mode;
+    }
+
+    @Override
+    public String getReserver() {
+        return reserver;
+    }
+    
+    public void setReserver(String reserver) {
+        this.reserver = reserver;
+    }
+    
+    @Override
+    public ReservationStrategy getReservationStrategy() {
+        return ReservationStrategy.Start;
+    }
+
+    @Override
+    public int getExpectedReservationInterval() {
+        return -1;
+    }
+
+    @Override
+    public int getExpectedReleaseInterval() {
+        return -1;
+    }
+
+    @Override
+    public Date getUpdateTime() {
+        return updateTime;
     }
 }
diff --git a/server/src/com/cloud/vm/UserVmManager.java b/server/src/com/cloud/vm/UserVmManager.java
index 212141d0b4b..c254e96d960 100644
--- a/server/src/com/cloud/vm/UserVmManager.java
+++ b/server/src/com/cloud/vm/UserVmManager.java
@@ -24,6 +24,8 @@ import com.cloud.agent.api.VmStatsEntry;
 import com.cloud.api.ServerApiException;
 import com.cloud.api.commands.AttachVolumeCmd;
 import com.cloud.api.commands.CreateTemplateCmd;
+import com.cloud.api.commands.CreateVMGroupCmd;
+import com.cloud.api.commands.DeleteVMGroupCmd;
 import com.cloud.api.commands.DestroyVMCmd;
 import com.cloud.api.commands.DetachVolumeCmd;
 import com.cloud.api.commands.RebootVMCmd;
@@ -92,11 +94,11 @@ public interface UserVmManager extends Manager, VirtualMachineManager
      * @param diskOffering the disk offering for the root disk (deploying from ISO) or the data disk (deploying from a normal template)
      * @return UserVmVO if created; null if not.
      */
-    UserVmVO createVirtualMachine(Long vmId, long userId, AccountVO account, DataCenterVO dc, ServiceOfferingVO offering, VMTemplateVO template, DiskOfferingVO diskOffering, String displayName, String group, String userData, List avoids, long startEventId, long size) throws InsufficientStorageCapacityException, InternalErrorException, ResourceAllocationException;
+    UserVmVO createVirtualMachine(Long vmId, long userId, AccountVO account, DataCenterVO dc, ServiceOfferingVO offering, VMTemplateVO template, DiskOfferingVO diskOffering, String displayName, String userData, List avoids, long startEventId, long size) throws InsufficientStorageCapacityException, InternalErrorException, ResourceAllocationException;
     
-	UserVmVO createDirectlyAttachedVM(Long vmId, long userId, AccountVO account, DataCenterVO dc, ServiceOfferingVO offering, VMTemplateVO template, DiskOfferingVO diskOffering, String displayName, String group, String userData, List a, List networkGroupVO, long startEventId, long size) throws InternalErrorException, ResourceAllocationException;
+	UserVmVO createDirectlyAttachedVM(Long vmId, long userId, AccountVO account, DataCenterVO dc, ServiceOfferingVO offering, VMTemplateVO template, DiskOfferingVO diskOffering, String displayName, String userData, List a, List networkGroupVO, long startEventId, long size) throws InternalErrorException, ResourceAllocationException;
 
-	UserVmVO createDirectlyAttachedVMExternal(Long vmId, long userId, AccountVO account, DataCenterVO dc, ServiceOfferingVO offering, VMTemplateVO template, DiskOfferingVO diskOffering, String displayName, String group, String userData, List a, List networkGroupVO, long startEventId, long size) throws InternalErrorException, ResourceAllocationException;
+	UserVmVO createDirectlyAttachedVMExternal(Long vmId, long userId, AccountVO account, DataCenterVO dc, ServiceOfferingVO offering, VMTemplateVO template, DiskOfferingVO diskOffering, String displayName, String userData, List a, List networkGroupVO, long startEventId, long size) throws InternalErrorException, ResourceAllocationException;
 
     /**
      * Destroys one virtual machine
@@ -241,6 +243,19 @@ public interface UserVmManager extends Manager, VirtualMachineManager
      */
     void releaseGuestIpAddress(UserVmVO userVm);
 
-	void updateVirtualMachine(UpdateVMCmd cmd);
+    /**
+     * Creates a vm group.
+     * @param name - name of the group
+     * @param accountId - accountId
+     */
+    InstanceGroupVO createVmGroup(CreateVMGroupCmd cmd) throws InvalidParameterValueException, PermissionDeniedException;
 
+    boolean deleteVmGroup(DeleteVMGroupCmd cmd) throws InvalidParameterValueException, PermissionDeniedException;
+    boolean deleteVmGroup(long groupId);
+
+    boolean addInstanceToGroup(long userVmId, String group);
+
+    InstanceGroupVO getGroupForVm(long vmId);
+
+	void updateVirtualMachine(UpdateVMCmd cmd);
 }
diff --git a/server/src/com/cloud/vm/UserVmManagerImpl.java b/server/src/com/cloud/vm/UserVmManagerImpl.java
index 76497c02fe5..db00c29b90f 100644
--- a/server/src/com/cloud/vm/UserVmManagerImpl.java
+++ b/server/src/com/cloud/vm/UserVmManagerImpl.java
@@ -66,6 +66,8 @@ import com.cloud.api.BaseCmd;
 import com.cloud.api.ServerApiException;
 import com.cloud.api.commands.AttachVolumeCmd;
 import com.cloud.api.commands.CreateTemplateCmd;
+import com.cloud.api.commands.CreateVMGroupCmd;
+import com.cloud.api.commands.DeleteVMGroupCmd;
 import com.cloud.api.commands.DestroyVMCmd;
 import com.cloud.api.commands.DetachVolumeCmd;
 import com.cloud.api.commands.RebootVMCmd;
@@ -96,6 +98,7 @@ import com.cloud.dc.DataCenterVO;
 import com.cloud.dc.HostPodVO;
 import com.cloud.dc.Vlan.VlanType;
 import com.cloud.dc.VlanVO;
+import com.cloud.dc.dao.AccountVlanMapDao;
 import com.cloud.dc.dao.DataCenterDao;
 import com.cloud.dc.dao.HostPodDao;
 import com.cloud.dc.dao.VlanDao;
@@ -134,8 +137,8 @@ import com.cloud.network.dao.SecurityGroupDao;
 import com.cloud.network.dao.SecurityGroupVMMapDao;
 import com.cloud.network.security.NetworkGroupManager;
 import com.cloud.network.security.NetworkGroupVO;
+import com.cloud.offering.NetworkOffering;
 import com.cloud.offering.ServiceOffering;
-import com.cloud.offering.ServiceOffering.GuestIpType;
 import com.cloud.offerings.NetworkOfferingVO;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.service.dao.ServiceOfferingDao;
@@ -151,7 +154,6 @@ import com.cloud.storage.StoragePoolVO;
 import com.cloud.storage.VMTemplateHostVO;
 import com.cloud.storage.VMTemplateStorageResourceAssoc.Status;
 import com.cloud.storage.VMTemplateVO;
-import com.cloud.storage.VirtualMachineTemplate.BootloaderType;
 import com.cloud.storage.Volume;
 import com.cloud.storage.Volume.VolumeType;
 import com.cloud.storage.VolumeVO;
@@ -166,6 +168,8 @@ import com.cloud.storage.dao.VMTemplateDao;
 import com.cloud.storage.dao.VMTemplateHostDao;
 import com.cloud.storage.dao.VolumeDao;
 import com.cloud.user.Account;
+import com.cloud.storage.snapshot.SnapshotManager;
+import com.cloud.template.VirtualMachineTemplate.BootloaderType;
 import com.cloud.user.AccountManager;
 import com.cloud.user.AccountVO;
 import com.cloud.user.User;
@@ -185,6 +189,7 @@ import com.cloud.utils.component.Inject;
 import com.cloud.utils.concurrency.NamedThreadFactory;
 import com.cloud.utils.db.DB;
 import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.db.SearchCriteria;
 import com.cloud.utils.db.Transaction;
 import com.cloud.utils.exception.CloudRuntimeException;
 import com.cloud.utils.exception.ExecutionException;
@@ -193,7 +198,9 @@ import com.cloud.vm.DomainRouter.Role;
 import com.cloud.vm.VirtualMachine.Event;
 import com.cloud.vm.VirtualMachine.Type;
 import com.cloud.vm.dao.DomainRouterDao;
+import com.cloud.vm.dao.InstanceGroupVMMapDao;
 import com.cloud.vm.dao.UserVmDao;
+import com.cloud.vm.dao.InstanceGroupDao;
 
 @Local(value={UserVmManager.class})
 public class UserVmManagerImpl implements UserVmManager {
@@ -224,6 +231,7 @@ public class UserVmManagerImpl implements UserVmManager {
     @Inject CapacityDao _capacityDao = null;
     @Inject NetworkManager _networkMgr = null;
     @Inject StorageManager _storageMgr = null;
+    @Inject SnapshotManager _snapshotMgr = null;
     @Inject AgentManager _agentMgr = null;
     @Inject ConfigurationManager _configMgr = null;
     @Inject AccountDao _accountDao = null;
@@ -237,12 +245,15 @@ public class UserVmManagerImpl implements UserVmManager {
     @Inject AsyncJobManager _asyncMgr;
     @Inject protected StoragePoolHostDao _storagePoolHostDao;
     @Inject VlanDao _vlanDao;
+    @Inject AccountVlanMapDao _accountVlanMapDao;
     @Inject StoragePoolDao _storagePoolDao;
     @Inject VMTemplateHostDao _vmTemplateHostDao;
     @Inject NetworkGroupManager _networkGroupManager;
     @Inject ServiceOfferingDao _serviceOfferingDao;
     @Inject EventDao _eventDao = null;
-    
+    @Inject InstanceGroupDao _vmGroupDao;
+    @Inject InstanceGroupVMMapDao _groupVMMapDao;
+
     private IpAddrAllocator _IpAllocator;
     ScheduledExecutorService _executor = null;
     int _expungeInterval;
@@ -410,10 +421,10 @@ public class UserVmManagerImpl implements UserVmManager {
     	// If the account is not an admin, check that the volume and the virtual machine are owned by the account that was passed in
     	if (account != null) {
     	    if (!isAdmin(account.getType())) {
-                if (account.getId().longValue() != volume.getAccountId())
+                if (account.getId() != volume.getAccountId())
                     throw new PermissionDeniedException("Unable to find volume with ID: " + volumeId + " for account: " + account.getAccountName() + ". Permission denied.");
 
-                if (account.getId().longValue() != vm.getAccountId())
+                if (account.getId() != vm.getAccountId())
                     throw new PermissionDeniedException("Unable to find VM with ID: " + vmId + " for account: " + account.getAccountName() + ". Permission denied");
     	    } else {
     	        if (!_domainDao.isChildDomain(account.getDomainId(), volume.getDomainId()) ||
@@ -557,11 +568,31 @@ public class UserVmManagerImpl implements UserVmManager {
     }
     
     @Override
-    public void detachVolumeFromVM(DetachVolumeCmd cmmd) throws InternalErrorException, InvalidParameterValueException {
-    	
+    public void detachVolumeFromVM(DetachVolumeCmd cmmd) throws InternalErrorException, InvalidParameterValueException {    	
     	Account account = (Account) UserContext.current().getAccountObject();
+    	if ((cmmd.getId() == null && cmmd.getDeviceId() == null && cmmd.getVirtualMachineId() == null) ||
+    	    (cmmd.getId() != null && (cmmd.getDeviceId() != null || cmmd.getVirtualMachineId() != null)) ||
+    	    (cmmd.getId() == null && (cmmd.getDeviceId()==null || cmmd.getVirtualMachineId() == null))) {
+    	    throw new InvalidParameterValueException("Please provide either a volume id, or a tuple(device id, instance id)");
+    	}
+
     	Long volumeId = cmmd.getId();
+    	VolumeVO volume = null;
     	
+    	if(volumeId != null) {
+    		volume = _volsDao.findById(volumeId);
+    	} else {
+    		volume = _volsDao.findByInstanceAndDeviceId(cmmd.getVirtualMachineId(), cmmd.getDeviceId()).get(0);
+    	}
+
+    	Long vmId = null;
+    	
+    	if (cmmd.getVirtualMachineId() == null) {
+    		vmId = volume.getInstanceId();
+    	} else {
+    		vmId = cmmd.getVirtualMachineId();
+    	}
+
     	boolean isAdmin;
     	if (account == null) {
     		// Admin API call
@@ -572,7 +603,6 @@ public class UserVmManagerImpl implements UserVmManager {
     	}
 
     	// Check that the volume ID is valid
-    	VolumeVO volume = _volsDao.findById(volumeId);
     	if (volume == null)
     		throw new ServerApiException(BaseCmd.PARAM_ERROR, "Unable to find volume with ID: " + volumeId);
 
@@ -596,8 +626,6 @@ public class UserVmManagerImpl implements UserVmManager {
             throw new InvalidParameterValueException("Please specify a volume that has been created on a shared storage pool.");
         }
 
-        Long vmId = volume.getInstanceId();
-
         // Check that the volume is currently attached to a VM
         if (vmId == null) {
             throw new InvalidParameterValueException("The specified volume is not attached to a VM.");
@@ -627,7 +655,7 @@ public class UserVmManagerImpl implements UserVmManager {
     	Answer answer = null;
     	
     	if (sendCommand) {
-			AttachVolumeCommand cmd = new AttachVolumeCommand(false, vm.getInstanceName(), volume.getPoolType(), volume.getFolder(), volume.getPath(), volume.getName(), volume.getDeviceId());
+			AttachVolumeCommand cmd = new AttachVolumeCommand(false, vm.getInstanceName(), volume.getPoolType(), volume.getFolder(), volume.getPath(), volume.getName(), cmmd.getDeviceId() != null ? cmmd.getDeviceId() : volume.getDeviceId());
 			
 			try {
     			answer = _agentMgr.send(vm.getHostId(), cmd);
@@ -845,6 +873,7 @@ public class UserVmManagerImpl implements UserVmManager {
     		_eventDao.persist(event);
             return null;
         }
+        
 
         if (!_vmDao.updateIf(vm, Event.StartRequested, host.getId())) {
             String description = "Unable to start VM " + vm.toString() + " because the state is not correct.";
@@ -856,6 +885,7 @@ public class UserVmManagerImpl implements UserVmManager {
         }
         
         boolean started = false;
+       
         Transaction txn = Transaction.currentTxn();
         try {
             
@@ -906,6 +936,11 @@ public class UserVmManagerImpl implements UserVmManager {
 	        VolumeVO vol = rootVols.get(0);
 
 	        List vols = _volsDao.findCreatedByInstance(vm.getId());
+	        List vos = new ArrayList();
+	        /*compete with take snapshot*/
+	        for (VolumeVO userVmVol : vols) {
+	        	vos.add(_volsDao.lock(userVmVol.getId(), true));
+	        }
 
             Answer answer = null;
             int retry = _retry;
@@ -1342,7 +1377,7 @@ public class UserVmManagerImpl implements UserVmManager {
 
 	private Long accountAndUserValidation(Long virtualMachineId,Account account, Long userId, UserVmVO vmInstance) throws ServerApiException {
 		if (account != null) {
-            if (!isAdmin(account.getType()) && (account.getId().longValue() != vmInstance.getAccountId())) {
+            if (!isAdmin(account.getType()) && (account.getId() != vmInstance.getAccountId())) {
                 throw new ServerApiException(BaseCmd.VM_INVALID_PARAM_ERROR, "unable to find a virtual machine with id " + virtualMachineId + " for this account");
             } else if (!_domainDao.isChildDomain(account.getDomainId(),vmInstance.getDomainId())) {
                 throw new ServerApiException(BaseCmd.PARAM_ERROR, "Invalid virtual machine id (" + virtualMachineId + ") given, unable to upgrade virtual machine.");
@@ -1446,7 +1481,7 @@ public class UserVmManagerImpl implements UserVmManager {
     public void releaseGuestIpAddress(UserVmVO userVm)  {
     	ServiceOffering offering = _offeringDao.findById(userVm.getServiceOfferingId());
     	
-    	if (offering.getGuestIpType() != GuestIpType.Virtualized) {  		
+    	if (offering.getGuestIpType() != NetworkOffering.GuestIpType.Virtualized) {  		
     		IPAddressVO guestIP = (userVm.getGuestIpAddress() == null) ? null : _ipAddressDao.findById(userVm.getGuestIpAddress());
     		if (guestIP != null && guestIP.getAllocated() != null) {
     			_ipAddressDao.unassignIpAddress(userVm.getGuestIpAddress());
@@ -1644,7 +1679,7 @@ public class UserVmManagerImpl implements UserVmManager {
     }
     
     @Override @DB
-    public UserVmVO createVirtualMachine(Long vmId, long userId, AccountVO account, DataCenterVO dc, ServiceOfferingVO offering, VMTemplateVO template, DiskOfferingVO diskOffering, String displayName, String group, String userData, List avoids, long startEventId, long size) throws InternalErrorException, ResourceAllocationException {
+    public UserVmVO createVirtualMachine(Long vmId, long userId, AccountVO account, DataCenterVO dc, ServiceOfferingVO offering, VMTemplateVO template, DiskOfferingVO diskOffering, String displayName, String userData, List avoids, long startEventId, long size) throws InternalErrorException, ResourceAllocationException {
         long accountId = account.getId();
         long dataCenterId = dc.getId();
         long serviceOfferingId = offering.getId();
@@ -1714,11 +1749,11 @@ public class UserVmManagerImpl implements UserVmManager {
 
             while ((pod = _agentMgr.findPod(template, offering, dc, account.getId(), podsToAvoid)) != null) {
                 if (vm == null) {
-                    vm = new UserVmVO(vmId, name, template.getId(), guestOSId, accountId, account.getDomainId().longValue(),
+                    vm = new UserVmVO(vmId, name, template.getId(), guestOSId, accountId, account.getDomainId(),
                     		serviceOfferingId, null, null, router.getGuestNetmask(),
                     		null,null,null,
                     		routerId, pod.first().getId(), dataCenterId,
-                    		offering.getOfferHA(), displayName, group, userData);
+                    		offering.getOfferHA(), displayName, userData);
                     
                     if (diskOffering != null) {
                     	vm.setMirroredVols(diskOffering.isMirrored());
@@ -1757,6 +1792,10 @@ public class UserVmManagerImpl implements UserVmManager {
                 podsToAvoid.add(pod.first().getId());
             }
 
+            if(pod == null){
+                throw new ResourceAllocationException("Create VM " + ((vm == null) ? vmId : vm.toString()) + " failed. There are no pods with enough CPU/memory");
+            }
+            
             if ((vm == null) || (poolid == 0)) {
                 throw new ResourceAllocationException("Create VM " + ((vm == null) ? vmId : vm.toString()) + " failed due to no Storage Pool is available");
             }
@@ -1786,7 +1825,7 @@ public class UserVmManagerImpl implements UserVmManager {
         } catch (Throwable th) {
             s_logger.error("Unable to create vm", th);
             if (vm != null) {
-            	_vmDao.delete(vmId);
+            	_vmDao.expunge(vmId);
             }
             _accountMgr.decrementResourceCount(account.getId(), ResourceType.user_vm);
             _accountMgr.decrementResourceCount(account.getId(), ResourceType.volume, numVolumes);
@@ -2055,9 +2094,7 @@ public class UserVmManagerImpl implements UserVmManager {
             if(volume.getTemplateId() !=null){
                 templateId = volume.getTemplateId();
             }
-            if(volume.getDiskOfferingId() !=null){
-                diskOfferingId = volume.getDiskOfferingId();
-            }
+            diskOfferingId = volume.getDiskOfferingId();
             long sizeMB = volume.getSize()/(1024*1024);
             String eventParams = "id=" + volume.getId() +"\ndoId="+diskOfferingId+"\ntId="+templateId+"\ndcId="+volume.getDataCenterId()+"\nsize="+sizeMB;
             EventVO volEvent = new EventVO();
@@ -2127,6 +2164,7 @@ public class UserVmManagerImpl implements UserVmManager {
         	Enumeration it = ipAllocators.enumeration();
         	_IpAllocator = it.nextElement();
         }
+                
         return true;
     }
 
@@ -2429,6 +2467,7 @@ public class UserVmManagerImpl implements UserVmManager {
                 
                 _vmDao.remove(vm.getId());
                 _networkGroupManager.removeInstanceFromGroups(vm.getId());
+                removeInstanceFromGroup(vm.getId());
                 
                 s_logger.debug("vm is destroyed");
             } catch (Exception e) {
@@ -2490,7 +2529,7 @@ public class UserVmManagerImpl implements UserVmManager {
 
             if ((answer != null) && answer.getResult()) {
                 // delete the snapshot from the database
-                _snapshotDao.delete(snapshotId);
+                _snapshotDao.expunge(snapshotId);
                 success = true;
             }
             if (answer != null) {
@@ -2504,7 +2543,7 @@ public class UserVmManagerImpl implements UserVmManager {
     @Override @DB
     public SnapshotVO createTemplateSnapshot(long userId, long volumeId) {
         SnapshotVO createdSnapshot = null;
-        VolumeVO volume = _volsDao.findById(volumeId);
+        VolumeVO volume = _volsDao.lock(volumeId, true);
         
         Long id = null;
         
@@ -2546,7 +2585,7 @@ public class UserVmManagerImpl implements UserVmManager {
             Transaction txn = Transaction.currentTxn();
             txn.start();
             createdSnapshot = _snapshotDao.findById(id);
-            _snapshotDao.delete(id);
+            _snapshotDao.expunge(id);
             txn.commit();
             
             createdSnapshot = null;
@@ -2691,7 +2730,7 @@ public class UserVmManagerImpl implements UserVmManager {
     	}
 
         if (!isAdmin) {
-            if (account.getId().longValue() != volume.getAccountId()) {
+            if (account.getId() != volume.getAccountId()) {
                 throw new PermissionDeniedException("Unable to create a template from volume with id " + volumeId + ", permission denied.");
             }
         } else if ((account != null) && !_domainDao.isChildDomain(account.getDomainId(), volume.getDomainId())) {
@@ -2852,10 +2891,11 @@ public class UserVmManagerImpl implements UserVmManager {
                 Long accountId = volume.getAccountId();
                 
                 String origTemplateInstallPath = null;
-                Long origTemplateId = volume.getTemplateId();
+
                 
-                if (origTemplateId != null) {
-                	VMTemplateHostVO vmTemplateHostVO = _templateHostDao.findByHostTemplate(secondaryStorageHost.getId(), origTemplateId);
+                if (ImageFormat.ISO != _snapshotMgr.getImageFormat(volumeId)) {
+                    Long origTemplateId = volume.getTemplateId();
+                    VMTemplateHostVO vmTemplateHostVO = _templateHostDao.findByHostTemplate(secondaryStorageHost.getId(), origTemplateId);
                     origTemplateInstallPath = vmTemplateHostVO.getInstallPath();
                 }
                 
@@ -2948,7 +2988,7 @@ public class UserVmManagerImpl implements UserVmManager {
     
     @DB
     @Override
-	public UserVmVO createDirectlyAttachedVM(Long vmId, long userId, AccountVO account, DataCenterVO dc, ServiceOfferingVO offering, VMTemplateVO template, DiskOfferingVO diskOffering, String displayName, String group, String userData, List a, List  networkGroups, long startEventId, long size) throws InternalErrorException, ResourceAllocationException {
+	public UserVmVO createDirectlyAttachedVM(Long vmId, long userId, AccountVO account, DataCenterVO dc, ServiceOfferingVO offering, VMTemplateVO template, DiskOfferingVO diskOffering, String displayName, String userData, List a, List  networkGroups, long startEventId, long size) throws InternalErrorException, ResourceAllocationException {
     	
     	long accountId = account.getId();
 	    long dataCenterId = dc.getId();
@@ -3005,18 +3045,36 @@ public class UserVmManagerImpl implements UserVmManager {
             Set avoids = new HashSet();
             VlanVO guestVlan = null;
             List vlansForAccount = _vlanDao.listVlansForAccountByType(dc.getId(), account.getId(), VlanType.DirectAttached);
-           
+            List vlansForPod = null;
+            List zoneWideVlans = null;
+            
             boolean forAccount = false;
+            boolean forZone = false;
             if (vlansForAccount.size() > 0) {
             	forAccount = true;
             	guestVlan = vlansForAccount.get(0);//FIXME: iterate over all vlans
             }
+            else 
+            {
+            	//list zone wide vlans that are direct attached and tagged
+            	//if exists pick random one
+            	//set forZone = true
+            	
+            	//note the dao method below does a NEQ on vlan id, hence passing untagged
+            	zoneWideVlans = _vlanDao.searchForZoneWideVlans(dc.getId(),VlanType.DirectAttached.toString(),"untagged");
+            	
+            	if(zoneWideVlans!=null && zoneWideVlans.size()>0){
+            		forZone = true;
+            		guestVlan = zoneWideVlans.get(0);//FIXME: iterate over all vlans
+            	}
+            }
+            
             while ((pod = _agentMgr.findPod(template, offering, dc, account.getId(), avoids)) != null) {
                 if (s_logger.isDebugEnabled()) {
                     s_logger.debug("Attempting to create direct attached vm in pod " + pod.first().getName());
                 }
-                if (!forAccount) {
-                	List vlansForPod = _vlanDao.listVlansForPodByType(pod.first().getId(), VlanType.DirectAttached);
+                if (!forAccount && !forZone) {
+                	vlansForPod = _vlanDao.listVlansForPodByType(pod.first().getId(), VlanType.DirectAttached);
                 	if (vlansForPod.size() < 1) {
                 		avoids.add(pod.first().getId());
                 		if (s_logger.isDebugEnabled()) {
@@ -3043,7 +3101,38 @@ public class UserVmManagerImpl implements UserVmManager {
                 	}
                 	routerId = router.getId();
                 }
-                String guestIp = _ipAddressDao.assignIpAddress(accountId, account.getDomainId().longValue(), guestVlan.getId(), false);
+                String guestIp = null;
+                
+                if(forAccount)
+                {
+                	for(VlanVO vlanForAcc : vlansForAccount)
+                	{
+                		guestIp = _ipAddressDao.assignIpAddress(accountId, account.getDomainId(), vlanForAcc.getId(), false);
+                		if(guestIp!=null)
+                			break; //got an ip
+                	}
+                }
+                else if(!forAccount && !forZone)
+                {
+                	//i.e. for pod
+                	for(VlanVO vlanForPod : vlansForPod)
+                	{
+                		guestIp = _ipAddressDao.assignIpAddress(accountId, account.getDomainId(), vlanForPod.getId(), false);
+                		if(guestIp!=null)
+                			break;//got an ip
+                	}
+                }
+                else
+                {
+                	//for zone
+                	for(VlanVO vlanForZone : zoneWideVlans)
+                	{
+                		guestIp = _ipAddressDao.assignIpAddress(accountId, account.getDomainId(), vlanForZone.getId(), false);
+                		if(guestIp!=null)
+                			break;//found an ip
+                	}
+                }
+                
                 if (guestIp == null) {
                 	s_logger.debug("No guest IP available in pod id=" + pod.first().getId());
                 	avoids.add(pod.first().getId());
@@ -3054,11 +3143,11 @@ public class UserVmManagerImpl implements UserVmManager {
                 String externalMacAddress = macAddresses[1];
                 Long externalVlanDbId = null;
             
-	            vm = new UserVmVO(vmId, name, templateId, guestOSId, accountId, account.getDomainId().longValue(),
+	            vm = new UserVmVO(vmId, name, templateId, guestOSId, accountId, account.getDomainId(),
 	            		serviceOfferingId, guestMacAddress, guestIp, guestVlan.getVlanNetmask(),
 	            		null, externalMacAddress, externalVlanDbId,
 	            		routerId, pod.first().getId(), dataCenterId,
-	            		offering.getOfferHA(), displayName, group, userData);
+	            		offering.getOfferHA(), displayName, userData);
 	            
 	            if (diskOffering != null) {
                 	vm.setMirroredVols(diskOffering.isMirrored());
@@ -3069,7 +3158,7 @@ public class UserVmManagerImpl implements UserVmManager {
 	            boolean addedToGroups = _networkGroupManager.addInstanceToGroups(vmId, networkGroups);
 	            if (!addedToGroups) {
 	            	s_logger.warn("Not all specified network groups can be found");
-	            	_vmDao.delete(vm.getId());
+	            	_vmDao.expunge(vm.getId());
 	            	throw new InvalidParameterValueException("Not all specified network groups can be found");
 	            }
 	            
@@ -3077,7 +3166,7 @@ public class UserVmManagerImpl implements UserVmManager {
 	            try {
 	            	poolId = _storageMgr.createUserVM(account,  vm, template, dc, pod.first(), offering, diskOffering, a,size);
 	            } catch (CloudRuntimeException e) {
-	            	_vmDao.delete(vmId);
+	            	_vmDao.expunge(vmId);
 	                _ipAddressDao.unassignIpAddress(guestIp);
 	                s_logger.debug("Released a guest ip address because we could not find storage: ip=" + guestIp);
 	                guestIp = null;
@@ -3110,7 +3199,11 @@ public class UserVmManagerImpl implements UserVmManager {
 	        	else
 	        	{
 	        		s_logger.debug("failed to create VM instance : " + name);
+	        		throw new InternalErrorException("We could not find a suitable pool for creating this directly attached vm");
+	        		
 	        	}
+	            _accountMgr.decrementResourceCount(account.getId(), ResourceType.user_vm);
+	            _accountMgr.decrementResourceCount(account.getId(), ResourceType.volume, numVolumes);
 	            return null;
 	        }
 	        
@@ -3142,13 +3235,13 @@ public class UserVmManagerImpl implements UserVmManager {
             _accountMgr.decrementResourceCount(account.getId(), ResourceType.volume, numVolumes);
 
 	        s_logger.error("Unable to create vm", th);
-	        throw new CloudRuntimeException("Unable to create vm", th);
+	        throw new CloudRuntimeException("Unable to create vm: "+th.getMessage(), th);
 	    }
 	}
     
     @DB
     @Override
-	public UserVmVO createDirectlyAttachedVMExternal(Long vmId, long userId, AccountVO account, DataCenterVO dc, ServiceOfferingVO offering, VMTemplateVO template, DiskOfferingVO diskOffering, String displayName, String group, String userData, List a, List  networkGroups, long startEventId, long size) throws InternalErrorException, ResourceAllocationException {
+	public UserVmVO createDirectlyAttachedVMExternal(Long vmId, long userId, AccountVO account, DataCenterVO dc, ServiceOfferingVO offering, VMTemplateVO template, DiskOfferingVO diskOffering, String displayName, String userData, List a, List  networkGroups, long startEventId, long size) throws InternalErrorException, ResourceAllocationException {
 	    long accountId = account.getId();
 	    long dataCenterId = dc.getId();
 	    long serviceOfferingId = offering.getId();
@@ -3171,6 +3264,9 @@ public class UserVmManagerImpl implements UserVmManager {
         	throw new InternalErrorException("No template or ISO was specified for the VM.");
         }
 	    
+        boolean isIso = Storage.ImageFormat.ISO.equals(template.getFormat());
+        long numVolumes = (isIso || (diskOffering == null)) ? 1 : 2;
+        
 	    Transaction txn = Transaction.currentTxn();
 	    try {
 	        UserVmVO vm = null;
@@ -3212,11 +3308,11 @@ public class UserVmManagerImpl implements UserVmManager {
                 	publicIpAddr = publicIp.ipaddr;
                 	publicIpNetMask = publicIp.netMask;
                 }
-	            vm = new UserVmVO(vmId, name, templateId, guestOSId, accountId, account.getDomainId().longValue(),
+	            vm = new UserVmVO(vmId, name, templateId, guestOSId, accountId, account.getDomainId(),
 	            		serviceOfferingId, guestMacAddress, publicIpAddr, publicIpNetMask,
 	            		null, externalMacAddress, null,
 	            		routerId, pod.first().getId(), dataCenterId,
-	            		offering.getOfferHA(), displayName, group, userData);
+	            		offering.getOfferHA(), displayName, userData);
 	            
 	            if (diskOffering != null) {
                 	vm.setMirroredVols(diskOffering.isMirrored());
@@ -3227,8 +3323,6 @@ public class UserVmManagerImpl implements UserVmManager {
 	            _networkGroupManager.addInstanceToGroups(vmId, networkGroups);
 	            
 	            _accountMgr.incrementResourceCount(account.getId(), ResourceType.user_vm);
-	            boolean isIso = Storage.ImageFormat.ISO.equals(template.getFormat());
-	            long numVolumes = (isIso || (diskOffering == null)) ? 1 : 2;
 	            _accountMgr.incrementResourceCount(account.getId(), ResourceType.volume, numVolumes);
 	            txn.commit();
 	
@@ -3236,7 +3330,7 @@ public class UserVmManagerImpl implements UserVmManager {
 	            try {
 	            	poolId = _storageMgr.createUserVM(account,  vm, template, dc, pod.first(), offering, diskOffering,a,size);
 	            } catch (CloudRuntimeException e) {
-	            	_vmDao.delete(vmId);
+	            	_vmDao.expunge(vmId);
 	                _accountMgr.decrementResourceCount(account.getId(), ResourceType.user_vm);
 	                _accountMgr.decrementResourceCount(account.getId(), ResourceType.volume, numVolumes);
 	                if (s_logger.isDebugEnabled()) {
@@ -3260,7 +3354,9 @@ public class UserVmManagerImpl implements UserVmManager {
 	        	{
 	        		s_logger.debug("failed to create VM instance : " + name);
 	        	}
-	        		          
+
+	            _accountMgr.decrementResourceCount(account.getId(), ResourceType.user_vm);
+	            _accountMgr.decrementResourceCount(account.getId(), ResourceType.volume, numVolumes);
 	            txn.commit();
 	            return null;
 	        }
@@ -3289,11 +3385,15 @@ public class UserVmManagerImpl implements UserVmManager {
 	
 	        return _vmDao.findById(vmId);
 	    } catch (ResourceAllocationException rae) {
+            _accountMgr.decrementResourceCount(account.getId(), ResourceType.user_vm);
+            _accountMgr.decrementResourceCount(account.getId(), ResourceType.volume, numVolumes);
 	        if (s_logger.isInfoEnabled()) {
 	            s_logger.info("Failed to create VM for account " + accountId + " due to maximum number of virtual machines exceeded.");
 	        }
 	    	throw rae;
 	    } catch (Throwable th) {
+            _accountMgr.decrementResourceCount(account.getId(), ResourceType.user_vm);
+            _accountMgr.decrementResourceCount(account.getId(), ResourceType.volume, numVolumes);
 	        s_logger.error("Unable to create vm", th);
 	        throw new CloudRuntimeException("Unable to create vm", th);
 	    }
@@ -3329,7 +3429,7 @@ public class UserVmManagerImpl implements UserVmManager {
     		}
     	}
     }
-	
+
 	private static boolean isAdmin(short accountType) {
 	    return ((accountType == Account.ACCOUNT_TYPE_ADMIN) ||
 	            (accountType == Account.ACCOUNT_TYPE_DOMAIN_ADMIN) ||
@@ -3337,11 +3437,8 @@ public class UserVmManagerImpl implements UserVmManager {
 	}
 	
     @Override
-    public void updateVirtualMachine(UpdateVMCmd cmd) 
-    {
-
+    public void updateVirtualMachine(UpdateVMCmd cmd) {
         String displayName = cmd.getDisplayName();
-        String group = cmd.getGroup();
         Boolean ha = cmd.getHaEnable();
         Long id = cmd.getId();
         Account account = (Account)UserContext.current().getAccountObject();
@@ -3365,10 +3462,6 @@ public class UserVmManagerImpl implements UserVmManager {
         }
 
         userId = accountAndUserValidation(id, account, userId,vmInstance);  
-        
-        if (group == null) {
-    		group = vmInstance.getGroup();
-    	}
 
     	if (displayName == null) {
     		displayName = vmInstance.getDisplayName();
@@ -3379,25 +3472,20 @@ public class UserVmManagerImpl implements UserVmManager {
     	}
 
     	long accountId = vmInstance.getAccountId();
-
-        
         UserVmVO vm = _vmDao.findById(id);
         if (vm == null) {
             throw new CloudRuntimeException("Unable to find virual machine with id " + id);
         }
 
         boolean haEnabled = vm.isHaEnabled();
-        _vmDao.updateVM(id, displayName, group, ha);
+        _vmDao.updateVM(id, displayName, ha);
         if (haEnabled != ha) {
             String description = null;
             String type = null;
-            if (ha) 
-            {
+            if (ha) {
                 description = "Successfully enabled HA for virtual machine " + vm.getName();
                 type = EventTypes.EVENT_VM_ENABLE_HA;
-            } 
-            else 
-            {
+            } else {
                 description = "Successfully disabled HA for virtual machine " + vm.getName();
                 type = EventTypes.EVENT_VM_DISABLE_HA;
             }
@@ -3514,6 +3602,195 @@ public class UserVmManagerImpl implements UserVmManager {
         	return status;
         }
 	}
+
+    @Override @DB
+    public InstanceGroupVO createVmGroup(CreateVMGroupCmd cmd) throws InvalidParameterValueException, PermissionDeniedException {
+        Account account = (Account)UserContext.current().getAccountObject();
+        Long domainId = cmd.getDomainId();
+        String accountName = cmd.getAccountName();
+        Long accountId = null;
+        String groupName = cmd.getGroupName();
+
+        if (account != null) {
+            if (isAdmin(account.getType())) {
+                if ((domainId != null) && (accountName != null)) {
+                    if (!_domainDao.isChildDomain(account.getDomainId(), domainId)) {
+                        throw new PermissionDeniedException("Unable to create vm group in domain " + domainId + ", permission denied.");
+                    }
+
+                    Account userAccount = _accountDao.findActiveAccount(accountName, domainId);
+                    if (userAccount != null) {
+                        accountId = userAccount.getId();
+                    } else {
+                        throw new InvalidParameterValueException("Failed to create vm group " + groupName + ", unable to find account " + accountName + " in domain " + domainId);
+                    }
+                } else {
+                    // the admin must be creating the vm group
+                    accountId = account.getId();
+                }
+            } else {
+                accountId = account.getId();
+            }
+        }
+
+        if (accountId == null) {
+            throw new InvalidParameterValueException("Failed to create vm group " + groupName + ", unable to find account for which to create a group.");
+        }
+
+        //Check if name is already in use by this account
+        boolean isNameInUse = _vmGroupDao.isNameInUse(accountId, groupName);
+
+        if (isNameInUse) {
+            throw new InvalidParameterValueException("Unable to create vm group, a group with name " + groupName + " already exisits for account " + accountId);
+        }
+
+        return createVmGroup(groupName, accountId);
+    }
+
+    @DB
+	private InstanceGroupVO createVmGroup(String groupName, long accountId) {
+        Account account = null;
+	    final Transaction txn = Transaction.currentTxn();
+		txn.start();
+		try {
+			account = _accountDao.acquire(accountId); //to ensure duplicate vm group names are not created.
+			if (account == null) {
+				s_logger.warn("Failed to acquire lock on account");
+				return null;
+			}
+			InstanceGroupVO group = _vmGroupDao.findByAccountAndName(accountId, groupName);
+			if (group == null){
+				group = new InstanceGroupVO(groupName, accountId);
+				group =  _vmGroupDao.persist(group);
+			}
+			return group;
+		} finally {
+			if (account != null) {
+				_accountDao.release(accountId);
+			}
+			txn.commit();
+		}
+    }
+
+    @Override
+    public boolean deleteVmGroup(DeleteVMGroupCmd cmd) throws InvalidParameterValueException, PermissionDeniedException {
+        Account account = (Account)UserContext.current().getAccountObject();
+        Long groupId = cmd.getId();
+
+        // Verify input parameters
+        InstanceGroupVO group = _vmGroupDao.findById(groupId);
+        if ((group == null) || (group.getRemoved() != null)) {
+            throw new InvalidParameterValueException("unable to find a vm group with id " + groupId);
+        }
+
+        if (account != null) {
+            Account tempAccount = _accountDao.findById(group.getAccountId());
+            if (!isAdmin(account.getType()) && (account.getId() != group.getAccountId())) {
+                throw new PermissionDeniedException("unable to find a group with id " + groupId);
+            } else if (!_domainDao.isChildDomain(account.getDomainId(), tempAccount.getDomainId())) {
+                throw new PermissionDeniedException("Invalid group id (" + groupId + ") given, unable to update the group.");
+            }
+        }
+
+        return deleteVmGroup(groupId);
+    }
+
+    @Override
+    public boolean deleteVmGroup(long groupId) {    	
+    	//delete all the mappings from group_vm_map table
+        List groupVmMaps = _groupVMMapDao.listByGroupId(groupId);
+        for (InstanceGroupVMMapVO groupMap : groupVmMaps) {
+	        SearchCriteria sc = _groupVMMapDao.createSearchCriteria();
+	        sc.addAnd("instanceId", SearchCriteria.Op.EQ, groupMap.getInstanceId());
+	        _groupVMMapDao.expunge(sc);
+        }
+    	
+    	if (_vmGroupDao.remove(groupId)) {
+    		return true;
+    	} else {
+    		return false;
+    	}
+    }
+
+	@Override @DB
+	public boolean addInstanceToGroup(long userVmId, String groupName) {		
+		UserVmVO vm = _vmDao.findById(userVmId);
+
+        InstanceGroupVO group = _vmGroupDao.findByAccountAndName(vm.getAccountId(), groupName);
+    	//Create vm group if the group doesn't exist for this account
+        if (group == null) {
+        	group = createVmGroup(groupName, vm.getAccountId());
+        }
+
+		if (group != null) {
+			final Transaction txn = Transaction.currentTxn();
+			txn.start();
+			UserVm userVm = _vmDao.acquire(userVmId);
+			if (userVm == null) {
+				s_logger.warn("Failed to acquire lock on user vm id=" + userVmId);
+			}
+			try {
+				//don't let the group be deleted when we are assigning vm to it.
+				InstanceGroupVO ngrpLock = _vmGroupDao.lock(group.getId(), false);
+				if (ngrpLock == null) {
+					s_logger.warn("Failed to acquire lock on vm group id=" + group.getId() + " name=" + group.getName());
+					txn.rollback();
+					return false;
+				}
+				
+				//Currently don't allow to assign a vm to more than one group
+				if (_groupVMMapDao.listByInstanceId(userVmId) != null) {
+					//Delete all mappings from group_vm_map table
+			        List groupVmMaps = _groupVMMapDao.listByInstanceId(userVmId);
+			        for (InstanceGroupVMMapVO groupMap : groupVmMaps) {
+				        SearchCriteria sc = _groupVMMapDao.createSearchCriteria();
+				        sc.addAnd("instanceId", SearchCriteria.Op.EQ, groupMap.getInstanceId());
+				        _groupVMMapDao.expunge(sc);
+			        }
+				}
+				InstanceGroupVMMapVO groupVmMapVO = new InstanceGroupVMMapVO(group.getId(), userVmId);
+				_groupVMMapDao.persist(groupVmMapVO);
+				
+				txn.commit();
+				return true;
+			} finally {
+				if (userVm != null) {
+					_vmDao.release(userVmId);
+				}
+			}
+	    }
+		return false;
+	}
 	
+	@Override
+	public InstanceGroupVO getGroupForVm(long vmId) {
+		//TODO - in future releases vm can be assigned to multiple groups; but currently return just one group per vm
+		try {
+			List groupsToVmMap =  _groupVMMapDao.listByInstanceId(vmId);
+
+            if(groupsToVmMap != null && groupsToVmMap.size() != 0){
+            	InstanceGroupVO group = _vmGroupDao.findById(groupsToVmMap.get(0).getGroupId());
+            	return group;
+            } else {
+            	return null;
+            }
+		}
+		catch (Exception e){
+			s_logger.warn("Error trying to get group for a vm: "+e);
+			return null;
+		}
+	}
 	
+	public void removeInstanceFromGroup(long vmId) {
+		try {
+			List groupVmMaps = _groupVMMapDao.listByInstanceId(vmId);
+	        for (InstanceGroupVMMapVO groupMap : groupVmMaps) {
+		        SearchCriteria sc = _groupVMMapDao.createSearchCriteria();
+		        sc.addAnd("instanceId", SearchCriteria.Op.EQ, groupMap.getInstanceId());
+		        _groupVMMapDao.expunge(sc);
+	        }
+		} catch (Exception e){
+			s_logger.warn("Error trying to remove vm from group: "+e);
+		}
+	}
 }
diff --git a/server/src/com/cloud/vm/VmManager.java b/server/src/com/cloud/vm/VmManager.java
index 115bb18936b..7c33a8969dd 100644
--- a/server/src/com/cloud/vm/VmManager.java
+++ b/server/src/com/cloud/vm/VmManager.java
@@ -17,25 +17,54 @@
  */
 package com.cloud.vm;
 
-import com.cloud.offerings.NetworkOfferingVO;
+import java.util.List;
+
+import com.cloud.deploy.DeploymentPlan;
+import com.cloud.exception.AgentUnavailableException;
+import com.cloud.exception.ConcurrentOperationException;
+import com.cloud.exception.InsufficientCapacityException;
+import com.cloud.exception.StorageUnavailableException;
+import com.cloud.network.NetworkConfigurationVO;
 import com.cloud.service.ServiceOfferingVO;
 import com.cloud.storage.DiskOfferingVO;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.user.AccountVO;
+import com.cloud.utils.Pair;
+import com.cloud.utils.component.Manager;
 
 /**
  * Manages allocating resources to vms.
  */
-public interface VmManager {
+public interface VmManager extends Manager {
     
-    VMInstanceVO allocate(VMInstanceVO vm, 
-            ServiceOfferingVO serviceOffering, 
-            NetworkOfferingVO[] networkOfferings, 
-            DiskOfferingVO[] diskOffering);
+     VirtualMachineProfile allocate(T vm,
+            VMTemplateVO template,
+            ServiceOfferingVO serviceOffering,
+            Pair rootDiskOffering,
+            List> dataDiskOfferings,
+            List> networks, 
+            DeploymentPlan plan,
+            AccountVO owner) throws InsufficientCapacityException, StorageUnavailableException;
     
-    void create(VMInstanceVO vm);
+     VirtualMachineProfile allocate(T vm,
+            VMTemplateVO template,
+            ServiceOfferingVO serviceOffering,
+            Long rootSize,
+            Pair dataDiskOffering,
+            List> networks,
+            DeploymentPlan plan,
+            AccountVO owner) throws InsufficientCapacityException, StorageUnavailableException;
     
-    void start();
+     VirtualMachineProfile allocate(T vm,
+            VMTemplateVO template,
+            ServiceOfferingVO serviceOffering,
+            List networkProfiles,
+            DeploymentPlan plan,
+            AccountVO owner) throws InsufficientCapacityException, StorageUnavailableException;
     
-    void stop();
+     T start(T vm, DeploymentPlan plan) throws InsufficientCapacityException, StorageUnavailableException, ConcurrentOperationException;
+    
+     T stop(T vm) throws AgentUnavailableException, ConcurrentOperationException;
     
     void destroy();
     
diff --git a/server/src/com/cloud/vm/dao/NicDao.java b/server/src/com/cloud/vm/dao/NicDao.java
new file mode 100644
index 00000000000..6f4e3616a3e
--- /dev/null
+++ b/server/src/com/cloud/vm/dao/NicDao.java
@@ -0,0 +1,14 @@
+/**
+ * 
+ */
+package com.cloud.vm.dao;
+
+import java.util.List;
+
+import com.cloud.utils.db.GenericDao;
+import com.cloud.vm.NicVO;
+
+public interface NicDao extends GenericDao {
+    List listBy(long instanceId);
+
+}
diff --git a/server/src/com/cloud/vm/dao/NicDaoImpl.java b/server/src/com/cloud/vm/dao/NicDaoImpl.java
new file mode 100644
index 00000000000..c6b1eda5ac8
--- /dev/null
+++ b/server/src/com/cloud/vm/dao/NicDaoImpl.java
@@ -0,0 +1,34 @@
+/**
+ * 
+ */
+package com.cloud.vm.dao;
+
+import java.util.List;
+
+import javax.ejb.Local;
+
+import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.SearchBuilder;
+import com.cloud.utils.db.SearchCriteria;
+import com.cloud.vm.NicVO;
+
+@Local(value=NicDao.class)
+public class NicDaoImpl extends GenericDaoBase implements NicDao {
+    private final SearchBuilder InstanceSearch;
+    
+    protected NicDaoImpl() {
+        super();
+        
+        InstanceSearch = createSearchBuilder();
+        InstanceSearch.and("instance", InstanceSearch.entity().getInstanceId(), SearchCriteria.Op.EQ);
+        InstanceSearch.done();
+    }
+    
+    @Override
+    public List listBy(long instanceId) {
+        SearchCriteria sc = InstanceSearch.create();
+        sc.setParameters("instance", instanceId);
+        return listIncludingRemovedBy(sc);
+    }
+    
+}
diff --git a/server/test/async-job-component.xml b/server/test/async-job-component.xml
index 6694f44a7fb..38602dc3e92 100644
--- a/server/test/async-job-component.xml
+++ b/server/test/async-job-component.xml
@@ -79,6 +79,7 @@
         
         
         
+        
         
         
         
@@ -150,6 +151,8 @@
         
         
         
+        
+        
         
         
         
diff --git a/server/wscript_build b/server/wscript_build
new file mode 100644
index 00000000000..f6660680f77
--- /dev/null
+++ b/server/wscript_build
@@ -0,0 +1,4 @@
+import Options
+
+if not Options.options.PRESERVECONFIG:
+	bld.install_files_filtered("${SERVERSYSCONFDIR}","conf/*")
diff --git a/setup/bindir/cloud-migrate-databases.in b/setup/bindir/cloud-migrate-databases.in
index dab1515d50c..6adffa75d3d 100644
--- a/setup/bindir/cloud-migrate-databases.in
+++ b/setup/bindir/cloud-migrate-databases.in
@@ -150,6 +150,23 @@ class From21datamigratedTo21postprocessed(cloud_utils.MigrationStep):
 	to_level = "2.1"
 	def run(self): self.context.run_sql_resource("postprocess-20to21.sql")
 
+class From21To213(cloud_utils.MigrationStep):
+	def __str__(self): return "Dropping obsolete indexes"
+	from_level = "2.1"
+	to_level = "2.1.3"
+	def run(self): self.context.run_sql_resource("index-212to213.sql")
+
+class From213To22data(cloud_utils.MigrationStep):
+	def __str__(self): return "Migrating data"
+	from_level = "2.1.3"
+	to_level = "2.2-01"
+	def run(self): self.context.run_sql_resource("data-21to22.sql")
+
+class From22dataTo22(cloud_utils.MigrationStep):
+	def __str__(self): return "Migrating indexes"
+	from_level = "2.2-01"
+	to_level = "2.2"
+	def run(self): self.context.run_sql_resource("index-21to22.sql")
 
 # command line harness functions
 
diff --git a/setup/bindir/cloud-setup-databases.in b/setup/bindir/cloud-setup-databases.in
index d1d0154916d..9543cb19571 100755
--- a/setup/bindir/cloud-setup-databases.in
+++ b/setup/bindir/cloud-setup-databases.in
@@ -9,6 +9,11 @@ from random import choice
 import string
 from optparse import OptionParser
 import commands
+import MySQLdb
+
+# squelch mysqldb spurious warnings
+import warnings
+warnings.simplefilter('ignore')
 
 # ---- This snippet of code adds the sources path and the waf configured PYTHONDIR to the Python path ----
 # ---- We do this so cloud_utils can be looked up in the following order:
@@ -43,7 +48,7 @@ except ImportError:
 #---------------------- option parsing and command line checks ------------------------
 
 
-usage = """%prog user:password@mysqlhost:port  [--deploy-as=rootuser:rootpassword] [--auto=/path/to/server-setup.xml]
+usage = """%prog user:password@mysqlhost:port  [--deploy-as=rootuser:rootpassword] [--auto=/path/to/server-setup.xml]
 
 This command sets up the CloudStack Management Server and CloudStack Usage Server database configuration (connection credentials and host information) based on the first argument.
 
@@ -128,22 +133,34 @@ def get_creds(parser,options,args):
 	host,port = parse_hostport(hostinfo)
 	return (user,password,host,port)
 
-def run_mysql(text,user,password,host,port,extraargs=None):
-  cmd = ["mysql",
-    "--user=%s"%user,
-    "--host=%s"%host,
-  ]
-  if password: 
-    cmd.append("--password=%s"%password)
-  if password: 
-    cmd.append("--port=%s"%port)
-  if extraargs:
-    cmd.extend(extraargs)
-    
-  p = subprocess.Popen(cmd,stdin=subprocess.PIPE)
-  p.communicate(text)
-  ret = p.wait()
-  if ret != 0: raise CalledProcessError(ret,cmd)
+def run_mysql(text,user,password,host,port,debug=False):
+  kwargs = {}
+  kwargs['host'] = host
+  kwargs['user'] = user
+  if password: kwargs['passwd']   = password
+  if port: kwargs['port']   = port
+
+  conn = MySQLdb.connect(**kwargs)
+  cur = conn.cursor()
+  import re
+  exp = re.compile("DELIMITER (.*)$",re.M)
+  pairs = [";"]+[x.strip() for x in exp.split(text)]
+  delims = []
+  chunks = []
+  while pairs:
+      delims.append( pairs[0] )
+      chunks.append( pairs[1] )
+      pairs = pairs[2:]
+
+  for delim,chunk in zip(delims,chunks):
+      for stmt in chunk.split(delim):
+	stmt = stmt.strip()
+	if not stmt: continue
+	if debug: print stmt
+	cur.execute(stmt)
+  cur.close()
+  conn.commit()
+  conn.close()
 
 def ifaces():
     status,lines = commands.getstatusoutput('LANG=C /sbin/ip address show')
@@ -154,7 +171,8 @@ def ifaces():
 
 def ip(iface):
     status,lines = commands.getstatusoutput('LANG=C /sbin/ip address show %s'%iface)
-    assert status == 0
+    if status != 0: return False
+    #used to say: assert status == 0 but it caused a bug in ifaces without IP
     lines = [ l for l in lines.splitlines() if l.startswith('    inet ') ]
     if not lines: return None
     toks = lines[0].split()
@@ -224,7 +242,7 @@ def setupconfigfile(fn,myipaddr,username,password,host,port):
 (options, args) = parser.parse_args()
 
 if len(args) != 2: e("Wrong number of arguments")
-if args[1] not in ["xenserver","kvm"]: e("You need to specify either xenserver or kvm after the database credentials")
+if args[1] not in ["xenserver","kvm","vmware"]: e("You need to specify either xenserver or kvm or vmware after the database credentials")
 virttech = args[1]
 
 user,password,host,port= get_creds(parser,options,args)
@@ -239,8 +257,8 @@ if options.serversetup and not os.path.isfile(options.serversetup):
 	e("%s is not a valid file"%options.serversetup)
 
 
-dbfilepath = "@SETUPDATADIR@"
-dbppaths = [ os.path.join("@MSCONF@","db.properties") ] # , os.path.join("@USAGESYSCONFDIR@","db.properties") ]
+dbfilepath = r"@SETUPDATADIR@"
+dbppaths = [ os.path.join(r"@MSCONF@","db.properties") ] # , os.path.join("@USAGESYSCONFDIR@","db.properties") ]
 dbppaths = [ x for x in dbppaths if os.path.exists(x) ]
 if not dbppaths:
 	print "No services to set up installed on this system.  Refusing to continue."
@@ -249,28 +267,29 @@ if not dbppaths:
 #run sanity checks
 # checkutc()
 checkdbserverhostname(host)
-checkhostname()
+if sys.platform != "win32": checkhostname()
 try: checkselinux()
 except OSError,e:
 	if e.errno == 2: pass
 	else: raise
-checknetwork()
+if sys.platform != 'win32': checknetwork()
 
 
 #initialize variables
-ipaddr = firstip(ifaces())
+if sys.platform != 'win32': ipaddr = firstip(ifaces())
+else: ipaddr = None
 if not ipaddr: ipaddr='127.0.0.1'
 
 
 if rootuser:
 	print "Testing specified deployment credentials on server %s:%s"%(host,port)
-	try: run_mysql("SELECT * from mysql.user limit 0",rootuser,rootpassword,host,port)
+        try: run_mysql("SELECT * from mysql.user limit 0",rootuser,rootpassword,host,port,debug=options.debug)
 	except CalledProcessError:
 		print "The deployment credentials you specified are not valid.  Refusing to continue."
 		sys.exit(19)
 else:
 	print "Testing specified connection credentials on server %s:%s"%(host,port)
-	try: run_mysql("SELECT * from cloud.user limit 0",user,password,host,port)
+        try: run_mysql("SELECT * from cloud.user limit 0",user,password,host,port,debug=options.debug)
 	except CalledProcessError:
 		print "The connection credentials you specified are not valid.  Refusing to continue."
 		sys.exit(19)
@@ -287,7 +306,9 @@ if rootuser:
 
 	replacements = (
 		("CREATE USER cloud identified by 'cloud';",
-			""),	
+			"CREATE USER %s@`localhost` identified by '%s'; CREATE USER %s@`%%` identified by '%s';"%(
+					(user,password,user,password)
+				)),
 		("cloud identified by 'cloud';",
 			"%s identified by '%s';"%(user,password)),
 		("cloud@`localhost` identified by 'cloud'",
@@ -315,22 +336,27 @@ if rootuser:
 		if not os.path.exists(p): continue
 		text = file(p).read()
 		for t,r in replacements: text = text.replace(t,r)
-		print "Applying file %s to the database on server %s:%s"%(p,host,port)
-		try: run_mysql(text,rootuser,rootpassword,host,port)
+                print "Applying file %s to the database on server %s:%s"%(p,host,port)
+		try: run_mysql(text,rootuser,rootpassword,host,port,debug=options.debug)
 		except CalledProcessError: sys.exit(20)
 		
 	if options.serversetup:
-		systemjars = "@SYSTEMJARS@".split()
-		pipe = subprocess.Popen(["build-classpath"]+systemjars,stdout=subprocess.PIPE)
-		systemcp,throwaway = pipe.communicate()
-		systemcp = systemcp.strip()
-		if pipe.wait(): # this means that build-classpath failed miserably
-			systemcp = "@SYSTEMCLASSPATH@"
-		pcp = os.path.pathsep.join( glob.glob( os.path.join ( "@PREMIUMJAVADIR@" , "*" ) ) )
-		mscp = "@MSCLASSPATH@"
-		depscp = "@DEPSCLASSPATH@"
 		conf = os.path.dirname(dbppaths[0])
-		classpath = os.path.pathsep.join([pcp,systemcp,depscp,mscp,conf])
+		pcp = os.path.pathsep.join( glob.glob( os.path.join ( r"@PREMIUMJAVADIR@" , "*" ) ) )
+		if sys.platform == 'win32':
+			mscp = r"@MSCLASSPATH@"
+			depscp = r"@DEPSCLASSPATH@"
+			classpath = os.path.pathsep.join([pcp,depscp,mscp,conf])
+		else:
+			systemjars = r"@SYSTEMJARS@".split()
+			pipe = subprocess.Popen(["build-classpath"]+systemjars,stdout=subprocess.PIPE)
+			systemcp,throwaway = pipe.communicate()
+			systemcp = systemcp.strip()
+			if pipe.wait(): # this means that build-classpath failed miserably
+				systemcp = r"@SYSTEMCLASSPATH@"
+			mscp = r"@MSCLASSPATH@"
+			depscp = r"@DEPSCLASSPATH@"
+			classpath = os.path.pathsep.join([pcp,systemcp,depscp,mscp,conf])
 		print "Performing unattended automated setup using file %s"%options.serversetup
 		cmd = ["java","-cp",classpath,"com.cloud.test.DatabaseConfig",options.serversetup]
 		if options.debug: print "Running command: %s"%" ".join(cmd)
@@ -343,12 +369,19 @@ if rootuser:
 			p = os.path.join(dbfilepath,"%s.sql"%f)
 			text = file(p).read()
 			print "Applying file %s to the database on server %s:%s"%(p,host,port)
-			try: run_mysql(text,rootuser,rootpassword,host,port)
+                        try: run_mysql(text,rootuser,rootpassword,host,port,debug=options.debug)
 			except CalledProcessError: sys.exit(22)
 
 	for f in ["templates.%s"%virttech,"create-index-fk"]:
 		p = os.path.join(dbfilepath,"%s.sql"%f)
 		text = file(p).read()
 		print "Applying file %s to the database on server %s:%s"%(p,host,port)
-		try: run_mysql(text,rootuser,rootpassword,host,port)
+                try: run_mysql(text,rootuser,rootpassword,host,port,debug=options.debug)
+		except CalledProcessError: sys.exit(22)
+
+	p = os.path.join(dbfilepath,"schema-level.sql")
+	if os.path.isfile(p):
+		text = file(p).read()
+		print "Applying file %s to the database on server %s:%s"%(p,host,port)
+                try: run_mysql(text,rootuser,rootpassword,host,port,debug=options.debug)
 		except CalledProcessError: sys.exit(22)
diff --git a/setup/db/create-database.sql b/setup/db/create-database.sql
index 3a2ad436170..704568edc60 100644
--- a/setup/db/create-database.sql
+++ b/setup/db/create-database.sql
@@ -24,7 +24,7 @@ BEGIN
   IF foo > 0 THEN 
          DROP USER 'cloud'@'%' ;
   END IF;
-END ;$$
+END $$
 DELIMITER ;
 
 CALL `mysql`.`cloud_drop_user_if_exists`() ;
diff --git a/setup/db/create-index-fk.sql b/setup/db/create-index-fk.sql
old mode 100644
new mode 100755
index 16bcb289747..c1b3d6ed727
--- a/setup/db/create-index-fk.sql
+++ b/setup/db/create-index-fk.sql
@@ -26,10 +26,11 @@ ALTER TABLE `cloud`.`op_ha_work` ADD INDEX `i_op_ha_work__step`(`step`);
 ALTER TABLE `cloud`.`op_ha_work` ADD INDEX `i_op_ha_work__type`(`type`);
 ALTER TABLE `cloud`.`op_ha_work` ADD INDEX `i_op_ha_work__mgmt_server_id`(`mgmt_server_id`);
 
-ALTER TABLE `cloud`.`network_profiles` ADD CONSTRAINT `fk_network_profiles__account_id` FOREIGN KEY `fk_network_profiles__account_id`(`account_id`) REFERENCES `account`(`id`) ON DELETE CASCADE;
+ALTER TABLE `cloud`.`account_network_ref` ADD CONSTRAINT `fk_account_network_ref__account_id` FOREIGN KEY `fk_account_network_ref__account_id`(`account_id`) REFERENCES `account`(`id`) ON DELETE CASCADE;
+ALTER TABLE `cloud`.`account_network_ref` ADD CONSTRAINT `fk_account_network_ref__network_configuration_id` FOREIGN KEY `fk_account_network_ref__network_configuration_id`(`network_configuration_id`) REFERENCES `network_configurations`(`id`) ON DELETE CASCADE;
 
 ALTER TABLE `cloud`.`nics` ADD CONSTRAINT `fk_nics__instance_id` FOREIGN KEY `fk_nics__instance_id`(`instance_id`) REFERENCES `vm_instance`(`id`) ON DELETE CASCADE;
-ALTER TABLE `cloud`.`nics` ADD CONSTRAINT `fk_nics__network_profile_id` FOREIGN KEY `fk_nics__network_profile_id`(`network_profile_id`) REFERENCES `network_profiles`(`id`);
+ALTER TABLE `cloud`.`nics` ADD CONSTRAINT `fk_nics__network_configuration_id` FOREIGN KEY `fk_nics__network_configuration_id`(`network_configuration_id`) REFERENCES `network_configurations`(`id`);
 
 ALTER TABLE `cloud`.`op_dc_ip_address_alloc` ADD INDEX `i_op_dc_ip_address_alloc__pod_id__data_center_id__taken` (`pod_id`, `data_center_id`, `taken`, `instance_id`);
 ALTER TABLE `cloud`.`op_dc_ip_address_alloc` ADD UNIQUE `i_op_dc_ip_address_alloc__ip_address__data_center_id`(`ip_address`, `data_center_id`);
@@ -63,7 +64,6 @@ ALTER TABLE `cloud`.`storage_pool_details` ADD INDEX `i_storage_pool_details__na
 
 ALTER TABLE `cloud`.`user` ADD INDEX `i_user__secret_key_removed`(`secret_key`, `removed`);
 ALTER TABLE `cloud`.`user` ADD INDEX `i_user__removed`(`removed`);
-ALTER TABLE `cloud`.`user` ADD UNIQUE `i_user__username__removed`(`username`, `removed`);
 ALTER TABLE `cloud`.`user` ADD UNIQUE `i_user__api_key`(`api_key`);
 ALTER TABLE `cloud`.`user` ADD CONSTRAINT `fk_user__account_id` FOREIGN KEY `fk_user__account_id` (`account_id`) REFERENCES `account` (`id`) ON DELETE CASCADE;
 ALTER TABLE `cloud`.`user` ADD INDEX `i_user__account_id`(`account_id`);
@@ -119,16 +119,16 @@ ALTER TABLE `cloud`.`vm_instance` ADD INDEX `i_vm_instance__last_host_id`(`last_
 
 ALTER TABLE `cloud`.`vm_instance` ADD CONSTRAINT `fk_vm_instance__template_id` FOREIGN KEY `fk_vm_instance__template_id` (`vm_template_id`) REFERENCES `vm_template` (`id`);
 ALTER TABLE `cloud`.`vm_instance` ADD INDEX `i_vm_instance__template_id`(`vm_template_id`);
+ALTER TABLE `cloud`.`vm_instance` ADD CONSTRAINT `fk_vm_instance__account_id` FOREIGN KEY `fk_vm_instance__account_id` (`account_id`) REFERENCES `account` (`id`);
+ALTER TABLE `cloud`.`vm_instance` ADD INDEX `i_vm_instance__account_id`(`account_id`);
+ALTER TABLE `cloud`.`vm_instance` ADD CONSTRAINT `fk_vm_instance__service_offering_id` FOREIGN KEY `fk_vm_instance__service_offering_id` (`service_offering_id`) REFERENCES `service_offering` (`id`);
+ALTER TABLE `cloud`.`vm_instance` ADD INDEX `i_vm_instance__service_offering_id`(`service_offering_id`);
 
 ALTER TABLE `cloud`.`service_offering` ADD CONSTRAINT `fk_service_offering__id` FOREIGN KEY `fk_service_offering__id`(`id`) REFERENCES `disk_offering`(`id`) ON DELETE CASCADE;
 
 ALTER TABLE `cloud`.`user_vm` ADD CONSTRAINT `fk_user_vm__domain_router_id` FOREIGN KEY `fk_user_vm__domain_router_id` (`domain_router_id`) REFERENCES `domain_router` (`id`);
 ALTER TABLE `cloud`.`user_vm` ADD INDEX `i_user_vm__domain_router_id`(`domain_router_id`);
-ALTER TABLE `cloud`.`user_vm` ADD CONSTRAINT `fk_user_vm__service_offering_id` FOREIGN KEY `fk_user_vm__service_offering_id` (`service_offering_id`) REFERENCES `service_offering` (`id`);
-ALTER TABLE `cloud`.`user_vm` ADD INDEX `i_user_vm__service_offering_id`(`service_offering_id`);
 ALTER TABLE `cloud`.`user_vm` ADD CONSTRAINT `fk_user_vm__id` FOREIGN KEY `fk_user_vm__id` (`id`) REFERENCES `vm_instance`(`id`) ON DELETE CASCADE;
-ALTER TABLE `cloud`.`user_vm` ADD CONSTRAINT `fk_user_vm__account_id` FOREIGN KEY `fk_user_vm__account_id` (`account_id`) REFERENCES `account` (`id`);
-ALTER TABLE `cloud`.`user_vm` ADD INDEX `i_user_vm__account_id`(`account_id`);
 ALTER TABLE `cloud`.`user_vm` ADD CONSTRAINT `fk_user_vm__external_ip_address` FOREIGN KEY `fk_user_vm__external_ip_address` (`external_ip_address`) REFERENCES `user_ip_address` (`public_ip_address`);
 ALTER TABLE `cloud`.`user_vm` ADD INDEX `i_user_vm__external_ip_address`(`external_ip_address`);
 ALTER TABLE `cloud`.`user_vm` ADD CONSTRAINT `fk_user_vm__external_vlan_db_id` FOREIGN KEY `fk_user_vm__external_vlan_db_id` (`external_vlan_db_id`) REFERENCES `vlan` (`id`);
@@ -138,9 +138,6 @@ ALTER TABLE `cloud`.`domain_router` ADD CONSTRAINT `fk_domain_router__public_ip_
 ALTER TABLE `cloud`.`domain_router` ADD INDEX `i_domain_router__public_ip_address`(`public_ip_address`);
 ALTER TABLE `cloud`.`domain_router` ADD CONSTRAINT `fk_domain_router__id` FOREIGN KEY `fk_domain_router__id` (`id`) REFERENCES `vm_instance`(`id`) ON DELETE CASCADE;
 
-ALTER TABLE `cloud`.`domain_router` ADD CONSTRAINT `fk_domain_router__account_id` FOREIGN KEY `fk_domain_router__account_id` (`account_id`) REFERENCES `account` (`id`);
-ALTER TABLE `cloud`.`domain_router` ADD INDEX `i_domain_router__account_id`(`account_id`);
-
 ALTER TABLE `cloud`.`domain_router` ADD CONSTRAINT `fk_domain_router__vlan_id` FOREIGN KEY `fk_domain_router__vlan_id` (`vlan_db_id`) REFERENCES `vlan` (`id`);
 ALTER TABLE `cloud`.`domain_router` ADD INDEX `i_domain_router__vlan_id`(`vlan_db_id`);
 
@@ -159,6 +156,10 @@ ALTER TABLE `cloud`.`template_host_ref` ADD INDEX `i_template_host_ref__host_id`
 ALTER TABLE `cloud`.`template_host_ref` ADD CONSTRAINT `fk_template_host_ref__template_id` FOREIGN KEY `fk_template_host_ref__template_id` (`template_id`) REFERENCES `vm_template` (`id`) ON DELETE CASCADE;
 ALTER TABLE `cloud`.`template_host_ref` ADD INDEX `i_template_host_ref__template_id`(`template_id`);
 
+ALTER TABLE `cloud`.`upload` ADD CONSTRAINT `fk_upload__host_id` FOREIGN KEY `fk_upload__host_id` (`host_id`) REFERENCES `host` (`id`) ON DELETE CASCADE;
+ALTER TABLE `cloud`.`upload` ADD INDEX `i_upload__host_id`(`host_id`);
+ALTER TABLE `cloud`.`upload` ADD INDEX `i_upload__type_id`(`type_id`);
+
 ALTER TABLE `cloud`.`template_zone_ref` ADD CONSTRAINT `fk_template_zone_ref__zone_id` FOREIGN KEY `fk_template_zone_ref__zone_id` (`zone_id`) REFERENCES `data_center` (`id`) ON DELETE CASCADE;
 ALTER TABLE `cloud`.`template_zone_ref` ADD INDEX `i_template_zone_ref__zone_id`(`zone_id`);
 ALTER TABLE `cloud`.`template_zone_ref` ADD CONSTRAINT `fk_template_zone_ref__template_id` FOREIGN KEY `fk_template_zone_ref__template_id` (`template_id`) REFERENCES `vm_template` (`id`) ON DELETE CASCADE;
@@ -250,3 +251,11 @@ ALTER TABLE `cloud`.`network_group_vm_map` ADD CONSTRAINT `fk_network_group_vm_m
 
 ALTER TABLE `cloud`.`op_nwgrp_work` ADD INDEX `i_op_nwgrp_work__instance_id`(`instance_id`);
 ALTER TABLE `cloud`.`op_nwgrp_work` ADD INDEX `i_op_nwgrp_work__mgmt_server_id`(`mgmt_server_id`);
+
+ALTER TABLE `cloud`.`host_master` ADD UNIQUE `i_host_master__service_address`(`service_address`);
+
+ALTER TABLE `cloud`.`instance_group` ADD CONSTRAINT `fk_instance_group__account_id` FOREIGN KEY `fk_instance_group__account_id` (`account_id`) REFERENCES `account` (`id`);
+
+ALTER TABLE `cloud`.`instance_group_vm_map` ADD CONSTRAINT `fk_instance_group_vm_map___group_id` FOREIGN KEY `fk_instance_group_vm_map___group_id` (`group_id`) REFERENCES `instance_group` (`id`) ON DELETE CASCADE;
+ALTER TABLE `cloud`.`instance_group_vm_map` ADD CONSTRAINT `fk_instance_group_vm_map___instance_id` FOREIGN KEY `fk_instance_group_vm_map___instance_id` (`instance_id`) REFERENCES `user_vm` (`id`) ON DELETE CASCADE;
+
diff --git a/setup/db/create-schema.sql b/setup/db/create-schema.sql
old mode 100644
new mode 100755
index d58e157bffe..e98fd6768f8
--- a/setup/db/create-schema.sql
+++ b/setup/db/create-schema.sql
@@ -24,6 +24,7 @@ DROP TABLE IF EXISTS `cloud`.`pricing`;
 DROP TABLE IF EXISTS `cloud`.`sequence`;
 DROP TABLE IF EXISTS `cloud`.`user_vm`;
 DROP TABLE IF EXISTS `cloud`.`template_host_ref`;
+DROP TABLE IF EXISTS `cloud`.`upload`;
 DROP TABLE IF EXISTS `cloud`.`template_zone_ref`;
 DROP TABLE IF EXISTS `cloud`.`ha_work`;
 DROP TABLE IF EXISTS `cloud`.`dc_vnet_alloc`;
@@ -74,40 +75,78 @@ DROP TABLE IF EXISTS `cloud`.`storage_pool_details`;
 DROP TABLE IF EXISTS `cloud`.`ext_lun_details`;
 DROP TABLE IF EXISTS `cloud`.`cluster`;
 DROP TABLE IF EXISTS `cloud`.`nics`;
-DROP TABLE IF EXISTS `cloud`.`network_profiles`;
+DROP TABLE IF EXISTS `cloud`.`network_configurations`;
 DROP TABLE IF EXISTS `cloud`.`network_offerings`;
+DROP TABLE IF EXISTS `cloud`.`host_master`;
+DROP TABLE IF EXISTS `cloud`.`hypervisor_properties`;
+DROP TABLE IF EXISTS `cloud`.`account_network_ref`;
+DROP TABLE IF EXISTS `cloud`.`instance_group`;
+DROP TABLE IF EXISTS `cloud`.`instance_group_vm_map`;
 
-CREATE TABLE `cloud`.`network_profiles` (
+CREATE TABLE `cloud`.`hypervsior_properties` (
+  `hypervisor` varchar(32) NOT NULL UNIQUE COMMENT 'hypervisor type',
+  `max_storage_devices` int(10) NOT NULL COMMENT 'maximum number of storage devices',
+  `cdrom_device` int(10) NOT NULL COMMENT 'device id reserved for cdrom',
+  `max_network_devices` int(10) NOT NULL COMMENT 'maximum number of network devices'
+) ENGINE=InnoDB DEFAULT CHARSET=utf8;
+
+CREATE TABLE `cloud`.`network_configurations` (
   `id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
   `name` varchar(255) COMMENT 'name for this network',
-  `account_id` bigint unsigned NOT NULL COMMENT 'Owner of this network configuration',
   `traffic_type` varchar(32) NOT NULL COMMENT 'type of traffic going through this network',
   `broadcast_domain_type` varchar(32) NOT NULL COMMENT 'type of broadcast domain used',
-  `gateway` varchar(15) NOT NULL COMMENT 'gateway for this network profile',
-  `cidr` varchar(32) NOT NULL COMMENT 'network cidr',
-  `mode` varchar(32) NOT NULL COMMENT 'How to retrieve ip address in this network',
+  `broadcast_uri` varchar(255) COMMENT 'broadcast domain specifier',
+  `gateway` varchar(15) COMMENT 'gateway for this network configuration',
+  `cidr` varchar(18) COMMENT 'network cidr', 
+  `mode` varchar(32) COMMENT 'How to retrieve ip address in this network',
+  `network_offering_id` bigint unsigned NOT NULL COMMENT 'network offering id that this configuration is created from',
+  `data_center_id` bigint unsigned NOT NULL COMMENT 'data center id that this configuration is used in',
+  `guru_name` varchar(255) NOT NULL COMMENT 'who is responsible for this type of network configuration',
+  `state` varchar(32) NOT NULL COMMENT 'what state is this configuration in', 
+  PRIMARY KEY (`id`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8;
+
+CREATE TABLE `cloud`.`account_network_ref` (
+  `id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
+  `account_id` bigint unsigned NOT NULL COMMENT 'account id',
+  `network_configuration_id` bigint unsigned NOT NULL COMMENT 'network configuration id',
   PRIMARY KEY (`id`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
 
 CREATE TABLE `cloud`.`nics` (
   `id` bigint unsigned NOT NULL UNIQUE AUTO_INCREMENT COMMENT 'id',
   `instance_id` bigint unsigned NOT NULL COMMENT 'vm instance id',
-  `ip4_address` varchar(15) COMMENT 'ip4 address',
   `mac_address` varchar(17) COMMENT 'mac address',
-  `network_profile_id` bigint unsigned NOT NULL COMMENT 'network id',  
-  `vlan` varchar(64) COMMENT 'Virtualized network identifier',
+  `ip4_address` varchar(15) COMMENT 'ip4 address',
+  `netmask` varchar(15) COMMENT 'netmask for ip4 address',
+  `gateway` varchar(15) COMMENT 'gateway',
+  `ip_type` varchar(32) COMMENT 'type of ip',
+  `broadcast_uri` varchar(255) COMMENT 'broadcast uri',
+  `network_configuration_id` bigint unsigned NOT NULL COMMENT 'network configuration id',
+  `mode` varchar(32) COMMENT 'mode of getting ip address',  
   `state` varchar(32) NOT NULL COMMENT 'state of the creation',
+  `reserver_name` varchar(255) COMMENT 'Name of the component that reserved the ip address',
+  `reservation_id` varchar(64) COMMENT 'id for the reservation',
+  `device_id` int(10) COMMENT 'device id for the network when plugged into the virtual machine',
+  `update_time` timestamp NOT NULL COMMENT 'time the state was changed',
+  `isolation_uri` varchar(255) COMMENT 'id for isolation',
+  `ip6_address` varchar(32) COMMENT 'ip6 address', 
   PRIMARY KEY (`id`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
 
 CREATE TABLE `cloud`.`network_offerings` (
   `id` bigint unsigned NOT NULL UNIQUE AUTO_INCREMENT COMMENT 'id',
-  `name` varchar(64) NOT NULL COMMENT 'network offering',
-  `type` varchar(32) NOT NULL COMMENT 'type of network',
+  `name` varchar(64) NOT NULL unique COMMENT 'network offering',
+  `type` varchar(32) COMMENT 'type of network',
   `display_text` varchar(255) NOT NULL COMMENT 'text to display to users',
-  `nw_rate` smallint unsigned default 200 COMMENT 'network rate throttle mbits/s',
-  `mc_rate` smallint unsigned default 10 COMMENT 'mcast rate throttle mbits/s',
-  `concurrent_connections` smallint unsigned COMMENT 'concurrent connections supported on this network',
+  `nw_rate` smallint unsigned COMMENT 'network rate throttle mbits/s',
+  `mc_rate` smallint unsigned COMMENT 'mcast rate throttle mbits/s',
+  `concurrent_connections` int(10) unsigned COMMENT 'concurrent connections supported on this network',
+  `traffic_type` varchar(32) NOT NULL COMMENT 'traffic type carried on this network',
+  `tags` varchar(4096) COMMENT 'tags supported by this offering',
+  `system_only` int(1) unsigned NOT NULL DEFAULT 0 COMMENT 'Is this network offering for system use only',
+  `created` datetime NOT NULL COMMENT 'time the entry was created',
+  `removed` datetime DEFAULT NULL COMMENT 'time the entry was removed',
   PRIMARY KEY (`id`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
 
@@ -193,6 +232,7 @@ INSERT INTO `cloud`.`sequence` (name, value) VALUES ('vm_template_seq', 200);
 INSERT INTO `cloud`.`sequence` (name, value) VALUES ('public_mac_address_seq', 1);
 INSERT INTO `cloud`.`sequence` (name, value) VALUES ('private_mac_address_seq', 1);
 INSERT INTO `cloud`.`sequence` (name, value) VALUES ('storage_pool_seq', 200);
+INSERT INTO `cloud`.`sequence` (name, value) VALUES ('volume_seq', 1);
 
 CREATE TABLE  `cloud`.`disk_template_ref` (
   `id` bigint unsigned NOT NULL auto_increment,
@@ -222,7 +262,7 @@ CREATE TABLE `cloud`.`volumes` (
   `data_center_id` bigint unsigned NOT NULL COMMENT 'data center this volume belongs to',
   `iscsi_name` varchar(255) COMMENT 'iscsi target name',
   `host_ip` varchar(15)  COMMENT 'host ip address for convenience',
-  `volume_type` varchar(64) COMMENT 'root, swap or data',
+  `volume_type` varchar(64) NOT NULL COMMENT 'root, swap or data',
   `resource_type` varchar(64) COMMENT 'pool-based or host-based',
   `pool_type` varchar(64) COMMENT 'type of the pool',
   `mirror_state` varchar(64) COMMENT 'not_mirrored, active or defunct',
@@ -233,9 +273,13 @@ CREATE TABLE `cloud`.`volumes` (
   `recreatable` tinyint(1) unsigned NOT NULL DEFAULT 0 COMMENT 'Is this volume recreatable?',
   `destroyed` tinyint(1) COMMENT 'indicates whether the volume was destroyed by the user or not',
   `created` datetime COMMENT 'Date Created',
+  `attached` datetime COMMENT 'Date Attached',
   `updated` datetime COMMENT 'Date updated for attach/detach',
   `removed` datetime COMMENT 'Date removed.  not null if removed',
   `status` varchar(32) COMMENT 'Async API volume creation status',
+  `state` varchar(32) COMMENT 'State machine',
+  `source_id` bigint unsigned  COMMENT 'id for the source',
+  `source_type` varchar(32) COMMENT 'source from which the volume is created -- snapshot, diskoffering, template, blank',
   PRIMARY KEY (`id`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
 
@@ -501,26 +545,6 @@ CREATE TABLE  `cloud`.`vm_template` (
   PRIMARY KEY  (`id`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
 
-CREATE TABLE  `cloud`.`vm_instance2` (
-  `id` bigint unsigned UNIQUE NOT NULL,
-  `instance_name` varchar(255) NOT NULL COMMENT 'name of the vm instance running on the hosts',
-  `state` varchar(32) NOT NULL,
-  `update_count` bigint unsigned NOT NULL DEFAULT 0 COMMENT 'date state was updated',
-  `vm_template_id` bigint unsigned,
-  `guest_os_id` bigint unsigned NOT NULL,
-  `host_id` bigint unsigned,
-  `last_host_id` bigint unsigned COMMENT 'tentative host for first run or last host that it has been running on',
-  `proxy_id` bigint unsigned NULL COMMENT 'console proxy allocated in previous session',
-  `proxy_assign_time` DATETIME NULL COMMENT 'time when console proxy was assigned',
-  `vnc_password` varchar(255) NOT NULL COMMENT 'vnc password',
-  `ha_enabled` tinyint(1) NOT NULL DEFAULT 0 COMMENT 'Should HA be enabled for this VM',
-  `update_time` datetime COMMENT 'date the destroy was requested',
-  `created` datetime NOT NULL COMMENT 'date created',
-  `removed` datetime COMMENT 'date removed if not null',
-  `type` varchar(32) NOT NULL COMMENT 'type of vm it is',
-  PRIMARY KEY  (`id`)
-) ENGINE=InnoDB DEFAULT CHARSET=utf8;
-
 CREATE TABLE  `cloud`.`vm_instance` (
   `id` bigint unsigned UNIQUE NOT NULL,
   `name` varchar(255) NOT NULL,
@@ -545,19 +569,19 @@ CREATE TABLE  `cloud`.`vm_instance` (
   `created` datetime NOT NULL COMMENT 'date created',
   `removed` datetime COMMENT 'date removed if not null',
   `type` varchar(32) NOT NULL COMMENT 'type of vm it is',
+  `account_id` bigint unsigned NOT NULL COMMENT 'user id of owner',
+  `domain_id` bigint unsigned NOT NULL,
+  `service_offering_id` bigint unsigned NOT NULL COMMENT 'service offering id',
   PRIMARY KEY  (`id`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
 
 CREATE TABLE `cloud`.`user_vm` (
   `id` bigint unsigned UNIQUE NOT NULL,
-  `group` varchar(255),
   `iso_id` bigint unsigned,
   `display_name` varchar(255),
   `domain_router_id` bigint unsigned COMMENT 'router id',
   `vnet` varchar(18) COMMENT 'vnet',
   `dc_vlan` varchar(18) COMMENT 'zone vlan',
-  `account_id` bigint unsigned NOT NULL COMMENT 'user id of owner',
-  `domain_id` bigint unsigned NOT NULL,
   `guest_ip_address` varchar(15) COMMENT 'ip address within the guest network',
   `guest_mac_address` varchar(17) COMMENT 'mac address within the guest network',
   `guest_netmask` varchar(15) COMMENT 'netmask within the guest network',
@@ -565,7 +589,6 @@ CREATE TABLE `cloud`.`user_vm` (
   `external_mac_address` varchar(17)  COMMENT 'mac address within the external network',
   `external_vlan_db_id` bigint unsigned  COMMENT 'foreign key into vlan table',
   `user_data` varchar(2048),
-  `service_offering_id` bigint unsigned NOT NULL COMMENT 'service offering id',
   PRIMARY KEY (`id`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
 
@@ -587,13 +610,26 @@ CREATE TABLE `cloud`.`domain_router` (
   `dc_vlan` varchar(18) COMMENT 'vnet',
   `vlan_db_id` bigint unsigned COMMENT 'Foreign key into vlan id table',
   `vlan_id` varchar(255) COMMENT 'optional VLAN ID for DomainRouter that can be used in rundomr.sh',
-  `account_id` bigint unsigned NOT NULL COMMENT 'account id of owner',
-  `domain_id` bigint unsigned NOT NULL,
   `dhcp_ip_address` bigint unsigned NOT NULL DEFAULT 2 COMMENT 'next ip address for dhcp for this domR',
   `role` varchar(64) NOT NULL COMMENT 'type of role played by this router',
   PRIMARY KEY (`id`)
 ) ENGINE = InnoDB DEFAULT CHARSET=utf8 COMMENT = 'information about the domR instance';
 
+CREATE TABLE  `cloud`.`upload` (
+  `id` bigint unsigned NOT NULL auto_increment,
+  `host_id` bigint unsigned NOT NULL,
+  `type_id` bigint unsigned NOT NULL,
+  `type` varchar(255),
+  `created` DATETIME NOT NULL,
+  `last_updated` DATETIME,
+  `job_id` varchar(255),
+  `upload_pct` int(10) unsigned,
+  `upload_state` varchar(255),
+  `error_str` varchar(255),
+  `url` varchar(255),
+  PRIMARY KEY  (`id`)
+) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
+
 CREATE TABLE  `cloud`.`template_host_ref` (
   `id` bigint unsigned NOT NULL auto_increment,
   `host_id` bigint unsigned NOT NULL,
@@ -630,10 +666,10 @@ CREATE TABLE  `cloud`.`console_proxy` (
   `dns1` varchar(15) COMMENT 'dns1',
   `dns2` varchar(15) COMMENT 'dns2',
   `domain` varchar(255) COMMENT 'domain',
-  `public_mac_address` varchar(17) NOT NULL unique COMMENT 'mac address of the public facing network card',
+  `public_mac_address` varchar(17) unique COMMENT 'mac address of the public facing network card',
   `public_ip_address` varchar(15) UNIQUE COMMENT 'public ip address for the console proxy',
   `public_netmask` varchar(15)  COMMENT 'public netmask used for the console proxy',
-  `guest_mac_address` varchar(17) NOT NULL unique COMMENT 'mac address of the guest facing network card',
+  `guest_mac_address` varchar(17) unique COMMENT 'mac address of the guest facing network card',
   `guest_ip_address`  varchar(15) UNIQUE COMMENT 'guest ip address for the console proxy',
   `guest_netmask` varchar(15)  COMMENT 'guest netmask used for the console proxy',
   `vlan_db_id` bigint unsigned COMMENT 'Foreign key into vlan id table',
@@ -834,8 +870,8 @@ CREATE TABLE `cloud`.`security_group` (
   `id` bigint unsigned NOT NULL auto_increment,
   `name` varchar(255) NOT NULL,
   `description` varchar(4096) NULL,
-  `domain_id` bigint unsigned NULL,
-  `account_id` bigint unsigned NULL,
+  `domain_id` bigint unsigned NOT NULL,
+  `account_id` bigint unsigned NOT NULL,
   PRIMARY KEY  (`id`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
 
@@ -880,7 +916,7 @@ CREATE TABLE `cloud`.`load_balancer` (
 CREATE TABLE  `cloud`.`storage_pool` (
   `id` bigint unsigned UNIQUE NOT NULL,
   `name` varchar(255) COMMENT 'should be NOT NULL',
-  `uuid` varchar(255) UNIQUE NOT NULL,
+  `uuid` varchar(255) NOT NULL,
   `pool_type` varchar(32) NOT NULL,
   `port` int unsigned NOT NULL,
   `data_center_id` bigint unsigned NOT NULL,
@@ -1042,4 +1078,29 @@ CREATE TABLE `cloud`.`op_vm_ruleset_log` (
   PRIMARY KEY (`id`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
 
+CREATE TABLE `cloud`.`host_master` (
+  `id` bigint unsigned UNIQUE NOT NULL AUTO_INCREMENT COMMENT 'id',
+  `type` varchar(32) NOT NULL,
+  `service_address` varchar(255) NOT NULL,
+  `admin` varchar(32) NOT NULL,
+  `password` varchar(32),
+  PRIMARY KEY (`id`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8;
+
+CREATE TABLE `cloud`.`instance_group` (
+  `id` bigint unsigned NOT NULL UNIQUE auto_increment,
+  `account_id` bigint unsigned NOT NULL COMMENT 'owner.  foreign key to account table',
+  `name` varchar(255) NOT NULL,
+  `removed` datetime COMMENT 'date the group was removed',
+  `created` datetime COMMENT 'date the group was created',
+  PRIMARY KEY  (`id`)
+) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
+
+CREATE TABLE `cloud`.`instance_group_vm_map` (
+  `id` bigint unsigned NOT NULL auto_increment,
+  `group_id` bigint unsigned NOT NULL,
+  `instance_id` bigint unsigned NOT NULL,
+  PRIMARY KEY  (`id`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8;
+
 SET foreign_key_checks = 1;
diff --git a/setup/db/data-21to22.sql b/setup/db/data-21to22.sql
new file mode 100644
index 00000000000..5d83a8c3637
--- /dev/null
+++ b/setup/db/data-21to22.sql
@@ -0,0 +1,14 @@
+--data upgrade from 21 to 22
+use cloud;
+
+START TRANSACTION;
+
+DELETE FROM configuration where name='upgrade.url';
+DELETE FROM configuration where name='router.template.id';
+UPDATE vm_template set unique_name='routing_old'  where id=1;
+INSERT INTO vm_template (id, unique_name, name, public, created, type, hvm, bits, account_id, url, checksum, enable_password, display_text, format, guest_os_id, featured, cross_zones)
+    VALUES (10, 'routing', 'SystemVM Template', 0, now(), 'ext3', 0, 64, 1, 'http://download.cloud.com/releases/2.2/systemvm.vhd.bz2', 'bcc7f290f4c27ab4d0fe95d1012829ea', 0, 'SystemVM Template', 'VHD', 15, 0, 1);
+
+ALTER TABLE `vm_instance` DROP COLUMN `group`;
+
+COMMIT;
diff --git a/setup/db/deploy-db-dev.sh b/setup/db/deploy-db-dev.sh
index 3e20af62042..fb1b53b6de5 100755
--- a/setup/db/deploy-db-dev.sh
+++ b/setup/db/deploy-db-dev.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 # deploy-db.sh -- deploys the database configuration.
-
+#
 # set -x
 
 if [ "$1" == "" ]; then
diff --git a/setup/db/index-212to213.sql b/setup/db/index-212to213.sql
new file mode 100644
index 00000000000..c3adf7b3d14
--- /dev/null
+++ b/setup/db/index-212to213.sql
@@ -0,0 +1,2 @@
+ALTER TABLE `cloud`.`user` DROP KEY `i_user__username__removed`;
+
diff --git a/setup/db/index-21to22.sql b/setup/db/index-21to22.sql
new file mode 100644
index 00000000000..ebc4b8477d4
--- /dev/null
+++ b/setup/db/index-21to22.sql
@@ -0,0 +1,4 @@
+ALTER TABLE `cloud`.`instance_group` ADD CONSTRAINT `fk_instance_group__account_id` FOREIGN KEY `fk_instance_group__account_id` (`account_id`) REFERENCES `account` (`id`);
+
+ALTER TABLE `cloud`.`instance_group_vm_map` ADD CONSTRAINT `fk_instance_group_vm_map___group_id` FOREIGN KEY `fk_instance_group_vm_map___group_id` (`group_id`) REFERENCES `instance_group` (`id`) ON DELETE CASCADE;
+ALTER TABLE `cloud`.`instance_group_vm_map` ADD CONSTRAINT `fk_instance_group_vm_map___instance_id` FOREIGN KEY `fk_instance_group_vm_map___instance_id` (`instance_id`) REFERENCES `user_vm` (`id`) ON DELETE CASCADE;
diff --git a/setup/db/migration/schema-21to22.sql b/setup/db/migration/schema-21to22.sql
deleted file mode 100644
index 108ed2a9e20..00000000000
--- a/setup/db/migration/schema-21to22.sql
+++ /dev/null
@@ -1,9 +0,0 @@
---
--- Schema upgrade from 2.1 to 2.2
---
-
-ALTER TABLE `cloud`.`data_center` MODIFY COLUMN `guest_network_cidr` varchar(18); -- modify column width to 18 from 15
-
-ALTER TABLE `cloud`.`resource_count` ADD COLUMN `domain_id` bigint unsigned; -- add the new column
-ALTER TABLE `cloud`.`resource_count` MODIFY COLUMN `account_id` bigint unsigned; -- modify the column to allow NULL values
-ALTER TABLE `cloud`.`storage_pool` add COLUMN STATUS varchar(32) not null; -- new status column for maintenance mode support for primary storage
diff --git a/setup/db/schema-21to22.sql b/setup/db/schema-21to22.sql
new file mode 100644
index 00000000000..9fbd99612fa
--- /dev/null
+++ b/setup/db/schema-21to22.sql
@@ -0,0 +1,33 @@
+--
+-- Schema upgrade from 2.1 to 2.2
+--
+
+
+CREATE TABLE `cloud`.`instance_group` (
+  `id` bigint unsigned NOT NULL UNIQUE auto_increment,
+  `account_id` bigint unsigned NOT NULL COMMENT 'owner.  foreign key to account table',
+  `name` varchar(255) NOT NULL,
+  `removed` datetime COMMENT 'date the group was removed',
+  `created` datetime COMMENT 'date the group was created',
+  PRIMARY KEY  (`id`)
+) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
+
+CREATE TABLE `cloud`.`instance_group_vm_map` (
+  `id` bigint unsigned NOT NULL auto_increment,
+  `group_id` bigint unsigned NOT NULL,
+  `instance_id` bigint unsigned NOT NULL,
+  PRIMARY KEY  (`id`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8;
+
+
+ALTER TABLE `cloud`.`data_center` MODIFY COLUMN `guest_network_cidr` varchar(18); -- modify column width to 18 from 15
+
+ALTER TABLE `cloud`.`resource_count` ADD COLUMN `domain_id` bigint unsigned; -- add the new column
+ALTER TABLE `cloud`.`resource_count` MODIFY COLUMN `account_id` bigint unsigned; -- modify the column to allow NULL values
+ALTER TABLE `cloud`.`storage_pool` add COLUMN STATUS varchar(32) not null; -- new status column for maintenance mode support for primary storage
+ALTER TABLE `cloud`.`volumes` ADD COLUMN `source_id` bigint unsigned;  -- id for the source
+ALTER TABLE `cloud`.`volumes` ADD COLUMN `source_type` varchar(32); --source from which the volume is created i.e. snapshot, diskoffering, template, blank
+ALTER TABLE `cloud`.`volumes` ADD COLUMN 'attached' datetime; --date and time the volume was attached
+
+
+
diff --git a/setup/db/schema-level.sql b/setup/db/schema-level.sql
new file mode 100644
index 00000000000..e3b0eea48fb
--- /dev/null
+++ b/setup/db/schema-level.sql
@@ -0,0 +1 @@
+INSERT INTO `cloud`.`configuration` (category, instance, component, name, value, description) VALUES ('Hidden', 'DEFAULT', 'database', 'schema.level', '2.2', 'The schema level of this database');
diff --git a/setup/db/server-setup.xml b/setup/db/server-setup.xml
old mode 100644
new mode 100755
index 7db947bee0b..9ca63ba9643
--- a/setup/db/server-setup.xml
+++ b/setup/db/server-setup.xml
@@ -12,14 +12,14 @@
   
     
       1
-      ZONE1
-      72.52.126.11
-      72.52.126.12
-      192.168.10.253
-      192.168.10.254
-      172.16.0.1
-      255.255.0.0
-      172.16.1.1-172.16.255.253
+      NM
+      4.2.2.2
+      10.10.10.14
+      4.2.2.2
+      4.2.2.2
+	  10.91.28.1
+	  255.255.255.0
+	  560-579      
       10.1.1.0/24
     
   
@@ -35,24 +35,46 @@
   
     
       1
-      POD1
+      NM
       1 
-      192.168.2.1
-      192.168.2.0/24
-      192.168.2.20-192.168.2.170
+      10.91.28.1
+      10.91.28.0/24
+      10.91.28.160-10.91.28.179
     
   
-
 
+
+
+
   
     
       1
       30
       VirtualNetwork
-      192.168.30.1
+      10.91.30.1
       255.255.255.0
-      192.168.30.10-192.168.30.19
+      10.91.30.160-10.91.30.179
     
+  
+
   
+    /usr/bin/qemu-kvm
+    
+      
+      
+      
+      
+    
+    
+      
+      
+      
+      
+    
+    
+    
+  
+
+
diff --git a/tools/waf/javadir.py b/tools/waf/javadir.py
new file mode 100644
index 00000000000..9c71a64c80c
--- /dev/null
+++ b/tools/waf/javadir.py
@@ -0,0 +1,22 @@
+#!/usr/bin/env python
+
+import Options, Utils
+import os
+
+def detect(conf):
+	conf.check_message_1('Detecting JAVADIR')
+	javadir = getattr(Options.options, 'JAVADIR', '')
+	if javadir:
+		conf.env.JAVADIR = javadir
+		conf.check_message_2("%s (forced through --javadir)"%conf.env.JAVADIR,"GREEN")
+	else:
+		conf.env.JAVADIR = os.path.join(conf.env.DATADIR,'java')
+		conf.check_message_2("%s (using default ${DATADIR}/java directory)"%conf.env.JAVADIR,"GREEN")
+
+def set_options(opt):
+        inst_dir = opt.get_option_group('--datadir') # get the group that contains bindir
+        if not inst_dir: raise Utils.WafError, "DATADIR not set.  Did you load the gnu_dirs tool options with opt.tool_options('gnu_dirs') before running opt.tool_options('javadir')?"
+	inst_dir.add_option('--javadir', # add javadir to the group that contains bindir
+		help = 'Java class and jar files [Default: ${DATADIR}/java]',
+		default = '',
+		dest = 'JAVADIR')
diff --git a/tools/waf/mkisofs.py b/tools/waf/mkisofs.py
new file mode 100644
index 00000000000..440073d2de4
--- /dev/null
+++ b/tools/waf/mkisofs.py
@@ -0,0 +1,68 @@
+import Utils
+from TaskGen import feature, before
+from Configure import ConfigurationError
+import Task
+import os
+
+# fixme: this seems to hang waf with 100% CPU
+
+def detect(conf):
+	conf.find_program("mkisofs",var='MKISOFS')
+	if not conf.env.MKISOFS: conf.find_program("genisoimage",mandatory=True,var='MKISOFS')
+
+def iso_up(task):
+	tgt = task.outputs[0].bldpath(task.env)
+	if os.path.exists(tgt): os.unlink(tgt)
+	inps = []
+	for inp in task.inputs:
+		if inp.id&3==Node.BUILD:
+			src = inp.bldpath(task.env)
+			srcname = src
+			srcname = sep.join(srcname.split(sep)[1:]) # chop off default/
+		else:
+			src = inp.srcpath(task.env)
+			srcname = src
+			srcname = sep.join(srcname.split(sep)[1:]) # chop off ../
+                if task.generator.rename: srcname = task.generator.rename(srcname)
+                inps.append(srcname+'='+src)
+	ret = Utils.exec_command(
+		[
+			task.generator.env.MKISOFS,
+			"-quiet",
+			"-r",
+                        "-graft-points",
+			"-o",tgt,
+		] + inps, shell=False)
+	if ret != 0: return ret
+
+def apply_iso(self):
+	Utils.def_attrs(self,fun=iso_up)
+	self.default_install_path=0
+	lst=self.to_list(self.source)
+	self.meths.remove('apply_core')
+	self.dict=getattr(self,'dict',{})
+	out = self.path.find_or_declare(self.target)
+	ins = []
+	for x in Utils.to_list(self.source):
+		node = self.path.find_resource(x)
+		if not node:raise Utils.WafError('cannot find input file %s for processing'%x)
+		ins.append(node)
+	if self.dict and not self.env['DICT_HASH']:
+		self.env=self.env.copy()
+		keys=list(self.dict.keys())
+		keys.sort()
+		lst=[self.dict[x]for x in keys]
+		self.env['DICT_HASH']=str(Utils.h_list(lst))
+	tsk=self.create_task('iso',ins,out)
+	tsk.fun=self.fun
+	tsk.dict=self.dict
+	tsk.dep_vars=['DICT_HASH']
+	tsk.install_path=self.install_path
+	tsk.chmod=self.chmod
+	if not tsk.env:
+		tsk.debug()
+		raise Utils.WafError('task without an environment')
+
+Task.task_type_from_func('iso',func=iso_up)
+feature('iso')(apply_iso)
+before('apply_core')(apply_iso)
\ No newline at end of file
diff --git a/tools/waf/tar.py b/tools/waf/tar.py
new file mode 100644
index 00000000000..bf9a91b7521
--- /dev/null
+++ b/tools/waf/tar.py
@@ -0,0 +1,68 @@
+import Utils
+import Options
+import tarfile
+from TaskGen import feature, before
+import Task
+import os, sys
+
+# this is a clever little thing
+# given a list of nodes, build or source
+# construct a tar file containing them
+# rooted in the parameter root =, specified in the task generator
+# and renaming the names of the files according to a rename(x) function passed to the task generator as well
+# if a build node's result of rename() has the same name as a source node, the build node will take precedence
+# for as long as the build node appears later than the source node (this is an implementation detail of waf we are relying on)
+def tar_up(task):
+	tgt = task.outputs[0].bldpath(task.env)
+	if os.path.exists(tgt): os.unlink(tgt)
+        if tgt.lower().endswith(".bz2"): z = tarfile.open(tgt,"w:bz2")
+        elif tgt.lower().endswith(".gz"): z = tarfile.open(tgt,"w:gz")
+        elif tgt.lower().endswith(".tgz"): z = tarfile.open(tgt,"w:gz")
+	else: z = tarfile.open(tgt,"w")
+	fileset = {}
+	for inp in task.inputs:
+		src = inp.srcpath(task.env)
+		if src.startswith(".."):
+			srcname = Utils.relpath(src,os.path.join("..",".")) # file in source dir
+		else:
+			srcname = Utils.relpath(src,os.path.join(task.env.variant(),".")) # file in artifacts dir
+		srcname = srcname.split(os.path.sep,len(task.generator.root.split(os.path.sep)))[-1]
+		if task.generator.rename: srcname = task.generator.rename(srcname)
+		fileset[srcname] = src
+	for srcname,src in fileset.items():
+		ti = tarfile.TarInfo(srcname)
+		ti.mode = 0755
+		ti.size = os.path.getsize(src)
+                openmode = 'r'
+                if Options.platform == 'win32': openmode = openmode + 'b'
+                f = file(src,openmode)
+		z.addfile(ti,fileobj=f)
+		f.close()
+	z.close()
+	if task.chmod: os.chmod(tgt,task.chmod)
+	return 0
+
+def apply_tar(self):
+	Utils.def_attrs(self,fun=tar_up)
+	self.default_install_path=0
+	lst=self.to_list(self.source)
+	self.meths.remove('apply_core')
+	self.dict=getattr(self,'dict',{})
+	out = self.path.find_or_declare(self.target)
+	ins = []
+	for x in Utils.to_list(self.source):
+		node = self.path.find_resource(x)
+		if not node:raise Utils.WafError('cannot find input file %s for processing'%x)
+		ins.append(node)
+	tsk=self.create_task('tar',ins,out)
+	tsk.fun=self.fun
+	tsk.dict=self.dict
+	tsk.install_path=self.install_path
+	tsk.chmod=self.chmod
+	if not tsk.env:
+		tsk.debug()
+		raise Utils.WafError('task without an environment')
+
+Task.task_type_from_func('tar',func=tar_up)
+feature('tar')(apply_tar)
+before('apply_core')(apply_tar)
diff --git a/tools/waf/tomcat.py b/tools/waf/tomcat.py
new file mode 100644
index 00000000000..e314c32beb0
--- /dev/null
+++ b/tools/waf/tomcat.py
@@ -0,0 +1,41 @@
+#!/usr/bin/env python
+
+import Options, Utils
+import os
+
+def detect(conf):
+	if not conf.env.DATADIR:
+		conf.fatal("DATADIR not found in the environment.  Did you run conf.check_tool('gnu_dirs') before running check_tool('tomcat')?")
+	conf.check_message_1('Detecting Tomcat')
+	conf.env.TOMCATHOME = ''
+	tomcathome = getattr(Options.options, 'TOMCATHOME', '')
+	if tomcathome:
+		conf.env.TOMCATHOME = tomcathome
+		method = "forced through --with-tomcat"
+	else:
+		if    "TOMCAT_HOME" in conf.environ and conf.environ['TOMCAT_HOME'].strip():
+			conf.env.TOMCATHOME = conf.environ["TOMCAT_HOME"]
+			method = 'got through environment variable %TOMCAT_HOME%'
+		elif  "CATALINA_HOME" in conf.environ and conf.environ['CATALINA_HOME'].strip():
+			conf.env.TOMCATHOME = conf.environ['CATALINA_HOME']
+			method = 'got through environment variable %CATALINA_HOME%'
+		elif os.path.isdir(os.path.join(conf.env.DATADIR,"tomcat6")):
+			conf.env.TOMCATHOME = os.path.join(conf.env.DATADIR,"tomcat6")
+			method = 'detected existence of Tomcat directory under $DATADIR'
+		elif os.path.isdir("/usr/share/tomcat6"):
+			conf.env.TOMCATHOME = "/usr/share/tomcat6"
+			method = 'detected existence of standard Linux system directory'
+	if not conf.env.TOMCATHOME:
+		conf.fatal("Could not detect Tomcat")
+	elif not os.path.isdir(conf.env.TOMCATHOME):
+		conf.fatal("Tomcat cannot be found at %s"%conf.env.TOMCATHOME)
+	else:
+		conf.check_message_2("%s (%s)"%(conf.env.TOMCATHOME,method),"GREEN")
+
+def set_options(opt):
+        inst_dir = opt.get_option_group('--datadir') # get the group that contains bindir
+        if not inst_dir: raise Utils.WafError, "DATADIR not set.  Did you load the gnu_dirs tool options with opt.tool_options('gnu_dirs') before running opt.tool_options('tomcat')?"
+	inst_dir.add_option('--with-tomcat', # add javadir to the group that contains bindir
+		help = 'Path to installed Tomcat 6 environment [Default: ${DATADIR}/tomcat6 (unless %%CATALINA_HOME%% is set)]',
+		default = '',
+		dest = 'TOMCATHOME')
\ No newline at end of file
diff --git a/tools/waf/usermgmt.py b/tools/waf/usermgmt.py
new file mode 100644
index 00000000000..65fd889d330
--- /dev/null
+++ b/tools/waf/usermgmt.py
@@ -0,0 +1,124 @@
+import Utils, Build
+from TaskGen import feature, before
+from Configure import ConfigurationError
+import Options
+import Task
+import os
+
+def detect(conf):
+	if Options.platform == 'win32': raise Utils.WafError('the usermgmt tool only works on Linux')
+	if Options.platform == 'darwin': raise Utils.WafError('the usermgmt tool only works on Linux')
+	path_list = ["/usr/local/sbin","/usr/sbin","/sbin"] + os.environ.get('PATH','').split(os.pathsep)
+	conf.find_program("useradd",var='USERADD',mandatory=True,path_list=path_list)
+	conf.find_program("userdel",var='USERDEL',mandatory=True,path_list=path_list)
+
+def set_options(opt):
+	if Options.platform == 'win32': raise Utils.WafError('the usermgmt tool only works on Linux')
+	if Options.platform == 'darwin': raise Utils.WafError('the usermgmt tool only works on Linux')
+	og = opt.get_option_group('--force')
+	og.add_option('--nochown',
+		action = 'store_true',
+		help = 'do not create or remove user accounts or change file ownership on installed files',
+		default = False,
+		dest = 'NOUSERMGMT')
+
+def _subst_add_destdir(x,bld):
+	a = "${DESTDIR}" + x
+	a = a.replace("${DESTDIR}",Options.options.destdir)
+	a = Utils.subst_vars(a,bld.env)
+	if a.startswith("//"): a = a[1:]
+	return a
+Build.BuildContext.subst_add_destdir = staticmethod(_subst_add_destdir)
+
+def _setownership(ctx,path,owner,group,mode=None):
+	if Options.platform == 'win32': return
+	if Options.platform == 'darwin': return
+	if not hasattr(os,"getuid"): return
+	if os.getuid() != 0: return
+	if Options.options.NOUSERMGMT: return
+
+	import pwd
+	import grp
+	import stat
+	from os import chown as _chown, chmod as _chmod
+
+	def f(bld,path,owner,group,mode):
+		
+		try: uid = pwd.getpwnam(owner).pw_uid
+		except KeyError,e:
+			raise Utils.WafError("Before using setownership() you have to create the user with bld.createuser(username...)")
+		try: gid = grp.getgrnam(group).gr_gid
+		except KeyError,e:
+			raise Utils.WafError("Before using setownership() you have to create the user with bld.createuser(username...)")
+		
+		path = bld.subst_add_destdir(path,bld)
+		current_uid,current_gid = os.stat(path).st_uid,os.stat(path).st_gid
+		if current_uid != uid:
+			Utils.pprint("GREEN","* setting owner of %s to UID %s"%(path,uid))
+			_chown(path,uid,current_gid)
+			current_uid = uid
+		if current_gid != gid:
+			Utils.pprint("GREEN","* setting group of %s to GID %s"%(path,gid))
+			_chown(path,current_uid,gid)
+			current_gid = gid
+		if mode is not None:
+			current_mode = stat.S_IMODE(os.stat(path).st_mode)
+			if current_mode != mode:
+				Utils.pprint("GREEN","* adjusting permissions on %s to mode %o"%(path,mode))
+				_chmod(path,mode)
+				current_mode = mode
+	
+	if ctx.is_install > 0:
+		ctx.add_post_fun(lambda ctx: f(ctx,path,owner,group,mode))
+Build.BuildContext.setownership = _setownership
+
+def _createuser(ctx,user,homedir,shell):
+	if Options.platform == 'win32': return
+	if Options.platform == 'darwin': return
+	if not hasattr(os,"getuid"): return
+	if os.getuid() != 0: return
+	if Options.options.NOUSERMGMT: return
+	
+	def f(ctx,user,homedir,shell):
+		import pwd
+		try:
+			pwd.getpwnam(user).pw_uid
+			user_exists = True
+		except KeyError,e:
+			user_exists = False
+		if user_exists: return
+		
+		Utils.pprint("GREEN","* creating user %s"%user)
+		cmd = [
+		  ctx.env.USERADD,
+		  '-M',
+		  '-r',
+		  '-s',shell,
+		  '-d',homedir,
+		  user,
+		]
+		ret = Utils.exec_command(cmd)
+		if ret: raise Utils.WafError("Failed to run command %s"%cmd)
+	
+	def g(ctx,user,homedir,shell):
+		import pwd
+		try:
+			pwd.getpwnam(user).pw_uid
+			user_exists = True
+		except KeyError,e:
+			user_exists = False
+		if not user_exists: return
+		
+		Utils.pprint("GREEN","* removing user %s"%user)
+		cmd = [
+		  ctx.env.USERDEL,
+		  user,
+		]
+		ret = Utils.exec_command(cmd)
+		if ret: raise Utils.WafError("Failed to run command %s"%cmd)
+	
+	if ctx.is_install > 0:
+		ctx.add_pre_fun(lambda ctx: f(ctx,user,homedir,shell))
+	elif ctx.is_install < 0:
+		ctx.add_pre_fun(lambda ctx: g(ctx,user,homedir,shell))
+Build.BuildContext.createuser = _createuser
\ No newline at end of file
diff --git a/ui/images/help_actionicon.png b/ui/images/help_actionicon.png
new file mode 100644
index 00000000000..77f87eff354
Binary files /dev/null and b/ui/images/help_actionicon.png differ
diff --git a/ui/images/search_closeicon.gif b/ui/images/search_closeicon.gif
new file mode 100644
index 00000000000..c6148ab69d0
Binary files /dev/null and b/ui/images/search_closeicon.gif differ
diff --git a/ui/images/search_closeicon_hover.gif b/ui/images/search_closeicon_hover.gif
new file mode 100644
index 00000000000..f011186d669
Binary files /dev/null and b/ui/images/search_closeicon_hover.gif differ
diff --git a/ui/index.html b/ui/index.jsp
old mode 100644
new mode 100755
similarity index 82%
rename from ui/index.html
rename to ui/index.jsp
index 3e318cdf753..dc68a470ba8
--- a/ui/index.html
+++ b/ui/index.jsp
@@ -1,3 +1,9 @@
+<%@ page import="java.util.Date" %>
+
+<%
+long milliseconds = new Date().getTime();
+%>
+
 
 
 
@@ -5,7 +11,7 @@
 	  
 	  
 	
-	
+	
 
 	cloud.com - User Console
 	
@@ -19,7 +25,7 @@
 		- Default Cloud.com styling of the site.  This file contains the easiest portion of the site
         that can be styled to your companie's need such as logo, top navigation, and dialogs.		
 	-->
-	
+	
 	
 	
 	
@@ -37,12 +43,21 @@
 	
 	
 	
-	
+	
 	
 	
-	
-	
-	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
+	
 		
 	
 	
diff --git a/ui/content/tab_accounts.html b/ui/jsp/tab_accounts.jsp
old mode 100644
new mode 100755
similarity index 98%
rename from ui/content/tab_accounts.html
rename to ui/jsp/tab_accounts.jsp
index 5a63bb419c5..8bd1bb882b8
--- a/ui/content/tab_accounts.html
+++ b/ui/jsp/tab_accounts.jsp
@@ -1,4 +1,7 @@
-
+<%@ page import="java.util.Date" %>
+<%
+long milliseconds = new Date().getTime();
+%>
 	
 
 

diff --git a/ui/content/tab_configuration.html b/ui/jsp/tab_configuration.jsp
old mode 100644
new mode 100755
similarity index 99%
rename from ui/content/tab_configuration.html
rename to ui/jsp/tab_configuration.jsp
index 9bdbd581bb9..c9f574702fc
--- a/ui/content/tab_configuration.html
+++ b/ui/jsp/tab_configuration.jsp
@@ -1,4 +1,7 @@
-
+<%@ page import="java.util.Date" %>
+<%
+long milliseconds = new Date().getTime();
+%>
 	
 
 
@@ -430,7 +433,7 @@
 					
 					
 				
-				
  • 
 					
 					
 					
 				
    • 
-				
  • 
+				
  • 
 				    Cluster Options
 				
    • 
-				
  • 
+				
  • 
 				    
 				    
 					
diff --git a/ui/content/tab_instances.html b/ui/jsp/tab_instances.jsp
old mode 100644
new mode 100755
similarity index 99%
rename from ui/content/tab_instances.html
rename to ui/jsp/tab_instances.jsp
index 9bc3b94701d..6676b55432e
--- a/ui/content/tab_instances.html
+++ b/ui/jsp/tab_instances.jsp
@@ -1,4 +1,7 @@
-	
+<%@ page import="java.util.Date" %>
+<%
+long milliseconds = new Date().getTime();
+%>
 
 
 
@@ -704,8 +707,7 @@
                     
    
                         Step 3: Optional
    
                     
    
-                        To create a new instance, please first select a zone you wish to have your virtual
-                        instance hosted on.
    
+                        You can choose to name and group your virtual machine for easy identification. You can also choose additional data storage. (These options can be added at any time.) 
    
                 
    • 
                 
    
                     
    
diff --git a/ui/content/tab_networking.html b/ui/jsp/tab_networking.jsp
old mode 100644
new mode 100755
similarity index 99%
rename from ui/content/tab_networking.html
rename to ui/jsp/tab_networking.jsp
index e78b41eb636..ae81f2d107e
--- a/ui/content/tab_networking.html
+++ b/ui/jsp/tab_networking.jsp
@@ -1,4 +1,8 @@
-
+<%@ page import="java.util.Date" %>
+<%
+long milliseconds = new Date().getTime();
+%>
+
 
 
 
    
diff --git a/ui/content/tab_storage.html b/ui/jsp/tab_storage.jsp
old mode 100644
new mode 100755
similarity index 99%
rename from ui/content/tab_storage.html
rename to ui/jsp/tab_storage.jsp
index 2250ebddacd..f84a9405cde
--- a/ui/content/tab_storage.html
+++ b/ui/jsp/tab_storage.jsp
@@ -1,4 +1,7 @@
-
+<%@ page import="java.util.Date" %>
+<%
+long milliseconds = new Date().getTime();
+%>
 	
 
    
     
-            
    
+            
    
                 
    
                     Type
    
             
    
@@ -631,7 +634,7 @@
                 
    
                     Size
    
             
    
-             
    
+             
    
                 
    
                     State
    
             
    
@@ -703,7 +706,7 @@
             
    
                 Name
    
         
    
-        
    
+        
    
             
    
                 Type
    
         
    
@@ -723,7 +726,7 @@
             
    
                 size
    
         
    
-        
    
+        
    
             
    
                 state
    
         
    
@@ -1516,7 +1519,14 @@
 					
 					
-								
+						
+				
  • 
+					
+					
+				
    • 
 			
 		
 	
    
diff --git a/ui/content/tab_templates.html b/ui/jsp/tab_templates.jsp
old mode 100644
new mode 100755
similarity index 94%
rename from ui/content/tab_templates.html
rename to ui/jsp/tab_templates.jsp
index 3438c971de2..d4e54e4161f
--- a/ui/content/tab_templates.html
+++ b/ui/jsp/tab_templates.jsp
@@ -1,4 +1,7 @@
-
+<%@ page import="java.util.Date" %>
+<%
+long milliseconds = new Date().getTime();
+%>
 	
 
    		
     
@@ -610,9 +613,9 @@
 
 
 
-
-').css({width:this.offsetWidth+"px",height:this.offsetHeight+"px",position:"absolute",opacity:"0.001",zIndex:1E3}).css(d(this).offset()).appendTo("body")})},stop:function(){d("div.ui-draggable-iframeFix").each(function(){this.parentNode.removeChild(this)})}});d.ui.plugin.add("draggable","opacity",{start:function(a,b){a=d(b.helper);b=d(this).data("draggable").options;
+if(a.css("opacity"))b._opacity=a.css("opacity");a.css("opacity",b.opacity)},stop:function(a,b){a=d(this).data("draggable").options;a._opacity&&d(b.helper).css("opacity",a._opacity)}});d.ui.plugin.add("draggable","scroll",{start:function(){var a=d(this).data("draggable");if(a.scrollParent[0]!=document&&a.scrollParent[0].tagName!="HTML")a.overflowOffset=a.scrollParent.offset()},drag:function(a){var b=d(this).data("draggable"),c=b.options,f=false;if(b.scrollParent[0]!=document&&b.scrollParent[0].tagName!=
+"HTML"){if(!c.axis||c.axis!="x")if(b.overflowOffset.top+b.scrollParent[0].offsetHeight-a.pageY=0;h--){var i=c.snapElements[h].left,k=i+c.snapElements[h].width,j=c.snapElements[h].top,l=j+c.snapElements[h].height;if(i-e=j&&f<=l||h>=j&&h<=l||fl)&&(e>=i&&
+e<=k||g>=i&&g<=k||ek);default:return false}};d.ui.ddmanager={current:null,droppables:{"default":[]},prepareOffsets:function(a,b){var c=d.ui.ddmanager.droppables[a.options.scope]||[],e=b?b.type:null,g=(a.currentItem||a.element).find(":data(droppable)").andSelf(),f=0;a:for(;f
    ').css({position:this.element.css("position"),width:this.element.outerWidth(),height:this.element.outerHeight(),
+top:this.element.css("top"),left:this.element.css("left")}));this.element=this.element.parent().data("resizable",this.element.data("resizable"));this.elementIsWrapper=true;this.element.css({marginLeft:this.originalElement.css("marginLeft"),marginTop:this.originalElement.css("marginTop"),marginRight:this.originalElement.css("marginRight"),marginBottom:this.originalElement.css("marginBottom")});this.originalElement.css({marginLeft:0,marginTop:0,marginRight:0,marginBottom:0});this.originalResizeStyle=
+this.originalElement.css("resize");this.originalElement.css("resize","none");this._proportionallyResizeElements.push(this.originalElement.css({position:"static",zoom:1,display:"block"}));this.originalElement.css({margin:this.originalElement.css("margin")});this._proportionallyResize()}this.handles=a.handles||(!d(".ui-resizable-handle",this.element).length?"e,s,se":{n:".ui-resizable-n",e:".ui-resizable-e",s:".ui-resizable-s",w:".ui-resizable-w",se:".ui-resizable-se",sw:".ui-resizable-sw",ne:".ui-resizable-ne",
+nw:".ui-resizable-nw"});if(this.handles.constructor==String){if(this.handles=="all")this.handles="n,e,s,w,se,sw,ne,nw";var c=this.handles.split(",");this.handles={};for(var e=0;e
    ');/sw|se|ne|nw/.test(g)&&f.css({zIndex:++a.zIndex});"se"==g&&f.addClass("ui-icon ui-icon-gripsmall-diagonal-se");this.handles[g]=".ui-resizable-"+g;this.element.append(f)}}this._renderAxis=function(h){h=h||this.element;for(var i in this.handles){if(this.handles[i].constructor==
+String)this.handles[i]=d(this.handles[i],this.element).show();if(this.elementIsWrapper&&this.originalElement[0].nodeName.match(/textarea|input|select|button/i)){var j=d(this.handles[i],this.element),l=0;l=/sw|ne|nw|se|n|s/.test(i)?j.outerHeight():j.outerWidth();j=["padding",/ne|nw|n/.test(i)?"Top":/se|sw|s/.test(i)?"Bottom":/^e$/.test(i)?"Right":"Left"].join("");h.css(j,l);this._proportionallyResize()}d(this.handles[i])}};this._renderAxis(this.element);this._handles=d(".ui-resizable-handle",this.element).disableSelection();
+this._handles.mouseover(function(){if(!b.resizing){if(this.className)var h=this.className.match(/ui-resizable-(se|sw|ne|nw|n|e|s|w)/i);b.axis=h&&h[1]?h[1]:"se"}});if(a.autoHide){this._handles.hide();d(this.element).addClass("ui-resizable-autohide").hover(function(){d(this).removeClass("ui-resizable-autohide");b._handles.show()},function(){if(!b.resizing){d(this).addClass("ui-resizable-autohide");b._handles.hide()}})}this._mouseInit()},destroy:function(){this._mouseDestroy();var b=function(c){d(c).removeClass("ui-resizable ui-resizable-disabled ui-resizable-resizing").removeData("resizable").unbind(".resizable").find(".ui-resizable-handle").remove()};
+if(this.elementIsWrapper){b(this.element);var a=this.element;a.after(this.originalElement.css({position:a.css("position"),width:a.outerWidth(),height:a.outerHeight(),top:a.css("top"),left:a.css("left")})).remove()}this.originalElement.css("resize",this.originalResizeStyle);b(this.originalElement);return this},_mouseCapture:function(b){var a=false;for(var c in this.handles)if(d(this.handles[c])[0]==b.target)a=true;return!this.options.disabled&&a},_mouseStart:function(b){var a=this.options,c=this.element.position(),
+e=this.element;this.resizing=true;this.documentScroll={top:d(document).scrollTop(),left:d(document).scrollLeft()};if(e.is(".ui-draggable")||/absolute/.test(e.css("position")))e.css({position:"absolute",top:c.top,left:c.left});d.browser.opera&&/relative/.test(e.css("position"))&&e.css({position:"relative",top:"auto",left:"auto"});this._renderProxy();c=m(this.helper.css("left"));var g=m(this.helper.css("top"));if(a.containment){c+=d(a.containment).scrollLeft()||0;g+=d(a.containment).scrollTop()||0}this.offset=
+this.helper.offset();this.position={left:c,top:g};this.size=this._helper?{width:e.outerWidth(),height:e.outerHeight()}:{width:e.width(),height:e.height()};this.originalSize=this._helper?{width:e.outerWidth(),height:e.outerHeight()}:{width:e.width(),height:e.height()};this.originalPosition={left:c,top:g};this.sizeDiff={width:e.outerWidth()-e.width(),height:e.outerHeight()-e.height()};this.originalMousePosition={left:b.pageX,top:b.pageY};this.aspectRatio=typeof a.aspectRatio=="number"?a.aspectRatio:
+this.originalSize.width/this.originalSize.height||1;a=d(".ui-resizable-"+this.axis).css("cursor");d("body").css("cursor",a=="auto"?this.axis+"-resize":a);e.addClass("ui-resizable-resizing");this._propagate("start",b);return true},_mouseDrag:function(b){var a=this.helper,c=this.originalMousePosition,e=this._change[this.axis];if(!e)return false;c=e.apply(this,[b,b.pageX-c.left||0,b.pageY-c.top||0]);if(this._aspectRatio||b.shiftKey)c=this._updateRatio(c,b);c=this._respectSize(c,b);this._propagate("resize",
+b);a.css({top:this.position.top+"px",left:this.position.left+"px",width:this.size.width+"px",height:this.size.height+"px"});!this._helper&&this._proportionallyResizeElements.length&&this._proportionallyResize();this._updateCache(c);this._trigger("resize",b,this.ui());return false},_mouseStop:function(b){this.resizing=false;var a=this.options,c=this;if(this._helper){var e=this._proportionallyResizeElements,g=e.length&&/textarea/i.test(e[0].nodeName);e=g&&d.ui.hasScroll(e[0],"left")?0:c.sizeDiff.height;
+g={width:c.size.width-(g?0:c.sizeDiff.width),height:c.size.height-e};e=parseInt(c.element.css("left"),10)+(c.position.left-c.originalPosition.left)||null;var f=parseInt(c.element.css("top"),10)+(c.position.top-c.originalPosition.top)||null;a.animate||this.element.css(d.extend(g,{top:f,left:e}));c.helper.height(c.size.height);c.helper.width(c.size.width);this._helper&&!a.animate&&this._proportionallyResize()}d("body").css("cursor","auto");this.element.removeClass("ui-resizable-resizing");this._propagate("stop",
+b);this._helper&&this.helper.remove();return false},_updateCache:function(b){this.offset=this.helper.offset();if(k(b.left))this.position.left=b.left;if(k(b.top))this.position.top=b.top;if(k(b.height))this.size.height=b.height;if(k(b.width))this.size.width=b.width},_updateRatio:function(b){var a=this.position,c=this.size,e=this.axis;if(b.height)b.width=c.height*this.aspectRatio;else if(b.width)b.height=c.width/this.aspectRatio;if(e=="sw"){b.left=a.left+(c.width-b.width);b.top=null}if(e=="nw"){b.top=
+a.top+(c.height-b.height);b.left=a.left+(c.width-b.width)}return b},_respectSize:function(b){var a=this.options,c=this.axis,e=k(b.width)&&a.maxWidth&&a.maxWidthb.width,h=k(b.height)&&a.minHeight&&a.minHeight>b.height;if(f)b.width=a.minWidth;if(h)b.height=a.minHeight;if(e)b.width=a.maxWidth;if(g)b.height=a.maxHeight;var i=this.originalPosition.left+this.originalSize.width,j=this.position.top+this.size.height,
+l=/sw|nw|w/.test(c);c=/nw|ne|n/.test(c);if(f&&l)b.left=i-a.minWidth;if(e&&l)b.left=i-a.maxWidth;if(h&&c)b.top=j-a.minHeight;if(g&&c)b.top=j-a.maxHeight;if((a=!b.width&&!b.height)&&!b.left&&b.top)b.top=null;else if(a&&!b.top&&b.left)b.left=null;return b},_proportionallyResize:function(){if(this._proportionallyResizeElements.length)for(var b=this.helper||this.element,a=0;a
    ');var a=d.browser.msie&&d.browser.version<7,c=a?1:0;a=a?2:-1;this.helper.addClass(this._helper).css({width:this.element.outerWidth()+a,height:this.element.outerHeight()+a,position:"absolute",left:this.elementOffset.left-c+"px",top:this.elementOffset.top-c+"px",zIndex:++b.zIndex});this.helper.appendTo("body").disableSelection()}else this.helper=this.element},_change:{e:function(b,a){return{width:this.originalSize.width+
+a}},w:function(b,a){return{left:this.originalPosition.left+a,width:this.originalSize.width-a}},n:function(b,a,c){return{top:this.originalPosition.top+c,height:this.originalSize.height-c}},s:function(b,a,c){return{height:this.originalSize.height+c}},se:function(b,a,c){return d.extend(this._change.s.apply(this,arguments),this._change.e.apply(this,[b,a,c]))},sw:function(b,a,c){return d.extend(this._change.s.apply(this,arguments),this._change.w.apply(this,[b,a,c]))},ne:function(b,a,c){return d.extend(this._change.n.apply(this,
+arguments),this._change.e.apply(this,[b,a,c]))},nw:function(b,a,c){return d.extend(this._change.n.apply(this,arguments),this._change.w.apply(this,[b,a,c]))}},_propagate:function(b,a){d.ui.plugin.call(this,b,[a,this.ui()]);b!="resize"&&this._trigger(b,a,this.ui())},plugins:{},ui:function(){return{originalElement:this.originalElement,element:this.element,helper:this.helper,position:this.position,size:this.size,originalSize:this.originalSize,originalPosition:this.originalPosition}}});d.extend(d.ui.resizable,
+{version:"1.8.2"});d.ui.plugin.add("resizable","alsoResize",{start:function(){var b=d(this).data("resizable").options,a=function(c){d(c).each(function(){d(this).data("resizable-alsoresize",{width:parseInt(d(this).width(),10),height:parseInt(d(this).height(),10),left:parseInt(d(this).css("left"),10),top:parseInt(d(this).css("top"),10)})})};if(typeof b.alsoResize=="object"&&!b.alsoResize.parentNode)if(b.alsoResize.length){b.alsoResize=b.alsoResize[0];a(b.alsoResize)}else d.each(b.alsoResize,function(c){a(c)});
+else a(b.alsoResize)},resize:function(){var b=d(this).data("resizable"),a=b.options,c=b.originalSize,e=b.originalPosition,g={height:b.size.height-c.height||0,width:b.size.width-c.width||0,top:b.position.top-e.top||0,left:b.position.left-e.left||0},f=function(h,i){d(h).each(function(){var j=d(this),l=d(this).data("resizable-alsoresize"),p={};d.each((i&&i.length?i:["width","height","top","left"])||["width","height","top","left"],function(n,o){if((n=(l[o]||0)+(g[o]||0))&&n>=0)p[o]=n||null});if(/relative/.test(j.css("position"))&&
+d.browser.opera){b._revertToRelativePosition=true;j.css({position:"absolute",top:"auto",left:"auto"})}j.css(p)})};typeof a.alsoResize=="object"&&!a.alsoResize.nodeType?d.each(a.alsoResize,function(h,i){f(h,i)}):f(a.alsoResize)},stop:function(){var b=d(this).data("resizable");if(b._revertToRelativePosition&&d.browser.opera){b._revertToRelativePosition=false;el.css({position:"relative"})}d(this).removeData("resizable-alsoresize-start")}});d.ui.plugin.add("resizable","animate",{stop:function(b){var a=
+d(this).data("resizable"),c=a.options,e=a._proportionallyResizeElements,g=e.length&&/textarea/i.test(e[0].nodeName),f=g&&d.ui.hasScroll(e[0],"left")?0:a.sizeDiff.height;g={width:a.size.width-(g?0:a.sizeDiff.width),height:a.size.height-f};f=parseInt(a.element.css("left"),10)+(a.position.left-a.originalPosition.left)||null;var h=parseInt(a.element.css("top"),10)+(a.position.top-a.originalPosition.top)||null;a.element.animate(d.extend(g,h&&f?{top:h,left:f}:{}),{duration:c.animateDuration,easing:c.animateEasing,
+step:function(){var i={width:parseInt(a.element.css("width"),10),height:parseInt(a.element.css("height"),10),top:parseInt(a.element.css("top"),10),left:parseInt(a.element.css("left"),10)};e&&e.length&&d(e[0]).css({width:i.width,height:i.height});a._updateCache(i);a._propagate("resize",b)}})}});d.ui.plugin.add("resizable","containment",{start:function(){var b=d(this).data("resizable"),a=b.element,c=b.options.containment;if(a=c instanceof d?c.get(0):/parent/.test(c)?a.parent().get(0):c){b.containerElement=
+d(a);if(/document/.test(c)||c==document){b.containerOffset={left:0,top:0};b.containerPosition={left:0,top:0};b.parentData={element:d(document),left:0,top:0,width:d(document).width(),height:d(document).height()||document.body.parentNode.scrollHeight}}else{var e=d(a),g=[];d(["Top","Right","Left","Bottom"]).each(function(i,j){g[i]=m(e.css("padding"+j))});b.containerOffset=e.offset();b.containerPosition=e.position();b.containerSize={height:e.innerHeight()-g[3],width:e.innerWidth()-g[1]};c=b.containerOffset;
+var f=b.containerSize.height,h=b.containerSize.width;h=d.ui.hasScroll(a,"left")?a.scrollWidth:h;f=d.ui.hasScroll(a)?a.scrollHeight:f;b.parentData={element:a,left:c.left,top:c.top,width:h,height:f}}}},resize:function(b){var a=d(this).data("resizable"),c=a.options,e=a.containerOffset,g=a.position;b=a._aspectRatio||b.shiftKey;var f={top:0,left:0},h=a.containerElement;if(h[0]!=document&&/static/.test(h.css("position")))f=e;if(g.left<(a._helper?e.left:0)){a.size.width+=a._helper?a.position.left-e.left:
+a.position.left-f.left;if(b)a.size.height=a.size.width/c.aspectRatio;a.position.left=c.helper?e.left:0}if(g.top<(a._helper?e.top:0)){a.size.height+=a._helper?a.position.top-e.top:a.position.top;if(b)a.size.width=a.size.height*c.aspectRatio;a.position.top=a._helper?e.top:0}a.offset.left=a.parentData.left+a.position.left;a.offset.top=a.parentData.top+a.position.top;c=Math.abs((a._helper?a.offset.left-f.left:a.offset.left-f.left)+a.sizeDiff.width);e=Math.abs((a._helper?a.offset.top-f.top:a.offset.top-
+e.top)+a.sizeDiff.height);g=a.containerElement.get(0)==a.element.parent().get(0);f=/relative|absolute/.test(a.containerElement.css("position"));if(g&&f)c-=a.parentData.left;if(c+a.size.width>=a.parentData.width){a.size.width=a.parentData.width-c;if(b)a.size.height=a.size.width/a.aspectRatio}if(e+a.size.height>=a.parentData.height){a.size.height=a.parentData.height-e;if(b)a.size.width=a.size.height*a.aspectRatio}},stop:function(){var b=d(this).data("resizable"),a=b.options,c=b.containerOffset,e=b.containerPosition,
+g=b.containerElement,f=d(b.helper),h=f.offset(),i=f.outerWidth()-b.sizeDiff.width;f=f.outerHeight()-b.sizeDiff.height;b._helper&&!a.animate&&/relative/.test(g.css("position"))&&d(this).css({left:h.left-e.left-c.left,width:i,height:f});b._helper&&!a.animate&&/static/.test(g.css("position"))&&d(this).css({left:h.left-e.left-c.left,width:i,height:f})}});d.ui.plugin.add("resizable","ghost",{start:function(){var b=d(this).data("resizable"),a=b.options,c=b.size;b.ghost=b.originalElement.clone();b.ghost.css({opacity:0.25,
+display:"block",position:"relative",height:c.height,width:c.width,margin:0,left:0,top:0}).addClass("ui-resizable-ghost").addClass(typeof a.ghost=="string"?a.ghost:"");b.ghost.appendTo(b.helper)},resize:function(){var b=d(this).data("resizable");b.ghost&&b.ghost.css({position:"relative",height:b.size.height,width:b.size.width})},stop:function(){var b=d(this).data("resizable");b.ghost&&b.helper&&b.helper.get(0).removeChild(b.ghost.get(0))}});d.ui.plugin.add("resizable","grid",{resize:function(){var b=
+d(this).data("resizable"),a=b.options,c=b.size,e=b.originalSize,g=b.originalPosition,f=b.axis;a.grid=typeof a.grid=="number"?[a.grid,a.grid]:a.grid;var h=Math.round((c.width-e.width)/(a.grid[0]||1))*(a.grid[0]||1);a=Math.round((c.height-e.height)/(a.grid[1]||1))*(a.grid[1]||1);if(/^(se|s|e)$/.test(f)){b.size.width=e.width+h;b.size.height=e.height+a}else if(/^(ne)$/.test(f)){b.size.width=e.width+h;b.size.height=e.height+a;b.position.top=g.top-a}else{if(/^(sw)$/.test(f)){b.size.width=e.width+h;b.size.height=
+e.height+a}else{b.size.width=e.width+h;b.size.height=e.height+a;b.position.top=g.top-a}b.position.left=g.left-h}}});var m=function(b){return parseInt(b,10)||0},k=function(b){return!isNaN(parseInt(b,10))}})(jQuery);
+;
+/*
+ * jQuery UI Selectable 1.8.2
+ *
+ * Copyright (c) 2010 AUTHORS.txt (http://jqueryui.com/about)
+ * Dual licensed under the MIT (MIT-LICENSE.txt)
+ * and GPL (GPL-LICENSE.txt) licenses.
+ *
+ * http://docs.jquery.com/UI/Selectables
+ *
+ * Depends:
+ *	jquery.ui.core.js
+ *	jquery.ui.mouse.js
+ *	jquery.ui.widget.js
+ */
+(function($) {
+
+$.widget("ui.selectable", $.ui.mouse, {
+	options: {
+		appendTo: 'body',
+		autoRefresh: true,
+		distance: 0,
+		filter: '*',
+		tolerance: 'touch'
+	},
+	_create: function() {
+		var self = this;
+
+		this.element.addClass("ui-selectable");
+
+		this.dragged = false;
+
+		// cache selectee children based on filter
+		var selectees;
+		this.refresh = function() {
+			selectees = $(self.options.filter, self.element[0]);
+			selectees.each(function() {
+				var $this = $(this);
+				var pos = $this.offset();
+				$.data(this, "selectable-item", {
+					element: this,
+					$element: $this,
+					left: pos.left,
+					top: pos.top,
+					right: pos.left + $this.outerWidth(),
+					bottom: pos.top + $this.outerHeight(),
+					startselected: false,
+					selected: $this.hasClass('ui-selected'),
+					selecting: $this.hasClass('ui-selecting'),
+					unselecting: $this.hasClass('ui-unselecting')
+				});
+			});
+		};
+		this.refresh();
+
+		this.selectees = selectees.addClass("ui-selectee");
+
+		this._mouseInit();
+
+		this.helper = $("
    ");
+	},
+
+	destroy: function() {
+		this.selectees
+			.removeClass("ui-selectee")
+			.removeData("selectable-item");
+		this.element
+			.removeClass("ui-selectable ui-selectable-disabled")
+			.removeData("selectable")
+			.unbind(".selectable");
+		this._mouseDestroy();
+
+		return this;
+	},
+
+	_mouseStart: function(event) {
+		var self = this;
+
+		this.opos = [event.pageX, event.pageY];
+
+		if (this.options.disabled)
+			return;
+
+		var options = this.options;
+
+		this.selectees = $(options.filter, this.element[0]);
+
+		this._trigger("start", event);
+
+		$(options.appendTo).append(this.helper);
+		// position helper (lasso)
+		this.helper.css({
+			"z-index": 100,
+			"position": "absolute",
+			"left": event.clientX,
+			"top": event.clientY,
+			"width": 0,
+			"height": 0
+		});
+
+		if (options.autoRefresh) {
+			this.refresh();
+		}
+
+		this.selectees.filter('.ui-selected').each(function() {
+			var selectee = $.data(this, "selectable-item");
+			selectee.startselected = true;
+			if (!event.metaKey) {
+				selectee.$element.removeClass('ui-selected');
+				selectee.selected = false;
+				selectee.$element.addClass('ui-unselecting');
+				selectee.unselecting = true;
+				// selectable UNSELECTING callback
+				self._trigger("unselecting", event, {
+					unselecting: selectee.element
+				});
+			}
+		});
+
+		$(event.target).parents().andSelf().each(function() {
+			var selectee = $.data(this, "selectable-item");
+			if (selectee) {
+				var doSelect = !event.metaKey || !selectee.$element.hasClass('ui-selected');
+				selectee.$element
+					.removeClass(doSelect ? "ui-unselecting" : "ui-selected")
+					.addClass(doSelect ? "ui-selecting" : "ui-unselecting");
+				selectee.unselecting = !doSelect;
+				selectee.selecting = doSelect;
+				selectee.selected = doSelect;
+				// selectable (UN)SELECTING callback
+				if (doSelect) {
+					self._trigger("selecting", event, {
+						selecting: selectee.element
+					});
+				} else {
+					self._trigger("unselecting", event, {
+						unselecting: selectee.element
+					});
+				}
+				return false;
+			}
+		});
+
+	},
+
+	_mouseDrag: function(event) {
+		var self = this;
+		this.dragged = true;
+
+		if (this.options.disabled)
+			return;
+
+		var options = this.options;
+
+		var x1 = this.opos[0], y1 = this.opos[1], x2 = event.pageX, y2 = event.pageY;
+		if (x1 > x2) { var tmp = x2; x2 = x1; x1 = tmp; }
+		if (y1 > y2) { var tmp = y2; y2 = y1; y1 = tmp; }
+		this.helper.css({left: x1, top: y1, width: x2-x1, height: y2-y1});
+
+		this.selectees.each(function() {
+			var selectee = $.data(this, "selectable-item");
+			//prevent helper from being selected if appendTo: selectable
+			if (!selectee || selectee.element == self.element[0])
+				return;
+			var hit = false;
+			if (options.tolerance == 'touch') {
+				hit = ( !(selectee.left > x2 || selectee.right < x1 || selectee.top > y2 || selectee.bottom < y1) );
+			} else if (options.tolerance == 'fit') {
+				hit = (selectee.left > x1 && selectee.right < x2 && selectee.top > y1 && selectee.bottom < y2);
+			}
+
+			if (hit) {
+				// SELECT
+				if (selectee.selected) {
+					selectee.$element.removeClass('ui-selected');
+					selectee.selected = false;
+				}
+				if (selectee.unselecting) {
+					selectee.$element.removeClass('ui-unselecting');
+					selectee.unselecting = false;
+				}
+				if (!selectee.selecting) {
+					selectee.$element.addClass('ui-selecting');
+					selectee.selecting = true;
+					// selectable SELECTING callback
+					self._trigger("selecting", event, {
+						selecting: selectee.element
+					});
+				}
+			} else {
+				// UNSELECT
+				if (selectee.selecting) {
+					if (event.metaKey && selectee.startselected) {
+						selectee.$element.removeClass('ui-selecting');
+						selectee.selecting = false;
+						selectee.$element.addClass('ui-selected');
+						selectee.selected = true;
+					} else {
+						selectee.$element.removeClass('ui-selecting');
+						selectee.selecting = false;
+						if (selectee.startselected) {
+							selectee.$element.addClass('ui-unselecting');
+							selectee.unselecting = true;
+						}
+						// selectable UNSELECTING callback
+						self._trigger("unselecting", event, {
+							unselecting: selectee.element
+						});
+					}
+				}
+				if (selectee.selected) {
+					if (!event.metaKey && !selectee.startselected) {
+						selectee.$element.removeClass('ui-selected');
+						selectee.selected = false;
+
+						selectee.$element.addClass('ui-unselecting');
+						selectee.unselecting = true;
+						// selectable UNSELECTING callback
+						self._trigger("unselecting", event, {
+							unselecting: selectee.element
+						});
+					}
+				}
+			}
+		});
+
+		return false;
+	},
+
+	_mouseStop: function(event) {
+		var self = this;
+
+		this.dragged = false;
+
+		var options = this.options;
+
+		$('.ui-unselecting', this.element[0]).each(function() {
+			var selectee = $.data(this, "selectable-item");
+			selectee.$element.removeClass('ui-unselecting');
+			selectee.unselecting = false;
+			selectee.startselected = false;
+			self._trigger("unselected", event, {
+				unselected: selectee.element
+			});
+		});
+		$('.ui-selecting', this.element[0]).each(function() {
+			var selectee = $.data(this, "selectable-item");
+			selectee.$element.removeClass('ui-selecting').addClass('ui-selected');
+			selectee.selecting = false;
+			selectee.selected = true;
+			selectee.startselected = true;
+			self._trigger("selected", event, {
+				selected: selectee.element
+			});
+		});
+		this._trigger("stop", event);
+
+		this.helper.remove();
+
+		return false;
+	}
+
+});
+
+$.extend($.ui.selectable, {
+	version: "1.8.2"
+});
+
+})(jQuery);
+(function(e){e.widget("ui.selectable",e.ui.mouse,{options:{appendTo:"body",autoRefresh:true,distance:0,filter:"*",tolerance:"touch"},_create:function(){var c=this;this.element.addClass("ui-selectable");this.dragged=false;var f;this.refresh=function(){f=e(c.options.filter,c.element[0]);f.each(function(){var d=e(this),b=d.offset();e.data(this,"selectable-item",{element:this,$element:d,left:b.left,top:b.top,right:b.left+d.outerWidth(),bottom:b.top+d.outerHeight(),startselected:false,selected:d.hasClass("ui-selected"),
+selecting:d.hasClass("ui-selecting"),unselecting:d.hasClass("ui-unselecting")})})};this.refresh();this.selectees=f.addClass("ui-selectee");this._mouseInit();this.helper=e("
    ")},destroy:function(){this.selectees.removeClass("ui-selectee").removeData("selectable-item");this.element.removeClass("ui-selectable ui-selectable-disabled").removeData("selectable").unbind(".selectable");this._mouseDestroy();return this},_mouseStart:function(c){var f=this;this.opos=[c.pageX,
+c.pageY];if(!this.options.disabled){var d=this.options;this.selectees=e(d.filter,this.element[0]);this._trigger("start",c);e(d.appendTo).append(this.helper);this.helper.css({"z-index":100,position:"absolute",left:c.clientX,top:c.clientY,width:0,height:0});d.autoRefresh&&this.refresh();this.selectees.filter(".ui-selected").each(function(){var b=e.data(this,"selectable-item");b.startselected=true;if(!c.metaKey){b.$element.removeClass("ui-selected");b.selected=false;b.$element.addClass("ui-unselecting");
+b.unselecting=true;f._trigger("unselecting",c,{unselecting:b.element})}});e(c.target).parents().andSelf().each(function(){var b=e.data(this,"selectable-item");if(b){var g=!c.metaKey||!b.$element.hasClass("ui-selected");b.$element.removeClass(g?"ui-unselecting":"ui-selected").addClass(g?"ui-selecting":"ui-unselecting");b.unselecting=!g;b.selecting=g;(b.selected=g)?f._trigger("selecting",c,{selecting:b.element}):f._trigger("unselecting",c,{unselecting:b.element});return false}})}},_mouseDrag:function(c){var f=
+this;this.dragged=true;if(!this.options.disabled){var d=this.options,b=this.opos[0],g=this.opos[1],h=c.pageX,i=c.pageY;if(b>h){var j=h;h=b;b=j}if(g>i){j=i;i=g;g=j}this.helper.css({left:b,top:g,width:h-b,height:i-g});this.selectees.each(function(){var a=e.data(this,"selectable-item");if(!(!a||a.element==f.element[0])){var k=false;if(d.tolerance=="touch")k=!(a.left>h||a.righti||a.bottomb&&a.rightg&&a.bottom *",opacity:false,placeholder:false,revert:false,scroll:true,scrollSensitivity:20,scrollSpeed:20,scope:"default",tolerance:"intersect",zIndex:1E3},_create:function(){this.containerCache={};this.element.addClass("ui-sortable");
+this.refresh();this.floating=this.items.length?/left|right/.test(this.items[0].item.css("float")):false;this.offset=this.element.offset();this._mouseInit()},destroy:function(){this.element.removeClass("ui-sortable ui-sortable-disabled").removeData("sortable").unbind(".sortable");this._mouseDestroy();for(var a=this.items.length-1;a>=0;a--)this.items[a].item.removeData("sortable-item");return this},_setOption:function(a,b){if(a==="disabled"){this.options[a]=b;this.widget()[b?"addClass":"removeClass"]("ui-sortable-disabled")}else d.Widget.prototype._setOption.apply(this,
+arguments)},_mouseCapture:function(a,b){if(this.reverting)return false;if(this.options.disabled||this.options.type=="static")return false;this._refreshItems(a);var c=null,e=this;d(a.target).parents().each(function(){if(d.data(this,"sortable-item")==e){c=d(this);return false}});if(d.data(a.target,"sortable-item")==e)c=d(a.target);if(!c)return false;if(this.options.handle&&!b){var f=false;d(this.options.handle,c).find("*").andSelf().each(function(){if(this==a.target)f=true});if(!f)return false}this.currentItem=
+c;this._removeCurrentsFromItems();return true},_mouseStart:function(a,b,c){b=this.options;var e=this;this.currentContainer=this;this.refreshPositions();this.helper=this._createHelper(a);this._cacheHelperProportions();this._cacheMargins();this.scrollParent=this.helper.scrollParent();this.offset=this.currentItem.offset();this.offset={top:this.offset.top-this.margins.top,left:this.offset.left-this.margins.left};this.helper.css("position","absolute");this.cssPosition=this.helper.css("position");d.extend(this.offset,
+{click:{left:a.pageX-this.offset.left,top:a.pageY-this.offset.top},parent:this._getParentOffset(),relative:this._getRelativeOffset()});this.originalPosition=this._generatePosition(a);this.originalPageX=a.pageX;this.originalPageY=a.pageY;b.cursorAt&&this._adjustOffsetFromHelper(b.cursorAt);this.domPosition={prev:this.currentItem.prev()[0],parent:this.currentItem.parent()[0]};this.helper[0]!=this.currentItem[0]&&this.currentItem.hide();this._createPlaceholder();b.containment&&this._setContainment();
+if(b.cursor){if(d("body").css("cursor"))this._storedCursor=d("body").css("cursor");d("body").css("cursor",b.cursor)}if(b.opacity){if(this.helper.css("opacity"))this._storedOpacity=this.helper.css("opacity");this.helper.css("opacity",b.opacity)}if(b.zIndex){if(this.helper.css("zIndex"))this._storedZIndex=this.helper.css("zIndex");this.helper.css("zIndex",b.zIndex)}if(this.scrollParent[0]!=document&&this.scrollParent[0].tagName!="HTML")this.overflowOffset=this.scrollParent.offset();this._trigger("start",
+a,this._uiHash());this._preserveHelperProportions||this._cacheHelperProportions();if(!c)for(c=this.containers.length-1;c>=0;c--)this.containers[c]._trigger("activate",a,e._uiHash(this));if(d.ui.ddmanager)d.ui.ddmanager.current=this;d.ui.ddmanager&&!b.dropBehaviour&&d.ui.ddmanager.prepareOffsets(this,a);this.dragging=true;this.helper.addClass("ui-sortable-helper");this._mouseDrag(a);return true},_mouseDrag:function(a){this.position=this._generatePosition(a);this.positionAbs=this._convertPositionTo("absolute");
+if(!this.lastPositionAbs)this.lastPositionAbs=this.positionAbs;if(this.options.scroll){var b=this.options,c=false;if(this.scrollParent[0]!=document&&this.scrollParent[0].tagName!="HTML"){if(this.overflowOffset.top+this.scrollParent[0].offsetHeight-a.pageY=0;b--){c=this.items[b];var e=c.item[0],f=this._intersectsWithPointer(c);if(f)if(e!=this.currentItem[0]&&this.placeholder[f==1?"next":"prev"]()[0]!=e&&!d.ui.contains(this.placeholder[0],e)&&(this.options.type=="semi-dynamic"?!d.ui.contains(this.element[0],e):true)){this.direction=f==1?"down":"up";if(this.options.tolerance=="pointer"||this._intersectsWithSides(c))this._rearrange(a,
+c);else break;this._trigger("change",a,this._uiHash());break}}this._contactContainers(a);d.ui.ddmanager&&d.ui.ddmanager.drag(this,a);this._trigger("sort",a,this._uiHash());this.lastPositionAbs=this.positionAbs;return false},_mouseStop:function(a,b){if(a){d.ui.ddmanager&&!this.options.dropBehaviour&&d.ui.ddmanager.drop(this,a);if(this.options.revert){var c=this;b=c.placeholder.offset();c.reverting=true;d(this.helper).animate({left:b.left-this.offset.parent.left-c.margins.left+(this.offsetParent[0]==
+document.body?0:this.offsetParent[0].scrollLeft),top:b.top-this.offset.parent.top-c.margins.top+(this.offsetParent[0]==document.body?0:this.offsetParent[0].scrollTop)},parseInt(this.options.revert,10)||500,function(){c._clear(a)})}else this._clear(a,b);return false}},cancel:function(){var a=this;if(this.dragging){this._mouseUp();this.options.helper=="original"?this.currentItem.css(this._storedCSS).removeClass("ui-sortable-helper"):this.currentItem.show();for(var b=this.containers.length-1;b>=0;b--){this.containers[b]._trigger("deactivate",
+null,a._uiHash(this));if(this.containers[b].containerCache.over){this.containers[b]._trigger("out",null,a._uiHash(this));this.containers[b].containerCache.over=0}}}this.placeholder[0].parentNode&&this.placeholder[0].parentNode.removeChild(this.placeholder[0]);this.options.helper!="original"&&this.helper&&this.helper[0].parentNode&&this.helper.remove();d.extend(this,{helper:null,dragging:false,reverting:false,_noFinalSort:null});this.domPosition.prev?d(this.domPosition.prev).after(this.currentItem):
+d(this.domPosition.parent).prepend(this.currentItem);return this},serialize:function(a){var b=this._getItemsAsjQuery(a&&a.connected),c=[];a=a||{};d(b).each(function(){var e=(d(a.item||this).attr(a.attribute||"id")||"").match(a.expression||/(.+)[-=_](.+)/);if(e)c.push((a.key||e[1]+"[]")+"="+(a.key&&a.expression?e[1]:e[2]))});return c.join("&")},toArray:function(a){var b=this._getItemsAsjQuery(a&&a.connected),c=[];a=a||{};b.each(function(){c.push(d(a.item||this).attr(a.attribute||"id")||"")});return c},
+_intersectsWith:function(a){var b=this.positionAbs.left,c=b+this.helperProportions.width,e=this.positionAbs.top,f=e+this.helperProportions.height,g=a.left,h=g+a.width,i=a.top,k=i+a.height,j=this.offset.click.top,l=this.offset.click.left;j=e+j>i&&e+jg&&b+la[this.floating?"width":"height"]?j:g0?"down":"up")},_getDragHorizontalDirection:function(){var a=
+this.positionAbs.left-this.lastPositionAbs.left;return a!=0&&(a>0?"right":"left")},refresh:function(a){this._refreshItems(a);this.refreshPositions();return this},_connectWith:function(){var a=this.options;return a.connectWith.constructor==String?[a.connectWith]:a.connectWith},_getItemsAsjQuery:function(a){var b=[],c=[],e=this._connectWith();if(e&&a)for(a=e.length-1;a>=0;a--)for(var f=d(e[a]),g=f.length-1;g>=0;g--){var h=d.data(f[g],"sortable");if(h&&h!=this&&!h.options.disabled)c.push([d.isFunction(h.options.items)?
+h.options.items.call(h.element):d(h.options.items,h.element).not(".ui-sortable-helper").not(".ui-sortable-placeholder"),h])}c.push([d.isFunction(this.options.items)?this.options.items.call(this.element,null,{options:this.options,item:this.currentItem}):d(this.options.items,this.element).not(".ui-sortable-helper").not(".ui-sortable-placeholder"),this]);for(a=c.length-1;a>=0;a--)c[a][0].each(function(){b.push(this)});return d(b)},_removeCurrentsFromItems:function(){for(var a=this.currentItem.find(":data(sortable-item)"),
+b=0;b=0;f--)for(var g=d(e[f]),h=g.length-1;h>=0;h--){var i=d.data(g[h],"sortable");if(i&&i!=this&&!i.options.disabled){c.push([d.isFunction(i.options.items)?
+i.options.items.call(i.element[0],a,{item:this.currentItem}):d(i.options.items,i.element),i]);this.containers.push(i)}}for(f=c.length-1;f>=0;f--){a=c[f][1];e=c[f][0];h=0;for(g=e.length;h=0;b--){var c=this.items[b],e=this.options.toleranceElement?d(this.options.toleranceElement,
+c.item):c.item;if(!a){c.width=e.outerWidth();c.height=e.outerHeight()}e=e.offset();c.left=e.left;c.top=e.top}if(this.options.custom&&this.options.custom.refreshContainers)this.options.custom.refreshContainers.call(this);else for(b=this.containers.length-1;b>=0;b--){e=this.containers[b].element.offset();this.containers[b].containerCache.left=e.left;this.containers[b].containerCache.top=e.top;this.containers[b].containerCache.width=this.containers[b].element.outerWidth();this.containers[b].containerCache.height=
+this.containers[b].element.outerHeight()}return this},_createPlaceholder:function(a){var b=a||this,c=b.options;if(!c.placeholder||c.placeholder.constructor==String){var e=c.placeholder;c.placeholder={element:function(){var f=d(document.createElement(b.currentItem[0].nodeName)).addClass(e||b.currentItem[0].className+" ui-sortable-placeholder").removeClass("ui-sortable-helper")[0];if(!e)f.style.visibility="hidden";return f},update:function(f,g){if(!(e&&!c.forcePlaceholderSize)){g.height()||g.height(b.currentItem.innerHeight()-
+parseInt(b.currentItem.css("paddingTop")||0,10)-parseInt(b.currentItem.css("paddingBottom")||0,10));g.width()||g.width(b.currentItem.innerWidth()-parseInt(b.currentItem.css("paddingLeft")||0,10)-parseInt(b.currentItem.css("paddingRight")||0,10))}}}}b.placeholder=d(c.placeholder.element.call(b.element,b.currentItem));b.currentItem.after(b.placeholder);c.placeholder.update(b,b.placeholder)},_contactContainers:function(a){for(var b=null,c=null,e=this.containers.length-1;e>=0;e--)if(!d.ui.contains(this.currentItem[0],
+this.containers[e].element[0]))if(this._intersectsWith(this.containers[e].containerCache)){if(!(b&&d.ui.contains(this.containers[e].element[0],b.element[0]))){b=this.containers[e];c=e}}else if(this.containers[e].containerCache.over){this.containers[e]._trigger("out",a,this._uiHash(this));this.containers[e].containerCache.over=0}if(b)if(this.containers.length===1){this.containers[c]._trigger("over",a,this._uiHash(this));this.containers[c].containerCache.over=1}else if(this.currentContainer!=this.containers[c]){b=
+1E4;e=null;for(var f=this.positionAbs[this.containers[c].floating?"left":"top"],g=this.items.length-1;g>=0;g--)if(d.ui.contains(this.containers[c].element[0],this.items[g].item[0])){var h=this.items[g][this.containers[c].floating?"left":"top"];if(Math.abs(h-f)this.containment[2])f=this.containment[2]+this.offset.click.left;if(a.pageY-this.offset.click.top>this.containment[3])g=this.containment[3]+this.offset.click.top}if(b.grid){g=this.originalPageY+Math.round((g-this.originalPageY)/b.grid[1])*b.grid[1];g=this.containment?!(g-this.offset.click.topthis.containment[3])?g:!(g-this.offset.click.topthis.containment[2])?f:!(f-this.offset.click.left=0;e--)if(d.ui.contains(this.containers[e].element[0],this.currentItem[0])&&!b){c.push(function(f){return function(g){f._trigger("receive",g,this._uiHash(this))}}.call(this,this.containers[e]));c.push(function(f){return function(g){f._trigger("update",
+g,this._uiHash(this))}}.call(this,this.containers[e]))}}for(e=this.containers.length-1;e>=0;e--){b||c.push(function(f){return function(g){f._trigger("deactivate",g,this._uiHash(this))}}.call(this,this.containers[e]));if(this.containers[e].containerCache.over){c.push(function(f){return function(g){f._trigger("out",g,this._uiHash(this))}}.call(this,this.containers[e]));this.containers[e].containerCache.over=0}}this._storedCursor&&d("body").css("cursor",this._storedCursor);this._storedOpacity&&this.helper.css("opacity",
+this._storedOpacity);if(this._storedZIndex)this.helper.css("zIndex",this._storedZIndex=="auto"?"":this._storedZIndex);this.dragging=false;if(this.cancelHelperRemoval){if(!b){this._trigger("beforeStop",a,this._uiHash());for(e=0;e li > :first-child,> :not(li):even",icons:{header:"ui-icon-triangle-1-e",headerSelected:"ui-icon-triangle-1-s"},navigation:false,navigationFilter:function(){return this.href.toLowerCase()==location.href.toLowerCase()}},_create:function(){var a=this.options,b=this;this.running=0;this.element.addClass("ui-accordion ui-widget ui-helper-reset");
+this.element.children("li").addClass("ui-accordion-li-fix");this.headers=this.element.find(a.header).addClass("ui-accordion-header ui-helper-reset ui-state-default ui-corner-all").bind("mouseenter.accordion",function(){c(this).addClass("ui-state-hover")}).bind("mouseleave.accordion",function(){c(this).removeClass("ui-state-hover")}).bind("focus.accordion",function(){c(this).addClass("ui-state-focus")}).bind("blur.accordion",function(){c(this).removeClass("ui-state-focus")});this.headers.next().addClass("ui-accordion-content ui-helper-reset ui-widget-content ui-corner-bottom");
+if(a.navigation){var d=this.element.find("a").filter(a.navigationFilter);if(d.length){var f=d.closest(".ui-accordion-header");this.active=f.length?f:d.closest(".ui-accordion-content").prev()}}this.active=this._findActive(this.active||a.active).toggleClass("ui-state-default").toggleClass("ui-state-active").toggleClass("ui-corner-all").toggleClass("ui-corner-top");this.active.next().addClass("ui-accordion-content-active");this._createIcons();this.resize();this.element.attr("role","tablist");this.headers.attr("role",
+"tab").bind("keydown",function(g){return b._keydown(g)}).next().attr("role","tabpanel");this.headers.not(this.active||"").attr("aria-expanded","false").attr("tabIndex","-1").next().hide();this.active.length?this.active.attr("aria-expanded","true").attr("tabIndex","0"):this.headers.eq(0).attr("tabIndex","0");c.browser.safari||this.headers.find("a").attr("tabIndex","-1");a.event&&this.headers.bind(a.event+".accordion",function(g){b._clickHandler.call(b,g,this);g.preventDefault()})},_createIcons:function(){var a=
+this.options;if(a.icons){c("").addClass("ui-icon "+a.icons.header).prependTo(this.headers);this.active.find(".ui-icon").toggleClass(a.icons.header).toggleClass(a.icons.headerSelected);this.element.addClass("ui-accordion-icons")}},_destroyIcons:function(){this.headers.children(".ui-icon").remove();this.element.removeClass("ui-accordion-icons")},destroy:function(){var a=this.options;this.element.removeClass("ui-accordion ui-widget ui-helper-reset").removeAttr("role").unbind(".accordion").removeData("accordion");
+this.headers.unbind(".accordion").removeClass("ui-accordion-header ui-helper-reset ui-state-default ui-corner-all ui-state-active ui-corner-top").removeAttr("role").removeAttr("aria-expanded").removeAttr("tabIndex");this.headers.find("a").removeAttr("tabIndex");this._destroyIcons();var b=this.headers.next().css("display","").removeAttr("role").removeClass("ui-helper-reset ui-widget-content ui-corner-bottom ui-accordion-content ui-accordion-content-active");if(a.autoHeight||a.fillHeight)b.css("height",
+"");return this},_setOption:function(a,b){c.Widget.prototype._setOption.apply(this,arguments);a=="active"&&this.activate(b);if(a=="icons"){this._destroyIcons();b&&this._createIcons()}},_keydown:function(a){var b=c.ui.keyCode;if(!(this.options.disabled||a.altKey||a.ctrlKey)){var d=this.headers.length,f=this.headers.index(a.target),g=false;switch(a.keyCode){case b.RIGHT:case b.DOWN:g=this.headers[(f+1)%d];break;case b.LEFT:case b.UP:g=this.headers[(f-1+d)%d];break;case b.SPACE:case b.ENTER:this._clickHandler({target:a.target},
+a.target);a.preventDefault()}if(g){c(a.target).attr("tabIndex","-1");c(g).attr("tabIndex","0");g.focus();return false}return true}},resize:function(){var a=this.options,b;if(a.fillSpace){if(c.browser.msie){var d=this.element.parent().css("overflow");this.element.parent().css("overflow","hidden")}b=this.element.parent().height();c.browser.msie&&this.element.parent().css("overflow",d);this.headers.each(function(){b-=c(this).outerHeight(true)});this.headers.next().each(function(){c(this).height(Math.max(0,
+b-c(this).innerHeight()+c(this).height()))}).css("overflow","auto")}else if(a.autoHeight){b=0;this.headers.next().each(function(){b=Math.max(b,c(this).height())}).height(b)}return this},activate:function(a){this.options.active=a;a=this._findActive(a)[0];this._clickHandler({target:a},a);return this},_findActive:function(a){return a?typeof a=="number"?this.headers.filter(":eq("+a+")"):this.headers.not(this.headers.not(a)):a===false?c([]):this.headers.filter(":eq(0)")},_clickHandler:function(a,b){var d=
+this.options;if(!d.disabled)if(a.target){a=c(a.currentTarget||b);b=a[0]==this.active[0];d.active=d.collapsible&&b?false:c(".ui-accordion-header",this.element).index(a);if(!(this.running||!d.collapsible&&b)){this.active.removeClass("ui-state-active ui-corner-top").addClass("ui-state-default ui-corner-all").find(".ui-icon").removeClass(d.icons.headerSelected).addClass(d.icons.header);if(!b){a.removeClass("ui-state-default ui-corner-all").addClass("ui-state-active ui-corner-top").find(".ui-icon").removeClass(d.icons.header).addClass(d.icons.headerSelected);
+a.next().addClass("ui-accordion-content-active")}e=a.next();f=this.active.next();g={options:d,newHeader:b&&d.collapsible?c([]):a,oldHeader:this.active,newContent:b&&d.collapsible?c([]):e,oldContent:f};d=this.headers.index(this.active[0])>this.headers.index(a[0]);this.active=b?c([]):a;this._toggle(e,f,g,b,d)}}else if(d.collapsible){this.active.removeClass("ui-state-active ui-corner-top").addClass("ui-state-default ui-corner-all").find(".ui-icon").removeClass(d.icons.headerSelected).addClass(d.icons.header);
+this.active.next().addClass("ui-accordion-content-active");var f=this.active.next(),g={options:d,newHeader:c([]),oldHeader:d.active,newContent:c([]),oldContent:f},e=this.active=c([]);this._toggle(e,f,g)}},_toggle:function(a,b,d,f,g){var e=this.options,k=this;this.toShow=a;this.toHide=b;this.data=d;var i=function(){if(k)return k._completed.apply(k,arguments)};this._trigger("changestart",null,this.data);this.running=b.size()===0?a.size():b.size();if(e.animated){d={};d=e.collapsible&&f?{toShow:c([]),
+toHide:b,complete:i,down:g,autoHeight:e.autoHeight||e.fillSpace}:{toShow:a,toHide:b,complete:i,down:g,autoHeight:e.autoHeight||e.fillSpace};if(!e.proxied)e.proxied=e.animated;if(!e.proxiedDuration)e.proxiedDuration=e.duration;e.animated=c.isFunction(e.proxied)?e.proxied(d):e.proxied;e.duration=c.isFunction(e.proxiedDuration)?e.proxiedDuration(d):e.proxiedDuration;f=c.ui.accordion.animations;var h=e.duration,j=e.animated;if(j&&!f[j]&&!c.easing[j])j="slide";f[j]||(f[j]=function(l){this.slide(l,{easing:j,
+duration:h||700})});f[j](d)}else{if(e.collapsible&&f)a.toggle();else{b.hide();a.show()}i(true)}b.prev().attr("aria-expanded","false").attr("tabIndex","-1").blur();a.prev().attr("aria-expanded","true").attr("tabIndex","0").focus()},_completed:function(a){var b=this.options;this.running=a?0:--this.running;if(!this.running){b.clearStyle&&this.toShow.add(this.toHide).css({height:"",overflow:""});this.toHide.removeClass("ui-accordion-content-active");this._trigger("change",null,this.data)}}});c.extend(c.ui.accordion,
+{version:"1.8.2",animations:{slide:function(a,b){a=c.extend({easing:"swing",duration:300},a,b);if(a.toHide.size())if(a.toShow.size()){var d=a.toShow.css("overflow"),f=0,g={},e={},k;b=a.toShow;k=b[0].style.width;b.width(parseInt(b.parent().width(),10)-parseInt(b.css("paddingLeft"),10)-parseInt(b.css("paddingRight"),10)-(parseInt(b.css("borderLeftWidth"),10)||0)-(parseInt(b.css("borderRightWidth"),10)||0));c.each(["height","paddingTop","paddingBottom"],function(i,h){e[h]="hide";i=(""+c.css(a.toShow[0],
+h)).match(/^([\d+-.]+)(.*)$/);g[h]={value:i[1],unit:i[2]||"px"}});a.toShow.css({height:0,overflow:"hidden"}).show();a.toHide.filter(":hidden").each(a.complete).end().filter(":visible").animate(e,{step:function(i,h){if(h.prop=="height")f=h.end-h.start===0?0:(h.now-h.start)/(h.end-h.start);a.toShow[0].style[h.prop]=f*g[h.prop].value+g[h.prop].unit},duration:a.duration,easing:a.easing,complete:function(){a.autoHeight||a.toShow.css("height","");a.toShow.css("width",k);a.toShow.css({overflow:d});a.complete()}})}else a.toHide.animate({height:"hide"},
+a);else a.toShow.animate({height:"show"},a)},bounceslide:function(a){this.slide(a,{easing:a.down?"easeOutBounce":"swing",duration:a.down?1E3:200})}}})})(jQuery);
+;/*
+ * jQuery UI Autocomplete 1.8.2
+ *
+ * Copyright (c) 2010 AUTHORS.txt (http://jqueryui.com/about)
+ * Dual licensed under the MIT (MIT-LICENSE.txt)
+ * and GPL (GPL-LICENSE.txt) licenses.
+ *
+ * http://docs.jquery.com/UI/Autocomplete
+ *
+ * Depends:
+ *	jquery.ui.core.js
+ *	jquery.ui.widget.js
+ *	jquery.ui.position.js
+ */
+(function(e){e.widget("ui.autocomplete",{options:{minLength:1,delay:300},_create:function(){var a=this,c=this.element[0].ownerDocument;this.element.addClass("ui-autocomplete-input").attr("autocomplete","off").attr({role:"textbox","aria-autocomplete":"list","aria-haspopup":"true"}).bind("keydown.autocomplete",function(d){var b=e.ui.keyCode;switch(d.keyCode){case b.PAGE_UP:a._move("previousPage",d);break;case b.PAGE_DOWN:a._move("nextPage",d);break;case b.UP:a._move("previous",d);d.preventDefault();
+break;case b.DOWN:a._move("next",d);d.preventDefault();break;case b.ENTER:case b.NUMPAD_ENTER:a.menu.active&&d.preventDefault();case b.TAB:if(!a.menu.active)return;a.menu.select(d);break;case b.ESCAPE:a.element.val(a.term);a.close(d);break;case b.LEFT:case b.RIGHT:case b.SHIFT:case b.CONTROL:case b.ALT:case b.COMMAND:case b.COMMAND_RIGHT:case b.INSERT:case b.CAPS_LOCK:case b.END:case b.HOME:break;default:clearTimeout(a.searching);a.searching=setTimeout(function(){a.search(null,d)},a.options.delay);
+break}}).bind("focus.autocomplete",function(){a.selectedItem=null;a.previous=a.element.val()}).bind("blur.autocomplete",function(d){clearTimeout(a.searching);a.closing=setTimeout(function(){a.close(d);a._change(d)},150)});this._initSource();this.response=function(){return a._response.apply(a,arguments)};this.menu=e("
      ").addClass("ui-autocomplete").appendTo("body",c).mousedown(function(){setTimeout(function(){clearTimeout(a.closing)},13)}).menu({focus:function(d,b){b=b.item.data("item.autocomplete");
+false!==a._trigger("focus",null,{item:b})&&/^key/.test(d.originalEvent.type)&&a.element.val(b.value)},selected:function(d,b){b=b.item.data("item.autocomplete");false!==a._trigger("select",d,{item:b})&&a.element.val(b.value);a.close(d);d=a.previous;if(a.element[0]!==c.activeElement){a.element.focus();a.previous=d}a.selectedItem=b},blur:function(){a.menu.element.is(":visible")&&a.element.val(a.term)}}).zIndex(this.element.zIndex()+1).css({top:0,left:0}).hide().data("menu");e.fn.bgiframe&&this.menu.element.bgiframe()},
+destroy:function(){this.element.removeClass("ui-autocomplete-input").removeAttr("autocomplete").removeAttr("role").removeAttr("aria-autocomplete").removeAttr("aria-haspopup");this.menu.element.remove();e.Widget.prototype.destroy.call(this)},_setOption:function(a){e.Widget.prototype._setOption.apply(this,arguments);a==="source"&&this._initSource()},_initSource:function(){var a,c;if(e.isArray(this.options.source)){a=this.options.source;this.source=function(d,b){b(e.ui.autocomplete.filter(a,d.term))}}else if(typeof this.options.source===
+"string"){c=this.options.source;this.source=function(d,b){e.getJSON(c,d,b)}}else this.source=this.options.source},search:function(a,c){a=a!=null?a:this.element.val();if(a.length").data("item.autocomplete",
+c).append(""+c.label+"").appendTo(a)},_move:function(a,c){if(this.menu.element.is(":visible"))if(this.menu.first()&&/^previous/.test(a)||this.menu.last()&&/^next/.test(a)){this.element.val(this.term);this.menu.deactivate()}else this.menu[a](c);else this.search(null,c)},widget:function(){return this.menu.element}});e.extend(e.ui.autocomplete,{escapeRegex:function(a){return a.replace(/([\^\$\(\)\[\]\{\}\*\.\+\?\|\\])/gi,"\\$1")},filter:function(a,c){var d=new RegExp(e.ui.autocomplete.escapeRegex(c),
+"i");return e.grep(a,function(b){return d.test(b.label||b.value||b)})}})})(jQuery);
+(function(e){e.widget("ui.menu",{_create:function(){var a=this;this.element.addClass("ui-menu ui-widget ui-widget-content ui-corner-all").attr({role:"listbox","aria-activedescendant":"ui-active-menuitem"}).click(function(c){if(e(c.target).closest(".ui-menu-item a").length){c.preventDefault();a.select(c)}});this.refresh()},refresh:function(){var a=this;this.element.children("li:not(.ui-menu-item):has(a)").addClass("ui-menu-item").attr("role","menuitem").children("a").addClass("ui-corner-all").attr("tabindex",
+-1).mouseenter(function(c){a.activate(c,e(this).parent())}).mouseleave(function(){a.deactivate()})},activate:function(a,c){this.deactivate();if(this.hasScroll()){var d=c.offset().top-this.element.offset().top,b=this.element.attr("scrollTop"),f=this.element.height();if(d<0)this.element.attr("scrollTop",b+d);else d>f&&this.element.attr("scrollTop",b+d-f+c.height())}this.active=c.eq(0).children("a").addClass("ui-state-hover").attr("id","ui-active-menuitem").end();this._trigger("focus",a,{item:c})},deactivate:function(){if(this.active){this.active.children("a").removeClass("ui-state-hover").removeAttr("id");
+this._trigger("blur");this.active=null}},next:function(a){this.move("next",".ui-menu-item:first",a)},previous:function(a){this.move("prev",".ui-menu-item:last",a)},first:function(){return this.active&&!this.active.prev().length},last:function(){return this.active&&!this.active.next().length},move:function(a,c,d){if(this.active){a=this.active[a+"All"](".ui-menu-item").eq(0);a.length?this.activate(d,a):this.activate(d,this.element.children(c))}else this.activate(d,this.element.children(c))},nextPage:function(a){if(this.hasScroll())if(!this.active||
+this.last())this.activate(a,this.element.children(":first"));else{var c=this.active.offset().top,d=this.element.height(),b=this.element.children("li").filter(function(){var f=e(this).offset().top-c-d+e(this).height();return f<10&&f>-10});b.length||(b=this.element.children(":last"));this.activate(a,b)}else this.activate(a,this.element.children(!this.active||this.last()?":first":":last"))},previousPage:function(a){if(this.hasScroll())if(!this.active||this.first())this.activate(a,this.element.children(":last"));
+else{var c=this.active.offset().top,d=this.element.height();result=this.element.children("li").filter(function(){var b=e(this).offset().top-c+d-e(this).height();return b<10&&b>-10});result.length||(result=this.element.children(":first"));this.activate(a,result)}else this.activate(a,this.element.children(!this.active||this.first()?":last":":first"))},hasScroll:function(){return this.element.height()      ").addClass("ui-button-text").html(this.options.label).appendTo(b.empty()).text(),d=this.options.icons,e=d.primary&&d.secondary;if(d.primary||d.secondary){b.addClass("ui-button-text-icon"+(e?"s":""));d.primary&&b.prepend("");d.secondary&&b.append("");if(!this.options.text){b.addClass(e?"ui-button-icons-only":"ui-button-icon-only").removeClass("ui-button-text-icons ui-button-text-icon");
+this.hasTitle||b.attr("title",c)}}else b.addClass("ui-button-text-only")}}});a.widget("ui.buttonset",{_create:function(){this.element.addClass("ui-buttonset");this._init()},_init:function(){this.refresh()},_setOption:function(b,c){b==="disabled"&&this.buttons.button("option",b,c);a.Widget.prototype._setOption.apply(this,arguments)},refresh:function(){this.buttons=this.element.find(":button, :submit, :reset, :checkbox, :radio, a, :data(button)").filter(":ui-button").button("refresh").end().not(":ui-button").button().end().map(function(){return a(this).button("widget")[0]}).removeClass("ui-corner-all ui-corner-left ui-corner-right").filter(":first").addClass("ui-corner-left").end().filter(":last").addClass("ui-corner-right").end().end()},
+destroy:function(){this.element.removeClass("ui-buttonset");this.buttons.map(function(){return a(this).button("widget")[0]}).removeClass("ui-corner-left ui-corner-right").end().button("destroy");a.Widget.prototype.destroy.call(this)}})})(jQuery);
+;/*
+ * jQuery UI Dialog 1.8.2
+ *
+ * Copyright (c) 2010 AUTHORS.txt (http://jqueryui.com/about)
+ * Dual licensed under the MIT (MIT-LICENSE.txt)
+ * and GPL (GPL-LICENSE.txt) licenses.
+ *
+ * http://docs.jquery.com/UI/Dialog
+ *
+ * Depends:
+ *	jquery.ui.core.js
+ *	jquery.ui.widget.js
+ *  jquery.ui.button.js
+ *	jquery.ui.draggable.js
+ *	jquery.ui.mouse.js
+ *	jquery.ui.position.js
+ *	jquery.ui.resizable.js
+ */
+(function(c){c.widget("ui.dialog",{options:{autoOpen:true,buttons:{},closeOnEscape:true,closeText:"close",dialogClass:"",draggable:true,hide:null,height:"auto",maxHeight:false,maxWidth:false,minHeight:150,minWidth:150,modal:false,position:"center",resizable:true,show:null,stack:true,title:"",width:300,zIndex:1E3},_create:function(){this.originalTitle=this.element.attr("title");var a=this,b=a.options,d=b.title||a.originalTitle||" ",e=c.ui.dialog.getTitleId(a.element),g=(a.uiDialog=c("
      ")).appendTo(document.body).hide().addClass("ui-dialog ui-widget ui-widget-content ui-corner-all "+
+b.dialogClass).css({zIndex:b.zIndex}).attr("tabIndex",-1).css("outline",0).keydown(function(i){if(b.closeOnEscape&&i.keyCode&&i.keyCode===c.ui.keyCode.ESCAPE){a.close(i);i.preventDefault()}}).attr({role:"dialog","aria-labelledby":e}).mousedown(function(i){a.moveToTop(false,i)});a.element.show().removeAttr("title").addClass("ui-dialog-content ui-widget-content").appendTo(g);var f=(a.uiDialogTitlebar=c("
      ")).addClass("ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix").prependTo(g),
+h=c('').addClass("ui-dialog-titlebar-close ui-corner-all").attr("role","button").hover(function(){h.addClass("ui-state-hover")},function(){h.removeClass("ui-state-hover")}).focus(function(){h.addClass("ui-state-focus")}).blur(function(){h.removeClass("ui-state-focus")}).click(function(i){a.close(i);return false}).appendTo(f);(a.uiDialogTitlebarCloseText=c("")).addClass("ui-icon ui-icon-closethick").text(b.closeText).appendTo(h);c("").addClass("ui-dialog-title").attr("id",
+e).html(d).prependTo(f);if(c.isFunction(b.beforeclose)&&!c.isFunction(b.beforeClose))b.beforeClose=b.beforeclose;f.find("*").add(f).disableSelection();b.draggable&&c.fn.draggable&&a._makeDraggable();b.resizable&&c.fn.resizable&&a._makeResizable();a._createButtons(b.buttons);a._isOpen=false;c.fn.bgiframe&&g.bgiframe()},_init:function(){this.options.autoOpen&&this.open()},destroy:function(){var a=this;a.overlay&&a.overlay.destroy();a.uiDialog.hide();a.element.unbind(".dialog").removeData("dialog").removeClass("ui-dialog-content ui-widget-content").hide().appendTo("body");
+a.uiDialog.remove();a.originalTitle&&a.element.attr("title",a.originalTitle);return a},widget:function(){return this.uiDialog},close:function(a){var b=this,d;if(false!==b._trigger("beforeClose",a)){b.overlay&&b.overlay.destroy();b.uiDialog.unbind("keypress.ui-dialog");b._isOpen=false;if(b.options.hide)b.uiDialog.hide(b.options.hide,function(){b._trigger("close",a)});else{b.uiDialog.hide();b._trigger("close",a)}c.ui.dialog.overlay.resize();if(b.options.modal){d=0;c(".ui-dialog").each(function(){if(this!==
+b.uiDialog[0])d=Math.max(d,c(this).css("z-index"))});c.ui.dialog.maxZ=d}return b}},isOpen:function(){return this._isOpen},moveToTop:function(a,b){var d=this,e=d.options;if(e.modal&&!a||!e.stack&&!e.modal)return d._trigger("focus",b);if(e.zIndex>c.ui.dialog.maxZ)c.ui.dialog.maxZ=e.zIndex;if(d.overlay){c.ui.dialog.maxZ+=1;d.overlay.$el.css("z-index",c.ui.dialog.overlay.maxZ=c.ui.dialog.maxZ)}a={scrollTop:d.element.attr("scrollTop"),scrollLeft:d.element.attr("scrollLeft")};c.ui.dialog.maxZ+=1;d.uiDialog.css("z-index",
+c.ui.dialog.maxZ);d.element.attr(a);d._trigger("focus",b);return d},open:function(){if(!this._isOpen){var a=this,b=a.options,d=a.uiDialog;a.overlay=b.modal?new c.ui.dialog.overlay(a):null;d.next().length&&d.appendTo("body");a._size();a._position(b.position);d.show(b.show);a.moveToTop(true);b.modal&&d.bind("keypress.ui-dialog",function(e){if(e.keyCode===c.ui.keyCode.TAB){var g=c(":tabbable",this),f=g.filter(":first");g=g.filter(":last");if(e.target===g[0]&&!e.shiftKey){f.focus(1);return false}else if(e.target===
+f[0]&&e.shiftKey){g.focus(1);return false}}});c([]).add(d.find(".ui-dialog-content :tabbable:first")).add(d.find(".ui-dialog-buttonpane :tabbable:first")).add(d).filter(":first").focus();a._trigger("open");a._isOpen=true;return a}},_createButtons:function(a){var b=this,d=false,e=c("
      ").addClass("ui-dialog-buttonpane ui-widget-content ui-helper-clearfix");b.uiDialog.find(".ui-dialog-buttonpane").remove();typeof a==="object"&&a!==null&&c.each(a,function(){return!(d=true)});if(d){c.each(a,
+function(g,f){g=c('').text(g).click(function(){f.apply(b.element[0],arguments)}).appendTo(e);c.fn.button&&g.button()});e.appendTo(b.uiDialog)}},_makeDraggable:function(){function a(f){return{position:f.position,offset:f.offset}}var b=this,d=b.options,e=c(document),g;b.uiDialog.draggable({cancel:".ui-dialog-content, .ui-dialog-titlebar-close",handle:".ui-dialog-titlebar",containment:"document",start:function(f,h){g=d.height==="auto"?"auto":c(this).height();c(this).height(c(this).height()).addClass("ui-dialog-dragging");
+b._trigger("dragStart",f,a(h))},drag:function(f,h){b._trigger("drag",f,a(h))},stop:function(f,h){d.position=[h.position.left-e.scrollLeft(),h.position.top-e.scrollTop()];c(this).removeClass("ui-dialog-dragging").height(g);b._trigger("dragStop",f,a(h));c.ui.dialog.overlay.resize()}})},_makeResizable:function(a){function b(f){return{originalPosition:f.originalPosition,originalSize:f.originalSize,position:f.position,size:f.size}}a=a===undefined?this.options.resizable:a;var d=this,e=d.options,g=d.uiDialog.css("position");
+a=typeof a==="string"?a:"n,e,s,w,se,sw,ne,nw";d.uiDialog.resizable({cancel:".ui-dialog-content",containment:"document",alsoResize:d.element,maxWidth:e.maxWidth,maxHeight:e.maxHeight,minWidth:e.minWidth,minHeight:d._minHeight(),handles:a,start:function(f,h){c(this).addClass("ui-dialog-resizing");d._trigger("resizeStart",f,b(h))},resize:function(f,h){d._trigger("resize",f,b(h))},stop:function(f,h){c(this).removeClass("ui-dialog-resizing");e.height=c(this).height();e.width=c(this).width();d._trigger("resizeStop",
+f,b(h));c.ui.dialog.overlay.resize()}}).css("position",g).find(".ui-resizable-se").addClass("ui-icon ui-icon-grip-diagonal-se")},_minHeight:function(){var a=this.options;return a.height==="auto"?a.minHeight:Math.min(a.minHeight,a.height)},_position:function(a){var b=[],d=[0,0];a=a||c.ui.dialog.prototype.options.position;if(typeof a==="string"||typeof a==="object"&&"0"in a){b=a.split?a.split(" "):[a[0],a[1]];if(b.length===1)b[1]=b[0];c.each(["left","top"],function(e,g){if(+b[e]===b[e]){d[e]=b[e];b[e]=
+g}})}else if(typeof a==="object"){if("left"in a){b[0]="left";d[0]=a.left}else if("right"in a){b[0]="right";d[0]=-a.right}if("top"in a){b[1]="top";d[1]=a.top}else if("bottom"in a){b[1]="bottom";d[1]=-a.bottom}}(a=this.uiDialog.is(":visible"))||this.uiDialog.show();this.uiDialog.css({top:0,left:0}).position({my:b.join(" "),at:b.join(" "),offset:d.join(" "),of:window,collision:"fit",using:function(e){var g=c(this).css(e).offset().top;g<0&&c(this).css("top",e.top-g)}});a||this.uiDialog.hide()},_setOption:function(a,
+b){var d=this,e=d.uiDialog,g=e.is(":data(resizable)"),f=false;switch(a){case "beforeclose":a="beforeClose";break;case "buttons":d._createButtons(b);break;case "closeText":d.uiDialogTitlebarCloseText.text(""+b);break;case "dialogClass":e.removeClass(d.options.dialogClass).addClass("ui-dialog ui-widget ui-widget-content ui-corner-all "+b);break;case "disabled":b?e.addClass("ui-dialog-disabled"):e.removeClass("ui-dialog-disabled");break;case "draggable":b?d._makeDraggable():e.draggable("destroy");break;
+case "height":f=true;break;case "maxHeight":g&&e.resizable("option","maxHeight",b);f=true;break;case "maxWidth":g&&e.resizable("option","maxWidth",b);f=true;break;case "minHeight":g&&e.resizable("option","minHeight",b);f=true;break;case "minWidth":g&&e.resizable("option","minWidth",b);f=true;break;case "position":d._position(b);break;case "resizable":g&&!b&&e.resizable("destroy");g&&typeof b==="string"&&e.resizable("option","handles",b);!g&&b!==false&&d._makeResizable(b);break;case "title":c(".ui-dialog-title",
+d.uiDialogTitlebar).html(""+(b||" "));break;case "width":f=true;break}c.Widget.prototype._setOption.apply(d,arguments);f&&d._size()},_size:function(){var a=this.options,b;this.element.css({width:"auto",minHeight:0,height:0});b=this.uiDialog.css({height:"auto",width:a.width}).height();this.element.css(a.height==="auto"?{minHeight:Math.max(a.minHeight-b,0),height:"auto"}:{minHeight:0,height:Math.max(a.height-b,0)}).show();this.uiDialog.is(":data(resizable)")&&this.uiDialog.resizable("option","minHeight",
+this._minHeight())}});c.extend(c.ui.dialog,{version:"1.8.2",uuid:0,maxZ:0,getTitleId:function(a){a=a.attr("id");if(!a){this.uuid+=1;a=this.uuid}return"ui-dialog-title-"+a},overlay:function(a){this.$el=c.ui.dialog.overlay.create(a)}});c.extend(c.ui.dialog.overlay,{instances:[],oldInstances:[],maxZ:0,events:c.map("focus,mousedown,mouseup,keydown,keypress,click".split(","),function(a){return a+".dialog-overlay"}).join(" "),create:function(a){if(this.instances.length===0){setTimeout(function(){c.ui.dialog.overlay.instances.length&&
+c(document).bind(c.ui.dialog.overlay.events,function(d){return c(d.target).zIndex()>=c.ui.dialog.overlay.maxZ})},1);c(document).bind("keydown.dialog-overlay",function(d){if(a.options.closeOnEscape&&d.keyCode&&d.keyCode===c.ui.keyCode.ESCAPE){a.close(d);d.preventDefault()}});c(window).bind("resize.dialog-overlay",c.ui.dialog.overlay.resize)}var b=(this.oldInstances.pop()||c("
      ").addClass("ui-widget-overlay")).appendTo(document.body).css({width:this.width(),height:this.height()});c.fn.bgiframe&&
+b.bgiframe();this.instances.push(b);return b},destroy:function(a){this.oldInstances.push(this.instances.splice(c.inArray(a,this.instances),1)[0]);this.instances.length===0&&c([document,window]).unbind(".dialog-overlay");a.remove();var b=0;c.each(this.instances,function(){b=Math.max(b,this.css("z-index"))});this.maxZ=b},height:function(){var a,b;if(c.browser.msie&&c.browser.version<7){a=Math.max(document.documentElement.scrollHeight,document.body.scrollHeight);b=Math.max(document.documentElement.offsetHeight,
+document.body.offsetHeight);return a
      ");if(!b.values)b.values=[this._valueMin(),this._valueMin()];if(b.values.length&&b.values.length!==2)b.values=[b.values[0],b.values[0]]}else this.range=d("
      ");this.range.appendTo(this.element).addClass("ui-slider-range");if(b.range==="min"||b.range==="max")this.range.addClass("ui-slider-range-"+b.range);this.range.addClass("ui-widget-header")}d(".ui-slider-handle",this.element).length===0&&d("").appendTo(this.element).addClass("ui-slider-handle");
+if(b.values&&b.values.length)for(;d(".ui-slider-handle",this.element).length").appendTo(this.element).addClass("ui-slider-handle");this.handles=d(".ui-slider-handle",this.element).addClass("ui-state-default ui-corner-all");this.handle=this.handles.eq(0);this.handles.add(this.range).filter("a").click(function(c){c.preventDefault()}).hover(function(){b.disabled||d(this).addClass("ui-state-hover")},function(){d(this).removeClass("ui-state-hover")}).focus(function(){if(b.disabled)d(this).blur();
+else{d(".ui-slider .ui-state-focus").removeClass("ui-state-focus");d(this).addClass("ui-state-focus")}}).blur(function(){d(this).removeClass("ui-state-focus")});this.handles.each(function(c){d(this).data("index.ui-slider-handle",c)});this.handles.keydown(function(c){var e=true,f=d(this).data("index.ui-slider-handle"),g,h,i;if(!a.options.disabled){switch(c.keyCode){case d.ui.keyCode.HOME:case d.ui.keyCode.END:case d.ui.keyCode.PAGE_UP:case d.ui.keyCode.PAGE_DOWN:case d.ui.keyCode.UP:case d.ui.keyCode.RIGHT:case d.ui.keyCode.DOWN:case d.ui.keyCode.LEFT:e=
+false;if(!a._keySliding){a._keySliding=true;d(this).addClass("ui-state-active");g=a._start(c,f);if(g===false)return}break}i=a.options.step;g=a.options.values&&a.options.values.length?(h=a.values(f)):(h=a.value());switch(c.keyCode){case d.ui.keyCode.HOME:h=a._valueMin();break;case d.ui.keyCode.END:h=a._valueMax();break;case d.ui.keyCode.PAGE_UP:h=a._trimAlignValue(g+(a._valueMax()-a._valueMin())/5);break;case d.ui.keyCode.PAGE_DOWN:h=a._trimAlignValue(g-(a._valueMax()-a._valueMin())/5);break;case d.ui.keyCode.UP:case d.ui.keyCode.RIGHT:if(g===
+a._valueMax())return;h=a._trimAlignValue(g+i);break;case d.ui.keyCode.DOWN:case d.ui.keyCode.LEFT:if(g===a._valueMin())return;h=a._trimAlignValue(g-i);break}a._slide(c,f,h);return e}}).keyup(function(c){var e=d(this).data("index.ui-slider-handle");if(a._keySliding){a._keySliding=false;a._stop(c,e);a._change(c,e);d(this).removeClass("ui-state-active")}});this._refreshValue();this._animateOff=false},destroy:function(){this.handles.remove();this.range.remove();this.element.removeClass("ui-slider ui-slider-horizontal ui-slider-vertical ui-slider-disabled ui-widget ui-widget-content ui-corner-all").removeData("slider").unbind(".slider");
+this._mouseDestroy();return this},_mouseCapture:function(a){var b=this.options,c,e,f,g,h,i;if(b.disabled)return false;this.elementSize={width:this.element.outerWidth(),height:this.element.outerHeight()};this.elementOffset=this.element.offset();c={x:a.pageX,y:a.pageY};e=this._normValueFromMouse(c);f=this._valueMax()-this._valueMin()+1;h=this;this.handles.each(function(j){var k=Math.abs(e-h.values(j));if(f>k){f=k;g=d(this);i=j}});if(b.range===true&&this.values(1)===b.min){i+=1;g=d(this.handles[i])}if(this._start(a,
+i)===false)return false;this._mouseSliding=true;h._handleIndex=i;g.addClass("ui-state-active").focus();b=g.offset();this._clickOffset=!d(a.target).parents().andSelf().is(".ui-slider-handle")?{left:0,top:0}:{left:a.pageX-b.left-g.width()/2,top:a.pageY-b.top-g.height()/2-(parseInt(g.css("borderTopWidth"),10)||0)-(parseInt(g.css("borderBottomWidth"),10)||0)+(parseInt(g.css("marginTop"),10)||0)};e=this._normValueFromMouse(c);this._slide(a,i,e);return this._animateOff=true},_mouseStart:function(){return true},
+_mouseDrag:function(a){var b=this._normValueFromMouse({x:a.pageX,y:a.pageY});this._slide(a,this._handleIndex,b);return false},_mouseStop:function(a){this.handles.removeClass("ui-state-active");this._mouseSliding=false;this._stop(a,this._handleIndex);this._change(a,this._handleIndex);this._clickOffset=this._handleIndex=null;return this._animateOff=false},_detectOrientation:function(){this.orientation=this.options.orientation==="vertical"?"vertical":"horizontal"},_normValueFromMouse:function(a){var b;
+if(this.orientation==="horizontal"){b=this.elementSize.width;a=a.x-this.elementOffset.left-(this._clickOffset?this._clickOffset.left:0)}else{b=this.elementSize.height;a=a.y-this.elementOffset.top-(this._clickOffset?this._clickOffset.top:0)}b=a/b;if(b>1)b=1;if(b<0)b=0;if(this.orientation==="vertical")b=1-b;a=this._valueMax()-this._valueMin();return this._trimAlignValue(this._valueMin()+b*a)},_start:function(a,b){var c={handle:this.handles[b],value:this.value()};if(this.options.values&&this.options.values.length){c.value=
+this.values(b);c.values=this.values()}return this._trigger("start",a,c)},_slide:function(a,b,c){var e;if(this.options.values&&this.options.values.length){e=this.values(b?0:1);if(this.options.values.length===2&&this.options.range===true&&(b===0&&c>e||b===1&&c1){this.options.values[a]=this._trimAlignValue(b);this._refreshValue();this._change(null,a)}if(arguments.length)if(d.isArray(arguments[0])){c=this.options.values;e=arguments[0];for(f=0;fthis._valueMax())return this._valueMax();var b=this.options.step>0?this.options.step:1,c=a%b;a=a-c;if(Math.abs(c)*2>=b)a+=c>0?b:-b;return parseFloat(a.toFixed(5))},_valueMin:function(){return this.options.min},_valueMax:function(){return this.options.max},_refreshValue:function(){var a=
+this.options.range,b=this.options,c=this,e=!this._animateOff?b.animate:false,f,g={},h,i,j,k;if(this.options.values&&this.options.values.length)this.handles.each(function(l){f=(c.values(l)-c._valueMin())/(c._valueMax()-c._valueMin())*100;g[c.orientation==="horizontal"?"left":"bottom"]=f+"%";d(this).stop(1,1)[e?"animate":"css"](g,b.animate);if(c.options.range===true)if(c.orientation==="horizontal"){if(l===0)c.range.stop(1,1)[e?"animate":"css"]({left:f+"%"},b.animate);if(l===1)c.range[e?"animate":"css"]({width:f-
+h+"%"},{queue:false,duration:b.animate})}else{if(l===0)c.range.stop(1,1)[e?"animate":"css"]({bottom:f+"%"},b.animate);if(l===1)c.range[e?"animate":"css"]({height:f-h+"%"},{queue:false,duration:b.animate})}h=f});else{i=this.value();j=this._valueMin();k=this._valueMax();f=k!==j?(i-j)/(k-j)*100:0;g[c.orientation==="horizontal"?"left":"bottom"]=f+"%";this.handle.stop(1,1)[e?"animate":"css"](g,b.animate);if(a==="min"&&this.orientation==="horizontal")this.range.stop(1,1)[e?"animate":"css"]({width:f+"%"},
+b.animate);if(a==="max"&&this.orientation==="horizontal")this.range[e?"animate":"css"]({width:100-f+"%"},{queue:false,duration:b.animate});if(a==="min"&&this.orientation==="vertical")this.range.stop(1,1)[e?"animate":"css"]({height:f+"%"},b.animate);if(a==="max"&&this.orientation==="vertical")this.range[e?"animate":"css"]({height:100-f+"%"},{queue:false,duration:b.animate})}}});d.extend(d.ui.slider,{version:"1.8.2"})})(jQuery);
+;/*
+ * jQuery UI Tabs 1.8.2
+ *
+ * Copyright (c) 2010 AUTHORS.txt (http://jqueryui.com/about)
+ * Dual licensed under the MIT (MIT-LICENSE.txt)
+ * and GPL (GPL-LICENSE.txt) licenses.
+ *
+ * http://docs.jquery.com/UI/Tabs
+ *
+ * Depends:
+ *	jquery.ui.core.js
+ *	jquery.ui.widget.js
+ */
+(function(d){function s(){return++u}function v(){return++w}var u=0,w=0;d.widget("ui.tabs",{options:{add:null,ajaxOptions:null,cache:false,cookie:null,collapsible:false,disable:null,disabled:[],enable:null,event:"click",fx:null,idPrefix:"ui-tabs-",load:null,panelTemplate:"
      ",remove:null,select:null,show:null,spinner:"Loading…",tabTemplate:'
    • #{label}
      • '},_create:function(){this._tabify(true)},_setOption:function(c,e){if(c=="selected")this.options.collapsible&&
+e==this.options.selected||this.select(e);else{this.options[c]=e;this._tabify()}},_tabId:function(c){return c.title&&c.title.replace(/\s/g,"_").replace(/[^A-Za-z0-9\-_:\.]/g,"")||this.options.idPrefix+s()},_sanitizeSelector:function(c){return c.replace(/:/g,"\\:")},_cookie:function(){var c=this.cookie||(this.cookie=this.options.cookie.name||"ui-tabs-"+v());return d.cookie.apply(null,[c].concat(d.makeArray(arguments)))},_ui:function(c,e){return{tab:c,panel:e,index:this.anchors.index(c)}},_cleanup:function(){this.lis.filter(".ui-state-processing").removeClass("ui-state-processing").find("span:data(label.tabs)").each(function(){var c=
+d(this);c.html(c.data("label.tabs")).removeData("label.tabs")})},_tabify:function(c){function e(g,f){g.css({display:""});!d.support.opacity&&f.opacity&&g[0].style.removeAttribute("filter")}this.list=this.element.find("ol,ul").eq(0);this.lis=d("li:has(a[href])",this.list);this.anchors=this.lis.map(function(){return d("a",this)[0]});this.panels=d([]);var a=this,b=this.options,h=/^#.+/;this.anchors.each(function(g,f){var j=d(f).attr("href"),l=j.split("#")[0],p;if(l&&(l===location.toString().split("#")[0]||
+(p=d("base")[0])&&l===p.href)){j=f.hash;f.href=j}if(h.test(j))a.panels=a.panels.add(a._sanitizeSelector(j));else if(j!="#"){d.data(f,"href.tabs",j);d.data(f,"load.tabs",j.replace(/#.*$/,""));j=a._tabId(f);f.href="#"+j;f=d("#"+j);if(!f.length){f=d(b.panelTemplate).attr("id",j).addClass("ui-tabs-panel ui-widget-content ui-corner-bottom").insertAfter(a.panels[g-1]||a.list);f.data("destroy.tabs",true)}a.panels=a.panels.add(f)}else b.disabled.push(g)});if(c){this.element.addClass("ui-tabs ui-widget ui-widget-content ui-corner-all");
+this.list.addClass("ui-tabs-nav ui-helper-reset ui-helper-clearfix ui-widget-header ui-corner-all");this.lis.addClass("ui-state-default ui-corner-top");this.panels.addClass("ui-tabs-panel ui-widget-content ui-corner-bottom");if(b.selected===undefined){location.hash&&this.anchors.each(function(g,f){if(f.hash==location.hash){b.selected=g;return false}});if(typeof b.selected!="number"&&b.cookie)b.selected=parseInt(a._cookie(),10);if(typeof b.selected!="number"&&this.lis.filter(".ui-tabs-selected").length)b.selected=
+this.lis.index(this.lis.filter(".ui-tabs-selected"));b.selected=b.selected||(this.lis.length?0:-1)}else if(b.selected===null)b.selected=-1;b.selected=b.selected>=0&&this.anchors[b.selected]||b.selected<0?b.selected:0;b.disabled=d.unique(b.disabled.concat(d.map(this.lis.filter(".ui-state-disabled"),function(g){return a.lis.index(g)}))).sort();d.inArray(b.selected,b.disabled)!=-1&&b.disabled.splice(d.inArray(b.selected,b.disabled),1);this.panels.addClass("ui-tabs-hide");this.lis.removeClass("ui-tabs-selected ui-state-active");
+if(b.selected>=0&&this.anchors.length){this.panels.eq(b.selected).removeClass("ui-tabs-hide");this.lis.eq(b.selected).addClass("ui-tabs-selected ui-state-active");a.element.queue("tabs",function(){a._trigger("show",null,a._ui(a.anchors[b.selected],a.panels[b.selected]))});this.load(b.selected)}d(window).bind("unload",function(){a.lis.add(a.anchors).unbind(".tabs");a.lis=a.anchors=a.panels=null})}else b.selected=this.lis.index(this.lis.filter(".ui-tabs-selected"));this.element[b.collapsible?"addClass":
+"removeClass"]("ui-tabs-collapsible");b.cookie&&this._cookie(b.selected,b.cookie);c=0;for(var i;i=this.lis[c];c++)d(i)[d.inArray(c,b.disabled)!=-1&&!d(i).hasClass("ui-tabs-selected")?"addClass":"removeClass"]("ui-state-disabled");b.cache===false&&this.anchors.removeData("cache.tabs");this.lis.add(this.anchors).unbind(".tabs");if(b.event!="mouseover"){var k=function(g,f){f.is(":not(.ui-state-disabled)")&&f.addClass("ui-state-"+g)},n=function(g,f){f.removeClass("ui-state-"+g)};this.lis.bind("mouseover.tabs",
+function(){k("hover",d(this))});this.lis.bind("mouseout.tabs",function(){n("hover",d(this))});this.anchors.bind("focus.tabs",function(){k("focus",d(this).closest("li"))});this.anchors.bind("blur.tabs",function(){n("focus",d(this).closest("li"))})}var m,o;if(b.fx)if(d.isArray(b.fx)){m=b.fx[0];o=b.fx[1]}else m=o=b.fx;var q=o?function(g,f){d(g).closest("li").addClass("ui-tabs-selected ui-state-active");f.hide().removeClass("ui-tabs-hide").animate(o,o.duration||"normal",function(){e(f,o);a._trigger("show",
+null,a._ui(g,f[0]))})}:function(g,f){d(g).closest("li").addClass("ui-tabs-selected ui-state-active");f.removeClass("ui-tabs-hide");a._trigger("show",null,a._ui(g,f[0]))},r=m?function(g,f){f.animate(m,m.duration||"normal",function(){a.lis.removeClass("ui-tabs-selected ui-state-active");f.addClass("ui-tabs-hide");e(f,m);a.element.dequeue("tabs")})}:function(g,f){a.lis.removeClass("ui-tabs-selected ui-state-active");f.addClass("ui-tabs-hide");a.element.dequeue("tabs")};this.anchors.bind(b.event+".tabs",
+function(){var g=this,f=d(this).closest("li"),j=a.panels.filter(":not(.ui-tabs-hide)"),l=d(a._sanitizeSelector(this.hash));if(f.hasClass("ui-tabs-selected")&&!b.collapsible||f.hasClass("ui-state-disabled")||f.hasClass("ui-state-processing")||a._trigger("select",null,a._ui(this,l[0]))===false){this.blur();return false}b.selected=a.anchors.index(this);a.abort();if(b.collapsible)if(f.hasClass("ui-tabs-selected")){b.selected=-1;b.cookie&&a._cookie(b.selected,b.cookie);a.element.queue("tabs",function(){r(g,
+j)}).dequeue("tabs");this.blur();return false}else if(!j.length){b.cookie&&a._cookie(b.selected,b.cookie);a.element.queue("tabs",function(){q(g,l)});a.load(a.anchors.index(this));this.blur();return false}b.cookie&&a._cookie(b.selected,b.cookie);if(l.length){j.length&&a.element.queue("tabs",function(){r(g,j)});a.element.queue("tabs",function(){q(g,l)});a.load(a.anchors.index(this))}else throw"jQuery UI Tabs: Mismatching fragment identifier.";d.browser.msie&&this.blur()});this.anchors.bind("click.tabs",
+function(){return false})},destroy:function(){var c=this.options;this.abort();this.element.unbind(".tabs").removeClass("ui-tabs ui-widget ui-widget-content ui-corner-all ui-tabs-collapsible").removeData("tabs");this.list.removeClass("ui-tabs-nav ui-helper-reset ui-helper-clearfix ui-widget-header ui-corner-all");this.anchors.each(function(){var e=d.data(this,"href.tabs");if(e)this.href=e;var a=d(this).unbind(".tabs");d.each(["href","load","cache"],function(b,h){a.removeData(h+".tabs")})});this.lis.unbind(".tabs").add(this.panels).each(function(){d.data(this,
+"destroy.tabs")?d(this).remove():d(this).removeClass("ui-state-default ui-corner-top ui-tabs-selected ui-state-active ui-state-hover ui-state-focus ui-state-disabled ui-tabs-panel ui-widget-content ui-corner-bottom ui-tabs-hide")});c.cookie&&this._cookie(null,c.cookie);return this},add:function(c,e,a){if(a===undefined)a=this.anchors.length;var b=this,h=this.options;e=d(h.tabTemplate.replace(/#\{href\}/g,c).replace(/#\{label\}/g,e));c=!c.indexOf("#")?c.replace("#",""):this._tabId(d("a",e)[0]);e.addClass("ui-state-default ui-corner-top").data("destroy.tabs",
+true);var i=d("#"+c);i.length||(i=d(h.panelTemplate).attr("id",c).data("destroy.tabs",true));i.addClass("ui-tabs-panel ui-widget-content ui-corner-bottom ui-tabs-hide");if(a>=this.lis.length){e.appendTo(this.list);i.appendTo(this.list[0].parentNode)}else{e.insertBefore(this.lis[a]);i.insertBefore(this.panels[a])}h.disabled=d.map(h.disabled,function(k){return k>=a?++k:k});this._tabify();if(this.anchors.length==1){h.selected=0;e.addClass("ui-tabs-selected ui-state-active");i.removeClass("ui-tabs-hide");
+this.element.queue("tabs",function(){b._trigger("show",null,b._ui(b.anchors[0],b.panels[0]))});this.load(0)}this._trigger("add",null,this._ui(this.anchors[a],this.panels[a]));return this},remove:function(c){var e=this.options,a=this.lis.eq(c).remove(),b=this.panels.eq(c).remove();if(a.hasClass("ui-tabs-selected")&&this.anchors.length>1)this.select(c+(c+1=c?--h:h});this._tabify();this._trigger("remove",
+null,this._ui(a.find("a")[0],b[0]));return this},enable:function(c){var e=this.options;if(d.inArray(c,e.disabled)!=-1){this.lis.eq(c).removeClass("ui-state-disabled");e.disabled=d.grep(e.disabled,function(a){return a!=c});this._trigger("enable",null,this._ui(this.anchors[c],this.panels[c]));return this}},disable:function(c){var e=this.options;if(c!=e.selected){this.lis.eq(c).addClass("ui-state-disabled");e.disabled.push(c);e.disabled.sort();this._trigger("disable",null,this._ui(this.anchors[c],this.panels[c]))}return this},
+select:function(c){if(typeof c=="string")c=this.anchors.index(this.anchors.filter("[href$="+c+"]"));else if(c===null)c=-1;if(c==-1&&this.options.collapsible)c=this.options.selected;this.anchors.eq(c).trigger(this.options.event+".tabs");return this},load:function(c){var e=this,a=this.options,b=this.anchors.eq(c)[0],h=d.data(b,"load.tabs");this.abort();if(!h||this.element.queue("tabs").length!==0&&d.data(b,"cache.tabs"))this.element.dequeue("tabs");else{this.lis.eq(c).addClass("ui-state-processing");
+if(a.spinner){var i=d("span",b);i.data("label.tabs",i.html()).html(a.spinner)}this.xhr=d.ajax(d.extend({},a.ajaxOptions,{url:h,success:function(k,n){d(e._sanitizeSelector(b.hash)).html(k);e._cleanup();a.cache&&d.data(b,"cache.tabs",true);e._trigger("load",null,e._ui(e.anchors[c],e.panels[c]));try{a.ajaxOptions.success(k,n)}catch(m){}},error:function(k,n){e._cleanup();e._trigger("load",null,e._ui(e.anchors[c],e.panels[c]));try{a.ajaxOptions.error(k,n,c,b)}catch(m){}}}));e.element.dequeue("tabs");return this}},
+abort:function(){this.element.queue([]);this.panels.stop(false,true);this.element.queue("tabs",this.element.queue("tabs").splice(-2,2));if(this.xhr){this.xhr.abort();delete this.xhr}this._cleanup();return this},url:function(c,e){this.anchors.eq(c).removeData("cache.tabs").data("load.tabs",e);return this},length:function(){return this.anchors.length}});d.extend(d.ui.tabs,{version:"1.8.2"});d.extend(d.ui.tabs.prototype,{rotation:null,rotate:function(c,e){var a=this,b=this.options,h=a._rotate||(a._rotate=
+function(i){clearTimeout(a.rotation);a.rotation=setTimeout(function(){var k=b.selected;a.select(++k
      ')}function E(a,b){d.extend(a,
+b);for(var c in b)if(b[c]==null||b[c]==undefined)a[c]=b[c];return a}d.extend(d.ui,{datepicker:{version:"1.8.2"}});var y=(new Date).getTime();d.extend(J.prototype,{markerClassName:"hasDatepicker",log:function(){this.debug&&console.log.apply("",arguments)},_widgetDatepicker:function(){return this.dpDiv},setDefaults:function(a){E(this._defaults,a||{});return this},_attachDatepicker:function(a,b){var c=null;for(var e in this._defaults){var f=a.getAttribute("date:"+e);if(f){c=c||{};try{c[e]=eval(f)}catch(h){c[e]=
+f}}}e=a.nodeName.toLowerCase();f=e=="div"||e=="span";if(!a.id){this.uuid+=1;a.id="dp"+this.uuid}var i=this._newInst(d(a),f);i.settings=d.extend({},b||{},c||{});if(e=="input")this._connectDatepicker(a,i);else f&&this._inlineDatepicker(a,i)},_newInst:function(a,b){return{id:a[0].id.replace(/([^A-Za-z0-9_])/g,"\\\\$1"),input:a,selectedDay:0,selectedMonth:0,selectedYear:0,drawMonth:0,drawYear:0,inline:b,dpDiv:!b?this.dpDiv:d('
      ')}},
+_connectDatepicker:function(a,b){var c=d(a);b.append=d([]);b.trigger=d([]);if(!c.hasClass(this.markerClassName)){this._attachments(c,b);c.addClass(this.markerClassName).keydown(this._doKeyDown).keypress(this._doKeyPress).keyup(this._doKeyUp).bind("setData.datepicker",function(e,f,h){b.settings[f]=h}).bind("getData.datepicker",function(e,f){return this._get(b,f)});this._autoSize(b);d.data(a,"datepicker",b)}},_attachments:function(a,b){var c=this._get(b,"appendText"),e=this._get(b,"isRTL");b.append&&
+b.append.remove();if(c){b.append=d(''+c+"");a[e?"before":"after"](b.append)}a.unbind("focus",this._showDatepicker);b.trigger&&b.trigger.remove();c=this._get(b,"showOn");if(c=="focus"||c=="both")a.focus(this._showDatepicker);if(c=="button"||c=="both"){c=this._get(b,"buttonText");var f=this._get(b,"buttonImage");b.trigger=d(this._get(b,"buttonImageOnly")?d("").addClass(this._triggerClass).attr({src:f,alt:c,title:c}):d('').addClass(this._triggerClass).html(f==
+""?c:d("").attr({src:f,alt:c,title:c})));a[e?"before":"after"](b.trigger);b.trigger.click(function(){d.datepicker._datepickerShowing&&d.datepicker._lastInput==a[0]?d.datepicker._hideDatepicker():d.datepicker._showDatepicker(a[0]);return false})}},_autoSize:function(a){if(this._get(a,"autoSize")&&!a.inline){var b=new Date(2009,11,20),c=this._get(a,"dateFormat");if(c.match(/[DM]/)){var e=function(f){for(var h=0,i=0,g=0;gh){h=f[g].length;i=g}return i};b.setMonth(e(this._get(a,
+c.match(/MM/)?"monthNames":"monthNamesShort")));b.setDate(e(this._get(a,c.match(/DD/)?"dayNames":"dayNamesShort"))+20-b.getDay())}a.input.attr("size",this._formatDate(a,b).length)}},_inlineDatepicker:function(a,b){var c=d(a);if(!c.hasClass(this.markerClassName)){c.addClass(this.markerClassName).append(b.dpDiv).bind("setData.datepicker",function(e,f,h){b.settings[f]=h}).bind("getData.datepicker",function(e,f){return this._get(b,f)});d.data(a,"datepicker",b);this._setDate(b,this._getDefaultDate(b),
+true);this._updateDatepicker(b);this._updateAlternate(b)}},_dialogDatepicker:function(a,b,c,e,f){a=this._dialogInst;if(!a){this.uuid+=1;this._dialogInput=d('');this._dialogInput.keydown(this._doKeyDown);d("body").append(this._dialogInput);a=this._dialogInst=this._newInst(this._dialogInput,false);a.settings={};d.data(this._dialogInput[0],"datepicker",a)}E(a.settings,e||{});b=b&&b.constructor==
+Date?this._formatDate(a,b):b;this._dialogInput.val(b);this._pos=f?f.length?f:[f.pageX,f.pageY]:null;if(!this._pos)this._pos=[document.documentElement.clientWidth/2-100+(document.documentElement.scrollLeft||document.body.scrollLeft),document.documentElement.clientHeight/2-150+(document.documentElement.scrollTop||document.body.scrollTop)];this._dialogInput.css("left",this._pos[0]+20+"px").css("top",this._pos[1]+"px");a.settings.onSelect=c;this._inDialog=true;this.dpDiv.addClass(this._dialogClass);this._showDatepicker(this._dialogInput[0]);
+d.blockUI&&d.blockUI(this.dpDiv);d.data(this._dialogInput[0],"datepicker",a);return this},_destroyDatepicker:function(a){var b=d(a),c=d.data(a,"datepicker");if(b.hasClass(this.markerClassName)){var e=a.nodeName.toLowerCase();d.removeData(a,"datepicker");if(e=="input"){c.append.remove();c.trigger.remove();b.removeClass(this.markerClassName).unbind("focus",this._showDatepicker).unbind("keydown",this._doKeyDown).unbind("keypress",this._doKeyPress).unbind("keyup",this._doKeyUp)}else if(e=="div"||e=="span")b.removeClass(this.markerClassName).empty()}},
+_enableDatepicker:function(a){var b=d(a),c=d.data(a,"datepicker");if(b.hasClass(this.markerClassName)){var e=a.nodeName.toLowerCase();if(e=="input"){a.disabled=false;c.trigger.filter("button").each(function(){this.disabled=false}).end().filter("img").css({opacity:"1.0",cursor:""})}else if(e=="div"||e=="span")b.children("."+this._inlineClass).children().removeClass("ui-state-disabled");this._disabledInputs=d.map(this._disabledInputs,function(f){return f==a?null:f})}},_disableDatepicker:function(a){var b=
+d(a),c=d.data(a,"datepicker");if(b.hasClass(this.markerClassName)){var e=a.nodeName.toLowerCase();if(e=="input"){a.disabled=true;c.trigger.filter("button").each(function(){this.disabled=true}).end().filter("img").css({opacity:"0.5",cursor:"default"})}else if(e=="div"||e=="span")b.children("."+this._inlineClass).children().addClass("ui-state-disabled");this._disabledInputs=d.map(this._disabledInputs,function(f){return f==a?null:f});this._disabledInputs[this._disabledInputs.length]=a}},_isDisabledDatepicker:function(a){if(!a)return false;
+for(var b=0;b-1}},_doKeyUp:function(a){a=d.datepicker._getInst(a.target);if(a.input.val()!=a.lastVal)try{if(d.datepicker.parseDate(d.datepicker._get(a,"dateFormat"),a.input?a.input.val():null,d.datepicker._getFormatConfig(a))){d.datepicker._setDateFromField(a);d.datepicker._updateAlternate(a);d.datepicker._updateDatepicker(a)}}catch(b){d.datepicker.log(b)}return true},_showDatepicker:function(a){a=a.target||
+a;if(a.nodeName.toLowerCase()!="input")a=d("input",a.parentNode)[0];if(!(d.datepicker._isDisabledDatepicker(a)||d.datepicker._lastInput==a)){var b=d.datepicker._getInst(a);d.datepicker._curInst&&d.datepicker._curInst!=b&&d.datepicker._curInst.dpDiv.stop(true,true);var c=d.datepicker._get(b,"beforeShow");E(b.settings,c?c.apply(a,[a,b]):{});b.lastVal=null;d.datepicker._lastInput=a;d.datepicker._setDateFromField(b);if(d.datepicker._inDialog)a.value="";if(!d.datepicker._pos){d.datepicker._pos=d.datepicker._findPos(a);
+d.datepicker._pos[1]+=a.offsetHeight}var e=false;d(a).parents().each(function(){e|=d(this).css("position")=="fixed";return!e});if(e&&d.browser.opera){d.datepicker._pos[0]-=document.documentElement.scrollLeft;d.datepicker._pos[1]-=document.documentElement.scrollTop}c={left:d.datepicker._pos[0],top:d.datepicker._pos[1]};d.datepicker._pos=null;b.dpDiv.css({position:"absolute",display:"block",top:"-1000px"});d.datepicker._updateDatepicker(b);c=d.datepicker._checkOffset(b,c,e);b.dpDiv.css({position:d.datepicker._inDialog&&
+d.blockUI?"static":e?"fixed":"absolute",display:"none",left:c.left+"px",top:c.top+"px"});if(!b.inline){c=d.datepicker._get(b,"showAnim");var f=d.datepicker._get(b,"duration"),h=function(){d.datepicker._datepickerShowing=true;var i=d.datepicker._getBorders(b.dpDiv);b.dpDiv.find("iframe.ui-datepicker-cover").css({left:-i[0],top:-i[1],width:b.dpDiv.outerWidth(),height:b.dpDiv.outerHeight()})};b.dpDiv.zIndex(d(a).zIndex()+1);d.effects&&d.effects[c]?b.dpDiv.show(c,d.datepicker._get(b,"showOptions"),f,
+h):b.dpDiv[c||"show"](c?f:null,h);if(!c||!f)h();b.input.is(":visible")&&!b.input.is(":disabled")&&b.input.focus();d.datepicker._curInst=b}}},_updateDatepicker:function(a){var b=this,c=d.datepicker._getBorders(a.dpDiv);a.dpDiv.empty().append(this._generateHTML(a)).find("iframe.ui-datepicker-cover").css({left:-c[0],top:-c[1],width:a.dpDiv.outerWidth(),height:a.dpDiv.outerHeight()}).end().find("button, .ui-datepicker-prev, .ui-datepicker-next, .ui-datepicker-calendar td a").bind("mouseout",function(){d(this).removeClass("ui-state-hover");
+this.className.indexOf("ui-datepicker-prev")!=-1&&d(this).removeClass("ui-datepicker-prev-hover");this.className.indexOf("ui-datepicker-next")!=-1&&d(this).removeClass("ui-datepicker-next-hover")}).bind("mouseover",function(){if(!b._isDisabledDatepicker(a.inline?a.dpDiv.parent()[0]:a.input[0])){d(this).parents(".ui-datepicker-calendar").find("a").removeClass("ui-state-hover");d(this).addClass("ui-state-hover");this.className.indexOf("ui-datepicker-prev")!=-1&&d(this).addClass("ui-datepicker-prev-hover");
+this.className.indexOf("ui-datepicker-next")!=-1&&d(this).addClass("ui-datepicker-next-hover")}}).end().find("."+this._dayOverClass+" a").trigger("mouseover").end();c=this._getNumberOfMonths(a);var e=c[1];e>1?a.dpDiv.addClass("ui-datepicker-multi-"+e).css("width",17*e+"em"):a.dpDiv.removeClass("ui-datepicker-multi-2 ui-datepicker-multi-3 ui-datepicker-multi-4").width("");a.dpDiv[(c[0]!=1||c[1]!=1?"add":"remove")+"Class"]("ui-datepicker-multi");a.dpDiv[(this._get(a,"isRTL")?"add":"remove")+"Class"]("ui-datepicker-rtl");
+a==d.datepicker._curInst&&d.datepicker._datepickerShowing&&a.input&&a.input.is(":visible")&&!a.input.is(":disabled")&&a.input.focus()},_getBorders:function(a){var b=function(c){return{thin:1,medium:2,thick:3}[c]||c};return[parseFloat(b(a.css("border-left-width"))),parseFloat(b(a.css("border-top-width")))]},_checkOffset:function(a,b,c){var e=a.dpDiv.outerWidth(),f=a.dpDiv.outerHeight(),h=a.input?a.input.outerWidth():0,i=a.input?a.input.outerHeight():0,g=document.documentElement.clientWidth+d(document).scrollLeft(),
+k=document.documentElement.clientHeight+d(document).scrollTop();b.left-=this._get(a,"isRTL")?e-h:0;b.left-=c&&b.left==a.input.offset().left?d(document).scrollLeft():0;b.top-=c&&b.top==a.input.offset().top+i?d(document).scrollTop():0;b.left-=Math.min(b.left,b.left+e>g&&g>e?Math.abs(b.left+e-g):0);b.top-=Math.min(b.top,b.top+f>k&&k>f?Math.abs(f+i):0);return b},_findPos:function(a){for(var b=this._get(this._getInst(a),"isRTL");a&&(a.type=="hidden"||a.nodeType!=1);)a=a[b?"previousSibling":"nextSibling"];
+a=d(a).offset();return[a.left,a.top]},_hideDatepicker:function(a){var b=this._curInst;if(!(!b||a&&b!=d.data(a,"datepicker")))if(this._datepickerShowing){a=this._get(b,"showAnim");var c=this._get(b,"duration"),e=function(){d.datepicker._tidyDialog(b);this._curInst=null};d.effects&&d.effects[a]?b.dpDiv.hide(a,d.datepicker._get(b,"showOptions"),c,e):b.dpDiv[a=="slideDown"?"slideUp":a=="fadeIn"?"fadeOut":"hide"](a?c:null,e);a||e();if(a=this._get(b,"onClose"))a.apply(b.input?b.input[0]:null,[b.input?b.input.val():
+"",b]);this._datepickerShowing=false;this._lastInput=null;if(this._inDialog){this._dialogInput.css({position:"absolute",left:"0",top:"-100px"});if(d.blockUI){d.unblockUI();d("body").append(this.dpDiv)}}this._inDialog=false}},_tidyDialog:function(a){a.dpDiv.removeClass(this._dialogClass).unbind(".ui-datepicker-calendar")},_checkExternalClick:function(a){if(d.datepicker._curInst){a=d(a.target);a[0].id!=d.datepicker._mainDivId&&a.parents("#"+d.datepicker._mainDivId).length==0&&!a.hasClass(d.datepicker.markerClassName)&&
+!a.hasClass(d.datepicker._triggerClass)&&d.datepicker._datepickerShowing&&!(d.datepicker._inDialog&&d.blockUI)&&d.datepicker._hideDatepicker()}},_adjustDate:function(a,b,c){a=d(a);var e=this._getInst(a[0]);if(!this._isDisabledDatepicker(a[0])){this._adjustInstDate(e,b+(c=="M"?this._get(e,"showCurrentAtPos"):0),c);this._updateDatepicker(e)}},_gotoToday:function(a){a=d(a);var b=this._getInst(a[0]);if(this._get(b,"gotoCurrent")&&b.currentDay){b.selectedDay=b.currentDay;b.drawMonth=b.selectedMonth=b.currentMonth;
+b.drawYear=b.selectedYear=b.currentYear}else{var c=new Date;b.selectedDay=c.getDate();b.drawMonth=b.selectedMonth=c.getMonth();b.drawYear=b.selectedYear=c.getFullYear()}this._notifyChange(b);this._adjustDate(a)},_selectMonthYear:function(a,b,c){a=d(a);var e=this._getInst(a[0]);e._selectingMonthYear=false;e["selected"+(c=="M"?"Month":"Year")]=e["draw"+(c=="M"?"Month":"Year")]=parseInt(b.options[b.selectedIndex].value,10);this._notifyChange(e);this._adjustDate(a)},_clickMonthYear:function(a){a=this._getInst(d(a)[0]);
+a.input&&a._selectingMonthYear&&!d.browser.msie&&a.input.focus();a._selectingMonthYear=!a._selectingMonthYear},_selectDay:function(a,b,c,e){var f=d(a);if(!(d(e).hasClass(this._unselectableClass)||this._isDisabledDatepicker(f[0]))){f=this._getInst(f[0]);f.selectedDay=f.currentDay=d("a",e).html();f.selectedMonth=f.currentMonth=b;f.selectedYear=f.currentYear=c;this._selectDate(a,this._formatDate(f,f.currentDay,f.currentMonth,f.currentYear))}},_clearDate:function(a){a=d(a);this._getInst(a[0]);this._selectDate(a,
+"")},_selectDate:function(a,b){a=this._getInst(d(a)[0]);b=b!=null?b:this._formatDate(a);a.input&&a.input.val(b);this._updateAlternate(a);var c=this._get(a,"onSelect");if(c)c.apply(a.input?a.input[0]:null,[b,a]);else a.input&&a.input.trigger("change");if(a.inline)this._updateDatepicker(a);else{this._hideDatepicker();this._lastInput=a.input[0];typeof a.input[0]!="object"&&a.input.focus();this._lastInput=null}},_updateAlternate:function(a){var b=this._get(a,"altField");if(b){var c=this._get(a,"altFormat")||
+this._get(a,"dateFormat"),e=this._getDate(a),f=this.formatDate(c,e,this._getFormatConfig(a));d(b).each(function(){d(this).val(f)})}},noWeekends:function(a){a=a.getDay();return[a>0&&a<6,""]},iso8601Week:function(a){a=new Date(a.getTime());a.setDate(a.getDate()+4-(a.getDay()||7));var b=a.getTime();a.setMonth(0);a.setDate(1);return Math.floor(Math.round((b-a)/864E5)/7)+1},parseDate:function(a,b,c){if(a==null||b==null)throw"Invalid arguments";b=typeof b=="object"?b.toString():b+"";if(b=="")return null;
+for(var e=(c?c.shortYearCutoff:null)||this._defaults.shortYearCutoff,f=(c?c.dayNamesShort:null)||this._defaults.dayNamesShort,h=(c?c.dayNames:null)||this._defaults.dayNames,i=(c?c.monthNamesShort:null)||this._defaults.monthNamesShort,g=(c?c.monthNames:null)||this._defaults.monthNames,k=c=-1,l=-1,u=-1,j=false,o=function(p){(p=z+1-1){k=1;l=u;do{e=this._getDaysInMonth(c,k-1);if(l<=e)break;k++;l-=e}while(1)}v=this._daylightSavingAdjust(new Date(c,
+k-1,l));if(v.getFullYear()!=c||v.getMonth()+1!=k||v.getDate()!=l)throw"Invalid date";return v},ATOM:"yy-mm-dd",COOKIE:"D, dd M yy",ISO_8601:"yy-mm-dd",RFC_822:"D, d M y",RFC_850:"DD, dd-M-y",RFC_1036:"D, d M y",RFC_1123:"D, d M yy",RFC_2822:"D, d M yy",RSS:"D, d M y",TICKS:"!",TIMESTAMP:"@",W3C:"yy-mm-dd",_ticksTo1970:(718685+Math.floor(492.5)-Math.floor(19.7)+Math.floor(4.925))*24*60*60*1E7,formatDate:function(a,b,c){if(!b)return"";var e=(c?c.dayNamesShort:null)||this._defaults.dayNamesShort,f=(c?
+c.dayNames:null)||this._defaults.dayNames,h=(c?c.monthNamesShort:null)||this._defaults.monthNamesShort;c=(c?c.monthNames:null)||this._defaults.monthNames;var i=function(o){(o=j+112?a.getHours()+2:0);return a},_setDate:function(a,b,c){var e=!b,f=a.selectedMonth,h=a.selectedYear;b=this._restrictMinMax(a,this._determineDate(a,b,new Date));a.selectedDay=a.currentDay=b.getDate();a.drawMonth=a.selectedMonth=a.currentMonth=b.getMonth();a.drawYear=a.selectedYear=a.currentYear=b.getFullYear();if((f!=a.selectedMonth||h!=a.selectedYear)&&!c)this._notifyChange(a);this._adjustInstDate(a);if(a.input)a.input.val(e?"":this._formatDate(a))},_getDate:function(a){return!a.currentYear||
+a.input&&a.input.val()==""?null:this._daylightSavingAdjust(new Date(a.currentYear,a.currentMonth,a.currentDay))},_generateHTML:function(a){var b=new Date;b=this._daylightSavingAdjust(new Date(b.getFullYear(),b.getMonth(),b.getDate()));var c=this._get(a,"isRTL"),e=this._get(a,"showButtonPanel"),f=this._get(a,"hideIfNoPrevNext"),h=this._get(a,"navigationAsDateFormat"),i=this._getNumberOfMonths(a),g=this._get(a,"showCurrentAtPos"),k=this._get(a,"stepMonths"),l=i[0]!=1||i[1]!=1,u=this._daylightSavingAdjust(!a.currentDay?
+new Date(9999,9,9):new Date(a.currentYear,a.currentMonth,a.currentDay)),j=this._getMinMaxDate(a,"min"),o=this._getMinMaxDate(a,"max");g=a.drawMonth-g;var m=a.drawYear;if(g<0){g+=12;m--}if(o){var n=this._daylightSavingAdjust(new Date(o.getFullYear(),o.getMonth()-i[0]*i[1]+1,o.getDate()));for(n=j&&nn;){g--;if(g<0){g=11;m--}}}a.drawMonth=g;a.drawYear=m;n=this._get(a,"prevText");n=!h?n:this.formatDate(n,this._daylightSavingAdjust(new Date(m,g-k,1)),this._getFormatConfig(a));
+n=this._canAdjustMonth(a,-1,m,g)?''+n+"":f?"":''+n+"";var r=this._get(a,"nextText");r=!h?r:this.formatDate(r,this._daylightSavingAdjust(new Date(m,
+g+k,1)),this._getFormatConfig(a));f=this._canAdjustMonth(a,+1,m,g)?''+r+"":f?"":''+r+"";k=this._get(a,"currentText");r=this._get(a,"gotoCurrent")&&
+a.currentDay?u:b;k=!h?k:this.formatDate(k,r,this._getFormatConfig(a));h=!a.inline?'":"";e=e?'
      '+(c?h:"")+(this._isInRange(a,r)?'":"")+(c?"":h)+"
      ":"";h=parseInt(this._get(a,"firstDay"),10);h=isNaN(h)?0:h;k=this._get(a,"showWeek");r=this._get(a,"dayNames");this._get(a,"dayNamesShort");var s=this._get(a,"dayNamesMin"),z=this._get(a,"monthNames"),v=this._get(a,"monthNamesShort"),p=this._get(a,"beforeShowDay"),w=this._get(a,"showOtherMonths"),G=this._get(a,"selectOtherMonths");this._get(a,"calculateWeek");for(var K=this._getDefaultDate(a),H="",C=0;C1)switch(D){case 0:x+=" ui-datepicker-group-first";t=" ui-corner-"+(c?"right":"left");break;case i[1]-1:x+=" ui-datepicker-group-last";t=" ui-corner-"+(c?"left":"right");break;default:x+=" ui-datepicker-group-middle";t="";break}x+='">'}x+='
      '+(/all|left/.test(t)&&C==0?c?
+f:n:"")+(/all|right/.test(t)&&C==0?c?n:f:"")+this._generateMonthYearHeader(a,g,m,j,o,C>0||D>0,z,v)+'
      ';var A=k?'":"";for(t=0;t<7;t++){var q=(t+h)%7;A+="=5?' class="ui-datepicker-week-end"':"")+'>'+s[q]+""}x+=A+"";A=this._getDaysInMonth(m,g);if(m==a.selectedYear&&g==a.selectedMonth)a.selectedDay=Math.min(a.selectedDay,
+A);t=(this._getFirstDayOfMonth(m,g)-h+7)%7;A=l?6:Math.ceil((t+A)/7);q=this._daylightSavingAdjust(new Date(m,g,1-t));for(var N=0;N";var O=!k?"":'";for(t=0;t<7;t++){var F=p?p.apply(a.input?a.input[0]:null,[q]):[true,""],B=q.getMonth()!=g,I=B&&!G||!F[0]||j&&qo;O+='";q.setDate(q.getDate()+1);q=this._daylightSavingAdjust(q)}x+=O+""}g++;if(g>11){g=0;m++}x+="
      '+this._get(a,"weekHeader")+"
      '+this._get(a,"calculateWeek")(q)+""+(B&&!w?" ":I?''+q.getDate()+
+"":''+q.getDate()+"")+"
      "+(l?"
      "+(i[0]>0&&D==i[1]-1?'
      ':""):"");L+=x}H+=L}H+=e+(d.browser.msie&&parseInt(d.browser.version,10)<7&&!a.inline?'':
+"");a._keyEvent=false;return H},_generateMonthYearHeader:function(a,b,c,e,f,h,i,g){var k=this._get(a,"changeMonth"),l=this._get(a,"changeYear"),u=this._get(a,"showMonthAfterYear"),j='
      ',o="";if(h||!k)o+=''+i[b]+"";else{i=e&&e.getFullYear()==c;var m=f&&f.getFullYear()==c;o+='"}u||(j+=o+(h||!(k&&l)?" ":""));if(h||!l)j+=''+c+"";else{g=this._get(a,"yearRange").split(":");var r=(new Date).getFullYear();i=function(s){s=s.match(/c[+-].*/)?c+parseInt(s.substring(1),10):s.match(/[+-].*/)?r+parseInt(s,10):parseInt(s,10);return isNaN(s)?r:s};b=i(g[0]);g=Math.max(b,
+i(g[1]||""));b=e?Math.max(b,e.getFullYear()):b;g=f?Math.min(g,f.getFullYear()):g;for(j+='"}j+=this._get(a,"yearSuffix");if(u)j+=(h||!(k&&l)?" ":"")+o;j+="
      ";return j},_adjustInstDate:function(a,b,c){var e=
+a.drawYear+(c=="Y"?b:0),f=a.drawMonth+(c=="M"?b:0);b=Math.min(a.selectedDay,this._getDaysInMonth(e,f))+(c=="D"?b:0);e=this._restrictMinMax(a,this._daylightSavingAdjust(new Date(e,f,b)));a.selectedDay=e.getDate();a.drawMonth=a.selectedMonth=e.getMonth();a.drawYear=a.selectedYear=e.getFullYear();if(c=="M"||c=="Y")this._notifyChange(a)},_restrictMinMax:function(a,b){var c=this._getMinMaxDate(a,"min");a=this._getMinMaxDate(a,"max");b=c&&ba?a:b},_notifyChange:function(a){var b=this._get(a,
+"onChangeMonthYear");if(b)b.apply(a.input?a.input[0]:null,[a.selectedYear,a.selectedMonth+1,a])},_getNumberOfMonths:function(a){a=this._get(a,"numberOfMonths");return a==null?[1,1]:typeof a=="number"?[1,a]:a},_getMinMaxDate:function(a,b){return this._determineDate(a,this._get(a,b+"Date"),null)},_getDaysInMonth:function(a,b){return 32-(new Date(a,b,32)).getDate()},_getFirstDayOfMonth:function(a,b){return(new Date(a,b,1)).getDay()},_canAdjustMonth:function(a,b,c,e){var f=this._getNumberOfMonths(a);
+c=this._daylightSavingAdjust(new Date(c,e+(b<0?b:f[0]*f[1]),1));b<0&&c.setDate(this._getDaysInMonth(c.getFullYear(),c.getMonth()));return this._isInRange(a,c)},_isInRange:function(a,b){var c=this._getMinMaxDate(a,"min");a=this._getMinMaxDate(a,"max");return(!c||b.getTime()>=c.getTime())&&(!a||b.getTime()<=a.getTime())},_getFormatConfig:function(a){var b=this._get(a,"shortYearCutoff");b=typeof b!="string"?b:(new Date).getFullYear()%100+parseInt(b,10);return{shortYearCutoff:b,dayNamesShort:this._get(a,
+"dayNamesShort"),dayNames:this._get(a,"dayNames"),monthNamesShort:this._get(a,"monthNamesShort"),monthNames:this._get(a,"monthNames")}},_formatDate:function(a,b,c,e){if(!b){a.currentDay=a.selectedDay;a.currentMonth=a.selectedMonth;a.currentYear=a.selectedYear}b=b?typeof b=="object"?b:this._daylightSavingAdjust(new Date(e,c,b)):this._daylightSavingAdjust(new Date(a.currentYear,a.currentMonth,a.currentDay));return this.formatDate(this._get(a,"dateFormat"),b,this._getFormatConfig(a))}});d.fn.datepicker=
+function(a){if(!d.datepicker.initialized){d(document).mousedown(d.datepicker._checkExternalClick).find("body").append(d.datepicker.dpDiv);d.datepicker.initialized=true}var b=Array.prototype.slice.call(arguments,1);if(typeof a=="string"&&(a=="isDisabled"||a=="getDate"||a=="widget"))return d.datepicker["_"+a+"Datepicker"].apply(d.datepicker,[this[0]].concat(b));if(a=="option"&&arguments.length==2&&typeof arguments[1]=="string")return d.datepicker["_"+a+"Datepicker"].apply(d.datepicker,[this[0]].concat(b));
+return this.each(function(){typeof a=="string"?d.datepicker["_"+a+"Datepicker"].apply(d.datepicker,[this].concat(b)):d.datepicker._attachDatepicker(this,a)})};d.datepicker=new J;d.datepicker.initialized=false;d.datepicker.uuid=(new Date).getTime();d.datepicker.version="1.8.2";window["DP_jQuery_"+y]=d})(jQuery);
+;/*
+ * jQuery UI Progressbar 1.8.2
+ *
+ * Copyright (c) 2010 AUTHORS.txt (http://jqueryui.com/about)
+ * Dual licensed under the MIT (MIT-LICENSE.txt)
+ * and GPL (GPL-LICENSE.txt) licenses.
+ *
+ * http://docs.jquery.com/UI/Progressbar
+ *
+ * Depends:
+ *   jquery.ui.core.js
+ *   jquery.ui.widget.js
+ */
+(function(b){b.widget("ui.progressbar",{options:{value:0},_create:function(){this.element.addClass("ui-progressbar ui-widget ui-widget-content ui-corner-all").attr({role:"progressbar","aria-valuemin":this._valueMin(),"aria-valuemax":this._valueMax(),"aria-valuenow":this._value()});this.valueDiv=b("
      ").appendTo(this.element);this._refreshValue()},destroy:function(){this.element.removeClass("ui-progressbar ui-widget ui-widget-content ui-corner-all").removeAttr("role").removeAttr("aria-valuemin").removeAttr("aria-valuemax").removeAttr("aria-valuenow");
+this.valueDiv.remove();b.Widget.prototype.destroy.apply(this,arguments)},value:function(a){if(a===undefined)return this._value();this._setOption("value",a);return this},_setOption:function(a,c){switch(a){case "value":this.options.value=c;this._refreshValue();this._trigger("change");break}b.Widget.prototype._setOption.apply(this,arguments)},_value:function(){var a=this.options.value;if(typeof a!=="number")a=0;if(athis._valueMax())a=this._valueMax();return a},
+_valueMin:function(){return 0},_valueMax:function(){return 100},_refreshValue:function(){var a=this.value();this.valueDiv[a===this._valueMax()?"addClass":"removeClass"]("ui-corner-right").width(a+"%");this.element.attr("aria-valuenow",a)}});b.extend(b.ui.progressbar,{version:"1.8.2"})})(jQuery);
+;/*
+ * jQuery UI Effects 1.8.2
+ *
+ * Copyright (c) 2010 AUTHORS.txt (http://jqueryui.com/about)
+ * Dual licensed under the MIT (MIT-LICENSE.txt)
+ * and GPL (GPL-LICENSE.txt) licenses.
+ *
+ * http://docs.jquery.com/UI/Effects/
+ */
+jQuery.effects||function(f){function k(c){var a;if(c&&c.constructor==Array&&c.length==3)return c;if(a=/rgb\(\s*([0-9]{1,3})\s*,\s*([0-9]{1,3})\s*,\s*([0-9]{1,3})\s*\)/.exec(c))return[parseInt(a[1],10),parseInt(a[2],10),parseInt(a[3],10)];if(a=/rgb\(\s*([0-9]+(?:\.[0-9]+)?)\%\s*,\s*([0-9]+(?:\.[0-9]+)?)\%\s*,\s*([0-9]+(?:\.[0-9]+)?)\%\s*\)/.exec(c))return[parseFloat(a[1])*2.55,parseFloat(a[2])*2.55,parseFloat(a[3])*2.55];if(a=/#([a-fA-F0-9]{2})([a-fA-F0-9]{2})([a-fA-F0-9]{2})/.exec(c))return[parseInt(a[1],
+16),parseInt(a[2],16),parseInt(a[3],16)];if(a=/#([a-fA-F0-9])([a-fA-F0-9])([a-fA-F0-9])/.exec(c))return[parseInt(a[1]+a[1],16),parseInt(a[2]+a[2],16),parseInt(a[3]+a[3],16)];if(/rgba\(0, 0, 0, 0\)/.exec(c))return l.transparent;return l[f.trim(c).toLowerCase()]}function q(c,a){var b;do{b=f.curCSS(c,a);if(b!=""&&b!="transparent"||f.nodeName(c,"body"))break;a="backgroundColor"}while(c=c.parentNode);return k(b)}function m(){var c=document.defaultView?document.defaultView.getComputedStyle(this,null):this.currentStyle,
+a={},b,d;if(c&&c.length&&c[0]&&c[c[0]])for(var e=c.length;e--;){b=c[e];if(typeof c[b]=="string"){d=b.replace(/\-(\w)/g,function(g,h){return h.toUpperCase()});a[d]=c[b]}}else for(b in c)if(typeof c[b]==="string")a[b]=c[b];return a}function n(c){var a,b;for(a in c){b=c[a];if(b==null||f.isFunction(b)||a in r||/scrollbar/.test(a)||!/color/i.test(a)&&isNaN(parseFloat(b)))delete c[a]}return c}function s(c,a){var b={_:0},d;for(d in a)if(c[d]!=a[d])b[d]=a[d];return b}function j(c,a,b,d){if(typeof c=="object"){d=
+a;b=null;a=c;c=a.effect}if(f.isFunction(a)){d=a;b=null;a={}}if(f.isFunction(b)){d=b;b=null}if(typeof a=="number"||f.fx.speeds[a]){d=b;b=a;a={}}a=a||{};b=b||a.duration;b=f.fx.off?0:typeof b=="number"?b:f.fx.speeds[b]||f.fx.speeds._default;d=d||a.complete;return[c,a,b,d]}f.effects={};f.each(["backgroundColor","borderBottomColor","borderLeftColor","borderRightColor","borderTopColor","color","outlineColor"],function(c,a){f.fx.step[a]=function(b){if(!b.colorInit){b.start=q(b.elem,a);b.end=k(b.end);b.colorInit=
+true}b.elem.style[a]="rgb("+Math.max(Math.min(parseInt(b.pos*(b.end[0]-b.start[0])+b.start[0],10),255),0)+","+Math.max(Math.min(parseInt(b.pos*(b.end[1]-b.start[1])+b.start[1],10),255),0)+","+Math.max(Math.min(parseInt(b.pos*(b.end[2]-b.start[2])+b.start[2],10),255),0)+")"}});var l={aqua:[0,255,255],azure:[240,255,255],beige:[245,245,220],black:[0,0,0],blue:[0,0,255],brown:[165,42,42],cyan:[0,255,255],darkblue:[0,0,139],darkcyan:[0,139,139],darkgrey:[169,169,169],darkgreen:[0,100,0],darkkhaki:[189,
+183,107],darkmagenta:[139,0,139],darkolivegreen:[85,107,47],darkorange:[255,140,0],darkorchid:[153,50,204],darkred:[139,0,0],darksalmon:[233,150,122],darkviolet:[148,0,211],fuchsia:[255,0,255],gold:[255,215,0],green:[0,128,0],indigo:[75,0,130],khaki:[240,230,140],lightblue:[173,216,230],lightcyan:[224,255,255],lightgreen:[144,238,144],lightgrey:[211,211,211],lightpink:[255,182,193],lightyellow:[255,255,224],lime:[0,255,0],magenta:[255,0,255],maroon:[128,0,0],navy:[0,0,128],olive:[128,128,0],orange:[255,
+165,0],pink:[255,192,203],purple:[128,0,128],violet:[128,0,128],red:[255,0,0],silver:[192,192,192],white:[255,255,255],yellow:[255,255,0],transparent:[255,255,255]},o=["add","remove","toggle"],r={border:1,borderBottom:1,borderColor:1,borderLeft:1,borderRight:1,borderTop:1,borderWidth:1,margin:1,padding:1};f.effects.animateClass=function(c,a,b,d){if(f.isFunction(b)){d=b;b=null}return this.each(function(){var e=f(this),g=e.attr("style")||" ",h=n(m.call(this)),p,t=e.attr("className");f.each(o,function(u,
+i){c[i]&&e[i+"Class"](c[i])});p=n(m.call(this));e.attr("className",t);e.animate(s(h,p),a,b,function(){f.each(o,function(u,i){c[i]&&e[i+"Class"](c[i])});if(typeof e.attr("style")=="object"){e.attr("style").cssText="";e.attr("style").cssText=g}else e.attr("style",g);d&&d.apply(this,arguments)})})};f.fn.extend({_addClass:f.fn.addClass,addClass:function(c,a,b,d){return a?f.effects.animateClass.apply(this,[{add:c},a,b,d]):this._addClass(c)},_removeClass:f.fn.removeClass,removeClass:function(c,a,b,d){return a?
+f.effects.animateClass.apply(this,[{remove:c},a,b,d]):this._removeClass(c)},_toggleClass:f.fn.toggleClass,toggleClass:function(c,a,b,d,e){return typeof a=="boolean"||a===undefined?b?f.effects.animateClass.apply(this,[a?{add:c}:{remove:c},b,d,e]):this._toggleClass(c,a):f.effects.animateClass.apply(this,[{toggle:c},a,b,d])},switchClass:function(c,a,b,d,e){return f.effects.animateClass.apply(this,[{add:a,remove:c},b,d,e])}});f.extend(f.effects,{version:"1.8.2",save:function(c,a){for(var b=0;b
      ").addClass("ui-effects-wrapper").css({fontSize:"100%",background:"transparent",border:"none",margin:0,padding:0});c.wrap(b);b=c.parent();if(c.css("position")=="static"){b.css({position:"relative"});c.css({position:"relative"})}else{f.extend(a,{position:c.css("position"),zIndex:c.css("z-index")});f.each(["top","left","bottom","right"],function(d,e){a[e]=c.css(e);if(isNaN(parseInt(a[e],10)))a[e]="auto"});
+c.css({position:"relative",top:0,left:0})}return b.css(a).show()},removeWrapper:function(c){if(c.parent().is(".ui-effects-wrapper"))return c.parent().replaceWith(c);return c},setTransition:function(c,a,b,d){d=d||{};f.each(a,function(e,g){unit=c.cssUnit(g);if(unit[0]>0)d[g]=unit[0]*b+unit[1]});return d}});f.fn.extend({effect:function(c){var a=j.apply(this,arguments);a={options:a[1],duration:a[2],callback:a[3]};var b=f.effects[c];return b&&!f.fx.off?b.call(this,a):this},_show:f.fn.show,show:function(c){if(!c||
+typeof c=="number"||f.fx.speeds[c])return this._show.apply(this,arguments);else{var a=j.apply(this,arguments);a[1].mode="show";return this.effect.apply(this,a)}},_hide:f.fn.hide,hide:function(c){if(!c||typeof c=="number"||f.fx.speeds[c])return this._hide.apply(this,arguments);else{var a=j.apply(this,arguments);a[1].mode="hide";return this.effect.apply(this,a)}},__toggle:f.fn.toggle,toggle:function(c){if(!c||typeof c=="number"||f.fx.speeds[c]||typeof c=="boolean"||f.isFunction(c))return this.__toggle.apply(this,
+arguments);else{var a=j.apply(this,arguments);a[1].mode="toggle";return this.effect.apply(this,a)}},cssUnit:function(c){var a=this.css(c),b=[];f.each(["em","px","%","pt"],function(d,e){if(a.indexOf(e)>0)b=[parseFloat(a),e]});return b}});f.easing.jswing=f.easing.swing;f.extend(f.easing,{def:"easeOutQuad",swing:function(c,a,b,d,e){return f.easing[f.easing.def](c,a,b,d,e)},easeInQuad:function(c,a,b,d,e){return d*(a/=e)*a+b},easeOutQuad:function(c,a,b,d,e){return-d*(a/=e)*(a-2)+b},easeInOutQuad:function(c,
+a,b,d,e){if((a/=e/2)<1)return d/2*a*a+b;return-d/2*(--a*(a-2)-1)+b},easeInCubic:function(c,a,b,d,e){return d*(a/=e)*a*a+b},easeOutCubic:function(c,a,b,d,e){return d*((a=a/e-1)*a*a+1)+b},easeInOutCubic:function(c,a,b,d,e){if((a/=e/2)<1)return d/2*a*a*a+b;return d/2*((a-=2)*a*a+2)+b},easeInQuart:function(c,a,b,d,e){return d*(a/=e)*a*a*a+b},easeOutQuart:function(c,a,b,d,e){return-d*((a=a/e-1)*a*a*a-1)+b},easeInOutQuart:function(c,a,b,d,e){if((a/=e/2)<1)return d/2*a*a*a*a+b;return-d/2*((a-=2)*a*a*a-2)+
+b},easeInQuint:function(c,a,b,d,e){return d*(a/=e)*a*a*a*a+b},easeOutQuint:function(c,a,b,d,e){return d*((a=a/e-1)*a*a*a*a+1)+b},easeInOutQuint:function(c,a,b,d,e){if((a/=e/2)<1)return d/2*a*a*a*a*a+b;return d/2*((a-=2)*a*a*a*a+2)+b},easeInSine:function(c,a,b,d,e){return-d*Math.cos(a/e*(Math.PI/2))+d+b},easeOutSine:function(c,a,b,d,e){return d*Math.sin(a/e*(Math.PI/2))+b},easeInOutSine:function(c,a,b,d,e){return-d/2*(Math.cos(Math.PI*a/e)-1)+b},easeInExpo:function(c,a,b,d,e){return a==0?b:d*Math.pow(2,
+10*(a/e-1))+b},easeOutExpo:function(c,a,b,d,e){return a==e?b+d:d*(-Math.pow(2,-10*a/e)+1)+b},easeInOutExpo:function(c,a,b,d,e){if(a==0)return b;if(a==e)return b+d;if((a/=e/2)<1)return d/2*Math.pow(2,10*(a-1))+b;return d/2*(-Math.pow(2,-10*--a)+2)+b},easeInCirc:function(c,a,b,d,e){return-d*(Math.sqrt(1-(a/=e)*a)-1)+b},easeOutCirc:function(c,a,b,d,e){return d*Math.sqrt(1-(a=a/e-1)*a)+b},easeInOutCirc:function(c,a,b,d,e){if((a/=e/2)<1)return-d/2*(Math.sqrt(1-a*a)-1)+b;return d/2*(Math.sqrt(1-(a-=2)*
+a)+1)+b},easeInElastic:function(c,a,b,d,e){c=1.70158;var g=0,h=d;if(a==0)return b;if((a/=e)==1)return b+d;g||(g=e*0.3);if(h
      ").css({position:"absolute",visibility:"visible",left:-f*(h/d),top:-e*(i/c)}).parent().addClass("ui-effects-explode").css({position:"absolute",overflow:"hidden",width:h/d,height:i/c,left:g.left+f*(h/d)+(a.options.mode=="show"?(f-Math.floor(d/2))*(h/d):0),top:g.top+e*(i/c)+(a.options.mode=="show"?(e-Math.floor(c/2))*(i/c):0),opacity:a.options.mode=="show"?0:1}).animate({left:g.left+f*(h/d)+(a.options.mode=="show"?0:(f-Math.floor(d/2))*(h/d)),top:g.top+ +e*(i/c)+(a.options.mode=="show"?0:(e-Math.floor(c/2))*(i/c)),opacity:a.options.mode=="show"?1:0},a.duration||500);setTimeout(function(){a.options.mode=="show"?b.css({visibility:"visible"}):b.css({visibility:"visible"}).hide();a.callback&&a.callback.apply(b[0]);b.dequeue();j("div.ui-effects-explode").remove()},a.duration||500)})}})(jQuery); +;/* + * jQuery UI Effects Fold 1.8.2 + * + * Copyright (c) 2010 AUTHORS.txt (http://jqueryui.com/about) + * Dual licensed under the MIT (MIT-LICENSE.txt) + * and GPL (GPL-LICENSE.txt) licenses. + * + * http://docs.jquery.com/UI/Effects/Fold + * + * Depends: + * jquery.effects.core.js + */ +(function(c){c.effects.fold=function(a){return this.queue(function(){var b=c(this),j=["position","top","left"],d=c.effects.setMode(b,a.options.mode||"hide"),g=a.options.size||15,h=!!a.options.horizFirst,k=a.duration?a.duration/2:c.fx.speeds._default/2;c.effects.save(b,j);b.show();var e=c.effects.createWrapper(b).css({overflow:"hidden"}),f=d=="show"!=h,l=f?["width","height"]:["height","width"];f=f?[e.width(),e.height()]:[e.height(),e.width()];var i=/([0-9]+)%/.exec(g);if(i)g=parseInt(i[1],10)/100* +f[d=="hide"?0:1];if(d=="show")e.css(h?{height:0,width:g}:{height:g,width:0});h={};i={};h[l[0]]=d=="show"?f[0]:g;i[l[1]]=d=="show"?f[1]:0;e.animate(h,k,a.options.easing).animate(i,k,a.options.easing,function(){d=="hide"&&b.hide();c.effects.restore(b,j);c.effects.removeWrapper(b);a.callback&&a.callback.apply(b[0],arguments);b.dequeue()})})}})(jQuery); +;/* + * jQuery UI Effects Highlight 1.8.2 + * + * Copyright (c) 2010 AUTHORS.txt (http://jqueryui.com/about) + * Dual licensed under the MIT (MIT-LICENSE.txt) + * and GPL (GPL-LICENSE.txt) licenses. + * + * http://docs.jquery.com/UI/Effects/Highlight + * + * Depends: + * jquery.effects.core.js + */ +(function(b){b.effects.highlight=function(c){return this.queue(function(){var a=b(this),e=["backgroundImage","backgroundColor","opacity"],d=b.effects.setMode(a,c.options.mode||"show"),f={backgroundColor:a.css("backgroundColor")};if(d=="hide")f.opacity=0;b.effects.save(a,e);a.show().css({backgroundImage:"none",backgroundColor:c.options.color||"#ffff99"}).animate(f,{queue:false,duration:c.duration,easing:c.options.easing,complete:function(){d=="hide"&&a.hide();b.effects.restore(a,e);d=="show"&&!b.support.opacity&& +this.style.removeAttribute("filter");c.callback&&c.callback.apply(this,arguments);a.dequeue()}})})}})(jQuery); +;/* + * jQuery UI Effects Pulsate 1.8.2 + * + * Copyright (c) 2010 AUTHORS.txt (http://jqueryui.com/about) + * Dual licensed under the MIT (MIT-LICENSE.txt) + * and GPL (GPL-LICENSE.txt) licenses. + * + * http://docs.jquery.com/UI/Effects/Pulsate + * + * Depends: + * jquery.effects.core.js + */ +(function(d){d.effects.pulsate=function(a){return this.queue(function(){var b=d(this),c=d.effects.setMode(b,a.options.mode||"show");times=(a.options.times||5)*2-1;duration=a.duration?a.duration/2:d.fx.speeds._default/2;isVisible=b.is(":visible");animateTo=0;if(!isVisible){b.css("opacity",0).show();animateTo=1}if(c=="hide"&&isVisible||c=="show"&&!isVisible)times--;for(c=0;c
      ').appendTo(document.body).addClass(a.options.className).css({top:d.top,left:d.left,height:b.innerHeight(),width:b.innerWidth(),position:"absolute"}).animate(c,a.duration,a.options.easing,function(){f.remove();a.callback&&a.callback.apply(b[0],arguments); +b.dequeue()})})}})(jQuery); +; \ No newline at end of file diff --git a/ui/new/scripts/jquery.cookies.js b/ui/new/scripts/jquery.cookies.js new file mode 100755 index 00000000000..6036754e815 --- /dev/null +++ b/ui/new/scripts/jquery.cookies.js @@ -0,0 +1,96 @@ +/** + * Cookie plugin + * + * Copyright (c) 2006 Klaus Hartl (stilbuero.de) + * Dual licensed under the MIT and GPL licenses: + * http://www.opensource.org/licenses/mit-license.php + * http://www.gnu.org/licenses/gpl.html + * + */ + +/** + * Create a cookie with the given name and value and other optional parameters. + * + * @example $.cookie('the_cookie', 'the_value'); + * @desc Set the value of a cookie. + * @example $.cookie('the_cookie', 'the_value', { expires: 7, path: '/', domain: 'jquery.com', secure: true }); + * @desc Create a cookie with all available options. + * @example $.cookie('the_cookie', 'the_value'); + * @desc Create a session cookie. + * @example $.cookie('the_cookie', null); + * @desc Delete a cookie by passing null as value. Keep in mind that you have to use the same path and domain + * used when the cookie was set. + * + * @param String name The name of the cookie. + * @param String value The value of the cookie. + * @param Object options An object literal containing key/value pairs to provide optional cookie attributes. + * @option Number|Date expires Either an integer specifying the expiration date from now on in days or a Date object. + * If a negative value is specified (e.g. a date in the past), the cookie will be deleted. + * If set to null or omitted, the cookie will be a session cookie and will not be retained + * when the the browser exits. + * @option String path The value of the path atribute of the cookie (default: path of page that created the cookie). + * @option String domain The value of the domain attribute of the cookie (default: domain of page that created the cookie). + * @option Boolean secure If true, the secure attribute of the cookie will be set and the cookie transmission will + * require a secure protocol (like HTTPS). + * @type undefined + * + * @name $.cookie + * @cat Plugins/Cookie + * @author Klaus Hartl/klaus.hartl@stilbuero.de + */ + +/** + * Get the value of a cookie with the given name. + * + * @example $.cookie('the_cookie'); + * @desc Get the value of a cookie. + * + * @param String name The name of the cookie. + * @return The value of the cookie. + * @type String + * + * @name $.cookie + * @cat Plugins/Cookie + * @author Klaus Hartl/klaus.hartl@stilbuero.de + */ +jQuery.cookie = function(name, value, options) { + if (typeof value != 'undefined') { // name and value given, set cookie + options = options || {}; + if (value === null) { + value = ''; + options.expires = -1; + } + var expires = ''; + if (options.expires && (typeof options.expires == 'number' || options.expires.toUTCString)) { + var date; + if (typeof options.expires == 'number') { + date = new Date(); + date.setTime(date.getTime() + (options.expires * 24 * 60 * 60 * 1000)); + } else { + date = options.expires; + } + expires = '; expires=' + date.toUTCString(); // use expires attribute, max-age is not supported by IE + } + // CAUTION: Needed to parenthesize options.path and options.domain + // in the following expressions, otherwise they evaluate to undefined + // in the packed version for some reason... + var path = options.path ? '; path=' + (options.path) : ''; + var domain = options.domain ? '; domain=' + (options.domain) : ''; + var secure = options.secure ? '; secure' : ''; + document.cookie = [name, '=', encodeURIComponent(value), expires, path, domain, secure].join(''); + } else { // only name given, get cookie + var cookieValue = null; + if (document.cookie && document.cookie != '') { + var cookies = document.cookie.split(';'); + for (var i = 0; i < cookies.length; i++) { + var cookie = jQuery.trim(cookies[i]); + // Does this cookie string begin with the name we want? + if (cookie.substring(0, name.length + 1) == (name + '=')) { + cookieValue = decodeURIComponent(cookie.substring(name.length + 1)); + break; + } + } + } + return cookieValue; + } +}; \ No newline at end of file diff --git a/ui/new/scripts/jquery.md5.js b/ui/new/scripts/jquery.md5.js new file mode 100755 index 00000000000..54885ab451d --- /dev/null +++ b/ui/new/scripts/jquery.md5.js @@ -0,0 +1,229 @@ + /** + * jQuery MD5 hash algorithm function + * + * + * Calculate the md5 hash of a String + * String $.md5 ( String str ) + * + * + * Calculates the MD5 hash of str using the RSA Data Security, Inc. MD5 Message-Digest Algorithm, and returns that hash. + * MD5 (Message-Digest algorithm 5) is a widely-used cryptographic hash function with a 128-bit hash value. MD5 has been employed in a wide variety of security applications, and is also commonly used to check the integrity of data. The generated hash is also non-reversable. Data cannot be retrieved from the message digest, the digest uniquely identifies the data. + * MD5 was developed by Professor Ronald L. Rivest in 1994. Its 128 bit (16 byte) message digest makes it a faster implementation than SHA-1. + * This script is used to process a variable length message into a fixed-length output of 128 bits using the MD5 algorithm. It is fully compatible with UTF-8 encoding. It is very useful when u want to transfer encrypted passwords over the internet. If you plan using UTF-8 encoding in your project don't forget to set the page encoding to UTF-8 (Content-Type meta tag). + * This function orginally get from the WebToolkit and rewrite for using as the jQuery plugin. + * + * Example + * Code + * + * $.md5("I'm Persian."); + * + * Result + * + * "b8c901d0f02223f9761016cfff9d68df" + * + * + * @alias Muhammad Hussein Fattahizadeh < muhammad [AT] semnanweb [DOT] com > + * @link http://www.semnanweb.com/jquery-plugin/md5.html + * @see http://www.webtoolkit.info/ + * @license http://www.gnu.org/licenses/gpl.html [GNU General Public License] + * @param {jQuery} {md5:function(string)) + * @return string + */ + + (function($){ + + var rotateLeft = function(lValue, iShiftBits) { + return (lValue << iShiftBits) | (lValue >>> (32 - iShiftBits)); + } + + var addUnsigned = function(lX, lY) { + var lX4, lY4, lX8, lY8, lResult; + lX8 = (lX & 0x80000000); + lY8 = (lY & 0x80000000); + lX4 = (lX & 0x40000000); + lY4 = (lY & 0x40000000); + lResult = (lX & 0x3FFFFFFF) + (lY & 0x3FFFFFFF); + if (lX4 & lY4) return (lResult ^ 0x80000000 ^ lX8 ^ lY8); + if (lX4 | lY4) { + if (lResult & 0x40000000) return (lResult ^ 0xC0000000 ^ lX8 ^ lY8); + else return (lResult ^ 0x40000000 ^ lX8 ^ lY8); + } else { + return (lResult ^ lX8 ^ lY8); + } + } + + var F = function(x, y, z) { + return (x & y) | ((~ x) & z); + } + + var G = function(x, y, z) { + return (x & z) | (y & (~ z)); + } + + var H = function(x, y, z) { + return (x ^ y ^ z); + } + + var I = function(x, y, z) { + return (y ^ (x | (~ z))); + } + + var FF = function(a, b, c, d, x, s, ac) { + a = addUnsigned(a, addUnsigned(addUnsigned(F(b, c, d), x), ac)); + return addUnsigned(rotateLeft(a, s), b); + }; + + var GG = function(a, b, c, d, x, s, ac) { + a = addUnsigned(a, addUnsigned(addUnsigned(G(b, c, d), x), ac)); + return addUnsigned(rotateLeft(a, s), b); + }; + + var HH = function(a, b, c, d, x, s, ac) { + a = addUnsigned(a, addUnsigned(addUnsigned(H(b, c, d), x), ac)); + return addUnsigned(rotateLeft(a, s), b); + }; + + var II = function(a, b, c, d, x, s, ac) { + a = addUnsigned(a, addUnsigned(addUnsigned(I(b, c, d), x), ac)); + return addUnsigned(rotateLeft(a, s), b); + }; + + var convertToWordArray = function(string) { + var lWordCount; + var lMessageLength = string.length; + var lNumberOfWordsTempOne = lMessageLength + 8; + var lNumberOfWordsTempTwo = (lNumberOfWordsTempOne - (lNumberOfWordsTempOne % 64)) / 64; + var lNumberOfWords = (lNumberOfWordsTempTwo + 1) * 16; + var lWordArray = Array(lNumberOfWords - 1); + var lBytePosition = 0; + var lByteCount = 0; + while (lByteCount < lMessageLength) { + lWordCount = (lByteCount - (lByteCount % 4)) / 4; + lBytePosition = (lByteCount % 4) * 8; + lWordArray[lWordCount] = (lWordArray[lWordCount] | (string.charCodeAt(lByteCount) << lBytePosition)); + lByteCount++; + } + lWordCount = (lByteCount - (lByteCount % 4)) / 4; + lBytePosition = (lByteCount % 4) * 8; + lWordArray[lWordCount] = lWordArray[lWordCount] | (0x80 << lBytePosition); + lWordArray[lNumberOfWords - 2] = lMessageLength << 3; + lWordArray[lNumberOfWords - 1] = lMessageLength >>> 29; + return lWordArray; + }; + + var wordToHex = function(lValue) { + var WordToHexValue = "", WordToHexValueTemp = "", lByte, lCount; + for (lCount = 0; lCount <= 3; lCount++) { + lByte = (lValue >>> (lCount * 8)) & 255; + WordToHexValueTemp = "0" + lByte.toString(16); + WordToHexValue = WordToHexValue + WordToHexValueTemp.substr(WordToHexValueTemp.length - 2, 2); + } + return WordToHexValue; + }; + + var uTF8Encode = function(string) { + string = string.replace(/\x0d\x0a/g, "\x0a"); + var output = ""; + for (var n = 0; n < string.length; n++) { + var c = string.charCodeAt(n); + if (c < 128) { + output += String.fromCharCode(c); + } else if ((c > 127) && (c < 2048)) { + output += String.fromCharCode((c >> 6) | 192); + output += String.fromCharCode((c & 63) | 128); + } else { + output += String.fromCharCode((c >> 12) | 224); + output += String.fromCharCode(((c >> 6) & 63) | 128); + output += String.fromCharCode((c & 63) | 128); + } + } + return output; + }; + + $.extend({ + md5: function(string) { + var x = Array(); + var k, AA, BB, CC, DD, a, b, c, d; + var S11=7, S12=12, S13=17, S14=22; + var S21=5, S22=9 , S23=14, S24=20; + var S31=4, S32=11, S33=16, S34=23; + var S41=6, S42=10, S43=15, S44=21; + string = uTF8Encode(string); + x = convertToWordArray(string); + a = 0x67452301; b = 0xEFCDAB89; c = 0x98BADCFE; d = 0x10325476; + for (k = 0; k < x.length; k += 16) { + AA = a; BB = b; CC = c; DD = d; + a = FF(a, b, c, d, x[k+0], S11, 0xD76AA478); + d = FF(d, a, b, c, x[k+1], S12, 0xE8C7B756); + c = FF(c, d, a, b, x[k+2], S13, 0x242070DB); + b = FF(b, c, d, a, x[k+3], S14, 0xC1BDCEEE); + a = FF(a, b, c, d, x[k+4], S11, 0xF57C0FAF); + d = FF(d, a, b, c, x[k+5], S12, 0x4787C62A); + c = FF(c, d, a, b, x[k+6], S13, 0xA8304613); + b = FF(b, c, d, a, x[k+7], S14, 0xFD469501); + a = FF(a, b, c, d, x[k+8], S11, 0x698098D8); + d = FF(d, a, b, c, x[k+9], S12, 0x8B44F7AF); + c = FF(c, d, a, b, x[k+10], S13, 0xFFFF5BB1); + b = FF(b, c, d, a, x[k+11], S14, 0x895CD7BE); + a = FF(a, b, c, d, x[k+12], S11, 0x6B901122); + d = FF(d, a, b, c, x[k+13], S12, 0xFD987193); + c = FF(c, d, a, b, x[k+14], S13, 0xA679438E); + b = FF(b, c, d, a, x[k+15], S14, 0x49B40821); + a = GG(a, b, c, d, x[k+1], S21, 0xF61E2562); + d = GG(d, a, b, c, x[k+6], S22, 0xC040B340); + c = GG(c, d, a, b, x[k+11], S23, 0x265E5A51); + b = GG(b, c, d, a, x[k+0], S24, 0xE9B6C7AA); + a = GG(a, b, c, d, x[k+5], S21, 0xD62F105D); + d = GG(d, a, b, c, x[k+10], S22, 0x2441453); + c = GG(c, d, a, b, x[k+15], S23, 0xD8A1E681); + b = GG(b, c, d, a, x[k+4], S24, 0xE7D3FBC8); + a = GG(a, b, c, d, x[k+9], S21, 0x21E1CDE6); + d = GG(d, a, b, c, x[k+14], S22, 0xC33707D6); + c = GG(c, d, a, b, x[k+3], S23, 0xF4D50D87); + b = GG(b, c, d, a, x[k+8], S24, 0x455A14ED); + a = GG(a, b, c, d, x[k+13], S21, 0xA9E3E905); + d = GG(d, a, b, c, x[k+2], S22, 0xFCEFA3F8); + c = GG(c, d, a, b, x[k+7], S23, 0x676F02D9); + b = GG(b, c, d, a, x[k+12], S24, 0x8D2A4C8A); + a = HH(a, b, c, d, x[k+5], S31, 0xFFFA3942); + d = HH(d, a, b, c, x[k+8], S32, 0x8771F681); + c = HH(c, d, a, b, x[k+11], S33, 0x6D9D6122); + b = HH(b, c, d, a, x[k+14], S34, 0xFDE5380C); + a = HH(a, b, c, d, x[k+1], S31, 0xA4BEEA44); + d = HH(d, a, b, c, x[k+4], S32, 0x4BDECFA9); + c = HH(c, d, a, b, x[k+7], S33, 0xF6BB4B60); + b = HH(b, c, d, a, x[k+10], S34, 0xBEBFBC70); + a = HH(a, b, c, d, x[k+13], S31, 0x289B7EC6); + d = HH(d, a, b, c, x[k+0], S32, 0xEAA127FA); + c = HH(c, d, a, b, x[k+3], S33, 0xD4EF3085); + b = HH(b, c, d, a, x[k+6], S34, 0x4881D05); + a = HH(a, b, c, d, x[k+9], S31, 0xD9D4D039); + d = HH(d, a, b, c, x[k+12], S32, 0xE6DB99E5); + c = HH(c, d, a, b, x[k+15], S33, 0x1FA27CF8); + b = HH(b, c, d, a, x[k+2], S34, 0xC4AC5665); + a = II(a, b, c, d, x[k+0], S41, 0xF4292244); + d = II(d, a, b, c, x[k+7], S42, 0x432AFF97); + c = II(c, d, a, b, x[k+14], S43, 0xAB9423A7); + b = II(b, c, d, a, x[k+5], S44, 0xFC93A039); + a = II(a, b, c, d, x[k+12], S41, 0x655B59C3); + d = II(d, a, b, c, x[k+3], S42, 0x8F0CCC92); + c = II(c, d, a, b, x[k+10], S43, 0xFFEFF47D); + b = II(b, c, d, a, x[k+1], S44, 0x85845DD1); + a = II(a, b, c, d, x[k+8], S41, 0x6FA87E4F); + d = II(d, a, b, c, x[k+15], S42, 0xFE2CE6E0); + c = II(c, d, a, b, x[k+6], S43, 0xA3014314); + b = II(b, c, d, a, x[k+13], S44, 0x4E0811A1); + a = II(a, b, c, d, x[k+4], S41, 0xF7537E82); + d = II(d, a, b, c, x[k+11], S42, 0xBD3AF235); + c = II(c, d, a, b, x[k+2], S43, 0x2AD7D2BB); + b = II(b, c, d, a, x[k+9], S44, 0xEB86D391); + a = addUnsigned(a, AA); + b = addUnsigned(b, BB); + c = addUnsigned(c, CC); + d = addUnsigned(d, DD); + } + var tempValue = wordToHex(a) + wordToHex(b) + wordToHex(c) + wordToHex(d); + return tempValue.toLowerCase(); + } + }); + })(jQuery); diff --git a/ui/new/scripts/jquery.timers.js b/ui/new/scripts/jquery.timers.js new file mode 100755 index 00000000000..bb51157d405 --- /dev/null +++ b/ui/new/scripts/jquery.timers.js @@ -0,0 +1,138 @@ +/** + * jQuery.timers - Timer abstractions for jQuery + * Written by Blair Mitchelmore (blair DOT mitchelmore AT gmail DOT com) + * Licensed under the WTFPL (http://sam.zoy.org/wtfpl/). + * Date: 2009/10/16 + * + * @author Blair Mitchelmore + * @version 1.2 + * + **/ + +jQuery.fn.extend({ + everyTime: function(interval, label, fn, times) { + return this.each(function() { + jQuery.timer.add(this, interval, label, fn, times); + }); + }, + oneTime: function(interval, label, fn) { + return this.each(function() { + jQuery.timer.add(this, interval, label, fn, 1); + }); + }, + stopTime: function(label, fn) { + return this.each(function() { + jQuery.timer.remove(this, label, fn); + }); + } +}); + +jQuery.extend({ + timer: { + global: [], + guid: 1, + dataKey: "jQuery.timer", + regex: /^([0-9]+(?:\.[0-9]*)?)\s*(.*s)?$/, + powers: { + // Yeah this is major overkill... + 'ms': 1, + 'cs': 10, + 'ds': 100, + 's': 1000, + 'das': 10000, + 'hs': 100000, + 'ks': 1000000 + }, + timeParse: function(value) { + if (value == undefined || value == null) + return null; + var result = this.regex.exec(jQuery.trim(value.toString())); + if (result[2]) { + var num = parseFloat(result[1]); + var mult = this.powers[result[2]] || 1; + return num * mult; + } else { + return value; + } + }, + add: function(element, interval, label, fn, times) { + var counter = 0; + + if (jQuery.isFunction(label)) { + if (!times) + times = fn; + fn = label; + label = interval; + } + + interval = jQuery.timer.timeParse(interval); + + if (typeof interval != 'number' || isNaN(interval) || interval < 0) + return; + + if (typeof times != 'number' || isNaN(times) || times < 0) + times = 0; + + times = times || 0; + + var timers = jQuery.data(element, this.dataKey) || jQuery.data(element, this.dataKey, {}); + + if (!timers[label]) + timers[label] = {}; + + fn.timerID = fn.timerID || this.guid++; + + var handler = function() { + if ((++counter > times && times !== 0) || fn.call(element, counter) === false) + jQuery.timer.remove(element, label, fn); + }; + + handler.timerID = fn.timerID; + + if (!timers[label][fn.timerID]) + timers[label][fn.timerID] = window.setInterval(handler,interval); + + this.global.push( element ); + + }, + remove: function(element, label, fn) { + var timers = jQuery.data(element, this.dataKey), ret; + + if ( timers ) { + + if (!label) { + for ( label in timers ) + this.remove(element, label, fn); + } else if ( timers[label] ) { + if ( fn ) { + if ( fn.timerID ) { + window.clearInterval(timers[label][fn.timerID]); + delete timers[label][fn.timerID]; + } + } else { + for ( var fn in timers[label] ) { + window.clearInterval(timers[label][fn]); + delete timers[label][fn]; + } + } + + for ( ret in timers[label] ) break; + if ( !ret ) { + ret = null; + delete timers[label]; + } + } + + for ( ret in timers ) break; + if ( !ret ) + jQuery.removeData(element, this.dataKey); + } + } + } +}); + +jQuery(window).bind("unload", function() { + jQuery.each(jQuery.timer.global, function(index, item) { + jQuery.timer.remove(item); + }); +}); diff --git a/ui/resources/resource.properties b/ui/resources/resource.properties deleted file mode 100644 index b7a3baead51..00000000000 --- a/ui/resources/resource.properties +++ /dev/null @@ -1,5 +0,0 @@ -computer = computer -disk = disk -computer_disk_hahaha = computer disk hahaha -monitor = monitor -keyboard = keyboard diff --git a/ui/resources/resource_zh.properties b/ui/resources/resource_zh.properties deleted file mode 100644 index 948c8e13203..00000000000 --- a/ui/resources/resource_zh.properties +++ /dev/null @@ -1,5 +0,0 @@ -computer = 電腦 -disk = 硬碟 -computer_disk_hahaha = 電腦 硬碟 哈哈哈 !!! -monitor = 瑩幕 -keyboard = 鍵盤 diff --git a/ui/scripts/cloud.core.configuration.js b/ui/scripts/cloud.core.configuration.js index d3ffd9022ae..1bfcb7b1199 100644 --- a/ui/scripts/cloud.core.configuration.js +++ b/ui/scripts/cloud.core.configuration.js @@ -596,8 +596,12 @@ function showConfigurationTab() { array1.push("&cidr="+encodeURIComponent(newCidr)); if(newStartip != oldStartip) array1.push("&startIp="+encodeURIComponent(newStartip)); - if(newEndip != oldEndip && newEndip != null && newEndip.length > 0) + if(newEndip != oldEndip && newEndip != null && newEndip.length > 0) { + if(newStartip == oldStartip) { + array1.push("&startIp="+encodeURIComponent(newStartip)); //startIp needs to be passed to updatePod API when endIp is passed to updatePod API. + } array1.push("&endIp="+encodeURIComponent(newEndip)); + } if(newGateway != oldGateway && newGateway != null && newGateway.length > 0) array1.push("&gateway="+encodeURIComponent(newGateway)); @@ -697,10 +701,7 @@ function showConfigurationTab() { isTagged = thisDialog.find("#add_publicip_vlan_tagged").val() == "tagged"; } - if (isDirect && isTagged) - isValid &= validateString("Account", thisDialog.find("#add_publicip_vlan_account"), thisDialog.find("#add_publicip_vlan_account_errormsg"), false); //required - else - isValid &= validateString("Account", thisDialog.find("#add_publicip_vlan_account"), thisDialog.find("#add_publicip_vlan_account_errormsg"), true); //optional + isValid &= validateString("Account", thisDialog.find("#add_publicip_vlan_account"), thisDialog.find("#add_publicip_vlan_account_errormsg"), true); //optional if (isTagged) { isValid &= validateNumber("VLAN", thisDialog.find("#add_publicip_vlan_vlan"), thisDialog.find("#add_publicip_vlan_vlan_errormsg"), 2, 4095); @@ -717,15 +718,11 @@ function showConfigurationTab() { } else { vlan = "&vlan=untagged"; } - var directParams = ""; - if (isDirect && isTagged) { - directParams = "&domainId="+trim(thisDialog.find("#add_publicip_vlan_domain").val())+"&account="+trim(thisDialog.find("#add_publicip_vlan_account").val()); - } else if (isDirect && !isTagged) { - directParams = "&podId="+trim(thisDialog.find("#add_publicip_vlan_pod").val()); - } else if (!isDirect && (dialogAddVlanForZone.find("#add_publicip_vlan_scope").val()=="account-specific")) { //public VLAN, account-specific - directParams = "&domainId="+trim(thisDialog.find("#add_publicip_vlan_domain").val())+"&account="+trim(thisDialog.find("#add_publicip_vlan_account").val()); - } - + + var scopeParams = ""; + if(dialogAddVlanForZone.find("#add_publicip_vlan_scope").val()=="account-specific") + scopeParams = "&domainId="+trim(thisDialog.find("#add_publicip_vlan_domain").val())+"&account="+trim(thisDialog.find("#add_publicip_vlan_account").val()); + var type = "true"; if (getNetworkType() == "vlan") type = trim(thisDialog.find("#add_publicip_vlan_type").val()); var gateway = trim(thisDialog.find("#add_publicip_vlan_gateway").val()); @@ -747,7 +744,7 @@ function showConfigurationTab() { template.fadeIn("slow"); $.ajax({ - data: createURL("command=createVlanIpRange&forVirtualNetwork="+type+"&zoneId="+id+vlan+directParams+"&gateway="+encodeURIComponent(gateway)+"&netmask="+encodeURIComponent(netmask)+"&startip="+encodeURIComponent(startip)+"&endip="+encodeURIComponent(endip)+"&response=json"), + data: createURL("command=createVlanIpRange&forVirtualNetwork="+type+"&zoneId="+id+vlan+scopeParams+"&gateway="+encodeURIComponent(gateway)+"&netmask="+encodeURIComponent(netmask)+"&startip="+encodeURIComponent(startip)+"&endip="+encodeURIComponent(endip)+"&response=json"), dataType: "json", success: function(json) { var vlan = json.createvlaniprangeresponse; @@ -1041,21 +1038,16 @@ function showConfigurationTab() { dialogAddVlanForZone.find("#add_publicip_vlan_type").change(function(event) { var addPublicipVlanTagged = dialogAddVlanForZone.find("#add_publicip_vlan_tagged").empty(); - if ($(this).val() == "false") { //direct VLAN (only tagged option) - dialogAddVlanForZone.find("#add_publicip_vlan_scope_container").hide(); + // default value of "#add_publicip_vlan_scope" is "zone-wide". Calling change() will hide "#add_publicip_vlan_domain_container", "#add_publicip_vlan_account_container". + dialogAddVlanForZone.find("#add_publicip_vlan_scope").change(); + + if ($(this).val() == "false") { //direct VLAN (only tagged option) addPublicipVlanTagged.append(''); - dialogAddVlanForZone.find("#add_publicip_vlan_vlan_container").show(); - dialogAddVlanForZone.find("#add_publicip_vlan_domain_container").show(); - dialogAddVlanForZone.find("#add_publicip_vlan_account_container").show(); + dialogAddVlanForZone.find("#add_publicip_vlan_vlan_container").show(); dialogAddVlanForZone.find("#add_publicip_vlan_pod_container").hide(); - } else { //public VLAN - dialogAddVlanForZone.find("#add_publicip_vlan_scope_container").show(); - - // default value of "#add_publicip_vlan_scope" is "zone-wide". Calling change() will hide "#add_publicip_vlan_domain_container", "#add_publicip_vlan_account_container". - dialogAddVlanForZone.find("#add_publicip_vlan_scope").change(); - + } else { //public VLAN addPublicipVlanTagged.append('').append(''); if (dialogAddVlanForZone.find("#add_publicip_vlan_tagged") == "tagged") { @@ -1070,20 +1062,14 @@ function showConfigurationTab() { }); if (getNetworkType() != "vnet") { - dialogAddVlanForZone.find("#add_publicip_vlan_tagged").change(function(event) { - if (dialogAddVlanForZone.find("#add_publicip_vlan_type").val() == "false") { //direct VLAN (only tagged option) - dialogAddVlanForZone.find("#add_publicip_vlan_scope_container").hide(); - dialogAddVlanForZone.find("#add_publicip_vlan_vlan_container").show(); - dialogAddVlanForZone.find("#add_publicip_vlan_domain_container").show(); - dialogAddVlanForZone.find("#add_publicip_vlan_account_container").show(); - dialogAddVlanForZone.find("#add_publicip_vlan_pod_container").hide(); + dialogAddVlanForZone.find("#add_publicip_vlan_tagged").change(function(event) { + // default value of "#add_publicip_vlan_scope" is "zone-wide". Calling change() will hide "#add_publicip_vlan_domain_container", "#add_publicip_vlan_account_container". + dialogAddVlanForZone.find("#add_publicip_vlan_scope").change(); + if (dialogAddVlanForZone.find("#add_publicip_vlan_type").val() == "false") { //direct VLAN (only tagged option) + dialogAddVlanForZone.find("#add_publicip_vlan_vlan_container").show(); + dialogAddVlanForZone.find("#add_publicip_vlan_pod_container").hide(); } else { //public VLAN - dialogAddVlanForZone.find("#add_publicip_vlan_scope_container").show(); - - // default value of "#add_publicip_vlan_scope" is "zone-wide". Calling change() will hide "#add_publicip_vlan_domain_container", "#add_publicip_vlan_account_container". - dialogAddVlanForZone.find("#add_publicip_vlan_scope").change(); - if ($(this).val() == "tagged") { dialogAddVlanForZone.find("#add_publicip_vlan_vlan_container").show(); dialogAddVlanForZone.find("#add_publicip_vlan_pod_container").hide(); @@ -1098,7 +1084,7 @@ function showConfigurationTab() { dialogAddVlanForZone.find("#add_publicip_vlan_container").hide(); } - dialogAddVlanForZone.find("#add_publicip_vlan_scope").change(function(event) { + dialogAddVlanForZone.find("#add_publicip_vlan_scope").change(function(event) { if($(this).val() == "zone-wide") { dialogAddVlanForZone.find("#add_publicip_vlan_domain_container").hide(); dialogAddVlanForZone.find("#add_publicip_vlan_account_container").hide(); @@ -1244,7 +1230,7 @@ function showConfigurationTab() { dialogEditService.find("#service_name").text(svcName); dialogEditService.find("#edit_service_name").val(svcName); - dialogEditService.find("#edit_service_display").val(template.find("#service_display").text()); + dialogEditService.find("#edit_service_display").val(template.find("#service_displaytext").text()); dialogEditService.find("#edit_service_offerha").val(toBooleanValue(template.find("#service_offerha").text())); dialogEditService @@ -1260,9 +1246,9 @@ function showConfigurationTab() { var moreCriteria = []; var name = trim(thisDialog.find("#edit_service_name").val()); - moreCriteria.push("&name="+encodeURIComponent(name)); + moreCriteria.push("&name="+encodeURIComponent(escape(name))); var displaytext = trim(thisDialog.find("#edit_service_display").val()); - moreCriteria.push("&displayText="+encodeURIComponent(displaytext)); + moreCriteria.push("&displayText="+encodeURIComponent(escape(displaytext))); var offerha = trim(thisDialog.find("#edit_service_offerha").val()); moreCriteria.push("&offerha="+offerha); @@ -1316,17 +1302,17 @@ function showConfigurationTab() { function serviceJSONToTemplate(json, template) { template.attr("id", "service_"+json.id); (index++ % 2 == 0)? template.addClass("smallrow_even"): template.addClass("smallrow_odd"); - template.data("svcId", json.id).data("svcName", sanitizeXSS(json.name)); + template.data("svcId", json.id).data("svcName", sanitizeXSS(unescape(json.name))); template.find("#service_id").text(json.id); - template.find("#service_name").text(json.name); - template.find("#service_displaytext").text(json.displaytext); + template.find("#service_name").text(unescape(json.name)); + template.find("#service_displaytext").text(unescape(json.displaytext)); template.find("#service_storagetype").text(json.storagetype); template.find("#service_cpu").text(json.cpunumber + " x " + convertHz(json.cpuspeed)); template.find("#service_memory").text(convertBytes(parseInt(json.memory)*1024*1024)); template.find("#service_offerha").text(toBooleanText(json.offerha)); template.find("#service_networktype").text(toNetworkType(json.usevirtualnetwork)); - template.find("#service_tags").text(json.tags); + template.find("#service_tags").text(unescape(json.tags)); setDateField(json.created, template.find("#service_created")); } @@ -1454,10 +1440,10 @@ function showConfigurationTab() { var array1 = []; var name = trim(thisDialog.find("#add_service_name").val()); - array1.push("&name="+encodeURIComponent(name)); + array1.push("&name="+encodeURIComponent(escape(name))); var display = trim(thisDialog.find("#add_service_display").val()); - array1.push("&displayText="+encodeURIComponent(display)); + array1.push("&displayText="+encodeURIComponent(escape(display))); var storagetype = trim(thisDialog.find("#add_service_storagetype").val()); array1.push("&storageType="+storagetype); @@ -1480,7 +1466,7 @@ function showConfigurationTab() { var tags = trim(thisDialog.find("#add_service_tags").val()); if(tags != null && tags.length > 0) - array1.push("&tags="+encodeURIComponent(tags)); + array1.push("&tags="+encodeURIComponent(escape(tags))); thisDialog.dialog("close"); $.ajax({ @@ -1529,7 +1515,7 @@ function showConfigurationTab() { var isValid = true; isValid &= validateString("Name", thisDialog.find("#add_disk_name"), thisDialog.find("#add_disk_name_errormsg")); isValid &= validateString("Description", thisDialog.find("#add_disk_description"), thisDialog.find("#add_disk_description_errormsg")); - isValid &= validateNumber("Disk size", thisDialog.find("#add_disk_disksize"), thisDialog.find("#add_disk_disksize_errormsg"), 1, null); + isValid &= validateNumber("Disk size", thisDialog.find("#add_disk_disksize"), thisDialog.find("#add_disk_disksize_errormsg"), 0, null); isValid &= validateString("Tags", thisDialog.find("#add_disk_tags"), thisDialog.find("#add_disk_tags_errormsg"), true); //optional if (!isValid) return; @@ -1544,17 +1530,17 @@ function showConfigurationTab() { var array1 = []; var name = trim(thisDialog.find("#add_disk_name").val()); - array1.push("&name="+encodeURIComponent(name)); + array1.push("&name="+encodeURIComponent(escape(name))); var description = trim(thisDialog.find("#add_disk_description").val()); - array1.push("&displaytext="+encodeURIComponent(description)); + array1.push("&displaytext="+encodeURIComponent(escape(description))); var disksize = trim(thisDialog.find("#add_disk_disksize").val()); array1.push("&disksize="+disksize); var tags = trim(thisDialog.find("#add_disk_tags").val()); if(tags != null && tags.length > 0) - array1.push("&tags="+encodeURIComponent(tags)); + array1.push("&tags="+encodeURIComponent(escape(tags))); thisDialog.dialog("close"); $.ajax({ @@ -1649,7 +1635,7 @@ function showConfigurationTab() { var dialogBox = $(this); dialogBox.dialog("close"); $.ajax({ - data: createURL("command=updateDiskOffering&name="+encodeURIComponent(name)+"&displayText="+encodeURIComponent(display)+"&id="+diskId+"&response=json"), + data: createURL("command=updateDiskOffering&name="+encodeURIComponent(escape(name))+"&displayText="+encodeURIComponent(escape(display))+"&id="+diskId+"&response=json"), dataType: "json", success: function(json) { template.find("#disk_description").text(display); @@ -1699,15 +1685,14 @@ function showConfigurationTab() { } else { template.addClass("smallrow_odd"); } - template.data("diskId", json.id).data("diskName", sanitizeXSS(json.name)); + template.data("diskId", json.id).data("diskName", sanitizeXSS(unescape(json.name))); template.find("#disk_id").text(json.id); - template.find("#disk_name").text(json.name); - template.find("#disk_description").text(json.displaytext); + template.find("#disk_name").text(unescape(json.name)); + template.find("#disk_description").text(unescape(json.displaytext)); template.find("#disk_disksize").text(convertBytes(json.disksize)); - template.find("#disk_tags").text(json.tags); - template.find("#disk_domain").text(json.domain); - template.find("#disk_ismirrored").text(json.ismirrored); + template.find("#disk_tags").text(unescape(json.tags)); + template.find("#disk_domain").text(unescape(json.domain)); } function listDiskOfferings() { diff --git a/ui/scripts/cloud.core.domains.js b/ui/scripts/cloud.core.domains.js index 921b158b177..572ebe8dbad 100644 --- a/ui/scripts/cloud.core.domains.js +++ b/ui/scripts/cloud.core.domains.js @@ -65,8 +65,8 @@ function showDomainsTab() { } function drawTree(id, level, container) { - $.ajax({ - data: createURL("command=listDomainChildren&id="+id+"&response=json"), + $.ajax({ + data: createURL("command=listDomainChildren&id="+id+"&response=json&pageSize=-1"), dataType: "json", async: false, success: function(json) { @@ -320,8 +320,8 @@ function showDomainsTab() { rightPanelDetailContent.hide(); rightPanelSearchResult.show(); var keyword = searchInput.val(); - $.ajax({ - data: createURL("command=listDomains&keyword="+keyword+"&response=json"+maxPageSize), + $.ajax({ + data: createURL("command=listDomains&keyword="+keyword+"&response=json&pageSize=-1"), //pageSize=-1 will return all items (no limitation) dataType: "json", async: false, success: function(json) { @@ -349,8 +349,8 @@ function showDomainsTab() { //draw root node function drawRootNode(rootDomainId) { treeContentBox.empty(); - $.ajax({ - data: createURL("command=listDomains&id="+rootDomainId+"&response=json"), + $.ajax({ + data: createURL("command=listDomains&id="+rootDomainId+"&response=json&pageSize=-1"), //pageSize=-1 will return all items (no limitation) dataType: "json", async: false, success: function(json) { diff --git a/ui/scripts/cloud.core.hosts.js b/ui/scripts/cloud.core.hosts.js index 65423fa6f67..af7aac1e64d 100644 --- a/ui/scripts/cloud.core.hosts.js +++ b/ui/scripts/cloud.core.hosts.js @@ -23,73 +23,72 @@ function showHostsTab() { var sIndex = 0; var pIndex = 0; - // Dialog Setup - if (getHypervisorType() != "kvm") { //"xenserver" - $("#host_action_new_routing").show(); - activateDialog($("#dialog_add_routing").dialog({ - autoOpen: false, - modal: true, - zIndex: 2000 - })); - - var dialogAddRouting = $("#dialog_add_routing"); - + // Dialog Setup + $("#host_action_new_routing").show(); + activateDialog($("#dialog_add_routing").dialog({ + autoOpen: false, + modal: true, + zIndex: 2000 + })); + + var dialogAddRouting = $("#dialog_add_routing"); + + $.ajax({ + data: createURL("command=listZones&available=true&response=json"+maxPageSize), + dataType: "json", + success: function(json) { + var zones = json.listzonesresponse.zone; + var zoneSelect = dialogAddRouting.find("#host_zone").empty(); + if (zones != null && zones.length > 0) { + for (var i = 0; i < zones.length; i++) + zoneSelect.append(""); + } + //dialogAddRouting.find("#host_zone").change(); + } + }); + + dialogAddRouting.find("#host_zone").bind("change", function(event) { + var zoneId = $(this).val(); $.ajax({ - data: createURL("command=listZones&available=true&response=json"+maxPageSize), + data: createURL("command=listPods&zoneId="+zoneId+"&response=json"+maxPageSize), dataType: "json", + async: false, success: function(json) { - var zones = json.listzonesresponse.zone; - var zoneSelect = dialogAddRouting.find("#host_zone").empty(); - if (zones != null && zones.length > 0) { - for (var i = 0; i < zones.length; i++) - zoneSelect.append(""); + var pods = json.listpodsresponse.pod; + var podSelect = dialogAddRouting.find("#host_pod").empty(); + if (pods != null && pods.length > 0) { + for (var i = 0; i < pods.length; i++) { + podSelect.append(""); + } } - //dialogAddRouting.find("#host_zone").change(); + dialogAddRouting.find("#host_pod").change(); } }); - - dialogAddRouting.find("#host_zone").bind("change", function(event) { - var zoneId = $(this).val(); - $.ajax({ - data: createURL("command=listPods&zoneId="+zoneId+"&response=json"+maxPageSize), - dataType: "json", - async: false, - success: function(json) { - var pods = json.listpodsresponse.pod; - var podSelect = dialogAddRouting.find("#host_pod").empty(); - if (pods != null && pods.length > 0) { - for (var i = 0; i < pods.length; i++) { - podSelect.append(""); - } - } - dialogAddRouting.find("#host_pod").change(); - } - }); - }); - - dialogAddRouting.find("#host_pod").bind("change", function(event) { - var podId = $(this).val(); - if(podId == null || podId.length == 0) - return; - var clusterSelect = dialogAddRouting.find("#cluster_select").empty(); - $.ajax({ - data: createURL("command=listClusters&response=json&podid=" + podId+maxPageSize), - dataType: "json", - success: function(json) { - var items = json.listclustersresponse.cluster; - if(items != null && items.length > 0) { - for(var i=0; i" + items[i].name + ""); - dialogAddRouting.find("input[value=existing_cluster_radio]").attr("checked", true); - } - else { - clusterSelect.append(""); - dialogAddRouting.find("input[value=new_cluster_radio]").attr("checked", true); - } - } - }); - }); - } + }); + + dialogAddRouting.find("#host_pod").bind("change", function(event) { + var podId = $(this).val(); + if(podId == null || podId.length == 0) + return; + var clusterSelect = dialogAddRouting.find("#cluster_select").empty(); + $.ajax({ + data: createURL("command=listClusters&response=json&podid=" + podId+maxPageSize), + dataType: "json", + success: function(json) { + var items = json.listclustersresponse.cluster; + if(items != null && items.length > 0) { + for(var i=0; i" + items[i].name + ""); + dialogAddRouting.find("input[value=existing_cluster_radio]").attr("checked", true); + } + else { + clusterSelect.append(""); + dialogAddRouting.find("input[value=new_cluster_radio]").attr("checked", true); + } + } + }); + }); + activateDialog($("#dialog_update_os").dialog({ autoOpen: false, modal: true, @@ -566,7 +565,7 @@ function showHostsTab() { } else if (state == "Maintenance") { template.find(".grid_links").find("#host_action_reconnect_container, #host_action_enable_maint_container").hide(); } else if (state == "Disconnected") { - template.find(".grid_links").find("#host_action_reconnect_container, #host_action_enable_maint_container, #host_action_cancel_maint_container, #host_action_remove_container").hide(); + template.find(".grid_links").find("#host_action_reconnect_container, #host_action_enable_maint_container, #host_action_cancel_maint_container").hide(); } else { alert("Unsupported Host State: " + state); } @@ -575,105 +574,101 @@ function showHostsTab() { var submenuContent = $("#submenu_content_routing"); - // Add New Routing Host - if (getHypervisorType() != "kvm") { - $("#host_action_new_routing").bind("click", function(event) { - dialogAddRouting.find("#new_cluster_name").val(""); - dialogAddRouting.find("#host_zone").change(); //refresh cluster dropdown - - dialogAddRouting - .dialog('option', 'buttons', { - "Add": function() { - var dialogBox = $(this); - var clusterRadio = dialogBox.find("input[name=cluster]:checked").val(); + // Add New Routing Host + $("#host_action_new_routing").bind("click", function(event) { + dialogAddRouting.find("#new_cluster_name").val(""); + dialogAddRouting.find("#host_zone").change(); //refresh cluster dropdown + + dialogAddRouting + .dialog('option', 'buttons', { + "Add": function() { + var dialogBox = $(this); + var clusterRadio = dialogBox.find("input[name=cluster]:checked").val(); + + // validate values + var isValid = true; + isValid &= validateString("Host name", dialogBox.find("#host_hostname"), dialogBox.find("#host_hostname_errormsg")); + isValid &= validateString("User name", dialogBox.find("#host_username"), dialogBox.find("#host_username_errormsg")); + isValid &= validateString("Password", dialogBox.find("#host_password"), dialogBox.find("#host_password_errormsg")); + if (!isValid) return; - // validate values - var isValid = true; - isValid &= validateString("Host name", dialogBox.find("#host_hostname"), dialogBox.find("#host_hostname_errormsg")); - isValid &= validateString("User name", dialogBox.find("#host_username"), dialogBox.find("#host_username_errormsg")); - isValid &= validateString("Password", dialogBox.find("#host_password"), dialogBox.find("#host_password_errormsg")); - if(clusterRadio == "new_cluster_radio") - isValid &= validateString("Cluster name", dialogBox.find("#new_cluster_name"), dialogBox.find("#new_cluster_name_errormsg")); - if (!isValid) return; - - var array1 = []; - - var zoneId = dialogBox.find("#host_zone").val(); - array1.push("&zoneId="+zoneId); - - var podId = dialogBox.find("#host_pod").val(); - array1.push("&podId="+podId); - - var username = trim(dialogBox.find("#host_username").val()); - array1.push("&username="+encodeURIComponent(username)); - - var password = trim(dialogBox.find("#host_password").val()); - array1.push("&password="+encodeURIComponent(password)); - - if(clusterRadio == "new_cluster_radio") { - var newClusterName = trim(dialogBox.find("#new_cluster_name").val()); - array1.push("&clustername="+encodeURIComponent(newClusterName)); - } - else if(clusterRadio == "existing_cluster_radio") { - var clusterId = dialogBox.find("#cluster_select").val(); - // We will default to no cluster if someone selects Join Cluster with no cluster available. - if (clusterId != '-1') { - array1.push("&clusterid="+clusterId); - } - } - - var hostname = trim(dialogBox.find("#host_hostname").val()); - var url; - if(hostname.indexOf("http://")==-1) - url = "http://" + hostname; - else - url = hostname; - array1.push("&url="+encodeURIComponent(url)); - - var template = $("#routing_template").clone(true); - var loadingImg = template.find(".adding_loading"); - var rowContainer = template.find("#row_container"); - loadingImg.find(".adding_text").text("Adding...."); - loadingImg.show(); - rowContainer.hide(); - submenuContent.find("#grid_content").append(template.fadeIn("slow")); - - dialogBox.dialog("close"); - $.ajax({ - data: createURL("command=addHost&response=json" + array1.join("")), - dataType: "json", - success: function(json) { - var items = json.addhostresponse.host; - routingJSONToTemplate(items[0], template); - loadingImg.hide(); - rowContainer.show(); - changeGridRowsTotal(submenuContent.find("#grid_rows_total"), 1); - - if(items.length > 1) { - for(var i=1; i 1) { + for(var i=1; iPlease confirm you want to destroy your virtual machine: "+vmName+". Destroying your virtual machine would include deleting the ROOT volume and all attached data disk volumes.

      ") + .html("

      Please confirm you want to destroy your virtual machine: "+vmName+". Destroying your virtual machine will also delete the ROOT volume, but not attached data disk volumes.

      ") .dialog('option', 'buttons', { "Confirm": function() { $(this).dialog("close"); @@ -576,7 +576,7 @@ function showInstancesTab(p_domainId, p_account) { if (offerings != null && offerings.length > 0) { for (var i = 0; i < offerings.length; i++) { - var option = $("").data("name", offerings[i].name); + var option = $("").data("name", sanitizeXSS(unescape(offerings[i].name))); offeringSelect.append(option); } } @@ -611,7 +611,7 @@ function showInstancesTab(p_domainId, p_account) { vmInstance.find(".row_loading").show(); vmInstance.find(".loadingmessage_container .loadingmessage_top p").html("Your virtual instance has been upgraded. Please restart your virtual instance for the new service offering to take effect."); vmInstance.find(".loadingmessage_container").fadeIn("slow"); - vmInstance.find("#vm_service").html("Service: " + sanitizeXSS(result.virtualmachine[0].serviceofferingname)); + vmInstance.find("#vm_service").html("Service: " + sanitizeXSS(unescape(result.virtualmachine[0].serviceofferingname))); if (result.virtualmachine[0].haenable =='true') { vmInstance.find("#vm_ha").html("HA: Enabled"); vmInstance.find("#vm_action_ha").text("Disable HA"); @@ -1017,6 +1017,7 @@ function showInstancesTab(p_domainId, p_account) { if(volumes[i].type=="ROOT") { if (volumes[i].vmstate == "Stopped") { detail.find("#volume_action_detach_disk, #volume_acton_separator").hide(); + detail.find("#volume_action_create_template").show(); } else { detail.find("#volume_action_detach_disk, #volume_acton_separator, #volume_action_create_template").hide(); } @@ -1109,7 +1110,7 @@ function showInstancesTab(p_domainId, p_account) { instanceTemplate.find("#vm_ip_address").html("IP Address: " + instanceJSON.ipaddress); instanceTemplate.find("#vm_zone").html("Zone: " + sanitizeXSS(instanceJSON.zonename)); instanceTemplate.find("#vm_template").html("Template: " + sanitizeXSS(instanceJSON.templatename)); - instanceTemplate.find("#vm_service").html("Service: " + sanitizeXSS(instanceJSON.serviceofferingname)); + instanceTemplate.find("#vm_service").html("Service: " + sanitizeXSS(unescape(instanceJSON.serviceofferingname))); if (instanceJSON.haenable =='true') { instanceTemplate.find("#vm_ha").html("HA: Enabled"); instanceTemplate.find("#vm_action_ha").text("Disable HA"); @@ -1277,7 +1278,7 @@ function showInstancesTab(p_domainId, p_account) { continue; var checked = "checked"; if (first == false) checked = ""; - var listItem = $("
      • "); + var listItem = $("
      • "); $("#wizard_service_offering").append(listItem); first = false; } @@ -1306,14 +1307,14 @@ function showInstancesTab(p_domainId, p_account) { var html = "
    • " +"" - +"" + +"" +"
      • "; $("#wizard_root_disk_offering").append(html); var html2 = "
    • " +"" - +"" + +"" +"
      • "; $("#wizard_data_disk_offering").append(html2); } diff --git a/ui/scripts/cloud.core.js b/ui/scripts/cloud.core.js index abfa94d4d48..ebee3b350d8 100644 --- a/ui/scripts/cloud.core.js +++ b/ui/scripts/cloud.core.js @@ -626,7 +626,11 @@ function noNull(val) { // Prevent cross-site-script(XSS) attack. // used right before adding user input to the DOM tree. e.g. DOM_element.html(sanitizeXSS(user_input)); function sanitizeXSS(val) { +<<<<<<< HEAD if((val == null) || (typeof val != "string")) +======= + if(val == null || typeof(val) != "string") +>>>>>>> master return val; val = val.replace(//g, ">"); //replace > whose unicode is \u003e diff --git a/ui/scripts/cloud.core.network.js b/ui/scripts/cloud.core.network.js index 1a5545ed7d5..782ce057b04 100644 --- a/ui/scripts/cloud.core.network.js +++ b/ui/scripts/cloud.core.network.js @@ -409,7 +409,8 @@ function showNetworkingTab(p_domainId, p_account) { template.find("#row_container #protocol").text(json.protocol); template.find("#row_container_edit #protocol").text(json.protocol); - template.find("#row_container #vm_name").text(json.vmname); + var vmName = getVmName(json.vmname, json.vmdisplayname); //json doesn't include vmdisplayname property(incorrect). Waiting for Bug 6241 to be fixed.... + template.find("#row_container #vm_name").text(vmName); var virtualMachineId = json.virtualmachineid; $.ajax({ diff --git a/ui/scripts/cloud.core.storage.js b/ui/scripts/cloud.core.storage.js index 9008c14e1e7..f8eef15c1b2 100644 --- a/ui/scripts/cloud.core.storage.js +++ b/ui/scripts/cloud.core.storage.js @@ -16,14 +16,14 @@ * */ -// Version: 1.9.1.452 +// Version: @VERSION@ function showStorageTab(domainId, targetTab) { var currentSubMenu; var populateZoneField = function(isAdmin) { $.ajax({ - data: createURL("command=listZones&available=true&response=json"+maxPageSize), + data: createURL("command=listZones&available=true&response=json"+maxPageSize), dataType: "json", success: function(json) { var zones = json.listzonesresponse.zone; @@ -50,7 +50,7 @@ function showStorageTab(domainId, targetTab) { var populateDiskOfferingField = function() { $.ajax({ - data: createURL("command=listDiskOfferings&response=json"), + data: createURL("command=listDiskOfferings&response=json"), dataType: "json", success: function(json) { var offerings = json.listdiskofferingsresponse.diskoffering; @@ -75,7 +75,7 @@ function showStorageTab(domainId, targetTab) { var volumeVmSelect = $("#dialog_attach_volume").find("#volume_vm").empty(); if (instances != null && instances.length > 0) { for (var i = 0; i < instances.length; i++) { - volumeVmSelect.append(""); + volumeVmSelect.append(""); } } $.ajax({ @@ -86,7 +86,7 @@ function showStorageTab(domainId, targetTab) { var instances = json.listvirtualmachinesresponse.virtualmachine; if (instances != null && instances.length > 0) { for (var i = 0; i < instances.length; i++) { - volumeVmSelect.append(""); + volumeVmSelect.append(""); } } } @@ -274,8 +274,10 @@ function showStorageTab(domainId, targetTab) { setDateField(json.created, template.find("#volume_created")); if(json.type=="ROOT") { - } else { - // DataDisk + if (json.vmstate == "Stopped") + template.find("#volume_action_create_template_span").show(); + } + else { //json.type=="DATADISK": "detach disk" is allowed, "create template" is disallowed. if (json.virtualmachineid != undefined) { if (json.storagetype == "shared" && (json.vmstate == "Running" || json.vmstate == "Stopped")) { template.find("#volume_action_detach_span").show(); @@ -531,7 +533,8 @@ function showStorageTab(domainId, targetTab) { var name = thisDialog.find("#name").val(); var displayText = thisDialog.find("#display_text").val(); - var osTypeId = thisDialog.find("#os_type").val(); + var osTypeId = thisDialog.find("#os_type").val(); + var password = thisDialog.find("#password").val(); thisDialog.dialog("close"); var loadingImg = template.find(".adding_loading"); @@ -541,7 +544,7 @@ function showStorageTab(domainId, targetTab) { rowContainer.hide(); $.ajax({ - data: createURL("command=createTemplate&snapshotid="+snapshotId+"&name="+name+"&displaytext="+displayText+"&ostypeid="+osTypeId+"&response=json"), + data: createURL("command=createTemplate&snapshotid="+snapshotId+"&name="+name+"&displaytext="+displayText+"&ostypeid="+osTypeId+"&passwordEnabled="+password+"&response=json"), dataType: "json", success: function(json) { var jobId = json.createtemplateresponse.jobid; @@ -592,28 +595,29 @@ function showStorageTab(domainId, targetTab) { } }); - $("#volume_action_snapshot_grid, #volume_action_take_snapshot_container, #volume_action_recurring_snapshot_container").show(); - $("#submenu_snapshot").show().bind("click", function(event) { - event.preventDefault(); - - currentSubMenu.addClass("submenu_links_off").removeClass("submenu_links_on"); - $(this).addClass("submenu_links_on").removeClass("submenu_links_off"); - currentSubMenu = $(this); - - $("#submenu_content_snapshot").show(); - $("#submenu_content_pool").hide(); - $("#submenu_content_storage").hide(); - $("#submenu_content_volume").hide(); - - var submenuContent = $("#submenu_content_snapshot"); - if (isAdmin) - submenuContent.find("#adv_search_domain_li, #adv_search_account_li").show(); - else //There are no fields in Advanced Search Dialog Box for non-admin user. So, hide Advanced Search Link. - submenuContent.find("#advanced_search_link").hide(); - - currentPage = 1; - listSnapshots(); - }); + $("#volume_action_snapshot_grid, #volume_action_take_snapshot_container, #volume_action_recurring_snapshot_container").show(); + $("#submenu_snapshot").show().bind("click", function(event) { + event.preventDefault(); + + currentSubMenu.addClass("submenu_links_off").removeClass("submenu_links_on"); + $(this).addClass("submenu_links_on").removeClass("submenu_links_off"); + currentSubMenu = $(this); + + $("#submenu_content_snapshot").show(); + $("#submenu_content_pool").hide(); + $("#submenu_content_storage").hide(); + $("#submenu_content_volume").hide(); + + var submenuContent = $("#submenu_content_snapshot"); + if (isAdmin) + submenuContent.find("#adv_search_domain_li, #adv_search_account_li").show(); + else //There are no fields in Advanced Search Dialog Box for non-admin user. So, hide Advanced Search Link. + submenuContent.find("#advanced_search_link").hide(); + + currentPage = 1; + listSnapshots(); + }); + if (getHypervisorType() == "kvm") { $("#dialog_add_pool #pool_cluster_container").hide(); } @@ -979,7 +983,7 @@ function showStorageTab(domainId, targetTab) { rowContainer.hide(); $.ajax({ - data: createURL("command=deleteVolume&id="+volumeId+"&response=json"), + data: createURL("command=deleteVolume&id="+volumeId+"&response=json"), dataType: "json", success: function(json) { volumeTemplate.slideUp("slow", function(){ @@ -1596,7 +1600,8 @@ function showStorageTab(domainId, targetTab) { var name = thisDialog.find("#name").val(); var displayText = thisDialog.find("#display_text").val(); - var osTypeId = thisDialog.find("#os_type").val(); + var osTypeId = thisDialog.find("#os_type").val(); + var password = thisDialog.find("#password").val(); thisDialog.dialog("close"); var loadingImg = template.find(".adding_loading"); @@ -1606,7 +1611,7 @@ function showStorageTab(domainId, targetTab) { rowContainer.hide(); $.ajax({ - data: createURL("command=createTemplate&snapshotid="+snapshotId+"&name="+name+"&displaytext="+displayText+"&ostypeid="+osTypeId+"&response=json"), + data: createURL("command=createTemplate&snapshotid="+snapshotId+"&name="+name+"&displaytext="+displayText+"&ostypeid="+osTypeId+"&passwordEnabled="+password+"&response=json"), dataType: "json", success: function(json) { var jobId = json.createtemplateresponse.jobid; diff --git a/ui/scripts/cloud.core.templates.js b/ui/scripts/cloud.core.templates.js index 47f73b6e99b..12d1d91ccfa 100644 --- a/ui/scripts/cloud.core.templates.js +++ b/ui/scripts/cloud.core.templates.js @@ -51,6 +51,12 @@ function showTemplatesTab() { zIndex: 2000 })); + activateDialog($("#dialog_create_vm_from_iso").dialog({ + width:300, + autoOpen: false, + modal: true, + zIndex: 2000 + })); var g_zoneIds = [], g_zoneNames = []; var addTemplateZoneField = $("#dialog_add_template #add_template_zone"); @@ -87,7 +93,7 @@ function showTemplatesTab() { } $.ajax({ - data: createURL("command=listOsTypes&response=json"), + data: createURL("command=listOsTypes&response=json"+maxPageSize), dataType: "json", success: function(json) { types = json.listostypesresponse.ostype; @@ -104,27 +110,36 @@ function showTemplatesTab() { }); $.ajax({ - data: createURL("command=listServiceOfferings&response=json"), + data: createURL("command=listServiceOfferings&response=json"+maxPageSize), dataType: "json", success: function(json) { var items = json.listserviceofferingsresponse.serviceoffering; if(items != null && items.length > 0 ) { - var serviceOfferingField = $("#dialog_create_vm_from_template #service_offering").empty(); - for(var i = 0; i < items.length; i++) - serviceOfferingField.append(""); + var templateServiceOfferingField = $("#dialog_create_vm_from_template #service_offering").empty(); + var isoServiceOfferingField = $("#dialog_create_vm_from_iso #service_offering").empty(); + for(var i = 0; i < items.length; i++) { + templateServiceOfferingField.append(""); + isoServiceOfferingField.append(""); + } } } }); $.ajax({ - data: createURL("command=listDiskOfferings&response=json"), + data: createURL("command=listDiskOfferings&response=json"+maxPageSize), dataType: "json", success: function(json) { var items = json.listdiskofferingsresponse.diskoffering; if(items != null && items.length > 0 ) { - var diskOfferingField = $("#dialog_create_vm_from_template #disk_offering").empty(); - for(var i = 0; i < items.length; i++) - diskOfferingField.append(""); + var templateDiskOfferingField = $("#dialog_create_vm_from_template #disk_offering").empty(); + templateDiskOfferingField.append(""); + + var isoDiskOfferingField = $("#dialog_create_vm_from_iso #disk_offering").empty(); + + for(var i = 0; i < items.length; i++) { + templateDiskOfferingField.append(""); + isoDiskOfferingField.append(""); + } } } @@ -523,12 +538,20 @@ function showTemplatesTab() { function createVMFromTemplateOrISO(actionLink, sourceType) { var parentElementId = actionLink.data("parentElementId"); var thisTemplate = $("#"+parentElementId); - var id = (sourceType == "template")? thisTemplate.data("templateId"): thisTemplate.data("isoId"); var name = thisTemplate.data("name"); - var zoneId = thisTemplate.data("zoneId"); - - var createVmDialog = $("#dialog_create_vm_from_template"); - createVmDialog.find("#template").text(name); + var zoneId = thisTemplate.data("zoneId"); + + var id; + var createVmDialog; + if(sourceType == "template") { + id = thisTemplate.data("templateId"); + createVmDialog = $("#dialog_create_vm_from_template"); + } + else { //sourceType == "iso" + id = thisTemplate.data("isoId"); + createVmDialog = $("#dialog_create_vm_from_iso"); + } + createVmDialog.find("#source_name").text(name); createVmDialog .dialog('option', 'buttons', { @@ -540,12 +563,22 @@ function showTemplatesTab() { isValid &= validateString("Name", thisDialog.find("#name"), thisDialog.find("#name_errormsg"), true); isValid &= validateString("Group", thisDialog.find("#group"), thisDialog.find("#group_errormsg"), true); if (!isValid) return; - - var name = trim(thisDialog.find("#name").val()); - var group = trim(thisDialog.find("#group").val()); - var serviceOfferingId = thisDialog.find("#service_offering").val(); + + thisDialog.dialog("close"); + + var array1 = []; + var name = trim(thisDialog.find("#name").val()); + array1.push("&displayname="+encodeURIComponent(name)); + + var group = trim(thisDialog.find("#group").val()); + array1.push("&group="+encodeURIComponent(group)); + + var serviceOfferingId = thisDialog.find("#service_offering").val(); + array1.push("&serviceOfferingId="+serviceOfferingId); + var diskOfferingId = thisDialog.find("#disk_offering").val(); - thisDialog.dialog("close"); + if(diskOfferingId != null && diskOfferingId.length > 0) + array1.push("&diskOfferingId="+diskOfferingId); var loadingImg = thisTemplate.find(".adding_loading"); var rowContainer = thisTemplate.find("#row_container"); @@ -554,7 +587,7 @@ function showTemplatesTab() { rowContainer.hide(); $.ajax({ - data: createURL("command=deployVirtualMachine&zoneId="+zoneId+"&serviceOfferingId="+serviceOfferingId+"&diskOfferingId="+diskOfferingId+"&templateId="+id+"&group="+encodeURIComponent(group)+"&displayname="+encodeURIComponent(name)+"&response=json"), + data: createURL("command=deployVirtualMachine&zoneId="+zoneId+"&templateId="+id+array1.join("")+"&response=json"), dataType: "json", success: function(json) { var jobId = json.deployvirtualmachineresponse.jobid; diff --git a/ui/test/scripts/cloud.core.test.js b/ui/test/scripts/cloud.core.test.js index 3f8cb95694d..e70cedcf53d 100644 --- a/ui/test/scripts/cloud.core.test.js +++ b/ui/test/scripts/cloud.core.test.js @@ -102,7 +102,7 @@ function trim(val) { // Prevent cross-site-script(XSS) attack. // used right before adding user input to the DOM tree. e.g. DOM_element.html(sanitizeXSS(user_input)); function sanitizeXSS(val) { - if(val == null) + if(val == null || typeof(val) != "string") return val; val = val.replace(//g, ">"); //replace > whose unicode is \u003e @@ -290,7 +290,7 @@ $(document).ready(function() { submenuContent.find("#grid_content").prepend(template.fadeIn("slow")); var username = thisDialog.find("#add_user_username").val(); - var password = $.md5(thisDialog.find("#add_user_password").val()); + var password = $.md5(encodeURIComponent(thisDialog.find("#add_user_password").val())); var email = thisDialog.find("#add_user_email").val(); if(email == "") email = username; @@ -318,7 +318,7 @@ $(document).ready(function() { $.ajax({ type: "POST", - data: createURL("command=createUser&username="+encodeURIComponent(username)+"&password="+encodeURIComponent(password)+"&email="+encodeURIComponent(email)+"&firstname="+encodeURIComponent(firstname)+"&lastname="+encodeURIComponent(lastname)+"&account="+account+"&accounttype="+accountType+"&domainid="+domainId+moreCriteria.join("")+"&response=json"), + data: createURL("command=createUser&username="+encodeURIComponent(username)+"&password="+password+"&email="+encodeURIComponent(email)+"&firstname="+encodeURIComponent(firstname)+"&lastname="+encodeURIComponent(lastname)+"&account="+account+"&accounttype="+accountType+"&domainid="+domainId+moreCriteria.join("")+"&response=json"), dataType: "json", async: false, success: function(json) { diff --git a/ui/wscript_build b/ui/wscript_build new file mode 100644 index 00000000000..ab3805ecef5 --- /dev/null +++ b/ui/wscript_build @@ -0,0 +1,41 @@ +import Utils, os + +# binary unsubstitutable files: +bld.install_files("${MSENVIRON}/webapps/client",bld.path.ant_glob("*.ico **/*png **/*jpg **/*gif",src=True,bld=False,dir=False,flat=True),cwd=bld.path,relative_trick=True) + +# text substitutable files (substitute with tokens from the environment bld.env): +bld.substitute('*html **/*html **/*js **/*css **/*properties **/*jsp *jsp',install_to="${MSENVIRON}/webapps/client") + +# -> minification of UI files +def minifyjs(task): + tgt = task.outputs[0].bldpath(task.env) + inputfiles = [] + outputfile = ['--js_output_file',tgt] + for inp in task.inputs: + src = inp.srcpath(task.env) + inputfiles.append(src) + newinputfiles = [] + for inputfile in inputfiles: + if inputfile not in newinputfiles: + newinputfiles.append('--js') + newinputfiles.append(inputfile) + compilerjar = os.path.join(bld.srcnode.abspath(),'tools','gcc','compiler.jar') + return Utils.exec_command(["java",'-jar',compilerjar] + newinputfiles + outputfile,log=True) + +javascripts = [ + ['scripts/jquery-1.4.2.min.js','scripts/date.js'], + Utils.to_list(bld.path.ant_glob('scripts/jquery*js')), + ['scripts/cloud.core.js','scripts/cloud.core.callbacks.js'], + Utils.to_list(bld.path.ant_glob('scripts/cloud*js')), +] +sourcefiles = [] +for lst in javascripts: + for x in lst: + if x not in sourcefiles: sourcefiles.append(x) +tgen = bld( + rule = minifyjs, + source = sourcefiles, + target = 'scripts/cloud.min.js', + name = 'minifyjs', +) +bld.install_files("${MSENVIRON}/webapps/client/scripts", "scripts/cloud.min.js") diff --git a/utils/bindir/cloud-gitrevs.in b/utils/bindir/cloud-gitrevs.in new file mode 100644 index 00000000000..ffa17be9bf8 --- /dev/null +++ b/utils/bindir/cloud-gitrevs.in @@ -0,0 +1,13 @@ +#!/usr/bin/env python + +import os, sys + +path = os.path.join("@DOCDIR@","sccs-info") +try: text = file(path).read(-1) +except IOError,e: + if e.errno == 2: + sys.stderr.write("error: SCCS info file %r cannot be found\n"%path) + sys.exit(1) + else: raise +lines = [ s.strip() for s in text.split("\n") if s.startswith('Git Revision: ') ] +print " ".join( [ s[14:] for s in lines ] ) \ No newline at end of file diff --git a/utils/bindir/cloud-sccs.in b/utils/bindir/cloud-sccs.in new file mode 100755 index 00000000000..cfd40d5fb58 --- /dev/null +++ b/utils/bindir/cloud-sccs.in @@ -0,0 +1,12 @@ +#!/usr/bin/env python + +import os, sys + +path = os.path.join("@DOCDIR@","sccs-info") +try: text = file(path).read(-1) +except IOError,e: + if e.errno == 2: + sys.stderr.write("error: SCCS info file %r cannot be found\n"%path) + sys.exit(1) + else: raise +print text \ No newline at end of file diff --git a/utils/src/com/cloud/utils/CloudResourceBundle.java b/utils/src/com/cloud/utils/CloudResourceBundle.java new file mode 100755 index 00000000000..69e77275928 --- /dev/null +++ b/utils/src/com/cloud/utils/CloudResourceBundle.java @@ -0,0 +1,29 @@ +package com.cloud.utils; + +import java.util.Locale; +import java.util.ResourceBundle; + +public class CloudResourceBundle { + + private ResourceBundle _bundle; + + public CloudResourceBundle(ResourceBundle bundle) { + _bundle = bundle; + } + + public static CloudResourceBundle getBundle(String baseName, Locale locale) { + return new CloudResourceBundle(ResourceBundle.getBundle(baseName, locale)); + } + + private String getString(String key) { + try { + return _bundle.getString(key); + } catch(Exception e) { + return key; //if translation is not found, just return original word (i.e. English). + } + } + + public String t(String key) { + return getString(key); + } +} diff --git a/utils/src/com/cloud/utils/DateUtil.java b/utils/src/com/cloud/utils/DateUtil.java index 436bb8df231..d9bbad4592b 100644 --- a/utils/src/com/cloud/utils/DateUtil.java +++ b/utils/src/com/cloud/utils/DateUtil.java @@ -18,6 +18,7 @@ package com.cloud.utils; +import java.net.URI; import java.text.DateFormat; import java.text.ParseException; import java.text.SimpleDateFormat; @@ -235,14 +236,12 @@ public class DateUtil { throw new CloudRuntimeException("Incorrect interval: "+type.toString()); } - return scheduleTime.getTime(); } // test only public static void main(String[] args) { - TimeZone localTimezone = Calendar.getInstance().getTimeZone(); TimeZone gmtTimezone = TimeZone.getTimeZone("GMT"); TimeZone estTimezone = TimeZone.getTimeZone("EST"); diff --git a/utils/src/com/cloud/utils/EnumerationImpl.java b/utils/src/com/cloud/utils/EnumerationImpl.java index a30ffefa5bc..c8c7e417399 100755 --- a/utils/src/com/cloud/utils/EnumerationImpl.java +++ b/utils/src/com/cloud/utils/EnumerationImpl.java @@ -26,7 +26,7 @@ import java.util.Iterator; * * Why the heck didn't Iterator extend from Enumeration, I will probably never know. Tell me Lee Boyton! **/ -public class EnumerationImpl implements Enumeration { +public class EnumerationImpl implements Enumeration, Iterator { Iterator _it; // Can't use this. @@ -37,11 +37,28 @@ public class EnumerationImpl implements Enumeration { _it = it; } + @Override public boolean hasMoreElements() { return _it.hasNext(); } + @Override public T nextElement() { return _it.next(); } + + @Override + public boolean hasNext() { + return _it.hasNext(); + } + + @Override + public T next() { + return _it.next(); + } + + @Override + public void remove() { + throw new UnsupportedOperationException("Enumerations do not support remove operation"); + } } diff --git a/utils/src/com/cloud/utils/Journal.java b/utils/src/com/cloud/utils/Journal.java new file mode 100644 index 00000000000..bbf721e4893 --- /dev/null +++ b/utils/src/com/cloud/utils/Journal.java @@ -0,0 +1,88 @@ +/** + * + */ +package com.cloud.utils; + +import java.util.ArrayList; + +import org.apache.log4j.Level; +import org.apache.log4j.Logger; + +/** + * Journal is used to kept what has happened during a process so someone can track + * what happens during a process. + * + */ +public class Journal { + String _name; + ArrayList> _entries; + + public Journal(String name) { + _name = name; + _entries = new ArrayList>(); + } + + + final private void log(String msg, Object... params) { + Pair entry = new Pair(msg, params); + assert msg != null : "Message can not be null or else it's useless!"; + _entries.add(entry); + } + + public void record(String msg, Object... params) { + log(msg, params); + } + + public void record(Logger logger, Level p, String msg, Object... params) { + if (logger.isEnabledFor(p)) { + StringBuilder buf = new StringBuilder(); + toString(buf, msg, params); + String entry = buf.toString(); + log(entry); + logger.log(p, entry); + } else { + log(msg, params); + } + } + + protected void toString(StringBuilder buf, String msg, Object[] params) { + buf.append(msg); + if (params != null) { + buf.append(" - "); + int i = 0; + for (Object obj : params) { + buf.append('P').append(i).append('='); + buf.append(obj != null ? obj.toString() : "null"); + buf.append(", "); + } + buf.delete(buf.length() - 2, buf.length()); + } + } + + public String toString(String separator) { + StringBuilder buf = new StringBuilder(_name).append(": "); + for (Pair entry : _entries) { + toString(buf, entry.first(), entry.second()); + buf.append(separator); + } + return buf.toString(); + } + + @Override + public String toString() { + return toString("; "); + } + + public static class LogJournal extends Journal { + Logger _logger; + public LogJournal(String name, Logger logger) { + super(name); + _logger = logger; + } + + @Override + public void record(String msg, Object... params) { + record(_logger, Level.DEBUG, msg, params); + } + } +} diff --git a/utils/src/com/cloud/utils/SerialVersionUID.java b/utils/src/com/cloud/utils/SerialVersionUID.java index 6c26f4891e7..be9765bf87c 100755 --- a/utils/src/com/cloud/utils/SerialVersionUID.java +++ b/utils/src/com/cloud/utils/SerialVersionUID.java @@ -55,6 +55,7 @@ public interface SerialVersionUID { public static final long StorageUnavailableException = Base | 0x19; public static final long InfficientVirtualNetworkCapacityException = Base | 0x1a; public static final long DiscoveryException = Base | 0x1b; - public static final long CloudAuthenticationException = Base | 0x1c; - public static final long AsyncCommandQueued = Base | 0x1d; + public static final long ConflictingNetworkSettingException = Base | 0x1c; + public static final long CloudAuthenticationException = Base | 0x1d; + public static final long AsyncCommandQueued = Base | 0x1e; } diff --git a/utils/src/com/cloud/utils/component/Adapters.java b/utils/src/com/cloud/utils/component/Adapters.java index e1b504ed535..86b06b85a5e 100755 --- a/utils/src/com/cloud/utils/component/Adapters.java +++ b/utils/src/com/cloud/utils/component/Adapters.java @@ -17,8 +17,12 @@ */ package com.cloud.utils.component; +import java.util.Collection; import java.util.Enumeration; +import java.util.HashMap; +import java.util.Iterator; import java.util.List; +import java.util.Map; import com.cloud.utils.EnumerationImpl; @@ -30,13 +34,13 @@ import com.cloud.utils.EnumerationImpl; * the iterator even during dynamic reloading. * **/ -public class Adapters { - private List _adapters; +public class Adapters implements Iterable { + private Map _map; protected String _name; public Adapters(String name, List adapters) { _name = name; - _adapters = adapters; + set(adapters); } /** @@ -49,18 +53,31 @@ public class Adapters { } public Enumeration enumeration() { - return new EnumerationImpl(_adapters.iterator()); + return new EnumerationImpl(_map.values().iterator()); + } + + @Override + public Iterator iterator() { + return new EnumerationImpl(_map.values().iterator()); } - protected List get() { - return _adapters; + protected Collection get() { + return _map.values(); } protected void set(List adapters) { - this._adapters = adapters; + HashMap map = new HashMap(adapters.size()); + for (T adapter : adapters) { + map.put(adapter.getName(), adapter); + } + this._map = map; + } + + public T get(String name) { + return _map.get(name); } public boolean isSet() { - return _adapters.size() != 0; + return _map.size() != 0; } -} +} \ No newline at end of file diff --git a/utils/src/com/cloud/utils/component/ComponentLocator.java b/utils/src/com/cloud/utils/component/ComponentLocator.java index aad5619850f..a9a6a314f27 100755 --- a/utils/src/com/cloud/utils/component/ComponentLocator.java +++ b/utils/src/com/cloud/utils/component/ComponentLocator.java @@ -100,7 +100,7 @@ public class ComponentLocator extends Thread implements ComponentLocatorMBean { } @Override - public synchronized void run() { + public synchronized void run() { Iterator> itAdapters = _adapterMap.values().iterator(); while (itAdapters.hasNext()) { Adapters adapters = itAdapters.next(); @@ -359,6 +359,8 @@ public class ComponentLocator extends Thread implements ComponentLocatorMBean { instance = locator.getManager(fc); } else if (GenericDao.class.isAssignableFrom(fc)) { instance = locator.getDao(fc); + } else if (Adapters.class.isAssignableFrom(fc)) { + instance = locator.getAdapters(inject.adapter()); } if (instance == null) { @@ -441,7 +443,7 @@ public class ComponentLocator extends Thread implements ComponentLocatorMBean { Set>>> entries = map.entrySet(); for (Map.Entry>> entry : entries) { Adapters adapters = (Adapters)_adapterMap.get(entry.getKey()); - List lst = adapters.get(); + List lst = new ArrayList(); for (Info info : entry.getValue()) { s_logger.info("Instantiating Adapter: " + info.name); info.instance = (Adapter)createInstance(info.clazz, true); @@ -460,6 +462,7 @@ public class ComponentLocator extends Thread implements ComponentLocatorMBean { lst.add(info.instance); s_logger.info("Instantiated Adapter: " + info.name); } + adapters.set(lst); } } @@ -487,6 +490,7 @@ public class ComponentLocator extends Thread implements ComponentLocatorMBean { } } + @Override public String getParentName() { return _parentLocator != null ? _parentLocator.getName() : "None"; } @@ -499,6 +503,7 @@ public class ComponentLocator extends Thread implements ComponentLocatorMBean { return (T)createInstance(clazz, true, args); } + @Override public Map> getAdapters() { HashMap> result = new HashMap>(); for (Map.Entry> entry : _adapterMap.entrySet()) { @@ -522,6 +527,7 @@ public class ComponentLocator extends Thread implements ComponentLocatorMBean { return parentResults; } + @Override public Collection getManagers() { Collection names = _parentLocator != null ? _parentLocator.getManagers() : new HashSet(); for (Map.Entry> entry : _managerMap.entrySet()) { @@ -530,6 +536,7 @@ public class ComponentLocator extends Thread implements ComponentLocatorMBean { return names; } + @Override public Collection getDaos() { Collection names = _parentLocator != null ? _parentLocator.getDaos() : new HashSet(); for (Map.Entry>> entry : _daoMap.entrySet()) { diff --git a/utils/src/com/cloud/utils/component/Inject.java b/utils/src/com/cloud/utils/component/Inject.java index cff0b6cb67f..8c948d75e43 100644 --- a/utils/src/com/cloud/utils/component/Inject.java +++ b/utils/src/com/cloud/utils/component/Inject.java @@ -26,4 +26,5 @@ import java.lang.annotation.Target; @Target(FIELD) @Retention(RUNTIME) public @interface Inject { + Class adapter() default Adapter.class; } diff --git a/utils/src/com/cloud/utils/component/Injector.java b/utils/src/com/cloud/utils/component/Injector.java new file mode 100644 index 00000000000..e278ab5ba08 --- /dev/null +++ b/utils/src/com/cloud/utils/component/Injector.java @@ -0,0 +1,19 @@ +/** + * + */ +package com.cloud.utils.component; + +import java.lang.annotation.Annotation; +import java.lang.reflect.AnnotatedElement; + +/** + * Injector implements customized Injectors for ComponentLocator. + * + */ +public interface Injector { + /** + * Can this injector handle injecting into this type of class? + */ + boolean canInject(AnnotatedElement element, Annotation ann); + +} diff --git a/utils/src/com/cloud/utils/concurrency/SynchronizationEvent.java b/utils/src/com/cloud/utils/concurrency/SynchronizationEvent.java new file mode 100644 index 00000000000..93da2b093ef --- /dev/null +++ b/utils/src/com/cloud/utils/concurrency/SynchronizationEvent.java @@ -0,0 +1,86 @@ +/** + * Copyright (C) 2010 Cloud.com, Inc. All rights reserved. + * + * This software is licensed under the GNU General Public License v3 or later. + * + * It is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or any later version. + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + */ +package com.cloud.utils.concurrency; + +import org.apache.log4j.Logger; + +public class SynchronizationEvent { + protected final static Logger s_logger = Logger.getLogger(SynchronizationEvent.class); + + private boolean signalled; + + public SynchronizationEvent() { + signalled = false; + } + + public SynchronizationEvent(boolean signalled) { + this.signalled = signalled; + } + + public void setEvent() { + synchronized(this) { + signalled = true; + notifyAll(); + } + } + + public void resetEvent() { + synchronized(this) { + signalled = false; + } + } + + public boolean waitEvent() { + synchronized(this) { + if(signalled) + return true; + + while(true) { + try { + wait(); + assert(signalled); + return signalled; + } catch (InterruptedException e) { + s_logger.debug("unexpected awaken signal in wait()"); + } + } + } + } + + public boolean waitEvent(long timeOutMiliseconds) { + synchronized(this) { + if(signalled) + return true; + + try { + wait(timeOutMiliseconds); + return signalled; + } catch (InterruptedException e) { + // TODO, we don't honor time out semantics when the waiting thread is interrupted + s_logger.debug("unexpected awaken signal in wait(...)"); + return false; + } + } + } + + public boolean isSignalled() { + synchronized(this) { + return signalled; + } + } +} diff --git a/utils/src/com/cloud/utils/db/GenericDao.java b/utils/src/com/cloud/utils/db/GenericDao.java index c74dea10259..da785808dd6 100755 --- a/utils/src/com/cloud/utils/db/GenericDao.java +++ b/utils/src/com/cloud/utils/db/GenericDao.java @@ -101,7 +101,7 @@ public interface GenericDao { * @return object if acquired; null if not. If null, you need to call findById to see if it is actually not found. */ T acquire(ID id); - + /** * Acquires a database wide lock on the id of the entity. This ensures * that only one is being used. The timeout is the configured default. @@ -125,14 +125,14 @@ public interface GenericDao { * Look for all active rows. * @return list of entity beans. */ - List listAllActive(); + List listAll(); /** * Look for all active rows. * @param filter filter to limit the results * @return list of entity beans. */ - List listAllActive(Filter filter); + List listAll(Filter filter); /** @@ -143,7 +143,7 @@ public interface GenericDao { */ List search(SearchCriteria sc, Filter filter); - List searchAll(SearchCriteria sc, final Filter filter, final Boolean lock, final boolean cache); + List searchIncludingRemoved(SearchCriteria sc, final Filter filter, final Boolean lock, final boolean cache); /** * Customized search with SearchCritiria @@ -151,20 +151,20 @@ public interface GenericDao { * @param filter * @return list of entity beans. */ - public List searchAll(SearchCriteria sc, Filter filter); + public List searchIncludingRemoved(SearchCriteria sc, Filter filter); /** * Retrieves the entire table. * @return collection of entity beans. **/ - List listAll(); + List listAllIncludingRemoved(); /** * Retrieves the entire table. * @param filter filter to limit the returns. * @return collection of entity beans. **/ - List listAll(Filter filter); + List listAllIncludingRemoved(Filter filter); /** * Persist the entity bean. The id field of the entity is updated with @@ -184,18 +184,26 @@ public interface GenericDao { boolean remove(ID id); /** - * remove the entity bean. + * Remove based on the search criteria. This will delete if the VO object + * does not have a REMOVED column. + * @param sc search criteria to match + * @return rows removed. + */ + int remove(SearchCriteria sc); + + /** + * Expunge actually delete the row even if it's REMOVED. * @param id * @return true if removed. */ - boolean delete(ID id); + boolean expunge(ID id); /** * remove the entity bean specified by the search criteria * @param sc * @return number of rows deleted */ - int delete(final SearchCriteria sc); + int expunge(final SearchCriteria sc); /** * expunge the removed rows. @@ -212,5 +220,4 @@ public interface GenericDao { */ boolean configure(String name, Map params) throws ConfigurationException; - int remove(SearchCriteria sc); } \ No newline at end of file diff --git a/utils/src/com/cloud/utils/db/GenericDaoBase.java b/utils/src/com/cloud/utils/db/GenericDaoBase.java index 641bd55fe43..54e40da4fbc 100755 --- a/utils/src/com/cloud/utils/db/GenericDaoBase.java +++ b/utils/src/com/cloud/utils/db/GenericDaoBase.java @@ -21,6 +21,10 @@ import java.io.Serializable; import java.lang.reflect.Field; import java.lang.reflect.ParameterizedType; import java.lang.reflect.Type; +import java.net.MalformedURLException; +import java.net.URI; +import java.net.URISyntaxException; +import java.net.URL; import java.sql.Connection; import java.sql.PreparedStatement; import java.sql.ResultSet; @@ -45,8 +49,11 @@ import javax.persistence.EnumType; import javax.persistence.Enumerated; import javax.persistence.TableGenerator; +import net.sf.cglib.proxy.Callback; +import net.sf.cglib.proxy.CallbackFilter; import net.sf.cglib.proxy.Enhancer; import net.sf.cglib.proxy.Factory; +import net.sf.cglib.proxy.NoOp; import net.sf.ehcache.Cache; import net.sf.ehcache.CacheManager; import net.sf.ehcache.Element; @@ -126,8 +133,11 @@ public abstract class GenericDaoBase implements Gene protected final Map, Attribute> _allColumns; protected Enhancer _enhancer; protected Factory _factory; + protected Enhancer _searchEnhancer; protected int _timeoutSeconds; + protected final static CallbackFilter s_callbackFilter = new UpdateFilter(); + protected static final String FOR_UPDATE_CLAUSE = " FOR UPDATE "; protected static final String SHARE_MODE_CLAUSE = " LOCK IN SHARE MODE"; protected static final String SELECT_LAST_INSERT_ID_SQL = "SELECT LAST_INSERT_ID()"; @@ -157,9 +167,10 @@ public abstract class GenericDaoBase implements Gene return dao; } + @Override @SuppressWarnings("unchecked") public GenericSearchBuilder createSearchBuilder(Class resultType) { - final T entity = (T)_enhancer.create(); + final T entity = (T)_searchEnhancer.create(); final Factory factory = (Factory)entity; GenericSearchBuilder builder = new GenericSearchBuilder(entity, resultType, _allAttributes); factory.setCallback(0, builder); @@ -210,11 +221,18 @@ public abstract class GenericDaoBase implements Gene _tgs.put(tg.name(), tg); } + Callback[] callbacks = new Callback[] { NoOp.INSTANCE, new UpdateBuilder(_allAttributes) }; + _enhancer = new Enhancer(); _enhancer.setSuperclass(_entityBeanType); - _enhancer.setCallback(new UpdateBuilder(_allAttributes)); + _enhancer.setCallbackFilter(s_callbackFilter); + _enhancer.setCallbacks(callbacks); _factory = (Factory)_enhancer.create(); + _searchEnhancer = new Enhancer(); + _searchEnhancer.setSuperclass(_entityBeanType); + _searchEnhancer.setCallback(new UpdateBuilder(_allAttributes)); + if (s_logger.isTraceEnabled()) { s_logger.trace("Select SQL: " + _partialSelectSql.first().toString()); s_logger.trace("Remove SQL: " + (_removeSql != null ? _removeSql.first() : "No remove sql")); @@ -235,8 +253,7 @@ public abstract class GenericDaoBase implements Gene @Override @DB(txn=false) @SuppressWarnings("unchecked") public T createForUpdate(final ID id) { - final T entity = (T)_enhancer.create(); - final Factory factory = (Factory)entity; + final T entity = (T)_factory.newInstance(new Callback[] {NoOp.INSTANCE, new UpdateBuilder(_allAttributes)}); if (id != null) { try { _idField.set(entity, id); @@ -244,7 +261,6 @@ public abstract class GenericDaoBase implements Gene } catch (final IllegalAccessException e) { } } - factory.setCallback(0, new UpdateBuilder(_allAttributes)); return entity; } @@ -254,7 +270,6 @@ public abstract class GenericDaoBase implements Gene } @Override - @SuppressWarnings("unchecked") public K getNextInSequence(final Class clazz, final String name) { final TableGenerator tg = _tgs.get(name); assert (tg != null) : "Couldn't find Table generator using " + name; @@ -282,10 +297,11 @@ public abstract class GenericDaoBase implements Gene } sc.addAnd(_removed.second().field.getName(), SearchCriteria.Op.NULL); } - return searchAll(sc, filter, lock, cache); + return searchIncludingRemoved(sc, filter, lock, cache); } - public List searchAll(SearchCriteria sc, final Filter filter, final Boolean lock, final boolean cache) { + @Override + public List searchIncludingRemoved(SearchCriteria sc, final Filter filter, final Boolean lock, final boolean cache) { String clause = sc != null ? sc.getWhereClause() : null; if (clause != null && clause.length() == 0) { clause = null; @@ -296,7 +312,7 @@ public abstract class GenericDaoBase implements Gene str.append(clause); } - Collection, Attribute, Attribute>> joins = null; + Collection>> joins = null; if (sc != null) { joins = sc.getJoins(); if (joins != null) { @@ -325,9 +341,10 @@ public abstract class GenericDaoBase implements Gene prepareAttribute(++i, pstmt, value.first(), value.second()); } } + if (joins != null) { - for (Ternary, Attribute, Attribute> join : joins) { - for (final Pair value : join.first().getValues()) { + for (JoinBuilder> join : joins) { + for (final Pair value : join.getT().getValues()) { prepareAttribute(++i, pstmt, value.first(), value.second()); } } @@ -351,7 +368,7 @@ public abstract class GenericDaoBase implements Gene } @Override @SuppressWarnings("unchecked") @DB - public List searchAll(SearchCriteria sc, final Filter filter) { + public List searchIncludingRemoved(SearchCriteria sc, final Filter filter) { String clause = sc != null ? sc.getWhereClause() : null; if (clause != null && clause.length() == 0) { clause = null; @@ -362,7 +379,7 @@ public abstract class GenericDaoBase implements Gene str.append(clause); } - Collection, Attribute, Attribute>> joins = null; + Collection>> joins = null; if (sc != null) { joins = sc.getJoins(); if (joins != null) { @@ -377,7 +394,6 @@ public abstract class GenericDaoBase implements Gene final Transaction txn = Transaction.currentTxn(); PreparedStatement pstmt = s_initStmt; - final List result = new ArrayList(); try { pstmt = txn.prepareAutoCloseStatement(sql); int i = 0; @@ -386,9 +402,10 @@ public abstract class GenericDaoBase implements Gene prepareAttribute(++i, pstmt, value.first(), value.second()); } } + if (joins != null) { - for (Ternary, Attribute, Attribute> join : joins) { - for (final Pair value : join.first().getValues()) { + for (JoinBuilder> join : joins) { + for (final Pair value : join.getT().getValues()) { prepareAttribute(++i, pstmt, value.first(), value.second()); } } @@ -396,9 +413,9 @@ public abstract class GenericDaoBase implements Gene s_logger.trace("join search statement is " + pstmt.toString()); } } + ResultSet rs = pstmt.executeQuery(); SelectType st = sc.getSelectType(); - ResultSetMetaData md = rs.getMetaData(); ArrayList results = new ArrayList(); List fields = sc.getSelectFields(); while (rs.next()) { @@ -429,14 +446,6 @@ public abstract class GenericDaoBase implements Gene final Class type = field.getType(); if (type == String.class) { field.set(entity, rs.getString(index)); - } else if (type == int.class) { - field.set(entity, rs.getInt(index)); - } else if (type == Integer.class) { - if (rs.getObject(index) == null) { - field.set(entity, null); - } else { - field.set(entity, rs.getInt(index)); - } } else if (type == long.class) { field.setLong(entity, rs.getLong(index)); } else if (type == Long.class) { @@ -445,6 +454,26 @@ public abstract class GenericDaoBase implements Gene } else { field.set(entity, rs.getLong(index)); } + } else if (type.isEnum()) { + final Enumerated enumerated = field.getAnnotation(Enumerated.class); + final EnumType enumType = (enumerated == null) ? EnumType.STRING : enumerated.value(); + + final Enum[] enums = (Enum[])field.getType().getEnumConstants(); + for (final Enum e : enums) { + if ((enumType == EnumType.STRING && e.name().equalsIgnoreCase(rs.getString(index))) || + (enumType == EnumType.ORDINAL && e.ordinal() == rs.getInt(index))) { + field.set(entity, e); + return; + } + } + } else if (type == int.class) { + field.set(entity, rs.getInt(index)); + } else if (type == Integer.class) { + if (rs.getObject(index) == null) { + field.set(entity, null); + } else { + field.set(entity, rs.getInt(index)); + } } else if (type == Date.class) { final Object data = rs.getDate(index); if (data == null) { @@ -452,14 +481,15 @@ public abstract class GenericDaoBase implements Gene return; } field.set(entity, DateUtil.parseDateString(s_gmtTimeZone, rs.getString(index))); - } else if (type == short.class) { - field.setShort(entity, rs.getShort(index)); - } else if (type == Short.class) { - if (rs.getObject(index) == null) { + } else if (type == Calendar.class) { + final Object data = rs.getDate(index); + if (data == null) { field.set(entity, null); - } else { - field.set(entity, rs.getShort(index)); + return; } + final Calendar cal = Calendar.getInstance(); + cal.setTime(DateUtil.parseDateString(s_gmtTimeZone, rs.getString(index))); + field.set(entity, cal); } else if (type == boolean.class) { field.setBoolean(entity, rs.getBoolean(index)); } else if (type == Boolean.class) { @@ -468,6 +498,28 @@ public abstract class GenericDaoBase implements Gene } else { field.set(entity, rs.getBoolean(index)); } + } else if (type == URI.class) { + try { + String str = rs.getString(index); + field.set(entity, str == null ? null : new URI(str)); + } catch (URISyntaxException e) { + throw new CloudRuntimeException("Invalid URI: " + rs.getString(index), e); + } + } else if (type == URL.class) { + try { + String str = rs.getString(index); + field.set(entity, str != null ? new URL(str) : null); + } catch (MalformedURLException e) { + throw new CloudRuntimeException("Invalid URL: " + rs.getString(index), e); + } + } else if (type == short.class) { + field.setShort(entity, rs.getShort(index)); + } else if (type == Short.class) { + if (rs.getObject(index) == null) { + field.set(entity, null); + } else { + field.set(entity, rs.getShort(index)); + } } else if (type == float.class) { field.setFloat(entity, rs.getFloat(index)); } else if (type == Float.class) { @@ -492,27 +544,6 @@ public abstract class GenericDaoBase implements Gene } else { field.set(entity, rs.getByte(index)); } - } else if (type == Calendar.class) { - final Object data = rs.getDate(index); - if (data == null) { - field.set(entity, null); - return; - } - final Calendar cal = Calendar.getInstance(); - cal.setTime(DateUtil.parseDateString(s_gmtTimeZone, rs.getString(index))); - field.set(entity, cal); - } else if (type.isEnum()) { - final Enumerated enumerated = field.getAnnotation(Enumerated.class); - final EnumType enumType = (enumerated == null) ? EnumType.STRING : enumerated.value(); - - final Enum[] enums = (Enum[])field.getType().getEnumConstants(); - for (final Enum e : enums) { - if ((enumType == EnumType.STRING && e.name().equalsIgnoreCase(rs.getString(index))) || - (enumType == EnumType.ORDINAL && e.ordinal() == rs.getInt(index))) { - field.set(entity, e); - return; - } - } } else if (type == byte[].class) { field.set(entity, rs.getBytes(index)); } else { @@ -626,7 +657,7 @@ public abstract class GenericDaoBase implements Gene @DB(txn=false) protected int update(final ID id, final UpdateBuilder ub) { - SearchCriteria sc = createSearchCriteria(); + SearchCriteria sc = createSearchCriteria(); sc.addAnd(_idAttributes.get(_table)[0], SearchCriteria.Op.EQ, id); int rowsUpdated = update(ub, sc, null); if (_cache != null) { @@ -711,37 +742,27 @@ public abstract class GenericDaoBase implements Gene return _entityBeanType; } - protected T findOneBy(final SearchCriteria sc) { + protected T findOneIncludingRemovedBy(final SearchCriteria sc) { Filter filter = new Filter(1); - List results = searchAll(sc, filter, null, false); + List results = searchIncludingRemoved(sc, filter, null, false); assert results.size() <= 1 : "Didn't the limiting worked?"; return results.size() == 0 ? null : results.get(0); } @DB(txn=false) - protected T findOneActiveBy(final SearchCriteria sc) { + protected T findOneBy(final SearchCriteria sc) { if (_removed != null) { sc.addAnd(_removed.second().field.getName(), SearchCriteria.Op.NULL); } - return findOneBy(sc); - } - - @DB(txn=false) - protected List listActiveBy(final SearchCriteria sc, final Filter filter) { - if (_removed != null) { - sc.addAnd(_removed.second().field.getName(), SearchCriteria.Op.NULL); - } - return listBy(sc, filter); - } - - @DB(txn=false) - protected List listActiveBy(final SearchCriteria sc) { - return listActiveBy(sc, null); + return findOneIncludingRemovedBy(sc); } @DB(txn=false) protected List listBy(final SearchCriteria sc, final Filter filter) { - return searchAll(sc, filter, null, false); + if (_removed != null) { + sc.addAnd(_removed.second().field.getName(), SearchCriteria.Op.NULL); + } + return listIncludingRemovedBy(sc, filter); } @DB(txn=false) @@ -749,6 +770,16 @@ public abstract class GenericDaoBase implements Gene return listBy(sc, null); } + @DB(txn=false) + protected List listIncludingRemovedBy(final SearchCriteria sc, final Filter filter) { + return searchIncludingRemoved(sc, filter, null, false); + } + + @DB(txn=false) + protected List listIncludingRemovedBy(final SearchCriteria sc) { + return listIncludingRemovedBy(sc, null); + } + @Override @DB(txn=false) @SuppressWarnings("unchecked") public T findById(final ID id) { @@ -761,7 +792,6 @@ public abstract class GenericDaoBase implements Gene } @Override @DB(txn=false) - @SuppressWarnings("unchecked") public T findById(final ID id, boolean fresh) { if(!fresh) return findById(id); @@ -801,6 +831,7 @@ public abstract class GenericDaoBase implements Gene return acquire(id, _timeoutSeconds); } + @Override public T acquire(final ID id, int seconds) { Transaction txn = Transaction.currentTxn(); T t = null; @@ -827,8 +858,8 @@ public abstract class GenericDaoBase implements Gene } @Override @DB(txn=false) - public List listAll() { - return listAll(null); + public List listAllIncludingRemoved() { + return listAllIncludingRemoved(null); } @DB(txn=false) @@ -862,7 +893,7 @@ public abstract class GenericDaoBase implements Gene } @Override @DB(txn=false) - public List listAll(final Filter filter) { + public List listAllIncludingRemoved(final Filter filter) { final StringBuilder sql = createPartialSelectSql(null, false); addFilter(sql, filter); @@ -893,14 +924,14 @@ public abstract class GenericDaoBase implements Gene } @Override @DB(txn=false) - public List listAllActive() { - return listAllActive(null); + public List listAll() { + return listAll(null); } @Override @DB(txn=false) - public List listAllActive(final Filter filter) { + public List listAll(final Filter filter) { if (_removed == null) { - return listAll(filter); + return listAllIncludingRemoved(filter); } final StringBuilder sql = createPartialSelectSql(null, true); @@ -911,7 +942,7 @@ public abstract class GenericDaoBase implements Gene } @Override - public boolean delete(final ID id) { + public boolean expunge(final ID id) { final Transaction txn = Transaction.currentTxn(); PreparedStatement pstmt = s_initStmt; String sql = null; @@ -941,7 +972,8 @@ public abstract class GenericDaoBase implements Gene } // FIXME: Does not work for joins. - public int delete(final SearchCriteria sc) { + @Override + public int expunge(final SearchCriteria sc) { final StringBuilder str = new StringBuilder("DELETE FROM "); str.append(_table); str.append(" WHERE "); @@ -984,25 +1016,28 @@ public abstract class GenericDaoBase implements Gene } @DB(txn=false) - protected void addJoins(StringBuilder str, Collection, Attribute, Attribute>> joins) { - int fromIndex = str.lastIndexOf("FROM") + 5; - if (str.lastIndexOf("WHERE") == -1) { + protected void addJoins(StringBuilder str, Collection>> joins) { + int fromIndex = str.lastIndexOf("WHERE"); + if (fromIndex == -1) { + fromIndex = str.length(); str.append(" WHERE "); } else { str.append(" AND "); } - - for (Ternary, Attribute, Attribute> join : joins) { - str.insert(fromIndex, join.third().table + ", "); - str.append(join.second().table).append(".").append(join.second().columnName).append("=").append(join.third().table).append(".").append(join.third().columnName); - str.append(" AND (").append(join.first().getWhereClause()).append(") AND "); - } - - for (Ternary, Attribute, Attribute> join : joins) { - if (join.first().getJoins() != null) { - addJoins(str, join.first().getJoins()); - } + + for (JoinBuilder> join : joins) { + StringBuilder onClause = new StringBuilder(); + onClause.append(" ").append(join.getType().getName()).append(" ").append(join.getSecondAttribute().table).append(" ON ").append(join.getFirstAttribute().table).append(".").append(join.getFirstAttribute().columnName).append("=").append(join.getSecondAttribute().table).append(".").append(join.getSecondAttribute().columnName).append(" "); + str.insert(fromIndex, onClause); + str.append(" (").append(join.getT().getWhereClause()).append(") AND "); + fromIndex+=onClause.length(); + } + + for (JoinBuilder> join : joins) { + if (join.getT().getJoins() != null) { + addJoins(str, join.getT().getJoins()); } + } str.delete(str.length() - 4, str.length()); } @@ -1176,8 +1211,12 @@ public abstract class GenericDaoBase implements Gene if (type == EnumType.STRING) { pstmt.setString(j, value == null ? null : value.toString()); } else if (type == EnumType.ORDINAL) { - pstmt.setInt(j, value == null ? null : ((Enum)value).ordinal()); + pstmt.setInt(j, value == null ? null : ((Enum)value).ordinal()); } + } else if (attr.field.getType() == URI.class) { + pstmt.setString(j, value == null ? null : value.toString()); + } else if (attr.field.getType() == URL.class) { + pstmt.setURL(j, (URL)value); } else if (attr.field.getType() == byte[].class) { pstmt.setBytes(j, (byte[])value); } else { @@ -1204,7 +1243,7 @@ public abstract class GenericDaoBase implements Gene @SuppressWarnings("unchecked") @DB(txn=false) protected T toEntityBean(final ResultSet result, final boolean cache) throws SQLException { - final T entity = (T)_factory.newInstance(new UpdateBuilder(_allAttributes)); + final T entity = (T)_factory.newInstance(new Callback[] {NoOp.INSTANCE, new UpdateBuilder(_allAttributes)}); toEntityBean(result, entity); @@ -1258,7 +1297,7 @@ public abstract class GenericDaoBase implements Gene @Override public boolean remove(final ID id) { if (_removeSql == null) { - return delete(id); + return expunge(id); } final Transaction txn = Transaction.currentTxn(); @@ -1288,7 +1327,7 @@ public abstract class GenericDaoBase implements Gene @Override public int remove(SearchCriteria sc) { if (_removeSql == null) { - return delete(sc); + return expunge(sc); } T vo = createForUpdate(); @@ -1326,7 +1365,7 @@ public abstract class GenericDaoBase implements Gene createCache(params); final boolean load = Boolean.parseBoolean((String)params.get("cache.preload")); if (load) { - listAllActive(); + listAll(); } return true; @@ -1341,13 +1380,13 @@ public abstract class GenericDaoBase implements Gene public static UpdateBuilder getUpdateBuilder(final T entityObject) { final Factory factory = (Factory)entityObject; assert(factory != null); - return (UpdateBuilder)factory.getCallback(0); + return (UpdateBuilder)factory.getCallback(1); } @SuppressWarnings("unchecked") @Override @DB(txn=false) public SearchBuilder createSearchBuilder() { - final T entity = (T)_enhancer.create(); + final T entity = (T)_searchEnhancer.create(); final Factory factory = (Factory)entity; SearchBuilder builder = new SearchBuilder(entity, _allAttributes); factory.setCallback(0, builder); @@ -1359,4 +1398,5 @@ public abstract class GenericDaoBase implements Gene SearchBuilder builder = createSearchBuilder(); return builder.create(); } + } diff --git a/utils/src/com/cloud/utils/db/GenericSearchBuilder.java b/utils/src/com/cloud/utils/db/GenericSearchBuilder.java index f7931fb8cb8..11609eb2f3d 100644 --- a/utils/src/com/cloud/utils/db/GenericSearchBuilder.java +++ b/utils/src/com/cloud/utils/db/GenericSearchBuilder.java @@ -27,7 +27,6 @@ import net.sf.cglib.proxy.Factory; import net.sf.cglib.proxy.MethodInterceptor; import net.sf.cglib.proxy.MethodProxy; -import com.cloud.utils.Ternary; import com.cloud.utils.db.SearchCriteria.Func; import com.cloud.utils.db.SearchCriteria.Op; import com.cloud.utils.db.SearchCriteria.SelectType; @@ -43,11 +42,11 @@ import com.cloud.utils.exception.CloudRuntimeException; * @param VO object this Search is build for. * @param Result object that should contain the results. */ -public class GenericSearchBuilder implements DaoSearch, MethodInterceptor { +public class GenericSearchBuilder implements MethodInterceptor { final protected Map _attrs; protected ArrayList _conditions; - protected HashMap, Attribute, Attribute>> _joins; + protected HashMap>> _joins; protected ArrayList _selects; private final ArrayList _groupBys; private final Class _resultType; @@ -130,10 +129,10 @@ public class SearchCriteria { this._counter = 0; this._joins = null; if (sb._joins != null) { - _joins = new HashMap, Attribute, Attribute>>(sb._joins.size()); - for (Map.Entry, Attribute, Attribute>> entry : sb._joins.entrySet()) { - Ternary, Attribute, Attribute> value = entry.getValue(); - _joins.put(entry.getKey(), new Ternary, Attribute, Attribute>(value.first().create(), value.second(), value.third())); + _joins = new HashMap>>(sb._joins.size()); + for (Map.Entry>> entry : sb._joins.entrySet()) { + JoinBuilder> value = entry.getValue(); + _joins.put(entry.getKey(), new JoinBuilder>(value.getT().create(),value.getFirstAttribute(), value.getSecondAttribute(), value.getType())); } } _selects = sb._selects; @@ -187,25 +186,26 @@ public class SearchCriteria { } public void setJoinParameters(String joinName, String conditionName, Object... params) { - Ternary, Attribute, Attribute> join = _joins.get(joinName); + JoinBuilder> join = _joins.get(joinName); assert (join != null) : "Incorrect join name specified: " + joinName; - join.first().setParameters(conditionName, params); + join.getT().setParameters(conditionName, params); + } public void addJoinAnd(String joinName, String field, Op op, Object... values) { - Ternary, Attribute, Attribute> join = _joins.get(joinName); + JoinBuilder> join = _joins.get(joinName); assert (join != null) : "Incorrect join name specified: " + joinName; - join.first().addAnd(field, op, values); + join.getT().addAnd(field, op, values); } public void addJoinOr(String joinName, String field, Op op, Object... values) { - Ternary, Attribute, Attribute> join = _joins.get(joinName); + JoinBuilder> join = _joins.get(joinName); assert (join != null) : "Incorrect join name specified: " + joinName; - join.first().addOr(field, op, values); + join.getT().addOr(field, op, values); } public SearchCriteria getJoin(String joinName) { - return _joins.get(joinName).first(); + return _joins.get(joinName).getT(); } public List getGroupBy() { @@ -290,12 +290,11 @@ public class SearchCriteria { return params; } - public Collection, Attribute, Attribute>> getJoins() { + public Collection>> getJoins() { return _joins != null ? _joins.values() : null; } private void getParams(ArrayList> params, Condition condition, Object[] objs) { - //Object[] objs = _params.get(condition.name); if (condition.op == Op.SC) { assert (objs != null && objs.length > 0) : " Where's your search criteria object? " + condition.name; params.addAll(((SearchCriteria)objs[0]).getValues()); diff --git a/server/src/com/cloud/network/dao/NetworkProfileDaoImpl.java b/utils/src/com/cloud/utils/db/StateMachine.java similarity index 68% rename from server/src/com/cloud/network/dao/NetworkProfileDaoImpl.java rename to utils/src/com/cloud/utils/db/StateMachine.java index 734e378faec..d8b16e3ff4e 100644 --- a/server/src/com/cloud/network/dao/NetworkProfileDaoImpl.java +++ b/utils/src/com/cloud/utils/db/StateMachine.java @@ -1,7 +1,7 @@ /** * Copyright (C) 2010 Cloud.com, Inc. All rights reserved. * - * This software is licensed under the GNU General Public License v3 or later. + * This software is licensed under the GNU General Public License v3 or later. * * It is free software: you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -15,15 +15,17 @@ * along with this program. If not, see . * */ -package com.cloud.network.dao; +package com.cloud.utils.db; -import javax.ejb.Local; +import static java.lang.annotation.ElementType.FIELD; +import static java.lang.annotation.RetentionPolicy.RUNTIME; -import com.cloud.network.NetworkProfileVO; -import com.cloud.utils.db.GenericDaoBase; +import java.lang.annotation.Retention; +import java.lang.annotation.Target; -@Local(value=NetworkProfileDao.class) -public class NetworkProfileDaoImpl extends GenericDaoBase { - protected NetworkProfileDaoImpl() { - } +@Target(FIELD) +@Retention(RUNTIME) +public @interface StateMachine { + public Class state(); + public Class event(); } diff --git a/utils/src/com/cloud/utils/db/UpdateBuilder.java b/utils/src/com/cloud/utils/db/UpdateBuilder.java index 6c1d464541d..fb5936e69c9 100755 --- a/utils/src/com/cloud/utils/db/UpdateBuilder.java +++ b/utils/src/com/cloud/utils/db/UpdateBuilder.java @@ -18,16 +18,13 @@ package com.cloud.utils.db; import java.lang.reflect.Method; -import java.util.ArrayList; import java.util.Collection; import java.util.HashMap; -import java.util.List; import java.util.Map; import net.sf.cglib.proxy.MethodInterceptor; import net.sf.cglib.proxy.MethodProxy; -import com.cloud.utils.Pair; import com.cloud.utils.Ternary; import com.cloud.utils.exception.CloudRuntimeException; @@ -40,6 +37,7 @@ public class UpdateBuilder implements MethodInterceptor { _changes = new HashMap>(); } + @Override public Object intercept(Object object, Method method, Object[] args, MethodProxy methodProxy) throws Throwable { String name = method.getName(); if (name.startsWith("set")) { @@ -125,7 +123,6 @@ public class UpdateBuilder implements MethodInterceptor { } public StringBuilder toSql(String tables) { - List> setters = new ArrayList>(); if (_changes.isEmpty()) { return null; } diff --git a/utils/src/com/cloud/utils/db/UpdateFilter.java b/utils/src/com/cloud/utils/db/UpdateFilter.java new file mode 100644 index 00000000000..b011d08888b --- /dev/null +++ b/utils/src/com/cloud/utils/db/UpdateFilter.java @@ -0,0 +1,16 @@ +/** + * + */ +package com.cloud.utils.db; + +import java.lang.reflect.Method; + +import net.sf.cglib.proxy.CallbackFilter; + +public class UpdateFilter implements CallbackFilter { + @Override + public int accept(Method method) { + String name = method.getName(); + return (name.startsWith("set") || name.startsWith("incr") || name.startsWith("decr")) ? 1 : 0; + } +} \ No newline at end of file diff --git a/utils/src/com/cloud/utils/fsm/FiniteState.java b/utils/src/com/cloud/utils/fsm/FiniteState.java new file mode 100644 index 00000000000..7ce9b2a6ed9 --- /dev/null +++ b/utils/src/com/cloud/utils/fsm/FiniteState.java @@ -0,0 +1,55 @@ +/** + * Copyright (C) 2010 Cloud.com, Inc. All rights reserved. + * + * This software is licensed under the GNU General Public License v3 or later. + * + * It is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or any later version. + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + */ +package com.cloud.utils.fsm; + +import java.util.List; +import java.util.Set; + +/** + * Interface for a state in the finite state machine. + * + * @param State + * @param Event + */ +public interface FiniteState { + /** + * @return the state machine being used. + */ + StateMachine getStateMachine(); + + /** + * get next state based on the event. + * @param event + * @return next State + */ + S getNextState(E event); + + /** + * Get the states that could have traveled to the current state + * via this event. + * @param event + * @return array of states + */ + List getFromStates(E event); + + /** + * Get the possible events that can happen from the current state. + * @return array of events. + */ + Set getPossibleEvents(); +} diff --git a/server/src/com/cloud/network/dao/NetworkProfileDao.java b/utils/src/com/cloud/utils/fsm/FiniteStateObject.java similarity index 78% rename from server/src/com/cloud/network/dao/NetworkProfileDao.java rename to utils/src/com/cloud/utils/fsm/FiniteStateObject.java index 2938d5de2f3..277af6fb597 100644 --- a/server/src/com/cloud/network/dao/NetworkProfileDao.java +++ b/utils/src/com/cloud/utils/fsm/FiniteStateObject.java @@ -1,7 +1,7 @@ /** * Copyright (C) 2010 Cloud.com, Inc. All rights reserved. * - * This software is licensed under the GNU General Public License v3 or later. + * This software is licensed under the GNU General Public License v3 or later. * * It is free software: you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -15,11 +15,13 @@ * along with this program. If not, see . * */ -package com.cloud.network.dao; - -import com.cloud.network.NetworkProfileVO; -import com.cloud.utils.db.GenericDao; - -public interface NetworkProfileDao extends GenericDao { +package com.cloud.utils.fsm; +public interface FiniteStateObject { + /** + * @return finite state. + */ + FiniteState getState(); + + void setState(S state); } diff --git a/utils/src/com/cloud/utils/net/NetUtils.java b/utils/src/com/cloud/utils/net/NetUtils.java index d0e3428f094..df63bcb505b 100755 --- a/utils/src/com/cloud/utils/net/NetUtils.java +++ b/utils/src/com/cloud/utils/net/NetUtils.java @@ -668,7 +668,15 @@ public class NetUtils { return null; } /*reserve gateway*/ - return getIpRangeFromCidr(getLinkLocalGateway(), 32 - size); + String[] range = getIpRangeFromCidr(getLinkLocalGateway(), 32 - size); + + if (range[0].equalsIgnoreCase(getLinkLocalGateway())) { + /*remove the gateway*/ + long ip = ip2Long(range[0]); + ip += 1; + range[0] = long2Ip(ip); + } + return range; } public static String getLinkLocalIpEnd() { diff --git a/version-info.in b/version-info.in index 19b45b89907..36eaac3b0a4 100644 --- a/version-info.in +++ b/version-info.in @@ -1,6 +1,5 @@ This is the version information for the manifests in the JAR files. -JAVACFLAGS (taken from ant debuglevel property): @JAVACFLAGS@ Implementation_Version: @Implementation_Version@ Ant arguments: @ant_args@ diff --git a/vnet/wscript_build b/vnet/wscript_build new file mode 100644 index 00000000000..270368b32c3 --- /dev/null +++ b/vnet/wscript_build @@ -0,0 +1,21 @@ +import Utils + +if bld.env.DISTRO not in ['Windows','Mac']: + # build / install declarations of vnet + files = """vnetd/connection.c vnetd/select.c vnetd/timer.c vnetd/spinlock.c vnetd/skbuff.c + vnetd/vnetd.c vnet-module/skb_util.c vnet-module/sxpr_util.c vnet-module/timer_util.c + vnet-module/etherip.c vnet-module/vnet.c vnet-module/vnet_eval.c vnet-module/vnet_forward.c + vnet-module/vif.c vnet-module/tunnel.c vnet-module/sa.c vnet-module/varp.c + libxutil/allocate.c libxutil/enum.c libxutil/file_stream.c libxutil/hash_table.c + libxutil/iostream.c libxutil/lexis.c libxutil/socket_stream.c libxutil/string_stream.c + libxutil/sxpr.c libxutil/sxpr_parser.c libxutil/sys_net.c libxutil/sys_string.c libxutil/util.c""" + files = [ "src/%s"%s for s in Utils.to_list(files) ] + bld( + name='vnetd', + features='cc cprogram', + source= files, + includes="src/libxutil src/vnet-module src/vnetd", + lib='dl pthread'.split(), + target='%s-vnetd'%bld.env.PACKAGE, + install_path="${SBINDIR}" + ) diff --git a/wscript b/wscript index 2d42fed035b..c1bc46c63d9 100644 --- a/wscript +++ b/wscript @@ -9,7 +9,6 @@ APPNAME = 'cloud' import shutil,os import email,time import optparse -import platform import Utils,Node,Options,Logs,Scripting,Environment,Build,Configure from subprocess import Popen as _Popen,PIPE import os @@ -144,8 +143,9 @@ def svninfo(*args): else: url = "SVN " + url[0].strip() return rev + "\n" + url -def gitinfo(*args): - try: p = _Popen(['git','remote','show','-n','origin']+list(args),stdin=PIPE,stdout=PIPE,stderr=PIPE) +def gitinfo(dir=None): + if dir and not _isdir(dir): return '' + try: p = _Popen(['git','remote','show','-n','origin'],stdin=PIPE,stdout=PIPE,stderr=PIPE,cwd=dir) except OSError,e: if e.errno == 2: return '' # svn command is not installed raise @@ -158,7 +158,7 @@ def gitinfo(*args): except IndexError: url = [ s[5:] for s in stdout if s.startswith("URL") ][0] assert url - p = _Popen(['git','log','-1']+list(args),stdin=PIPE,stdout=PIPE,stderr=PIPE) + p = _Popen(['git','log','-1'],stdin=PIPE,stdout=PIPE,stderr=PIPE,cwd=dir) stdout,stderr = p.communicate('') retcode = p.wait() if retcode: return @@ -169,6 +169,15 @@ def gitinfo(*args): return "Git Revision: %s"%commitid + "\n" + "Git URL: %s"%url +def allgitinfo(): + t = gitinfo() + if not t: return t + + u = gitinfo(_join(pardir,"cloudstack-proprietary")) + if not u: return t + + return t + "\n\ncloustack-proprietary:\n" + u + def _getbuildnumber(): # FIXME implement for git n = Options.options.BUILDNUMBER if n: @@ -189,14 +198,6 @@ def _getbuildnumber(): # FIXME implement for git return rev Utils.getbuildnumber = _getbuildnumber -def _subst_add_destdir(x,bld): - a = "${DESTDIR}" + x - a = a.replace("${DESTDIR}",Options.options.destdir) - a = Utils.subst_vars(a,bld.env) - if a.startswith("//"): a = a[1:] - return a -Build.BuildContext.subst_add_destdir = staticmethod(_subst_add_destdir) - def mkdir_p(directory): if not _isdir(directory): Utils.pprint("GREEN","Creating directory %s and necessary parents"%directory) @@ -230,6 +231,7 @@ def discover_ant_targets_and_properties(files): propsinpropfiles = [ l.strip().split("=",1) for f in files if f.endswith(".properties") for l in file(f).readlines() if "=" in l and not l.startswith("#") ] props = dict( propsinxml + propsinpropfiles ) props["base.dir"] = '.' + props["p.base.dir"] = '.' result = [] for name,target in targets.items(): @@ -268,7 +270,9 @@ def runant(tsk): "-Dthirdparty.classpath=%s"%(tsk.env.CLASSPATH.replace(os.pathsep,",")), ] stanzas += tsk.generator.antargs + tsk.generator.anttgts - return Utils.exec_command(" ".join(stanzas),cwd=tsk.generator.bld.srcnode.abspath(),env=environ,log=True) + ret = Utils.exec_command(" ".join(stanzas),cwd=tsk.generator.bld.srcnode.abspath(),env=environ,log=True) + if ret != 0: raise Utils.WafError("Ant command %s failed with error value %s"%(stanzas,ret)) + return ret Utils.runant = runant @throws_command_errors @@ -331,68 +335,45 @@ def _install_files_filtered(self,destdir,listoffiles,**kwargs): return ret Build.BuildContext.install_files_filtered = _install_files_filtered -def _setownership(ctx,path,owner,group,mode=None): - def f(bld,path,owner,group,mode): - dochown = not Options.options.NOCHOWN \ - and hasattr(os,"getuid") and os.getuid() == 0 \ - and _chown \ - and _chmod \ - and pwd \ - and grp \ - and stat - if not dochown: return - - try: uid = pwd.getpwnam(owner).pw_uid - except KeyError,e: raise Utils.WafError("If installing as root, please either create a %s user or use the --nochown parameter of waf install to install the files as root"%owner) - try: gid = grp.getgrnam(group).gr_gid - except KeyError,e: raise Utils.WafError("If installing as root, please either create a %s group or use the --nochown parameter of waf install to install the files as root"%group) - - path = _subst_add_destdir(path,bld) - current_uid,current_gid = os.stat(path).st_uid,os.stat(path).st_gid - if current_uid != uid: - Utils.pprint("GREEN","* setting owner of %s to UID %s"%(path,uid)) - _chown(path,uid,current_gid) - current_uid = uid - if current_gid != gid: - Utils.pprint("GREEN","* setting group of %s to GID %s"%(path,gid)) - _chown(path,current_uid,gid) - current_gid = gid - if mode is not None: - current_mode = stat.S_IMODE(os.stat(path).st_mode) - if current_mode != mode: - Utils.pprint("GREEN","* adjusting permissions on %s to mode %o"%(path,mode)) - _chmod(path,mode) - current_mode = mode - - if Options.is_install: - ctx.add_post_fun(lambda ctx: f(ctx,path,owner,group,mode)) -Build.BuildContext.setownership = _setownership +def _substitute(self,listoffiles,install_to=None,cwd=None,dict=None,name=None,**kwargs): + if cwd is None: cwd = self.path + tgenkwargs = {} + if name is not None: tgenkwargs["name"] = name + if isinstance(listoffiles,str) and '**' in listoffiles: + listoffiles = cwd.ant_glob(listoffiles,flat=True) + elif isinstance(listoffiles,str) and '*' in listoffiles: + listoffiles = [ n for x in listoffiles.split() for n in _glob(cwd.abspath() + os.sep + x.replace("/",os.sep)) ] + for src in Utils.to_list(listoffiles): + tgt = src + ".subst" + inst = src # Utils.relpath(src,relative_to) <- disabled + tgen = self(features='subst', source=src, target=tgt, **tgenkwargs) + if dict is not None: tgen.dict = dict + else: tgen.dict = self.env.get_merged_dict() + self.path.find_or_declare(tgt) + if install_to is not None: self.install_as("%s/%s"%(install_to,inst), tgt, **kwargs) +Build.BuildContext.substitute = _substitute def set_options(opt): """Register command line options""" opt.tool_options('gnu_dirs') - if platform.system() not in ['Windows',"Darwin"]: opt.tool_options('compiler_cc') opt.tool_options('python') + opt.tool_options('tar',tooldir='tools/waf') + opt.tool_options('mkisofs',tooldir='tools/waf') + if Options.platform not in ['darwin','win32']: opt.tool_options('usermgmt',tooldir='tools/waf') + if Options.platform not in ['darwin','win32']: opt.tool_options('javadir',tooldir='tools/waf') + opt.tool_options('tomcat',tooldir='tools/waf') + if Options.platform not in ['darwin',"win32"]: opt.tool_options('compiler_cc') - inst_dir = opt.get_option_group('--bindir') # get the group that contains bindir - inst_dir.add_option('--javadir', # add javadir to the group that contains bindir - help = 'Java class and jar files [Default: ${DATADIR}/java]', - default = '', - dest = 'JAVADIR') - inst_dir.add_option('--with-tomcat', # add javadir to the group that contains bindir - help = 'Path to installed Tomcat 6 environment [Default: ${DATADIR}/tomcat6 (unless %%CATALINA_HOME%% is set)]', - default = '', - dest = 'TOMCATHOME') - inst_dir = opt.get_option_group('--srcdir') # get the group that contains the srcdir - inst_dir.add_option('--with-db-host', # add javadir to the group that contains bindir + inst_dir = opt.get_option_group('--srcdir') + inst_dir.add_option('--with-db-host', help = 'Database host to use for waf deploydb [Default: 127.0.0.1]', default = '127.0.0.1', dest = 'DBHOST') - inst_dir.add_option('--with-db-user', # add javadir to the group that contains bindir + inst_dir.add_option('--with-db-user', help = 'Database user to use for waf deploydb [Default: root]', default = 'root', dest = 'DBUSER') - inst_dir.add_option('--with-db-pw', # add javadir to the group that contains bindir + inst_dir.add_option('--with-db-pw', help = 'Database password to use for waf deploydb [Default: ""]', default = '', dest = 'DBPW') @@ -410,12 +391,7 @@ def set_options(opt): help = 'does ---no-dep-check', default = False, dest = 'NODEPCHECK') - inst_dir = opt.get_option_group('--force') # get the group that contains the force - inst_dir.add_option('--nochown', - action='store_true', - help = 'skip chown and chmod upon install (skipped on Windows or by non-root users by default)', - default = False, - dest = 'NOCHOWN') + inst_dir = opt.get_option_group('--force') inst_dir.add_option('--preserve-config', action='store_true', help = 'do not install configuration files', @@ -424,7 +400,7 @@ def set_options(opt): debugopts = optparse.OptionGroup(opt.parser,'run/debug options') opt.add_option_group(debugopts) - debugopts.add_option('--debug-port', # add javadir to the group that contains bindir + debugopts.add_option('--debug-port', help = 'Port on which the debugger will listen when running waf debug [Default: 8787]', default = '8787', dest = 'DEBUGPORT') @@ -441,15 +417,15 @@ def set_options(opt): rpmopts = optparse.OptionGroup(opt.parser,'RPM/DEB build options') opt.add_option_group(rpmopts) - rpmopts.add_option('--build-number', # add javadir to the group that contains bindir + rpmopts.add_option('--build-number', help = 'Build number [Default: SVN revision number for builds from checkouts, or empty for builds from source releases]', default = '', dest = 'BUILDNUMBER') - rpmopts.add_option('--prerelease', # add javadir to the group that contains bindir + rpmopts.add_option('--prerelease', help = 'Branch name to append to the release number (if specified, alter release number to be a prerelease); this option requires --build-number=X [Default: nothing]', default = '', dest = 'PRERELEASE') - rpmopts.add_option('--skip-dist', # add javadir to the group that contains bindir + rpmopts.add_option('--skip-dist', action='store_true', help = 'Normally, dist() is called during package build. This makes the package build assume that a distribution tarball has already been made, and use that. This option is also valid during distcheck and dist.', default = False, @@ -457,7 +433,7 @@ def set_options(opt): distopts = optparse.OptionGroup(opt.parser,'dist options') opt.add_option_group(distopts) - distopts.add_option('--oss', # add javadir to the group that contains bindir + distopts.add_option('--oss', help = 'Only include open source components', action = 'store_true', default = False, @@ -476,6 +452,18 @@ def showconfig(conf): continue Utils.pprint("BLUE"," %s: %s"%(key,val)) +def _getconfig(self): + lines = [] + for key,val in sorted(self.env.get_merged_dict().items()): + if "CLASSPATH" in key: + lines.append(" %s:"%key) + for v in val.split(pathsep): + lines.append(" %s"%v) + continue + lines.append(" %s: %s"%(key,val)) + return "\n".join(lines) +Build.BuildContext.getconfig = _getconfig + def list_targets(ctx): """return the list of buildable and installable targets""" @@ -528,7 +516,7 @@ def dist_hook(): if Options.options.OSS: [ shutil.rmtree(f) for f in "cloudstack-proprietary".split() if _exists(f) ] - stdout = svninfo("..") or gitinfo() + stdout = svninfo("..") or allgitinfo() if stdout: f = file("sccs-info","w") f.write(stdout) @@ -579,12 +567,15 @@ def rpm(context): if not Options.options.blddir: outputdir = _join(context.curdir,blddir,"rpmbuild") else: outputdir = _join(_abspath(Options.options.blddir),"rpmbuild") Utils.pprint("GREEN","Building RPMs") + tarball = Scripting.dist() + sourcedir = _join(outputdir,"SOURCES") - #if _isdir(outputdir): shutil.rmtree(outputdir) + if _exists(sourcedir): shutil.rmtree(sourcedir) for a in ["RPMS/noarch","SRPMS","BUILD","SPECS","SOURCES"]: mkdir_p(_join(outputdir,a)) + shutil.copy(tarball,_join(sourcedir,tarball)) + specfile = "%s.spec"%APPNAME - shutil.copy(tarball,_join(outputdir,"SOURCES")) checkdeps = lambda: c(["rpmbuild","--define","_topdir %s"%outputdir,"--nobuild",specfile]) dorpm = lambda: c(["rpmbuild","--define","_topdir %s"%outputdir,"-ba",specfile]+buildnumber+prerelease) try: checkdeps() @@ -622,13 +613,15 @@ def deb(context): if not Options.options.blddir: outputdir = _join(context.curdir,blddir,"debbuild") else: outputdir = _join(_abspath(Options.options.blddir),"debbuild") Utils.pprint("GREEN","Building DEBs") - tarball = Scripting.dist() - - #if _isdir(outputdir): shutil.rmtree(outputdir) + + tarball = Scripting.dist() + srcdir = "%s/%s-%s"%(outputdir,APPNAME,VERSION) + + if _exists(srcdir): shutil.rmtree(srcdir) mkdir_p(outputdir) + f = tarfile.open(tarball,'r:bz2') f.extractall(path=outputdir) - srcdir = "%s/%s-%s"%(outputdir,APPNAME,VERSION) if tempchangelog: f = file(_join(srcdir,"debian","changelog"),"w") f.write(tempchangelog) @@ -642,7 +635,6 @@ def deb(context): Utils.pprint("YELLOW","Dependencies might be missing. Trying to auto-install them...") installdebdeps(context) dodeb() - shutil.rmtree(srcdir) def uninstallrpms(context): """uninstalls any Cloud Stack RPMs on this system""" @@ -684,58 +676,26 @@ def installdebdeps(context): @throws_command_errors def deploydb(ctx,virttech=None): - if not virttech: raise Utils.WafError('use deploydb_xenserver or deploydb_kvm rather than deploydb') + if not virttech: raise Utils.WafError('use deploydb_xenserver, deploydb_vmware or deploydb_kvm rather than deploydb') ctx = _getbuildcontext() - srcdir = ctx.path.abspath() - builddir = ctx.path.abspath(ctx.env) + setupdatabases = _join(ctx.env.BINDIR,"cloud-setup-databases") + serversetup = _join(ctx.env.SETUPDATADIR,"server-setup.xml") - dbhost = ctx.env.DBHOST - dbuser = ctx.env.DBUSER - dbpw = ctx.env.DBPW - dbdir = ctx.env.DBDIR + if not _exists(setupdatabases): # Needs install! + Scripting.install(ctx) + + cmd = [ + ctx.env.PYTHON, + setupdatabases, + "cloud@%s"%ctx.env.DBHOST, + virttech, + "--auto=%s"%serversetup, + "--deploy-as=%s:%s"%(ctx.env.DBUSER,ctx.env.DBPW), + ] - if not _exists(_join(builddir,"client","tomcatconf","db.properties")): raise Utils.WafError("Please build at least once to generate the db.properties configuration file") - - cp = [] - cp += [ _join(builddir,"client","tomcatconf") ] - cp += [ _join("test","conf") ] - cp += _glob(_join(builddir,"target", "jar", "*.jar")) - cp += [ctx.env.CLASSPATH] - cp = pathsep.join(cp) - - before = "" - for f in ["create-database","create-schema"]: - p = _join("setup","db",f+".sql") - p = dev_override(p) - before = before + file(p).read() - Utils.pprint("GREEN","Reading database code from %s"%p) - - cmd = ["mysql","--user=%s"%dbuser,"-h",dbhost,"--password=%s"%dbpw] - Utils.pprint("GREEN","Deploying database scripts to %s (user %s)"%(dbhost,dbuser)) Utils.pprint("BLUE"," ".join(cmd)) - p = _Popen(cmd,stdin=PIPE,stdout=None,stderr=None) - p.communicate(before) - retcode = p.wait() - if retcode: raise CalledProcessError(retcode,cmd) - - serversetup = dev_override(_join("setup","db","server-setup.xml")) - Utils.pprint("GREEN","Configuring database with com.cloud.test.DatabaseConfig") - run_java("com.cloud.test.DatabaseConfig",cp,['-Dlog4j.configuration=log4j-stdout.properties'],[serversetup]) - - after = "" - for f in ["templates.%s"%virttech,"create-index-fk"]: - p = _join("setup","db",f+".sql") - p = dev_override(p) - after = after + file(p).read() - Utils.pprint("GREEN","Reading database code from %s"%p) - - cmd = ["mysql","--user=%s"%dbuser,"-h",dbhost,"--password=%s"%dbpw] - Utils.pprint("GREEN","Deploying post-configuration database scripts to %s (user %s)"%(dbhost,dbuser)) - Utils.pprint("BLUE"," ".join(cmd)) - p = _Popen(cmd,stdin=PIPE,stdout=None,stderr=None) - p.communicate(after) - retcode = p.wait() + retcode = Utils.exec_command(cmd,shell=False,stdout=None,stderr=None,log=True) if retcode: raise CalledProcessError(retcode,cmd) def deploydb_xenserver(ctx): @@ -744,6 +704,9 @@ def deploydb_xenserver(ctx): def deploydb_kvm(ctx): """re-deploys the database using the MySQL connection information and the KVM templates.sql""" return deploydb(ctx,"kvm") +def deploydb_vmware(ctx): + """re-deploys the database using the MySQL connection information and the KVM templates.sql""" + return deploydb(ctx,"vmware") def run(args): """runs the management server""" @@ -768,6 +731,8 @@ def run(args): "-Dcatalina.home=" + conf.env.MSENVIRON, "-Djava.io.tmpdir="+_join(conf.env.MSENVIRON,"temp"), ] + if not _exists(_join(conf.env.BINDIR,"cloud-setup-databases")): Scripting.install(conf) + cp = [conf.env.MSCONF] cp += _glob(_join(conf.env.MSENVIRON,'bin',"*.jar")) cp += _glob(_join(conf.env.MSENVIRON,'lib',"*.jar")) @@ -782,8 +747,6 @@ def run(args): #vendorconfs = _glob( _join(conf.env.MSCONF,"vendor","*") ) #if vendorconfs: cp = plugins + cp - #Scripting.install(conf) - run_java("org.apache.catalina.startup.Bootstrap",cp,options,["start"]) def debug(ctx): @@ -792,14 +755,16 @@ def debug(ctx): @throws_command_errors def run_agent(args): - """runs the management server""" + """runs the agent""" # FIXME: make this use the run/debug options conf = _getbuildcontext() + if not _exists(_join(conf.env.LIBEXECDIR,"agent-runner")): Scripting.install(conf) _check_call("sudo",[_join(conf.env.LIBEXECDIR,"agent-runner")]) @throws_command_errors def run_console_proxy(args): - """runs the management server""" + """runs the console proxy""" # FIXME: make this use the run/debug options conf = _getbuildcontext() + if not _exists(_join(conf.env.LIBEXECDIR,"console-proxy-runner")): Scripting.install(conf) _check_call("sudo",[_join(conf.env.LIBEXECDIR,"console-proxy-runner")]) def simulate_agent(args): @@ -826,7 +791,7 @@ def simulate_agent(args): cp += [conf.env.DEPSCLASSPATH] cp += [conf.env.AGENTSIMULATORCLASSPATH] - #Scripting.install(conf) + if not _exists(_join(conf.env.LIBEXECDIR,"agent-runner")): Scripting.install(conf) run_java("com.cloud.agent.AgentSimulator",cp,arguments=args) diff --git a/wscript_build b/wscript_build index 59e33345f80..7065266758a 100644 --- a/wscript_build +++ b/wscript_build @@ -8,7 +8,7 @@ import shutil,os import Utils,Node,Options,Logs,Scripting,Environment,Build,Configure from os import unlink as _unlink, makedirs as _makedirs, getcwd as _getcwd, chdir as _chdir -from os.path import abspath as _abspath, basename as _basename, dirname as _dirname, exists as _exists, isdir as _isdir, split as _split, join as _join, sep, pathsep, pardir +from os.path import abspath as _abspath, basename as _basename, dirname as _dirname, exists as _exists, isdir as _isdir, split as _split, join as _join, sep, pathsep, pardir, curdir from glob import glob as _glob try: set([1,2,3]) except Exception: from Sets import set @@ -42,54 +42,21 @@ sccsinfo = _join(sourcedir,"sccs-info") if _exists(sccsinfo): bld.install_files("${DOCDIR}","sccs-info") tgen = bld(features='subst', name='configure-info', source="configure-info.in", target="configure-info") -def gen_configure_info(): - lines = [] - for key,val in sorted(bld.env.get_merged_dict().items()): - if "CLASSPATH" in key: - lines.append(" %s:"%key) - for v in val.split(pathsep): - lines.append(" %s"%v) - continue - lines.append(" %s: %s"%(key,val)) - return "\n".join(lines) -tgen.dict = {"CONFIGUREVARS":gen_configure_info()} +tgen.dict = {"CONFIGUREVARS":bld.getconfig()} bld.install_files("${DOCDIR}","configure-info") # ==================== Java compilation =========================== -sfl = lambda filespec: filelist(filespec,src=True,bld=False,dir=False,flat=True) - # discover ant targets and properties antxmlfiles = _glob(_join("build","*.xml")) + _glob(_join("build","*.properties")) if buildpremium: antxmlfiles += _glob(_join("build","premium","*.xml")) + _glob(_join("cloudstack-proprietary","build","*.xml")) targets,antprops = Utils.discover_ant_targets_and_properties(antxmlfiles) -javac_env = bld.env.copy() -# FIXME trigger recompilation / cache avoidance when debuglevel changes. ATM this does not happen. -if antprops.get('debuglevel',None): javac_env.append_value("JAVACFLAGS",["-g:%s"%antprops['debuglevel']]) Implementation_Version = bld.env.VERSION buildnumber = Utils.getbuildnumber() if buildnumber: Implementation_Version += "." + buildnumber -# compile .class files using waf -classpathentries = [ _join(builddir,x[2]) for x in targets ] -javawclasspath = pathsep.join( classpathentries + [ _abspath(x) for x in bld.env.CLASSPATH.split(pathsep) ] ) -javac_tgens = [ - bld( features='javac', name=name, srcdir=srcdir, outdir=classdir, - after=" ".join(deps), classpath=javawclasspath, env=javac_env ) - for name,srcdir,classdir,jarpath,deps in targets ] -[ bld.process_after(tgen) for tgen in javac_tgens ] - -# compile jar files using ant -# ant only needs to be reinvoked if the version with build number changes -# we here trim all the depended targets from the target list: -build_targets = [ x[0] for x in targets ] -nondep_build_targets = list ( set([ x[0] for x in targets ]) - set([ x for dependencies in [ y[4] for y in targets ] for x in dependencies ]) ) -ant_sources = " ".join(antxmlfiles + [ sfl(x[1] + "/**/*.java") for x in targets ]) -ant_jars = [ x[3] for x in targets ] - -# because build-console-viewer does not use compile-java macro, we have to hardcode it here - +# this is to trigger recompilation / cache avoidance if the relevant environment for ant changes ant_args = [ "-Dimpl.version=%s"%Implementation_Version, "-Dtarget.dir=%s"%Utils.relpath(_join(builddir,"target")), @@ -97,17 +64,20 @@ ant_args = [ ] if not buildpremium: ant_args += ["-DOSS=true"] -# this is to trigger recompilation / cache avoidance if the relevant environment for ant changes tgen = bld(features='subst', name='version-info', source="version-info.in", target="version-info") -tgen.dict = { "Implementation_Version":Implementation_Version,"JAVACFLAGS":javac_env.JAVACFLAGS,"ant_args":ant_args } +tgen.dict = { "Implementation_Version":Implementation_Version,"ant_args":ant_args } bld.install_files("${DOCDIR}","version-info") -build_targets += ["version-info"] -ant_sources += " version-info" +# compile jar files using ant +# ant only needs to be reinvoked if the version with build number changes +# we here trim all the depended targets from the target list: +build_targets = [ x[0] for x in targets ] +nondep_build_targets = list ( set([ x[0] for x in targets ]) - set([ x for dependencies in [ y[4] for y in targets ] for x in dependencies ]) ) +ant_sources = " ".join(antxmlfiles + [ filelist(x[1] + "/**/*.java",src=True,bld=False,dir=False,flat=True) for x in targets ]) + ' version-info' +ant_jars = [ x[3] for x in targets ] bld.path.ensure_dir_node_from_path("target/jar") -tgen = bld(rule=Utils.runant, name='runant', source=ant_sources, target=ant_jars, anttgts = nondep_build_targets, antargs=ant_args, after=build_targets) -bld.process_after(tgen) +tgen = bld(rule=Utils.runant, name='runant', source=ant_sources, target=ant_jars, anttgts = nondep_build_targets, antargs=ant_args) # install jar files [ bld.install_files('${JAVADIR}',jar) for jar in ant_jars if _basename(jar).startswith("cloud-") ] @@ -117,69 +87,22 @@ bld.process_after(tgen) # =================== C / Python compilation ========================= -if bld.env.DISTRO not in ['Windows','Mac']: - # build / install declarations of the daemonization utility - except for Windows - bld( - name='daemonize', - features='cc cprogram', - source='daemonize/daemonize.c', - target='daemonize/cloud-daemonize') - - # build / install declarations of vnet - files = """vnetd/connection.c vnetd/select.c vnetd/timer.c vnetd/spinlock.c vnetd/skbuff.c - vnetd/vnetd.c vnet-module/skb_util.c vnet-module/sxpr_util.c vnet-module/timer_util.c - vnet-module/etherip.c vnet-module/vnet.c vnet-module/vnet_eval.c vnet-module/vnet_forward.c - vnet-module/vif.c vnet-module/tunnel.c vnet-module/sa.c vnet-module/varp.c - libxutil/allocate.c libxutil/enum.c libxutil/file_stream.c libxutil/hash_table.c - libxutil/iostream.c libxutil/lexis.c libxutil/socket_stream.c libxutil/string_stream.c - libxutil/sxpr.c libxutil/sxpr_parser.c libxutil/sys_net.c libxutil/sys_string.c libxutil/util.c""" - files = [ "vnet/src/%s"%s for s in Utils.to_list(files) ] - bld( - name='vnetd', - features='cc cprogram', - source= files, - includes="vnet/src/libxutil vnet/src/vnet-module vnet/src/vnetd", - lib='dl pthread'.split(), - target='vnet/%s-vnetd'%bld.env.PACKAGE, - install_path="${SBINDIR}" - ) - - obj = bld(features = 'py',name='pythonmodules') - obj.find_sources_in_dirs('python/lib', exts=['.py']) +bld.recurse(["vnet","daemonize","python"],'build') # ===================== End C / Python compilation ========================== # ================ Third-party / dependency installation =============== -bld.install_files('${JAVADIR}','deps/*.jar') -if buildpremium: bld.install_files('${PREMIUMJAVADIR}','cloudstack-proprietary/thirdparty/*.jar') +bld.recurse(["deps"],'build') +if buildpremium: bld.recurse(['cloudstack-proprietary/thirdparty'],'build') # =================== End 3rdparty/dep install ======================== # =================== Build install declaratoin of console proxy project ======== -# -> binary unsubstitutable files -start_path = bld.path.find_dir("console-proxy") -bld.install_files("${CPLIBDIR}", - start_path.ant_glob("images/**",src=True,bld=False,dir=False,flat=True), - cwd=start_path,relative_trick=True) - -# -> source files with tokens -patterns = 'console-proxy/css/** console-proxy/js/** console-proxy/ui/** console-proxy/scripts/**' -src_files = Utils.to_list(filelist(patterns,flat=True)) -subst_files = [ x+".subst" for x in src_files ] -inst_files = [ Utils.relpath(x,"console-proxy") for x in src_files ] -for src,tgt,inst in zip(src_files,subst_files,inst_files): - tgen = bld(features='subst', source=src, target=tgt) - tgen.dict = bld.env.get_merged_dict() - bld.path.find_or_declare(tgt) - bld.install_as("${CPLIBDIR}/%s"%inst, tgt) - -# -> configuration -if not Options.options.PRESERVECONFIG: - bld.install_files_filtered("${CPSYSCONFDIR}",filelist("console-proxy/conf.dom0/*")) +bld.recurse(["console-proxy"],'build') # ================== End console proxy =================== @@ -187,64 +110,7 @@ if not Options.options.PRESERVECONFIG: # ================ Creation of patch.tgz's ============================ # done here because the patches require substituted files -patterns = "patches/**" -src_files = Utils.to_list(filelist(patterns,flat=True)) -subst_files = [ x+".subst" for x in src_files ] -inst_files = src_files -for src,tgt,inst in zip(src_files,subst_files,inst_files): - tgen = bld(features='subst', name='patchsubst', source=src, target=tgt) - tgen.dict = bld.env.get_merged_dict() - bld.path.find_or_declare(tgt) - -# this is a clever little thing -# given a list of nodes, build or source -# construct a tar file containing them -# rooted in the parameter root =, specified in the task generator -# and renaming the names of the files according to a rename(x) function passed to the task generator as well -# if a build node's result of rename() has the same name as a source node, the build node will take precedence -# for as long as the build node appears later than the source node (this is an implementation detail of waf we are relying on) -def tar_up(task): - tgt = task.outputs[0].bldpath(task.env) - if _exists(tgt): _unlink(tgt) - z = tarfile.open(tgt,"w:gz") - fileset = {} - for inp in task.inputs: - src = inp.srcpath(task.env) - if src.startswith(".."): - srcname = Utils.relpath(src,_join("..",".")) # file in source dir - else: - srcname = Utils.relpath(src,_join(task.env.variant(),".")) # file in artifacts dir - if task.generator.rename: srcname = task.generator.rename(srcname) - for dummy in task.generator.root.split("/"): - splittedname = srcname.split("/") - srcname = "/".join(splittedname[1:]) - fileset[srcname] = src - for srcname,src in fileset.items(): - ti = tarfile.TarInfo(srcname) - ti.mode = 0755 - ti.size = os.path.getsize(src) - f = file(src) - z.addfile(ti,fileobj=f) - f.close() - z.close() - return 0 - -if bld.env.DISTRO != "Windows": - for virttech in [ _basename(x) for x in _glob(_join("patches","*")) ]: - if virttech == "shared": - continue - patchfiles = filelist('patches/%s/** patches/shared/**'%virttech,src=True,bld=True,dir=False,flat=True) - tgen = bld( - rule = tar_up, - source = patchfiles, - target = '%s-patch.tgz'%virttech, - name = '%s-patch_tgz'%virttech, - root = "patches/%s"%virttech, - rename = lambda x: re.sub(".subst$","",x), - after = 'patchsubst', - ) - bld.process_after(tgen) - bld.install_as("${AGENTLIBDIR}/scripts/vm/hypervisor/%s/patch.tgz"%virttech, "%s-patch.tgz"%virttech) +bld.recurse(["patches"],'build') # ================== End creation of patch.tgz's ==================== @@ -326,7 +192,7 @@ def zip_up(task): if srcname.startswith("cloudstack-proprietary"): srcname = "/".join(srcname.split("/")[1:]) # chop off cloudstack proprietary srcname = "/".join(srcname.split("/")[1:]) # chop off project name # post-process the paths - if True in [ srcname.startswith(strt) for strt in [ 'scripts/run','scripts/config_ssl','scripts/config_auth','scripts/ssvm-check' ] ]: + if True in [ srcname.startswith(strt) for strt in [ 'scripts/run','scripts/_run','scripts/config_ssl','scripts/config_auth','scripts/ssvm-check' ] ]: srcname = "/".join(srcname.split("/")[1:]) elif srcname.startswith('storage'): srcname = "scripts/" + srcname @@ -338,6 +204,8 @@ def zip_up(task): z.close() return 0 +bld.path.ensure_dir_node_from_path("target/oss") +bld.path.ensure_dir_node_from_path("target/premium") if bld.env.DISTRO not in ["Windows","Mac"]: tgen = bld( rule = copydeps, @@ -348,32 +216,79 @@ if bld.env.DISTRO not in ["Windows","Mac"]: tgen = bld( rule = zip_up, source = " ".join( [sources,artifacts,deps,systems] ), - target = 'systemvm.zip', + target = 'target/oss/systemvm.zip', name = 'systemvm_zip', - after = 'copydeps getsystemjars runant', ) - bld.process_after(tgen) + bld.install_files("${AGENTLIBDIR}/vms", "target/oss/systemvm.zip") + if buildpremium: tgen = bld( rule = zip_up, source = " ".join( [premiumsources,premiumartifacts,thirdparties] ), - target = 'systemvm-premium.zip', + target = 'target/premium/systemvm.zip', name = 'systemvm-premium_zip', - after = 'runant', ) - bld.process_after(tgen) - -if bld.env.DISTRO not in [ "Windows", "Mac"]: - bld.install_files("${AGENTLIBDIR}/vms", "systemvm.zip") - -if buildpremium: - bld.install_files("${AGENTLIBDIR}/vms", "systemvm-premium.zip") + #no need to install the premium one, we have ISOs below + #bld.install_files("${AGENTLIBDIR}/vms", "systemvm-premium.zip") # ================== End systemvm patch creation ==================== +# ================== systemvm ISO creation ==================== + +def iso_up(task): + tgt = task.outputs[0].bldpath(task.env) + if os.path.exists(tgt): os.unlink(tgt) + inps = [] + for inp in task.inputs: + if inp.id&3==Node.BUILD: + src = inp.bldpath(task.env) + srcname = src + srcname = sep.join(srcname.split(sep)[1:]) # chop off default/ + else: + src = inp.srcpath(task.env) + srcname = src + srcname = sep.join(srcname.split(sep)[1:]) # chop off ../ + if task.generator.rename: srcname = task.generator.rename(srcname) + inps.append(srcname+'='+src) + ret = Utils.exec_command( + [ + task.generator.env.MKISOFS, + "-quiet", + "-r", + "-graft-points", + "-o",tgt, + ] + inps, shell=False) + if ret != 0: return ret + +if bld.env.DISTRO not in ["Windows","Mac"]: + # systemvm.zip cannot be built on Windows or Mac because system deps do not exist there + tgen = bld( + rule = iso_up, + source = "patches/systemvm-patch.tgz target/oss/systemvm.zip patches/systemvm/root/.ssh/authorized_keys", + target = 'target/oss/systemvm.iso', + name = 'systemvm_iso', + rename = lambda x: x.split(sep)[-1].replace('systemvm-',''), + ) + bld.install_as("${AGENTLIBDIR}/vms/systemvm.iso", "target/oss/systemvm.iso") + +if buildpremium: + tgen = bld( + rule = iso_up, + source = "patches/systemvm-patch.tgz target/premium/systemvm.zip patches/systemvm/root/.ssh/authorized_keys", + target = 'target/premium/systemvm.iso', + name = 'systemvm-premium_iso', + rename = lambda x: x.split(sep)[-1].replace('systemvm-',''), + ) + bld.install_as("${AGENTLIBDIR}/vms/systemvm-premium.iso", "target/premium/systemvm.iso") + +# ================== End systemvm ISO creation ==================== + + # =================== Empty directory / symlink creation on install target ==================== +bld.createuser(bld.env.MSUSER,bld.env.MSENVIRON,'/bin/sh') + # 7. make log and cache dirs (this actually runs first) if bld.env.DISTRO in 'Windows Mac': pass else: @@ -422,14 +337,8 @@ else: # =================== Subst / installation of agent scripts project ======== -src_files = Utils.to_list(filelist("scripts/**",flat=True)) -subst_files = [ x+".subst" for x in src_files ] -inst_files = src_files -for src,tgt,inst in zip(src_files,subst_files,inst_files): - tgen = bld(features='subst', name='scriptssubst', source=src, target=tgt) - tgen.dict = bld.env.get_merged_dict() - bld.path.find_or_declare(tgt) - bld.install_as("${AGENTLIBDIR}/%s"%inst, tgt, chmod=0755) +bld.recurse(["scripts"],'build') +if buildpremium: bld.recurse(["cloudstack-proprietary/scripts"],'build') # ================== End agent scripts =================== @@ -446,91 +355,17 @@ bld.install_files_filtered("${SBINDIR}","*/sbindir/* cloudstack-proprietary/*/sb # ================== Installation of scripts / bindirs / configuration files =========================== # build / install declarations of test project -if buildpremium: - proj = 'test' - start_path = bld.path.find_dir("cloudstack-proprietary/test/scripts") - bld.install_files('${LIBDIR}/${PACKAGE}/test', - start_path.ant_glob("**",src=True,bld=False,dir=False,flat=True), - cwd=start_path,relative_trick=True) - start_path = bld.path.find_dir("cloudstack-proprietary/test/metadata") - bld.install_files('${SHAREDSTATEDIR}/${PACKAGE}/test', - start_path.ant_glob("**",src=True,bld=False,dir=False,flat=True), - cwd=start_path,relative_trick=True) - if not Options.options.PRESERVECONFIG: - bld.install_files('${SYSCONFDIR}/%s/%s'%(bld.env.PACKAGE,proj),'cloudstack-proprietary/test/conf/*') +if buildpremium: bld.recurse(["cloudstack-proprietary/test"],'build') # build / install declarations of server project <- this is actually now in client project -bld.install_files("${MSENVIRON}/webapps/client/WEB-INF",'client/WEB-INF/web.xml') # install web.xml file -if not Options.options.PRESERVECONFIG: - bld.install_files_filtered("${SERVERSYSCONFDIR}","server/conf/*") +bld.recurse(["client","server"],'build') +if buildpremium: bld.recurse(["cloudstack-proprietary/premium"],'build') # build / install declarations of agent project -proj = 'agent' -start_path = bld.path.find_dir(proj) -bld.install_files("${AGENTLIBDIR}", - start_path.ant_glob("storagepatch/**",src=True,bld=False,dir=False,flat=True), - cwd=start_path,relative_trick=True) -if not Options.options.PRESERVECONFIG: - bld.install_files_filtered("${AGENTSYSCONFDIR}","%s/conf/*"%proj) +bld.recurse(["agent"],'build') # build / install declarations of client UI project - -# -> binary unsubstitutable files -start_path = bld.path.find_dir("ui") -bld.install_files("${MSENVIRON}/webapps/client", - start_path.ant_glob("*.ico **/*png **/*jpg **/*gif",src=True,bld=False,dir=False,flat=True), - cwd=start_path,relative_trick=True) - -# -> source files with tokens -patterns = 'ui/*html ui/**/*html ui/**/*js ui/**/*css ui/**/*properties ui/**/*jsp' -src_files = Utils.to_list(filelist(patterns,flat=True)) -subst_files = [ x+".subst" for x in src_files ] -inst_files = [ Utils.relpath(x,"ui") for x in src_files ] -for src,tgt,inst in zip(src_files,subst_files,inst_files): - tgen = bld(features='subst', source=src, target=tgt) - tgen.dict = bld.env.get_merged_dict() - bld.path.find_or_declare(tgt) - bld.install_as("${MSENVIRON}/webapps/client/%s"%inst, tgt) - -# -> minification of UI files -def minifyjs(task): - tgt = task.outputs[0].bldpath(task.env) - inputfiles = [] - outputfile = ['--js_output_file',tgt] - for inp in task.inputs: - src = inp.srcpath(task.env) - inputfiles.append(src) - newinputfiles = [] - for inputfile in inputfiles: - if inputfile not in newinputfiles: - newinputfiles.append('--js') - newinputfiles.append(inputfile) - compilerjar = _join(sourcedir,'tools','gcc','compiler.jar') - return Utils.exec_command(["java",'-jar',compilerjar] + newinputfiles + outputfile,log=True) - -javascripts = [ - ['ui/scripts/jquery-1.4.2.min.js','ui/scripts/date.js'], - Utils.to_list(filelist('ui/scripts/jquery*js')), - ['ui/scripts/cloud.core.js','ui/scripts/cloud.core.callbacks.js'], - Utils.to_list(filelist('ui/scripts/cloud*js')), -] -sourcefiles = [] -for lst in javascripts: - for x in lst: - if x not in sourcefiles: sourcefiles.append(x) -tgen = bld( - rule = minifyjs, - source = sourcefiles, - target = 'ui/scripts/cloud.min.js', - name = 'minifyjs', -) -bld.install_files("${MSENVIRON}/webapps/client/scripts", "ui/scripts/cloud.min.js") - -# substitute and install generic tomcat config -if not Options.options.PRESERVECONFIG: - bld.install_files_filtered("${MSCONF}","*/tomcatconf/* cloudstack-proprietary/*/tomcatconf/*") - bld.install_files("${MSCONF}",'client/tomcatconf/db.properties',chmod=0640) - bld.setownership("${MSCONF}/db.properties","root",bld.env.MSUSER) +bld.recurse(["ui"],'build') # apply distro-specific config on top of the 'all' generic cloud-management config globspec = _join("*","distro",bld.env.DISTRO.lower(),"*") # matches premium/distro/centos/SYSCONFDIR @@ -547,12 +382,7 @@ for dsdir in distrospecificdirs: bld.install_files_filtered(dsdirwithvar, files, cwd=start_path, relative_trick=True,chmod=mode) # build / install declarations of usage -if buildpremium: - if not Options.options.PRESERVECONFIG: - #print filelist("usage/conf/* cloudstack-proprietary/usage/conf/*") - #assert False - bld.install_files_filtered("${USAGESYSCONFDIR}","usage/conf/* cloudstack-proprietary/usage/conf/*") - bld.symlink_as("${USAGESYSCONFDIR}/db.properties",Utils.subst_vars("${MSCONF}/db.properties",bld.env)) +if buildpremium: bld.recurse(["cloudstack-proprietary/usage"],'build') # install db data files bld.install_files_filtered("${SETUPDATADIR}",filelist("*/db/* cloudstack-proprietary/*/db/*",excl=Node.exclude_regs + "\ncloud-gate\ncloud-bridge")) @@ -580,3 +410,8 @@ for vendor in _glob(_join("vendor","*")) + _glob(_join("cloudstack-proprietary", bld.install_files_filtered("${MSCONF}/%s"%vendor,filelist("%s/tomcatconf/*"%vendor)) # ====================== End vendor-specific plugins ==================== + + +# ====================== Magic! ========================================= + +bld.use_the_magic() diff --git a/wscript_configure b/wscript_configure index a7111b847cc..42ff5d6d9b5 100644 --- a/wscript_configure +++ b/wscript_configure @@ -4,10 +4,11 @@ - detects Tomcat (on Windows) - detects or configures directories according to command-line options""" -import platform import Utils,Node,Options,Logs,Scripting,Environment,Build,Configure from os import unlink as _unlink, makedirs as _makedirs, getcwd as _getcwd, chdir as _chdir -from os.path import abspath as _abspath, basename as _basename, dirname as _dirname, exists as _exists, isdir as _isdir, split as _split, join as _join, sep, pathsep, pardir +try: from os import getuid as _getuid +except ImportError: pass +from os.path import abspath as _abspath, basename as _basename, dirname as _dirname, exists as _exists, isdir as _isdir, split as _split, join as _join, expanduser as _expanduser, sep, pathsep, pardir from glob import glob as _glob @@ -88,15 +89,11 @@ hard_deps = [ ] -conf.check_tool('misc') -conf.check_tool('java') -conf.check_tool("gnu_dirs") -conf.check_tool("python") -conf.check_python_version((2,4,0)) +conf.env.VERSION = Utils.g_module.VERSION conf.check_message_1('Detecting distribution') -if platform.system() == 'Windows': conf.env.DISTRO = "Windows" -elif platform.system() == 'Darwin': conf.env.DISTRO = "Mac" +if Options.platform == 'win32': conf.env.DISTRO = "Windows" +elif Options.platform == 'darwin': conf.env.DISTRO = "Mac" elif _exists("/etc/network"): conf.env.DISTRO = "Ubuntu" elif _exists("/etc/fedora-release"): conf.env.DISTRO = "Fedora" elif _exists("/etc/centos-release") or _exists("/etc/redhat-release"): conf.env.DISTRO = "CentOS" @@ -105,50 +102,45 @@ if conf.env.DISTRO == "unknown": c = "YELLOW" else: c = "GREEN" conf.check_message_2(conf.env.DISTRO,c) -if conf.env.DISTRO not in ["Windows","Mac"]: - conf.check_tool('compiler_cc') - conf.check_cc(lib='pthread') - conf.check_cc(lib='dl') +conf.check_message_1('Detecting installation prefix') +if Options.options.prefix == Options.default_prefix: + if conf.env.DISTRO == 'Windows': + conf.env.PREFIX = 'C:\\CloudStack' + elif _getuid() != 0: # not root + conf.env.PREFIX = _join(_expanduser("~"),"cloudstack") +conf.check_message_2("%s"%conf.env.PREFIX,"GREEN") -# waf uses slashes somewhere along the line in some paths. we fix them on windows. -if conf.env.DISTRO in ['Windows']: +conf.check_tool('misc') + +conf.check_tool("gnu_dirs") +if conf.env.DISTRO == 'Windows': + # waf uses slashes somewhere along the line in some paths. we fix them on windows. for pth in [ x for x in conf.env.get_merged_dict().keys() if x.endswith("DIR") ]: + if not pth: continue + if not conf.env[pth]: continue conf.env[pth] = conf.env[pth].replace("/","\\") -for a in "DBHOST DBUSER DBPW DBDIR".split(): - conf.env[a] = getattr(Options.options, a, '') +conf.check_tool('tar') +try: conf.check_tool('mkisofs') +except Configure.ConfigurationError,e: + raise Configure.ConfigurationError, "The program genisoimage (or mkisofs) could not be found.\nOn Linux: ./waf installrpmdeps or ./waf installdebdeps according to your distro's package format.\nOn Windows: Use cygwin to install the mkisofs package, then ensure that the program is in your PATH." +conf.check_tool('java') -conf.check_message_1('Determining management server user name') -msuser = getattr(Options.options, 'MSUSER', '') -if msuser: - conf.env.MSUSER = msuser - conf.check_message_2("%s (forced through --tomcat-user)"%conf.env.MSUSER,"GREEN") -else: - if conf.env.DISTRO in ['Windows','Mac']: - conf.env.MSUSER = 'root' - conf.check_message_2("%s (not used on Windows or Mac)"%conf.env.MSUSER,"GREEN") - else: - conf.env.MSUSER = conf.env.PACKAGE - conf.check_message_2("%s (Linux default)"%conf.env.MSUSER,"GREEN") +conf.check_tool("python") +conf.check_python_version((2,4,0)) -conf.check_message_1('Detecting Tomcat') -tomcathome = getattr(Options.options, 'TOMCATHOME', '') -if tomcathome: - conf.env.TOMCATHOME = tomcathome - conf.check_message_2("%s (forced through --with-tomcat)"%conf.env.TOMCATHOME,"GREEN") -else: - if "TOMCAT_HOME" in conf.environ and conf.environ['TOMCAT_HOME'].strip(): - conf.env.TOMCATHOME = conf.environ["TOMCAT_HOME"] - conf.check_message_2("%s (got through environment variable %%TOMCAT_HOME%%)"%conf.env.TOMCATHOME,"GREEN") - elif "CATALINA_HOME" in conf.environ and conf.environ['CATALINA_HOME'].strip(): - conf.env.TOMCATHOME = conf.environ['CATALINA_HOME'] - conf.check_message_2("%s (got through environment variable %%CATALINA_HOME%%)"%conf.env.TOMCATHOME,"GREEN") - elif _isdir("/usr/share/tomcat6"): - conf.env.TOMCATHOME = "/usr/share/tomcat6" - conf.check_message_2("%s (detected existence of system directory)"%conf.env.TOMCATHOME,"GREEN") - else: - conf.env.TOMCATHOME = _join(conf.env.DATADIR,'tomcat6') - conf.check_message_2("%s (assumed presence of Tomcat there)"%conf.env.TOMCATHOME,"GREEN") +conf.check_message_1('Detecting Python MySQL module') +try: import MySQLdb +except ImportError,e: + raise Configure.ConfigurationError, "The Python MySQLdb module could not be found.\nOn Linux: ./waf installrpmdeps or ./waf installdebdeps according to your distro's package format.\nOn Windows: Install MySQL 5.1 on your machine, then install the Python MySQLdb module for Python %s.\nThe module for Python 2.6 / win32 / MySQL 5.1 is available here: http://soemin.googlecode.com/files/MySQL-python-1.2.3c1.win32-py2.6.exe"%conf.env.PYTHON_VERSION +conf.check_message_2('MySQLdb','GREEN') +conf.check_message_1('Database info for developer setup') +for a in "DBHOST DBUSER DBPW".split(): conf.env[a] = getattr(Options.options, a, '') +conf.check_message_2("user: %r, password: %r, host: %r"%(conf.env.DBUSER,conf.env.DBPW,conf.env.DBHOST),'GREEN') + +try: conf.check_tool("tomcat") +except Configure.ConfigurationError,e: + conf.fatal("Tomcat directory %r not found. Either install Tomcat using ./waf installrpmdeps or ./waf installdebdeps, or manually install Tomcat to a directory in your system and set the environment variable TOMCAT_HOME to point to it."%conf.env.TOMCATHOME) conf.env.AGENTPATH = _join(conf.env.PACKAGE,"agent") conf.env.CPPATH = _join(conf.env.PACKAGE,"console-proxy") @@ -162,32 +154,28 @@ if conf.env.DISTRO in ['Windows','Mac']: conf.env.MSCONF = _join(conf.env.TOMCATHOME,"conf") conf.env.MSLOGDIR = _join(conf.env.TOMCATHOME,"logs") conf.env.MSMNTDIR = _join(conf.env.TOMCATHOME,"mnt") + conf.env.JAVADIR = _join(conf.env.MSENVIRON,"lib") + conf.env.PREMIUMJAVADIR = conf.env.JAVADIR + conf.env.PLUGINJAVADIR = conf.env.JAVADIR + conf.env.SYSTEMJAVADIR = conf.env.JAVADIR else: conf.env.MSENVIRON = _join(conf.env.DATADIR,conf.env.MSPATH) conf.env.MSCONF = _join(conf.env.SYSCONFDIR,conf.env.MSPATH) conf.env.MSLOGDIR = _join(conf.env.LOCALSTATEDIR,"log",conf.env.MSPATH) conf.env.MSMNTDIR = _join(conf.env.SHAREDSTATEDIR,conf.env.PACKAGE,"mnt") - -conf.env.PIDDIR = _join(conf.env.LOCALSTATEDIR,"run") -conf.env.LOCKDIR = _join(conf.env.LOCALSTATEDIR,"lock","subsys") - -conf.check_message_1('Detecting JAVADIR') -javadir = getattr(Options.options, 'JAVADIR', '') -if javadir: - conf.env.JAVADIR = javadir - conf.check_message_2("%s (forced through --javadir)"%conf.env.JAVADIR,"GREEN") -elif conf.env.DISTRO in ['Windows','Mac']: - conf.env.JAVADIR = _join(conf.env['TOMCATHOME'],'lib') - conf.check_message_2("%s (using Tomcat's lib/ directory)"%conf.env.JAVADIR,"GREEN") -else: - conf.env.JAVADIR = _join(conf.env.DATADIR,'java') - conf.check_message_2("%s (using default ${DATADIR}/java directory)"%conf.env.JAVADIR,"GREEN") - -if conf.env.DISTRO in ["Windows","Mac"]: - conf.env.PREMIUMJAVADIR = conf.env.JAVADIR - conf.env.PLUGINJAVADIR = conf.env.JAVADIR - conf.env.SYSTEMJAVADIR = conf.env.JAVADIR -else: + conf.check_tool('compiler_cc') + conf.check_cc(lib='pthread') + conf.check_cc(lib='dl') + conf.check_tool('usermgmt') + conf.check_message_1('Determining management server user name') + msuser = getattr(Options.options, 'MSUSER', '') + if msuser: + conf.env.MSUSER = msuser + conf.check_message_2("%s (forced through --tomcat-user)"%conf.env.MSUSER,"GREEN") + else: + conf.env.MSUSER = conf.env.PACKAGE + conf.check_message_2("%s (Linux default)"%conf.env.MSUSER,"GREEN") + conf.check_tool("javadir") conf.env.PREMIUMJAVADIR = _join(conf.env.JAVADIR,"%s-premium"%conf.env.PACKAGE) conf.env.PLUGINJAVADIR = _join(conf.env.JAVADIR,"%s-plugins"%conf.env.PACKAGE) conf.env.SYSTEMJAVADIR = "/usr/share/java" @@ -196,6 +184,30 @@ in_javadir = lambda name: _join(conf.env.JAVADIR,_basename(name)) # $PREFIX/shar in_system_javadir = lambda name: _join(conf.env.SYSTEMJAVADIR,name) # /usr/share/java in_premiumjavadir = lambda name: _join(conf.env.PREMIUMJAVADIR,name) # $PREFIX/share/java/cloud-premium +conf.env.AGENTLIBDIR = Utils.subst_vars(_join("${LIBDIR}","${AGENTPATH}"),conf.env) +conf.env.AGENTSYSCONFDIR = Utils.subst_vars(_join("${SYSCONFDIR}","${AGENTPATH}"),conf.env) +conf.env.AGENTLOGDIR = Utils.subst_vars(_join("${LOCALSTATEDIR}","log","${AGENTPATH}"),conf.env) + +conf.env.USAGELOGDIR = Utils.subst_vars(_join("${LOCALSTATEDIR}","log","${USAGEPATH}"),conf.env) +conf.env.USAGESYSCONFDIR = Utils.subst_vars(_join("${SYSCONFDIR}","${USAGEPATH}"),conf.env) + +conf.env.CPLIBDIR = Utils.subst_vars(_join("${LIBDIR}","${CPPATH}"),conf.env) +conf.env.CPSYSCONFDIR = Utils.subst_vars(_join("${SYSCONFDIR}","${CPPATH}"),conf.env) +conf.env.CPLOGDIR = Utils.subst_vars(_join("${LOCALSTATEDIR}","log","${CPPATH}"),conf.env) + +conf.env.MSLOG = _join(conf.env.MSLOGDIR,"management-server.log") +conf.env.APISERVERLOG = _join(conf.env.MSLOGDIR,"api-server.log") +conf.env.AGENTLOG = _join(conf.env.AGENTLOGDIR,"agent.log") +conf.env.USAGELOG = _join(conf.env.USAGELOGDIR,"usage.log") +conf.env.CPLOG = _join(conf.env.CPLOGDIR,"console-proxy.log") + +conf.env.SETUPDATADIR = Utils.subst_vars(_join("${DATADIR}","${SETUPPATH}"),conf.env) + +conf.env.SERVERSYSCONFDIR = Utils.subst_vars(_join("${SYSCONFDIR}","${SERVERPATH}"),conf.env) +conf.env.PIDDIR = _join(conf.env.LOCALSTATEDIR,"run") +conf.env.LOCKDIR = _join(conf.env.LOCALSTATEDIR,"lock","subsys") + + conf.check_message_1('Building classpaths') # == Here we build the run-time classpaths == @@ -246,30 +258,7 @@ compilecp+= _glob(_join(conf.env.TOMCATHOME,'lib',"*.jar")) conf.env.CLASSPATH = pathsep.join(compilecp) conf.check_message_2('Done','GREEN') -conf.env.VERSION = Utils.g_module.VERSION - -conf.env.AGENTLIBDIR = Utils.subst_vars(_join("${LIBDIR}","${AGENTPATH}"),conf.env) -conf.env.AGENTSYSCONFDIR = Utils.subst_vars(_join("${SYSCONFDIR}","${AGENTPATH}"),conf.env) -conf.env.AGENTLOGDIR = Utils.subst_vars(_join("${LOCALSTATEDIR}","log","${AGENTPATH}"),conf.env) - -conf.env.USAGELOGDIR = Utils.subst_vars(_join("${LOCALSTATEDIR}","log","${USAGEPATH}"),conf.env) -conf.env.USAGESYSCONFDIR = Utils.subst_vars(_join("${SYSCONFDIR}","${USAGEPATH}"),conf.env) - -conf.env.CPLIBDIR = Utils.subst_vars(_join("${LIBDIR}","${CPPATH}"),conf.env) -conf.env.CPSYSCONFDIR = Utils.subst_vars(_join("${SYSCONFDIR}","${CPPATH}"),conf.env) -conf.env.CPLOGDIR = Utils.subst_vars(_join("${LOCALSTATEDIR}","log","${CPPATH}"),conf.env) - -conf.env.MSLOG = _join(conf.env.MSLOGDIR,"management-server.log") -conf.env.APISERVERLOG = _join(conf.env.MSLOGDIR,"api-server.log") -conf.env.AGENTLOG = _join(conf.env.AGENTLOGDIR,"agent.log") -conf.env.USAGELOG = _join(conf.env.USAGELOGDIR,"usage.log") -conf.env.CPLOG = _join(conf.env.CPLOGDIR,"console-proxy.log") - -conf.env.SETUPDATADIR = Utils.subst_vars(_join("${DATADIR}","${SETUPPATH}"),conf.env) - -conf.env.SERVERSYSCONFDIR = Utils.subst_vars(_join("${SYSCONFDIR}","${SERVERPATH}"),conf.env) - - # log4j config and property config files require backslash escapes on Windows +# log4j config and property config files require backslash escapes on Windows if conf.env.DISTRO in ["Windows"]: for log in "MSLOG APISERVERLOG AGENTLIBDIR USAGELOG AGENTLOG".split(): conf.env[log] = conf.env[log].replace("\\","\\\\")