cloudstack/docs/en-US/Release_Notes.xml

<?xml version='1.0' encoding='utf-8' ?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
<!ENTITY % BOOK_ENTITIES SYSTEM "cloudstack.ent">
%BOOK_ENTITIES;
]>
<!-- Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
-->
<book>
  <xi:include href="Book_Info_Release_Notes_4.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
  <xi:include href="Preface.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
  <chapter id="welcome-4.2">
    <title>Welcome to &PRODUCT; 4.2</title>
    <para>Welcome to the 4.2.0 release of &PRODUCT;, the second major release from the Apache
      CloudStack project since its graduation from the Apache Incubator. &PRODUCT; 4.2 includes more
      than 70 new features and enhancements. The focus of the release is on three major
      areas:</para>
    <itemizedlist>
      <listitem>
        <para>Improved support for both legacy-style and cloud-style workloads</para>
      </listitem>
      <listitem>
        <para>New third-party plug-in architecture</para>
      </listitem>
      <listitem>
        <para>Networking enhancements</para>
      </listitem>
    </itemizedlist>
    <para>In addition to these major new areas of functionality, &PRODUCT; 4.2 provides many
      additional enhancements in a variety of product areas. All of the new features are summarized
      later in this Release Note.</para>
    <para>This document contains information specific to this release of &PRODUCT;, including
      upgrade instructions from prior releases, new features added to &PRODUCT;, API changes, and
      issues fixed in the release. For installation instructions, please see the <ulink
        url="http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html/Installation_Guide/index.html"
        >Installation Guide</ulink>. For usage and administration instructions, please see the
        <ulink
        url="http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html/Admin_Guide/index.html"
        >&PRODUCT; Administrator's Guide</ulink>. Developers and users who wish to work with the API
      will find instruction in the <ulink
        url="http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.0.1-incubating/html/API_Developers_Guide/index.html"
        >&PRODUCT; API Developer's Guide</ulink></para>
    <para>If you find any errors or problems in this guide, please see <xref linkend="feedback"/>.
      We hope you enjoy working with &PRODUCT;!</para>
  </chapter>
  <chapter id="version-4.2">
    <title>Version 4.2.0</title>
    <section id="whats-new-in-4.2">
      <para>&PRODUCT; 4.2 includes the following new features.</para>
      <section id="workloads">
        <title>Features to Support Heterogeneous Workloads</title>
        <para>The following new features help &PRODUCT; 4.2 better support both legacy and cloud-era
          style zones.</para>
        <section id="regions">
          <title>Regions</title>
          <para>To increase reliability of the cloud, you can optionally group resources into
            geographic regions. A region is the largest available organizational unit within a cloud
            deployment. A region is made up of several availability zones, where each zone is
            equivalent to a datacenter. Each region is controlled by its own cluster of Management
            Servers, running in one of the zones. The zones in a region are typically located in
            close geographical proximity. Regions are a useful technique for providing fault
            tolerance and disaster recovery.</para>
          <para>By grouping zones into regions, the cloud can achieve higher availability and
            scalability. User accounts can span regions, so that users can deploy VMs in multiple,
            widely-dispersed regions. Even if one of the regions becomes unavailable, the services
            are still available to the end-user through VMs deployed in another region. And by
            grouping communities of zones under their own nearby Management Servers, the latency of
            communications within the cloud is reduced compared to managing widely-dispersed zones
            from a single central Management Server.</para>
          <para>Usage records can also be consolidated and tracked at the region level, creating
            reports or invoices for each geographic region.</para>
          <mediaobject>
            <imageobject>
              <imagedata fileref="./images/region-overview.png"/>
            </imageobject>
            <textobject>
              <phrase>region-overview.png: Nested structure of a region.</phrase>
            </textobject>
          </mediaobject>
        </section>
        <section id="object-store">
          <title>Object Storage Plugin Architecture</title>
          <para>Artifacts such as templates, ISOs and snapshots are kept in storage which &PRODUCT;
            refers to as secondary storage. To improve scalability and performance, as when a number
            of hosts access secondary storage concurrently, object storage can be used for secondary
            storage. Object storage can also provide built-in high availability capability. When
            using object storage, access to secondary storage data can be made available across
            multiple zones in a region. This is a huge benefit, as it is no longer necessary to copy
            templates, snapshots etc. across zones as would be needed in an NFS-only
            environment.</para>
          <para>Object storage is provided through third-party software such as Amazon Simple
            Storage Service (S3) or any other object storage that supports the S3 interface. These
            third party object storages can be integrated with &PRODUCT; by writing plugin software
            that uses the object storage plugin capability introduced in &PRODUCT; 4.2. Several new
            pluggable service interfaces are available so that different storage providers can
            develop vendor-specific plugins based on the well-defined contracts that can be
            seemlessly managed by &PRODUCT;.</para>
        </section>
        <section id="zone-wide-primary-storage">
          <title>Zone-Wide Primary Storage</title>
          <para>(Supported on KVM and VMware)</para>
          <para>In &PRODUCT; 4.2, you can provision primary storage on a per-zone basis. Data
            volumes in the primary storage can be attached to any VM on any host in the zone.</para>
          <para>In previous &PRODUCT; versions, each cluster had its own primary storage. Data in
            the primary storage was directly available only to VMs within that cluster. If a VM in a
            different cluster needed some of the data, it must be copied from one cluster to
            another, using the zone's secondary storage as an intermediate step. This operation was
            unnecessarily time-consuming.</para>
        </section>
        <section id="vmware-datacenter">
          <title>VMware Datacenter Now Visible As a &PRODUCT; Zone</title>
          <para>In order to support zone-wide functions for VMware, changes have been made so that
            &PRODUCT; is now aware of VMware Datacenters and can map each Datacenter to a &PRODUCT;
            zone. Previously, &PRODUCT; was only aware of VMware Clusters, a smaller organizational
            unit than Datacenters. This implies that a single &PRODUCT; zone could possibly contain
            clusters from different VMware Datacenters. In order for zone-wide functions, such as
            zone-wide primary storage, to work for VMware hosts, &PRODUCT; has to make sure that a
            zone contains only a single VMware Datacenter. Therefore, when you are creating a new
            &PRODUCT; zone, you will now be able to select a VMware Datacenter for the zone. If you
            are provisioning multiple VMware Datacenters, each one will be set up as a single zone
            in &PRODUCT;. </para>
          <note>
            <para>If you are upgrading from a previous &PRODUCT; version, and your existing
              deployment contains a zone with clusters from multiple VMware Datacenters, that zone
              will not be forcibly migrated to the new model. It will continue to function as
              before. However, any new zone-wide operations, such as zone-wide primary storage, will
              not be available in that zone.</para>
          </note>
          <para/>
        </section>
      </section>
      <section id="third-party-plugin">
        <title>Third-Party UI Plugin Framework</title>
        <para>Using the new third-party plugin framework, you can write and install extensions to
          &PRODUCT;. The installed and enabled plugins will appear in the UI alongside the
          Citrix-provided features.</para>
        <para>The basic procedure for adding a UI plugin is explained in the Developer Guide. In
          summary, the plugin developer creates the plugin code itself (in Javascript), a thumbnail
          image, the plugin listing, and a CSS file. The &PRODUCT; administrator adds the folder
          containing the plugin code under the &PRODUCT; PLUGINS folder and adds the plugin name to
          a configuration file (plugins.js).</para>
        <para>The next time the user refreshes the UI in the browser, the plugin will appear under
          the Plugins button in the left navigation bar.</para>
        <mediaobject>
          <imageobject>
            <imagedata fileref="./images/plugin4.jpg"/>
          </imageobject>
          <textobject>
            <phrase>plugin4.jpg: The plugin appears in the UI</phrase>
          </textobject>
        </mediaobject>
      </section>
      <section id="networking">
        <title>Networking Enhancements</title>
        <para>The following new features provide additional networking functionality in &PRODUCT;
          4.2.</para>
        <section id="ipv6">
          <title>IPv6 (Technical Preview)</title>
          <para>&PRODUCT; 4.2 introduces initial support for IPv6. This feature is provided as a
            technical preview only. Full support is planned for a future release.</para>
        </section>
        <section id="portable-ip">
          <title>Portable IPs</title>
          <para>Portable IPs in &PRODUCT; are elastic IPs that can be transferred across
            geographically separated zones. As an administrator, you can provision a pool of
            portable IPs at region level and are available for user consumption. The users can
            acquire portable IPs if admin has provisioned portable public IPs at the region level
            they are part of. These IPs can be used for any service within an advanced zone. You can
            also use portable IPs for EIP service in Basic zones. Additionally, a portable IP can be
            transferred from one network to another network.</para>
        </section>
        <section id="ntier-apps">
          <title>N-Tier Applications</title>
          <para>In &PRODUCT; 3.0.6, a functionality was added to allow users to create a multi-tier
            application connected to a single instance of a Virtual Router that supports inter-VLAN
            routing. Such a multi-tier application is called a virtual private cloud (VPC). Users
            were also able to connect their multi-tier applications to a private Gateway or a
            Site-to-Site VPN tunnel and route certain traffic to those gateways. For &PRODUCT; 4.2,
            additional features are implemented to enhance VPC applications.</para>
          <itemizedlist>
            <listitem>
              <para><xref linkend="kvm-vpc"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="deploy-vm-vpc"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="add-loadbalancer-rule-vpc"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="current-lb-vpc"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="across-tiers-lb"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="ns-support"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="configure-acl"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="acl-private-gateway"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="allow-acl"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="acl-deny"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="add-vm-tier-sharednw"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="add-gateway-vpc"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="sourcenat-private-gateway"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="eightvpn"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="static-route"/></para>
            </listitem>
            <listitem>
              <para><xref linkend="blacklist-route"/></para>
            </listitem>
          </itemizedlist>
          <section id="kvm-vpc">
            <title>Support for KVM</title>
            <para>VPC is now supported on KVM hypervisors.</para>
          </section>
          <section id="deploy-vm-vpc">
            <title>Support for Simultaneously Deploying a VM on VPC and Multiple Shared
              Networks</title>
            <para>Support for the ability to simultaneously deploy a VM on a VPC tier and one or
              more Shared networks is supported.</para>
          </section>
          <section id="add-loadbalancer-rule-vpc">
            <title>Load Balancing Support for VPC</title>
            <para>In a VPC, you can configure two types of load balancing&mdash;external LB and
              internal LB. External LB is nothing but a LB rule created to redirect the traffic
              received at a public IP of the VPC virtual router. The traffic is load balanced within
              a tier based on your configuration. Citrix NetScaler and VPC virtual router are
              supported for external LB. When you use internal LB service, traffic received at a
              tier is load balanced across different VMs within that tier. For example, traffic
              reached at Web tier is redirected to another VM in that tier. External load balancing
              devices are not supported for internal LB. The service is provided by a internal LB VM
              configured on the target tier.</para>
            <section id="current-lb-vpc">
              <title>Load Balancing Within a Tier (External LB)</title>
              <para>A &PRODUCT; user or administrator may create load balancing rules that balance
                traffic received at a public IP to one or more VMs that belong to a network tier
                that provides load balancing service in a VPC. A user creates a rule, specifies an
                algorithm, and assigns the rule to a set of VMs within a tier.</para>
            </section>
            <section id="across-tiers-lb">
              <title>Load Balancing Across Tiers</title>
              <para>&PRODUCT; supports sharing workload across different tiers within your VPC.
                Assume that multiple tiers are set up in your environment, such as Web tier and
                Application tier. Traffic to each tier is balanced on the VPC virtual router on the
                public side. If you want the traffic coming from the Web tier to the Application
                tier to be balanced, use the internal load balancing feature offered by
                &PRODUCT;.</para>
            </section>
            <section id="ns-support">
              <title>Netscaler Support for VPC</title>
              <para>Citrix NetScaler is supported for external LB. Certified version for this
                feature is NetScaler 10.0 Build 74.4006.e.</para>
            </section>
          </section>
          <section id="configure-acl">
            <title>Enhanced Access Control List</title>
            <para>Network Access Control List (ACL) on the VPC virtual router is enhanced. The
              network ACLs can be created for the tiers only if the NetworkACL service is supported.
              In &PRODUCT; terminology, Network ACL is a group of Network ACL items. Network ACL
              items are nothing but numbered rules that are evaluated in order, starting with the
              lowest numbered rule. These rules determine whether traffic is allowed in or out of
              any tier associated with the network ACL. You need to add the Network ACL items to the
              Network ACL, then associate the Network ACL with a tier. Network ACL is associated
              with a VPC and can be assigned to multiple VPC tiers within a VPC. A Tier is
              associated with a Network ACL at all the times. Each tier can be associated with only
              one ACL. </para>
            <para>The default Network ACL is used when no ACL is associated. Default behavior is all
              incoming traffic to guest networks is blocked and all outgoing traffic from guest
              networks is allowed. Default network ACL cannot be removed or modified.</para>
            <section id="acl-private-gateway">
              <title>ACL on Private Gateway</title>
              <para>The traffic on the VPC private gateway is controlled by creating both ingress
                and egress network ACL rules. The ACLs contains both allow and deny rules. As per
                the rule, all the ingress traffic to the private gateway interface and all the
                egress traffic out from the private gateway interface are blocked. You can change
                this default behaviour while creating a private gateway.</para>
            </section>
            <section id="allow-acl">
              <title>Allow ACL on All Level 4 Protocols</title>
              <para>In addition to the existing protocol support for ICMP, TCP, UDP, support for All
                Level 4 protocols is added. The protocol numbers from 0 to 255 are supported.</para>
            </section>
            <section id="acl-deny">
              <title>Support for ACL Deny Rules</title>
              <para>In addition to the existing support for ACL Allow rules, support for ACL Deny
                rules has been added in &PRODUCT; 4.2. As part of this, two operations are
                supported: Number and Action. You can configure a rule, allow or deny, by using
                action. Use Number to add a rule number.</para>
            </section>
          </section>
          <section id="add-vm-tier-sharednw">
            <title>Deploying VMs to a VPC Tier and Shared Networks</title>
            <para>&PRODUCT; allows you to deploy VMs on a VPC tier and one or more shared networks.
              With this feature, the VMs deployed in a multi-tier application can receive services
              offered by a service provider over the shared network. One example of such a service
              is monitoring service.</para>
          </section>
          <section id="add-gateway-vpc">
            <title>Adding a Private Gateway to a VPC</title>
            <para>A private gateway can be added by the root admin only. The VPC private network has
              1:1 relationship with the NIC of the physical network. You can configure multiple
              private gateways to a single VPC. No gateways with duplicated VLAN and IP are allowed
              in the same data center.</para>
            <section id="sourcenat-private-gateway">
              <title>Source NAT on Private Gateway</title>
              <para>You might want to deploy multiple VPCs with the same super CIDR and guest tier
                CIDR. Therefore, multiple guest VMs from different VPCs can have the same IPs to
                reach a enterprise data center through the private gateway. In such cases, a NAT
                service need to be configured on the private gateway. If Source NAT is enabled, the
                guest VMs in VPC reaches the enterprise network via private gateway IP address by
                using the NAT service. </para>
              <para>The Source NAT service on a private gateway can be enabled while adding the
                private gateway. On deletion of a private gateway, source NAT rules specific to the
                private gateway are deleted.</para>
            </section>
            <section id="eightvpn">
              <title>VPN Gateways</title>
              <para>Support up to 8 VPN Gateways is added.</para>
            </section>
            <section id="static-route">
              <title>Creating a Static Route</title>
              <para>&PRODUCT; enables you to specify routing for the VPN connection you create. You
                can enter one or CIDR addresses to indicate which traffic is to be routed back to
                the gateway.</para>
            </section>
            <section id="blacklist-route">
              <title>Blacklisting Routes</title>
              <para>&PRODUCT; enables you to block a list of routes so that they are not assigned to
                any of the VPC private gateways. Specify the list of routes that you want to
                blacklist in the <code>blacklisted.routes</code> global parameter. Note that the
                parameter update affects only new static route creations. If you block an existing
                static route, it remains intact and continue functioning. You cannot add a static
                route if the route is blacklisted for the zone. </para>
            </section>
          </section>
        </section>
        <section id="vlan-assign-isolated-nw">
          <title>Assigning VLANs to Isolated Networks</title>
          <para>&PRODUCT; provides you the ability to control VLAN assignment to Isolated networks.
            You can assign a VLAN ID when a network is created, just the way it's done for Shared
            networks.</para>
          <para>The former behaviour also is supported &mdash; VLAN is randomly allocated to a
            network from the VNET range of the physical network when the network turns to
            Implemented state. The VLAN is released back to the VNET pool when the network shuts
            down as a part of the Network Garbage Collection. The VLAN can be re-used either by the
            same network when it is implemented again, or by any other network. On each subsequent
            implementation of a network, a new VLAN can be assigned.</para>
          <note>
            <para>You cannot change a VLAN once it's assigned to the network. The VLAN remains with
              the network for its entire life cycle.</para>
          </note>
        </section>
        <section id="persistent-network">
          <title>Persistent Networks</title>
          <para>&PRODUCT; 4.2 supports Persistent Networks. The network that you can provision
            without having to deploy any VMs on it is called a Persistent Network. A Persistent
            Network can be part of a VPC or a non-VPC environment. With the addition of this
            feature, you will have the ability to create a network in &PRODUCT; in which physical
            devices can be deployed without having to run any VMs. Additionally, you can deploy
            physical devices on that network. Another advantages is that you can create a VPC with a
            tier that consists only physical devices. For example, you might create a VPC for a
            three-tier application, deploy VMs for Web and Application tier, and use physical
            machines for the Database tier. Another use case is that if you are providing services
            by using physical hardware, you can define the network as persistent and therefore even
            if all its VMs are destroyed the services will not be discontinued.</para>
        </section>
        <section id="vnmc-cisco">
          <title>Cisco VNMC Support</title>
          <para>Cisco Virtual Network Management Center (VNMC) provides centralized multi-device and
            policy management for Cisco Network Virtual Services. When Cisco VNMC is integrated with
            ASA 1000v Cloud Firewall and Cisco Nexus 1000v dvSwitch in &PRODUCT; you will be able
            to: </para>
          <itemizedlist>
            <listitem>
              <para>Configure Cisco ASA 1000v Firewalls</para>
            </listitem>
            <listitem>
              <para>Create and apply security profiles that contain ACL policy sets for both ingress
                and egress traffic, and NAT policy sets</para>
            </listitem>
          </itemizedlist>
          <para>&PRODUCT; supports Cisco VNMC on Cisco Nexus 1000v dvSwich-enabled VMware
            hypervisors.</para>
        </section>
        <section id="vmware-vswitch">
          <title>VMware vNetwork Distributed vSwitch</title>
          <para>&PRODUCT; supports VMware vSphere Distributed Switch (VDS) for virtual network
            configuration in a VMware vSphere environment. Each vCenter server instance can support
            up to 128 VDSs and each VDS can manage up to 500 VMware hosts. &PRODUCT; supports
            configuring virtual networks in a deployment with a mix of Virtual Distributed Switch,
            Standard Virtual Switch and Nexus 1000v Virtual Switch. </para>
        </section>
        <section id="reserved-ip-addresses-non-csvms">
          <title>IP Reservation in Isolated Guest Networks</title>
          <para>In Isolated guest networks in &PRODUCT; 4.2, a part of the guest IP address space
            can be reserved for non-&PRODUCT; VMs or physical servers. To do so, you configure a
            range of Reserved IP addresses by specifying the CIDR when a guest network is in
            Implemented state. The advantage of having this feature is that if your customers wish
            to have non-&PRODUCT; controlled VMs or physical servers on the same network, they can
            use a part of the IP address space that is primarily provided to the guest network. When
            IP reservation is configured, the administrator can add additional VMs or physical
            servers that are not part of &PRODUCT; to the same network and assign them the Reserved
            IP addresses. &PRODUCT; guest VMs cannot acquire IPs from the Reserved IP Range.</para>
        </section>
        <section id="ip-vlan-tenant">
          <title>Dedicated Resources: Public IP Addresses and VLANs Per Account</title>
          <para>&PRODUCT; provides you the ability to reserve a set of public IP addresses and VLANs
            exclusively for an account. During zone creation, you can continue to define a set of
            VLANs and multiple public IP ranges. This feature extends the functionality to enable
            you to dedicate a fixed set of VLANs and guest IP addresses for a tenant.</para>
          <para>This feature provides you the following capabilities:</para>
          <itemizedlist>
            <listitem>
              <para>Reserve a VLAN range and public IP address range from an Advanced zone and
                assign it to an account</para>
            </listitem>
            <listitem>
              <para>Disassociate a VLAN and public IP address range from an account</para>
            </listitem>
          </itemizedlist>
          <note>
            <para>Ensure that you check whether the required range is available and conforms to
              account limits. The maximum IPs per account limit cannot be superseded.</para>
          </note>
        </section>
        <section id="egress-firewall">
          <title>Enhanced Juniper SRX Support for Egress Firewall Rules</title>
          <para>Egress firewall rules were previously supported on virtual routers, and now they are
            also supported on Juniper SRX external networking devices.</para>
          <para>Egress traffic originates from a private network to a public network, such as the
            Internet. By default, the egress traffic is blocked, so no outgoing traffic is allowed
            from a guest network to the Internet. However, you can control the egress traffic in an
            Advanced zone by creating egress firewall rules. When an egress firewall rule is
            applied, the traffic specific to the rule is allowed and the remaining traffic is
            blocked. When all the firewall rules are removed the default policy, Block, is
            applied.</para>
          <note>
            <para>Egress firewall rules are not supported on Shared networks. They are supported
              only on Isolated guest networks.</para>
          </note>
        </section>
        <section id="default-egress-policy">
          <title>Configuring the Default Egress Policy</title>
          <para>The default egress policy for Isolated guest network can be configured by using
            Network offering. Use the create network offering option to determine whether the
            default policy should be block or allow all the traffic to the public network from a
            guest network. Use this network offering to create the network. If no policy is
            specified, by default all the traffic is allowed from the guest network that you create
            by using this network offering.</para>
          <para>You have two options: Allow and Deny.</para>
          <para>If you select Allow for a network offering, by default egress traffic is allowed.
            However, when an egress rule is configured for a guest network, rules are applied to
            block the specified traffic and rest are allowed. If no egress rules are configured for
            the network, egress traffic is accepted. If you select Deny for a network offering, by
            default egress traffic for the guest network is blocked. However, when an egress rules
            is configured for a guest network, rules are applied to allow the specified traffic.
            While implementing a guest network, &PRODUCT; adds the firewall egress rule specific to
            the default egress policy for the guest network.</para>
          <para>This feature is supported only on virtual router and Juniper SRX.</para>
        </section>
        <section id="non-contiguous-vlan">
          <title>Non-Contiguous VLAN Ranges</title>
          <para>&PRODUCT; provides you with the flexibility to add non contiguous VLAN ranges to
            your network. The administrator can either update an existing VLAN range or add multiple
            non contiguous VLAN ranges while creating a zone. You can also use the
            UpdatephysicalNetwork API to extend the VLAN range.</para>
        </section>
        <section id="pvlan">
          <title>Isolation in Advanced Zone Using Private VLAN</title>
          <para>Isolation of guest traffic in shared networks can be achieved by using Private VLANs
            (PVLAN). PVLANs provide Layer 2 isolation between ports within the same VLAN. In a
            PVLAN-enabled shared network, a user VM cannot reach other user VM though they can reach
            the DHCP server and gateway, this would in turn allow users to control traffic within a
            network and help them deploy multiple applications without communication between
            application as well as prevent communication with other users’ VMs.</para>
          <itemizedlist>
            <listitem>
              <para>Isolate VMs in a shared networks by using Private VLANs.</para>
            </listitem>
            <listitem>
              <para>Supported on KVM, XenServer, and VMware hypervisors.</para>
            </listitem>
            <listitem>
              <para>PVLAN-enabled shared network can be a part of multiple networks of a guest VM.
              </para>
            </listitem>
          </itemizedlist>
          <para>For further reading:</para>
          <itemizedlist>
            <listitem>
              <para><ulink
                  url="http://www.cisco.com/en/US/docs/switches/lan/catalyst3750/software/release/12.2_25_see/configuration/guide/swpvlan.html#wp1038379"
                  >Understanding Private VLANs</ulink></para>
            </listitem>
            <listitem>
              <para><ulink url="http://tools.ietf.org/html/rfc5517">Cisco Systems' Private VLANs:
                  Scalable Security in a Multi-Client Environment</ulink></para>
            </listitem>
            <listitem>
              <para><ulink url="http://kb.vmware.com">Private VLAN (PVLAN) on vNetwork Distributed
                  Switch - Concept Overview (1010691)</ulink></para>
            </listitem>
          </itemizedlist>
        </section>
        <section id="multiple-ip-nic">
          <title>Configuring Multiple IP Addresses on a Single NIC</title>
          <para>(Supported on XenServer, KVM, and VMware hypervisors)</para>
          <para>&PRODUCT; now provides you the ability to associate multiple private IP addresses
            per guest VM NIC. This feature is supported on all the network
            configurations&mdash;Basic, Advanced, and VPC. Security Groups, Static NAT and Port
            forwarding services are supported on these additional IPs. In addition to the primary
            IP, you can assign additional IPs to the guest VM NIC. Up to 256 IP addresses are
            allowed per NIC.</para>
          <para>As always, you can specify an IP from the guest subnet; if not specified, an IP is
            automatically picked up from the guest VM subnet. You can view the IPs associated with
            for each guest VM NICs on the UI. You can apply NAT on these additional guest IPs by
            using firewall configuration in the &PRODUCT; UI. You must specify the NIC to which the
            IP should be associated.</para>
        </section>
        <section id="multiple-ip-range">
          <title>Adding Multiple IP Ranges</title>
          <para>(Supported on KVM, xenServer, and VMware hypervisors)</para>
          <para>&PRODUCT; 4.2 provides you with the flexibility to add guest IP ranges from
            different subnets in Basic zones and security groups-enabled Advanced zones. For
            security groups-enabled Advanced zones, it implies multiple subnets can be added to the
            same VLAN. With the addition of this feature, you will be able to add IP address ranges
            from the same subnet or from a different one when IP address are exhausted. This would
            in turn allows you to employ higher number of subnets and thus reduce the address
            management overhead.</para>
          <para>Ensure that you manually configure the gateway of the new subnet before adding the
            IP range. Note that &PRODUCT; supports only one gateway for a subnet; overlapping
            subnets are not currently supported.</para>
          <para>You can also delete IP ranges. This operation fails if an IP from the remove range
            is in use. If the remove range contains the IP address on which the DHCP server is
            running, &PRODUCT; acquires a new IP from the same subnet. If no IP is available in the
            subnet, the remove operation fails.</para>
          <note>
            <para>The feature can only be implemented on IPv4 addresses.</para>
          </note>
        </section>
        <section id="add-remove-network-vm">
          <title>Support for Multiple Networks in VMs</title>
          <para>(Supported on XenServer, VMware and KVM hypervisors)</para>
          <para>&PRODUCT; 4.2 provides you the ability to add and remove multiple networks to a VM.
            You can remove a network from a VM and add a new network. You can also change the
            default network of a VM. With this functionality, hybrid or traditional server loads can
            be accommodated with ease. </para>
          <para>For adding or removing a NIC to work on VMware, ensure that vm-tools are running on
            guest VMs.</para>
        </section>
        <section id="gslb">
          <title>Global Server Load Balancing</title>
          <para>&PRODUCT; 4.2 supports Global Server Load Balancing (GSLB) functionalities to
            provide business continuity by load balancing traffic to an instance on active zones
            only in case of zone failures . &PRODUCT; achieve this by extending its functionality of
            integrating with NetScaler Application Delivery Controller (ADC), which also provides
            various GSLB capabilities, such as disaster recovery and load balancing. The DNS
            redirection technique is used to achieve GSLB in &PRODUCT;. In order to support this
            functionality, region level services and service provider are introduced. A new service
            'GSLB' is introduced as a region level service. The GSLB service provider is introduced
            that will provider the GSLB service. Currently, NetScaler is the supported GSLB provider
            in &PRODUCT;. GSLB functionality works in an Active-Active data center environment.
          </para>
        </section>
        <section id="lb-on-shared-vlan">
          <title>Enhanced Load Balancing Services Using External Provider on Shared VLANs</title>
          <para>Network services like Firewall, Load Balancing, and NAT are now supported in shared
            networks created in an advanced zone. In effect, the following network services shall be
            made available to a VM in a shared network: Source NAT, Static NAT, Port Forwarding,
            Firewall and Load balancing. Subset of these service can be chosen while creating a
            network offering for shared networks. Services available in a shared network is defined
            by the network offering and the service chosen in the network offering. For example, if
            network offering for a shared network has source NAT service enabled, a public IP shall
            be provisioned and source NAT is configured on the firewall device to provide public
            access to the VMs on the shared network. Static NAT, Port Forwarding, Load Balancing,
            and Firewall services shall be available only on the acquired public IPs associated with
            a shared network.</para>
          <para>Additionally, Netscaler and Juniper SRX firewall device can be configured inline or
            side-by-side mode.</para>
        </section>
        <section id="health-check">
          <title>Health Checks for Load Balanced Instances</title>
          <note>
            <para>This feature is supported only on NetScaler version 10.0 and beyond.</para>
          </note>
          <para>(NetScaler load balancer only) A load balancer rule distributes requests among a
            pool of services (a service in this context means an application running on a virtual
            machine). When creating a load balancer rule, you can specify a health check which will
            ensure that the rule forwards requests only to services that are healthy (running and
            available). When a health check is in effect, the load balancer will stop forwarding
            requests to any resources that it has found to be unhealthy. If the resource later
            becomes available again, the periodic health check (periodicity is configurable) will
            discover it and the resource will once again be made available to the load
            balancer.</para>
          <para>To configure how often the health check is performed by default, use the global
            configuration setting healthcheck.update.interval. This default applies to all the
            health check policies in the cloud. You can override this value for an individual health
            check policy.</para>
        </section>
      </section>
      <section id="host-and-vm-enhancements">
        <title>Host and Virtual Machine Enhancements</title>
        <para>The following new features expand the ways you can use hosts and virtual
          machines.</para>
        <section id="vmware-drs">
          <title>VMware DRS Support</title>
          <para>The VMware vSphere Distributed Resources Scheduler (DRS) is supported.</para>
        </section>
        <section id="windows-8">
          <title>Windows 8 and Windows Server 2012 as VM Guest OS</title>
          <para>(Supported on XenServer, VMware, and KVM)</para>
          <para>Windows 8 and Windows Server 2012 can now be used as OS types on guest virtual
            machines. The OS would be made available the same as any other, by uploading an ISO or a
            template. The instructions for uploading ISOs and templates are given in the
            Administrator's Guide. </para>
          <note>
            <para><emphasis role="bold">Limitation:</emphasis> When used with VMware hosts, this
              feature works only for the following versions: vSphere ESXi 5.1 and ESXi 5.0 Patch
              4.</para>
          </note>
          <para/>
        </section>
        <section id="change-account">
          <title>Change Account Ownership of Virtual Machines</title>
          <para>A root administrator can now change the ownership of any virtual machine from one
            account to any other account. A domain or sub-domain administrator can do the same for
            VMs within the domain from one account to any other account in the domain.</para>
        </section>
        <section id="dedicated-resources">
          <title>Private Pod, Cluster, or Host</title>
          <para>Dedicating pod, cluster or host to a specific domain/account means that the
            domain/account will have sole access to the dedicated pod, cluster or hosts such that
            scalability, security and manageability within a domain/account can be improved. The
            resources which belong to that tenant will be placed into that dedicated pod, cluster or
            host.</para>
        </section>
        <section id="resize-volume">
          <title>Resizing Volumes</title>
          <para>&PRODUCT; provides the ability to resize data disks; &PRODUCT; controls volume size
            by using disk offerings. This provides &PRODUCT; administrators with the flexibility to
            choose how much space they want to make available to the end users. Volumes within the
            disk offerings with the same storage tag can be resized. For example, if you only want
            to offer 10, 50, and 100 GB offerings, the allowed resize should stay within those
            limits. That implies if you define a 10 GB, a 50 GB and a 100 GB disk offerings, a user
            can upgrade from 10 GB to 50 GB, or 50 GB to 100 GB. If you create a custom-sized disk
            offering, then you have the option to resize the volume by specifying a new, larger
            size. Additionally, using the resizeVolume API, a data volume can be moved from a static
            disk offering to a custom disk offering with the size specified. This functionality
            allows those who might be billing by certain volume sizes or disk offerings to stick to
            that model, while providing the flexibility to migrate to whatever custom size
            necessary. This feature is supported on KVM, XenServer, and VMware hosts. However,
            shrinking volumes is not supported on VMware hosts</para>
        </section>
        <section id="volume-snapshot-enhancement">
          <title>VMware Volume Snapshot Improved Performance</title>
          <para>When you take a snapshot of a data volume on VMware, &PRODUCT; will now use a more
            efficient storage technique to improve performance.</para>
          <para>Previously, every snapshot was immediately exported from vCenter to a mounted NFS
            share and packaged into an OVA file format. This operation consumed time and resources.
            Starting from 4.2, the original file formats (e.g., VMDK) provided by vCenter will be
            retained. An OVA file will only be created as needed, on demand.</para>
          <para>The new process applies only to newly created snapshots after upgrade to &PRODUCT;
            4.2. Snapshots that have already been taken and stored in OVA format will continue to
            exist in that format, and will continue to work as expected.</para>
        </section>
        <section id="storage-migration">
          <title>Storage Migration: XenMotion and vMotion</title>
          <para>(Supported on XenServer and VMware)</para>
          <para>Storage migration allows VMs to be moved from one host to another, where the VMs are
            not located on storage shared between the two hosts. It provides the option to live
            migrate a VM’s disks along with the VM itself. It is now possible to migrate a VM from
            one XenServer resource pool / VMware cluster to another, or to migrate a VM whose disks
            are on local storage, or even to migrate a VM’s disks from one storage repository to
            another, all while the VM is running.</para>
        </section>
        <section id="vmware-configure-linked-clones">
          <title>Configuring Usage of Linked Clones on VMware</title>
          <para>(For ESX hypervisor in conjunction with vCenter)</para>
          <para>In &PRODUCT; 4.2, the creation of VMs as full clones is allowed. In previous
            versions, only linked clones were possible.</para>
          <para>For a full description of clone types, refer to VMware documentation. In summary: A
            full clone is a copy of an existing virtual machine which, once created, does not depend
            in any way on the original virtual machine. A linked clone is also a copy of an existing
            virtual machine, but it has ongoing dependency on the original. A linked clone shares
            the virtual disk of the original VM, and retains access to all files that were present
            at the time the clone was created.</para>
          <para>A new global configuration setting has been added, vmware.create.full.clone. When
            the administrator sets this to true, end users can create guest VMs only as full clones.
            The default value is true for new installations. For customers upgrading from a previous
            version of &PRODUCT;, the default value of vmware.create.full.clone is false.</para>
        </section>
        <section id="host-deployment-rules">
          <title>VM Deployment Rules</title>
          <para>Rules can be set up to ensure that particular VMs are not placed on the same
            physical host. These "anti-affinity rules" can increase the reliability of applications
            by ensuring that the failure of a single host can not take down the entire group of VMs
            supporting a given application. See Affinity Groups in the &PRODUCT; 4.2 Administration
            Guide.</para>
        </section>
        <section id="cpu-ram-dynamic-scaling">
          <title>CPU and Memory Scaling for Running VMs</title>
          <para>(Supported on VMware and XenServer)</para>
          <para>You can now change the CPU and RAM values for a running virtual machine. In previous
            versions of &PRODUCT;, this could only be done on a stopped VM.</para>
          <para>It is not always possible to accurately predict the CPU and RAM requirements when
            you first deploy a VM. You might need to increase or decrease these resources at any
            time during the life of a VM. With the new ability to dynamically modify CPU and RAM
            levels, you can change these resources for a running VM without incurring any
            downtime.</para>
          <para>Dynamic CPU and RAM scaling can be used in the following cases:</para>
          <itemizedlist>
            <listitem>
              <para>New VMs that are created after the installation of &PRODUCT; 4.2. If you are
                upgrading from a previous version of &PRODUCT;, your existing VMs created with
                previous versions will not have the dynamic scaling capability.</para>
            </listitem>
            <listitem>
              <para>User VMs on hosts running VMware and XenServer.</para>
            </listitem>
            <listitem>
              <para>System VMs on VMware.</para>
            </listitem>
            <listitem>
              <para>VM Tools or XenServer Tools must be installed on the virtual machine.</para>
            </listitem>
            <listitem>
              <para>The new requested CPU and RAM values must be within the constraints allowed by
                the hypervisor and the VM operating system.</para>
            </listitem>
          </itemizedlist>
          <para>To configure this feature, use the following new global configuration
            variables:</para>
          <itemizedlist>
            <listitem>
              <para>enable.dynamic.scale.vm: Set to True to enable the feature. By default, the
                feature is turned off.</para>
            </listitem>
            <listitem>
              <para>scale.retry: How many times to attempt the scaling operation. Default =
                2.</para>
            </listitem>
          </itemizedlist>
        </section>
        <section id="cpu-ram-overcommit">
          <title>CPU and Memory Over-Provisioning</title>
          <para>(Supported for XenServer, KVM, and VMware)</para>
          <para>In &PRODUCT; 4.2, CPU and memory (RAM) over-provisioning factors can be set for each
            cluster to change the number of VMs that can run on each host in the cluster. This helps
            optimize the use of resources. By increasing the over-provisioning ratio, more resource
            capacity will be used. If the ratio is set to 1, no over-provisioning is done.</para>
          <para>In previous releases, &PRODUCT; did not perform memory over-provisioning. It
            performed CPU over-provisioning based on a ratio configured by the administrator in the
            global configuration setting cpu.overprovisioning.factor. Starting in 4.2, the
            administrator can specify a memory over-provisioning ratio, and can specify both CPU and
            memory over-provisioning ratios on a per-cluster basis, rather than only on a global
            basis.</para>
          <para>In any given cloud, the optimum number of VMs for each host is affected by such
            things as the hypervisor, storage, and hardware configuration. These may be different
            for each cluster in the same cloud. A single global over-provisioning setting could not
            provide the best utilization for all the different clusters in the cloud. It had to be
            set for the lowest common denominator. The new per-cluster setting provides a finer
            granularity for better utilization of resources, no matter where the &PRODUCT; placement
            algorithm decides to place a VM.</para>
        </section>
        <section id="baremetal">
          <title>Kickstart Installation for Bare Metal Provisioning</title>
          <para>&PRODUCT; 4.2 supports the kick start installation method for RPM-based Linux
            operating systems on baremetal hosts in basic zones. Users can provision a baremetal
            host managed by &PRODUCT; as long as they have the kick start file and corresponding OS
            installation ISO ready.</para>
          <para>Tested on CentOS 5.5, CentOS 6.2, CentOS 6.3, Ubuntu 12.04.</para>
          <para>For more information, see the Baremetal Installation Guide.</para>
        </section>
        <section id="baremetal-ucs">
          <title>Enhanced Bare Metal Support on Cisco UCS</title>
          <para>You can now more easily provision new Cisco UCS server blades into &PRODUCT; for use
            as bare metal hosts. The goal is to enable easy expansion of the cloud by leveraging the
            programmability of the UCS converged infrastructure and &PRODUCT;’s knowledge of the
            cloud architecture and ability to orchestrate. With this new feature, &PRODUCT; can
            automatically understand the UCS environment, server profiles, etc. to make it easy to
            deploy a bare metal OS on a Cisco UCS.</para>
        </section>
        <section id="update-vm-image">
          <title>Changing a VM's Base Image</title>
          <para>Every VM is created from a base image, which is a template or ISO which has been
            created and stored in &PRODUCT;. Both cloud administrators and end users can create and
            modify templates, ISOs, and VMs.</para>
          <para>In &PRODUCT; 4.2, there is a new way to modify an existing VM. You can change an
            existing VM from one base image to another. For example, suppose there is a template
            based on a particular operating system, and the OS vendor releases a software patch. The
            administrator or user naturally wants to apply the patch and then make sure existing VMs
            start using it. Whether a software update is involved or not, it's also possible to
            simply switch a VM from its current template to any other desired template.</para>
        </section>
        <section id="reset-vm-reboot">
          <title>Reset VM on Reboot</title>
          <para>In &PRODUCT; 4.2, you can specify that you want to discard the root disk and create
            a new one whenever a given VM is rebooted. This is useful for secure environments that
            need a fresh start on every boot and for desktops that should not retain state. The IP
            address of the VM will not change due to this operation.</para>
        </section>
        <section id="vm-snapshots">
          <title>Virtual Machine Snapshots for VMware</title>
          <para>(VMware hosts only) In addition to the existing &PRODUCT; ability to snapshot
            individual VM volumes, you can now take a VM snapshot to preserve all the VM's data
            volumes as well as (optionally) its CPU/memory state. This is useful for quick restore
            of a VM. For example, you can snapshot a VM, then make changes such as software
            upgrades. If anything goes wrong, simply restore the VM to its previous state using the
            previously saved VM snapshot. </para>
          <para>The snapshot is created using the VMware native snapshot facility. The VM snapshot
            includes not only the data volumes, but optionally also whether the VM is running or
            turned off (CPU state) and the memory contents. The snapshot is stored in &PRODUCT;'s
            primary storage.</para>
          <para>VM snapshots can have a parent/child relationship. Each successive snapshot of the
            same VM is the child of the snapshot that came before it. Each time you take an
            additional snapshot of the same VM, it saves only the differences between the current
            state of the VM and the state stored in the most recent previous snapshot. The previous
            snapshot becomes a parent, and the new snapshot is its child. It is possible to create a
            long chain of these parent/child snapshots, which amount to a "redo" record leading from
            the current state of the VM back to the original.</para>
        </section>
        <section id="vm-userdata">
          <title>Increased Userdata Size When Deploying a VM</title>
          <para>You can now specify up to 32KB of userdata when deploying a virtual machine through
            the &PRODUCT; UI or the deployVirtualMachine API call. </para>
        </section>
        <section id="vmware-cluster-limit">
          <title>Set VMware Cluster Size Limit Depending on VMware Version</title>
          <para>The maximum number of hosts in a vSphere cluster is determined by the VMware
            hypervisor software. For VMware versions 4.2, 4.1, 5.0, and 5.1, the limit is 32
            hosts.</para>
          <para>For &PRODUCT; 4.2, the global configuration setting vmware.percluster.host.max has
            been removed. The maximum number of hosts in a VMware cluster is now determined by the
            underlying hypervisor software.</para>
          <note>
            <para>Best Practice: It is advisable for VMware clusters in &PRODUCT; to be smaller than
              the VMware hypervisor's maximum size. A cluster size of up to 8 hosts has been found
              optimal for most real-world situations.</para>
          </note>
        </section>
        <section id="limit-accounts-domains-rn">
          <title>Limiting Resource Usage</title>
          <para>Previously in &PRODUCT;, resource usage limit was imposed based on the resource
            count, that is, restrict a user or domain on the basis of the number of VMs, volumes, or
            snapshots used. In &PRODUCT; 4.2, a new set of resource types has been added to the
            existing pool of resources (VMs, Volumes, and Snapshots) to support the customization
            model&mdash;need-basis usage, such as large VM or small VM. The new resource types are
            now broadly classified as CPU, RAM, Primary storage, and Secondary storage. &PRODUCT;
            4.2 allows the root administrator to impose resource usage limit by the following
            resource types for Domain, Project and Accounts. </para>
          <itemizedlist>
            <listitem>
              <para>CPUs</para>
            </listitem>
            <listitem>
              <para>Memory (RAM)</para>
            </listitem>
            <listitem>
              <para>Primary Storage (Volumes)</para>
            </listitem>
            <listitem>
              <para>Secondary Storage (Snapshots, Templates, ISOs)</para>
            </listitem>
          </itemizedlist>
        </section>
      </section>
      <section id="ops">
        <title>Monitoring, Maintenance, and Operations Enhancements</title>
        <!-- <section id="mixed-zone">
        <title>Basic and Advanced Zone Views</title>
        <para>A new dropdown in the UI provides the choice between Basic and Advanced views.
          Depending on which type of zone you are working with, you can change the view to show only
          &PRODUCT; features and choices that are relevant for that type of zone. This makes it
          easier to work with basic zones, since features that are not relevant will not be shown in
          the UI.</para>
      </section> -->
        <section id="events-pubsub">
          <title>Publish and Subscribe for Event Notification</title>
          <para>An event is essentially a significant or meaningful change in the state of both
            virtual and physical resources associated with a cloud environment. In &PRODUCT; an
            event could be a state change of virtual or psychical resources, an action performed by
            an user (action events), or policy based events (alerts). In &PRODUCT; 4.2, a new event
            notification framework has been added. This framework provides a means for the
            Management Server components to publish and subscribe to &PRODUCT; events. Event
            notification is achieved by implementing the concept of event bus abstraction in the
            Management Server. </para>
          <para>A new event for state change, resource state change, is introduced as part of Event
            notification framework. Every resource, such as user VM, volume, NIC, network, public
            IP, snapshot, and template, is associated with a state machine and generates events as
            part of the state change. That implies that a change in the state of a resource results
            in a state change event, and the event is published in the corresponding state machine
            on the event bus. All the &PRODUCT; events (alerts, action events, usage events) and the
            additional category of resource state change events, are published on to the events
            bus.</para>
        </section>
        <section id="delete-alerts">
          <title>Deleting and Archiving Events and Alerts</title>
          <para>In addition to viewing a list of events and alerts in the UI, the administrator can
            now delete and archive them. In order to support deleting and archiving alerts, the
            following global parameters have been added:</para>
          <itemizedlist>
            <listitem>
              <para><emphasis role="bold">alert.purge.delay</emphasis>: The alerts older than
                specified number of days are purged. Set the value to 0 to never purge alerts
                automatically.</para>
            </listitem>
            <listitem>
              <para><emphasis role="bold">alert.purge.interval</emphasis>: The interval in seconds
                to wait before running the alert purge thread. The default is 86400 seconds (one
                day).</para>
            </listitem>
          </itemizedlist>
          <note>
            <para>Archived alerts or events cannot be viewed in the UI, or by using the API. They
              are maintained in the database for auditing or compliance purposes.</para>
          </note>
        </section>
        <section id="global-parameters">
          <title>Increased Granularity for Configuration Parameters</title>
          <para>Some configuration parameters which were previously available only at the global
            level of the cloud can now be set for smaller components of the cloud, such as at the
            zone level. To set these parameters, look for the new Settings tab in the UI. You will
            find it on the detail page for an account, cluster, zone, or primary storage.</para>
          <para>The account level parameters are: <code>remote.access.vpn.client.iprange</code>,
              <code>allow.public.user.templates</code>, <code>use.system.public.ips</code>, and
              <code>use.system.guest.vlans</code></para>
          <para>The cluster level parameters are
              <code>cluster.storage.allocated.capacity.notificationthreshold</code>,
              <code>cluster.storage.capacity.notificationthreshold</code>,
              <code>cluster.cpu.allocated.capacity.notificationthreshold</code>,
              <code>cluster.memory.allocated.capacity.notificationthreshold</code>, <code>
              cluster.cpu.allocated.capacity.disablethreshold</code>,
              <code>cluster.memory.allocated.capacity.disablethreshold</code>,
              <code>cpu.overprovisioning.factor</code>, <code>mem.overprovisioning.factor</code>,
              <code>vmware.reserve.cpu</code>, and <code>vmware.reserve.mem</code>.</para>
          <para>The zone level parameters are
              <code>pool.storage.allocated.capacity.disablethreshold</code>,
              <code>pool.storage.capacity.disablethreshold</code>,
              <code>storage.overprovisioning.factor</code>, <code>network.throttling.rate</code>,
              <code>guest.domain.suffix</code>, <code>router.template.xen</code>,
              <code>router.template.kvm</code>, <code>router.template.vmware</code>,
              <code>router.template.hyperv</code>, <code>router.template.lx</code>c,
              <code>enable.dynamic.scale.vm</code>, <code>use.external.dns</code>, and
              <code>blacklisted.routes</code>.</para>
        </section>
        <section id="api-request-throttling">
          <title>API Request Throttling</title>
          <para>In &PRODUCT; 4.2, you can limit the rate at which API requests can be placed for
            each account. This is useful to avoid malicious attacks on the Management Server,
            prevent performance degradation, and provide fairness to all accounts.</para>
          <para>If the number of API calls exceeds the threshold, an error message is returned for
            any additional API calls. The caller will have to retry these API calls at another
            time.</para>
          <para>To control the API request throttling, use the following new global configuration
            settings:</para>
          <itemizedlist>
            <listitem>
              <para>api.throttling.enabled - Enable/Disable API throttling. By default, this setting
                is false, so API throttling is not enabled.</para>
            </listitem>
            <listitem>
              <para>api.throttling.interval (in seconds) - Time interval during which the number of
                API requests is to be counted. When the interval has passed, the API count is reset
                to 0.</para>
            </listitem>
            <listitem>
              <para>api.throttling.max - Maximum number of APIs that can be placed within the
                api.throttling.interval period.</para>
            </listitem>
            <listitem>
              <para>api.throttling.cachesize - Cache size for storing API counters. Use a value
                higher than the total number of accounts managed by the cloud. One cache entry is
                needed for each account, to store the running API total for that account within the
                current time window.</para>
            </listitem>
          </itemizedlist>
        </section>
        <section id="external-alert-managers">
          <title>Sending Alerts to External SNMP and Syslog Managers</title>
          <para>In addition to showing administrator alerts on the Dashboard in the &PRODUCT; UI and
            sending them in email, &PRODUCT; now can also send the same alerts to external SNMP or
            Syslog management software. This is useful if you prefer to use an SNMP or Syslog
            manager to monitor your cloud.</para>
          <para>The supported protocol is SNMP version 2.</para>
        </section>
        <section id="default-pwd-engine">
          <title>Changing the Default Password Encryption</title>
          <para>Passwords are encoded when creating or updating users. The new default preferred
            encoder, replacing MD5, is SHA256. It is more secure than MD5 hashing. If you take no
            action to customize password encryption and authentication, SHA256 Salt will be
            used.</para>
          <para>If you prefer a different authentication mechanism, &PRODUCT; 4.2 provides a way for
            you to determine the default encoding and authentication mechanism for admin and user
            logins. Two new configurable lists have been introduced: userPasswordEncoders and
            userAuthenticators. userPasswordEncoders allow you to configure the order of preference
            for encoding passwords, and userAuthenticator allows you to configure the order in which
            authentication schemes are invoked to validate user passwords.</para>
          <para>The plain text user authenticator has been modified not to convert supplied
            passwords to their md5 sums before checking them with the database entries. It performs
            a simple string comparison between retrieved and supplied login passwords instead of
            comparing the retrieved md5 hash of the stored password against the supplied md5 hash of
            the password, because clients no longer hash the password.</para>
        </section>
        <section id="cloud-bugtool">
          <title>Log Collection Utility cloud-bugtool</title>
          <para>&PRODUCT; provides a command-line utility called cloud-bugtool to make it easier to
            collect the logs and other diagnostic data required for troubleshooting. This is
            especially useful when interacting with Citrix Technical Support.</para>
          <para>You can use cloud-bugtool to collect the following:</para>
          <itemizedlist>
            <listitem>
              <para>Basic system and environment information and network configuration including IP
                addresses, routing, and name resolver settings </para>
            </listitem>
            <listitem>
              <para>Information about running processes</para>
            </listitem>
            <listitem>
              <para>Management Server logs</para>
            </listitem>
            <listitem>
              <para>System logs in /var/log/</para>
            </listitem>
            <listitem>
              <para>Dump of the cloud database</para>
            </listitem>
          </itemizedlist>
          <warning>
            <para>cloud-bugtool collects information which might be considered sensitive and
              confidential. Using the <code>--nodb</code> option to avoid the cloud database can
              reduce this concern, though it is not guaranteed to exclude all sensitive data.</para>
          </warning>
          <para/>
        </section>
        <section id="rbd-primary-storage">
          <title>Snaphotting, Backups, Cloning and System VMs for RBD Primary Storage</title>
          <note>
            <para>These new RBD features require at least librbd 0.61.7 (Cuttlefish) and libvirt
              0.9.14 on the KVM hypervisors.</para>
          </note>
          <para>This release of &PRODUCT; will leverage the features of RBD format 2. This allows
            snapshotting and backing up those snapshots.</para>
          <para>Backups of snapshots to Secondary Storage are full copies of the RBD snapshot, they
            are not RBD diffs. This because when restoring a backup of a snapshot it is not
            mandatory that this backup is deployed on RBD again, it could also be a NFS Primary
            Storage.</para>
          <para>Another key feature of RBD format 2 is cloning. With this release templates will be
            copied to Primary Storage once and by using the cloning mechanism new disks will be
            cloned from this parent template. This saves space and decreases deployment time for
            instances dramatically.</para>
          <para>Before this release, a NFS Primary Storage was still required for running the System
            VMs from. The reason was a so called 'patch disk' that was generated by the hypervisor
            which contained metadata for the System VM. The scripts generating this disk didn't
            support RBD and thus System VMs had to be deployed from NFS. With 4.2 instead of the
            patch disk a VirtIO serial console is used to pass meta information to System VMs. This
            enabled the deployment of System VMs on RBD Primary Storage.</para>
        </section>
      </section>
    </section>
    <section id="issues-fixed-4.2">
      <title>Issues Fixed in 4.2.0</title>
      <para>Apache CloudStack uses <ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK"
          >Jira</ulink> to track its issues. All new features and bugs for 4.2.0 have been tracked
        in Jira, and have a standard naming convention of "CLOUDSTACK-NNNN" where "NNNN" is the
        issue number.</para>
      <para>This section includes a summary of known issues against 4.0.0 that were fixed in 4.2.0.
        Approximately 470 bugs were resolved or closed in the 4.2.0 cycle.</para>
      <informaltable>
        <tgroup cols="2" align="left" colsep="1" rowsep="1">
          <colspec colwidth="1*" colname="1" colnum="1"/>
          <colspec colwidth="2*" colname="2" colnum="2"/>
          <thead>
            <row>
              <entry>
                <para>Defect</para>
              </entry>
              <entry>
                <para>Description</para>
              </entry>
            </row>
          </thead>
          <tbody>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
            <row>
              <entry>
                <para/>
              </entry>
              <entry>
                <para/>
              </entry>
            </row>
          </tbody>
        </tgroup>
      </informaltable>
    </section>
    <section id="known-issues-4.2">
      <title>Known Issues in 4.2.0</title>
      <informaltable>
        <tgroup cols="2" align="left" colsep="1" rowsep="1">
          <colspec colwidth="1.0*" colname="1" colnum="1"/>
          <colspec colwidth="4.17*" colname="2" colnum="2"/>
          <thead>
            <row>
              <entry>
                <para>Issue ID</para>
              </entry>
              <entry>
                <para>Description</para>
              </entry>
            </row>
          </thead>
          <tbody>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-2709"
                    >CLOUDSTACK-3466</ulink></para></entry>
              <entry>
                <para>VM Migration across VMware clusters which are added with different switches
                  (Standard Switch,Vmware DVS, Cisco Nexus 1000v) is not supported.</para>
              </entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4207"
                    >CLOUDSTACK-4207</ulink></para></entry>
              <entry>
                <para>The following exception is observed when the Management Server is started
                  after the upgrade from any older versions to &PRODUCT; 4.2.</para>
                <para><code>jsonParseException: The JsonDeserializer
                    com.cloud.agent.transport.ArrayTypeAdaptor@2426e26f failed to deserialize json
                    object</code></para>
                <para>Ignore this exception, this would stop after you upgrade the System VM.
                  However, if you want to prevent this, stop system VM from the hypervisor before
                  upgrade.</para>
              </entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-2709"
                    >CLOUDSTACK-2709</ulink></para></entry>
              <entry>
                <para>Egress rules are not supported on Shared networks. </para>
              </entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-1747"
                    >CLOUDSTACK-1747</ulink></para></entry>
              <entry><para>The mvn deploydb command creates only 4.0 database, not 4.2
                  database.</para>
                <para>Due to tooling changes between 4.0 and 4.2, &PRODUCT; database is created by
                  using the 4.0 schema and updated to the 4.2 schema when the management server
                  starts for the first time. Neglect the same schema if the management server has
                  not started yet.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-1306"
                    >CLOUDSTACK-1306</ulink></para>
              </entry>
              <entry>
                <para>Enhance the error message that is displayed when trying to deploy a VM by
                  passing the static IPv4 addresses that are assigned to another VM or an IPv4
                  address that is outside the IP range.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-1236"
                    >CLOUDSTACK-1236</ulink></para>
              </entry>
              <entry>
                <para>Warning while adding a XenSever 6.1 host. The warning displayed is Unable to
                  create local link network.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-969"
                    >CLOUDSTACK-969</ulink></para>
              </entry>
              <entry>
                <para>api: zone response lists vlan in it as &quot;vlan range of zone&quot; but the
                  vlan belongs to physical network</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-963"
                    >CLOUDSTACK-963</ulink></para>
              </entry>
              <entry>
                <para>[cloud.utils.AnnotationHelper] class java.lang.Stringdoes not have a Table
                  annotation</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-458"
                    >CLOUDSTACK-458</ulink></para>
              </entry>
              <entry>
                <para>xen:snapshots:Storage gc fail to clean the failed snapshot images from
                  secondarystorage</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-315"
                    >CLOUDSTACK-315</ulink></para>
              </entry>
              <entry>
                <para>Infrastructure view does not show capacity values</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-300"
                    >CLOUDSTACK-300</ulink></para>
              </entry>
              <entry>
                <para>Creation of compute offering allow combination of local storage + HA</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-276"
                    >CLOUDSTACK-276</ulink></para>
              </entry>
              <entry>
                <para>SSVM ID is exposed in the Error Message thrown by AddTrafficType API</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-270"
                    >CLOUDSTACK-270</ulink></para>
              </entry>
              <entry>
                <para>Ui should not ask for a vlan range if the physical network isolation type is
                  not VLAN</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-245"
                    >CLOUDSTACK-245</ulink></para>
              </entry>
              <entry>
                <para>VPC ACLs are not stored and programmed consistently</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-231"
                    >CLOUDSTACK-231</ulink></para>
              </entry>
              <entry>
                <para>Tag creation using special charecters</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-124"
                    >CLOUDSTACK-124</ulink></para>
              </entry>
              <entry>
                <para>NetworkGarbageCollector not cleaning up networks</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-62"
                    >CLOUDSTACK-62</ulink></para>
              </entry>
              <entry>
                <para>console proxy does not support any keymaps besides us, jp</para>
              </entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4645"
                    >CLOUDSTACK-4645</ulink></para></entry>
              <entry><para>There is no upgrade path from 4.1.1 to 4.2.0.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4641"
                    >CLOUDSTACK-4641</ulink></para></entry>
              <entry><para>Create volume form snapshot command times out exactly after 1 hour in
                  case of KVM hosts.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4621"
                    >CLOUDSTACK-4621</ulink></para></entry>
              <entry><para>Changing the management server's Ethernet interface or MAC address leaves
                  the system in unstable state.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4615"
                    >CLOUDSTACK-4615</ulink></para></entry>
              <entry><para>(Baremetal) Baremetal agent is missing in the installer.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4598"
                    >CLOUDSTACK-4598</ulink></para></entry>
              <entry><para>Long delays during deploying a VM; both network and deployment planner
                  are delayed.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4596"
                    >CLOUDSTACK-4596</ulink></para></entry>
              <entry><para>The same IP range is allowed to be defined in different VLANs across
                  public and portable ranges.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4588"
                    >CLOUDSTACK-4588</ulink></para></entry>
              <entry><para>(VMware) VM deployment failed while creating a volume with null pointer
                  exception.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4578"
                    >CLOUDSTACK-4578</ulink></para></entry>
              <entry><para>(VMware) SSVM is not getting created if one host is down in a
                  cluster.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4551"
                    >CLOUDSTACK-4551</ulink></para></entry>
              <entry><para>Migrating the data volume from NFS to local storage does not change the
                  underlying disk offering.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4550"
                    >CLOUDSTACK-4550</ulink></para></entry>
              <entry><para>Migration does not work in the case of bridge naming while upgrading KVM
                  agents to version 4.2.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4549"
                    >CLOUDSTACK-4549</ulink></para></entry>
              <entry><para>Deploying VMs from template fails if the template is created from a
                  snapshot.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4540"
                    >CLOUDSTACK-4540</ulink></para></entry>
              <entry><para>(VMware) When deploying 30 parallel VMs , 16 VMs fails to get deployed
                  due to the following error: "VmDataCommand failed due to Exception:
                  java.lang.Exception Message: Timed out in waiting SSH execution
                result".</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4506"
                    >CLOUDSTACK-4506</ulink></para></entry>
              <entry><para>In a mixed hypervisor setup, destroying a VM whose host has been removed,
                  throws a null pointer exception. The Root volume of that VM also is not deleted
                  from the primary memory.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4442"
                    >CLOUDSTACK-4442</ulink></para></entry>
              <entry><para>Source NAT not applied when network starts up.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4405"
                    >CLOUDSTACK-4405</ulink></para></entry>
              <entry><para>(KVM) Migration between existing hosts and new hosts
                fails.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4402"
                    >CLOUDSTACK-4402</ulink></para></entry>
              <entry><para>No options to delete the primary storage if the last host with which it
                  was associated is already removed.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4366"
                    >CLOUDSTACK-4366</ulink></para></entry>
              <entry><para>(Ubuntu) Key translation fails for the Japanese keyboard for the Menu key
                  and Caps Lock buttons.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4351"
                    >CLOUDSTACK-4351</ulink></para></entry>
              <entry><para>Host/Hypervisor System Requirements has misleading or premature note in
                  the documentation.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4348"
                    >CLOUDSTACK-4348</ulink></para></entry>
              <entry><para>Regression truncation issues occurs when moving the cursor to the "plus"
                  buttons.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4300"
                    >CLOUDSTACK-4300</ulink></para></entry>
              <entry><para>(KVM) System VMs are not coming up after 2.2.14 to 4.2
                upgrade.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4292"
                    >CLOUDSTACK-4292</ulink></para></entry>
              <entry><para>The destroyedvm API failed with ArrayIndexexception while
                  expunging.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4247"
                    >CLOUDSTACK-4247</ulink></para></entry>
              <entry><para>(VMWARE) Network read and write statistics always returns
                zero.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4224"
                    >CLOUDSTACK-4224</ulink></para></entry>
              <entry><para>Deleting UCS returns unknown API.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4220"
                    >CLOUDSTACK-4220</ulink></para></entry>
              <entry><para>From 3.0.6 to 4.2 upgrade, Add VMWare DataCenter button is provided for
                  legacy zones.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4201"
                    >CLOUDSTACK-4201</ulink></para></entry>
              <entry><para>The listServiceOfferings API does not take virtualmachineid parameter of
                  SystemVM to return the Service Offerings available for the VM to change a Service
                  Offering.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4200"
                    >CLOUDSTACK-4200</ulink></para></entry>
              <entry><para>The listSystemVMs API and listRouters API fail to return hypervisor
                  property.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4148"
                    >CLOUDSTACK-4148</ulink></para></entry>
              <entry><para>The usage statistics are not triggered for Shared network.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4139"
                    >CLOUDSTACK-4139</ulink></para></entry>
              <entry><para>(VMWARE) Resizing the volumes fails if they are created from
                  snapshot.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4137"
                    >CLOUDSTACK-4137</ulink></para></entry>
              <entry><para>(KVM): After removing a cluster, manage cluster will not bring KVM hosts
                  to UP state. Manually restart the cloud-agent on KVM hosts.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4128"
                    >CLOUDSTACK-4128</ulink></para></entry>
              <entry><para>The System VMs start up does not check for existence of staging secondary
                  storage in a zone.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4099"
                    >CLOUDSTACK-4099</ulink></para></entry>
              <entry><para>Update the systemvm templates in DevCloud2.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4095"
                    >CLOUDSTACK-4095</ulink></para></entry>
              <entry><para>Region ID is displayed within the Database Transaction
                code.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4072"
                    >CLOUDSTACK-4072</ulink></para></entry>
              <entry><para>The mysql-connector-java rpm is required while upgrading from 2.2.14 to
                  4.2.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4036"
                    >CLOUDSTACK-4036</ulink></para></entry>
              <entry><para>The UI remains in processing state and the queryAsyncJobResult is being
                  called repeatedly for the scaleSystemVm API.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-4016"
                    >CLOUDSTACK-4016</ulink></para></entry>
              <entry><para>The listPublicIpAddresses lists the portable IP that was already
                  transferred to a different Isolated network.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3968"
                    >CLOUDSTACK-3968</ulink></para></entry>
              <entry><para>Distributed Port groups are not deleted when guest networks are removed.
                  The user account of this network is removed from &PRODUCT;</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3967"
                    >CLOUDSTACK-3967</ulink></para></entry>
              <entry><para>No support for usage statistics collection at the portable IP
                  level</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3953"
                    >CLOUDSTACK-3953</ulink></para></entry>
              <entry><para>The usage statistics are not collected for GSLB rules.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3911"
                    >CLOUDSTACK-3911</ulink></para></entry>
              <entry><para>No check available while adding public range in a zone to see whether the
                  same VLAN exists in a portable IP range.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3888"
                    >CLOUDSTACK-3888</ulink></para></entry>
              <entry><para>The UI does not return the mode (Strict/Preferred) when listing the
                  ServiceOffering that uses ImplicitDedicationPlanner.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3808"
                    >CLOUDSTACK-3808</ulink></para></entry>
              <entry><para>Attaching volumes does not work when root is at the zone-level primary
                  store and data at the cluster level or host level store.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3791"
                    >CLOUDSTACK-3791</ulink></para></entry>
              <entry><para>Download template fails with a null pointer exception.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3788"
                    >CLOUDSTACK-3788</ulink></para></entry>
              <entry><para>The weekly Snapshot is stuck in Allocated State.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3765"
                    >CLOUDSTACK-3765</ulink></para></entry>
              <entry><para>Upgrading CloudPlatform 4.2 build on centos5.5 does not
                work.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3737"
                    >CLOUDSTACK-3737</ulink></para></entry>
              <entry><para>Uploaded volume is not getting deleted from secondary storage after
                  attaching it to a guest VM.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3658"
                    >CLOUDSTACK-3658</ulink></para></entry>
              <entry><para>Several old object storage tables and columns are deprecated as a part of
                  4.1 to 4.2 database upgrade.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3627"
                    >CLOUDSTACK-3627</ulink></para></entry>
              <entry><para>Public IP interface (eth2) is not getting configured with Redundant
                  virtual router. The State is FAULT.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3608"
                    >CLOUDSTACK-3608</ulink></para></entry>
              <entry><para>The guest_os_hypervisor table in the database has repeated mappings of
                  hypervisor and guest OS.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3583"
                    >CLOUDSTACK-3583</ulink></para></entry>
              <entry><para>Stopping the Management server does not remove the PID.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3565"
                    >CLOUDSTACK-3565</ulink></para></entry>
              <entry><para>Restarting libvirtd service leads to destroying the storage
                pool.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3243"
                    >CLOUDSTACK-3243</ulink></para></entry>
              <entry><para>Wrong NFS mount point is given in the documentation.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-3138"
                    >CLOUDSTACK-3138</ulink></para></entry>
              <entry><para>Flaws in the documentation for the upgrade from 3.0.2 to
                4.1.0.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-2791"
                    >CLOUDSTACK-2791</ulink></para></entry>
              <entry><para>Installation instruction is wrong.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-1986"
                    >CLOUDSTACK-1986</ulink></para></entry>
              <entry><para>Key translation fails for the following Japanese keyboard keys: ¥_,\ |,
                  Muhenkan, Henkan, Hiragana/Katakana, Kanji Key, and Caps Lock.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-1775"
                    >CLOUDSTACK-1775</ulink></para></entry>
              <entry><para>Events related to User/Domain/Account are not being generated expect for
                  the USER-DISABLE,DOMAIN-DELETE and ACCOUNT.DISABLE events.</para></entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-732"
                    >CLOUDSTACK-732</ulink></para></entry>
              <entry><para>KVM snapshot is not supported.</para></entry>
            </row>
          </tbody>
        </tgroup>
      </informaltable>
    </section>
  </chapter>
  <chapter id="upgrade-instructions">
    <title>Upgrade Instructions for 4.2</title>
    <para>This section contains upgrade instructions from prior versions of CloudStack to Apache
      CloudStack 4.2.0. We include instructions on upgrading to Apache CloudStack from pre-Apache
      versions of Citrix CloudStack (last version prior to Apache is 3.0.2) and from the releases
      made while CloudStack was in the Apache Incubator.</para>
    <para>If you run into any issues during upgrades, please feel free to ask questions on
      users@cloudstack.apache.org or dev@cloudstack.apache.org.</para>
    <section id="upgrade-from-4.0-to-4.1">
      <title>Upgrade from 4.x.x to 4.2.0</title>
      <para>This section will guide you from &PRODUCT; 4.0.x versions to &PRODUCT; 4.2.0.</para>
      <para>Any steps that are hypervisor-specific will be called out with a note.</para>
      <warning>
        <title>Package Structure Changes</title>
        <para>The package structure for &PRODUCT; has changed significantly since the 4.0.x
          releases. If you've compiled your own packages, you'll notice that the package names and
          the number of packages has changed. This is <emphasis>not</emphasis> a bug.</para>
        <para>However, this does mean that the procedure is not as simple as an <command>apt-get
            upgrade</command> or <command>yum update</command>, so please follow this section
          carefully.</para>
      </warning>
      <para>We recommend reading through this section once or twice before beginning your upgrade
        procedure, and working through it on a test system before working on a production
        system.</para>
      <orderedlist>
        <listitem>
          <para>Most users of &PRODUCT; manage the installation and upgrades of &PRODUCT; with one
            of Linux's predominant package systems, RPM or APT. This guide assumes you'll be using
            RPM and Yum (for Red Hat Enterprise Linux or CentOS), or APT and Debian packages (for
            Ubuntu).</para>
        </listitem>
        <listitem>
          <para>Create RPM or Debian packages (as appropriate) and a repository from the 4.2.0
            source, or check the Apache CloudStack downloads page at <ulink
              url="http://cloudstack.apache.org/downloads.html"
              >http://cloudstack.apache.org/downloads.html</ulink> for package repositories supplied
            by community members. You will need them for step <xref linkend="upgrade-deb-packages"/>
            or step <xref linkend="upgrade-rpm-packages"/>.</para>
          <para>Instructions for creating packages from the &PRODUCT; source are in the <ulink
              url="http://cloudstack.apache.org/docs/en-US/index.html">Installation
            Guide</ulink>.</para>
        </listitem>
        <listitem>
          <para>Stop your management server or servers. Run this on all management server
            hosts:</para>
          <programlisting><prompt>#</prompt> service cloud-management stop</programlisting>
        </listitem>
        <listitem>
          <para>If you are running a usage server or usage servers, stop those as well:</para>
          <programlisting><prompt>#</prompt> service cloud-usage stop</programlisting>
        </listitem>
        <listitem>
          <para>Make a backup of your MySQL database. If you run into any issues or need to roll
            back the upgrade, this will assist in debugging or restoring your existing environment.
            You'll be prompted for your password.</para>
          <programlisting><prompt>#</prompt> mysqldump -u root -p cloud &gt; cloudstack-backup.sql</programlisting>
        </listitem>
        <listitem>
          <para>If you have made changes to
              <filename>/etc/cloud/management/components.xml</filename>, you'll need to carry these
            over manually to the new file,
              <filename>/etc/cloudstack/management/componentContext.xml</filename>. This is not done
            automatically. (If you're unsure, we recommend making a backup of the original
              <filename>components.xml</filename> to be on the safe side.</para>
        </listitem>
        <listitem>
          <para>After upgrading to 4.2, API clients are expected to send plain text passwords for
            login and user creation, instead of MD5 hash. If API client changes are not acceptable,
            following changes are to be made for backward compatibility:</para>
          <para>Modify componentsContext.xml, and make PlainTextUserAuthenticator as the default
            authenticator (1st entry in the userAuthenticators adapter list is default)</para>
          <programlisting language="XML">
&lt;!-- Security adapters --&gt;
&lt;bean id="userAuthenticators" class="com.cloud.utils.component.AdapterList"&gt;
  &lt;property name="Adapters"&gt;
    &lt;list&gt;
      &lt;ref bean="PlainTextUserAuthenticator"/&gt;
      &lt;ref bean="MD5UserAuthenticator"/&gt;
      &lt;ref bean="LDAPUserAuthenticator"/&gt;
    &lt;/list&gt;
  &lt;/property&gt;
&lt;/bean&gt;
            </programlisting>
          <para>PlainTextUserAuthenticator works the same way MD5UserAuthenticator worked prior to
            4.2.</para>
        </listitem>
        <listitem id="upgrade-deb-packages">
          <para>If you are using Ubuntu, follow this procedure to upgrade your packages. If not,
            skip to step <xref linkend="upgrade-rpm-packages"/>.</para>
          <note>
            <title>Community Packages</title>
            <para>This section assumes you're using the community supplied packages for &PRODUCT;.
              If you've created your own packages and APT repository, substitute your own URL for
              the ones used in these examples.</para>
          </note>
          <orderedlist id="debsteps">
            <listitem>
              <para>The first order of business will be to change the sources list for each system
                with &PRODUCT; packages. This means all management servers, and any hosts that have
                the KVM agent. (No changes should be necessary for hosts that are running VMware or
                Xen.)</para>
              <para>Start by opening <filename>/etc/apt/sources.list.d/cloudstack.list</filename> on
                any systems that have &PRODUCT; packages installed.</para>
              <para>This file should have one line, which contains:</para>
              <programlisting language="Bash">deb http://cloudstack.apt-get.eu/ubuntu precise 4.0</programlisting>
              <para>We'll change it to point to the new package repository:</para>
              <programlisting language="Bash">deb http://cloudstack.apt-get.eu/ubuntu precise 4.2</programlisting>
              <para>If you're using your own package repository, change this line to read as
                appropriate for your 4.2.0 repository.</para>
            </listitem>
            <listitem>
              <para>Now update your apt package list:</para>
              <programlisting language="Bash">$ sudo apt-get update</programlisting>
            </listitem>
            <listitem id="deb-master">
              <para>Now that you have the repository configured, it's time to install the
                  <filename>cloudstack-management</filename> package. This will pull in any other
                dependencies you need.</para>
              <programlisting language="Bash">$ sudo apt-get install cloudstack-management</programlisting>
            </listitem>
            <listitem id="kvm-agent-deb">
              <para>You will need to manually install the <filename>cloudstack-agent</filename>
                package:</para>
              <programlisting language="Bash">$ sudo apt-get install cloudstack-agent</programlisting>
              <para>During the installation of <filename>cloudstack-agent</filename>, APT will copy
                your <filename>agent.properties</filename>, <filename>log4j-cloud.xml</filename>,
                and <filename>environment.properties</filename> from
                  <filename>/etc/cloud/agent</filename> to
                  <filename>/etc/cloudstack/agent</filename>.</para>
              <para>When prompted whether you wish to keep your configuration, say Yes.</para>
            </listitem>
            <listitem>
              <para>Verify that the file
                  <filename>/etc/cloudstack/agent/environment.properties</filename> has a line that
                reads:</para>
              <programlisting language="Bash">paths.script=/usr/share/cloudstack-common</programlisting>
              <para>If not, add the line.</para>
            </listitem>
            <listitem>
              <para>Restart the agent:</para>
              <programlisting language="Bash">
service cloud-agent stop
killall jsvc
service cloudstack-agent start
                            </programlisting>
            </listitem>
            <listitem>
              <para>During the upgrade, <filename>log4j-cloud.xml</filename> was simply copied over,
                so the logs will continue to be added to
                  <filename>/var/log/cloud/agent/agent.log</filename>. There's nothing
                  <emphasis>wrong</emphasis> with this, but if you prefer to be consistent, you can
                change this by copying over the sample configuration file:</para>
              <programlisting language="Bash">
cd /etc/cloudstack/agent
mv log4j-cloud.xml.dpkg-dist log4j-cloud.xml
service cloudstack-agent restart
                            </programlisting>
            </listitem>
            <listitem>
              <para>Once the agent is running, you can uninstall the old cloud-* packages from your
                system:</para>
              <programlisting language="Bash">sudo dpkg --purge cloud-agent</programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>(VMware only) Additional steps are required for each VMware cluster. These steps
            will not affect running guests in the cloud. These steps are required only for clouds
            using VMware clusters:</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Stop the Management Server:</para>
              <programlisting>service cloudstack-management stop</programlisting>
            </listitem>
            <listitem>
              <para>Generate the encrypted equivalent of your vCenter password:</para>
              <programlisting>java -classpath /usr/share/cloudstack-common/lib/jasypt-1.9.0.jar org.jasypt.intf.cli.JasyptPBEStringEncryptionCLI encrypt.sh input="_your_vCenter_password_" password="`cat /etc/cloudstack/management/key`" verbose=false</programlisting>
              <para>Store the output from this step, we need to add this in cluster_details table
                and vmware_data_center tables in place of the plain text password</para>
            </listitem>
            <listitem>
              <para>Find the ID of the row of cluster_details table that you have to update:</para>
              <programlisting>mysql -u &lt;username&gt; -p&lt;password&gt;</programlisting>
              <programlisting>select * from cloud.cluster_details;</programlisting>
            </listitem>
            <listitem>
              <para>Update the plain text password with the encrypted one</para>
              <programlisting>update cloud.cluster_details set value = '_ciphertext_from_step_1_' where id = _id_from_step_2_;</programlisting>
            </listitem>
            <listitem>
              <para>Confirm that the table is updated:</para>
              <programlisting>select * from cloud.cluster_details; </programlisting>
            </listitem>
            <listitem>
              <para>Find the ID of the correct row of vmware_data_center that you want to
                update</para>
              <programlisting>select * from cloud.vmware_data_center; </programlisting>
            </listitem>
            <listitem>
              <para>update the plain text password with the encrypted one:</para>
              <programlisting>update cloud.vmware_data_center set password = '_ciphertext_from_step_1_' where id = _id_from_step_5_; </programlisting>
            </listitem>
            <listitem>
              <para>Confirm that the table is updated:</para>
              <programlisting>select * from cloud.vmware_data_center; </programlisting>
            </listitem>
            <listitem>
              <para>Start the &PRODUCT; Management server </para>
              <programlisting>service cloudstack-management start</programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>(KVM only) Additional steps are required for each KVM host. These steps will not
            affect running guests in the cloud. These steps are required only for clouds using KVM
            as hosts and only on the KVM hosts.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Manually clean up <filename>/var/cache/cloudstack</filename>.</para>
            </listitem>
            <listitem>
              <para>Copy the 4.2 tar file to the host, untar it, and change directory to the
                resulting directory.</para>
            </listitem>
            <listitem>
              <para>Stop the running agent.</para>
              <programlisting># service cloud-agent stop</programlisting>
            </listitem>
            <listitem>
              <para>Update the agent software.</para>
              <programlisting># ./install.sh</programlisting>
            </listitem>
            <listitem>
              <para>Choose "U" to update the packages.</para>
            </listitem>
            <listitem>
              <para>Start the agent.</para>
              <programlisting># service cloudstack-agent start</programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="upgrade-rpm-packages">
          <para>If you are using CentOS or RHEL, follow this procedure to upgrade your packages. If
            not, skip to step <xref linkend="restart-system-vms"/>.</para>
          <note>
            <title>Community Packages</title>
            <para>This section assumes you're using the community supplied packages for &PRODUCT;.
              If you've created your own packages and yum repository, substitute your own URL for
              the ones used in these examples.</para>
          </note>
          <orderedlist id="rpmsteps">
            <listitem>
              <para>The first order of business will be to change the yum repository for each system
                with &PRODUCT; packages. This means all management servers, and any hosts that have
                the KVM agent. </para>
              <para>(No changes should be necessary for hosts that are running VMware or
                Xen.)</para>
              <para>Start by opening <filename>/etc/yum.repos.d/cloudstack.repo</filename> on any
                systems that have &PRODUCT; packages installed.</para>
              <para>This file should have content similar to the following:</para>
              <programlisting language="Bash">
[apache-cloudstack]
name=Apache CloudStack
baseurl=http://cloudstack.apt-get.eu/rhel/4.0/
enabled=1
gpgcheck=0
                            </programlisting>
              <para>If you are using the community provided package repository, change the base url
                to http://cloudstack.apt-get.eu/rhel/4.2/</para>
              <para>If you're using your own package repository, change this line to read as
                appropriate for your 4.2.0 repository.</para>
            </listitem>
            <listitem id="rpm-master">
              <para>Now that you have the repository configured, it's time to install the
                  <filename>cloudstack-management</filename> package by upgrading the older
                  <filename>cloud-client</filename> package.</para>
              <programlisting language="Bash">$ sudo yum upgrade cloud-client</programlisting>
            </listitem>
            <listitem id="kvm-agent-rpm">
              <para>For KVM hosts, you will need to upgrade the <filename>cloud-agent</filename>
                package, similarly installing the new version as
                  <filename>cloudstack-agent</filename>.</para>
              <programlisting language="Bash">$ sudo yum upgrade cloud-agent</programlisting>
              <para>During the installation of <filename>cloudstack-agent</filename>, the RPM will
                copy your <filename>agent.properties</filename>,
                  <filename>log4j-cloud.xml</filename>, and
                  <filename>environment.properties</filename> from
                  <filename>/etc/cloud/agent</filename> to
                  <filename>/etc/cloudstack/agent</filename>.</para>
            </listitem>
            <listitem>
              <para>For CentOS 5.5, perform the following:</para>
              <orderedlist>
                <listitem>
                  <para>Run the following command:</para>
                  <programlisting>rpm -Uvh http://download.cloud.com/support/jsvc/jakarta-commons-daemon-jsvc-1.0.1-8.9.el6.x86_64.rpm</programlisting>
                </listitem>
                <listitem>
                  <para>Upgrade the Usage server.</para>
                  <programlisting>sudo yum upgrade cloud-usage</programlisting>
                </listitem>
              </orderedlist>
            </listitem>
            <listitem>
              <para>Verify that the file
                  <filename>/etc/cloudstack/agent/environment.properties</filename> has a line that
                reads:</para>
              <programlisting language="Bash">paths.script=/usr/share/cloudstack-common</programlisting>
              <para>If not, add the line.</para>
            </listitem>
            <listitem>
              <para>Restart the agent:</para>
              <programlisting language="Bash">
service cloud-agent stop
killall jsvc
service cloudstack-agent start
                            </programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="restart-system-vms">
          <para>Once you've upgraded the packages on your management servers, you'll need to restart
            the system VMs. Make sure port 8096 is open in your local host firewall to do
            this.</para>
          <para>There is a script that will do this for you, all you need to do is run the script
            and supply the IP address for your MySQL instance and your MySQL credentials:</para>
          <programlisting language="Bash"><prompt>#</prompt> nohup cloudstack-sysvmadm -d <replaceable>IP address</replaceable> -u cloud -p -a &gt; sysvm.log 2&gt;&amp;1 &amp;</programlisting>
          <para>You can monitor the log for progress. The process of restarting the system VMs can
            take an hour or more.</para>
          <programlisting language="Bash"><prompt>#</prompt> tail -f sysvm.log</programlisting>
          <para>The output to <filename>sysvm.log</filename> will look something like this:</para>
          <programlisting language="Bash">
Stopping and starting 1 secondary storage vm(s)...
Done stopping and starting secondary storage vm(s)
Stopping and starting 1 console proxy vm(s)...
Done stopping and starting console proxy vm(s).
Stopping and starting 4 running routing vm(s)...
Done restarting router(s).
                    </programlisting>
        </listitem>
        <listitem>
          <note>
            <title>For Xen Hosts: Copy vhd-utils</title>
            <para>This step is only for CloudStack installs that are using Xen hosts.</para>
          </note>
          <para>Copy the file <filename>vhd-utils</filename> to
              <filename>/usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver</filename>.</para>
        </listitem>
      </orderedlist>
    </section>
    <section id="upgrade-from-3.0.2-to-4.0">
      <title>Upgrade from 3.0.2 to 4.2.0</title>
      <para>This section will guide you from Citrix CloudStack 3.0.2 to Apache CloudStack 4.2.0.
        Sections that are hypervisor-specific will be called out with a note.</para>
      <orderedlist>
        <listitem>
          <note>
            <para>The following upgrade instructions apply only if you're using VMware hosts. If
              you're not using VMware hosts, skip this step and move on to <xref
                linkend="stopping-usage-servers"/>.</para>
          </note>
          <para>In each zone that includes VMware hosts, you need to add a new system VM template. </para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>While running the existing 3.0.2 system, log in to the UI as root
                administrator.</para>
            </listitem>
            <listitem>
              <para>In the left navigation bar, click Templates.</para>
            </listitem>
            <listitem>
              <para>In Select view, click Templates.</para>
            </listitem>
            <listitem>
              <para>Click Register template.</para>
              <para>The Register template dialog box is displayed.</para>
            </listitem>
            <listitem>
              <para>In the Register template dialog box, specify the following values (do not change
                these):</para>
              <informaltable>
                <tgroup cols="2" align="left" colsep="1" rowsep="1">
                  <colspec colwidth="1*" colname="1" colnum="1"/>
                  <colspec colwidth="2*" colname="2" colnum="2"/>
                  <thead>
                    <row>
                      <entry><para>Field</para></entry>
                      <entry><para>Value</para></entry>
                    </row>
                  </thead>
                  <tbody>
                    <row>
                      <entry><para>Name</para></entry>
                      <entry><para>systemvm-vmware-4.2</para></entry>
                    </row>
                    <row>
                      <entry><para>Description</para></entry>
                      <entry><para>systemvm-vmware-4.2</para></entry>
                    </row>
                    <row>
                      <entry><para>URL</para></entry>
                      <entry><para>http://download.cloud.com/templates/burbank/burbank-systemvm-08012012.ova</para></entry>
                    </row>
                    <row>
                      <entry><para>Zone</para></entry>
                      <entry><para>Choose the zone where this hypervisor is used</para></entry>
                    </row>
                    <row>
                      <entry><para>Hypervisor</para></entry>
                      <entry><para>VMware</para></entry>
                    </row>
                    <row>
                      <entry><para>Format</para></entry>
                      <entry><para>OVA</para></entry>
                    </row>
                    <row>
                      <entry><para>OS Type</para></entry>
                      <entry><para>Debian GNU/Linux 5.0 (32-bit)</para></entry>
                    </row>
                    <row>
                      <entry><para>Extractable</para></entry>
                      <entry><para>no</para></entry>
                    </row>
                    <row>
                      <entry><para>Password Enabled</para></entry>
                      <entry><para>no</para></entry>
                    </row>
                    <row>
                      <entry><para>Public</para></entry>
                      <entry><para>no</para></entry>
                    </row>
                    <row>
                      <entry><para>Featured</para></entry>
                      <entry><para>no</para></entry>
                    </row>
                  </tbody>
                </tgroup>
              </informaltable>
            </listitem>
            <listitem>
              <para>Watch the screen to be sure that the template downloads successfully and enters
                the READY state. Do not proceed until this is successful.</para>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="stopping-usage-servers">
          <para>Stop all Usage Servers if running. Run this on all Usage Server hosts.</para>
          <programlisting language="Bash"><prompt>#</prompt> service cloud-usage stop</programlisting>
        </listitem>
        <listitem>
          <para>Stop the Management Servers. Run this on all Management Server hosts.</para>
          <programlisting language="Bash"><prompt>#</prompt> service cloud-management stop</programlisting>
        </listitem>
        <listitem>
          <para>On the MySQL master, take a backup of the MySQL databases. We recommend performing
            this step even in test upgrades. If there is an issue, this will assist with
            debugging.</para>
          <para>In the following commands, it is assumed that you have set the root password on the
            database, which is a CloudStack recommended best practice. Substitute your own MySQL
            root password.</para>
          <programlisting><prompt>#</prompt> <command>mysqldump</command> -u root -p<replaceable>mysql_password</replaceable> cloud > <filename>cloud-backup.dmp</filename>
                        <prompt>#</prompt> <command>mysqldump</command> -u root -p<replaceable>mysql_password</replaceable> cloud_usage > <filename>cloud-usage-backup.dmp</filename></programlisting>
        </listitem>
        <listitem>
          <para>Either build RPM/DEB packages as detailed in the Installation Guide, or use one of
            the community provided yum/apt repositories to gain access to the &PRODUCT;
            binaries.</para>
        </listitem>
        <listitem id="upgrade-deb-packages-302">
          <para>If you are using Ubuntu, follow this procedure to upgrade your packages. If not,
            skip to step <xref linkend="upgrade-rpm-packages-302"/>.</para>
          <note>
            <title>Community Packages</title>
            <para>This section assumes you're using the community supplied packages for &PRODUCT;.
              If you've created your own packages and APT repository, substitute your own URL for
              the ones used in these examples.</para>
          </note>
          <orderedlist numeration="loweralpha" id="debsteps-302">
            <listitem>
              <para>The first order of business will be to change the sources list for each system
                with &PRODUCT; packages. This means all management servers, and any hosts that have
                the KVM agent. (No changes should be necessary for hosts that are running VMware or
                Xen.)</para>
              <para>Start by opening <filename>/etc/apt/sources.list.d/cloudstack.list</filename> on
                any systems that have &PRODUCT; packages installed.</para>
              <para>This file should have one line, which contains:</para>
              <programlisting language="Bash">deb http://cloudstack.apt-get.eu/ubuntu precise 4.0</programlisting>
              <para>We'll change it to point to the new package repository:</para>
              <programlisting language="Bash">deb http://cloudstack.apt-get.eu/ubuntu precise 4.2</programlisting>
              <para>If you're using your own package repository, change this line to read as
                appropriate for your 4.2.0 repository.</para>
            </listitem>
            <listitem>
              <para>Now update your apt package list:</para>
              <programlisting language="Bash">$ sudo apt-get update</programlisting>
            </listitem>
            <listitem id="deb-master-302">
              <para>Now that you have the repository configured, it's time to install the
                  <filename>cloudstack-management</filename> package. This will pull in any other
                dependencies you need.</para>
              <programlisting language="Bash">$ sudo apt-get install cloudstack-management</programlisting>
            </listitem>
            <listitem id="kvm-agent-deb-302">
              <para>You will need to manually install the <filename>cloudstack-agent</filename>
                package:</para>
              <programlisting language="Bash">$ sudo apt-get install cloudstack-agent</programlisting>
              <para>During the installation of <filename>cloudstack-agent</filename>, APT will copy
                your <filename>agent.properties</filename>, <filename>log4j-cloud.xml</filename>,
                and <filename>environment.properties</filename> from
                  <filename>/etc/cloud/agent</filename> to
                  <filename>/etc/cloudstack/agent</filename>.</para>
              <para>When prompted whether you wish to keep your configuration, say Yes.</para>
            </listitem>
            <listitem>
              <para>Verify that the file
                  <filename>/etc/cloudstack/agent/environment.properties</filename> has a line that
                reads:</para>
              <programlisting language="Bash">paths.script=/usr/share/cloudstack-common</programlisting>
              <para>If not, add the line.</para>
            </listitem>
            <listitem>
              <para>Restart the agent:</para>
              <programlisting language="Bash">
service cloud-agent stop
killall jsvc
service cloudstack-agent start
                            </programlisting>
            </listitem>
            <listitem>
              <para>During the upgrade, <filename>log4j-cloud.xml</filename> was simply copied over,
                so the logs will continue to be added to
                  <filename>/var/log/cloud/agent/agent.log</filename>. There's nothing
                  <emphasis>wrong</emphasis> with this, but if you prefer to be consistent, you can
                change this by copying over the sample configuration file:</para>
              <programlisting language="Bash">
cd /etc/cloudstack/agent
mv log4j-cloud.xml.dpkg-dist log4j-cloud.xml
service cloudstack-agent restart
                            </programlisting>
            </listitem>
            <listitem>
              <para>Once the agent is running, you can uninstall the old cloud-* packages from your
                system:</para>
              <programlisting language="Bash">sudo dpkg --purge cloud-agent</programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>(KVM only) Additional steps are required for each KVM host. These steps will not
            affect running guests in the cloud. These steps are required only for clouds using KVM
            as hosts and only on the KVM hosts.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Copy the CloudPlatform 4.2 tar file to the host, untar it, and change directory
                to the resulting directory.</para>
            </listitem>
            <listitem>
              <para>Stop the running agent.</para>
              <programlisting># service cloud-agent stop</programlisting>
            </listitem>
            <listitem>
              <para>Update the agent software.</para>
              <programlisting># ./install.sh</programlisting>
            </listitem>
            <listitem>
              <para>Choose "U" to update the packages.</para>
            </listitem>
            <listitem>
              <para>Start the agent.</para>
              <programlisting># service cloudstack-agent start</programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="upgrade-rpm-packages-302">
          <para>If you are using CentOS or RHEL, follow this procedure to upgrade your packages. If
            not, skip to step <xref linkend="correct-components-xml-302"/>.</para>
          <note>
            <title>Community Packages</title>
            <para>This section assumes you're using the community supplied packages for &PRODUCT;.
              If you've created your own packages and yum repository, substitute your own URL for
              the ones used in these examples.</para>
          </note>
          <orderedlist id="rpmsteps-302">
            <listitem>
              <para>The first order of business will be to change the yum repository for each system
                with &PRODUCT; packages. This means all management servers, and any hosts that have
                the KVM agent. (No changes should be necessary for hosts that are running VMware or
                Xen.)</para>
              <para>Start by opening <filename>/etc/yum.repos.d/cloudstack.repo</filename> on any
                systems that have &PRODUCT; packages installed.</para>
              <para>This file should have content similar to the following:</para>
              <programlisting language="Bash">
[apache-cloudstack]
name=Apache CloudStack
baseurl=http://cloudstack.apt-get.eu/rhel/4.0/
enabled=1
gpgcheck=0
                            </programlisting>
              <para>If you are using the community provided package repository, change the baseurl
                to http://cloudstack.apt-get.eu/rhel/4.2/</para>
              <para>If you're using your own package repository, change this line to read as
                appropriate for your 4.2.0 repository.</para>
            </listitem>
            <listitem id="rpm-master-302">
              <para>Now that you have the repository configured, it's time to install the
                  <filename>cloudstack-management</filename> package by upgrading the older
                  <filename>cloud-client</filename> package.</para>
              <programlisting language="Bash">$ sudo yum upgrade cloud-client</programlisting>
            </listitem>
            <listitem id="kvm-agent-rpm-302">
              <para>For KVM hosts, you will need to upgrade the <filename>cloud-agent</filename>
                package, similarly installing the new version as
                  <filename>cloudstack-agent</filename>.</para>
              <programlisting language="Bash">$ sudo yum upgrade cloud-agent</programlisting>
              <para>During the installation of <filename>cloudstack-agent</filename>, the RPM will
                copy your <filename>agent.properties</filename>,
                  <filename>log4j-cloud.xml</filename>, and
                  <filename>environment.properties</filename> from
                  <filename>/etc/cloud/agent</filename> to
                  <filename>/etc/cloudstack/agent</filename>.</para>
            </listitem>
            <listitem>
              <para>Verify that the file
                  <filename>/etc/cloudstack/agent/environment.properties</filename> has a line that
                reads:</para>
              <programlisting language="Bash">paths.script=/usr/share/cloudstack-common</programlisting>
              <para>If not, add the line.</para>
            </listitem>
            <listitem>
              <para>Restart the agent:</para>
              <programlisting language="Bash">
service cloud-agent stop
killall jsvc
service cloudstack-agent start
                            </programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="correct-components-xml-302">
          <para>If you have made changes to your copy of
              <filename>/etc/cloud/management/components.xml</filename> the changes will be
            preserved in the upgrade. However, you need to do the following steps to place these
            changes in a new version of the file which is compatible with version 4.2.0.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Make a backup copy of <filename>/etc/cloud/management/components.xml</filename>.
                For example:</para>
              <programlisting># mv /etc/cloud/management/components.xml /etc/cloud/management/components.xml-backup</programlisting>
            </listitem>
            <listitem>
              <para>Copy <filename>/etc/cloud/management/components.xml.rpmnew</filename> to create
                a new <filename>/etc/cloud/management/components.xml</filename>:</para>
              <programlisting># cp -ap /etc/cloud/management/components.xml.rpmnew /etc/cloud/management/components.xml</programlisting>
            </listitem>
            <listitem>
              <para>Merge your changes from the backup file into the new
                  <filename>components.xml</filename>.</para>
              <programlisting># vi /etc/cloudstack/management/components.xml</programlisting>
            </listitem>
          </orderedlist>
          <note>
            <para>If you have more than one management server node, repeat the upgrade steps on each
              node.</para>
          </note>
        </listitem>
        <listitem>
          <para>After upgrading to 4.2, API clients are expected to send plain text passwords for
            login and user creation, instead of MD5 hash. Incase, api client changes are not
            acceptable, following changes are to be made for backward compatibility:</para>
          <para>Modify componentsContext.xml, and make PlainTextUserAuthenticator as the default
            authenticator (1st entry in the userAuthenticators adapter list is default)</para>
          <programlisting language="XML">
&lt;!-- Security adapters --&gt;
&lt;bean id="userAuthenticators" class="com.cloud.utils.component.AdapterList"&gt;
  &lt;property name="Adapters"&gt;
    &lt;list&gt;
      &lt;ref bean="PlainTextUserAuthenticator"/&gt;
      &lt;ref bean="MD5UserAuthenticator"/&gt;
      &lt;ref bean="LDAPUserAuthenticator"/&gt;
    &lt;/list&gt;
  &lt;/property&gt;
&lt;/bean&gt;
            </programlisting>
          <para>PlainTextUserAuthenticator works the same way MD5UserAuthenticator worked prior to
            4.2.</para>
        </listitem>
        <listitem>
          <para>Start the first Management Server. Do not start any other Management Server nodes
            yet.</para>
          <programlisting language="Bash"><prompt>#</prompt> service cloudstack-management start</programlisting>
          <para>Wait until the databases are upgraded. Ensure that the database upgrade is complete.
            After confirmation, start the other Management Servers one at a time by running the same
            command on each node.</para>
          <note>
            <para>Failing to restart the Management Server indicates a problem in the upgrade.
              Having the Management Server restarted without any issues indicates that the upgrade
              is successfully completed.</para>
          </note>
        </listitem>
        <listitem>
          <para>Start all Usage Servers (if they were running on your previous version). Perform
            this on each Usage Server host.</para>
          <para><command># service cloudstack-usage start</command></para>
        </listitem>
        <listitem>
          <para>Additional steps are required for each KVM host. These steps will not affect running
            guests in the cloud. These steps are required only for clouds using KVM as hosts and
            only on the KVM hosts.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Configure a yum or apt repository containing the &PRODUCT; packages as outlined
                in the Installation Guide.</para>
            </listitem>
            <listitem>
              <para>Stop the running agent.</para>
              <para><command># service cloud-agent stop</command></para>
            </listitem>
            <listitem>
              <para>Update the agent software with one of the following command sets as appropriate
                for your environment.</para>
              <para><command># yum update cloud-*</command></para>
              <para><command># apt-get update</command></para>
              <para><command># apt-get upgrade cloud-*</command></para>
            </listitem>
            <listitem>
              <para>Edit <filename>/etc/cloudstack/agent/agent.properties</filename> to change the
                resource parameter from
                "com.cloud.agent.resource.computing.LibvirtComputingResource" to
                "com.cloud.hypervisor.kvm.resource.LibvirtComputingResource".</para>
            </listitem>
            <listitem>
              <para>Start the cloud agent and cloud management services.</para>
              <programlisting># service cloudstack-agent start</programlisting>
            </listitem>
            <listitem>
              <para>When the Management Server is up and running, log in to the CloudStack UI and
                restart the virtual router for proper functioning of all the features.</para>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>Log in to the CloudStack UI as administrator, and check the status of the hosts. All
            hosts should come to Up state (except those that you know to be offline). You may need
            to wait 20 or 30 minutes, depending on the number of hosts.</para>
          <note>
            <para>Troubleshooting: If login fails, clear your browser cache and reload the
              page.</para>
          </note>
          <para>Do not proceed to the next step until the hosts show in Up state.</para>
        </listitem>
        <listitem>
          <para>If you are upgrading from 3.0.2, perform the following:</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Ensure that the admin port is set to 8096 by using the "integration.api.port"
                global parameter.</para>
              <para>This port is used by the cloud-sysvmadm script at the end of the upgrade
                procedure. For information about how to set this parameter, see "Setting Global
                Configuration Parameters" in the Installation Guide.</para>
            </listitem>
            <listitem>
              <para>Restart the Management Server.</para>
              <note>
                <para>If you don't want the admin port to remain open, you can set it to null after
                  the upgrade is done and restart the management server.</para>
              </note>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>Run the <command>cloud-sysvmadm</command> script to stop, then start, all Secondary
            Storage VMs, Console Proxy VMs, and virtual routers. Run the script once on each
            management server. Substitute your own IP address of the MySQL instance, the MySQL user
            to connect as, and the password to use for that user. In addition to those parameters,
            provide the <command>-c</command> and <command>-r</command> arguments. For
            example:</para>
          <para><command># nohup cloud-sysvmadm -d 192.168.1.5 -u cloud -p password -c -r >
              sysvm.log 2>&amp;1 &amp;</command></para>
          <para><command># tail -f sysvm.log</command></para>
          <para>This might take up to an hour or more to run, depending on the number of accounts in
            the system.</para>
        </listitem>
        <listitem>
          <para>If needed, upgrade all Citrix XenServer hypervisor hosts in your cloud to a version
            supported by CloudStack 4.2.0. The supported versions are XenServer 5.6 SP2 and 6.0.2.
            Instructions for upgrade can be found in the CloudStack 4.2.0 Installation Guide under
            "Upgrading XenServer Versions."</para>
        </listitem>
        <listitem>
          <para>Now apply the XenServer hotfix XS602E003 (and any other needed hotfixes) to
            XenServer v6.0.2 hypervisor hosts.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Disconnect the XenServer cluster from CloudStack.</para>
              <para>In the left navigation bar of the CloudStack UI, select Infrastructure. Under
                Clusters, click View All. Select the XenServer cluster and click Actions -
                Unmanage.</para>
              <para>This may fail if there are hosts not in one of the states Up, Down,
                Disconnected, or Alert. You may need to fix that before unmanaging this
                cluster.</para>
              <para>Wait until the status of the cluster has reached Unmanaged. Use the CloudStack
                UI to check on the status. When the cluster is in the unmanaged state, there is no
                connection to the hosts in the cluster.</para>
            </listitem>
            <listitem>
              <para>To clean up the VLAN, log in to one XenServer host and run:</para>
              <para><command>/opt/xensource/bin/cloud-clean-vlan.sh</command></para>
            </listitem>
            <listitem>
              <para>Now prepare the upgrade by running the following on one XenServer host:</para>
              <para><command>/opt/xensource/bin/cloud-prepare-upgrade.sh</command></para>
              <para>If you see a message like "can't eject CD", log in to the VM and unmount the CD,
                then run this script again.</para>
            </listitem>
            <listitem>
              <para>Upload the hotfix to the XenServer hosts. Always start with the Xen pool master,
                then the slaves. Using your favorite file copy utility (e.g. WinSCP), copy the
                hotfixes to the host. Place them in a temporary folder such as /tmp. </para>
              <para>On the Xen pool master, upload the hotfix with this command:</para>
              <para><command>xe patch-upload file-name=XS602E003.xsupdate</command></para>
              <para>Make a note of the output from this command, which is a UUID for the hotfix
                file. You'll need it in another step later.</para>
              <note>
                <para>(Optional) If you are applying other hotfixes as well, you can repeat the
                  commands in this section with the appropriate hotfix number. For example,
                  XS602E004.xsupdate.</para>
              </note>
            </listitem>
            <listitem>
              <para>Manually live migrate all VMs on this host to another host. First, get a list of
                the VMs on this host:</para>
              <para><command># xe vm-list</command></para>
              <para>Then use this command to migrate each VM. Replace the example host name and VM
                name with your own:</para>
              <para><command># xe vm-migrate live=true host=<replaceable>host-name</replaceable>
                    vm=<replaceable>VM-name</replaceable></command></para>
              <note>
                <title>Troubleshooting</title>
                <para>If you see a message like "You attempted an operation on a VM which requires
                  PV drivers to be installed but the drivers were not detected," run:</para>
                <para><command>/opt/xensource/bin/make_migratable.sh
                    b6cf79c8-02ee-050b-922f-49583d9f1a14</command>.</para>
              </note>
            </listitem>
            <listitem>
              <para>Apply the hotfix. First, get the UUID of this host:</para>
              <programlisting># xe host-list</programlisting>
              <para>Then use the following command to apply the hotfix. Replace the example host
                UUID with the current host ID, and replace the hotfix UUID with the output from the
                patch-upload command you ran on this machine earlier. You can also get the hotfix
                UUID by running xe patch-list. </para>
              <programlisting><command>xe</command> patch-apply host-uuid=<replaceable>host-uuid</replaceable> uuid=<replaceable>hotfix-uuid</replaceable></programlisting>
            </listitem>
            <listitem>
              <para>Copy the following files from the CloudStack Management Server to the
                host.</para>
              <informaltable>
                <tgroup cols="2" align="left" colsep="1" rowsep="1">
                  <colspec colwidth="1*" colname="1" colnum="1"/>
                  <colspec colwidth="2*" colname="2" colnum="2"/>
                  <thead>
                    <row>
                      <entry><para>Copy from here...</para></entry>
                      <entry><para>...to here</para></entry>
                    </row>
                  </thead>
                  <tbody>
                    <row>
                      <entry><para>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/xenserver60/NFSSR.py</para></entry>
                      <entry><para>/opt/xensource/sm/NFSSR.py</para></entry>
                    </row>
                    <row>
                      <entry><para>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/setupxenserver.sh</para></entry>
                      <entry><para>/opt/xensource/bin/setupxenserver.sh</para></entry>
                    </row>
                    <row>
                      <entry><para>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/make_migratable.sh</para></entry>
                      <entry><para>/opt/xensource/bin/make_migratable.sh</para></entry>
                    </row>
                  </tbody>
                </tgroup>
              </informaltable>
            </listitem>
            <listitem>
              <para>(Only for hotfixes XS602E005 and XS602E007) You need to apply a new Cloud
                Support Pack.</para>
              <itemizedlist>
                <listitem>
                  <para>Download the CSP software onto the XenServer host from one of the following
                    links:</para>
                  <para>For hotfix XS602E005: <ulink
                      url="http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz"
                      >http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz</ulink></para>
                  <para>For hotfix XS602E007: <ulink
                      url="http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz"
                      >http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz</ulink></para>
                </listitem>
                <listitem>
                  <para>Extract the file:</para>
                  <programlisting># tar xf xenserver-cloud-supp.tgz</programlisting>
                </listitem>
                <listitem>
                  <para>Run the following script:</para>
                  <programlisting># xe-install-supplemental-pack xenserver-cloud-supp.iso</programlisting>
                </listitem>
                <listitem>
                  <para>If the XenServer host is part of a zone that uses basic networking, disable
                    Open vSwitch (OVS):</para>
                  <programlisting># xe-switch-network-backend  bridge</programlisting>
                </listitem>
              </itemizedlist>
            </listitem>
            <listitem>
              <para>Reboot this XenServer host.</para>
            </listitem>
            <listitem>
              <para>Run the following:</para>
              <programlisting>/opt/xensource/bin/setupxenserver.sh</programlisting>
              <note>
                <para>If the message "mv: cannot stat `/etc/cron.daily/logrotate': No such file or
                  directory" appears, you can safely ignore it.</para>
              </note>
            </listitem>
            <listitem>
              <para>Run the following:</para>
              <programlisting>for pbd in `xe pbd-list currently-attached=false| grep ^uuid | awk '{print $NF}'`; do xe pbd-plug uuid=$pbd ; </programlisting>
            </listitem>
            <listitem>
              <para>On each slave host in the Xen pool, repeat these steps, starting from "manually
                live migrate VMs."</para>
            </listitem>
          </orderedlist>
        </listitem>
      </orderedlist>
      <note>
        <title>Troubleshooting Tip</title>
        <para>If passwords which you know to be valid appear not to work after upgrade, or other UI
          issues are seen, try clearing your browser cache and reloading the UI page.</para>
      </note>
    </section>
    <section id="upgrade-from-2.2.x-to-4.1">
      <title>Upgrade from 2.2.14 to 4.2.0</title>
      <orderedlist>
        <listitem>
          <para>Ensure that you query your IPaddress usage records and process them; for example,
            issue invoices for any usage that you have not yet billed users for.</para>
          <para>Starting in 3.0.2, the usage record format for IP addresses is the same as the rest
            of the usage types. Instead of a single record with the assignment and release dates,
            separate records are generated per aggregation period with start and end dates. After
            upgrading to 4.2.0, any existing IP address usage records in the old format will no
            longer be available.</para>
        </listitem>
        <listitem>
          <para>If you are using version 2.2.0 - 2.2.13, first upgrade to 2.2.14 by using the
            instructions in the <ulink
              url="http://download.cloud.com/releases/2.2.0/CloudStack2.2.14ReleaseNotes.pdf">2.2.14
              Release Notes</ulink>.</para>
          <warning>
            <title>KVM Hosts</title>
            <para>If KVM hypervisor is used in your cloud, be sure you completed the step to insert
              a valid username and password into the host_details table on each KVM node as
              described in the 2.2.14 Release Notes. This step is critical, as the database will be
              encrypted after the upgrade to 4.2.0.</para>
          </warning>
        </listitem>
        <listitem>
          <para>While running the 2.2.14 system, log in to the UI as root administrator.</para>
        </listitem>
        <listitem>
          <para>Using the UI, add a new System VM template for each hypervisor type that is used in
            your cloud. In each zone, add a system VM template for each hypervisor used in that
            zone</para>
          <orderedlist>
            <listitem>
              <para>In the left navigation bar, click Templates.</para>
            </listitem>
            <listitem>
              <para>In Select view, click Templates.</para>
            </listitem>
            <listitem>
              <para>Click Register template.</para>
              <para>The Register template dialog box is displayed.</para>
            </listitem>
            <listitem>
              <para>In the Register template dialog box, specify the following values depending on
                the hypervisor type (do not change these):</para>
              <informaltable>
                <tgroup cols="2" align="left" colsep="1" rowsep="1">
                  <colspec colwidth="1*" colname="1" colnum="1"/>
                  <colspec colwidth="2*" colname="2" colnum="2"/>
                  <thead>
                    <row>
                      <entry><para>Hypervisor</para></entry>
                      <entry><para>Description</para></entry>
                    </row>
                  </thead>
                  <tbody>
                    <row>
                      <entry><para>XenServer</para></entry>
                      <entry><para>Name: systemvm-xenserver-4.2.0</para>
                        <para>Description: systemvm-xenserver-4.2.0</para>
                        <para>URL:http://download.cloud.com/templates/4.2/systemvmtemplate-2013-07-12-master-xen.vhd.bz2 </para>
                        <para>Zone: Choose the zone where this hypervisor is used</para>
                        <para>Hypervisor: XenServer</para>
                        <para>Format: VHD</para>
                        <para>OS Type: Debian GNU/Linux 6.0 (32-bit)</para>
                        <para>Extractable: no</para>
                        <para>Password Enabled: no</para>
                        <para>Public: no</para>
                        <para>Featured: no</para>
                      </entry>
                    </row>
                    <row>
                      <entry><para>KVM</para></entry>
                      <entry><para>Name: systemvm-kvm-4.2.0</para>
                        <para>Description: systemvm-kvm-4.2.0</para>
                        <para>URL:
                          http://download.cloud.com/templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2</para>
                        <para>Zone: Choose the zone where this hypervisor is used</para>
                        <para>Hypervisor: KVM</para>
                        <para>Format: QCOW2</para>
                        <para>OS Type: Debian GNU/Linux 5.0 (32-bit)</para>
                        <para>Extractable: no</para>
                        <para>Password Enabled: no</para>
                        <para>Public: no</para>
                        <para>Featured: no</para>
                      </entry>
                    </row>
                    <row>
                      <entry><para>VMware</para></entry>
                      <entry><para>Name: systemvm-vmware-4.2.0</para>
                        <para>Description: systemvm-vmware-4.2.0</para>
                        <para>URL:
                          http://download.cloud.com/templates/4.2/systemvmtemplate-4.2-vh7.ova</para>
                        <para>Zone: Choose the zone where this hypervisor is used</para>
                        <para>Hypervisor: VMware</para>
                        <para>Format: OVA</para>
                        <para>OS Type: Debian GNU/Linux 5.0 (32-bit)</para>
                        <para>Extractable: no</para>
                        <para>Password Enabled: no</para>
                        <para>Public: no</para>
                        <para>Featured: no</para>
                      </entry>
                    </row>
                  </tbody>
                </tgroup>
              </informaltable>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>Watch the screen to be sure that the template downloads successfully and enters the
            READY state. Do not proceed until this is successful</para>
        </listitem>
        <listitem>
          <para><emphasis role="bold">WARNING</emphasis>: If you use more than one type of
            hypervisor in your cloud, be sure you have repeated these steps to download the system
            VM template for each hypervisor type. Otherwise, the upgrade will fail.</para>
        </listitem>
        <listitem>
          <para>Stop all Usage Servers if running. Run this on all Usage Server hosts.</para>
          <programlisting># service cloud-usage stop</programlisting>
        </listitem>
        <listitem>
          <para>Stop the Management Servers. Run this on all Management Server hosts.</para>
          <programlisting language="Bash"><prompt>#</prompt> service cloud-management stop</programlisting>
        </listitem>
        <listitem>
          <para>On the MySQL master, take a backup of the MySQL databases. We recommend performing
            this step even in test upgrades. If there is an issue, this will assist with
            debugging.</para>
          <para>In the following commands, it is assumed that you have set the root password on the
            database, which is a CloudStack recommended best practice. Substitute your own MySQL
            root password.</para>
          <programlisting><prompt>#</prompt> <command>mysqldump</command> -u root -p<replaceable>mysql_password</replaceable> cloud > <filename>cloud-backup.dmp</filename>
                        <prompt>#</prompt> <command>mysqldump</command> -u root -p<replaceable>mysql_password</replaceable> cloud_usage > <filename>cloud-usage-backup.dmp</filename>
                    </programlisting>
        </listitem>
        <listitem>
          <para> Either build RPM/DEB packages as detailed in the Installation Guide, or use one of
            the community provided yum/apt repositories to gain access to the &PRODUCT; binaries.
          </para>
        </listitem>
        <listitem id="upgrade-deb-packages-22">
          <para>If you are using Ubuntu, follow this procedure to upgrade your packages. If not,
            skip to step <xref linkend="upgrade-rpm-packages-22"/>.</para>
          <note>
            <title>Community Packages</title>
            <para>This section assumes you're using the community supplied packages for &PRODUCT;.
              If you've created your own packages and APT repository, substitute your own URL for
              the ones used in these examples.</para>
          </note>
          <orderedlist id="debsteps-22">
            <listitem>
              <para>The first order of business will be to change the sources list for each system
                with &PRODUCT; packages. This means all management servers, and any hosts that have
                the KVM agent. (No changes should be necessary for hosts that are running VMware or
                Xen.)</para>
              <para>Start by opening <filename>/etc/apt/sources.list.d/cloudstack.list</filename> on
                any systems that have &PRODUCT; packages installed.</para>
              <para>This file should have one line, which contains:</para>
              <programlisting language="Bash">deb http://cloudstack.apt-get.eu/ubuntu precise 4.0</programlisting>
              <para>We'll change it to point to the new package repository:</para>
              <programlisting language="Bash">deb http://cloudstack.apt-get.eu/ubuntu precise 4.2</programlisting>
              <para>If you're using your own package repository, change this line to read as
                appropriate for your 4.2.0 repository.</para>
            </listitem>
            <listitem>
              <para>Now update your apt package list:</para>
              <programlisting language="Bash">$ sudo apt-get update</programlisting>
            </listitem>
            <listitem id="deb-master-22">
              <para>Now that you have the repository configured, it's time to install the
                  <filename>cloudstack-management</filename> package. This will pull in any other
                dependencies you need.</para>
              <programlisting language="Bash">$ sudo apt-get install cloudstack-management</programlisting>
            </listitem>
            <listitem id="kvm-agent-deb-22">
              <para>On KVM hosts, you will need to manually install the
                  <filename>cloudstack-agent</filename> package:</para>
              <programlisting language="Bash">$ sudo apt-get install cloudstack-agent</programlisting>
              <para>During the installation of <filename>cloudstack-agent</filename>, APT will copy
                your <filename>agent.properties</filename>, <filename>log4j-cloud.xml</filename>,
                and <filename>environment.properties</filename> from
                  <filename>/etc/cloud/agent</filename> to
                  <filename>/etc/cloudstack/agent</filename>.</para>
              <para>When prompted whether you wish to keep your configuration, say Yes.</para>
            </listitem>
            <listitem>
              <para>Verify that the file
                  <filename>/etc/cloudstack/agent/environment.properties</filename> has a line that
                reads:</para>
              <programlisting language="Bash">paths.script=/usr/share/cloudstack-common</programlisting>
              <para>If not, add the line.</para>
            </listitem>
            <listitem>
              <para>Restart the agent:</para>
              <programlisting language="Bash">
service cloud-agent stop
killall jsvc
service cloudstack-agent start
              </programlisting>
            </listitem>
            <listitem>
              <para>During the upgrade, <filename>log4j-cloud.xml</filename> was simply copied over,
                so the logs will continue to be added to
                  <filename>/var/log/cloud/agent/agent.log</filename>. There's nothing
                  <emphasis>wrong</emphasis> with this, but if you prefer to be consistent, you can
                change this by copying over the sample configuration file:</para>
              <programlisting language="Bash">
cd /etc/cloudstack/agent
mv log4j-cloud.xml.dpkg-dist log4j-cloud.xml
service cloudstack-agent restart
                            </programlisting>
            </listitem>
            <listitem>
              <para>Once the agent is running, you can uninstall the old cloud-* packages from your
                system:</para>
              <programlisting language="Bash">sudo dpkg --purge cloud-agent</programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="upgrade-rpm-packages-22">
          <para>If you are using CentOS or RHEL, follow this procedure to upgrade your packages. If
            not, skip to step <xref linkend="correct-components-xml-22"/>.</para>
          <note>
            <title>Community Packages</title>
            <para>This section assumes you're using the community supplied packages for &PRODUCT;.
              If you've created your own packages and yum repository, substitute your own URL for
              the ones used in these examples.</para>
          </note>
          <orderedlist id="rpmsteps-22">
            <listitem>
              <para>The first order of business will be to change the yum repository for each system
                with &PRODUCT; packages. This means all management servers, and any hosts that have
                the KVM agent. (No changes should be necessary for hosts that are running VMware or
                Xen.)</para>
              <para>Start by opening <filename>/etc/yum.repos.d/cloudstack.repo</filename> on any
                systems that have &PRODUCT; packages installed.</para>
              <para>This file should have content similar to the following:</para>
              <programlisting language="Bash">
[apache-cloudstack]
name=Apache CloudStack
baseurl=http://cloudstack.apt-get.eu/rhel/4.0/
enabled=1
gpgcheck=0
                            </programlisting>
              <para>If you are using the community provided package repository, change the baseurl
                to http://cloudstack.apt-get.eu/rhel/4.2/</para>
              <para>If you're using your own package repository, change this line to read as
                appropriate for your 4.2.0 repository.</para>
            </listitem>
            <listitem id="rpm-master-22">
              <para>Now that you have the repository configured, it's time to install the
                  <filename>cloudstack-management</filename> package by upgrading the older
                  <filename>cloud-client</filename> package.</para>
              <programlisting language="Bash">$ sudo yum upgrade cloud-client</programlisting>
            </listitem>
            <listitem id="kvm-agent-rpm-22">
              <para>For KVM hosts, you will need to upgrade the <filename>cloud-agent</filename>
                package, similarly installing the new version as
                  <filename>cloudstack-agent</filename>.</para>
              <programlisting language="Bash">$ sudo yum upgrade cloud-agent</programlisting>
              <para>During the installation of <filename>cloudstack-agent</filename>, the RPM will
                copy your <filename>agent.properties</filename>,
                  <filename>log4j-cloud.xml</filename>, and
                  <filename>environment.properties</filename> from
                  <filename>/etc/cloud/agent</filename> to
                  <filename>/etc/cloudstack/agent</filename>.</para>
            </listitem>
            <listitem>
              <para>Verify that the file
                  <filename>/etc/cloudstack/agent/environment.properties</filename> has a line that
                reads:</para>
              <programlisting language="Bash">paths.script=/usr/share/cloudstack-common</programlisting>
              <para>If not, add the line.</para>
            </listitem>
            <listitem>
              <para>Restart the agent:</para>
              <programlisting language="Bash">
service cloud-agent stop
killall jsvc
service cloudstack-agent start
                            </programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="correct-components-xml-22">
          <para>If you have made changes to your existing copy of the file components.xml in your
            previous-version CloudStack installation, the changes will be preserved in the upgrade.
            However, you need to do the following steps to place these changes in a new version of
            the file which is compatible with version 4.0.0-incubating.</para>
          <note>
            <para>How will you know whether you need to do this? If the upgrade output in the
              previous step included a message like the following, then some custom content was
              found in your old components.xml, and you need to merge the two files:</para>
          </note>
          <programlisting>warning: /etc/cloud/management/components.xml created as /etc/cloud/management/components.xml.rpmnew </programlisting>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Make a backup copy of your
                  <filename>/etc/cloud/management/components.xml</filename> file. For
                example:</para>
              <programlisting><prompt>#</prompt> <command>mv</command> <filename>/etc/cloud/management/components.xml</filename> <filename>/etc/cloud/management/components.xml-backup</filename></programlisting>
            </listitem>
            <listitem>
              <para>Copy <filename>/etc/cloud/management/components.xml.rpmnew</filename> to create
                a new <filename>/etc/cloud/management/components.xml</filename>:</para>
              <programlisting><prompt>#</prompt> <command>cp</command> -ap <filename>/etc/cloud/management/components.xml.rpmnew</filename> <filename>/etc/cloud/management/components.xml</filename></programlisting>
            </listitem>
            <listitem>
              <para>Merge your changes from the backup file into the new components.xml file.</para>
              <programlisting><prompt>#</prompt> <command>vi</command> <filename>/etc/cloud/management/components.xml</filename>
                            </programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>After upgrading to 4.2, API clients are expected to send plain text passwords for
            login and user creation, instead of MD5 hash. If API client changes are not acceptable,
            following changes are to be made for backward compatibility:</para>
          <para>Modify componentsContext.xml, and make PlainTextUserAuthenticator as the default
            authenticator (1st entry in the userAuthenticators adapter list is default)</para>
          <programlisting language="XML">
&lt;!-- Security adapters --&gt;
&lt;bean id="userAuthenticators" class="com.cloud.utils.component.AdapterList"&gt;
  &lt;property name="Adapters"&gt;
    &lt;list&gt;
      &lt;ref bean="PlainTextUserAuthenticator"/&gt;
      &lt;ref bean="MD5UserAuthenticator"/&gt;
      &lt;ref bean="LDAPUserAuthenticator"/&gt;
    &lt;/list&gt;
  &lt;/property&gt;
&lt;/bean&gt;
            </programlisting>
          <para>PlainTextUserAuthenticator works the same way MD5UserAuthenticator worked prior to
            4.2.</para>
        </listitem>
        <listitem>
          <para>If you have made changes to your existing copy of the
              <filename>/etc/cloud/management/db.properties</filename> file in your previous-version
            CloudStack installation, the changes will be preserved in the upgrade. However, you need
            to do the following steps to place these changes in a new version of the file which is
            compatible with version 4.0.0-incubating.</para>
          <orderedlist>
            <listitem>
              <para>Make a backup copy of your file
                  <filename>/etc/cloud/management/db.properties</filename>. For example:</para>
              <programlisting><prompt>#</prompt> <command>mv</command> <filename>/etc/cloud/management/db.properties</filename> <filename>/etc/cloud/management/db.properties-backup</filename></programlisting>
            </listitem>
            <listitem>
              <para>Copy <filename>/etc/cloud/management/db.properties.rpmnew</filename> to create a
                new <filename>/etc/cloud/management/db.properties</filename>:</para>
              <programlisting><prompt>#</prompt> <command>cp</command> -ap <filename>/etc/cloud/management/db.properties.rpmnew</filename> <filename>etc/cloud/management/db.properties</filename></programlisting>
            </listitem>
            <listitem>
              <para>Merge your changes from the backup file into the new db.properties file.</para>
              <programlisting><prompt>#</prompt> <command>vi</command> <filename>/etc/cloud/management/db.properties</filename></programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>On the management server node, run the following command. It is recommended that you
            use the command-line flags to provide your own encryption keys. See Password and Key
            Encryption in the Installation Guide.</para>
          <programlisting><prompt>#</prompt> <command>cloudstack-setup-encryption</command> -e <replaceable>encryption_type</replaceable> -m <replaceable>management_server_key</replaceable> -k <replaceable>database_key</replaceable></programlisting>
          <para>When used without arguments, as in the following example, the default encryption
            type and keys will be used:</para>
          <itemizedlist>
            <listitem>
              <para>(Optional) For encryption_type, use file or web to indicate the technique used
                to pass in the database encryption password. Default: file.</para>
            </listitem>
            <listitem>
              <para>(Optional) For management_server_key, substitute the default key that is used to
                encrypt confidential parameters in the properties file. Default: password. It is
                highly recommended that you replace this with a more secure value</para>
            </listitem>
            <listitem>
              <para>(Optional) For database_key, substitute the default key that is used to encrypt
                confidential parameters in the CloudStack database. Default: password. It is highly
                recommended that you replace this with a more secure value.</para>
            </listitem>
          </itemizedlist>
        </listitem>
        <listitem>
          <para>Repeat steps 10 - 14 on every management server node. If you provided your own
            encryption key in step 14, use the same key on all other management servers.</para>
        </listitem>
        <listitem>
          <para>Start the first Management Server. Do not start any other Management Server nodes
            yet.</para>
          <programlisting language="Bash"><prompt>#</prompt> service cloudstack-management start</programlisting>
          <para>Wait until the databases are upgraded. Ensure that the database upgrade is complete.
            You should see a message like "Complete! Done." After confirmation, start the other
            Management Servers one at a time by running the same command on each node.</para>
        </listitem>
        <listitem>
          <para>Start all Usage Servers (if they were running on your previous version). Perform
            this on each Usage Server host.</para>
          <programlisting language="Bash"><prompt>#</prompt> service cloudstack-usage start</programlisting>
        </listitem>
        <listitem>
          <para>(KVM only) Additional steps are required for each KVM host. These steps will not
            affect running guests in the cloud. These steps are required only for clouds using KVM
            as hosts and only on the KVM hosts.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Copy the CloudPlatform 4.2 tar file to the host, untar it, and change directory
                to the resulting directory.</para>
            </listitem>
            <listitem>
              <para>Stop the running agent.</para>
              <programlisting># service cloud-agent stop</programlisting>
            </listitem>
            <listitem>
              <para>Update the agent software.</para>
              <programlisting># ./install.sh</programlisting>
            </listitem>
            <listitem>
              <para>Choose "U" to update the packages.</para>
            </listitem>
            <listitem>
              <para>Start the agent.</para>
              <programlisting># service cloudstack-agent start</programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>(KVM only) Perform the following additional steps on each KVM host. </para>
          <para>These steps will not affect running guests in the cloud. These steps are required
            only for clouds using KVM as hosts and only on the KVM hosts.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para> Configure your CloudStack package repositories as outlined in the Installation
                Guide </para>
            </listitem>
            <listitem>
              <para>Stop the running agent.</para>
              <programlisting># service cloud-agent stop</programlisting>
            </listitem>
            <listitem>
              <para>Update the agent software with one of the following command sets as
                appropriate.</para>
              <programlisting><prompt>#</prompt> <command>yum</command> update cloud-*</programlisting>
              <programlisting>
                                <prompt>#</prompt> <command>apt-get</command> update
                                <prompt>#</prompt> <command>apt-get</command> upgrade cloud-*
                            </programlisting>
            </listitem>
            <listitem>
              <para>Start the agent.</para>
              <programlisting># service cloudstack-agent start</programlisting>
            </listitem>
            <listitem>
              <para> Copy the contents of the <filename>agent.properties</filename> file to the new
                  <filename>agent.properties</filename> file by using the following command</para>
              <programlisting><command>sed</command> -i 's/com.cloud.agent.resource.computing.LibvirtComputingResource/com.cloud.hypervisor.kvm.resource.LibvirtComputingResource/g' <filename>/etc/cloud/agent/agent.properties</filename></programlisting>
            </listitem>
            <listitem>
              <para>Start the cloud agent and cloud management services.</para>
            </listitem>
            <listitem>
              <para>When the Management Server is up and running, log in to the CloudStack UI and
                restart the virtual router for proper functioning of all the features.</para>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>Log in to the CloudStack UI as admin, and check the status of the hosts. All hosts
            should come to Up state (except those that you know to be offline). You may need to wait
            20 or 30 minutes, depending on the number of hosts.</para>
          <para>Do not proceed to the next step until the hosts show in the Up state. If the hosts
            do not come to the Up state, contact support.</para>
        </listitem>
        <listitem>
          <para>Run the following script to stop, then start, all Secondary Storage VMs, Console
            Proxy VMs, and virtual routers.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Run the command once on one management server. Substitute your own IP address of
                the MySQL instance, the MySQL user to connect as, and the password to use for that
                user. In addition to those parameters, provide the "-c" and "-r" arguments. For
                example:</para>
              <programlisting><prompt>#</prompt> <command>nohup cloud-sysvmadm</command> -d <replaceable>192.168.1.5</replaceable> -u cloud -p <replaceable>password</replaceable> -c -r > sysvm.log 2>&amp;1 &amp;
                                <prompt>#</prompt> <command>tail</command> -f <filename>sysvm.log</filename></programlisting>
              <para>This might take up to an hour or more to run, depending on the number of
                accounts in the system.</para>
            </listitem>
            <listitem>
              <para>After the script terminates, check the log to verify correct execution:</para>
              <programlisting><prompt>#</prompt> <command>tail</command> -f <filename>sysvm.log</filename></programlisting>
              <para>The content should be like the following:</para>
              <programlisting>
                                Stopping and starting 1 secondary storage vm(s)...
                                Done stopping and starting secondary storage vm(s)
                                Stopping and starting 1 console proxy vm(s)...
                                Done stopping and starting console proxy vm(s).
                                Stopping and starting 4 running routing vm(s)...
                                Done restarting router(s).
                            </programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>If you would like additional confirmation that the new system VM templates were
            correctly applied when these system VMs were rebooted, SSH into the System VM and check
            the version.</para>
          <para>Use one of the following techniques, depending on the hypervisor.</para>
          <formalpara>
            <title>XenServer or KVM:</title>
            <para>SSH in by using the link local IP address of the system VM. For example, in the
              command below, substitute your own path to the private key used to log in to the
              system VM and your own link local IP.</para>
          </formalpara>
          <para>Run the following commands on the XenServer or KVM host on which the system VM is
            present:</para>
          <programlisting><prompt>#</prompt> <command>ssh</command> -i <replaceable>private-key-path</replaceable> <replaceable>link-local-ip</replaceable> -p 3922
                        # cat /etc/cloudstack-release</programlisting>
          <para>The output should be like the following:</para>
          <programlisting>Cloudstack Release 4.0.0-incubating Mon Oct 9 15:10:04 PST 2012</programlisting>
          <formalpara>
            <title>ESXi</title>
            <para>SSH in using the private IP address of the system VM. For example, in the command
              below, substitute your own path to the private key used to log in to the system VM and
              your own private IP.</para>
          </formalpara>
          <para>Run the following commands on the Management Server:</para>
          <programlisting><prompt>#</prompt> <command>ssh</command> -i <replaceable>private-key-path</replaceable> <replaceable>private-ip</replaceable> -p 3922
                        <prompt>#</prompt> <command>cat</command> <filename>/etc/cloudstack-release</filename>
                    </programlisting>
          <para>The output should be like the following:</para>
          <programlisting>Cloudstack Release 4.0.0-incubating Mon Oct 9 15:10:04 PST 2012</programlisting>
        </listitem>
        <listitem>
          <para>If needed, upgrade all Citrix XenServer hypervisor hosts in your cloud to a version
            supported by CloudStack 4.0.0-incubating. The supported versions are XenServer 5.6 SP2
            and 6.0.2. Instructions for upgrade can be found in the CloudStack 4.0.0-incubating
            Installation Guide.</para>
        </listitem>
        <listitem>
          <para>Apply the XenServer hotfix XS602E003 (and any other needed hotfixes) to XenServer
            v6.0.2 hypervisor hosts.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Disconnect the XenServer cluster from CloudStack.</para>
              <para>In the left navigation bar of the CloudStack UI, select Infrastructure. Under
                Clusters, click View All. Select the XenServer cluster and click Actions -
                Unmanage.</para>
              <para>This may fail if there are hosts not in one of the states Up, Down,
                Disconnected, or Alert. You may need to fix that before unmanaging this
                cluster.</para>
              <para>Wait until the status of the cluster has reached Unmanaged. Use the CloudStack
                UI to check on the status. When the cluster is in the unmanaged state, there is no
                connection to the hosts in the cluster.</para>
            </listitem>
            <listitem>
              <para>To clean up the VLAN, log in to one XenServer host and run:</para>
              <programlisting>/opt/xensource/bin/cloud-clean-vlan.sh</programlisting>
            </listitem>
            <listitem>
              <para>Prepare the upgrade by running the following on one XenServer host:</para>
              <programlisting>/opt/xensource/bin/cloud-prepare-upgrade.sh</programlisting>
              <para>If you see a message like "can't eject CD", log in to the VM and umount the CD,
                then run this script again.</para>
            </listitem>
            <listitem>
              <para>Upload the hotfix to the XenServer hosts. Always start with the Xen pool master,
                then the slaves. Using your favorite file copy utility (e.g. WinSCP), copy the
                hotfixes to the host. Place them in a temporary folder such as /root or /tmp. </para>
              <para>On the Xen pool master, upload the hotfix with this command:</para>
              <programlisting>xe patch-upload file-name=XS602E003.xsupdate</programlisting>
              <para>Make a note of the output from this command, which is a UUID for the hotfix
                file. You'll need it in another step later.</para>
              <note>
                <para>(Optional) If you are applying other hotfixes as well, you can repeat the
                  commands in this section with the appropriate hotfix number. For example,
                  XS602E004.xsupdate.</para>
              </note>
            </listitem>
            <listitem>
              <para>Manually live migrate all VMs on this host to another host. First, get a list of
                the VMs on this host:</para>
              <programlisting># xe vm-list</programlisting>
              <para>Then use this command to migrate each VM. Replace the example host name and VM
                name with your own:</para>
              <programlisting><prompt>#</prompt> <command>xe</command> vm-migrate live=true host=<replaceable>host-name</replaceable> vm=<replaceable>VM-name</replaceable></programlisting>
              <note>
                <title>Troubleshooting</title>
                <para>If you see a message like "You attempted an operation on a VM which requires
                  PV drivers to be installed but the drivers were not detected," run:</para>
                <para><command>/opt/xensource/bin/make_migratable.sh
                    b6cf79c8-02ee-050b-922f-49583d9f1a14</command>.</para>
              </note>
            </listitem>
            <listitem>
              <para>Apply the hotfix. First, get the UUID of this host:</para>
              <para><command># xe host-list</command></para>
              <para>Then use the following command to apply the hotfix. Replace the example host
                UUID with the current host ID, and replace the hotfix UUID with the output from the
                patch-upload command you ran on this machine earlier. You can also get the hotfix
                UUID by running xe patch-list. </para>
              <para><command>xe patch-apply host-uuid=<replaceable>host-uuid</replaceable>
                    uuid=<replaceable>hotfix-uuid</replaceable></command></para>
            </listitem>
            <listitem>
              <para>Copy the following files from the CloudStack Management Server to the
                host.</para>
              <informaltable>
                <tgroup cols="2" align="left" colsep="1" rowsep="1">
                  <colspec colwidth="1*" colname="1" colnum="1"/>
                  <colspec colwidth="2*" colname="2" colnum="2"/>
                  <thead>
                    <row>
                      <entry><para>Copy from here...</para></entry>
                      <entry><para>...to here</para></entry>
                    </row>
                  </thead>
                  <tbody>
                    <row>
                      <entry><para><filename>/usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/xenserver60/NFSSR.py</filename></para></entry>
                      <entry><para><filename>/opt/xensource/sm/NFSSR.py</filename></para></entry>
                    </row>
                    <row>
                      <entry><para><filename>/usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/setupxenserver.sh</filename></para></entry>
                      <entry><para><filename>/opt/xensource/bin/setupxenserver.sh</filename></para></entry>
                    </row>
                    <row>
                      <entry><para><filename>/usr/lib64/cloudstack-common/scripts/vm/hypervisor/xenserver/make_migratable.sh</filename></para></entry>
                      <entry><para><filename>/opt/xensource/bin/make_migratable.sh</filename></para></entry>
                    </row>
                  </tbody>
                </tgroup>
              </informaltable>
            </listitem>
            <listitem>
              <para>(Only for hotfixes XS602E005 and XS602E007) You need to apply a new Cloud
                Support Pack.</para>
              <itemizedlist>
                <listitem>
                  <para>Download the CSP software onto the XenServer host from one of the following
                    links:</para>
                  <para>For hotfix XS602E005: <ulink
                      url="http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz"
                      >http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz</ulink></para>
                  <para>For hotfix XS602E007: <ulink
                      url="http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz"
                      >http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz</ulink></para>
                </listitem>
                <listitem>
                  <para>Extract the file:</para>
                  <para><command># tar xf xenserver-cloud-supp.tgz</command></para>
                </listitem>
                <listitem>
                  <para>Run the following script:</para>
                  <para><command># xe-install-supplemental-pack
                    xenserver-cloud-supp.iso</command></para>
                </listitem>
                <listitem>
                  <para>If the XenServer host is part of a zone that uses basic networking, disable
                    Open vSwitch (OVS):</para>
                  <para><command># xe-switch-network-backend bridge</command></para>
                </listitem>
              </itemizedlist>
            </listitem>
            <listitem>
              <para>Reboot this XenServer host.</para>
            </listitem>
            <listitem>
              <para>Run the following:</para>
              <para><command>/opt/xensource/bin/setupxenserver.sh</command></para>
              <note>
                <para>If the message "mv: cannot stat `/etc/cron.daily/logrotate': No such file or
                  directory" appears, you can safely ignore it.</para>
              </note>
            </listitem>
            <listitem>
              <para>Run the following:</para>
              <para><command>for pbd in `xe pbd-list currently-attached=false| grep ^uuid | awk
                  '{print $NF}'`; do xe pbd-plug uuid=$pbd ; </command>
              </para>
            </listitem>
            <listitem>
              <para>On each slave host in the Xen pool, repeat these steps, starting from "manually
                live migrate VMs."</para>
            </listitem>
          </orderedlist>
        </listitem>
      </orderedlist>
    </section>
  </chapter>
  <chapter id="api-changes">
    <title>API Changes in 4.2</title>
    <section id="added-API-commands-4.2">
      <title>Added API Commands in 4.2</title>
      <section>
        <title>Secondary Storage</title>
        <itemizedlist>
          <listitem>
            <para>addImageStore (Adds all types of secondary storage providers, S3/Swift/NFS)</para>
          </listitem>
          <listitem>
            <para>createSecondaryStagingStore (Adds a staging secondary storage in each zone)</para>
          </listitem>
          <listitem>
            <para>listImageStores (Lists all secondary storages, S3/Swift/NFS)</para>
          </listitem>
          <listitem>
            <para>listSecondaryStagingStores (Lists all staging secondary storages)</para>
          </listitem>
          <listitem>
            <para>addS3 (Adds a Amazon Simple Storage Service instance.) It is recommended to use
              addImageStore instead.</para>
          </listitem>
          <listitem>
            <para>listS3s (Lists all the Amazon Simple Storage Service instances.) It is recommended
              to use listImageStores instead.</para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>VM Snapshot</title>
        <itemizedlist>
          <listitem>
            <para>createVMSnapshot (Creates a virtual machine snapshot; see <xref
                linkend="vm-snapshots"/>)</para>
          </listitem>
          <listitem>
            <para>deleteVMSnapshot (Deletes a virtual machine snapshot)</para>
          </listitem>
          <listitem>
            <para>listVMSnapshot (Shows a virtual machine snapshot)</para>
          </listitem>
          <listitem>
            <para>revertToVMSnapshot (Returns a virtual machine to the state and data saved in a
              given snapshot)</para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Load Balancer Health Check</title>
        <itemizedlist>
          <listitem>
            <para>createLBHealthCheckPolicy (Creates a new health check policy for a load balancer
              rule; see <xref linkend="health-check"/>)</para>
          </listitem>
          <listitem>
            <para>deleteLBHealthCheckPolicy (Deletes an existing health check policy from a load
              balancer rule)</para>
          </listitem>
          <listitem>
            <para>listLBHealthCheckPolicies (Displays the health check policy for a load balancer
              rule)</para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Egress Firewall Rules</title>
        <itemizedlist>
          <listitem>
            <para>createEgressFirewallRules (Creates an egress firewall rule on the guest network;
              see <xref linkend="egress-firewall"/>)</para>
          </listitem>
          <listitem>
            <para>deleteEgressFirewallRules (Deletes a egress firewall rule on the guest
              network.)</para>
          </listitem>
          <listitem>
            <para>listEgressFirewallRules (Lists the egress firewall rules configured for a guest
              network.)</para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>SSH Key</title>
        <itemizedlist>
          <listitem>
            <para>resetSSHKeyForVirtualMachine (Resets the SSHkey for virtual machine.)</para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Bare Metal</title>
        <itemizedlist>
          <listitem>
            <para>addBaremetalHost (Adds a new host. Technically, this API command was present in
              v3.0.6, but its functionality was disabled. See <xref linkend="baremetal"/>)</para>
          </listitem>
          <listitem>
            <para>addBaremetalDhcp (Adds a DHCP server for bare metal hosts)</para>
          </listitem>
          <listitem>
            <para> addBaremetalPxePingServer (Adds a PXE PING server for bare metal hosts)</para>
          </listitem>
          <listitem>
            <para> addBaremetalPxeKickStartServer (Adds a PXE server for bare metal hosts)</para>
          </listitem>
          <listitem>
            <para> listBaremetalDhcp (Shows the DHCP servers currently defined for bare metal
              hosts)</para>
          </listitem>
          <listitem>
            <para> listBaremetalPxePingServer (Shows the PXE PING servers currently defined for bare
              metal hosts)</para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>NIC</title>
        <itemizedlist>
          <listitem>
            <para>addNicToVirtualMachine (Adds a new NIC to the specified VM on a selected network;
              see <xref linkend="multiple-ip-nic"/>)</para>
          </listitem>
          <listitem>
            <para>removeNicFromVirtualMachine (Removes the specified NIC from a selected VM.)</para>
          </listitem>
          <listitem>
            <para>updateDefaultNicForVirtualMachine (Updates the specified NIC to be the default one
              for a selected VM.)</para>
          </listitem>
          <listitem>
            <para>addIpToNic (Assigns secondary IP to a NIC.) </para>
          </listitem>
          <listitem>
            <para>removeIpFromNic (Assigns secondary IP to a NIC.) </para>
          </listitem>
          <listitem>
            <para>listNics (Lists the NICs associated with a VM.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Regions</title>
        <itemizedlist>
          <listitem>
            <para>addRegion (Registers a Region into another Region; see <xref linkend="regions"
              />)</para>
          </listitem>
          <listitem>
            <para>updateRegion (Updates Region details: ID, Name, Endpoint, User API Key, and User
              Secret Key.)</para>
          </listitem>
          <listitem>
            <para>removeRegion (Removes a Region from current Region.)</para>
          </listitem>
          <listitem>
            <para>listRegions (Get all the Regions. They can be filtered by using the ID or
              Name.)</para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>User</title>
        <itemizedlist>
          <listitem>
            <para>getUser (This API can only be used by the Admin. Get user account details by using
              the API Key.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>API Throttling</title>
        <itemizedlist>
          <listitem>
            <para>getApiLimit (Show number of remaining APIs for the invoking user in current
              window)</para>
          </listitem>
          <listitem>
            <para>resetApiLimit (For root admin, if accountId parameter is passed, it will reset
              count for that particular account, otherwise it will reset all counters)</para>
          </listitem>
          <listitem>
            <para>resetApiLimit (Reset the API count.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Locking</title>
        <itemizedlist>
          <listitem>
            <para>lockAccount (Locks an account)</para>
          </listitem>
          <listitem>
            <para>lockUser (Locks a user account)</para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>VM Scaling</title>
        <itemizedlist>
          <listitem>
            <para>scaleVirtualMachine (Scales the virtual machine to a new service offering.)</para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Migrate Volume</title>
        <itemizedlist>
          <listitem>
            <para>migrateVirtualMachineWithVolume (Attempts migrating VM with its volumes to a
              different host.) </para>
          </listitem>
          <listitem>
            <para>listStorageProviders (Lists storage providers.) </para>
          </listitem>
          <listitem>
            <para>findStoragePoolsForMigration (Lists storage pools available for migrating a
              volume.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Dedicated IP and VLAN</title>
        <itemizedlist>
          <listitem>
            <para>dedicatePublicIpRange (Dedicates a Public IP range to an account.) </para>
          </listitem>
          <listitem>
            <para>releasePublicIpRange (Releases a Public IP range back to the system pool.) </para>
          </listitem>
          <listitem>
            <para>dedicateGuestVlanRange (Dedicates a guest VLAN range to an account.) </para>
          </listitem>
          <listitem>
            <para>releaseDedicatedGuestVlanRange (Releases a dedicated guest VLAN range to the
              system.) </para>
          </listitem>
          <listitem>
            <para>listDedicatedGuestVlanRanges (Lists dedicated guest VLAN ranges.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Port Forwarding</title>
        <itemizedlist>
          <listitem>
            <para>updatePortForwardingRule (Updates a port forwarding rule. Only the private port
              and the VM can be updated.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Scale System VM</title>
        <itemizedlist>
          <listitem>
            <para>scaleSystemVm (Scale the service offering for a systemVM, console proxy, or
              secondary storage.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Deployment Planner</title>
        <itemizedlist>
          <listitem>
            <para>listDeploymentPlanners (Lists all the deployment planners available.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Archive and Delete Events and Alerts</title>
        <itemizedlist>
          <listitem>
            <para>archiveEvents (Archive one or more events.) </para>
          </listitem>
          <listitem>
            <para>deleteEvents (Delete one or more events.) </para>
          </listitem>
          <listitem>
            <para>archiveAlerts (Archive one or more alerts.) </para>
          </listitem>
          <listitem>
            <para>deleteAlerts (Delete one or more alerts.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Host Reservation</title>
        <itemizedlist>
          <listitem>
            <para>releaseHostReservation (Releases host reservation.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Resize Volume</title>
        <itemizedlist>
          <listitem>
            <para>resizeVolume (Resizes a volume.) </para>
          </listitem>
          <listitem>
            <para>updateVolume (Updates the volume.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Egress Firewall Rules</title>
        <itemizedlist>
          <listitem>
            <para>createEgressFirewallRule (Creates a egress firewall rule for a given network. )
            </para>
          </listitem>
          <listitem>
            <para>deleteEgressFirewallRule (Deletes an egress firewall rule.) </para>
          </listitem>
          <listitem>
            <para>listEgressFirewallRules (Lists all egress firewall rules for network.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Network ACL</title>
        <itemizedlist>
          <listitem>
            <para>updateNetworkACLItem (Updates ACL item with specified ID.) </para>
          </listitem>
          <listitem>
            <para>createNetworkACLList (Creates a Network ACL for the given VPC.) </para>
          </listitem>
          <listitem>
            <para>deleteNetworkACLList (Deletes a Network ACL.) </para>
          </listitem>
          <listitem>
            <para>replaceNetworkACLList (Replaces ACL associated with a Network or private gateway.)
            </para>
          </listitem>
          <listitem>
            <para>listNetworkACLLists (Lists all network ACLs.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Resource Detail</title>
        <itemizedlist>
          <listitem>
            <para>addResourceDetail (Adds detail for the Resource.) </para>
          </listitem>
          <listitem>
            <para>removeResourceDetail (Removes detail for the Resource.) </para>
          </listitem>
          <listitem>
            <para>listResourceDetails (List resource details.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Nicira Integration</title>
        <itemizedlist>
          <listitem>
            <para>addNiciraNvpDevice (Adds a Nicira NVP device.) </para>
          </listitem>
          <listitem>
            <para>deleteNiciraNvpDevice (Deletes a Nicira NVP device.) </para>
          </listitem>
          <listitem>
            <para>listNiciraNvpDevices (Lists Nicira NVP devices.) </para>
          </listitem>
          <listitem>
            <para>listNiciraNvpDeviceNetworks (Lists network that are using a Nicira NVP device.)
            </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>BigSwitch VNS</title>
        <itemizedlist>
          <listitem>
            <para>addBigSwitchVnsDevice (Adds a BigSwitch VNS device.) </para>
          </listitem>
          <listitem>
            <para>deleteBigSwitchVnsDevice (Deletes a BigSwitch VNS device.) </para>
          </listitem>
          <listitem>
            <para>listBigSwitchVnsDevices (Lists BigSwitch VNS devices.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Simulator</title>
        <itemizedlist>
          <listitem>
            <para>configureSimulator (Configures a simulator.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>API Discovery</title>
        <itemizedlist>
          <listitem>
            <para>listApis (Lists all the available APIs on the server, provided by the API
              Discovery plugin.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Global Load Balancer</title>
        <itemizedlist>
          <listitem>
            <para>createGlobalLoadBalancerRule (Creates a global load balancer rule.) </para>
          </listitem>
          <listitem>
            <para>deleteGlobalLoadBalancerRule (Deletes a global load balancer rule.) </para>
          </listitem>
          <listitem>
            <para>updateGlobalLoadBalancerRule (update global load balancer rules.) </para>
          </listitem>
          <listitem>
            <para>listGlobalLoadBalancerRules (Lists load balancer rules.) </para>
          </listitem>
          <listitem>
            <para>assignToGlobalLoadBalancerRule (Assign load balancer rule or list of load balancer
              rules to a global load balancer rules.) </para>
          </listitem>
          <listitem>
            <para>removeFromGlobalLoadBalancerRule (Removes a load balancer rule association with
              global load balancer rule) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Load Balancer</title>
        <itemizedlist>
          <listitem>
            <para>createLoadBalancer (Creates a Load Balancer) </para>
          </listitem>
          <listitem>
            <para>listLoadBalancers (Lists Load Balancers) </para>
          </listitem>
          <listitem>
            <para>deleteLoadBalancer (Deletes a load balancer) </para>
          </listitem>
          <listitem>
            <para>configureInternalLoadBalancerElement (Configures an Internal Load Balancer
              element.) </para>
          </listitem>
          <listitem>
            <para>createInternalLoadBalancerElement (Create an Internal Load Balancer element.)
            </para>
          </listitem>
          <listitem>
            <para>listInternalLoadBalancerElements (Lists all available Internal Load Balancer
              elements.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Affinity Group</title>
        <itemizedlist>
          <listitem>
            <para>createAffinityGroup (Creates an affinity or anti-affinity group.) </para>
          </listitem>
          <listitem>
            <para>deleteAffinityGroup (Deletes an affinity group.) </para>
          </listitem>
          <listitem>
            <para>listAffinityGroups (Lists all the affinity groups.)</para>
          </listitem>
          <listitem>
            <para>updateVMAffinityGroup (Updates the affinity or anti-affinity group associations of
              a VM. The VM has to be stopped and restarted for the new properties to take effect.)
            </para>
          </listitem>
          <listitem>
            <para>listAffinityGroupTypes (Lists affinity group types available.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Portable IP</title>
        <itemizedlist>
          <listitem>
            <para>createPortableIpRange (Adds a range of portable portable IPs to a Region.)</para>
          </listitem>
          <listitem>
            <para>deletePortableIpRange (Deletes a range of portable portable IPs associated with a
              Region.) </para>
          </listitem>
          <listitem>
            <para>listPortableIpRanges (Lists portable IP ranges.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Internal Load Balancer VM</title>
        <itemizedlist>
          <listitem>
            <para>stopInternalLoadBalancerVM (Stops an Internal LB VM.) </para>
          </listitem>
          <listitem>
            <para>startInternalLoadBalancerVM (Starts an existing Internal LB VM.) </para>
          </listitem>
          <listitem>
            <para>listInternalLoadBalancerVMs (List internal LB VMs.) </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Network Isolation</title>
        <itemizedlist>
          <listitem>
            <para>listNetworkIsolationMethods (Lists supported methods of network isolation.)
            </para>
          </listitem>
        </itemizedlist>
      </section>
      <section>
        <title>Dedicated Resources</title>
        <itemizedlist>
          <listitem>
            <para>dedicateZone (Dedicates a zone.) </para>
          </listitem>
          <listitem>
            <para>dedicatePod (Dedicates a pod.) </para>
          </listitem>
          <listitem>
            <para>dedicateCluster (Dedicate an existing cluster.) </para>
          </listitem>
          <listitem>
            <para>dedicateHost (Dedicates a host.) </para>
          </listitem>
          <listitem>
            <para>releaseDedicatedZone (Release dedication of zone.) </para>
          </listitem>
          <listitem>
            <para>releaseDedicatedPod (Release dedication for the pod.) </para>
          </listitem>
          <listitem>
            <para>releaseDedicatedCluster (Release dedication for cluster.) </para>
          </listitem>
          <listitem>
            <para>releaseDedicatedHost (Release dedication for host.) </para>
          </listitem>
          <listitem>
            <para>listDedicatedZones (List dedicated zones.) </para>
          </listitem>
          <listitem>
            <para>listDedicatedPods (Lists dedicated pods.) </para>
          </listitem>
          <listitem>
            <para>listDedicatedClusters (Lists dedicated clusters.) </para>
          </listitem>
          <listitem>
            <para>listDedicatedHosts (Lists dedicated hosts.) </para>
          </listitem>
        </itemizedlist>
      </section>
    </section>
    <section id="changed-api-commands-4.2">
      <title>Changed API Commands in 4.2</title>
      <informaltable>
        <tgroup cols="2" align="left" colsep="1" rowsep="1">
          <colspec colwidth="1.0*" colname="1" colnum="1"/>
          <colspec colwidth="4.26*" colname="2" colnum="2"/>
          <thead>
            <row>
              <entry>
                <para>API Commands</para>
              </entry>
              <entry>
                <para>Description</para>
              </entry>
            </row>
          </thead>
          <tbody>
            <row>
              <entry>
                <para>listNetworkACLs</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: aclid (optional), action
                  (optional), protocol (optional)</para>
                <para>The following new response parameters are added: aclid, action, number</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>copyTemplate</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: isdynamicallyscalable,
                  sshkeyenabled </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listRouters</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: ip6dns1, ip6dns2, role</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateConfiguration</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: accountid (optional),
                  clusterid (optional), storageid (optional), zoneid (optional)</para>
                <para>The following new response parameters are added: id, scope </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listVolumes</para>
              </entry>
              <entry>
                <para>The following request parameter is removed: details </para>
                <para>The following new response parameter is added: displayvolume </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>suspendProject</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: cpuavailable, cpulimit,
                  cputotal, ipavailable, iplimit, iptotal, memoryavailable, memorylimit,
                  memorytotal, networkavailable, networklimit, networktotal,
                  primarystorageavailable, primarystoragelimit, primarystoragetotal,
                  secondarystorageavailable, secondarystoragelimit, secondarystoragetotal,
                  snapshotavailable, snapshotlimit, snapshottotal, templateavailable, templatelimit,
                  templatetotal, vmavailable, vmlimit, vmrunning, vmstopped, vmtotal,
                  volumeavailable, volumelimit, volumetotal, vpcavailable, vpclimit, vpctotal
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listRemoteAccessVpns</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: id</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>registerTemplate</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: imagestoreuuid (optional),
                  isdynamicallyscalable (optional), isrouting (optional) </para>
                <para>The following new response parameters are added: isdynamicallyscalable,
                  sshkeyenabled </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>addTrafficMonitor</para>
              </entry>
              <entry>
                <para>The following response parameters are removed: privateinterface, privatezone,
                  publicinterface, publiczone, usageinterface, username </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createTemplate</para>
              </entry>
              <entry>
                <para>The following response parameters are removed: clusterid, clustername,
                  disksizeallocated, disksizetotal, disksizeused, ipaddress, path, podid, podname,
                  state, tags, type </para>
                <para>The following new response parameters are added: account, accountid, bootable,
                  checksum, crossZones, details, displaytext, domain, domainid, format, hostid,
                  hostname, hypervisor, isdynamicallyscalable, isextractable, isfeatured, ispublic,
                  isready, ostypeid, ostypename, passwordenabled, project, projectid, removed, size,
                  sourcetemplateid, sshkeyenabled, status, templatetag, templatetype, tags</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listLoadBalancerRuleInstances</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>migrateVolume</para>
              </entry>
              <entry>
                <para>The following new request parameters is added: livemigrate (optional) </para>
                <para>The following new response parameters is added: displayvolume </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createAccount</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: accountid (optional), userid
                  (optional)</para>
                <para>The following new response parameters are added: accountdetails, cpuavailable,
                  cpulimit, cputotal, defaultzoneid, ipavailable, iplimit, iptotal,
                  iscleanuprequired, isdefault, memoryavailable, memorylimit, memorytotal, name,
                  networkavailable, networkdomain, networklimit, networktotal,
                  primarystorageavailable, primarystoragelimit, primarystoragetotal,
                  projectavailable, projectlimit, projecttotal, receivedbytes,
                  secondarystorageavailable, secondarystoragelimit, secondarystoragetotal,
                  sentbytes, snapshotavailable, snapshotlimit, snapshottotal, templateavailable,
                  templatelimit, templatetotal, vmavailable, vmlimit, vmrunning, vmstopped, vmtotal,
                  volumeavailable, volumelimit, volumetotal, vpcavailable, vpclimit, vpctotal,
                  user</para>
                <para>The following parameters are removed: account, accountid, apikey, created,
                  email, firstname, lastname, secretkey, timezone, username </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updatePhysicalNetwork</para>
              </entry>
              <entry>
                <para>The following new request parameters is added: removevlan (optional) </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listTrafficMonitors</para>
              </entry>
              <entry>
                <para>The following response parameters are removed: privateinterface, privatezone,
                  publicinterface, publiczone, usageinterface, username </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>attachIso</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listProjects</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: cpuavailable, cpulimit,
                  cputotal, ipavailable, iplimit, iptotal, memoryavailable, memorylimit,
                  memorytotal, networkavailable, networklimit, networktotal,
                  primarystorageavailable, primarystoragelimit, primarystoragetotal,
                  secondarystorageavailable, secondarystoragelimit, secondarystoragetotal,
                  snapshotavailable, snapshotlimit, snapshottotal, templateavailable, templatelimit,
                  templatetotal, vmavailable, vmlimit, vmrunning, vmstopped, vmtotal,
                  volumeavailable, volumelimit, volumetotal, vpcavailable, vpclimit, vpctotal
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>enableAccount</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: cpuavailable, cpulimit,
                  cputotal, isdefault, memoryavailable, memorylimit, memorytotal,
                  primarystorageavailable, primarystoragelimit, primarystoragetotal,
                  secondarystorageavailable, secondarystoragelimit, secondarystoragetotal </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listPublicIpAddresses</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: isportable, vmipaddress
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>enableStorageMaintenance</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: hypervisor, scope,
                  suitableformigration </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listLoadBalancerRules</para>
              </entry>
              <entry>
                <para>The following new request parameters is added: networkid (optional) </para>
                <para>The following new response parameters is added: networkid</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>stopRouter</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: ip6dns1, ip6dns2, role
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listClusters</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: cpuovercommitratio,
                  memoryovercommitratio </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>attachVolume</para>
              </entry>
              <entry>
                <para>The following new response parameter is added: displayvolume</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateVPCOffering</para>
              </entry>
              <entry>
                <para>The following request parameters is made mandatory: id</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>resetSSHKeyForVirtualMachine</para>
              </entry>
              <entry>
                <para>The following new request parameter is added: keypair (required)</para>
                <para>The following parameter is removed: name</para>
                <para>The following new response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateCluster</para>
              </entry>
              <entry>
                <para>The following request parameters are removed: cpuovercommitratio,
                  memoryovercommitratio (optional) </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listPrivateGateways</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: aclid, sourcenatsupported
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>ldapConfig</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: listall (optional) </para>
                <para>The following parameters has been made optional: searchbase, hostname,
                  queryfilter</para>
                <para>The following new response parameter is added: ssl</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listTemplates</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: isdynamicallyscalable,
                  sshkeyenabled </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listNetworks</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: aclid, displaynetwork,
                  ip6cidr, ip6gateway, ispersistent, networkcidr, reservediprange </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>restartNetwork</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: isportable, vmipaddress
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>prepareTemplate</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: isdynamicallyscalable,
                  sshkeyenabled </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>rebootVirtualMachine</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>changeServiceForRouter</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: aclid (optional), action
                  (optional), protocol (optional)</para>
                <para>The following new response parameters are added: id, scope</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateZone</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: ip6dns1 (optional), ip6dns2
                  (optional) </para>
                <para>The following new response parameters are added: ip6dns1, ip6dns2 </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>ldapRemove</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: ssl</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateServiceOffering</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: deploymentplanner, isvolatile
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateStoragePool</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: hypervisor, scope,
                  suitableformigration</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listFirewallRules</para>
              </entry>
              <entry>
                <para>The following request parameter is removed: traffictype</para>
                <para>The following new response parameters are added: networkid</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateUser</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: iscallerchilddomain,
                  isdefault </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateProject</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: cpuavailable, cpulimit,
                  cputotal, ipavailable, iplimit, iptotal, memoryavailable, memorylimit,
                  memorytotal, networkavailable, networklimit, networktotal,
                  primarystorageavailable, primarystoragelimit, primarystoragetotal,
                  secondarystorageavailable, secondarystoragelimit, secondarystoragetotal,
                  snapshotavailable, snapshotlimit, snapshottotal, templateavailable, templatelimit,
                  templatetotal, vmavailable, vmlimit, vmrunning, vmstopped, vmtotal,
                  volumeavailable, volumelimit, volumetotal, vpcavailable, vpclimit, vpctotal
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateTemplate</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: isdynamicallyscalable
                  (optional), isrouting (optional) </para>
                <para>The following new response parameters are added: isdynamicallyscalable,
                  sshkeyenabled </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>disableUser</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: iscallerchilddomain,
                  isdefault </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>activateProject</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: cpuavailable, cpulimit,
                  cputotal, ipavailable, iplimit, iptotal, memoryavailable, memorylimit,
                  memorytotal, networkavailable, networklimit, networktotal,
                  primarystorageavailable, primarystoragelimit, primarystoragetotal,
                  secondarystorageavailable, secondarystoragelimit, secondarystoragetotal,
                  snapshotavailable, snapshotlimit, snapshottotal, templateavailable, templatelimit,
                  templatetotal, vmavailable, vmlimit, vmrunning, vmstopped, vmtotal,
                  volumeavailable, volumelimit, volumetotal, vpcavailable, vpclimit, vpctotal
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createNetworkACL</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: aclid (optional), action
                  (optional), number (optional) </para>
                <para>The following request parameter is now optional: networkid</para>
                <para>The following new response parameters are added: aclid, action, number </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>enableStaticNat</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: vmguestip (optional) </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>registerIso</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: imagestoreuuid (optional),
                  isdynamicallyscalable (optional)</para>
                <para>The following new response parameters are added: isdynamicallyscalable,
                  sshkeyenabled </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createIpForwardingRule</para>
              </entry>
              <entry>
                <para>The following new response parameter is added: vmguestip </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>resetPasswordForVirtualMachine</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createVolume</para>
              </entry>
              <entry>
                <para>The following new request parameter is added: displayvolume (optional)</para>
                <para>The following new response parameter is added: displayvolume</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>startRouter</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: ip6dns1, ip6dns2, role
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listCapabilities</para>
              </entry>
              <entry>
                <para>The following new response parameters are added: apilimitinterval and
                  apilimitmax.</para>
                <para>See <xref linkend="api-request-throttling"/>.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createServiceOffering</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: deploymentplanner (optional),
                  isvolatile (optional), serviceofferingdetails (optional). </para>
                <para>isvolatie indicates whether the service offering includes Volatile VM
                  capability, which will discard the VM's root disk and create a new one on reboot.
                  See <xref linkend="reset-vm-reboot"/>.</para>
                <para>The following new response parameters are added: deploymentplanner, isvolatile
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>restoreVirtualMachine</para>
              </entry>
              <entry>
                <para>The following request parameter is added: templateID (optional). This is used
                  to point to the new template ID when the base image is updated. The parameter
                  templateID can be an ISO ID in case of restore vm deployed using ISO. See <xref
                    linkend="update-vm-image"/>.</para>
                <para>The following response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createNetwork</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: aclid (optional),
                  displaynetwork (optional), endipv6 (optional), ip6cidr (optional), ip6gateway
                  (optional), isolatedpvlan (optional), startipv6 (optional)</para>
                <para>The following new response parameters are added: aclid, displaynetwork,
                  ip6cidr, ip6gateway, ispersistent, networkcidr, reservediprange </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createVlanIpRange</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: startipv6, endipv6,
                  ip6gateway, ip6cidr</para>
                <para>Changed parameters: startip (is now optional)</para>
                <para>The following new response parameters are added: startipv6, endipv6,
                  ip6gateway, ip6cidr</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CreateZone</para>
              </entry>
              <entry>
                <para>The following new request parameters are added: ip6dns1, ip6dns2</para>
                <para>The following new response parameters are added: ip6dns1, ip6dns2 </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>deployVirtualMachine</para>
              </entry>
              <entry>
                <para>The following request parameters are added: affinitygroupids (optional),
                  affinitygroupnames (optional), displayvm (optional), ip6address (optional) </para>
                <para>The following request parameter is modified: iptonetworklist has a new
                  possible value, ipv6</para>
                <para>The following new response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createNetworkOffering</para>
              </entry>
              <entry>
                <para>The following request parameters are added: details (optional),
                  egressdefaultpolicy (optional), ispersistent (optional) </para>
                <para>ispersistent determines if the network or network offering created or listed
                  by using this offering are persistent or not.</para>
                <para>The following response parameters are added: details, egressdefaultpolicy,
                  ispersistent </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listNetworks</para>
              </entry>
              <entry>
                <para>The following request parameters is added: isPersistent.</para>
                <para>This parameter determines if the network or network offering created or listed
                  by using this offering are persistent or not.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listNetworkOfferings</para>
              </entry>
              <entry>
                <para>The following request parameters is added: isPersistent.</para>
                <para>This parameter determines if the network or network offering created or listed
                  by using this offering are persistent or not.</para>
                <para>For listNetworkOfferings, the following response parameter has been added:
                  details, egressdefaultpolicy, ispersistent </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>addF5LoadBalancer</para>
                <para>configureNetscalerLoadBalancer</para>
                <para>addNetscalerLoadBalancer</para>
                <para>listF5LoadBalancers</para>
                <para>configureF5LoadBalancer</para>
                <para>listNetscalerLoadBalancers</para>
              </entry>
              <entry>
                <para>The following response parameter is removed: inline.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listRouters</para>
              </entry>
              <entry>
                <para>For nic responses, the following fields have been added.</para>
                <itemizedlist>
                  <listitem>
                    <para>ip6address</para>
                  </listitem>
                  <listitem>
                    <para>ip6gateway</para>
                  </listitem>
                  <listitem>
                    <para>ip6cidr</para>
                  </listitem>
                </itemizedlist>
              </entry>
            </row>
            <row>
              <entry>
                <para>listVirtualMachines</para>
              </entry>
              <entry>
                <para>The following request parameters are added: affinitygroupid (optional), vpcid
                  (optional) </para>
                <para>The following response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listRouters</para>
                <para>listZones</para>
              </entry>
              <entry>
                <para>For DomainRouter and DataCenter response, the following fields have been
                  added.</para>
                <itemizedlist>
                  <listitem>
                    <para>ip6dns1</para>
                  </listitem>
                  <listitem>
                    <para>ip6dns2</para>
                  </listitem>
                </itemizedlist>
                <para>For listZones, the following optional request parameters are added: name,
                  networktype</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listFirewallRules</para>
                <para>createFirewallRule</para>
              </entry>
              <entry>
                <para>The following request parameter is added: traffictype (optional).</para>
                <para>The following response parameter is added: networkid </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listUsageRecords</para>
              </entry>
              <entry>
                <para>The following response parameter is added: virtualsize.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>deleteIso</para>
              </entry>
              <entry>
                <para>The following request parameter is removed: forced</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>addCluster</para>
              </entry>
              <entry>
                <para>The following request parameters are added: guestvswitchtype (optional),
                  guestvswitchtype (optional), publicvswitchtype (optional), publicvswitchtype
                  (optional)</para>
                <para>See <xref linkend="cpu-ram-overcommit"/>.</para>
                <para>The following request parameters are removed: cpuovercommitratio,
                  memoryovercommitratio </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateCluster</para>
              </entry>
              <entry>
                <para>The following request parameters are added: cpuovercommitratio,
                  ramovercommitratio</para>
                <para>See <xref linkend="cpu-ram-overcommit"/>.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createStoragePool</para>
              </entry>
              <entry>
                <para>The following request parameters are added: hypervisor (optional), provider
                  (optional), scope (optional) </para>
                <para>The following request parameters have been made mandatory: podid,
                  clusterid</para>
                <para>See <xref linkend="zone-wide-primary-storage"/>.</para>
                <para>The following response parameter has been added: hypervisor, scope,
                  suitableformigration </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listStoragePools</para>
              </entry>
              <entry>
                <para>The following request parameter is added: scope (optional)</para>
                <para>See <xref linkend="zone-wide-primary-storage"/>.</para>
                <para>The following response parameters are added: hypervisor, scope,
                  suitableformigration </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateDiskOffering</para>
              </entry>
              <entry>
                <para>The following response parameter is added: displayoffering </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>changeServiceForVirtualMachine</para>
              </entry>
              <entry>
                <para>The following response parameter are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>recoverVirtualMachine</para>
              </entry>
              <entry>
                <para>The following response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listCapabilities</para>
              </entry>
              <entry>
                <para>The following response parameters are added: apilimitinterval, apilimitmax
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createRemoteAccessVpn</para>
              </entry>
              <entry>
                <para>The following response parameters are added: id</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>startVirtualMachine</para>
              </entry>
              <entry>
                <para>The following response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>detachIso</para>
              </entry>
              <entry>
                <para>The following response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateVPC</para>
              </entry>
              <entry>
                <para>The following request parameters has been made mandatory: id, name</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>associateIpAddress</para>
              </entry>
              <entry>
                <para>The following request parameters are added: isportable (optional), regionid
                  (optional) </para>
                <para>The following response parameters are added: isportable, vmipaddress </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listProjectAccounts</para>
              </entry>
              <entry>
                <para>The following response parameters are added: cpuavailable, cpulimit, cputotal,
                  ipavailable, iplimit, iptotal, memoryavailable, memorylimit, memorytotal,
                  networkavailable, networklimit, networktotal, primarystorageavailable,
                  primarystoragelimit, primarystoragetotal, secondarystorageavailable,
                  secondarystoragelimit, secondarystoragetotal, snapshotavailable, snapshotlimit,
                  snapshottotal, templateavailable, templatelimit, templatetotal, vmavailable,
                  vmlimit, vmrunning, vmstopped, vmtotal, volumeavailable, volumelimit, volumetotal,
                  vpcavailable, vpclimit, vpctotal </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>disableAccount</para>
              </entry>
              <entry>
                <para>The following response parameters are added: cpuavailable, cpulimit, cputotal,
                  isdefault, memoryavailable, memorylimit, memorytotal, primarystorageavailable,
                  primarystoragelimit, primarystoragetotal, secondarystorageavailable,
                  secondarystoragelimit, secondarystoragetotal </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listPortForwardingRules</para>
              </entry>
              <entry>
                <para>The following response parameters are added: vmguestip</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>migrateVirtualMachine</para>
              </entry>
              <entry>
                <para>The following response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>cancelStorageMaintenance</para>
              </entry>
              <entry>
                <para>The following response parameters are added: hypervisor, scope,
                  suitableformigration </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createPortForwardingRule</para>
              </entry>
              <entry><para>The following request parameter is added: vmguestip (optional) </para>The
                following response parameter is added: vmguestip</entry>
            </row>
            <row>
              <entry>
                <para>addVpnUser</para>
              </entry>
              <entry>
                <para>The following response parameter is added: state</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createVPCOffering</para>
              </entry>
              <entry>
                <para>The following request parameter is added: serviceproviderlist (optional)
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>assignVirtualMachine</para>
              </entry>
              <entry>
                <para>The following response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listConditions</para>
              </entry>
              <entry>
                <para>The following response parameters are added: account, counter, domain,
                  domainid, project, projectid, relationaloperator, threshold</para>
                <para>Removed response parameters: name, source, value </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createPrivateGateway</para>
              </entry>
              <entry>
                <para>The following request parameters are added: aclid (optional),
                  sourcenatsupported (optional) </para>
                <para>The following response parameters are added: aclid, sourcenatsupported </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateVirtualMachine</para>
              </entry>
              <entry>
                <para>The following request parameters are added: displayvm (optional),
                  isdynamicallyscalable (optional) </para>
                <para>The following response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>destroyRouter</para>
              </entry>
              <entry>
                <para>The following response parameters are added: ip6dns1, ip6dns2, role </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listServiceOfferings</para>
              </entry>
              <entry>
                <para>The following response parameters are added: deploymentplanner, isvolatile
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listUsageRecords</para>
              </entry>
              <entry>
                <para>The following response parameters are removed: virtualsize </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createProject</para>
              </entry>
              <entry>
                <para>The following response parameters are added: cpuavailable, cpulimit, cputotal,
                  ipavailable, iplimit, iptotal, memoryavailable, memorylimit, memorytotal,
                  networkavailable, networklimit, networktotal, primarystorageavailable,
                  primarystoragelimit, primarystoragetotal, secondarystorageavailable,
                  secondarystoragelimit, secondarystoragetotal, snapshotavailable, snapshotlimit,
                  snapshottotal, templateavailable, templatelimit, templatetotal, vmavailable,
                  vmlimit, vmrunning, vmstopped, vmtotal, volumeavailable, volumelimit, volumetotal,
                  vpcavailable, vpclimit, vpctotal </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>enableUser</para>
              </entry>
              <entry>
                <para>The following response parameters are added: iscallerchilddomain, isdefault
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createLoadBalancerRule</para>
              </entry>
              <entry>
                <para>The following response parameter is added: networkid </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateAccount</para>
              </entry>
              <entry>
                <para>The following response parameters are added: cpuavailable, cpulimit, cputotal,
                  isdefault, memoryavailable, memorylimit, memorytotal, primarystorageavailable,
                  primarystoragelimit, primarystoragetotal, secondarystorageavailable,
                  secondarystoragelimit, secondarystoragetotal </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>copyIso</para>
              </entry>
              <entry>
                <para>The following response parameters are added: isdynamicallyscalable,
                  sshkeyenabled </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>uploadVolume</para>
              </entry>
              <entry>
                <para>The following request parameters are added: imagestoreuuid (optional),
                  projectid (optional</para>
                <para>The following response parameters are added: displayvolume </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createDomain</para>
              </entry>
              <entry>
                <para>The following request parameter is added: domainid (optional) </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>stopVirtualMachine</para>
              </entry>
              <entry>
                <para>The following response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listAccounts</para>
              </entry>
              <entry>
                <para>The following response parameters are added: cpuavailable, cpulimit, cputotal,
                  isdefault, memoryavailable, memorylimit, memorytotal, primarystorageavailable,
                  primarystoragelimit, primarystoragetotal, secondarystorageavailable,
                  secondarystoragelimit, secondarystoragetotal </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createSnapshot</para>
              </entry>
              <entry>
                <para>The following response parameter is added: zoneid </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateIso</para>
              </entry>
              <entry>
                <para>The following request parameters are added: isdynamicallyscalable (optional),
                  isrouting (optional) </para>
                <para>The following response parameters are added: isdynamicallyscalable,
                  sshkeyenabled</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listIpForwardingRules</para>
              </entry>
              <entry>
                <para>The following response parameter is added: vmguestip </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateNetwork</para>
              </entry>
              <entry>
                <para>The following request parameters are added: displaynetwork (optional),
                  guestvmcidr (optional) </para>
                <para>The following response parameters are added: aclid, displaynetwork, ip6cidr,
                  ip6gateway, ispersistent, networkcidr, reservediprange </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>destroyVirtualMachine</para>
              </entry>
              <entry>
                <para>The following response parameters are added: diskioread, diskiowrite,
                  diskkbsread, diskkbswrite, displayvm, isdynamicallyscalable, affinitygroup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createDiskOffering</para>
              </entry>
              <entry>
                <para>The following request parameter is added: displayoffering (optional)</para>
                <para>The following response parameter is added: displayoffering </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>rebootRouter</para>
              </entry>
              <entry>
                <para>The following response parameters are added: ip6dns1, ip6dns2, role </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listConfigurations</para>
              </entry>
              <entry>
                <para>The following request parameters are added: accountid (optional), clusterid
                  (optional), storageid (optional), zoneid (optional) </para>
                <para>The following response parameters are added: id, scope</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createUser</para>
              </entry>
              <entry>
                <para>The following request parameter is added: userid (optional)</para>
                <para>The following response parameters are added: iscallerchilddomain,
                  isdefault</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listDiskOfferings</para>
              </entry>
              <entry>
                <para>The following response parameter is added: displayoffering</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>detachVolume</para>
              </entry>
              <entry>
                <para>The following response parameter is added: displayvolume </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>deleteUser</para>
              </entry>
              <entry>
                <para>The following response parameters are added: displaytext, success</para>
                <para>Removed parameters: id, account, accountid, accounttype, apikey, created,
                  domain, domainid, email, firstname, lastname, secretkey, state, timezone, username
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listSnapshots</para>
              </entry>
              <entry>
                <para>The following request parameter is added: zoneid (optional) </para>
                <para>The following response parameter is added: zoneid</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>markDefaultZoneForAccount</para>
              </entry>
              <entry>
                <para>The following response parameters are added: cpuavailable, cpulimit, cputotal,
                  isdefault, memoryavailable, memorylimit, memorytotal, primarystorageavailable,
                  primarystoragelimit, primarystoragetotal, secondarystorageavailable,
                  secondarystoragelimit, secondarystoragetotal </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>restartVPC</para>
              </entry>
              <entry>
                <para>The following request parameters are made mandatory: id</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateHypervisorCapabilities</para>
              </entry>
              <entry>
                <para>The following response parameters are added: hypervisor, hypervisorversion,
                  maxdatavolumeslimit, maxguestslimit, maxhostspercluster, securitygroupenabled,
                  storagemotionenabled </para>
                <para>Removed parameters: cpunumber, cpuspeed, created, defaultuse, displaytext,
                  domain, domainid, hosttags, issystem, limitcpuuse, memory, name, networkrate,
                  offerha, storagetype, systemvmtype, tags </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateLoadBalancerRule</para>
              </entry>
              <entry>
                <para>The following response parameter is added: networkid </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listVlanIpRanges</para>
              </entry>
              <entry>
                <para>The following response parameters are added: endipv6, ip6cidr, ip6gateway,
                  startipv6 </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listHypervisorCapabilities</para>
              </entry>
              <entry>
                <para>The following response parameters are added: maxdatavolumeslimit,
                  maxhostspercluster, storagemotionenabled </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>updateNetworkOffering</para>
              </entry>
              <entry>
                <para>The following response parameters are added: details, egressdefaultpolicy,
                  ispersistent </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>createVirtualRouterElement</para>
              </entry>
              <entry>
                <para>The following request parameters are added: providertype (optional)</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listVpnUsers</para>
              </entry>
              <entry>
                <para>The following response parameter is added: state</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listUsers</para>
              </entry>
              <entry>
                <para>The following response parameters are added: iscallerchilddomain, isdefault
                </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listSupportedNetworkServices</para>
              </entry>
              <entry>
                <para>The following response parameter is added: provider</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>listIsos</para>
              </entry>
              <entry>
                <para>The following response parameters are added: isdynamicallyscalable,
                  sshkeyenabled </para>
              </entry>
            </row>
          </tbody>
        </tgroup>
      </informaltable>
    </section>
    <section id="removed-api-4.2">
      <title>Deprecated APIs</title>
      <itemizedlist>
        <listitem>
          <para>addExternalLoadBalancer (Adds F5 external load balancer appliance.) </para>
        </listitem>
        <listitem>
          <para>deleteExternalLoadBalancer (Deletes a F5 external load balancer appliance added in a
            zone.) </para>
        </listitem>
        <listitem>
          <para>listExternalLoadBalancers (Lists F5 external load balancer appliances added in a
            zone.) </para>
        </listitem>
      </itemizedlist>
    </section>
  </chapter>
  <chapter id="version-4.1">
    <title>Version 4.1.0</title>
    <section id="what-new-in-4.1">
      <title>What’s New in 4.1</title>
      <para>Apache CloudStack 4.1.0 includes many new features. This section covers the most
        prominent new features and changes.</para>
      <section id="localization">
        <title>Localization</title>
        <para>The 4.1.0 release adds partial User Interface (UI) support for Catalan, Chinese,
          French, German, Italian, Japanese, Korean, Norwegian, Portuguese, Russian, and Spanish.
          Not all languages are complete.</para>
        <para>The 4.1.0 release also adds documentation translations for Chinese, Chinese (Taiwan),
          Italian, Japanese, Korean, and Portuguese.</para>
      </section>
      <section id="aws-style-regions">
        <title>Added Region Support</title>
        <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-241"
            >CLOUDSTACK-241</ulink>: This feature adds a "region" construct that spans several
          management servers. The objective of this feature is to add AWS EC2 like Regions
          implementation into CloudStack. Regions are dispersed and located in separate geographic
          areas. Availability Zones (or Zones in CloudStack) are distinct locations within a Region
          that are engineered to be isolated from failures in other Zones and provide inexpensive,
          low latency network connectivity to other Zones in the same Region.</para>
        <para>Regions are expected to add the following benefits</para>
        <itemizedlist>
          <listitem>
            <para>Higher availability of the services: users can deploy services across AZs and even
              if one of the AZ goes down the services are still available to the end-user through
              VMs deployed in other zones.</para>
          </listitem>
          <listitem>
            <para>Higher availability of the Management Server (MS): Since each MS Cluster only
              manages a single Region, if that MS Cluster goes down, only that particular Region is
              impacted. Admin should be able to access all the other Regions.</para>
          </listitem>
          <listitem>
            <para>Scalability: The scalability limit of CloudStack dramatically improves, as the
              scalability limit of MS Cluster is limited to a single Region.</para>
          </listitem>
          <listitem>
            <para>Object Store: With Regions construct, CloudStack would also allow users to define
              Object Store (Secondary Storage) across AZs. This helps users easily deploy VMs in
              different AZs using the same template, offerings.</para>
          </listitem>
          <listitem>
            <para>Geographical Grouping: Regions allow admins to group AZs (that have low latency
              and are geographically located nearby) into a broader region construct.</para>
          </listitem>
        </itemizedlist>
        <para>Currently the Region feature is exposed in the API, but does not have a UI
          component.</para>
      </section>
      <section id="ec2-query-api">
        <title>Support for EC2 Query API</title>
        <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-197"
            >CLOUDSTACK-197</ulink>: This introduces a query API for the AWS APIs that are currently
          only supported by SOAP. The AWS Java SDK and AWS PHP SDK should now be supported by the
          AWSAPI in CloudStack.</para>
        <para>Supported Query APIs in 4.1.0:</para>
        <itemizedlist>
          <listitem>
            <para><command>AllocateAddress</command></para>
          </listitem>
          <listitem>
            <para><command>AssociateAddress</command></para>
          </listitem>
          <listitem>
            <para><command>AttachVolume</command></para>
          </listitem>
          <listitem>
            <para><command>AuthorizeSecurityGroupIngress</command></para>
          </listitem>
          <listitem>
            <para><command>CreateImage</command></para>
          </listitem>
          <listitem>
            <para><command>CreateKeyPair</command></para>
          </listitem>
          <listitem>
            <para><command>CreateSecurityGroup</command></para>
          </listitem>
          <listitem>
            <para><command>CreateSnapshot</command></para>
          </listitem>
          <listitem>
            <para><command>CreateTags</command></para>
          </listitem>
          <listitem>
            <para><command>CreateVolume</command></para>
          </listitem>
          <listitem>
            <para><command>DeleteKeyPair</command></para>
          </listitem>
          <listitem>
            <para><command>DeleteSecurityGroup</command></para>
          </listitem>
          <listitem>
            <para><command>DeleteSnapshot</command></para>
          </listitem>
          <listitem>
            <para><command>DeleteTags</command></para>
          </listitem>
          <listitem>
            <para><command>DeleteVolume</command></para>
          </listitem>
          <listitem>
            <para><command>DeregisterImage</command></para>
          </listitem>
          <listitem>
            <para><command>DescribeAddresses</command></para>
          </listitem>
          <listitem>
            <para><command>DescribeAvailabilityZones</command></para>
          </listitem>
          <listitem>
            <para><command>DescribeImageAttribute</command></para>
          </listitem>
          <listitem>
            <para><command>DescribeImages</command></para>
          </listitem>
          <listitem>
            <para><command>DescribeInstanceAttribute</command></para>
          </listitem>
          <listitem>
            <para><command>DescribeInstances</command></para>
          </listitem>
          <listitem>
            <para><command>DescribeKeyPairs</command></para>
          </listitem>
          <listitem>
            <para><command>DescribeSecurityGroups</command></para>
          </listitem>
          <listitem>
            <para><command>DescribeSnapshots</command></para>
          </listitem>
          <listitem>
            <para><command>DescribeTags</command></para>
          </listitem>
          <listitem>
            <para><command>DescribeVolumes</command></para>
          </listitem>
          <listitem>
            <para><command>DetachVolume</command></para>
          </listitem>
          <listitem>
            <para><command>DisassociateAddress</command></para>
          </listitem>
          <listitem>
            <para><command>GetPasswordData</command></para>
          </listitem>
          <listitem>
            <para><command>ImportkeyPair</command></para>
          </listitem>
          <listitem>
            <para><command>ModifyImageAttribute</command></para>
          </listitem>
          <listitem>
            <para><command>RebootInstances</command></para>
          </listitem>
          <listitem>
            <para><command>RegisterImage</command></para>
          </listitem>
          <listitem>
            <para><command>ReleaseAddress</command></para>
          </listitem>
          <listitem>
            <para><command>ResetImageAttribute</command></para>
          </listitem>
          <listitem>
            <para><command>RevokeSecurityGroupIngress</command></para>
          </listitem>
          <listitem>
            <para><command>RunInstances</command></para>
          </listitem>
          <listitem>
            <para><command>StartInstances</command></para>
          </listitem>
          <listitem>
            <para><command>StopInstances</command></para>
          </listitem>
          <listitem>
            <para><command>TerminateInstances</command></para>
          </listitem>
        </itemizedlist>
        <para>See the <ulink
            url="https://cwiki.apache.org/CLOUDSTACK/ec2-functional-spec-for-query-api-support.html"
            >Feature Specification</ulink> for more information on the Query API support.</para>
      </section>
      <section id="cloudmonkey">
        <title>Auto-Completing Shell for CloudStack (CloudMonkey)</title>
        <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-132"
            >CLOUDSTACK-132</ulink>: Adds a auto-completing shell and command-line tool for
          &PRODUCT; written in Python, called <application>CloudMonkey</application>.</para>
        <para>CloudMonkey includes the following features:</para>
        <itemizedlist>
          <listitem>
            <para>Usable as a command line tool and interactive shell.</para>
          </listitem>
          <listitem>
            <para>All commands are lowercase unlike API.</para>
          </listitem>
          <listitem>
            <para>Api Discovery using sync feature, with build time api precaching for failsafe
              sync.</para>
          </listitem>
          <listitem>
            <para>Raw api execution support.</para>
          </listitem>
          <listitem>
            <para>Auto-completion via double <command>tab</command>.</para>
          </listitem>
          <listitem>
            <para>Reverse search using <command>Ctrl+R</command></para>
          </listitem>
          <listitem>
            <para>Emacs compatible key bindings.</para>
          </listitem>
          <listitem>
            <para>Output that's "pipeable" to other *nix programs.</para>
          </listitem>
          <listitem>
            <para>Unix shell execution.</para>
          </listitem>
          <listitem>
            <para>Support to handle asynchronous jobs using user defined blocking or non-blocking
              way.</para>
          </listitem>
          <listitem>
            <para>Tabular or JSON output with filtering of table columns.</para>
          </listitem>
          <listitem>
            <para>Colored output.</para>
          </listitem>
          <listitem>
            <para>API parameter value completion (based on predication, fuzzy results may fail
              sometimes).</para>
          </listitem>
        </itemizedlist>
        <para>CloudMonkey has a few requirements above and beyond CloudStack, and does not need to
          be run on the same machine as a management server. If you wish to run
            <application>CloudMonkey</application> you'll need Python 2.5 or later,
            <application>readline</application>, <application>Pygments</application>, and
            <application>prettytable</application>. CloudMonkey can be installed with
            <application>pip</application>:</para>
        <programlisting language="Bash"><prompt>$</prompt> pip install cloudmonkey</programlisting>
        <para>See the Developer's Guide and <ulink
            url="https://cwiki.apache.org/CLOUDSTACK/cloudstack-cloudmonkey-cli.html">the CloudStack
            wiki</ulink> for the latest information on <application>CloudMonkey</application>
          installation and use.</para>
      </section>
      <section id="apidiscover">
        <title>API Discovery Service</title>
        <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-926"
            >CLOUDSTACK-926</ulink>: CloudStack has more than 300 APIs and more are added in each
          major release. CloudStack admins can enable or disable APIs, or add plugins which provide
          more APIs. The API Discovery Service is a plugin which will help users discover the APIs
          available to them on a CloudStack Management Server.</para>
        <para>The discovery service implements a method called <command>listApis</command> which
          will return information about APIs for a user. It currently accepts an apiName to list api
          information of that particular API. The method ensures that user can only list APIs they
          are entitled to.</para>
        <para>All CloudStack APIs are implemented by annotated command class and PluggableService is
          a contract implemented by all the components such as the Management Server and all the
          plugins which provide an API. During load time, API discovery service asks all the
          pluggable services to return list of API cmd classes from whose fields and annotations it
          gathers information about each API, the information consists of name, description,
          parameter name, parameter description, etc.</para>
        <para>For more information on the implementation of the API Discovery Service for 4.1.0, see
          the <ulink url="https://cwiki.apache.org/CLOUDSTACK/api-discovery-service.html">CloudStack
            wiki</ulink>.</para>
      </section>
      <section id="events-framework">
        <title>Events Framework</title>
        <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-820"
            >CLOUDSTACK-820</ulink>: The Events Framework provides a mechanism to publish and
          subscribe to events in &PRODUCT;.</para>
      </section>
      <section id="additional-vmx-settings">
        <title>Additional VMX Settings</title>
        <para>###</para>
      </section>
      <section id="l3-nicira">
        <title>L3 Router Functionality in Nicira Nvp Plugin</title>
        <para>###</para>
      </section>
      <section id="persistent-networks">
        <title>Persistent Networks without Running VM</title>
        <para>###</para>
      </section>
      <section id="add-remove-network-vm">
        <title>Add/Remove Network on VM</title>
        <para>###</para>
      </section>
      <section id="resize-volumes">
        <title>Resize Volumes Feature</title>
        <para>###</para>
      </section>
      <section id="autoscale">
        <title>Autoscale</title>
        <para>###</para>
      </section>
      <section id="api-throttling">
        <title>API Request Throttling</title>
        <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-618"
            >CLOUDSTACK-618</ulink>: Limits the number of API requests per second that can be placed
          against a management server to avoid DoS attacks via API requests.</para>
        <para>The throttling is controlled by the <command>api.throttling.enabled</command>,
            <command>api.throttling.interval</command>, and <command>api.throttling.max</command>
          configuration settings. Note that <command>api.throttling.enabled</command> is set to
            <emphasis>false</emphasis> by default.</para>
      </section>
      <section id="s3-backed-storage">
        <title>S3 Backed Secondary Storage</title>
        <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-509"
            >CLOUDSTACK-509</ulink>: This enhancement backs NFS secondary storage with an
          S3-compatible object store. Periodically, a reaper thread synchronizes the templates,
          ISOs, and snapshots stored on a NFS secondary storage mount with a configured S3 object
          store. In addition to permitting the use of commodity or IaaS storage solutions for static
          assets, it provides a means of automatically synchronizing template and ISO assets across
          multiple zones.</para>
        <para>See the <ulink
            url="https://cwiki.apache.org/CLOUDSTACK/s3-backed-secondary-storage.html">&PRODUCT;
            wiki</ulink> for more information on this feature, currently the <ulink
            url="https://issues.apache.org/jira/browse/CLOUDSTACK-878">documentation is
            incomplete</ulink>.</para>
      </section>
      <section id="user-domain-admin-create-key">
        <title>User and Domain Admin Can Create API Key and Secret</title>
        <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-437"
            >CLOUDSTACK-437</ulink>: This feature adds the ability for domain admins and users to
          create their own API Key and Secret. Domain admins can create keys for themselves,
          subdomain admins, and for regular users, but not for other domain admins.</para>
      </section>
      <section id="inline-srx-f5-mode">
        <title>Support Inline Mode for F5 and SRX</title>
        <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-306"
            >CLOUDSTACK-306</ulink>: For &PRODUCT; deployments using the Juniper SRX (firewall) and
          F5 Big IP (load balancer), &PRODUCT; 4.1.0 supports putting the firewall in front of the
          load balancer, making the firewall device the gateway and putting the load balancer behind
          the public network.</para>
      </section>
      <section id="egress-firewall">
        <title>Egress Firewall Rules for Guest Networks</title>
        <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-299"
            >CLOUDSTACK-299</ulink>: This feature allows users to create egress (exit) traffic rules
          from private networks to public networks (<emphasis>e.g.</emphasis> from your internal
          network to the public Internet). By default all traffic is blocked from internal networks
          to the public networks, this allows you to open ports as necessary.</para>
        <para>Egress traffic rules are suppored only on virtual routers at this time, physical
          devices are not supported.</para>
      </section>
      <section id="reset-ssh-key">
        <title>Reset SSH Key to Access VM</title>
        <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-297"
            >CLOUDSTACK-297</ulink>: &PRODUCT; 4.1.0 introduces a new API
            <command>resetSSHKeyForVirtualMachine</command>, that can allow them to set or reset the
          SSH keypair assigned to a virtual machine.</para>
      </section>
    </section>
    <section id="issues-fixed-4.1">
      <title>Issues Fixed in 4.1.0</title>
      <para>Apache CloudStack uses <ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK"
          >Jira</ulink> to track its issues. All new features and bugs for 4.1.0 have been tracked
        in Jira, and have a standard naming convention of "CLOUDSTACK-NNNN" where "NNNN" is the
        issue number.</para>
      <para>This section includes a summary of known issues against 4.0.0 that were fixed in 4.1.0.
        Approximately 470 bugs were resolved or closed in the 4.1.0 cycle.</para>
      <informaltable>
        <tgroup cols="2" align="left" colsep="1" rowsep="1">
          <colspec colwidth="1*" colname="1" colnum="1"/>
          <colspec colwidth="2*" colname="2" colnum="2"/>
          <thead>
            <row>
              <entry>
                <para>Defect</para>
              </entry>
              <entry>
                <para>Description</para>
              </entry>
            </row>
          </thead>
          <tbody>
            <row>
              <entry>
                <para>CLOUDSTACK-46</para>
              </entry>
              <entry>
                <para>Remnants of mycloud remain.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-70</para>
              </entry>
              <entry>
                <para>Improve Network Restart Behaviour for Basic Zone: Restarting Network
                  Fail</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-94</para>
              </entry>
              <entry>
                <para>"API command, listIsos documentation clarity</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-95</para>
              </entry>
              <entry>
                <para>IP address allocation not working when a user tries to allocate IP addresses
                  in a Project</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-97</para>
              </entry>
              <entry>
                <para>Vmware network labels are ignored when creating a Zone using basic
                  networking</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-108</para>
              </entry>
              <entry>
                <para>VM should not be allowed to be deployed on two Isolated Networks of an Account
                  that were created from DefaultNetworkOfferingwithSourceNATService</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-118</para>
              </entry>
              <entry>
                <para>Status of host resorce stuck in "ErrorInMaintenance"</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-119</para>
              </entry>
              <entry>
                <para>Move Agent-Simulator in to the hypervisor plugin model</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-130</para>
              </entry>
              <entry>
                <para>Clarify docs on tags parameter in API reference</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-152</para>
              </entry>
              <entry>
                <para>Routes on the User VM are programmed incorrectly on a VM present on both
                  Isolated and Shared Guest Network</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-178</para>
              </entry>
              <entry>
                <para>Expose name parameter of VM in list Vm view.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-198</para>
              </entry>
              <entry>
                <para>vpn:failto add VPN Users deletes all the existing Vpn user</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-222</para>
              </entry>
              <entry>
                <para>Admin UI prompts to restart Management server with cancel edit
                  operation</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-225</para>
              </entry>
              <entry>
                <para>API Docs: Request params repeated with different description</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-226</para>
              </entry>
              <entry>
                <para>UpdatePhysicalNetworkcommand failed due to java.sql.BatchUpdateException ;
                  Tried to extend the existing Guest VLAN Range of one physical network into the
                  Guest VLAN range of the other physical network</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-227</para>
              </entry>
              <entry>
                <para>ReconnectHostCmd: NullPointerException: Unable to get host Information for
                  XenServer 6.0.2 host - on intentionally changing the traffic labels on the
                  physical network</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-228</para>
              </entry>
              <entry>
                <para>UI provides an option to reconnect a disconnected host - ServerApiException is
                  thrown on an attempt</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-232</para>
              </entry>
              <entry>
                <para>Zone infrastructure chart -- disable resource total display</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-235</para>
              </entry>
              <entry>
                <para>Network rate can be set in 2 places. Clarify docs on how this works</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-249</para>
              </entry>
              <entry>
                <para>Add host id to failed VM deploy alerts</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-250</para>
              </entry>
              <entry>
                <para>Incorrect description of maintenance mode in admin guide</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-256</para>
              </entry>
              <entry>
                <para>"vpn:As an admin user, not able to delete VPN user which is present in a
                  regular user's network.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-271</para>
              </entry>
              <entry>
                <para>updatePhysicalNetwork dies with an NPE when the vlan range is empty</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-274</para>
              </entry>
              <entry>
                <para>Two error codes mapped to same value in API</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-275</para>
              </entry>
              <entry>
                <para>hostid not always a UUID</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-277</para>
              </entry>
              <entry>
                <para>Message during CloudStack management server Installation: cannot access
                  /usr/share/cloud/bridge/lib: No such file or directory</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-279</para>
              </entry>
              <entry>
                <para>deleteProject fails when executed by the regular user (works fine for
                  root/domain admin)</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-284</para>
              </entry>
              <entry>
                <para>listVirtualMachines does not return deleted machines when zone is
                  specified</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-290</para>
              </entry>
              <entry>
                <para>3.0.0 template also needed for 2.2.14 to 3.0.5 direct upgrade.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-293</para>
              </entry>
              <entry>
                <para>"We do awful, hacky things in our spec file for client"</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-304</para>
              </entry>
              <entry>
                <para>Add synchronization for createSnapshot command per host basis</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-309</para>
              </entry>
              <entry>
                <para>iptables rules being deleted from wrong VM after a migration</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-318</para>
              </entry>
              <entry>
                <para>Adding XenServer Host Fails - 6.0.2 fails with 4.0.0</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-320</para>
              </entry>
              <entry>
                <para>"sessionKey query parameter should be case-insensitive, now only sessionkey is
                  accepted"</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-322</para>
              </entry>
              <entry>
                <para>During upgrade displays error - a foreign key constraint fails
                  (`cloud/#sql-f34_6e`..</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-332</para>
              </entry>
              <entry>
                <para>"count" property in list* API response should be equal to how many entries in
                  database, not how many objects in API response</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-333</para>
              </entry>
              <entry>
                <para>When Datacenter name in VCenter has spaces Primary Storage (VMFS) discovery
                  will fail</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-335</para>
              </entry>
              <entry>
                <para>KVM VPC load balancer not working</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-336</para>
              </entry>
              <entry>
                <para>listZones doesn't honour paging</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-343</para>
              </entry>
              <entry>
                <para>"Document what tools and packages are required to build, package and install
                  CloudStack 4.0</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-346</para>
              </entry>
              <entry>
                <para>Cannot add Vmware cluster with class loader conflict exception</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-347</para>
              </entry>
              <entry>
                <para>listNetworks API: return vlan information only when the caller is ROOT
                  admin</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-348</para>
              </entry>
              <entry>
                <para>deleteNetwork does not clean up network resource count correctly</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-354</para>
              </entry>
              <entry>
                <para>Display of storage statistics is wrong</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-355</para>
              </entry>
              <entry>
                <para>"Fix ""count"" in a bunch of API commands</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-357</para>
              </entry>
              <entry>
                <para>"ISOs can be deleted while still attached to a running VM, and they
                  subsequently cannot be detached from a running VM</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-359</para>
              </entry>
              <entry>
                <para>PropagateResourceEventCommand failes in cluster configuration</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-361</para>
              </entry>
              <entry>
                <para>Wrong creation of guest networks on a KVM host in Multiple Physical Networks
                  with guest traffic</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-364</para>
              </entry>
              <entry>
                <para>Docs point to download.cloud.com for AWS API script</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-368</para>
              </entry>
              <entry>
                <para>OVM - cannot create guest VM</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-369</para>
              </entry>
              <entry>
                <para>ASF 4.0 - unable to support XenServer 6.1 host</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-373</para>
              </entry>
              <entry>
                <para>"static NAT and Firewall is not working on external firewall device SRX, it
                  needs to be implemented</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-377</para>
              </entry>
              <entry>
                <para>provide deployment config access to marvin's testcase</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-378</para>
              </entry>
              <entry>
                <para>mavenize marvin on master</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-390</para>
              </entry>
              <entry>
                <para>Install Guide: Section 4.5.7 (Prepare the System VM Template): Links go to
                  cloud.com</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-397</para>
              </entry>
              <entry>
                <para>Install Guide: Section 11.1 (Guest Traffic): Diagram is the wrong
                  diagram</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-398</para>
              </entry>
              <entry>
                <para>Install Guide: Section 11.17.3 (Using VPN with Mac OSX): Not complete</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-404</para>
              </entry>
              <entry>
                <para>Update docs on the usage of cloud-setup-database</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-412</para>
              </entry>
              <entry>
                <para>Data truncation: Out of range value for column 'ram' at row </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-415</para>
              </entry>
              <entry>
                <para>restartNetwork call causes VM to be unreachable when Nicira based SDN is
                  used</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-416</para>
              </entry>
              <entry>
                <para>XCP 1.6beta2 (61002c) - can't add a host</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-417</para>
              </entry>
              <entry>
                <para>Handle password server securely to run on port 8080 on VR</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-424</para>
              </entry>
              <entry>
                <para>Updated userdata not propagating to the VR</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-427</para>
              </entry>
              <entry>
                <para>Change hardcoded step number references to dynamic link</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-428</para>
              </entry>
              <entry>
                <para>Storage capacity shown in UI is incorrect</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-435</para>
              </entry>
              <entry>
                <para>Vmware network labels are ignored when creating a Zone using basic
                  networking</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-441</para>
              </entry>
              <entry>
                <para>Running mgmt server using jetty fails to start api server</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-446</para>
              </entry>
              <entry>
                <para>"Host going to alert state, if you are adding already added host</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-448</para>
              </entry>
              <entry>
                <para>SSVM bootstrap failure on XenServer hosts with E3 CPU</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-456</para>
              </entry>
              <entry>
                <para>License tag in SPEC isn't what RPM is expecting</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-459</para>
              </entry>
              <entry>
                <para>[Optional Public IP assignment for EIP with Basic Zone] Associate IP Checkbox
                  in Create Network Offering Dialog is Displayed When Elastic LB is Selected</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-462</para>
              </entry>
              <entry>
                <para>A few corrections to make to the 4.0.0 installation guide</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-464</para>
              </entry>
              <entry>
                <para>"Regression in AWSAPI docs, entire sections removed</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-465</para>
              </entry>
              <entry>
                <para>French language file quotes are dropping javascript syntax error</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-467</para>
              </entry>
              <entry>
                <para>Developer's Guide points to cloud.com for API reference</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-479</para>
              </entry>
              <entry>
                <para>UpdateVirtualMachine api fails to propagate userdata to domr</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-481</para>
              </entry>
              <entry>
                <para>Installation Guide Doc Error</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-493</para>
              </entry>
              <entry>
                <para>2.2.x-3.0 DB upgrade support for Advance SG enabled network</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-499</para>
              </entry>
              <entry>
                <para>cloudmonkey CLI can't accept complex parameter</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-500</para>
              </entry>
              <entry>
                <para>Passwd-server iptables rules are dropped on domr on fresh start or on
                  reboot.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-501</para>
              </entry>
              <entry>
                <para>Apidocs and marvin does not know how to handle Autoscaling docs.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-504</para>
              </entry>
              <entry>
                <para>Duplicate guest password scripts in codebase.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-507</para>
              </entry>
              <entry>
                <para>fix api docs for listSSHKeyPair</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-508</para>
              </entry>
              <entry>
                <para>CLVM copies template to primary storage unnecessarily.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-510</para>
              </entry>
              <entry>
                <para>Add button not visible when adding public IPs to physical network.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-514</para>
              </entry>
              <entry>
                <para>Marvin and Cloudmonkey don't work when an API target uses https or an
                  alternate path.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-518</para>
              </entry>
              <entry>
                <para>API refactoring -- change @Parameter annotation and remove the @IdentityMapper
                  annotation.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-520</para>
              </entry>
              <entry>
                <para>Dependency jar names mismatch with install-non-oss.sh</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-521</para>
              </entry>
              <entry>
                <para>Build will hung up when doing test for TestAgentShell</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-522</para>
              </entry>
              <entry>
                <para>Log requests in cloudmonkey's log file.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-527</para>
              </entry>
              <entry>
                <para>List API performance optimization by using DB views and removing UUID
                  conversion.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-534</para>
              </entry>
              <entry>
                <para>Failed to add host</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-536</para>
              </entry>
              <entry>
                <para>remove citrix cloudpatform from 4.0 build - CloudStack is ASF project.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-539</para>
              </entry>
              <entry>
                <para>Cropped Text in UI under Quick View.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-552</para>
              </entry>
              <entry>
                <para>]Quick view details for a volume displays scroll bar in place of name of the
                  volume when the name of the volume has more no of characters.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-553</para>
              </entry>
              <entry>
                <para>"SRX - When adding SRX device make "Public Network" - default to "untrusted"
                  and "Private Network" - default to "trusted" as un-editable fields.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-556</para>
              </entry>
              <entry>
                <para>Erratic window behavior in Quick View tooltip.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-559</para>
              </entry>
              <entry>
                <para>source code import problem</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-560</para>
              </entry>
              <entry>
                <para>Usage server doesn't work in 4.0.0 due to missing db changes</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-572</para>
              </entry>
              <entry>
                <para>SG Enabled Advanced Zone - Not able to deploy a VM in an account specific
                  shared network</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-573</para>
              </entry>
              <entry>
                <para>"NPE at
                  ""com.cloud.network.NetworkManagerImpl.networkOfferingIsConfiguredForExternalNetworking(NetworkManagerImpl.java:4345)""
                  when create network from the network offering having NULL provider for the
                  service</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-578</para>
              </entry>
              <entry>
                <para>The already deleted same hostname is not deleted from /etc/hosts of
                  vRouter</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-584</para>
              </entry>
              <entry>
                <para>"typos in
                  ""Apache_CloudStack-4.0.0-incubating-CloudStack_Nicira_NVP_Guide-en-US""</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-590</para>
              </entry>
              <entry>
                <para>Incorrect Network Gateways Assigned to System VM</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-592</para>
              </entry>
              <entry>
                <para>"API bloat, unknown apis cmd classes</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-593</para>
              </entry>
              <entry>
                <para>"2 guest network, auto create vlan error</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-596</para>
              </entry>
              <entry>
                <para>DeployVM command takes a lot of time to return job id.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-599</para>
              </entry>
              <entry>
                <para>DhcpEntryCommand fails on Router VM on CS4.0 and vSphere5 with Advanced
                  Network Zone.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-600</para>
              </entry>
              <entry>
                <para>When rebooting KVM local storage VM host, libvirt definitions deleted</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-605</para>
              </entry>
              <entry>
                <para>Host physical CPU is incorrectly calculated for Vmware host</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-606</para>
              </entry>
              <entry>
                <para>Starting VM fails with 'ConcurrentOperationException' in a clustered MS
                  scenario</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-614</para>
              </entry>
              <entry>
                <para>"ListTemplates API is not returning ""Enable SSH Key"" attribute for any given
                  template</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-617</para>
              </entry>
              <entry>
                <para>Unable to edit a Sub domain</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-639</para>
              </entry>
              <entry>
                <para>API Refactoring: Adapters for ACL</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-648</para>
              </entry>
              <entry>
                <para>The normal users could change their own login password.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-660</para>
              </entry>
              <entry>
                <para>Network Traffic Labels are not functional in Marvin</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-683</para>
              </entry>
              <entry>
                <para>Image Is Missing in the Accessing VM Section</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-689</para>
              </entry>
              <entry>
                <para>RVR: Stop pending flag is not cleared when user start the disconnected router
                  from another host</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-691</para>
              </entry>
              <entry>
                <para>A warning dialog box shows after reloading the welcome page.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-693</para>
              </entry>
              <entry>
                <para>Adding a VPC virtual router to a NiciraNVP enabled network fails.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-694</para>
              </entry>
              <entry>
                <para>"Create a new VPC network offering with "connectivity" option needed for SDN
                  networking) is not allowed / VPC support for SDN networks</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-717</para>
              </entry>
              <entry>
                <para>cloudmonkey fails to parse/print response.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-720</para>
              </entry>
              <entry>
                <para>Fail to load a png image when accessing the web console.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-721</para>
              </entry>
              <entry>
                <para>Bytes sent/received in user statistics is empty (CloudStack 4.0)</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-725</para>
              </entry>
              <entry>
                <para>UI: Error when the Egress rules tab is selected for a network.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-734</para>
              </entry>
              <entry>
                <para>api_refactoring: CreateAccountCmd fails to send response due to NPE in service
                  layer</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-735</para>
              </entry>
              <entry>
                <para>Integration smoke tests: Fix expunge vm test on api_refactoring</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-736</para>
              </entry>
              <entry>
                <para>Integration smoke tests: Fix check for vm name for the deployvm smoke
                  test.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-793</para>
              </entry>
              <entry>
                <para>"Create cloudmonkey-helper, a plugin that helps autodiscover and sync api info
                  via an api over some endpoint</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-798</para>
              </entry>
              <entry>
                <para>Move usage related cmd classes from cloud-server to cloud-api</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-799</para>
              </entry>
              <entry>
                <para>[Load Test] Check router statistics falls behind in gathering stats by more
                  than 2 times the set value</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-819</para>
              </entry>
              <entry>
                <para>Create Account/User API logging password in access log</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-863</para>
              </entry>
              <entry>
                <para>Non-printable characters (ASCII control character) such as %00 or %0025 are
                  getting stored in raw/non encoded form in the database</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-870</para>
              </entry>
              <entry>
                <para>Client UI: Wrong character encoding for some language</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-928</para>
              </entry>
              <entry>
                <para>[Simulator] Latency for Agent Commands - change unit of wait from seconds to
                  milliseconds</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-938</para>
              </entry>
              <entry>
                <para>s2s VPN trouble</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-959</para>
              </entry>
              <entry>
                <para>Missing sub-sections in document section System Service Offering</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-968</para>
              </entry>
              <entry>
                <para>marvin: vlan should be an attribute of the physical_network and not the
                  zone</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-977</para>
              </entry>
              <entry>
                <para>Document how to use openvswitch with KVM hypervisor</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-978</para>
              </entry>
              <entry>
                <para>TypeError: instance.displayname is undefined while adding VM's to the LB
                  rule</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-985</para>
              </entry>
              <entry>
                <para>Different MAC address for RvR caused issue in short term network outage</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-987</para>
              </entry>
              <entry>
                <para>Sections missing in Working With Snapshots</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-993</para>
              </entry>
              <entry>
                <para>"admin"" user is not getting created when management server is started.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-995</para>
              </entry>
              <entry>
                <para>Not able to add the KVM host</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1002</para>
              </entry>
              <entry>
                <para>Not able to start VM</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1006</para>
              </entry>
              <entry>
                <para>need to disable service libvirt-guests in CentOS packaging RPMs, or in
                  installation docs</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1008</para>
              </entry>
              <entry>
                <para>"Egress"" tab should not be presented in the UI for Shared Networks</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1010</para>
              </entry>
              <entry>
                <para>Host count and Secondary storage count always shows 1 in UI</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1011</para>
              </entry>
              <entry>
                <para>KVM host getting disconnected in cluster environment</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1013</para>
              </entry>
              <entry>
                <para>running cloudstack overwrites default public/private ssh key</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1014</para>
              </entry>
              <entry>
                <para>Merge ManagementServer and ManagementServerEx</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1016</para>
              </entry>
              <entry>
                <para>Not able to deploy VM</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1021</para>
              </entry>
              <entry>
                <para>the vlan is not creat to right nic. when i creat multi guest network</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1024</para>
              </entry>
              <entry>
                <para>Regression: Unable to add Xenserver host with latest build.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1027</para>
              </entry>
              <entry>
                <para>"Update SSL certificate" button should properly reflect its
                  functionality</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1029</para>
              </entry>
              <entry>
                <para>Enter the token to specified project is malfunctioned</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1037</para>
              </entry>
              <entry>
                <para>"Make cloudmonkey awesome-er: Online help docs and api discovery, better
                  colored output, parameter value autocompletion</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1050</para>
              </entry>
              <entry>
                <para>No Documentation on Adding a Load Balancer Rule</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1051</para>
              </entry>
              <entry>
                <para>API dispatcher unable to find objectVO corresponding to
                  DeleteTemplatecmd</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1055</para>
              </entry>
              <entry>
                <para>"The overlay still exists when the ""Recurring Snapshots"" dialog is canceled
                  by pressing esc key.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1056</para>
              </entry>
              <entry>
                <para>S3 secondary storage fails to upload systemvm template due to KVMHA
                  directory</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1057</para>
              </entry>
              <entry>
                <para>regression of changeServiceForVirtualMachine API - fails to find service
                  offering by serviceOfferingId parameter</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1063</para>
              </entry>
              <entry>
                <para>"SG Enabled Advanced Zone - "Add Guest Networks" - When user tries to add a
                  guest Network with scope as "Account" he should NOT be presented with "Offering
                  for shared security group enabled"</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1064</para>
              </entry>
              <entry>
                <para>A type error occurs when trying to add account/register template...</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1068</para>
              </entry>
              <entry>
                <para>Names in VR list is useless</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1070</para>
              </entry>
              <entry>
                <para>javelin: NPE on executing registerIso API</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1071</para>
              </entry>
              <entry>
                <para>Netscaler element is not getting loaded as part of LoadBalancing Service
                  Providers</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1078</para>
              </entry>
              <entry>
                <para>Not able to start System Vms on Rhel 6.3 KVM host</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1079</para>
              </entry>
              <entry>
                <para>Deploying AWSAPI with mvn -pl :cloud-awsapi jetty:run fail</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1082</para>
              </entry>
              <entry>
                <para>UI doesn't throw any error message when trying to delete ip range from a
                  network that is in use.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1083</para>
              </entry>
              <entry>
                <para>listUsageRecords api: removed project results in NPE</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1087</para>
              </entry>
              <entry>
                <para>Update the Developer Guide for ASFCS 4.1 Release</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1088</para>
              </entry>
              <entry>
                <para>EnableStaticNat error will clear the data in database</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1094</para>
              </entry>
              <entry>
                <para>Ipv6 - hostname/hostname --fqdn does not return the name of the VM. But i am
                  able to reach the Vm using their names</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1095</para>
              </entry>
              <entry>
                <para>Ipv6 - dhclient command needs to be run manually on the Vms to get the Ipv6
                  address</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1100</para>
              </entry>
              <entry>
                <para>Expunge thread is not kicked off based on global configuration if the global
                  setting is less than 60 seconds</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1103</para>
              </entry>
              <entry>
                <para>"IpV6 - listNetwork() command does not retrun gateway,netmask,cidr</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1104</para>
              </entry>
              <entry>
                <para>Ipv6 - listVlanIpRanges() returns error 530</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1105</para>
              </entry>
              <entry>
                <para>"IpV6 - listVirtualMachines() does not return netmask,
                  gateway,ipaddress.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1107</para>
              </entry>
              <entry>
                <para>Ipv6 - Unable to extend Ip range for a Ipv6 network using craeteVlanIpRange()
                  command - Error code 530 returned</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1108</para>
              </entry>
              <entry>
                <para>Ipv6 - Not able to restart Networks</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1109</para>
              </entry>
              <entry>
                <para>"Ipv6 - Unable to expunge User Vms that are "Destroyed".</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1111</para>
              </entry>
              <entry>
                <para>Ipv6 - listRouters() does not return guestipaddress/</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1112</para>
              </entry>
              <entry>
                <para>"Errors in "Prepare the System VM Template"</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1113</para>
              </entry>
              <entry>
                <para>"Ipv6 - Not able to deploy a new VM in this network because of "Unable to
                  allocate Unique Ipv6 address"</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1114</para>
              </entry>
              <entry>
                <para>unable to execute listegressfirewallrules API due invalid value id</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1115</para>
              </entry>
              <entry>
                <para>In multiple shared network unable to login with default nic - KVM</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1123</para>
              </entry>
              <entry>
                <para>ListStoragePools API broken by refactor</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1138</para>
              </entry>
              <entry>
                <para>"Providing invalid values for gateway, netmask etc in the zoneWizard blocks
                  the VLAN container to load, throwing an error</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1139</para>
              </entry>
              <entry>
                <para>"After the Vm is "Expunged" we see the entry still being present in the router
                  in /etc/dhcphosts.txt</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1141</para>
              </entry>
              <entry>
                <para>"Ipv6 - After network restart (and reboot router), we do not see the existing
                  vms dnsentries not being programmed in the router.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1152</para>
              </entry>
              <entry>
                <para>Missing tag in host-add.xml</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1153</para>
              </entry>
              <entry>
                <para>"Ipv6 - Vm deployment fails with "n must be positive" error.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1154</para>
              </entry>
              <entry>
                <para>Account/Users related API failed due to RegionService inject exception.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1157</para>
              </entry>
              <entry>
                <para>No API Documentation on Listing Custom User Templates Using CS4 API</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1160</para>
              </entry>
              <entry>
                <para>References to version=3.0.3|4|5|6 in API classes needs to be removed.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1161</para>
              </entry>
              <entry>
                <para>Differences between 4.1 and master in
                  ongoing-config-of-external-firewalls-lb.xml</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1163</para>
              </entry>
              <entry>
                <para>Failed with NPE while creating firewall rule</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1168</para>
              </entry>
              <entry>
                <para>Create firewall rule broke</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1173</para>
              </entry>
              <entry>
                <para>ConsoleProxyResource instantiation exception.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1174</para>
              </entry>
              <entry>
                <para>Snapshots related SQL error.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1176</para>
              </entry>
              <entry>
                <para>Issue with snapshots(create/list)</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1181</para>
              </entry>
              <entry>
                <para>mvn deploy db failing with NPE</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1190</para>
              </entry>
              <entry>
                <para>Make APIChecker interface throw a single sensible exception.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1200</para>
              </entry>
              <entry>
                <para>"Unknown column 'vm_instance.disk_offering_id' in table vm_instance, db
                  exception shown in MS log</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1201</para>
              </entry>
              <entry>
                <para>"Failed to create ssh key for user "cloud"
                  /var/lib/cloud/management/.ssh/id_rsa and failed to start management server</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1202</para>
              </entry>
              <entry>
                <para>Fail to install KVM cloud-agent.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1203</para>
              </entry>
              <entry>
                <para>Fail to create advance zone with SG enabled when UI allows SG enabled
                  option.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1204</para>
              </entry>
              <entry>
                <para>Fail to create advance zone due to fail to add host</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1205</para>
              </entry>
              <entry>
                <para>Ipv6 - Ubuntu 12.10 guest Vms loses default route (after it expiration time ~
                  30 mts) when ipv6.autoconfig parameters are disabled except for
                  net.ipv6.conf.lo.autoconf which is enabled.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1206</para>
              </entry>
              <entry>
                <para>Failure in Copy of System template</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1210</para>
              </entry>
              <entry>
                <para>Make all pluggable services return list of api cmd classes</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1216</para>
              </entry>
              <entry>
                <para>UUID is null for admin and failed to register user key with 4.0</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1218</para>
              </entry>
              <entry>
                <para>"IPv6: Shared Network - After network restart with clean option, router is
                  assigned a different address. Name resolution for the existing guest Vms in the
                  network fails.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1219</para>
              </entry>
              <entry>
                <para>Ipv6 - Provide better error messages when deploying a Vm with Ip an address
                  that is outside the network's ip range / if the ip address already is assigned to
                  another Vm</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1220</para>
              </entry>
              <entry>
                <para>Ipv6 - Better error message when deploy Vm fails to get a free Ip
                  address</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1222</para>
              </entry>
              <entry>
                <para>API rate limit configs: removed double quote in upgrade script</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1223</para>
              </entry>
              <entry>
                <para>Exception while starting jetty server:
                  org.springframework.beans.factory.BeanCreationException Error creating bean with
                  name 'apiServer'</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1224</para>
              </entry>
              <entry>
                <para>Volume snapshot creation failing</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1226</para>
              </entry>
              <entry>
                <para>Error while running Cloudstack-setup-database</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1228</para>
              </entry>
              <entry>
                <para>Unable to Create System Vm's in the VMware Hypervisor setup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1229</para>
              </entry>
              <entry>
                <para>Incorrect SQL syntax to insert api limit related configuration items in
                  upgrade path script.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1231</para>
              </entry>
              <entry>
                <para>cloud-install-sys-tmplt failed due to missing path</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1232</para>
              </entry>
              <entry>
                <para>"Ipv6 - Guest Vms are not able to get Ipaddress when executing dhclient
                  command when using ""/96"" network.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1233</para>
              </entry>
              <entry>
                <para>Veewee configuration files are inappropriately identified as ASLv2 licensed
                  file</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1234</para>
              </entry>
              <entry>
                <para>Unable to start KVM agent with 4.1 build.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1237</para>
              </entry>
              <entry>
                <para>"Register Template fails with ""Cannot find template adapter for
                  XenServer""</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1239</para>
              </entry>
              <entry>
                <para>Unable to registerISO :unhandled exception executing api command:
                  registerIso</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1240</para>
              </entry>
              <entry>
                <para>Unable to registerTemplate : Cannot find template adapter for
                  XenServer.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1241</para>
              </entry>
              <entry>
                <para>Network apply rules logic is broken.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1242</para>
              </entry>
              <entry>
                <para>[F5-SRX-InlineMode] Failed to create LB rule with F5-SRX inlinemode
                  deployment</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1243</para>
              </entry>
              <entry>
                <para>Failed to cleanup account :java.lang.NullPointerException</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1244</para>
              </entry>
              <entry>
                <para>fail to push sysmvm.iso onto xen host</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1246</para>
              </entry>
              <entry>
                <para>"[ ALU beta CS 4.1 build2] ""Guest network"" missing in Add Zone wizard ( step
                  3, Setup Network \ Physical Network)</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1251</para>
              </entry>
              <entry>
                <para>Baremetal zone doesn't need primary/secondary storage in UI wizard.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1252</para>
              </entry>
              <entry>
                <para>Failed to download default template in VMware.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1260</para>
              </entry>
              <entry>
                <para>Failed to register template: Unable to find template adapter</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1261</para>
              </entry>
              <entry>
                <para>Cannot find template adapter for XenServer.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1262</para>
              </entry>
              <entry>
                <para>"Failed to Prepare Secondary Storage in VMware,</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1265</para>
              </entry>
              <entry>
                <para>logrotate dnsmasq configuration is wrong</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1267</para>
              </entry>
              <entry>
                <para>KVM's cloudstack-agent service doesn't log (log4j)</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1269</para>
              </entry>
              <entry>
                <para>Failed to start CPVM java.lang.NullPointerException Unable to start
                  SSVM</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1272</para>
              </entry>
              <entry>
                <para>Autoscale: createAutoScaleVmProfile fails due to unable to retrieve Service
                  Offering ip</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1274</para>
              </entry>
              <entry>
                <para>UpdateNetworkCmd throws NP</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1276</para>
              </entry>
              <entry>
                <para>Remove autoscanning for 4.1</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1277</para>
              </entry>
              <entry>
                <para>ApiResponseHelper.createUserVmResponse failed to populate password field set
                  from UserVm object</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1278</para>
              </entry>
              <entry>
                <para>Improper permissions on injectkeys.sh</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1288</para>
              </entry>
              <entry>
                <para>[F5-SRX-InlineMode] classCastException during network restart with cleanup
                  option true</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1289</para>
              </entry>
              <entry>
                <para>[F5-SRX-InlineMode] Usage stats are not generated for Juniper SRX Firewall in
                  inlinemode</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1290</para>
              </entry>
              <entry>
                <para>listNetoworks API takes too long to respond</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1292</para>
              </entry>
              <entry>
                <para>"[F5-SRX-InlineMode] Update network from SRX,F5 as service provideds to VR as
                  service provider does not delete firewall rules from SRX</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1295</para>
              </entry>
              <entry>
                <para>NPE in usage parsers due to missing @Component inject</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1299</para>
              </entry>
              <entry>
                <para>Errors in 4.5.5 section of installation guide</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1300</para>
              </entry>
              <entry>
                <para>section in wrong order in installation guide</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1303</para>
              </entry>
              <entry>
                <para>Ipv6 - java.lang.NullPointerException when executing listnetworks() and
                  deployVirtualMachine() after extending the Ipv4 range of a dual stack
                  network</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1307</para>
              </entry>
              <entry>
                <para>Noticed NPE when we put host in maintenance mode in clustered management
                  setup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1310</para>
              </entry>
              <entry>
                <para>ASF-build-master-nonoss-rhel63 - create advance zone FAIL -
                  CreatePhysicalNetworkCmd FAIL - MySQLIntegrityConstraintViolationException:
                  Duplicate entry '200-Public' for key 'physical_network_id'</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1312</para>
              </entry>
              <entry>
                <para>"Fix rolling upgrades from 4.0 to 4.1 in 4.1 release, fix db schemas to be
                  same as 4.0</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1313</para>
              </entry>
              <entry>
                <para>Working with Volumes Section Is Missing</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1315</para>
              </entry>
              <entry>
                <para>[F5-SRX-InlineMode] Network implement failed with Run time Exception during
                  network upgrade from VR to SRX-F5</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1319</para>
              </entry>
              <entry>
                <para>createCustomerVpnGateway response gives TypeError:
                  json.createvpncustomergatewayresponse is undefined</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1320</para>
              </entry>
              <entry>
                <para>Routers naming convention is changed to hostname.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1321</para>
              </entry>
              <entry>
                <para>[Site-to-Site VPN] No events are generated in case of status change in site to
                  site vpn connection</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1326</para>
              </entry>
              <entry>
                <para>KVM - Failed to start cloud agent from SSVM</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1328</para>
              </entry>
              <entry>
                <para>console view unable to connect - CPVM SSVM guest VM</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1329</para>
              </entry>
              <entry>
                <para>"API listRouters response returns hostname instead of Virtual Routers, UI
                  displays host entry for each VR</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1330</para>
              </entry>
              <entry>
                <para>ec2-run-instances - When -n option is used to deploy multiple Vms API returns
                  error even though few of the Vms have been deployed successfully</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1331</para>
              </entry>
              <entry>
                <para>Upgrade fails for a 2.2.14 Zone having multiple guest networks using
                  network_tags and Public Vlan</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1332</para>
              </entry>
              <entry>
                <para>IPV6 - Router and guest Vms should be able to use an IPV6 address for external
                  DNS entry</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1334</para>
              </entry>
              <entry>
                <para>vmware.root.disk.controller doesn't work</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1337</para>
              </entry>
              <entry>
                <para>Zone to zone template/ISO copy fails and template/ISO download also
                  fail</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1338</para>
              </entry>
              <entry>
                <para>Deploy VM failed using IS</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1339</para>
              </entry>
              <entry>
                <para>ASF 4.1: Management server becomes unresponsive</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1341</para>
              </entry>
              <entry>
                <para>URL for the KEYs file is wrong in the installation guide</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1342</para>
              </entry>
              <entry>
                <para>Document installation and usage of cloudmonkey for 4.1 docs</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1343</para>
              </entry>
              <entry>
                <para>Porting Baremetal related UI changes to ACS</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1344</para>
              </entry>
              <entry>
                <para>Typo in use.external.dns setting description</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1345</para>
              </entry>
              <entry>
                <para>BigSwitch plugin introduces 'VNS' isolation in UI without backend
                  implementation</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1346</para>
              </entry>
              <entry>
                <para>"Check to see if external devices are used in the network, is hardcoded for
                  specific devices</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1347</para>
              </entry>
              <entry>
                <para>"Not able to delete network. Error - "Unable to insert queue item into
                  database, DB is full?"</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1348</para>
              </entry>
              <entry>
                <para>API/UI: zoneObj is undefined.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1349</para>
              </entry>
              <entry>
                <para>"VPC network Adding Network ACls, PF rules - Unable to insert queue item into
                  database, DB is full? PF rules and NW Acls in Add state in DB</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1350</para>
              </entry>
              <entry>
                <para>Management server Stop and start causes previously downloaded ISOs and
                  templates to redownload &amp; reinstall.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1353</para>
              </entry>
              <entry>
                <para>KVM 6.3 snapshot Scheduling snapshot failed due to
                  java.lang.NullPointerException</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1357</para>
              </entry>
              <entry>
                <para>"Autoscale: Provisioned VMs from Netscaler not being added to lb vserver,
                  provserver fails with provserver_err_asynctaskpoll</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1360</para>
              </entry>
              <entry>
                <para>The clusterid field of the createStoragePool API command should be documented
                  as required.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1367</para>
              </entry>
              <entry>
                <para>NPE noticed in logs while AgentMonitor is monitoring the host ping
                  interval</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1368</para>
              </entry>
              <entry>
                <para>Shared network - Not able to delete network because of
                  java.lang.NullPointerException</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1369</para>
              </entry>
              <entry>
                <para>"Ipv6 - In dual Stack network, guest VM does not have the Ipv6 address of the
                  router programmed in /etc/resolv.conf for DNS resolution.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1370</para>
              </entry>
              <entry>
                <para>DeployVM Fail - VPC or non-VPC network</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1375</para>
              </entry>
              <entry>
                <para>deploydb failing with acs master</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1376</para>
              </entry>
              <entry>
                <para>Unable to migrate VM due to internal error process exited while connecting to
                  monitor</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1377</para>
              </entry>
              <entry>
                <para>HA fail - when host is shutdown, VMs and SSVMs are not failover to second host
                  in cluster.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1382</para>
              </entry>
              <entry>
                <para>vm deploy fails with Error "cannot find DeployPlannerSelector for vm"</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1383</para>
              </entry>
              <entry>
                <para>Deploying basic zone on 4.1 fails in NPE</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1386</para>
              </entry>
              <entry>
                <para>BASIC zone SSVM fail to start due to exception</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1388</para>
              </entry>
              <entry>
                <para>UI - ListUsers doesnt display any User except the Default Root Admin
                  User</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1391</para>
              </entry>
              <entry>
                <para>EventBus is not getting injected after javelin merge</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1394</para>
              </entry>
              <entry>
                <para>[F5-SRX-InlineMode] Failure in static nat configuration on SRX does not result
                  in LB configuration error in CS during LB rule configuration</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1397</para>
              </entry>
              <entry>
                <para>Static Nat configuration is failing with NPE</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1399</para>
              </entry>
              <entry>
                <para>Unhandled exception executing api command: stopVirtualMachine</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1402</para>
              </entry>
              <entry>
                <para>listRouters API response doesn't return linklocal IP and public IP
                  details</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1403</para>
              </entry>
              <entry>
                <para>Storage and console-proxy related error</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1411</para>
              </entry>
              <entry>
                <para>Issues with VMWare Hypervisor host_ids not updated when ESX(i) crashes in
                  instance table</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1414</para>
              </entry>
              <entry>
                <para>Redundant router: BACKUP switch cancelled due to lock timeout after a glitch
                  in network.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1417</para>
              </entry>
              <entry>
                <para>When invalid values are passed to createNetwork(), error message does not
                  indicate the parameter name that has invalid values.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1418</para>
              </entry>
              <entry>
                <para>As regular user, we are not allowed to deploy VM on a shared network.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1419</para>
              </entry>
              <entry>
                <para>Apache-ify and apply trademark logos in the UI</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1420</para>
              </entry>
              <entry>
                <para>Ensure trademarks are properly attributed in publican brand</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1423</para>
              </entry>
              <entry>
                <para>Unable to launch UI [HTTP Status 404].</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1425</para>
              </entry>
              <entry>
                <para>unhandled exception executing api command: migrateVirtualMachine &amp;
                  recoverVirtualMachine</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1427</para>
              </entry>
              <entry>
                <para>Failed to delete Guestnetwork which has LB with Netscaler</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1428</para>
              </entry>
              <entry>
                <para>[UI] Instance which are created without display name are not visible when
                  added to LB</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1429</para>
              </entry>
              <entry>
                <para>single account is unable to use same vnet across multiple physical
                  network</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1436</para>
              </entry>
              <entry>
                <para>4.1 management server fails to start from RPM build artifact</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1443</para>
              </entry>
              <entry>
                <para>As domain admin we are allowed to create shared network</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1446</para>
              </entry>
              <entry>
                <para>[UI]VPC Router type should be of type vpc and not system</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1447</para>
              </entry>
              <entry>
                <para>[UI]Persistent Status is not displayed for VPC Tier</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1449</para>
              </entry>
              <entry>
                <para>listAccounts and listProjectAccounts API lists all the users not
                  account-specific users for each account returned</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1451</para>
              </entry>
              <entry>
                <para>Getting EntityExistsException while creating more than one project in CS
                  4.1</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1452</para>
              </entry>
              <entry>
                <para>Public IP's are assigned to private interface with VPC Restart [PF/LB rules
                  are not functional</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1461</para>
              </entry>
              <entry>
                <para>"Ipv6 - From a Vm that that is part of 2 networks, non default network
                  router's details should not get programmed in the DNS entries of the guest
                  VM.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1463</para>
              </entry>
              <entry>
                <para>IPV6 - Ubuntu 12.10 - Multiple Nic - IPV6 address is assigned automatically
                  for 1 nic only. Need to do a manual dhclient request to get the ipv6 for other
                  nic.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1464</para>
              </entry>
              <entry>
                <para>"IPV6 - Multi nic - Ubuntu 1210 -When Vm is stopped and started/ rebooted, i
                  get multiple global IPV6 addresses being allocated for one of the nics.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1465</para>
              </entry>
              <entry>
                <para>List Zones returns null under create instance when logged is as user</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1467</para>
              </entry>
              <entry>
                <para>Failed to create Volume for the System VMs</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1469</para>
              </entry>
              <entry>
                <para>kvm agent: agent service fails to start up</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1470</para>
              </entry>
              <entry>
                <para>unhandled exception executing api command: deployVirtualMachine</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1472</para>
              </entry>
              <entry>
                <para>AssignVirtualMachine API with wrong Virtual Instance ID failed with NPE</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1473</para>
              </entry>
              <entry>
                <para>deleteDomain is failing with NPE</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1481</para>
              </entry>
              <entry>
                <para>"IPV6 - When Vm is part of 1 dual network and 1 ipv6 network, name resolution
                  using fqdn fails for the ipv6 network.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1482</para>
              </entry>
              <entry>
                <para>IPV6 - We are not allowed to create a shared IPV6 network with a VLAN which
                  already is associated with a IPV4 network</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1484</para>
              </entry>
              <entry>
                <para>API Throttling : api.throttling.enabled, Global setting missing</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1485</para>
              </entry>
              <entry>
                <para>Add Baremetal Provider back to 4.1 branch</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1487</para>
              </entry>
              <entry>
                <para>cloudstack-setup-agent fails to set private.network.device on KVM host
                  add</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1488</para>
              </entry>
              <entry>
                <para>"Ipv6 - When Vm is deployed as part of multiple networks, one of the IPV6
                  address assigned to guest VM is lost.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1490</para>
              </entry>
              <entry>
                <para>4.1 deb management fails to start due to tomcat dep problem</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1496</para>
              </entry>
              <entry>
                <para>List API Performance: listAccounts failing with OOME for high values of
                  pagesize (>1000) </para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1499</para>
              </entry>
              <entry>
                <para>ListAPI Performance for few APIs not as good as it was before API
                  optimization</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1503</para>
              </entry>
              <entry>
                <para>listHypervisor API not getting fired when logged in as User</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1505</para>
              </entry>
              <entry>
                <para>Unknown column 'domain.region_id' in 'field list'</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1509</para>
              </entry>
              <entry>
                <para>Failed to implement network elements and resources while provisioning for
                  persistent network(createVlanIpRange to an account</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1511</para>
              </entry>
              <entry>
                <para>[UI] Instances NIC details does not have Network Name</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1512</para>
              </entry>
              <entry>
                <para>[UI] Wrong message[message.configure.all.traffic.types] when trying to create
                  zone with mulitple physical networks without providing the traffic label</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1515</para>
              </entry>
              <entry>
                <para>None of the cloudstack packges are marked for upgrade when tried to upgrade
                  from.4.0/4.0.1 to 4.1</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1516</para>
              </entry>
              <entry>
                <para>Create documentation in languages that have translations available</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1517</para>
              </entry>
              <entry>
                <para>Check UI in languages available</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1521</para>
              </entry>
              <entry>
                <para>Redundant router: Services are not stopped when switch to BACKUP state</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1526</para>
              </entry>
              <entry>
                <para>Template registration fails in the VMware Setup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1531</para>
              </entry>
              <entry>
                <para>vmware create volume from snapshot will missing date</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1537</para>
              </entry>
              <entry>
                <para>Restart network with clean up set to true causes Autoscaled LB rule to get
                  mangled and unusable</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1541</para>
              </entry>
              <entry>
                <para>NPE while deleting snapshot :Unexpected exception while executing
                  org.apache.cloudstack.api.command.user.snapshot.DeleteSnapshotCmd</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1542</para>
              </entry>
              <entry>
                <para>unhandled exception while creating project</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1544</para>
              </entry>
              <entry>
                <para>The description and the response format for the deleteUser command are
                  incorrect</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1550</para>
              </entry>
              <entry>
                <para>createaccountresponse returns more than the user you requested for
                  creation</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1553</para>
              </entry>
              <entry>
                <para>AWS Regions-Not able to list accounts from the 2nd region after
                  user/account/domain details have been manually synced up from first region</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1555</para>
              </entry>
              <entry>
                <para>"AWS Regions - userapikey and usersecretkey parameters are not returned in the
                  response of addRegion, updateRegion listRegion api calls..</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1557</para>
              </entry>
              <entry>
                <para>EC2 REST API : cloudbridge database is missing on the CloudStack
                  Installation</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1562</para>
              </entry>
              <entry>
                <para>Replace the short-cut solution of supportting @DB with the formal one</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1565</para>
              </entry>
              <entry>
                <para>"Used Master Branch System VM Template: Default Route on the System VMs
                  (SSVM,CPVM and VR) is missing</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1566</para>
              </entry>
              <entry>
                <para>Baremetal API addBaremetalPxePingServer fail to add PXE PING server to
                  deployment causing create instance with PING style image to fail</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1569</para>
              </entry>
              <entry>
                <para>"AWS Regions - Not able to Edit domain/account/user from a region that is not
                  the owner region.""The content of elements must consist of well-formed character
                  data or markup."" - error message presented to the user.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1571</para>
              </entry>
              <entry>
                <para>"AWS Regions - When deleting domain/account/user from a region that is not the
                  owner, the request is not being forwarded to the owner region.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1574</para>
              </entry>
              <entry>
                <para>updateResourceCount API is failed saying to specify valida resource type even
                  after parsing the valid resource type</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1583</para>
              </entry>
              <entry>
                <para>AWS Regions - RabbitMQ Server did not recieve any event notification during
                  account creation</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1587</para>
              </entry>
              <entry>
                <para>Basic zone - CPVM fail to go to running state, Exception while trying to start
                  secondary storage vm</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1588</para>
              </entry>
              <entry>
                <para>AWS Regions - When registerUserKeys() is called for a user from a region that
                  is not the owner, it is handled by this region.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1600</para>
              </entry>
              <entry>
                <para>Typo in dpkg-buildpackage command</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1604</para>
              </entry>
              <entry>
                <para>deploy VM failed when global setting "vm.allocation.algorithm" is set to
                  "userdispersing</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1615</para>
              </entry>
              <entry>
                <para>"VMware Cluster discovery fails with if ESXi version is 5.0 Update 1, build
                  721882</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1620</para>
              </entry>
              <entry>
                <para>Cannot provision CentOS 6 VMs on XenServer 6.1</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1621</para>
              </entry>
              <entry>
                <para>listProjectInvitations fails with NPE for valid request</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1624</para>
              </entry>
              <entry>
                <para>API is not returning response in details:UI is also not returning any
                  output</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1625</para>
              </entry>
              <entry>
                <para>NPE with updateResourceCount when &amp;&amp; is passed thru API</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1630</para>
              </entry>
              <entry>
                <para>4.0.x cloud-aws-api not properly obsoleted</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1631</para>
              </entry>
              <entry>
                <para>4.1 RPM packaging broken</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1636</para>
              </entry>
              <entry>
                <para>AWS Regions - Remove the concept of having an owner region for
                  domain/account/user objects</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1642</para>
              </entry>
              <entry>
                <para>Add support CentOS 6.4</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1648</para>
              </entry>
              <entry>
                <para>Unable to add KVM host.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1649</para>
              </entry>
              <entry>
                <para>vmware vm os type error</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1651</para>
              </entry>
              <entry>
                <para>agent scripts still pointing to /var/log/cloud</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1656</para>
              </entry>
              <entry>
                <para>NicResponses in a UserVmResponse are not preserving the natural order</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1663</para>
              </entry>
              <entry>
                <para>AWS Regions - Events - There are no events being generated when a new domain
                  is added/edited</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1664</para>
              </entry>
              <entry>
                <para>Action Events are not logged due to spring change</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1665</para>
              </entry>
              <entry>
                <para>AWS Regions - Events - There are no events being generated when a new user is
                  added/edited/enabled/deleted/password changes/api &amp; secret keys are
                  generated</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1666</para>
              </entry>
              <entry>
                <para>KVM VPC NetworkUsage does not work</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1668</para>
              </entry>
              <entry>
                <para>IP conflict in VPC tier</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1671</para>
              </entry>
              <entry>
                <para>AWS Regions - Events - Domain Delete event does not include the UUID of the
                  domain that was deleted</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1674</para>
              </entry>
              <entry>
                <para>AWS Regions - Events - Account Deletion event does not include the UUID of the
                  account deleted</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1681</para>
              </entry>
              <entry>
                <para>Upgrade instructions mention incorrect name and description of systemvm-vmware
                  template in registering template section</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1684</para>
              </entry>
              <entry>
                <para>"api.throttling.enabled configuration setting should be set to ""false"" in
                  Config.java</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1688</para>
              </entry>
              <entry>
                <para>AWS Regions - Domain admin user is not able to use getUser() command to fetch
                  user details</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1690</para>
              </entry>
              <entry>
                <para>NPE from API server when starting mgmt server</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1694</para>
              </entry>
              <entry>
                <para>Issues to start/access Management Server after upgrade from 4.0 to 4.1</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1697</para>
              </entry>
              <entry>
                <para>Six DB tables are not available with upgraded setup(4.0 to 4.1) when compare
                  to 4.1 newly installation</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1706</para>
              </entry>
              <entry>
                <para>Failed to deploy VM with error "cannot find DeployPlannerSelector"</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1709</para>
              </entry>
              <entry>
                <para>AWS Regions - As part of adding a new region, project related entries should
                  not be synced from accounts table.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1710</para>
              </entry>
              <entry>
                <para>AWS Regions - As part of adding a new region,default_zone_id column for the
                  account entries should not be synced.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1711</para>
              </entry>
              <entry>
                <para>AWS Regions - Include all the details of the API call made in the Events
                  payload when changes in Admin/Account/User objects are made.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1713</para>
              </entry>
              <entry>
                <para>EC2 REST API: AWS API Installation Problem</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1714</para>
              </entry>
              <entry>
                <para>Doc section has wrong title: Setting Zone VLAN and Running VM Maximum</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1715</para>
              </entry>
              <entry>
                <para>"Missing ""host"" config setting in docs on management server load
                  balancing</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1716</para>
              </entry>
              <entry>
                <para>"AWS Regions - listRegions(),removeRegions(),updateRegions() should accept
                  UUID value instead of id.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1718</para>
              </entry>
              <entry>
                <para>AWS Regions - removeRegion() response returns updateregionresponse</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1719</para>
              </entry>
              <entry>
                <para>EC2 REST API: AWS APIs are not getting translated on the CloudStack Management
                  Server</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1720</para>
              </entry>
              <entry>
                <para>Have an upgrade path from 4.0.x to 4.1 and 4.0.x to 4.2.0</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1729</para>
              </entry>
              <entry>
                <para>Ensure adapter execution order in runtime</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1733</para>
              </entry>
              <entry>
                <para>[ACS41][UI] Add guest network is missing ip range fields and missing network
                  offering</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1736</para>
              </entry>
              <entry>
                <para>Ubuntu 12.04 cloud-setup-management Failed to configure CloudStack Management
                  Server</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1738</para>
              </entry>
              <entry>
                <para>StatsCollector is not running</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1740</para>
              </entry>
              <entry>
                <para>Failed to view console</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1746</para>
              </entry>
              <entry>
                <para>Cloudstack Usage Server won't start</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1747</para>
              </entry>
              <entry>
                <para>"mvn deploydb only creates 4.0 DB, not 4.1</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1750</para>
              </entry>
              <entry>
                <para>injectkeys script fails on OSX because cp does not have a -b option (backup of
                  destination file</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1761</para>
              </entry>
              <entry>
                <para>Available local storage disk capacity incorrectly reported in KVM to
                  manager</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1764</para>
              </entry>
              <entry>
                <para>ListTemplateCommand failed with java.lang.NumberFormatException and failed to
                  create default template.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1772</para>
              </entry>
              <entry>
                <para>the change in vnc listening port will cause live migration doesn't
                  work.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1773</para>
              </entry>
              <entry>
                <para>Disable baremetal functionality</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1776</para>
              </entry>
              <entry>
                <para>NPE on listSecondaryStorageHostsInAllZones in Upgraded setup from 4.0 to
                  4.1.0</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1785</para>
              </entry>
              <entry>
                <para>Redundant Router test cases failing during automation run.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1789</para>
              </entry>
              <entry>
                <para>Unable to download templates to Primary Storage if a host is in
                  maintenance.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1791</para>
              </entry>
              <entry>
                <para>Volumes with storage tags can't be attached.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1792</para>
              </entry>
              <entry>
                <para>"AWS Regions - RuntimeException while executing listAccounts(), when the
                  encryption keys are set to different values between regions.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1793</para>
              </entry>
              <entry>
                <para>L10n docs don't build in chinese, portuguese and japanese</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1795</para>
              </entry>
              <entry>
                <para>Customize AOP to fully support legacy CloudStack @DB and @ActionEvent
                  semantics.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1796</para>
              </entry>
              <entry>
                <para>Japanese docs don't build.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1802</para>
              </entry>
              <entry>
                <para>Upgrade 4.0 -> 4.1 - Not able to start management server becasue of missing
                  /etc/cloudstack/management/tomcat6.conf file</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1804</para>
              </entry>
              <entry>
                <para>Upgrade 4.0 -> 4.1 - DB upgrade fails</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1805</para>
              </entry>
              <entry>
                <para>com.mysql.jdbc.exceptions.jdbc4.CommunicationsException seen after long time
                  of inactivity resulting in not being able to log in to the management
                  server</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1810</para>
              </entry>
              <entry>
                <para>listTemplate API with templatefilter=featured|community is not returning any
                  lists</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1811</para>
              </entry>
              <entry>
                <para>"Upgrade 4.0->4.1 - When upgrade scripts fail, component loading continues and
                  management server starts.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1812</para>
              </entry>
              <entry>
                <para>create physical network fails while creating basic zone</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1825</para>
              </entry>
              <entry>
                <para>EC2 REST API: AWS APIs fail to execute due to BeanCreationException: Error
                  creating bean with name 'SAclDaoImpl'</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1826</para>
              </entry>
              <entry>
                <para>"Storage migration not working, seemingly due to uuid vs id</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1827</para>
              </entry>
              <entry>
                <para>Redundant router - When VR Master was stopped failover to VR Backup did not
                  occur.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1834</para>
              </entry>
              <entry>
                <para>"Events are not generated for registerUserKeys(), Enabling account and Editing
                  account.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1836</para>
              </entry>
              <entry>
                <para>License header failures for ja-JP .po translation file</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1839</para>
              </entry>
              <entry>
                <para>Upgrade 4.0 -> 4.1 - Upgraded DB has lot more keys and indexes for many tables
                  compare to the fresh installed 4.1 DB</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1841</para>
              </entry>
              <entry>
                <para>ASF 4.0 to 4.1 Upgrade: Missing Few Global Configuration parameters on the
                  Upgraded Setup.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1842</para>
              </entry>
              <entry>
                <para>ASF 4.0 to 4.1 Upgrade: Missing Ubuntu 12.04 Guest OS Types on the Upgraded
                  Setup.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1844</para>
              </entry>
              <entry>
                <para>Upgrade 4.0 -> 4.1 - KVM host agent.properties is not restored as part of
                  upgrading the binaries from 4.0 to 4.1.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1845</para>
              </entry>
              <entry>
                <para>KVM - storage migration often fails</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1846</para>
              </entry>
              <entry>
                <para>"KVM - storage pools can silently fail to be unregistered, leading to failure
                  to register later.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1848</para>
              </entry>
              <entry>
                <para>Cloudstack Packages are not got updated with scenario 4.0 to 4.1 upgrade where
                  MS is on Ubuntu 12.04.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1856</para>
              </entry>
              <entry>
                <para>Upgrade 4.0 -> 4.1 - Fresh install of 4.1 has 3 parameters missing in
                  db.properties compared to an upgraded 4.0 setup</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1873</para>
              </entry>
              <entry>
                <para>"Installation : JasyptPBEStringDecryptionCLI missing, failed to decrypt db
                  password</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1874</para>
              </entry>
              <entry>
                <para>AWS Regions - Account table in cloud_usage DB has region_id</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1876</para>
              </entry>
              <entry>
                <para>External Devices - network offering for external devices is not returned in
                  API listNetworkOfferings when creating instances.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1877</para>
              </entry>
              <entry>
                <para>Failed to connect to DB while starting Ubuntu management server after
                  upgrading the packages from 4.0 to 4.1.0</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1882</para>
              </entry>
              <entry>
                <para>“HTTP Status 404 。 The requested resource () is not available.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1890</para>
              </entry>
              <entry>
                <para>listProjects is not listing state in the response</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1900</para>
              </entry>
              <entry>
                <para>"Upgrade 4.0 -> 4.1, We do not have a copy of db.properties that comes from a
                  4.1 installation saved anywhere.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1929</para>
              </entry>
              <entry>
                <para>ASF 4.1 cloudstack agent fail to install in KVM host CENTOS 6.3 OS:
                  qemu-kvm-0.12.1.2-3.295.el6.10.x86_64 requires libusbredirparser.so.0</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1934</para>
              </entry>
              <entry>
                <para>NPE with listSupportedNetworkServices after upgrade from 4.0 to 4.1 (Ubuntu
                  MS)</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1935</para>
              </entry>
              <entry>
                <para>Cloud utilities are not renamed to Cloudstack after upgrade from 4.0 to 4.1
                  [Ubutnu MS]</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1936</para>
              </entry>
              <entry>
                <para>On CentOS, after a upgrade from 4.0.1 to 4.1 on a cloud node (cloud-agent),
                  the new cloustack-agent isn't add as a service (chkconfig)</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1951</para>
              </entry>
              <entry>
                <para>centos packaging: cloud-install-sys-tmplt can't find jasypt jar.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1971</para>
              </entry>
              <entry>
                <para>VM deployed to incorrect primary storage.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1972</para>
              </entry>
              <entry>
                <para>VM deployed to incorrect primary storage.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1978</para>
              </entry>
              <entry>
                <para>openvswitch - unable to start console session for SSVM CPVM user VM</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1980</para>
              </entry>
              <entry>
                <para>"[4.1]cloudstack-setup-bridge, cloudstack-setup-encryption &amp;
                  cloudstack-sysvmadm utilities are not available in Ubuntu 12.04 Management
                  Server.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1987</para>
              </entry>
              <entry>
                <para>Deleted service offerings owned by a domain show up to domain user.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1988</para>
              </entry>
              <entry>
                <para>AWS API using SOAP client - User Registration fails</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-1989</para>
              </entry>
              <entry>
                <para>"Query service offering by ID returns no result, but querying all returns
                  service offering</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-2003</para>
              </entry>
              <entry>
                <para>Deleting domain while deleted account is cleaning up leaves VMs expunging
                  forever due to 'Failed to update resource count</para>
              </entry>
            </row>
            <row>
              <entry>
                <para>CLOUDSTACK-2007</para>
              </entry>
              <entry>
                <para>Release Notes failing to build on jenkins.cs.</para>
              </entry>
            </row>
          </tbody>
        </tgroup>
      </informaltable>
    </section>
    <section id="known-issues-4.1">
      <title>Known Issues in 4.1.0</title>
      <informaltable>
        <tgroup cols="2" align="left" colsep="1" rowsep="1">
          <colspec colwidth="1*" colname="1" colnum="1"/>
          <colspec colwidth="2*" colname="2" colnum="2"/>
          <thead>
            <row>
              <entry>
                <para>Issue ID</para>
              </entry>
              <entry>
                <para>Description</para>
              </entry>
            </row>
          </thead>
          <tbody>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-2709"
                    >CLOUDSTACK-2709</ulink></para></entry>
              <entry>
                <para>Egress rules are are not supported on shared networks. </para>
              </entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-1747"
                    >CLOUDSTACK-1747</ulink></para></entry>
              <entry><para>mvn deploydb only creates 4.0 DB, not 4.1</para>
                <para>Due to tooling changes between 4.1 and 4.2, CloudStack's database is created
                  using the 4.0 schema and updated to the 4.1 schema when the management server
                  starts for the first time. It's OK to see the same schema if the management server
                  has not started yet.</para>
              </entry>
            </row>
            <row>
              <entry><para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-1824"
                    >CLOUDSTACK-1824</ulink></para></entry>
              <entry><para>Service CloudStack-Management is being displayed as cloud-management
                  service </para>
                <para>Many scripts and text entries have references to cloud-management rather than
                  cloudstack-management due to the changeover between 4.0 and 4.1 to rename
                  services. This is a minor issue and should be corrected by 4.2.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-1824"
                    >CLOUDSTACK-1824</ulink></para></entry>
              <entry><para>Service CloudStack-Management is being displayed as cloud-management
                  service</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-1510"
                    >CLOUDSTACK-1510</ulink></para>
              </entry>
              <entry>
                <para>NPE when primary storage is added with wrong path</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-1428"
                    >CLOUDSTACK-1428</ulink></para>
              </entry>
              <entry>
                <para>[UI] Instance which are created without display name are not visible when
                  added to LB</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-1306"
                    >CLOUDSTACK-1306</ulink></para>
              </entry>
              <entry>
                <para>Better Error message when trying to deploy Vm by passing static Ipv4 addresses
                  that are assigned to another VM/IP4 address is outside the iprange.</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-1236"
                    >CLOUDSTACK-1236</ulink></para>
              </entry>
              <entry>
                <para>Warning while adding Xen 6.1 host [Unable to create local link network]</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-969"
                    >CLOUDSTACK-969</ulink></para>
              </entry>
              <entry>
                <para>api: zone response lists vlan in it as &quot;vlan range of zone&quot; but the
                  vlan belongs to physical network</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-963"
                    >CLOUDSTACK-963</ulink></para>
              </entry>
              <entry>
                <para>[cloud.utils.AnnotationHelper] class java.lang.Stringdoes not have a Table
                  annotation</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-458"
                    >CLOUDSTACK-458</ulink></para>
              </entry>
              <entry>
                <para>xen:snapshots:Storage gc fail to clean the failed snapshot images from
                  secondarystorage</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-315"
                    >CLOUDSTACK-315</ulink></para>
              </entry>
              <entry>
                <para>Infrastructure view does not show capacity values</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-300"
                    >CLOUDSTACK-300</ulink></para>
              </entry>
              <entry>
                <para>Creation of compute offering allow combination of local storage + HA</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-282"
                    >CLOUDSTACK-282</ulink></para>
              </entry>
              <entry>
                <para>Virtual Routers do not properly resolve DNS SRV Records</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-276"
                    >CLOUDSTACK-276</ulink></para>
              </entry>
              <entry>
                <para>SSVM ID is exposed in the Error Message thrown by AddTrafficType API</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-270"
                    >CLOUDSTACK-270</ulink></para>
              </entry>
              <entry>
                <para>Ui should not ask for a vlan range if the physical network isolation type is
                  not VLAN</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-245"
                    >CLOUDSTACK-245</ulink></para>
              </entry>
              <entry>
                <para>VPC ACLs are not stored and programmed consistently</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-231"
                    >CLOUDSTACK-231</ulink></para>
              </entry>
              <entry>
                <para>Tag creation using special charecters</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-124"
                    >CLOUDSTACK-124</ulink></para>
              </entry>
              <entry>
                <para>NetworkGarbageCollector not cleaning up networks</para>
              </entry>
            </row>
            <row>
              <entry>
                <para><ulink url="https://issues.apache.org/jira/browse/CLOUDSTACK-62"
                    >CLOUDSTACK-62</ulink></para>
              </entry>
              <entry>
                <para>console proxy does not support any keymaps besides us, jp</para>
              </entry>
            </row>
          </tbody>
        </tgroup>
      </informaltable>
    </section>
  </chapter>
  <chapter id="upgrade-instructions">
    <title>Upgrade Instructions for 4.1</title>
    <para>This section contains upgrade instructions from prior versions of CloudStack to Apache
      CloudStack 4.1.0. We include instructions on upgrading to Apache CloudStack from pre-Apache
      versions of Citrix CloudStack (last version prior to Apache is 3.0.2) and from the releases
      made while CloudStack was in the Apache Incubator.</para>
    <para>If you run into any issues during upgrades, please feel free to ask questions on
      users@cloudstack.apache.org or dev@cloudstack.apache.org.</para>
    <section id="upgrade-from-4.0-to-4.1">
      <title>Upgrade from 4.0.x to 4.1.0</title>
      <para>This section will guide you from &PRODUCT; 4.0.x versions to &PRODUCT; 4.1.0.</para>
      <para>Any steps that are hypervisor-specific will be called out with a note.</para>
      <warning>
        <title>Package Structure Changes</title>
        <para>The package structure for &PRODUCT; has changed significantly since the 4.0.x
          releases. If you've compiled your own packages, you'll notice that the package names and
          the number of packages has changed. This is <emphasis>not</emphasis> a bug.</para>
        <para>However, this <emphasis>does</emphasis> mean that the procedure is not as simple as an
            <command>apt-get upgrade</command> or <command>yum update</command>, so please follow
          this section carefully.</para>
      </warning>
      <para>We recommend reading through this section once or twice before beginning your upgrade
        procedure, and working through it on a test system before working on a production
        system.</para>
      <orderedlist>
        <listitem>
          <para>Most users of &PRODUCT; manage the installation and upgrades of &PRODUCT; with one
            of Linux's predominant package systems, RPM or APT. This guide assumes you'll be using
            RPM and Yum (for Red Hat Enterprise Linux or CentOS), or APT and Debian packages (for
            Ubuntu).</para>
          <para>Create RPM or Debian packages (as appropriate) and a repository from the 4.1.0
            source, or check the Apache CloudStack downloads page at <ulink
              url="http://cloudstack.apache.org/downloads.html"
              >http://cloudstack.apache.org/downloads.html</ulink> for package repositories supplied
            by community members. You will need them for step <xref linkend="upgrade-deb-packages"/>
            or step <xref linkend="upgrade-rpm-packages"/>.</para>
          <para>Instructions for creating packages from the &PRODUCT; source are in the <ulink
              url="http://cloudstack.apache.org/docs/en-US/index.html">Installation
            Guide</ulink>.</para>
        </listitem>
        <listitem>
          <para>Stop your management server or servers. Run this on all management server
            hosts:</para>
          <programlisting><prompt>#</prompt> service cloud-management stop</programlisting>
        </listitem>
        <listitem>
          <para>If you are running a usage server or usage servers, stop those as well:</para>
          <programlisting><prompt>#</prompt> service cloud-usage stop</programlisting>
        </listitem>
        <listitem>
          <para>Make a backup of your MySQL database. If you run into any issues or need to roll
            back the upgrade, this will assist in debugging or restoring your existing environment.
            You'll be prompted for your password.</para>
          <programlisting><prompt>#</prompt> mysqldump -u root -p cloud &gt; cloudstack-backup.sql</programlisting>
        </listitem>
        <listitem>
          <para>If you have made changes to
              <filename>/etc/cloud/management/components.xml</filename>, you'll need to carry these
            over manually to the new file,
              <filename>/etc/cloudstack/management/componentContext.xml</filename>. This is not done
            automatically. (If you're unsure, we recommend making a backup of the original
              <filename>components.xml</filename> to be on the safe side.</para>
        </listitem>
        <listitem>
          <para>After upgrading to 4.1, API clients are expected to send plain text passwords for
            login and user creation, instead of MD5 hash. Incase, api client changes are not
            acceptable, following changes are to be made for backward compatibility:</para>
          <para>Modify componentsContext.xml, and make PlainTextUserAuthenticator as the default
            authenticator (1st entry in the userAuthenticators adapter list is default)</para>
          <programlisting language="XML">
&lt;!-- Security adapters --&gt;
&lt;bean id="userAuthenticators" class="com.cloud.utils.component.AdapterList"&gt;
  &lt;property name="Adapters"&gt;
    &lt;list&gt;
      &lt;ref bean="PlainTextUserAuthenticator"/&gt;
      &lt;ref bean="MD5UserAuthenticator"/&gt;
      &lt;ref bean="LDAPUserAuthenticator"/&gt;
    &lt;/list&gt;
  &lt;/property&gt;
&lt;/bean&gt;
            </programlisting>
          <para>PlainTextUserAuthenticator works the same way MD5UserAuthenticator worked prior to
            4.1.</para>
        </listitem>
        <listitem id="upgrade-deb-packages">
          <para>If you are using Ubuntu, follow this procedure to upgrade your packages. If not,
            skip to step <xref linkend="upgrade-rpm-packages"/>.</para>
          <note>
            <title>Community Packages</title>
            <para>This section assumes you're using the community supplied packages for &PRODUCT;.
              If you've created your own packages and APT repository, substitute your own URL for
              the ones used in these examples.</para>
          </note>
          <orderedlist id="debsteps">
            <listitem>
              <para>The first order of business will be to change the sources list for each system
                with &PRODUCT; packages. This means all management servers, and any hosts that have
                the KVM agent. (No changes should be necessary for hosts that are running VMware or
                Xen.)</para>
              <para>Start by opening <filename>/etc/apt/sources.list.d/cloudstack.list</filename> on
                any systems that have &PRODUCT; packages installed.</para>
              <para>This file should have one line, which contains:</para>
              <programlisting language="Bash">deb http://cloudstack.apt-get.eu/ubuntu precise 4.0</programlisting>
              <para>We'll change it to point to the new package repository:</para>
              <programlisting language="Bash">deb http://cloudstack.apt-get.eu/ubuntu precise 4.1</programlisting>
              <para>If you're using your own package repository, change this line to read as
                appropriate for your 4.1.0 repository.</para>
            </listitem>
            <listitem>
              <para>Now update your apt package list:</para>
              <programlisting language="Bash">$ sudo apt-get update</programlisting>
            </listitem>
            <listitem id="deb-master">
              <para>Now that you have the repository configured, it's time to install the
                  <filename>cloudstack-management</filename> package. This will pull in any other
                dependencies you need.</para>
              <programlisting language="Bash">$ sudo apt-get install cloudstack-management</programlisting>
            </listitem>
            <listitem id="kvm-agent-deb">
              <para>You will need to manually install the <filename>cloudstack-agent</filename>
                package:</para>
              <programlisting language="Bash">$ sudo apt-get install cloudstack-agent</programlisting>
              <para>During the installation of <filename>cloudstack-agent</filename>, APT will copy
                your <filename>agent.properties</filename>, <filename>log4j-cloud.xml</filename>,
                and <filename>environment.properties</filename> from
                  <filename>/etc/cloud/agent</filename> to
                  <filename>/etc/cloudstack/agent</filename>.</para>
              <para>When prompted whether you wish to keep your configuration, say Yes.</para>
            </listitem>
            <listitem>
              <para>Verify that the file
                  <filename>/etc/cloudstack/agent/environment.properties</filename> has a line that
                reads:</para>
              <programlisting language="Bash">paths.script=/usr/share/cloudstack-common</programlisting>
              <para>If not, add the line.</para>
            </listitem>
            <listitem>
              <para>Restart the agent:</para>
              <programlisting language="Bash">
service cloud-agent stop
killall jsvc
service cloudstack-agent start
                            </programlisting>
            </listitem>
            <listitem>
              <para>During the upgrade, <filename>log4j-cloud.xml</filename> was simply copied over,
                so the logs will continue to be added to
                  <filename>/var/log/cloud/agent/agent.log</filename>. There's nothing
                  <emphasis>wrong</emphasis> with this, but if you prefer to be consistent, you can
                change this by copying over the sample configuration file:</para>
              <programlisting language="Bash">
cd /etc/cloudstack/agent
mv log4j-cloud.xml.dpkg-dist log4j-cloud.xml
service cloudstack-agent restart
                            </programlisting>
            </listitem>
            <listitem>
              <para>Once the agent is running, you can uninstall the old cloud-* packages from your
                system:</para>
              <programlisting language="Bash">sudo dpkg --purge cloud-agent</programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="upgrade-rpm-packages">
          <para>If you are using CentOS or RHEL, follow this procedure to upgrade your packages. If
            not, skip to step <xref linkend="restart-system-vms"/>.</para>
          <note>
            <title>Community Packages</title>
            <para>This section assumes you're using the community supplied packages for &PRODUCT;.
              If you've created your own packages and yum repository, substitute your own URL for
              the ones used in these examples.</para>
          </note>
          <orderedlist id="rpmsteps">
            <listitem>
              <para>The first order of business will be to change the yum repository for each system
                with &PRODUCT; packages. This means all management servers, and any hosts that have
                the KVM agent. (No changes should be necessary for hosts that are running VMware or
                Xen.)</para>
              <para>Start by opening <filename>/etc/yum.repos.d/cloudstack.repo</filename> on any
                systems that have &PRODUCT; packages installed.</para>
              <para>This file should have content similar to the following:</para>
              <programlisting language="Bash">
[apache-cloudstack]
name=Apache CloudStack
baseurl=http://cloudstack.apt-get.eu/rhel/4.0/
enabled=1
gpgcheck=0
                            </programlisting>
              <para>If you are using the community provided package repository, change the baseurl
                to http://cloudstack.apt-get.eu/rhel/4.1/</para>
              <para>If you're using your own package repository, change this line to read as
                appropriate for your 4.1.0 repository.</para>
            </listitem>
            <listitem id="rpm-master">
              <para>Now that you have the repository configured, it's time to install the
                  <filename>cloudstack-management</filename> package by upgrading the older
                  <filename>cloud-client</filename> package.</para>
              <programlisting language="Bash">$ sudo yum upgrade cloud-client</programlisting>
            </listitem>
            <listitem id="kvm-agent-rpm">
              <para>For KVM hosts, you will need to upgrade the <filename>cloud-agent</filename>
                package, similarly installing the new version as
                  <filename>cloudstack-agent</filename>.</para>
              <programlisting language="Bash">$ sudo yum upgrade cloud-agent</programlisting>
              <para>During the installation of <filename>cloudstack-agent</filename>, the RPM will
                copy your <filename>agent.properties</filename>,
                  <filename>log4j-cloud.xml</filename>, and
                  <filename>environment.properties</filename> from
                  <filename>/etc/cloud/agent</filename> to
                  <filename>/etc/cloudstack/agent</filename>.</para>
            </listitem>
            <listitem>
              <para>Verify that the file
                  <filename>/etc/cloudstack/agent/environment.properties</filename> has a line that
                reads:</para>
              <programlisting language="Bash">paths.script=/usr/share/cloudstack-common</programlisting>
              <para>If not, add the line.</para>
            </listitem>
            <listitem>
              <para>Restart the agent:</para>
              <programlisting language="Bash">
service cloud-agent stop
killall jsvc
service cloudstack-agent start
                            </programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="restart-system-vms">
          <para>Once you've upgraded the packages on your management servers, you'll need to restart
            the system VMs. Make sure port 8096 is open in your local host firewall to do
            this.</para>
          <para>There is a script that will do this for you, all you need to do is run the script
            and supply the IP address for your MySQL instance and your MySQL credentials:</para>
          <programlisting language="Bash"><prompt>#</prompt> nohup cloudstack-sysvmadm -d <replaceable>IP address</replaceable> -u cloud -p -a &gt; sysvm.log 2&gt;&amp;1 &amp;</programlisting>
          <para>You can monitor the log for progress. The process of restarting the system VMs can
            take an hour or more.</para>
          <programlisting language="Bash"><prompt>#</prompt> tail -f sysvm.log</programlisting>
          <para>The output to <filename>sysvm.log</filename> will look something like this:</para>
          <programlisting language="Bash">
Stopping and starting 1 secondary storage vm(s)...
Done stopping and starting secondary storage vm(s)
Stopping and starting 1 console proxy vm(s)...
Done stopping and starting console proxy vm(s).
Stopping and starting 4 running routing vm(s)...
Done restarting router(s).
                    </programlisting>
        </listitem>
        <listitem>
          <note>
            <title>For Xen Hosts: Copy vhd-utils</title>
            <para>This step is only for CloudStack installs that are using Xen hosts.</para>
          </note>
          <para>Copy the file <filename>vhd-utils</filename> to
              <filename>/usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver</filename>.</para>
        </listitem>
      </orderedlist>
    </section>
    <section id="upgrade-from-3.0.2-to-4.0">
      <title>Upgrade from 3.0.2 to 4.1.0</title>
      <para>This section will guide you from Citrix CloudStack 3.0.2 to Apache CloudStack 4.1.0.
        Sections that are hypervisor-specific will be called out with a note.</para>
      <orderedlist>
        <listitem>
          <note>
            <para>The following upgrade instructions apply only if you're using VMware hosts. If
              you're not using VMware hosts, skip this step and move on to <xref
                linkend="stopping-usage-servers"/>.</para>
          </note>
          <para>In each zone that includes VMware hosts, you need to add a new system VM template. </para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>While running the existing 3.0.2 system, log in to the UI as root
                administrator.</para>
            </listitem>
            <listitem>
              <para>In the left navigation bar, click Templates.</para>
            </listitem>
            <listitem>
              <para>In Select view, click Templates.</para>
            </listitem>
            <listitem>
              <para>Click Register template.</para>
              <para>The Register template dialog box is displayed.</para>
            </listitem>
            <listitem>
              <para>In the Register template dialog box, specify the following values (do not change
                these):</para>
              <informaltable>
                <tgroup cols="2" align="left" colsep="1" rowsep="1">
                  <colspec colwidth="1*" colname="1" colnum="1"/>
                  <colspec colwidth="2*" colname="2" colnum="2"/>
                  <thead>
                    <row>
                      <entry><para>Field</para></entry>
                      <entry><para>Value</para></entry>
                    </row>
                  </thead>
                  <tbody>
                    <row>
                      <entry><para>Name</para></entry>
                      <entry><para>systemvm-vmware-4.1</para></entry>
                    </row>
                    <row>
                      <entry><para>Description</para></entry>
                      <entry><para>systemvm-vmware-4.1</para></entry>
                    </row>
                    <row>
                      <entry><para>URL</para></entry>
                      <entry><para>http://download.cloud.com/templates/burbank/burbank-systemvm-08012012.ova</para></entry>
                    </row>
                    <row>
                      <entry><para>Zone</para></entry>
                      <entry><para>Choose the zone where this hypervisor is used</para></entry>
                    </row>
                    <row>
                      <entry><para>Hypervisor</para></entry>
                      <entry><para>VMware</para></entry>
                    </row>
                    <row>
                      <entry><para>Format</para></entry>
                      <entry><para>OVA</para></entry>
                    </row>
                    <row>
                      <entry><para>OS Type</para></entry>
                      <entry><para>Debian GNU/Linux 5.0 (32-bit)</para></entry>
                    </row>
                    <row>
                      <entry><para>Extractable</para></entry>
                      <entry><para>no</para></entry>
                    </row>
                    <row>
                      <entry><para>Password Enabled</para></entry>
                      <entry><para>no</para></entry>
                    </row>
                    <row>
                      <entry><para>Public</para></entry>
                      <entry><para>no</para></entry>
                    </row>
                    <row>
                      <entry><para>Featured</para></entry>
                      <entry><para>no</para></entry>
                    </row>
                  </tbody>
                </tgroup>
              </informaltable>
            </listitem>
            <listitem>
              <para>Watch the screen to be sure that the template downloads successfully and enters
                the READY state. Do not proceed until this is successful.</para>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="stopping-usage-servers">
          <para>Stop all Usage Servers if running. Run this on all Usage Server hosts.</para>
          <programlisting language="Bash"><prompt>#</prompt> service cloud-usage stop</programlisting>
        </listitem>
        <listitem>
          <para>Stop the Management Servers. Run this on all Management Server hosts.</para>
          <programlisting language="Bash"><prompt>#</prompt> service cloud-management stop</programlisting>
        </listitem>
        <listitem>
          <para>On the MySQL master, take a backup of the MySQL databases. We recommend performing
            this step even in test upgrades. If there is an issue, this will assist with
            debugging.</para>
          <para>In the following commands, it is assumed that you have set the root password on the
            database, which is a CloudStack recommended best practice. Substitute your own MySQL
            root password.</para>
          <programlisting><prompt>#</prompt> <command>mysqldump</command> -u root -p<replaceable>mysql_password</replaceable> cloud > <filename>cloud-backup.dmp</filename>
                        <prompt>#</prompt> <command>mysqldump</command> -u root -p<replaceable>mysql_password</replaceable> cloud_usage > <filename>cloud-usage-backup.dmp</filename></programlisting>
        </listitem>
        <listitem>
          <para>Either build RPM/DEB packages as detailed in the Installation Guide, or use one of
            the community provided yum/apt repositories to gain access to the &PRODUCT;
            binaries.</para>
        </listitem>
        <listitem id="upgrade-deb-packages-302">
          <para>If you are using Ubuntu, follow this procedure to upgrade your packages. If not,
            skip to step <xref linkend="upgrade-rpm-packages-302"/>.</para>
          <note>
            <title>Community Packages</title>
            <para>This section assumes you're using the community supplied packages for &PRODUCT;.
              If you've created your own packages and APT repository, substitute your own URL for
              the ones used in these examples.</para>
          </note>
          <orderedlist id="debsteps-302">
            <listitem>
              <para>The first order of business will be to change the sources list for each system
                with &PRODUCT; packages. This means all management servers, and any hosts that have
                the KVM agent. (No changes should be necessary for hosts that are running VMware or
                Xen.)</para>
              <para>Start by opening <filename>/etc/apt/sources.list.d/cloudstack.list</filename> on
                any systems that have &PRODUCT; packages installed.</para>
              <para>This file should have one line, which contains:</para>
              <programlisting language="Bash">deb http://cloudstack.apt-get.eu/ubuntu precise 4.0</programlisting>
              <para>We'll change it to point to the new package repository:</para>
              <programlisting language="Bash">deb http://cloudstack.apt-get.eu/ubuntu precise 4.1</programlisting>
              <para>If you're using your own package repository, change this line to read as
                appropriate for your 4.1.0 repository.</para>
            </listitem>
            <listitem>
              <para>Now update your apt package list:</para>
              <programlisting language="Bash">$ sudo apt-get update</programlisting>
            </listitem>
            <listitem id="deb-master-302">
              <para>Now that you have the repository configured, it's time to install the
                  <filename>cloudstack-management</filename> package. This will pull in any other
                dependencies you need.</para>
              <programlisting language="Bash">$ sudo apt-get install cloudstack-management</programlisting>
            </listitem>
            <listitem id="kvm-agent-deb-302">
              <para>You will need to manually install the <filename>cloudstack-agent</filename>
                package:</para>
              <programlisting language="Bash">$ sudo apt-get install cloudstack-agent</programlisting>
              <para>During the installation of <filename>cloudstack-agent</filename>, APT will copy
                your <filename>agent.properties</filename>, <filename>log4j-cloud.xml</filename>,
                and <filename>environment.properties</filename> from
                  <filename>/etc/cloud/agent</filename> to
                  <filename>/etc/cloudstack/agent</filename>.</para>
              <para>When prompted whether you wish to keep your configuration, say Yes.</para>
            </listitem>
            <listitem>
              <para>Verify that the file
                  <filename>/etc/cloudstack/agent/environment.properties</filename> has a line that
                reads:</para>
              <programlisting language="Bash">paths.script=/usr/share/cloudstack-common</programlisting>
              <para>If not, add the line.</para>
            </listitem>
            <listitem>
              <para>Restart the agent:</para>
              <programlisting language="Bash">
service cloud-agent stop
killall jsvc
service cloudstack-agent start
                            </programlisting>
            </listitem>
            <listitem>
              <para>During the upgrade, <filename>log4j-cloud.xml</filename> was simply copied over,
                so the logs will continue to be added to
                  <filename>/var/log/cloud/agent/agent.log</filename>. There's nothing
                  <emphasis>wrong</emphasis> with this, but if you prefer to be consistent, you can
                change this by copying over the sample configuration file:</para>
              <programlisting language="Bash">
cd /etc/cloudstack/agent
mv log4j-cloud.xml.dpkg-dist log4j-cloud.xml
service cloudstack-agent restart
                            </programlisting>
            </listitem>
            <listitem>
              <para>Once the agent is running, you can uninstall the old cloud-* packages from your
                system:</para>
              <programlisting language="Bash">sudo dpkg --purge cloud-agent</programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="upgrade-rpm-packages-302">
          <para>If you are using CentOS or RHEL, follow this procedure to upgrade your packages. If
            not, skip to step <xref linkend="correct-components-xml-302"/>.</para>
          <note>
            <title>Community Packages</title>
            <para>This section assumes you're using the community supplied packages for &PRODUCT;.
              If you've created your own packages and yum repository, substitute your own URL for
              the ones used in these examples.</para>
          </note>
          <orderedlist id="rpmsteps-302">
            <listitem>
              <para>The first order of business will be to change the yum repository for each system
                with &PRODUCT; packages. This means all management servers, and any hosts that have
                the KVM agent. (No changes should be necessary for hosts that are running VMware or
                Xen.)</para>
              <para>Start by opening <filename>/etc/yum.repos.d/cloudstack.repo</filename> on any
                systems that have &PRODUCT; packages installed.</para>
              <para>This file should have content similar to the following:</para>
              <programlisting language="Bash">
[apache-cloudstack]
name=Apache CloudStack
baseurl=http://cloudstack.apt-get.eu/rhel/4.0/
enabled=1
gpgcheck=0
                            </programlisting>
              <para>If you are using the community provided package repository, change the baseurl
                to http://cloudstack.apt-get.eu/rhel/4.1/</para>
              <para>If you're using your own package repository, change this line to read as
                appropriate for your 4.1.0 repository.</para>
            </listitem>
            <listitem id="rpm-master-302">
              <para>Now that you have the repository configured, it's time to install the
                  <filename>cloudstack-management</filename> package by upgrading the older
                  <filename>cloud-client</filename> package.</para>
              <programlisting language="Bash">$ sudo yum upgrade cloud-client</programlisting>
            </listitem>
            <listitem id="kvm-agent-rpm-302">
              <para>For KVM hosts, you will need to upgrade the <filename>cloud-agent</filename>
                package, similarly installing the new version as
                  <filename>cloudstack-agent</filename>.</para>
              <programlisting language="Bash">$ sudo yum upgrade cloud-agent</programlisting>
              <para>During the installation of <filename>cloudstack-agent</filename>, the RPM will
                copy your <filename>agent.properties</filename>,
                  <filename>log4j-cloud.xml</filename>, and
                  <filename>environment.properties</filename> from
                  <filename>/etc/cloud/agent</filename> to
                  <filename>/etc/cloudstack/agent</filename>.</para>
            </listitem>
            <listitem>
              <para>Verify that the file
                  <filename>/etc/cloudstack/agent/environment.properties</filename> has a line that
                reads:</para>
              <programlisting language="Bash">paths.script=/usr/share/cloudstack-common</programlisting>
              <para>If not, add the line.</para>
            </listitem>
            <listitem>
              <para>Restart the agent:</para>
              <programlisting language="Bash">
service cloud-agent stop
killall jsvc
service cloudstack-agent start
                            </programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="correct-components-xml-302">
          <para>If you have made changes to your copy of
              <filename>/etc/cloud/management/components.xml</filename> the changes will be
            preserved in the upgrade. However, you need to do the following steps to place these
            changes in a new version of the file which is compatible with version 4.1.0.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Make a backup copy of <filename>/etc/cloud/management/components.xml</filename>.
                For example:</para>
              <programlisting># mv /etc/cloud/management/components.xml /etc/cloud/management/components.xml-backup</programlisting>
            </listitem>
            <listitem>
              <para>Copy <filename>/etc/cloud/management/components.xml.rpmnew</filename> to create
                a new <filename>/etc/cloud/management/components.xml</filename>:</para>
              <programlisting># cp -ap /etc/cloud/management/components.xml.rpmnew /etc/cloud/management/components.xml</programlisting>
            </listitem>
            <listitem>
              <para>Merge your changes from the backup file into the new
                  <filename>components.xml</filename>.</para>
              <programlisting># vi /etc/cloud/management/components.xml</programlisting>
            </listitem>
          </orderedlist>
          <note>
            <para>If you have more than one management server node, repeat the upgrade steps on each
              node.</para>
          </note>
        </listitem>
        <listitem>
          <para>After upgrading to 4.1, API clients are expected to send plain text passwords for
            login and user creation, instead of MD5 hash. Incase, api client changes are not
            acceptable, following changes are to be made for backward compatibility:</para>
          <para>Modify componentsContext.xml, and make PlainTextUserAuthenticator as the default
            authenticator (1st entry in the userAuthenticators adapter list is default)</para>
          <programlisting language="XML">
&lt;!-- Security adapters --&gt;
&lt;bean id="userAuthenticators" class="com.cloud.utils.component.AdapterList"&gt;
  &lt;property name="Adapters"&gt;
    &lt;list&gt;
      &lt;ref bean="PlainTextUserAuthenticator"/&gt;
      &lt;ref bean="MD5UserAuthenticator"/&gt;
      &lt;ref bean="LDAPUserAuthenticator"/&gt;
    &lt;/list&gt;
  &lt;/property&gt;
&lt;/bean&gt;
            </programlisting>
          <para>PlainTextUserAuthenticator works the same way MD5UserAuthenticator worked prior to
            4.1.</para>
        </listitem>
        <listitem>
          <para>Start the first Management Server. Do not start any other Management Server nodes
            yet.</para>
          <programlisting language="Bash"><prompt>#</prompt> service cloudstack-management start</programlisting>
          <para>Wait until the databases are upgraded. Ensure that the database upgrade is complete.
            After confirmation, start the other Management Servers one at a time by running the same
            command on each node.</para>
          <note>
            <para>Failing to restart the Management Server indicates a problem in the upgrade.
              Having the Management Server restarted without any issues indicates that the upgrade
              is successfully completed.</para>
          </note>
        </listitem>
        <listitem>
          <para>Start all Usage Servers (if they were running on your previous version). Perform
            this on each Usage Server host.</para>
          <para><command># service cloudstack-usage start</command></para>
        </listitem>
        <listitem>
          <note>
            <para>Additional steps are required for each KVM host. These steps will not affect
              running guests in the cloud. These steps are required only for clouds using KVM as
              hosts and only on the KVM hosts.</para>
          </note>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Configure a yum or apt respository containing the &PRODUCT; packages as outlined
                in the Installation Guide.</para>
            </listitem>
            <listitem>
              <para>Stop the running agent.</para>
              <para><command># service cloud-agent stop</command></para>
            </listitem>
            <listitem>
              <para>Update the agent software with one of the following command sets as appropriate
                for your environment.</para>
              <para><command># yum update cloud-*</command></para>
              <para><command># apt-get update</command></para>
              <para><command># apt-get upgrade cloud-*</command></para>
            </listitem>
            <listitem>
              <para>Start the agent.</para>
              <programlisting># service cloudstack-agent start</programlisting>
            </listitem>
            <listitem>
              <para>Edit <filename>/etc/cloud/agent/agent.properties</filename> to change the
                resource parameter from
                "com.cloud.agent.resource.computing.LibvirtComputingResource" to
                "com.cloud.hypervisor.kvm.resource.LibvirtComputingResource".</para>
            </listitem>
            <listitem>
              <para>Start the cloud agent and cloud management services.</para>
            </listitem>
            <listitem>
              <para>When the Management Server is up and running, log in to the CloudStack UI and
                restart the virtual router for proper functioning of all the features.</para>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>Log in to the CloudStack UI as administrator, and check the status of the hosts. All
            hosts should come to Up state (except those that you know to be offline). You may need
            to wait 20 or 30 minutes, depending on the number of hosts.</para>
          <note>
            <para>Troubleshooting: If login fails, clear your browser cache and reload the
              page.</para>
          </note>
          <para/>
          <para>Do not proceed to the next step until the hosts show in Up state.</para>
        </listitem>
        <listitem>
          <para>If you are upgrading from 3.0.2, perform the following:</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Ensure that the admin port is set to 8096 by using the "integration.api.port"
                global parameter.</para>
              <para>This port is used by the cloud-sysvmadm script at the end of the upgrade
                procedure. For information about how to set this parameter, see "Setting Global
                Configuration Parameters" in the Installation Guide.</para>
            </listitem>
            <listitem>
              <para>Restart the Management Server.</para>
              <note>
                <para>If you don't want the admin port to remain open, you can set it to null after
                  the upgrade is done and restart the management server.</para>
              </note>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>Run the <command>cloud-sysvmadm</command> script to stop, then start, all Secondary
            Storage VMs, Console Proxy VMs, and virtual routers. Run the script once on each
            management server. Substitute your own IP address of the MySQL instance, the MySQL user
            to connect as, and the password to use for that user. In addition to those parameters,
            provide the <command>-c</command> and <command>-r</command> arguments. For
            example:</para>
          <para><command># nohup cloud-sysvmadm -d 192.168.1.5 -u cloud -p password -c -r >
              sysvm.log 2>&amp;1 &amp;</command></para>
          <para><command># tail -f sysvm.log</command></para>
          <para>This might take up to an hour or more to run, depending on the number of accounts in
            the system.</para>
        </listitem>
        <listitem>
          <para>If needed, upgrade all Citrix XenServer hypervisor hosts in your cloud to a version
            supported by CloudStack 4.1.0. The supported versions are XenServer 5.6 SP2 and 6.0.2.
            Instructions for upgrade can be found in the CloudStack 4.1.0 Installation Guide under
            "Upgrading XenServer Versions."</para>
        </listitem>
        <listitem>
          <para>Now apply the XenServer hotfix XS602E003 (and any other needed hotfixes) to
            XenServer v6.0.2 hypervisor hosts.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Disconnect the XenServer cluster from CloudStack.</para>
              <para>In the left navigation bar of the CloudStack UI, select Infrastructure. Under
                Clusters, click View All. Select the XenServer cluster and click Actions -
                Unmanage.</para>
              <para>This may fail if there are hosts not in one of the states Up, Down,
                Disconnected, or Alert. You may need to fix that before unmanaging this
                cluster.</para>
              <para>Wait until the status of the cluster has reached Unmanaged. Use the CloudStack
                UI to check on the status. When the cluster is in the unmanaged state, there is no
                connection to the hosts in the cluster.</para>
            </listitem>
            <listitem>
              <para>To clean up the VLAN, log in to one XenServer host and run:</para>
              <para><command>/opt/xensource/bin/cloud-clean-vlan.sh</command></para>
            </listitem>
            <listitem>
              <para>Now prepare the upgrade by running the following on one XenServer host:</para>
              <para><command>/opt/xensource/bin/cloud-prepare-upgrade.sh</command></para>
              <para>If you see a message like "can't eject CD", log in to the VM and unmount the CD,
                then run this script again.</para>
            </listitem>
            <listitem>
              <para>Upload the hotfix to the XenServer hosts. Always start with the Xen pool master,
                then the slaves. Using your favorite file copy utility (e.g. WinSCP), copy the
                hotfixes to the host. Place them in a temporary folder such as /tmp. </para>
              <para>On the Xen pool master, upload the hotfix with this command:</para>
              <para><command>xe patch-upload file-name=XS602E003.xsupdate</command></para>
              <para>Make a note of the output from this command, which is a UUID for the hotfix
                file. You'll need it in another step later.</para>
              <note>
                <para>(Optional) If you are applying other hotfixes as well, you can repeat the
                  commands in this section with the appropriate hotfix number. For example,
                  XS602E004.xsupdate.</para>
              </note>
            </listitem>
            <listitem>
              <para>Manually live migrate all VMs on this host to another host. First, get a list of
                the VMs on this host:</para>
              <para><command># xe vm-list</command></para>
              <para>Then use this command to migrate each VM. Replace the example host name and VM
                name with your own:</para>
              <para><command># xe vm-migrate live=true host=<replaceable>host-name</replaceable>
                    vm=<replaceable>VM-name</replaceable></command></para>
              <note>
                <title>Troubleshooting</title>
                <para>If you see a message like "You attempted an operation on a VM which requires
                  PV drivers to be installed but the drivers were not detected," run:</para>
                <para><command>/opt/xensource/bin/make_migratable.sh
                    b6cf79c8-02ee-050b-922f-49583d9f1a14</command>.</para>
              </note>
            </listitem>
            <listitem>
              <para>Apply the hotfix. First, get the UUID of this host:</para>
              <programlisting># xe host-list</programlisting>
              <para>Then use the following command to apply the hotfix. Replace the example host
                UUID with the current host ID, and replace the hotfix UUID with the output from the
                patch-upload command you ran on this machine earlier. You can also get the hotfix
                UUID by running xe patch-list. </para>
              <programlisting><command>xe</command> patch-apply host-uuid=<replaceable>host-uuid</replaceable> uuid=<replaceable>hotfix-uuid</replaceable></programlisting>
            </listitem>
            <listitem>
              <para>Copy the following files from the CloudStack Management Server to the
                host.</para>
              <informaltable>
                <tgroup cols="2" align="left" colsep="1" rowsep="1">
                  <colspec colwidth="1*" colname="1" colnum="1"/>
                  <colspec colwidth="2*" colname="2" colnum="2"/>
                  <thead>
                    <row>
                      <entry><para>Copy from here...</para></entry>
                      <entry><para>...to here</para></entry>
                    </row>
                  </thead>
                  <tbody>
                    <row>
                      <entry><para>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/xenserver60/NFSSR.py</para></entry>
                      <entry><para>/opt/xensource/sm/NFSSR.py</para></entry>
                    </row>
                    <row>
                      <entry><para>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/setupxenserver.sh</para></entry>
                      <entry><para>/opt/xensource/bin/setupxenserver.sh</para></entry>
                    </row>
                    <row>
                      <entry><para>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/make_migratable.sh</para></entry>
                      <entry><para>/opt/xensource/bin/make_migratable.sh</para></entry>
                    </row>
                  </tbody>
                </tgroup>
              </informaltable>
            </listitem>
            <listitem>
              <para>(Only for hotfixes XS602E005 and XS602E007) You need to apply a new Cloud
                Support Pack.</para>
              <itemizedlist>
                <listitem>
                  <para>Download the CSP software onto the XenServer host from one of the following
                    links:</para>
                  <para>For hotfix XS602E005: <ulink
                      url="http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz"
                      >http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz</ulink></para>
                  <para>For hotfix XS602E007: <ulink
                      url="http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz"
                      >http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz</ulink></para>
                </listitem>
                <listitem>
                  <para>Extract the file:</para>
                  <programlisting># tar xf xenserver-cloud-supp.tgz</programlisting>
                </listitem>
                <listitem>
                  <para>Run the following script:</para>
                  <programlisting># xe-install-supplemental-pack xenserver-cloud-supp.iso</programlisting>
                </listitem>
                <listitem>
                  <para>If the XenServer host is part of a zone that uses basic networking, disable
                    Open vSwitch (OVS):</para>
                  <programlisting># xe-switch-network-backend  bridge</programlisting>
                </listitem>
              </itemizedlist>
            </listitem>
            <listitem>
              <para>Reboot this XenServer host.</para>
            </listitem>
            <listitem>
              <para>Run the following:</para>
              <programlisting>/opt/xensource/bin/setupxenserver.sh</programlisting>
              <note>
                <para>If the message "mv: cannot stat `/etc/cron.daily/logrotate': No such file or
                  directory" appears, you can safely ignore it.</para>
              </note>
            </listitem>
            <listitem>
              <para>Run the following:</para>
              <programlisting>for pbd in `xe pbd-list currently-attached=false| grep ^uuid | awk '{print $NF}'`; do xe pbd-plug uuid=$pbd ; </programlisting>
            </listitem>
            <listitem>
              <para>On each slave host in the Xen pool, repeat these steps, starting from "manually
                live migrate VMs."</para>
            </listitem>
          </orderedlist>
        </listitem>
      </orderedlist>
      <note>
        <title>Troubleshooting Tip</title>
        <para>If passwords which you know to be valid appear not to work after upgrade, or other UI
          issues are seen, try clearing your browser cache and reloading the UI page.</para>
      </note>
    </section>
    <section id="upgrade-from-2.2.x-to-4.1">
      <title>Upgrade from 2.2.14 to 4.1.0</title>
      <orderedlist>
        <listitem>
          <para>Ensure that you query your IPaddress usage records and process them; for example,
            issue invoices for any usage that you have not yet billed users for.</para>
          <para>Starting in 3.0.2, the usage record format for IP addresses is the same as the rest
            of the usage types. Instead of a single record with the assignment and release dates,
            separate records are generated per aggregation period with start and end dates. After
            upgrading to 4.1.0, any existing IP address usage records in the old format will no
            longer be available.</para>
        </listitem>
        <listitem>
          <para>If you are using version 2.2.0 - 2.2.13, first upgrade to 2.2.14 by using the
            instructions in the <ulink
              url="http://download.cloud.com/releases/2.2.0/CloudStack2.2.14ReleaseNotes.pdf">2.2.14
              Release Notes</ulink>.</para>
          <warning>
            <title>KVM Hosts</title>
            <para>If KVM hypervisor is used in your cloud, be sure you completed the step to insert
              a valid username and password into the host_details table on each KVM node as
              described in the 2.2.14 Release Notes. This step is critical, as the database will be
              encrypted after the upgrade to 4.1.0.</para>
          </warning>
        </listitem>
        <listitem>
          <para>While running the 2.2.14 system, log in to the UI as root administrator.</para>
        </listitem>
        <listitem>
          <para>Using the UI, add a new System VM template for each hypervisor type that is used in
            your cloud. In each zone, add a system VM template for each hypervisor used in that
            zone</para>
          <orderedlist>
            <listitem>
              <para>In the left navigation bar, click Templates.</para>
            </listitem>
            <listitem>
              <para>In Select view, click Templates.</para>
            </listitem>
            <listitem>
              <para>Click Register template.</para>
              <para>The Register template dialog box is displayed.</para>
            </listitem>
            <listitem>
              <para>In the Register template dialog box, specify the following values depending on
                the hypervisor type (do not change these):</para>
              <informaltable>
                <tgroup cols="2" align="left" colsep="1" rowsep="1">
                  <colspec colwidth="1*" colname="1" colnum="1"/>
                  <colspec colwidth="2*" colname="2" colnum="2"/>
                  <thead>
                    <row>
                      <entry><para>Hypervisor</para></entry>
                      <entry><para>Description</para></entry>
                    </row>
                  </thead>
                  <tbody>
                    <row>
                      <entry><para>XenServer</para></entry>
                      <entry><para>Name: systemvm-xenserver-4.1.0</para>
                        <para>Description: systemvm-xenserver-4.1.0</para>
                        <para>URL:
                          http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2</para>
                        <para>Zone: Choose the zone where this hypervisor is used</para>
                        <para>Hypervisor: XenServer</para>
                        <para>Format: VHD</para>
                        <para>OS Type: Debian GNU/Linux 5.0 (32-bit)</para>
                        <para>Extractable: no</para>
                        <para>Password Enabled: no</para>
                        <para>Public: no</para>
                        <para>Featured: no</para>
                      </entry>
                    </row>
                    <row>
                      <entry><para>KVM</para></entry>
                      <entry><para>Name: systemvm-kvm-4.1.0</para>
                        <para>Description: systemvm-kvm-4.1.0</para>
                        <para>URL:
                          http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2</para>
                        <para>Zone: Choose the zone where this hypervisor is used</para>
                        <para>Hypervisor: KVM</para>
                        <para>Format: QCOW2</para>
                        <para>OS Type: Debian GNU/Linux 5.0 (32-bit)</para>
                        <para>Extractable: no</para>
                        <para>Password Enabled: no</para>
                        <para>Public: no</para>
                        <para>Featured: no</para>
                      </entry>
                    </row>
                    <row>
                      <entry><para>VMware</para></entry>
                      <entry><para>Name: systemvm-vmware-4.1.0</para>
                        <para>Description: systemvm-vmware-4.1.0</para>
                        <para>URL:
                          http://download.cloud.com/templates/burbank/burbank-systemvm-08012012.ova</para>
                        <para>Zone: Choose the zone where this hypervisor is used</para>
                        <para>Hypervisor: VMware</para>
                        <para>Format: OVA</para>
                        <para>OS Type: Debian GNU/Linux 5.0 (32-bit)</para>
                        <para>Extractable: no</para>
                        <para>Password Enabled: no</para>
                        <para>Public: no</para>
                        <para>Featured: no</para>
                      </entry>
                    </row>
                  </tbody>
                </tgroup>
              </informaltable>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>Watch the screen to be sure that the template downloads successfully and enters the
            READY state. Do not proceed until this is successful</para>
        </listitem>
        <listitem>
          <para><emphasis role="bold">WARNING</emphasis>: If you use more than one type of
            hypervisor in your cloud, be sure you have repeated these steps to download the system
            VM template for each hypervisor type. Otherwise, the upgrade will fail.</para>
        </listitem>
        <listitem>
          <para>Stop all Usage Servers if running. Run this on all Usage Server hosts.</para>
          <programlisting># service cloud-usage stop</programlisting>
        </listitem>
        <listitem>
          <para>Stop the Management Servers. Run this on all Management Server hosts.</para>
          <programlisting language="Bash"><prompt>#</prompt> service cloud-management stop</programlisting>
        </listitem>
        <listitem>
          <para>On the MySQL master, take a backup of the MySQL databases. We recommend performing
            this step even in test upgrades. If there is an issue, this will assist with
            debugging.</para>
          <para>In the following commands, it is assumed that you have set the root password on the
            database, which is a CloudStack recommended best practice. Substitute your own MySQL
            root password.</para>
          <programlisting><prompt>#</prompt> <command>mysqldump</command> -u root -p<replaceable>mysql_password</replaceable> cloud > <filename>cloud-backup.dmp</filename>
                        <prompt>#</prompt> <command>mysqldump</command> -u root -p<replaceable>mysql_password</replaceable> cloud_usage > <filename>cloud-usage-backup.dmp</filename>
                    </programlisting>
        </listitem>
        <listitem>
          <para> Either build RPM/DEB packages as detailed in the Installation Guide, or use one of
            the community provided yum/apt repositories to gain access to the &PRODUCT; binaries.
          </para>
        </listitem>
        <listitem id="upgrade-deb-packages-22">
          <para>If you are using Ubuntu, follow this procedure to upgrade your packages. If not,
            skip to step <xref linkend="upgrade-rpm-packages-22"/>.</para>
          <note>
            <title>Community Packages</title>
            <para>This section assumes you're using the community supplied packages for &PRODUCT;.
              If you've created your own packages and APT repository, substitute your own URL for
              the ones used in these examples.</para>
          </note>
          <orderedlist id="debsteps-22">
            <listitem>
              <para>The first order of business will be to change the sources list for each system
                with &PRODUCT; packages. This means all management servers, and any hosts that have
                the KVM agent. (No changes should be necessary for hosts that are running VMware or
                Xen.)</para>
              <para>Start by opening <filename>/etc/apt/sources.list.d/cloudstack.list</filename> on
                any systems that have &PRODUCT; packages installed.</para>
              <para>This file should have one line, which contains:</para>
              <programlisting language="Bash">deb http://cloudstack.apt-get.eu/ubuntu precise 4.0</programlisting>
              <para>We'll change it to point to the new package repository:</para>
              <programlisting language="Bash">deb http://cloudstack.apt-get.eu/ubuntu precise 4.1</programlisting>
              <para>If you're using your own package repository, change this line to read as
                appropriate for your 4.1.0 repository.</para>
            </listitem>
            <listitem>
              <para>Now update your apt package list:</para>
              <programlisting language="Bash">$ sudo apt-get update</programlisting>
            </listitem>
            <listitem id="deb-master-22">
              <para>Now that you have the repository configured, it's time to install the
                  <filename>cloudstack-management</filename> package. This will pull in any other
                dependencies you need.</para>
              <programlisting language="Bash">$ sudo apt-get install cloudstack-management</programlisting>
            </listitem>
            <listitem id="kvm-agent-deb-22">
              <para>On KVM hosts, you will need to manually install the
                  <filename>cloudstack-agent</filename> package:</para>
              <programlisting language="Bash">$ sudo apt-get install cloudstack-agent</programlisting>
              <para>During the installation of <filename>cloudstack-agent</filename>, APT will copy
                your <filename>agent.properties</filename>, <filename>log4j-cloud.xml</filename>,
                and <filename>environment.properties</filename> from
                  <filename>/etc/cloud/agent</filename> to
                  <filename>/etc/cloudstack/agent</filename>.</para>
              <para>When prompted whether you wish to keep your configuration, say Yes.</para>
            </listitem>
            <listitem>
              <para>Verify that the file
                  <filename>/etc/cloudstack/agent/environment.properties</filename> has a line that
                reads:</para>
              <programlisting language="Bash">paths.script=/usr/share/cloudstack-common</programlisting>
              <para>If not, add the line.</para>
            </listitem>
            <listitem>
              <para>Restart the agent:</para>
              <programlisting language="Bash">
service cloud-agent stop
killall jsvc
service cloudstack-agent start
              </programlisting>
            </listitem>
            <listitem>
              <para>During the upgrade, <filename>log4j-cloud.xml</filename> was simply copied over,
                so the logs will continue to be added to
                  <filename>/var/log/cloud/agent/agent.log</filename>. There's nothing
                  <emphasis>wrong</emphasis> with this, but if you prefer to be consistent, you can
                change this by copying over the sample configuration file:</para>
              <programlisting language="Bash">
cd /etc/cloudstack/agent
mv log4j-cloud.xml.dpkg-dist log4j-cloud.xml
service cloudstack-agent restart
                            </programlisting>
            </listitem>
            <listitem>
              <para>Once the agent is running, you can uninstall the old cloud-* packages from your
                system:</para>
              <programlisting language="Bash">sudo dpkg --purge cloud-agent</programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="upgrade-rpm-packages-22">
          <para>If you are using CentOS or RHEL, follow this procedure to upgrade your packages. If
            not, skip to step <xref linkend="correct-components-xml-22"/>.</para>
          <note>
            <title>Community Packages</title>
            <para>This section assumes you're using the community supplied packages for &PRODUCT;.
              If you've created your own packages and yum repository, substitute your own URL for
              the ones used in these examples.</para>
          </note>
          <orderedlist id="rpmsteps-22">
            <listitem>
              <para>The first order of business will be to change the yum repository for each system
                with &PRODUCT; packages. This means all management servers, and any hosts that have
                the KVM agent. (No changes should be necessary for hosts that are running VMware or
                Xen.)</para>
              <para>Start by opening <filename>/etc/yum.repos.d/cloudstack.repo</filename> on any
                systems that have &PRODUCT; packages installed.</para>
              <para>This file should have content similar to the following:</para>
              <programlisting language="Bash">
[apache-cloudstack]
name=Apache CloudStack
baseurl=http://cloudstack.apt-get.eu/rhel/4.0/
enabled=1
gpgcheck=0
                            </programlisting>
              <para>If you are using the community provided package repository, change the baseurl
                to http://cloudstack.apt-get.eu/rhel/4.1/</para>
              <para>If you're using your own package repository, change this line to read as
                appropriate for your 4.1.0 repository.</para>
            </listitem>
            <listitem id="rpm-master-22">
              <para>Now that you have the repository configured, it's time to install the
                  <filename>cloudstack-management</filename> package by upgrading the older
                  <filename>cloud-client</filename> package.</para>
              <programlisting language="Bash">$ sudo yum upgrade cloud-client</programlisting>
            </listitem>
            <listitem id="kvm-agent-rpm-22">
              <para>For KVM hosts, you will need to upgrade the <filename>cloud-agent</filename>
                package, similarly installing the new version as
                  <filename>cloudstack-agent</filename>.</para>
              <programlisting language="Bash">$ sudo yum upgrade cloud-agent</programlisting>
              <para>During the installation of <filename>cloudstack-agent</filename>, the RPM will
                copy your <filename>agent.properties</filename>,
                  <filename>log4j-cloud.xml</filename>, and
                  <filename>environment.properties</filename> from
                  <filename>/etc/cloud/agent</filename> to
                  <filename>/etc/cloudstack/agent</filename>.</para>
            </listitem>
            <listitem>
              <para>Verify that the file
                  <filename>/etc/cloudstack/agent/environment.properties</filename> has a line that
                reads:</para>
              <programlisting language="Bash">paths.script=/usr/share/cloudstack-common</programlisting>
              <para>If not, add the line.</para>
            </listitem>
            <listitem>
              <para>Restart the agent:</para>
              <programlisting language="Bash">
service cloud-agent stop
killall jsvc
service cloudstack-agent start
                            </programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem id="correct-components-xml-22">
          <para>If you have made changes to your existing copy of the file components.xml in your
            previous-version CloudStack installation, the changes will be preserved in the upgrade.
            However, you need to do the following steps to place these changes in a new version of
            the file which is compatible with version 4.0.0-incubating.</para>
          <note>
            <para>How will you know whether you need to do this? If the upgrade output in the
              previous step included a message like the following, then some custom content was
              found in your old components.xml, and you need to merge the two files:</para>
          </note>
          <programlisting>warning: /etc/cloud/management/components.xml created as /etc/cloud/management/components.xml.rpmnew </programlisting>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Make a backup copy of your
                  <filename>/etc/cloud/management/components.xml</filename> file. For
                example:</para>
              <programlisting><prompt>#</prompt> <command>mv</command> <filename>/etc/cloud/management/components.xml</filename> <filename>/etc/cloud/management/components.xml-backup</filename></programlisting>
            </listitem>
            <listitem>
              <para>Copy <filename>/etc/cloud/management/components.xml.rpmnew</filename> to create
                a new <filename>/etc/cloud/management/components.xml</filename>:</para>
              <programlisting><prompt>#</prompt> <command>cp</command> -ap <filename>/etc/cloud/management/components.xml.rpmnew</filename> <filename>/etc/cloud/management/components.xml</filename></programlisting>
            </listitem>
            <listitem>
              <para>Merge your changes from the backup file into the new components.xml file.</para>
              <programlisting><prompt>#</prompt> <command>vi</command> <filename>/etc/cloud/management/components.xml</filename>
                            </programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>After upgrading to 4.1, API clients are expected to send plain text passwords for
            login and user creation, instead of MD5 hash. Incase, api client changes are not
            acceptable, following changes are to be made for backward compatibility:</para>
          <para>Modify componentsContext.xml, and make PlainTextUserAuthenticator as the default
            authenticator (1st entry in the userAuthenticators adapter list is default)</para>
          <programlisting language="XML">
&lt;!-- Security adapters --&gt;
&lt;bean id="userAuthenticators" class="com.cloud.utils.component.AdapterList"&gt;
  &lt;property name="Adapters"&gt;
    &lt;list&gt;
      &lt;ref bean="PlainTextUserAuthenticator"/&gt;
      &lt;ref bean="MD5UserAuthenticator"/&gt;
      &lt;ref bean="LDAPUserAuthenticator"/&gt;
    &lt;/list&gt;
  &lt;/property&gt;
&lt;/bean&gt;
            </programlisting>
          <para>PlainTextUserAuthenticator works the same way MD5UserAuthenticator worked prior to
            4.1.</para>
        </listitem>
        <listitem>
          <para>If you have made changes to your existing copy of the
              <filename>/etc/cloud/management/db.properties</filename> file in your previous-version
            CloudStack installation, the changes will be preserved in the upgrade. However, you need
            to do the following steps to place these changes in a new version of the file which is
            compatible with version 4.0.0-incubating.</para>
          <orderedlist>
            <listitem>
              <para>Make a backup copy of your file
                  <filename>/etc/cloud/management/db.properties</filename>. For example:</para>
              <programlisting><prompt>#</prompt> <command>mv</command> <filename>/etc/cloud/management/db.properties</filename> <filename>/etc/cloud/management/db.properties-backup</filename></programlisting>
            </listitem>
            <listitem>
              <para>Copy <filename>/etc/cloud/management/db.properties.rpmnew</filename> to create a
                new <filename>/etc/cloud/management/db.properties</filename>:</para>
              <programlisting><prompt>#</prompt> <command>cp</command> -ap <filename>/etc/cloud/management/db.properties.rpmnew</filename> <filename>etc/cloud/management/db.properties</filename></programlisting>
            </listitem>
            <listitem>
              <para>Merge your changes from the backup file into the new db.properties file.</para>
              <programlisting><prompt>#</prompt> <command>vi</command> <filename>/etc/cloud/management/db.properties</filename></programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>On the management server node, run the following command. It is recommended that you
            use the command-line flags to provide your own encryption keys. See Password and Key
            Encryption in the Installation Guide.</para>
          <programlisting><prompt>#</prompt> <command>cloud-setup-encryption</command> -e <replaceable>encryption_type</replaceable> -m <replaceable>management_server_key</replaceable> -k <replaceable>database_key</replaceable></programlisting>
          <para>When used without arguments, as in the following example, the default encryption
            type and keys will be used:</para>
          <itemizedlist>
            <listitem>
              <para>(Optional) For encryption_type, use file or web to indicate the technique used
                to pass in the database encryption password. Default: file.</para>
            </listitem>
            <listitem>
              <para>(Optional) For management_server_key, substitute the default key that is used to
                encrypt confidential parameters in the properties file. Default: password. It is
                highly recommended that you replace this with a more secure value</para>
            </listitem>
            <listitem>
              <para>(Optional) For database_key, substitute the default key that is used to encrypt
                confidential parameters in the CloudStack database. Default: password. It is highly
                recommended that you replace this with a more secure value.</para>
            </listitem>
          </itemizedlist>
        </listitem>
        <listitem>
          <para>Repeat steps 10 - 14 on every management server node. If you provided your own
            encryption key in step 14, use the same key on all other management servers.</para>
        </listitem>
        <listitem>
          <para>Start the first Management Server. Do not start any other Management Server nodes
            yet.</para>
          <programlisting language="Bash"><prompt>#</prompt> service cloudstack-management start</programlisting>
          <para>Wait until the databases are upgraded. Ensure that the database upgrade is complete.
            You should see a message like "Complete! Done." After confirmation, start the other
            Management Servers one at a time by running the same command on each node.</para>
        </listitem>
        <listitem>
          <para>Start all Usage Servers (if they were running on your previous version). Perform
            this on each Usage Server host.</para>
          <programlisting language="Bash"><prompt>#</prompt> service cloudstack-usage start</programlisting>
        </listitem>
        <listitem>
          <para>(KVM only) Additional steps are required for each KVM host. These steps will not
            affect running guests in the cloud. These steps are required only for clouds using KVM
            as hosts and only on the KVM hosts.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para> Configure your CloudStack package repositories as outlined in the Installation
                Guide </para>
            </listitem>
            <listitem>
              <para>Stop the running agent.</para>
              <programlisting># service cloud-agent stop</programlisting>
            </listitem>
            <listitem>
              <para>Update the agent software with one of the following command sets as
                appropriate.</para>
              <programlisting><prompt>#</prompt> <command>yum</command> update cloud-*</programlisting>
              <programlisting>
                                <prompt>#</prompt> <command>apt-get</command> update
                                <prompt>#</prompt> <command>apt-get</command> upgrade cloud-*
                            </programlisting>
            </listitem>
            <listitem>
              <para>Start the agent.</para>
              <programlisting># service cloudstack-agent start</programlisting>
            </listitem>
            <listitem>
              <para> Copy the contents of the <filename>agent.properties</filename> file to the new
                  <filename>agent.properties</filename> file by using the following command</para>
              <programlisting><command>sed</command> -i 's/com.cloud.agent.resource.computing.LibvirtComputingResource/com.cloud.hypervisor.kvm.resource.LibvirtComputingResource/g' <filename>/etc/cloud/agent/agent.properties</filename></programlisting>
            </listitem>
            <listitem>
              <para>Start the cloud agent and cloud management services.</para>
            </listitem>
            <listitem>
              <para>When the Management Server is up and running, log in to the CloudStack UI and
                restart the virtual router for proper functioning of all the features.</para>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>Log in to the CloudStack UI as admin, and check the status of the hosts. All hosts
            should come to Up state (except those that you know to be offline). You may need to wait
            20 or 30 minutes, depending on the number of hosts.</para>
          <para>Do not proceed to the next step until the hosts show in the Up state. If the hosts
            do not come to the Up state, contact support.</para>
        </listitem>
        <listitem>
          <para>Run the following script to stop, then start, all Secondary Storage VMs, Console
            Proxy VMs, and virtual routers.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Run the command once on one management server. Substitute your own IP address of
                the MySQL instance, the MySQL user to connect as, and the password to use for that
                user. In addition to those parameters, provide the "-c" and "-r" arguments. For
                example:</para>
              <programlisting><prompt>#</prompt> <command>nohup cloud-sysvmadm</command> -d <replaceable>192.168.1.5</replaceable> -u cloud -p <replaceable>password</replaceable> -c -r > sysvm.log 2>&amp;1 &amp;
                                <prompt>#</prompt> <command>tail</command> -f <filename>sysvm.log</filename></programlisting>
              <para>This might take up to an hour or more to run, depending on the number of
                accounts in the system.</para>
            </listitem>
            <listitem>
              <para>After the script terminates, check the log to verify correct execution:</para>
              <programlisting><prompt>#</prompt> <command>tail</command> -f <filename>sysvm.log</filename></programlisting>
              <para>The content should be like the following:</para>
              <programlisting>
                                Stopping and starting 1 secondary storage vm(s)...
                                Done stopping and starting secondary storage vm(s)
                                Stopping and starting 1 console proxy vm(s)...
                                Done stopping and starting console proxy vm(s).
                                Stopping and starting 4 running routing vm(s)...
                                Done restarting router(s).
                            </programlisting>
            </listitem>
          </orderedlist>
        </listitem>
        <listitem>
          <para>If you would like additional confirmation that the new system VM templates were
            correctly applied when these system VMs were rebooted, SSH into the System VM and check
            the version.</para>
          <para>Use one of the following techniques, depending on the hypervisor.</para>
          <formalpara>
            <title>XenServer or KVM:</title>
            <para>SSH in by using the link local IP address of the system VM. For example, in the
              command below, substitute your own path to the private key used to log in to the
              system VM and your own link local IP.</para>
          </formalpara>
          <para>Run the following commands on the XenServer or KVM host on which the system VM is
            present:</para>
          <programlisting><prompt>#</prompt> <command>ssh</command> -i <replaceable>private-key-path</replaceable> <replaceable>link-local-ip</replaceable> -p 3922
                        # cat /etc/cloudstack-release</programlisting>
          <para>The output should be like the following:</para>
          <programlisting>Cloudstack Release 4.0.0-incubating Mon Oct 9 15:10:04 PST 2012</programlisting>
          <formalpara>
            <title>ESXi</title>
            <para>SSH in using the private IP address of the system VM. For example, in the command
              below, substitute your own path to the private key used to log in to the system VM and
              your own private IP.</para>
          </formalpara>
          <para>Run the following commands on the Management Server:</para>
          <programlisting><prompt>#</prompt> <command>ssh</command> -i <replaceable>private-key-path</replaceable> <replaceable>private-ip</replaceable> -p 3922
                        <prompt>#</prompt> <command>cat</command> <filename>/etc/cloudstack-release</filename>
                    </programlisting>
          <para>The output should be like the following:</para>
          <programlisting>Cloudstack Release 4.0.0-incubating Mon Oct 9 15:10:04 PST 2012</programlisting>
        </listitem>
        <listitem>
          <para>If needed, upgrade all Citrix XenServer hypervisor hosts in your cloud to a version
            supported by CloudStack 4.0.0-incubating. The supported versions are XenServer 5.6 SP2
            and 6.0.2. Instructions for upgrade can be found in the CloudStack 4.0.0-incubating
            Installation Guide.</para>
        </listitem>
        <listitem>
          <para>Apply the XenServer hotfix XS602E003 (and any other needed hotfixes) to XenServer
            v6.0.2 hypervisor hosts.</para>
          <orderedlist numeration="loweralpha">
            <listitem>
              <para>Disconnect the XenServer cluster from CloudStack.</para>
              <para>In the left navigation bar of the CloudStack UI, select Infrastructure. Under
                Clusters, click View All. Select the XenServer cluster and click Actions -
                Unmanage.</para>
              <para>This may fail if there are hosts not in one of the states Up, Down,
                Disconnected, or Alert. You may need to fix that before unmanaging this
                cluster.</para>
              <para>Wait until the status of the cluster has reached Unmanaged. Use the CloudStack
                UI to check on the status. When the cluster is in the unmanaged state, there is no
                connection to the hosts in the cluster.</para>
            </listitem>
            <listitem>
              <para>To clean up the VLAN, log in to one XenServer host and run:</para>
              <programlisting>/opt/xensource/bin/cloud-clean-vlan.sh</programlisting>
            </listitem>
            <listitem>
              <para>Prepare the upgrade by running the following on one XenServer host:</para>
              <programlisting>/opt/xensource/bin/cloud-prepare-upgrade.sh</programlisting>
              <para>If you see a message like "can't eject CD", log in to the VM and umount the CD,
                then run this script again.</para>
            </listitem>
            <listitem>
              <para>Upload the hotfix to the XenServer hosts. Always start with the Xen pool master,
                then the slaves. Using your favorite file copy utility (e.g. WinSCP), copy the
                hotfixes to the host. Place them in a temporary folder such as /root or /tmp. </para>
              <para>On the Xen pool master, upload the hotfix with this command:</para>
              <programlisting>xe patch-upload file-name=XS602E003.xsupdate</programlisting>
              <para>Make a note of the output from this command, which is a UUID for the hotfix
                file. You'll need it in another step later.</para>
              <note>
                <para>(Optional) If you are applying other hotfixes as well, you can repeat the
                  commands in this section with the appropriate hotfix number. For example,
                  XS602E004.xsupdate.</para>
              </note>
            </listitem>
            <listitem>
              <para>Manually live migrate all VMs on this host to another host. First, get a list of
                the VMs on this host:</para>
              <programlisting># xe vm-list</programlisting>
              <para>Then use this command to migrate each VM. Replace the example host name and VM
                name with your own:</para>
              <programlisting><prompt>#</prompt> <command>xe</command> vm-migrate live=true host=<replaceable>host-name</replaceable> vm=<replaceable>VM-name</replaceable></programlisting>
              <note>
                <title>Troubleshooting</title>
                <para>If you see a message like "You attempted an operation on a VM which requires
                  PV drivers to be installed but the drivers were not detected," run:</para>
                <para><command>/opt/xensource/bin/make_migratable.sh
                    b6cf79c8-02ee-050b-922f-49583d9f1a14</command>.</para>
              </note>
            </listitem>
            <listitem>
              <para>Apply the hotfix. First, get the UUID of this host:</para>
              <para><command># xe host-list</command></para>
              <para>Then use the following command to apply the hotfix. Replace the example host
                UUID with the current host ID, and replace the hotfix UUID with the output from the
                patch-upload command you ran on this machine earlier. You can also get the hotfix
                UUID by running xe patch-list. </para>
              <para><command>xe patch-apply host-uuid=<replaceable>host-uuid</replaceable>
                    uuid=<replaceable>hotfix-uuid</replaceable></command></para>
            </listitem>
            <listitem>
              <para>Copy the following files from the CloudStack Management Server to the
                host.</para>
              <informaltable>
                <tgroup cols="2" align="left" colsep="1" rowsep="1">
                  <colspec colwidth="1*" colname="1" colnum="1"/>
                  <colspec colwidth="2*" colname="2" colnum="2"/>
                  <thead>
                    <row>
                      <entry><para>Copy from here...</para></entry>
                      <entry><para>...to here</para></entry>
                    </row>
                  </thead>
                  <tbody>
                    <row>
                      <entry><para><filename>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/xenserver60/NFSSR.py</filename></para></entry>
                      <entry><para><filename>/opt/xensource/sm/NFSSR.py</filename></para></entry>
                    </row>
                    <row>
                      <entry><para><filename>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/setupxenserver.sh</filename></para></entry>
                      <entry><para><filename>/opt/xensource/bin/setupxenserver.sh</filename></para></entry>
                    </row>
                    <row>
                      <entry><para><filename>/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/make_migratable.sh</filename></para></entry>
                      <entry><para><filename>/opt/xensource/bin/make_migratable.sh</filename></para></entry>
                    </row>
                  </tbody>
                </tgroup>
              </informaltable>
            </listitem>
            <listitem>
              <para>(Only for hotfixes XS602E005 and XS602E007) You need to apply a new Cloud
                Support Pack.</para>
              <itemizedlist>
                <listitem>
                  <para>Download the CSP software onto the XenServer host from one of the following
                    links:</para>
                  <para>For hotfix XS602E005: <ulink
                      url="http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz"
                      >http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz</ulink></para>
                  <para>For hotfix XS602E007: <ulink
                      url="http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz"
                      >http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz</ulink></para>
                </listitem>
                <listitem>
                  <para>Extract the file:</para>
                  <para><command># tar xf xenserver-cloud-supp.tgz</command></para>
                </listitem>
                <listitem>
                  <para>Run the following script:</para>
                  <para><command># xe-install-supplemental-pack
                    xenserver-cloud-supp.iso</command></para>
                </listitem>
                <listitem>
                  <para>If the XenServer host is part of a zone that uses basic networking, disable
                    Open vSwitch (OVS):</para>
                  <para><command># xe-switch-network-backend bridge</command></para>
                </listitem>
              </itemizedlist>
            </listitem>
            <listitem>
              <para>Reboot this XenServer host.</para>
            </listitem>
            <listitem>
              <para>Run the following:</para>
              <para><command>/opt/xensource/bin/setupxenserver.sh</command></para>
              <note>
                <para>If the message "mv: cannot stat `/etc/cron.daily/logrotate': No such file or
                  directory" appears, you can safely ignore it.</para>
              </note>
            </listitem>
            <listitem>
              <para>Run the following:</para>
              <para><command>for pbd in `xe pbd-list currently-attached=false| grep ^uuid | awk
                  '{print $NF}'`; do xe pbd-plug uuid=$pbd ; </command>
              </para>
            </listitem>
            <listitem>
              <para>On each slave host in the Xen pool, repeat these steps, starting from "manually
                live migrate VMs."</para>
            </listitem>
          </orderedlist>
        </listitem>
      </orderedlist>
    </section>
  </chapter>
  <!-- <chapter id="api-changes-4.1">
        <title>API Changes in 4.1.0</title>
    </chapter> -->
</book>