diff --git a/docs/en-US/Release_Notes.xml b/docs/en-US/Release_Notes.xml
index f1a159c68d8..8534b0815f3 100644
--- a/docs/en-US/Release_Notes.xml
+++ b/docs/en-US/Release_Notes.xml
@@ -530,6 +530,1513 @@ under the License.
+
+ Upgrade Instructions
+ This section contains upgrade instructions from prior versions of CloudStack to Apache
+ CloudStack 4.2.0. We include instructions on upgrading to Apache CloudStack from pre-Apache
+ versions of Citrix CloudStack (last version prior to Apache is 3.0.2) and from the releases
+ made while CloudStack was in the Apache Incubator.
+ If you run into any issues during upgrades, please feel free to ask questions on
+ users@cloudstack.apache.org or dev@cloudstack.apache.org.
+
+ Upgrade from 4.x.x to 4.2.0
+ This section will guide you from &PRODUCT; 4.0.x versions to &PRODUCT; 4.2.0.
+ Any steps that are hypervisor-specific will be called out with a note.
+
+ Package Structure Changes
+ The package structure for &PRODUCT; has changed significantly since the 4.0.x
+ releases. If you've compiled your own packages, you'll notice that the package names and
+ the number of packages has changed. This is not a bug.
+ However, this does mean that the procedure is not as simple as an apt-get
+ upgrade or yum update, so please follow this section
+ carefully.
+
+ We recommend reading through this section once or twice before beginning your upgrade
+ procedure, and working through it on a test system before working on a production
+ system.
+
+
+ Most users of &PRODUCT; manage the installation and upgrades of &PRODUCT; with one
+ of Linux's predominant package systems, RPM or APT. This guide assumes you'll be using
+ RPM and Yum (for Red Hat Enterprise Linux or CentOS), or APT and Debian packages (for
+ Ubuntu).
+ Create RPM or Debian packages (as appropriate) and a repository from the 4.2.0
+ source, or check the Apache CloudStack downloads page at http://cloudstack.apache.org/downloads.html for package repositories supplied
+ by community members. You will need them for step
+ or step .
+ Instructions for creating packages from the &PRODUCT; source are in the Installation
+ Guide.
+
+
+ Stop your management server or servers. Run this on all management server
+ hosts:
+ # service cloud-management stop
+
+
+ If you are running a usage server or usage servers, stop those as well:
+ # service cloud-usage stop
+
+
+ Make a backup of your MySQL database. If you run into any issues or need to roll
+ back the upgrade, this will assist in debugging or restoring your existing environment.
+ You'll be prompted for your password.
+ # mysqldump -u root -p cloud > cloudstack-backup.sql
+
+
+ If you have made changes to
+ /etc/cloud/management/components.xml, you'll need to carry these
+ over manually to the new file,
+ /etc/cloudstack/management/componentContext.xml. This is not done
+ automatically. (If you're unsure, we recommend making a backup of the original
+ components.xml to be on the safe side.
+
+
+ After upgrading to 4.2, API clients are expected to send plain text passwords for
+ login and user creation, instead of MD5 hash. If API client changes are not acceptable,
+ following changes are to be made for backward compatibility:
+ Modify componentsContext.xml, and make PlainTextUserAuthenticator as the default
+ authenticator (1st entry in the userAuthenticators adapter list is default)
+
+<!-- Security adapters -->
+<bean id="userAuthenticators" class="com.cloud.utils.component.AdapterList">
+ <property name="Adapters">
+ <list>
+ <ref bean="PlainTextUserAuthenticator"/>
+ <ref bean="MD5UserAuthenticator"/>
+ <ref bean="LDAPUserAuthenticator"/>
+ </list>
+ </property>
+</bean>
+
+ PlainTextUserAuthenticator works the same way MD5UserAuthenticator worked prior to
+ 4.2.
+
+
+ If you are using Ubuntu, follow this procedure to upgrade your packages. If not,
+ skip to step .
+
+ Community Packages
+ This section assumes you're using the community supplied packages for &PRODUCT;.
+ If you've created your own packages and APT repository, substitute your own URL for
+ the ones used in these examples.
+
+
+
+ The first order of business will be to change the sources list for each system
+ with &PRODUCT; packages. This means all management servers, and any hosts that have
+ the KVM agent. (No changes should be necessary for hosts that are running VMware or
+ Xen.)
+ Start by opening /etc/apt/sources.list.d/cloudstack.list on
+ any systems that have &PRODUCT; packages installed.
+ This file should have one line, which contains:
+ deb http://cloudstack.apt-get.eu/ubuntu precise 4.0
+ We'll change it to point to the new package repository:
+ deb http://cloudstack.apt-get.eu/ubuntu precise 4.2
+ If you're using your own package repository, change this line to read as
+ appropriate for your 4.2.0 repository.
+
+
+ Now update your apt package list:
+ $ sudo apt-get update
+
+
+ Now that you have the repository configured, it's time to install the
+ cloudstack-management package. This will pull in any other
+ dependencies you need.
+ $ sudo apt-get install cloudstack-management
+
+
+ You will need to manually install the cloudstack-agent
+ package:
+ $ sudo apt-get install cloudstack-agent
+ During the installation of cloudstack-agent, APT will copy
+ your agent.properties, log4j-cloud.xml,
+ and environment.properties from
+ /etc/cloud/agent to
+ /etc/cloudstack/agent.
+ When prompted whether you wish to keep your configuration, say Yes.
+
+
+ Verify that the file
+ /etc/cloudstack/agent/environment.properties has a line that
+ reads:
+ paths.script=/usr/share/cloudstack-common
+ If not, add the line.
+
+
+ Restart the agent:
+
+service cloud-agent stop
+killall jsvc
+service cloudstack-agent start
+
+
+
+ During the upgrade, log4j-cloud.xml was simply copied over,
+ so the logs will continue to be added to
+ /var/log/cloud/agent/agent.log. There's nothing
+ wrong with this, but if you prefer to be consistent, you can
+ change this by copying over the sample configuration file:
+
+cd /etc/cloudstack/agent
+mv log4j-cloud.xml.dpkg-dist log4j-cloud.xml
+service cloudstack-agent restart
+
+
+
+ Once the agent is running, you can uninstall the old cloud-* packages from your
+ system:
+ sudo dpkg --purge cloud-agent
+
+
+
+
+ If you are using CentOS or RHEL, follow this procedure to upgrade your packages. If
+ not, skip to step .
+
+ Community Packages
+ This section assumes you're using the community supplied packages for &PRODUCT;.
+ If you've created your own packages and yum repository, substitute your own URL for
+ the ones used in these examples.
+
+
+
+ The first order of business will be to change the yum repository for each system
+ with &PRODUCT; packages. This means all management servers, and any hosts that have
+ the KVM agent.
+ (No changes should be necessary for hosts that are running VMware or
+ Xen.)
+ Start by opening /etc/yum.repos.d/cloudstack.repo on any
+ systems that have &PRODUCT; packages installed.
+ This file should have content similar to the following:
+
+[apache-cloudstack]
+name=Apache CloudStack
+baseurl=http://cloudstack.apt-get.eu/rhel/4.0/
+enabled=1
+gpgcheck=0
+
+ If you are using the community provided package repository, change the base url
+ to http://cloudstack.apt-get.eu/rhel/4.2/
+ If you're using your own package repository, change this line to read as
+ appropriate for your 4.2.0 repository.
+
+
+ Now that you have the repository configured, it's time to install the
+ cloudstack-management package by upgrading the older
+ cloud-client package.
+ $ sudo yum upgrade cloud-client
+
+
+ For KVM hosts, you will need to upgrade the cloud-agent
+ package, similarly installing the new version as
+ cloudstack-agent.
+ $ sudo yum upgrade cloud-agent
+ During the installation of cloudstack-agent, the RPM will
+ copy your agent.properties,
+ log4j-cloud.xml, and
+ environment.properties from
+ /etc/cloud/agent to
+ /etc/cloudstack/agent.
+
+
+ For CentOS 5.5, perform the following:
+
+
+ Run the following command:
+ rpm -Uvh http://download.cloud.com/support/jsvc/jakarta-commons-daemon-jsvc-1.0.1-8.9.el6.x86_64.rpm
+
+
+ Upgrade the Usage server.
+ sudo yum upgrade cloud-usage
+
+
+
+
+ Verify that the file
+ /etc/cloudstack/agent/environment.properties has a line that
+ reads:
+ paths.script=/usr/share/cloudstack-common
+ If not, add the line.
+
+
+ Restart the agent:
+
+service cloud-agent stop
+killall jsvc
+service cloudstack-agent start
+
+
+
+
+
+ Once you've upgraded the packages on your management servers, you'll need to restart
+ the system VMs. Make sure port 8096 is open in your local host firewall to do
+ this.
+ There is a script that will do this for you, all you need to do is run the script
+ and supply the IP address for your MySQL instance and your MySQL credentials:
+ # nohup cloudstack-sysvmadm -d IP address -u cloud -p -a > sysvm.log 2>&1 &
+ You can monitor the log for progress. The process of restarting the system VMs can
+ take an hour or more.
+ # tail -f sysvm.log
+ The output to sysvm.log will look something like this:
+
+Stopping and starting 1 secondary storage vm(s)...
+Done stopping and starting secondary storage vm(s)
+Stopping and starting 1 console proxy vm(s)...
+Done stopping and starting console proxy vm(s).
+Stopping and starting 4 running routing vm(s)...
+Done restarting router(s).
+
+
+
+
+ For Xen Hosts: Copy vhd-utils
+ This step is only for CloudStack installs that are using Xen hosts.
+
+ Copy the file vhd-utils to
+ /usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver.
+
+
+
+
+ Upgrade from 3.0.2 to 4.2.0
+ This section will guide you from Citrix CloudStack 3.0.2 to Apache CloudStack 4.2.0.
+ Sections that are hypervisor-specific will be called out with a note.
+
+
+
+ The following upgrade instructions apply only if you're using VMware hosts. If
+ you're not using VMware hosts, skip this step and move on to .
+
+ In each zone that includes VMware hosts, you need to add a new system VM template.
+
+
+ While running the existing 3.0.2 system, log in to the UI as root
+ administrator.
+
+
+ In the left navigation bar, click Templates.
+
+
+ In Select view, click Templates.
+
+
+ Click Register template.
+ The Register template dialog box is displayed.
+
+
+ In the Register template dialog box, specify the following values (do not change
+ these):
+
+
+
+
+
+
+ Field
+ Value
+
+
+
+
+ Name
+ systemvm-vmware-4.2
+
+
+ Description
+ systemvm-vmware-4.2
+
+
+ URL
+ http://download.cloud.com/templates/burbank/burbank-systemvm-08012012.ova
+
+
+ Zone
+ Choose the zone where this hypervisor is used
+
+
+ Hypervisor
+ VMware
+
+
+ Format
+ OVA
+
+
+ OS Type
+ Debian GNU/Linux 5.0 (32-bit)
+
+
+ Extractable
+ no
+
+
+ Password Enabled
+ no
+
+
+ Public
+ no
+
+
+ Featured
+ no
+
+
+
+
+
+
+ Watch the screen to be sure that the template downloads successfully and enters
+ the READY state. Do not proceed until this is successful.
+
+
+
+
+ Stop all Usage Servers if running. Run this on all Usage Server hosts.
+ # service cloud-usage stop
+
+
+ Stop the Management Servers. Run this on all Management Server hosts.
+ # service cloud-management stop
+
+
+ On the MySQL master, take a backup of the MySQL databases. We recommend performing
+ this step even in test upgrades. If there is an issue, this will assist with
+ debugging.
+ In the following commands, it is assumed that you have set the root password on the
+ database, which is a CloudStack recommended best practice. Substitute your own MySQL
+ root password.
+ # mysqldump -u root -pmysql_password cloud > cloud-backup.dmp
+ # mysqldump -u root -pmysql_password cloud_usage > cloud-usage-backup.dmp
+
+
+ Either build RPM/DEB packages as detailed in the Installation Guide, or use one of
+ the community provided yum/apt repositories to gain access to the &PRODUCT;
+ binaries.
+
+
+ If you are using Ubuntu, follow this procedure to upgrade your packages. If not,
+ skip to step .
+
+ Community Packages
+ This section assumes you're using the community supplied packages for &PRODUCT;.
+ If you've created your own packages and APT repository, substitute your own URL for
+ the ones used in these examples.
+
+
+
+ The first order of business will be to change the sources list for each system
+ with &PRODUCT; packages. This means all management servers, and any hosts that have
+ the KVM agent. (No changes should be necessary for hosts that are running VMware or
+ Xen.)
+ Start by opening /etc/apt/sources.list.d/cloudstack.list on
+ any systems that have &PRODUCT; packages installed.
+ This file should have one line, which contains:
+ deb http://cloudstack.apt-get.eu/ubuntu precise 4.0
+ We'll change it to point to the new package repository:
+ deb http://cloudstack.apt-get.eu/ubuntu precise 4.2
+ If you're using your own package repository, change this line to read as
+ appropriate for your 4.2.0 repository.
+
+
+ Now update your apt package list:
+ $ sudo apt-get update
+
+
+ Now that you have the repository configured, it's time to install the
+ cloudstack-management package. This will pull in any other
+ dependencies you need.
+ $ sudo apt-get install cloudstack-management
+
+
+ You will need to manually install the cloudstack-agent
+ package:
+ $ sudo apt-get install cloudstack-agent
+ During the installation of cloudstack-agent, APT will copy
+ your agent.properties, log4j-cloud.xml,
+ and environment.properties from
+ /etc/cloud/agent to
+ /etc/cloudstack/agent.
+ When prompted whether you wish to keep your configuration, say Yes.
+
+
+ Verify that the file
+ /etc/cloudstack/agent/environment.properties has a line that
+ reads:
+ paths.script=/usr/share/cloudstack-common
+ If not, add the line.
+
+
+ Restart the agent:
+
+service cloud-agent stop
+killall jsvc
+service cloudstack-agent start
+
+
+
+ During the upgrade, log4j-cloud.xml was simply copied over,
+ so the logs will continue to be added to
+ /var/log/cloud/agent/agent.log. There's nothing
+ wrong with this, but if you prefer to be consistent, you can
+ change this by copying over the sample configuration file:
+
+cd /etc/cloudstack/agent
+mv log4j-cloud.xml.dpkg-dist log4j-cloud.xml
+service cloudstack-agent restart
+
+
+
+ Once the agent is running, you can uninstall the old cloud-* packages from your
+ system:
+ sudo dpkg --purge cloud-agent
+
+
+
+
+ If you are using CentOS or RHEL, follow this procedure to upgrade your packages. If
+ not, skip to step .
+
+ Community Packages
+ This section assumes you're using the community supplied packages for &PRODUCT;.
+ If you've created your own packages and yum repository, substitute your own URL for
+ the ones used in these examples.
+
+
+
+ The first order of business will be to change the yum repository for each system
+ with &PRODUCT; packages. This means all management servers, and any hosts that have
+ the KVM agent. (No changes should be necessary for hosts that are running VMware or
+ Xen.)
+ Start by opening /etc/yum.repos.d/cloudstack.repo on any
+ systems that have &PRODUCT; packages installed.
+ This file should have content similar to the following:
+
+[apache-cloudstack]
+name=Apache CloudStack
+baseurl=http://cloudstack.apt-get.eu/rhel/4.0/
+enabled=1
+gpgcheck=0
+
+ If you are using the community provided package repository, change the baseurl
+ to http://cloudstack.apt-get.eu/rhel/4.2/
+ If you're using your own package repository, change this line to read as
+ appropriate for your 4.2.0 repository.
+
+
+ Now that you have the repository configured, it's time to install the
+ cloudstack-management package by upgrading the older
+ cloud-client package.
+ $ sudo yum upgrade cloud-client
+
+
+ For KVM hosts, you will need to upgrade the cloud-agent
+ package, similarly installing the new version as
+ cloudstack-agent.
+ $ sudo yum upgrade cloud-agent
+ During the installation of cloudstack-agent, the RPM will
+ copy your agent.properties,
+ log4j-cloud.xml, and
+ environment.properties from
+ /etc/cloud/agent to
+ /etc/cloudstack/agent.
+
+
+ Verify that the file
+ /etc/cloudstack/agent/environment.properties has a line that
+ reads:
+ paths.script=/usr/share/cloudstack-common
+ If not, add the line.
+
+
+ Restart the agent:
+
+service cloud-agent stop
+killall jsvc
+service cloudstack-agent start
+
+
+
+
+
+ If you have made changes to your copy of
+ /etc/cloud/management/components.xml the changes will be
+ preserved in the upgrade. However, you need to do the following steps to place these
+ changes in a new version of the file which is compatible with version 4.2.0.
+
+
+ Make a backup copy of /etc/cloud/management/components.xml.
+ For example:
+ # mv /etc/cloud/management/components.xml /etc/cloud/management/components.xml-backup
+
+
+ Copy /etc/cloud/management/components.xml.rpmnew to create
+ a new /etc/cloud/management/components.xml:
+ # cp -ap /etc/cloud/management/components.xml.rpmnew /etc/cloud/management/components.xml
+
+
+ Merge your changes from the backup file into the new
+ components.xml.
+ # vi /etc/cloudstack/management/components.xml
+
+
+
+ If you have more than one management server node, repeat the upgrade steps on each
+ node.
+
+
+
+ After upgrading to 4.2, API clients are expected to send plain text passwords for
+ login and user creation, instead of MD5 hash. Incase, api client changes are not
+ acceptable, following changes are to be made for backward compatibility:
+ Modify componentsContext.xml, and make PlainTextUserAuthenticator as the default
+ authenticator (1st entry in the userAuthenticators adapter list is default)
+
+<!-- Security adapters -->
+<bean id="userAuthenticators" class="com.cloud.utils.component.AdapterList">
+ <property name="Adapters">
+ <list>
+ <ref bean="PlainTextUserAuthenticator"/>
+ <ref bean="MD5UserAuthenticator"/>
+ <ref bean="LDAPUserAuthenticator"/>
+ </list>
+ </property>
+</bean>
+
+ PlainTextUserAuthenticator works the same way MD5UserAuthenticator worked prior to
+ 4.2.
+
+
+ Start the first Management Server. Do not start any other Management Server nodes
+ yet.
+ # service cloudstack-management start
+ Wait until the databases are upgraded. Ensure that the database upgrade is complete.
+ After confirmation, start the other Management Servers one at a time by running the same
+ command on each node.
+
+ Failing to restart the Management Server indicates a problem in the upgrade.
+ Having the Management Server restarted without any issues indicates that the upgrade
+ is successfully completed.
+
+
+
+ Start all Usage Servers (if they were running on your previous version). Perform
+ this on each Usage Server host.
+ # service cloudstack-usage start
+
+
+
+ Additional steps are required for each KVM host. These steps will not affect
+ running guests in the cloud. These steps are required only for clouds using KVM as
+ hosts and only on the KVM hosts.
+
+
+
+ Configure a yum or apt repository containing the &PRODUCT; packages as outlined
+ in the Installation Guide.
+
+
+ Stop the running agent.
+ # service cloud-agent stop
+
+
+ Update the agent software with one of the following command sets as appropriate
+ for your environment.
+ # yum update cloud-*
+ # apt-get update
+ # apt-get upgrade cloud-*
+
+
+ Start the agent.
+ # service cloudstack-agent start
+
+
+ Edit /etc/cloudstack/agent/agent.properties to change the
+ resource parameter from
+ "com.cloud.agent.resource.computing.LibvirtComputingResource" to
+ "com.cloud.hypervisor.kvm.resource.LibvirtComputingResource".
+
+
+ Start the cloud agent and cloud management services.
+
+
+ When the Management Server is up and running, log in to the CloudStack UI and
+ restart the virtual router for proper functioning of all the features.
+
+
+
+
+ Log in to the CloudStack UI as administrator, and check the status of the hosts. All
+ hosts should come to Up state (except those that you know to be offline). You may need
+ to wait 20 or 30 minutes, depending on the number of hosts.
+
+ Troubleshooting: If login fails, clear your browser cache and reload the
+ page.
+
+
+ Do not proceed to the next step until the hosts show in Up state.
+
+
+ If you are upgrading from 3.0.2, perform the following:
+
+
+ Ensure that the admin port is set to 8096 by using the "integration.api.port"
+ global parameter.
+ This port is used by the cloud-sysvmadm script at the end of the upgrade
+ procedure. For information about how to set this parameter, see "Setting Global
+ Configuration Parameters" in the Installation Guide.
+
+
+ Restart the Management Server.
+
+ If you don't want the admin port to remain open, you can set it to null after
+ the upgrade is done and restart the management server.
+
+
+
+
+
+ Run the cloud-sysvmadm script to stop, then start, all Secondary
+ Storage VMs, Console Proxy VMs, and virtual routers. Run the script once on each
+ management server. Substitute your own IP address of the MySQL instance, the MySQL user
+ to connect as, and the password to use for that user. In addition to those parameters,
+ provide the -c and -r arguments. For
+ example:
+ # nohup cloud-sysvmadm -d 192.168.1.5 -u cloud -p password -c -r >
+ sysvm.log 2>&1 &
+ # tail -f sysvm.log
+ This might take up to an hour or more to run, depending on the number of accounts in
+ the system.
+
+
+ If needed, upgrade all Citrix XenServer hypervisor hosts in your cloud to a version
+ supported by CloudStack 4.2.0. The supported versions are XenServer 5.6 SP2 and 6.0.2.
+ Instructions for upgrade can be found in the CloudStack 4.2.0 Installation Guide under
+ "Upgrading XenServer Versions."
+
+
+ Now apply the XenServer hotfix XS602E003 (and any other needed hotfixes) to
+ XenServer v6.0.2 hypervisor hosts.
+
+
+ Disconnect the XenServer cluster from CloudStack.
+ In the left navigation bar of the CloudStack UI, select Infrastructure. Under
+ Clusters, click View All. Select the XenServer cluster and click Actions -
+ Unmanage.
+ This may fail if there are hosts not in one of the states Up, Down,
+ Disconnected, or Alert. You may need to fix that before unmanaging this
+ cluster.
+ Wait until the status of the cluster has reached Unmanaged. Use the CloudStack
+ UI to check on the status. When the cluster is in the unmanaged state, there is no
+ connection to the hosts in the cluster.
+
+
+ To clean up the VLAN, log in to one XenServer host and run:
+ /opt/xensource/bin/cloud-clean-vlan.sh
+
+
+ Now prepare the upgrade by running the following on one XenServer host:
+ /opt/xensource/bin/cloud-prepare-upgrade.sh
+ If you see a message like "can't eject CD", log in to the VM and unmount the CD,
+ then run this script again.
+
+
+ Upload the hotfix to the XenServer hosts. Always start with the Xen pool master,
+ then the slaves. Using your favorite file copy utility (e.g. WinSCP), copy the
+ hotfixes to the host. Place them in a temporary folder such as /tmp.
+ On the Xen pool master, upload the hotfix with this command:
+ xe patch-upload file-name=XS602E003.xsupdate
+ Make a note of the output from this command, which is a UUID for the hotfix
+ file. You'll need it in another step later.
+
+ (Optional) If you are applying other hotfixes as well, you can repeat the
+ commands in this section with the appropriate hotfix number. For example,
+ XS602E004.xsupdate.
+
+
+
+ Manually live migrate all VMs on this host to another host. First, get a list of
+ the VMs on this host:
+ # xe vm-list
+ Then use this command to migrate each VM. Replace the example host name and VM
+ name with your own:
+ # xe vm-migrate live=true host=host-name
+ vm=VM-name
+
+ Troubleshooting
+ If you see a message like "You attempted an operation on a VM which requires
+ PV drivers to be installed but the drivers were not detected," run:
+ /opt/xensource/bin/make_migratable.sh
+ b6cf79c8-02ee-050b-922f-49583d9f1a14.
+
+
+
+ Apply the hotfix. First, get the UUID of this host:
+ # xe host-list
+ Then use the following command to apply the hotfix. Replace the example host
+ UUID with the current host ID, and replace the hotfix UUID with the output from the
+ patch-upload command you ran on this machine earlier. You can also get the hotfix
+ UUID by running xe patch-list.
+ xe patch-apply host-uuid=host-uuid uuid=hotfix-uuid
+
+
+ Copy the following files from the CloudStack Management Server to the
+ host.
+
+
+
+
+
+
+ Copy from here...
+ ...to here
+
+
+
+
+ /usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/xenserver60/NFSSR.py
+ /opt/xensource/sm/NFSSR.py
+
+
+ /usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/setupxenserver.sh
+ /opt/xensource/bin/setupxenserver.sh
+
+
+ /usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/make_migratable.sh
+ /opt/xensource/bin/make_migratable.sh
+
+
+
+
+
+
+ (Only for hotfixes XS602E005 and XS602E007) You need to apply a new Cloud
+ Support Pack.
+
+
+ Download the CSP software onto the XenServer host from one of the following
+ links:
+ For hotfix XS602E005: http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz
+ For hotfix XS602E007: http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz
+
+
+ Extract the file:
+ # tar xf xenserver-cloud-supp.tgz
+
+
+ Run the following script:
+ # xe-install-supplemental-pack xenserver-cloud-supp.iso
+
+
+ If the XenServer host is part of a zone that uses basic networking, disable
+ Open vSwitch (OVS):
+ # xe-switch-network-backend bridge
+
+
+
+
+ Reboot this XenServer host.
+
+
+ Run the following:
+ /opt/xensource/bin/setupxenserver.sh
+
+ If the message "mv: cannot stat `/etc/cron.daily/logrotate': No such file or
+ directory" appears, you can safely ignore it.
+
+
+
+ Run the following:
+ for pbd in `xe pbd-list currently-attached=false| grep ^uuid | awk '{print $NF}'`; do xe pbd-plug uuid=$pbd ;
+
+
+ On each slave host in the Xen pool, repeat these steps, starting from "manually
+ live migrate VMs."
+
+
+
+
+
+ Troubleshooting Tip
+ If passwords which you know to be valid appear not to work after upgrade, or other UI
+ issues are seen, try clearing your browser cache and reloading the UI page.
+
+
+
+ Upgrade from 2.2.14 to 4.2.0
+
+
+ Ensure that you query your IPaddress usage records and process them; for example,
+ issue invoices for any usage that you have not yet billed users for.
+ Starting in 3.0.2, the usage record format for IP addresses is the same as the rest
+ of the usage types. Instead of a single record with the assignment and release dates,
+ separate records are generated per aggregation period with start and end dates. After
+ upgrading to 4.2.0, any existing IP address usage records in the old format will no
+ longer be available.
+
+
+ If you are using version 2.2.0 - 2.2.13, first upgrade to 2.2.14 by using the
+ instructions in the 2.2.14
+ Release Notes.
+
+ KVM Hosts
+ If KVM hypervisor is used in your cloud, be sure you completed the step to insert
+ a valid username and password into the host_details table on each KVM node as
+ described in the 2.2.14 Release Notes. This step is critical, as the database will be
+ encrypted after the upgrade to 4.2.0.
+
+
+
+ While running the 2.2.14 system, log in to the UI as root administrator.
+
+
+ Using the UI, add a new System VM template for each hypervisor type that is used in
+ your cloud. In each zone, add a system VM template for each hypervisor used in that
+ zone
+
+
+ In the left navigation bar, click Templates.
+
+
+ In Select view, click Templates.
+
+
+ Click Register template.
+ The Register template dialog box is displayed.
+
+
+ In the Register template dialog box, specify the following values depending on
+ the hypervisor type (do not change these):
+
+
+
+
+
+
+ Hypervisor
+ Description
+
+
+
+
+ XenServer
+ Name: systemvm-xenserver-4.2.0
+ Description: systemvm-xenserver-4.2.0
+ URL:http://download.cloud.com/templates/4.2/systemvmtemplate-2013-07-12-master-xen.vhd.bz2
+ Zone: Choose the zone where this hypervisor is used
+ Hypervisor: XenServer
+ Format: VHD
+ OS Type: Debian GNU/Linux 6.0 (32-bit)
+ Extractable: no
+ Password Enabled: no
+ Public: no
+ Featured: no
+
+
+
+ KVM
+ Name: systemvm-kvm-4.2.0
+ Description: systemvm-kvm-4.2.0
+ URL:
+ http://download.cloud.com/templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2
+ Zone: Choose the zone where this hypervisor is used
+ Hypervisor: KVM
+ Format: QCOW2
+ OS Type: Debian GNU/Linux 5.0 (32-bit)
+ Extractable: no
+ Password Enabled: no
+ Public: no
+ Featured: no
+
+
+
+ VMware
+ Name: systemvm-vmware-4.2.0
+ Description: systemvm-vmware-4.2.0
+ URL:
+ http://download.cloud.com/templates/4.2/systemvmtemplate-4.2-vh7.ova
+ Zone: Choose the zone where this hypervisor is used
+ Hypervisor: VMware
+ Format: OVA
+ OS Type: Debian GNU/Linux 5.0 (32-bit)
+ Extractable: no
+ Password Enabled: no
+ Public: no
+ Featured: no
+
+
+
+
+
+
+
+
+
+ Watch the screen to be sure that the template downloads successfully and enters the
+ READY state. Do not proceed until this is successful
+
+
+ WARNING: If you use more than one type of
+ hypervisor in your cloud, be sure you have repeated these steps to download the system
+ VM template for each hypervisor type. Otherwise, the upgrade will fail.
+
+
+ Stop all Usage Servers if running. Run this on all Usage Server hosts.
+ # service cloud-usage stop
+
+
+ Stop the Management Servers. Run this on all Management Server hosts.
+ # service cloud-management stop
+
+
+ On the MySQL master, take a backup of the MySQL databases. We recommend performing
+ this step even in test upgrades. If there is an issue, this will assist with
+ debugging.
+ In the following commands, it is assumed that you have set the root password on the
+ database, which is a CloudStack recommended best practice. Substitute your own MySQL
+ root password.
+ # mysqldump -u root -pmysql_password cloud > cloud-backup.dmp
+ # mysqldump -u root -pmysql_password cloud_usage > cloud-usage-backup.dmp
+
+
+
+ Either build RPM/DEB packages as detailed in the Installation Guide, or use one of
+ the community provided yum/apt repositories to gain access to the &PRODUCT; binaries.
+
+
+
+ If you are using Ubuntu, follow this procedure to upgrade your packages. If not,
+ skip to step .
+
+ Community Packages
+ This section assumes you're using the community supplied packages for &PRODUCT;.
+ If you've created your own packages and APT repository, substitute your own URL for
+ the ones used in these examples.
+
+
+
+ The first order of business will be to change the sources list for each system
+ with &PRODUCT; packages. This means all management servers, and any hosts that have
+ the KVM agent. (No changes should be necessary for hosts that are running VMware or
+ Xen.)
+ Start by opening /etc/apt/sources.list.d/cloudstack.list on
+ any systems that have &PRODUCT; packages installed.
+ This file should have one line, which contains:
+ deb http://cloudstack.apt-get.eu/ubuntu precise 4.0
+ We'll change it to point to the new package repository:
+ deb http://cloudstack.apt-get.eu/ubuntu precise 4.2
+ If you're using your own package repository, change this line to read as
+ appropriate for your 4.2.0 repository.
+
+
+ Now update your apt package list:
+ $ sudo apt-get update
+
+
+ Now that you have the repository configured, it's time to install the
+ cloudstack-management package. This will pull in any other
+ dependencies you need.
+ $ sudo apt-get install cloudstack-management
+
+
+ On KVM hosts, you will need to manually install the
+ cloudstack-agent package:
+ $ sudo apt-get install cloudstack-agent
+ During the installation of cloudstack-agent, APT will copy
+ your agent.properties, log4j-cloud.xml,
+ and environment.properties from
+ /etc/cloud/agent to
+ /etc/cloudstack/agent.
+ When prompted whether you wish to keep your configuration, say Yes.
+
+
+ Verify that the file
+ /etc/cloudstack/agent/environment.properties has a line that
+ reads:
+ paths.script=/usr/share/cloudstack-common
+ If not, add the line.
+
+
+ Restart the agent:
+
+service cloud-agent stop
+killall jsvc
+service cloudstack-agent start
+
+
+
+ During the upgrade, log4j-cloud.xml was simply copied over,
+ so the logs will continue to be added to
+ /var/log/cloud/agent/agent.log. There's nothing
+ wrong with this, but if you prefer to be consistent, you can
+ change this by copying over the sample configuration file:
+
+cd /etc/cloudstack/agent
+mv log4j-cloud.xml.dpkg-dist log4j-cloud.xml
+service cloudstack-agent restart
+
+
+
+ Once the agent is running, you can uninstall the old cloud-* packages from your
+ system:
+ sudo dpkg --purge cloud-agent
+
+
+
+
+ If you are using CentOS or RHEL, follow this procedure to upgrade your packages. If
+ not, skip to step .
+
+ Community Packages
+ This section assumes you're using the community supplied packages for &PRODUCT;.
+ If you've created your own packages and yum repository, substitute your own URL for
+ the ones used in these examples.
+
+
+
+ The first order of business will be to change the yum repository for each system
+ with &PRODUCT; packages. This means all management servers, and any hosts that have
+ the KVM agent. (No changes should be necessary for hosts that are running VMware or
+ Xen.)
+ Start by opening /etc/yum.repos.d/cloudstack.repo on any
+ systems that have &PRODUCT; packages installed.
+ This file should have content similar to the following:
+
+[apache-cloudstack]
+name=Apache CloudStack
+baseurl=http://cloudstack.apt-get.eu/rhel/4.0/
+enabled=1
+gpgcheck=0
+
+ If you are using the community provided package repository, change the baseurl
+ to http://cloudstack.apt-get.eu/rhel/4.2/
+ If you're using your own package repository, change this line to read as
+ appropriate for your 4.2.0 repository.
+
+
+ Now that you have the repository configured, it's time to install the
+ cloudstack-management package by upgrading the older
+ cloud-client package.
+ $ sudo yum upgrade cloud-client
+
+
+ For KVM hosts, you will need to upgrade the cloud-agent
+ package, similarly installing the new version as
+ cloudstack-agent.
+ $ sudo yum upgrade cloud-agent
+ During the installation of cloudstack-agent, the RPM will
+ copy your agent.properties,
+ log4j-cloud.xml, and
+ environment.properties from
+ /etc/cloud/agent to
+ /etc/cloudstack/agent.
+
+
+ Verify that the file
+ /etc/cloudstack/agent/environment.properties has a line that
+ reads:
+ paths.script=/usr/share/cloudstack-common
+ If not, add the line.
+
+
+ Restart the agent:
+
+service cloud-agent stop
+killall jsvc
+service cloudstack-agent start
+
+
+
+
+
+ If you have made changes to your existing copy of the file components.xml in your
+ previous-version CloudStack installation, the changes will be preserved in the upgrade.
+ However, you need to do the following steps to place these changes in a new version of
+ the file which is compatible with version 4.0.0-incubating.
+
+ How will you know whether you need to do this? If the upgrade output in the
+ previous step included a message like the following, then some custom content was
+ found in your old components.xml, and you need to merge the two files:
+
+ warning: /etc/cloud/management/components.xml created as /etc/cloud/management/components.xml.rpmnew
+
+
+ Make a backup copy of your
+ /etc/cloud/management/components.xml file. For
+ example:
+ # mv /etc/cloud/management/components.xml /etc/cloud/management/components.xml-backup
+
+
+ Copy /etc/cloud/management/components.xml.rpmnew to create
+ a new /etc/cloud/management/components.xml:
+ # cp -ap /etc/cloud/management/components.xml.rpmnew /etc/cloud/management/components.xml
+
+
+ Merge your changes from the backup file into the new components.xml file.
+ # vi /etc/cloud/management/components.xml
+
+
+
+
+
+ After upgrading to 4.2, API clients are expected to send plain text passwords for
+ login and user creation, instead of MD5 hash. Incase, api client changes are not
+ acceptable, following changes are to be made for backward compatibility:
+ Modify componentsContext.xml, and make PlainTextUserAuthenticator as the default
+ authenticator (1st entry in the userAuthenticators adapter list is default)
+
+<!-- Security adapters -->
+<bean id="userAuthenticators" class="com.cloud.utils.component.AdapterList">
+ <property name="Adapters">
+ <list>
+ <ref bean="PlainTextUserAuthenticator"/>
+ <ref bean="MD5UserAuthenticator"/>
+ <ref bean="LDAPUserAuthenticator"/>
+ </list>
+ </property>
+</bean>
+
+ PlainTextUserAuthenticator works the same way MD5UserAuthenticator worked prior to
+ 4.2.
+
+
+ If you have made changes to your existing copy of the
+ /etc/cloud/management/db.properties file in your previous-version
+ CloudStack installation, the changes will be preserved in the upgrade. However, you need
+ to do the following steps to place these changes in a new version of the file which is
+ compatible with version 4.0.0-incubating.
+
+
+ Make a backup copy of your file
+ /etc/cloud/management/db.properties. For example:
+ # mv /etc/cloud/management/db.properties /etc/cloud/management/db.properties-backup
+
+
+ Copy /etc/cloud/management/db.properties.rpmnew to create a
+ new /etc/cloud/management/db.properties:
+ # cp -ap /etc/cloud/management/db.properties.rpmnew etc/cloud/management/db.properties
+
+
+ Merge your changes from the backup file into the new db.properties file.
+ # vi /etc/cloud/management/db.properties
+
+
+
+
+ On the management server node, run the following command. It is recommended that you
+ use the command-line flags to provide your own encryption keys. See Password and Key
+ Encryption in the Installation Guide.
+ # cloudstack-setup-encryption -e encryption_type -m management_server_key -k database_key
+ When used without arguments, as in the following example, the default encryption
+ type and keys will be used:
+
+
+ (Optional) For encryption_type, use file or web to indicate the technique used
+ to pass in the database encryption password. Default: file.
+
+
+ (Optional) For management_server_key, substitute the default key that is used to
+ encrypt confidential parameters in the properties file. Default: password. It is
+ highly recommended that you replace this with a more secure value
+
+
+ (Optional) For database_key, substitute the default key that is used to encrypt
+ confidential parameters in the CloudStack database. Default: password. It is highly
+ recommended that you replace this with a more secure value.
+
+
+
+
+ Repeat steps 10 - 14 on every management server node. If you provided your own
+ encryption key in step 14, use the same key on all other management servers.
+
+
+ Start the first Management Server. Do not start any other Management Server nodes
+ yet.
+ # service cloudstack-management start
+ Wait until the databases are upgraded. Ensure that the database upgrade is complete.
+ You should see a message like "Complete! Done." After confirmation, start the other
+ Management Servers one at a time by running the same command on each node.
+
+
+ Start all Usage Servers (if they were running on your previous version). Perform
+ this on each Usage Server host.
+ # service cloudstack-usage start
+
+
+ (KVM only) Additional steps are required for each KVM host. These steps will not
+ affect running guests in the cloud. These steps are required only for clouds using KVM
+ as hosts and only on the KVM hosts.
+
+
+ Configure your CloudStack package repositories as outlined in the Installation
+ Guide
+
+
+ Stop the running agent.
+ # service cloud-agent stop
+
+
+ Update the agent software with one of the following command sets as
+ appropriate.
+ # yum update cloud-*
+
+ # apt-get update
+ # apt-get upgrade cloud-*
+
+
+
+ Start the agent.
+ # service cloudstack-agent start
+
+
+ Copy the contents of the agent.properties file to the new
+ agent.properties file by using the following command
+ sed -i 's/com.cloud.agent.resource.computing.LibvirtComputingResource/com.cloud.hypervisor.kvm.resource.LibvirtComputingResource/g' /etc/cloud/agent/agent.properties
+
+
+ Start the cloud agent and cloud management services.
+
+
+ When the Management Server is up and running, log in to the CloudStack UI and
+ restart the virtual router for proper functioning of all the features.
+
+
+
+
+ Log in to the CloudStack UI as admin, and check the status of the hosts. All hosts
+ should come to Up state (except those that you know to be offline). You may need to wait
+ 20 or 30 minutes, depending on the number of hosts.
+ Do not proceed to the next step until the hosts show in the Up state. If the hosts
+ do not come to the Up state, contact support.
+
+
+ Run the following script to stop, then start, all Secondary Storage VMs, Console
+ Proxy VMs, and virtual routers.
+
+
+ Run the command once on one management server. Substitute your own IP address of
+ the MySQL instance, the MySQL user to connect as, and the password to use for that
+ user. In addition to those parameters, provide the "-c" and "-r" arguments. For
+ example:
+ # nohup cloud-sysvmadm -d 192.168.1.5 -u cloud -p password -c -r > sysvm.log 2>&1 &
+ # tail -f sysvm.log
+ This might take up to an hour or more to run, depending on the number of
+ accounts in the system.
+
+
+ After the script terminates, check the log to verify correct execution:
+ # tail -f sysvm.log
+ The content should be like the following:
+
+ Stopping and starting 1 secondary storage vm(s)...
+ Done stopping and starting secondary storage vm(s)
+ Stopping and starting 1 console proxy vm(s)...
+ Done stopping and starting console proxy vm(s).
+ Stopping and starting 4 running routing vm(s)...
+ Done restarting router(s).
+
+
+
+
+
+ If you would like additional confirmation that the new system VM templates were
+ correctly applied when these system VMs were rebooted, SSH into the System VM and check
+ the version.
+ Use one of the following techniques, depending on the hypervisor.
+
+ XenServer or KVM:
+ SSH in by using the link local IP address of the system VM. For example, in the
+ command below, substitute your own path to the private key used to log in to the
+ system VM and your own link local IP.
+
+ Run the following commands on the XenServer or KVM host on which the system VM is
+ present:
+ # ssh -i private-key-path link-local-ip -p 3922
+ # cat /etc/cloudstack-release
+ The output should be like the following:
+ Cloudstack Release 4.0.0-incubating Mon Oct 9 15:10:04 PST 2012
+
+ ESXi
+ SSH in using the private IP address of the system VM. For example, in the command
+ below, substitute your own path to the private key used to log in to the system VM and
+ your own private IP.
+
+ Run the following commands on the Management Server:
+ # ssh -i private-key-path private-ip -p 3922
+ # cat /etc/cloudstack-release
+
+ The output should be like the following:
+ Cloudstack Release 4.0.0-incubating Mon Oct 9 15:10:04 PST 2012
+
+
+ If needed, upgrade all Citrix XenServer hypervisor hosts in your cloud to a version
+ supported by CloudStack 4.0.0-incubating. The supported versions are XenServer 5.6 SP2
+ and 6.0.2. Instructions for upgrade can be found in the CloudStack 4.0.0-incubating
+ Installation Guide.
+
+
+ Apply the XenServer hotfix XS602E003 (and any other needed hotfixes) to XenServer
+ v6.0.2 hypervisor hosts.
+
+
+ Disconnect the XenServer cluster from CloudStack.
+ In the left navigation bar of the CloudStack UI, select Infrastructure. Under
+ Clusters, click View All. Select the XenServer cluster and click Actions -
+ Unmanage.
+ This may fail if there are hosts not in one of the states Up, Down,
+ Disconnected, or Alert. You may need to fix that before unmanaging this
+ cluster.
+ Wait until the status of the cluster has reached Unmanaged. Use the CloudStack
+ UI to check on the status. When the cluster is in the unmanaged state, there is no
+ connection to the hosts in the cluster.
+
+
+ To clean up the VLAN, log in to one XenServer host and run:
+ /opt/xensource/bin/cloud-clean-vlan.sh
+
+
+ Prepare the upgrade by running the following on one XenServer host:
+ /opt/xensource/bin/cloud-prepare-upgrade.sh
+ If you see a message like "can't eject CD", log in to the VM and umount the CD,
+ then run this script again.
+
+
+ Upload the hotfix to the XenServer hosts. Always start with the Xen pool master,
+ then the slaves. Using your favorite file copy utility (e.g. WinSCP), copy the
+ hotfixes to the host. Place them in a temporary folder such as /root or /tmp.
+ On the Xen pool master, upload the hotfix with this command:
+ xe patch-upload file-name=XS602E003.xsupdate
+ Make a note of the output from this command, which is a UUID for the hotfix
+ file. You'll need it in another step later.
+
+ (Optional) If you are applying other hotfixes as well, you can repeat the
+ commands in this section with the appropriate hotfix number. For example,
+ XS602E004.xsupdate.
+
+
+
+ Manually live migrate all VMs on this host to another host. First, get a list of
+ the VMs on this host:
+ # xe vm-list
+ Then use this command to migrate each VM. Replace the example host name and VM
+ name with your own:
+ # xe vm-migrate live=true host=host-name vm=VM-name
+
+ Troubleshooting
+ If you see a message like "You attempted an operation on a VM which requires
+ PV drivers to be installed but the drivers were not detected," run:
+ /opt/xensource/bin/make_migratable.sh
+ b6cf79c8-02ee-050b-922f-49583d9f1a14.
+
+
+
+ Apply the hotfix. First, get the UUID of this host:
+ # xe host-list
+ Then use the following command to apply the hotfix. Replace the example host
+ UUID with the current host ID, and replace the hotfix UUID with the output from the
+ patch-upload command you ran on this machine earlier. You can also get the hotfix
+ UUID by running xe patch-list.
+ xe patch-apply host-uuid=host-uuid
+ uuid=hotfix-uuid
+
+
+ Copy the following files from the CloudStack Management Server to the
+ host.
+
+
+
+
+
+
+ Copy from here...
+ ...to here
+
+
+
+
+ /usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/xenserver60/NFSSR.py
+ /opt/xensource/sm/NFSSR.py
+
+
+ /usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/setupxenserver.sh
+ /opt/xensource/bin/setupxenserver.sh
+
+
+ /usr/lib64/cloudstack-common/scripts/vm/hypervisor/xenserver/make_migratable.sh
+ /opt/xensource/bin/make_migratable.sh
+
+
+
+
+
+
+ (Only for hotfixes XS602E005 and XS602E007) You need to apply a new Cloud
+ Support Pack.
+
+
+ Download the CSP software onto the XenServer host from one of the following
+ links:
+ For hotfix XS602E005: http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E005/56710/xe-phase-2/xenserver-cloud-supp.tgz
+ For hotfix XS602E007: http://coltrane.eng.hq.xensource.com/release/XenServer-6.x/XS-6.0.2/hotfixes/XS602E007/57824/xe-phase-2/xenserver-cloud-supp.tgz
+
+
+ Extract the file:
+ # tar xf xenserver-cloud-supp.tgz
+
+
+ Run the following script:
+ # xe-install-supplemental-pack
+ xenserver-cloud-supp.iso
+
+
+ If the XenServer host is part of a zone that uses basic networking, disable
+ Open vSwitch (OVS):
+ # xe-switch-network-backend bridge
+
+
+
+
+ Reboot this XenServer host.
+
+
+ Run the following:
+ /opt/xensource/bin/setupxenserver.sh
+
+ If the message "mv: cannot stat `/etc/cron.daily/logrotate': No such file or
+ directory" appears, you can safely ignore it.
+
+
+
+ Run the following:
+ for pbd in `xe pbd-list currently-attached=false| grep ^uuid | awk
+ '{print $NF}'`; do xe pbd-plug uuid=$pbd ;
+
+
+
+ On each slave host in the Xen pool, repeat these steps, starting from "manually
+ live migrate VMs."
+
+
+
+
+
+
Version 4.1.0