etan39
/
cloudstack
mirror of https://github.com/apache/cloudstack.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into 4.0

This commit is contained in:
Edison Su 2012-09-06 11:50:13 -07:00
parent ebfe12e846 f26929bf2d
commit bd5aebb2bd
23 changed files with 632 additions and 4961 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

41
LICENSE
View File

@ -375,7 +375,7 @@ Within the deps/awsapi-lib directory
cloud-gson.jar http://code.google.com/p/google-gson/
licensed under the Apache License, Version 2 http://www.apache.org/licenses/LICENSE-2.0.txt (as above)
$license.CopyrightNotice
from Json.simple Project http://code.google.com/p/json-simple/
json_simple-1.1.jar http://code.google.com/p/json-simple/source/checkout
@ -2368,7 +2368,8 @@ Within the target/jar directory
cloud-backport-util-concurrent-3.0.jar
licensed under the Apache License, Version 1.1 http://www.apache.org/licenses/LICENSE-1.1 (as follows)
Copyright © 2012 The Apache Software Foundation
/* ====================================================================
* The Apache Software License, Version 1.1
@ -2462,11 +2463,42 @@ Within the target/jar directory
cloud-google-gson-1.7.1.jar http://code.google.com/p/google-gson/
licensed under the Apache License, Version 2 http://www.apache.org/licenses/LICENSE-2.0.txt (as above)
$license.CopyrightNotice
from Jetty Committers http://jetty.codehaus.org/jetty/
jetty-6.1.26.jar http://repo1.maven.org/maven2/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26-sources.jar
jetty-util-6.1.26.jar http://repo1.maven.org/maven2/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26-sources.jar
licensed under the BSD (3-clause) http://www.opensource.org/licenses/BSD-3-Clause (as follows)
Copyright © 2009, Caringo, Inc.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holders nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
THE POSSIBILITY OF SUCH DAMAGE.
from Caringo, Inc. http://www.caringo.com/
CAStorSDK.jar http://www.castor.org/download.html
licensed under the BSD (3-clause) http://www.opensource.org/licenses/BSD-3-Clause (as follows)
Copyright © 2002-2011 Atsuhiko Yamanaka, JCraft,Inc.
@ -4118,7 +4150,8 @@ Within the ui/lib/jquery-ui directory
Within the ui/lib/qunit directory
licensed under the MIT License http://www.opensource.org/licenses/mit-license.php (as follows)
Copyright © 2012 John Resig, Jörn Zaefferer
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the

5
awsapi/pom.xml
View File

@ -94,6 +94,11 @@
<artifactId>jasypt</artifactId>
<version>${cs.jasypt.version}</version>
</dependency>
<dependency>
<groupId>com.caringo.client</groupId>
<artifactId>CAStorSDK</artifactId>
<version>1.3.1-CS40</version>
</dependency>
</dependencies>
<build>
<defaultGoal>install</defaultGoal>

4916
awsapi/resource/AmazonEC2/AmazonEC2.wsdl
View File

File diff suppressed because it is too large Load Diff

16
awsapi/resource/AmazonEC2/services.xml
View File

@ -1,4 +1,20 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- This file was auto-generated from WSDL -->
<!-- by the Apache Axis2 version: 1.5 Built on : Apr 30, 2009 (06:07:24 EDT) -->
<serviceGroup>

479
awsapi/src/com/cloud/bridge/io/S3CAStorBucketAdapter.java Normal file
View File

@ -0,0 +1,479 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package com.cloud.bridge.io;
import java.util.Arrays;
import java.util.HashSet;
import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.MalformedURLException;
import java.net.URL;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import javax.activation.DataHandler;
import javax.activation.DataSource;
import org.apache.log4j.Logger;
import com.cloud.bridge.service.core.s3.S3BucketAdapter;
import com.cloud.bridge.service.core.s3.S3MultipartPart;
import com.cloud.bridge.service.exception.ConfigurationException;
import com.cloud.bridge.service.exception.FileNotExistException;
import com.cloud.bridge.service.exception.InternalErrorException;
import com.cloud.bridge.service.exception.OutOfStorageException;
import com.cloud.bridge.service.exception.UnsupportedException;
import com.cloud.bridge.util.StringHelper;
import com.cloud.bridge.util.OrderedPair;
import com.caringo.client.locate.Locator;
import com.caringo.client.locate.StaticLocator;
import com.caringo.client.locate.ZeroconfLocator;
import com.caringo.client.ResettableFileInputStream;
import com.caringo.client.ScspClient;
import com.caringo.client.ScspExecutionException;
import com.caringo.client.ScspHeaders;
import com.caringo.client.ScspQueryArgs;
import com.caringo.client.ScspResponse;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.Header;
/**
* Creates an SCSP client to a CAStor cluster, configured in "storage.root",
* and use CAStor as the back-end storage instead of a file system.
*/
public class S3CAStorBucketAdapter implements S3BucketAdapter {
protected final static Logger s_logger = Logger.getLogger(S3CAStorBucketAdapter.class);
private static final int HTTP_OK = 200;
private static final int HTTP_CREATED = 201;
private static final int HTTP_UNSUCCESSFUL = 300;
private static final int HTTP_PRECONDITION_FAILED = 412;
// For ScspClient
private static final int DEFAULT_SCSP_PORT = 80;
private static final int DEFAULT_MAX_POOL_SIZE = 50;
private static final int DEFAULT_MAX_RETRIES = 5;
private static final int CONNECTION_TIMEOUT = 60 * 1000; // Request activity timeout - 1 minute
private static final int CM_IDLE_TIMEOUT = 60 * 1000; // HttpConnectionManager idle timeout - 1 minute
private static final int LOCATOR_RETRY_TIMEOUT = 0; // StaticLocator pool retry timeout
private ScspClient _scspClient; // talks to CAStor cluster
private Locator _locator; // maintains list of CAStor nodes
private String _domain; // domain where all CloudStack streams will live
private synchronized ScspClient myClient(String mountedRoot) {
if (_scspClient!=null) {
return _scspClient;
}
// The castor cluster is specified either by listing the ip addresses of some nodes, or
// by specifying "zeroconf=" and the cluster's mdns name -- this is "cluster" in castor's node.cfg.
// The "domain" to store streams can be specified. If not specified, streams will be written
// without a "domain" query arg, so they will go into the castor default domain.
// The port is optional and must be at the end of the config string, defaults to 80.
// Examples: "castor 172.16.78.130 172.16.78.131 80", "castor 172.16.78.130 domain=mycluster.example.com",
// "castor zeroconf=mycluster.example.com domain=mycluster.example.com 80"
String[] cfg = mountedRoot.split(" ");
int numIPs = cfg.length-1;
String possiblePort = cfg[cfg.length-1];
int castorPort = DEFAULT_SCSP_PORT;
try {
castorPort = Integer.parseInt(possiblePort);
--numIPs;
} catch (NumberFormatException nfe) {
// okay, it's an ip address, not a port number
}
if (numIPs <= 0) {
throw new ConfigurationException("No CAStor nodes specified in '" + mountedRoot + "'");
}
HashSet<String> ips = new HashSet<String>();
String clusterName = null;
for ( int i = 0; i < numIPs; ++i ) {
String option = cfg[i+1]; // ip address or zeroconf=mycluster.example.com or domain=mydomain.example.com
if (option.toLowerCase().startsWith("zeroconf=")) {
String[] confStr = option.split("=");
if (confStr.length != 2) {
throw new ConfigurationException("Could not parse cluster name from '" + option + "'");
}
clusterName = confStr[1];
} else if (option.toLowerCase().startsWith("domain=")) {
String[] confStr = option.split("=");
if (confStr.length != 2) {
throw new ConfigurationException("Could not parse domain name from '" + option + "'");
}
_domain = confStr[1];
} else {
ips.add(option);
}
}
if (clusterName == null && ips.isEmpty()) {
throw new ConfigurationException("No CAStor nodes specified in '" + mountedRoot + "'");
}
String[] castorNodes = ips.toArray(new String[0]); // list of configured nodes
if (clusterName == null) {
try {
_locator = new StaticLocator(castorNodes, castorPort, LOCATOR_RETRY_TIMEOUT);
_locator.start();
} catch (IOException e) {
throw new ConfigurationException("Could not create CAStor static locator for '" +
Arrays.toString(castorNodes) + "'");
}
} else {
try {
clusterName = clusterName.replace(".", "_"); // workaround needed for CAStorSDK 1.3.1
_locator = new ZeroconfLocator(clusterName);
_locator.start();
} catch (IOException e) {
throw new ConfigurationException("Could not create CAStor zeroconf locator for '" + clusterName + "'");
}
}
try {
s_logger.info("CAStor client starting: " + (_domain==null ? "default domain" : "domain " + _domain) + " " + (clusterName==null ? Arrays.toString(castorNodes) : clusterName) + " :" + castorPort);
_scspClient = new ScspClient(_locator, castorPort, DEFAULT_MAX_POOL_SIZE, DEFAULT_MAX_RETRIES, CONNECTION_TIMEOUT, CM_IDLE_TIMEOUT);
_scspClient.start();
} catch (Exception e) {
s_logger.error("Unable to create CAStor client for '" + mountedRoot + "': " + e.getMessage(), e);
throw new ConfigurationException("Unable to create CAStor client for '" + mountedRoot + "': " + e);
}
return _scspClient;
}
private String castorURL(String mountedRoot, String bucket, String fileName) {
// TODO: Replace this method with access to ScspClient's Locator,
// or add read method that returns the body as an unread
// InputStream for use by loadObject() and loadObjectRange().
myClient(mountedRoot); // make sure castorNodes and castorPort initialized
InetSocketAddress nodeAddr = _locator.locate();
if (nodeAddr == null) {
throw new ConfigurationException("Unable to locate CAStor node with locator " + _locator);
}
InetAddress nodeInetAddr = nodeAddr.getAddress();
if (nodeInetAddr == null) {
_locator.foundDead(nodeAddr);
throw new ConfigurationException("Unable to resolve CAStor node name '" + nodeAddr.getHostName() +
"' to IP address");
}
return "http://" + nodeInetAddr.getHostAddress() + ":" + nodeAddr.getPort() + "/" + bucket + "/" + fileName +
(_domain==null ? "" : "?domain=" + _domain);
}
private ScspQueryArgs domainQueryArg() {
ScspQueryArgs qa = new ScspQueryArgs();
if (this._domain != null)
qa.setValue("domain", this._domain);
return qa;
}
public S3CAStorBucketAdapter() {
// TODO: is there any way to initialize CAStor client here, can it
// get to config?
}
@Override
public void createContainer(String mountedRoot, String bucket) {
try {
ScspResponse bwResponse = myClient(mountedRoot).write(bucket, new ByteArrayInputStream("".getBytes()), 0, domainQueryArg(), new ScspHeaders());
if (bwResponse.getHttpStatusCode() != HTTP_CREATED) {
if (bwResponse.getHttpStatusCode() == HTTP_PRECONDITION_FAILED)
s_logger.error("CAStor unable to create bucket " + bucket + " because domain " +
(this._domain==null ? "(default)" : this._domain) + " does not exist");
else
s_logger.error("CAStor unable to create bucket " + bucket + ": " + bwResponse.getHttpStatusCode());
throw new OutOfStorageException("CAStor unable to create bucket " + bucket + ": " +
bwResponse.getHttpStatusCode());
}
} catch (ScspExecutionException e) {
s_logger.error("CAStor unable to create bucket " + bucket, e);
throw new OutOfStorageException("CAStor unable to create bucket " + bucket + ": " + e.getMessage());
}
}
@Override
public void deleteContainer(String mountedRoot, String bucket) {
try {
ScspResponse bwResponse = myClient(mountedRoot).delete("", bucket, domainQueryArg(), new ScspHeaders());
if (bwResponse.getHttpStatusCode() >= HTTP_UNSUCCESSFUL) {
s_logger.error("CAStor unable to delete bucket " + bucket + ": " + bwResponse.getHttpStatusCode());
throw new OutOfStorageException("CAStor unable to delete bucket " + bucket + ": " +
bwResponse.getHttpStatusCode());
}
} catch (ScspExecutionException e) {
s_logger.error("CAStor unable to delete bucket " + bucket, e);
throw new OutOfStorageException("CAStor unable to delete bucket " + bucket + ": " + e.getMessage());
}
}
@Override
public String saveObject(InputStream is, String mountedRoot, String bucket, String fileName)
{
// TODO: Currently this writes the object to a temporary file,
// so that the MD5 can be computed and so that we have the
// stream length needed by this version of CAStor SDK. Will
// change to calculate MD5 while streaming to CAStor and to
// either pass Content-length to this method or use newer SDK
// that doesn't require it.
FileOutputStream fos = null;
MessageDigest md5 = null;
try {
md5 = MessageDigest.getInstance("MD5");
} catch (NoSuchAlgorithmException e) {
s_logger.error("Unexpected exception " + e.getMessage(), e);
throw new InternalErrorException("Unable to get MD5 MessageDigest", e);
}
File spoolFile = null;
try {
spoolFile = File.createTempFile("castor", null);
} catch (IOException e) {
s_logger.error("Unexpected exception creating temporary CAStor spool file: " + e.getMessage(), e);
throw new InternalErrorException("Unable to create temporary CAStor spool file", e);
}
try {
String retVal;
int streamLen = 0;
try {
fos = new FileOutputStream(spoolFile);
byte[] buffer = new byte[4096];
int len = 0;
while( (len = is.read(buffer)) > 0) {
fos.write(buffer, 0, len);
streamLen = streamLen + len;
md5.update(buffer, 0, len);
}
//Convert MD5 digest to (lowercase) hex String
retVal = StringHelper.toHexString(md5.digest());
} catch(IOException e) {
s_logger.error("Unexpected exception " + e.getMessage(), e);
throw new OutOfStorageException(e);
} finally {
try {
if (null != fos)
fos.close();
} catch( Exception e ) {
s_logger.error("Can't close CAStor spool file " +
spoolFile.getAbsolutePath() + ": " + e.getMessage(), e);
throw new OutOfStorageException("Unable to close CAStor spool file: " + e.getMessage(), e);
}
}
try {
ScspResponse bwResponse =
myClient(mountedRoot).write(bucket + "/" + fileName,
new ResettableFileInputStream(spoolFile), streamLen,
domainQueryArg(), new ScspHeaders());
if (bwResponse.getHttpStatusCode() >= HTTP_UNSUCCESSFUL) {
s_logger.error("CAStor write responded with error " + bwResponse.getHttpStatusCode());
throw new OutOfStorageException("Unable to write object to CAStor " +
bucket + "/" + fileName + ": " + bwResponse.getHttpStatusCode());
}
} catch (ScspExecutionException e) {
s_logger.error("Unable to write object to CAStor " + bucket + "/" + fileName, e);
throw new OutOfStorageException("Unable to write object to CAStor " + bucket + "/" + fileName + ": " +
e.getMessage());
} catch (IOException ie) {
s_logger.error("Unable to write object to CAStor " + bucket + "/" + fileName, ie);
throw new OutOfStorageException("Unable to write object to CAStor " + bucket + "/" + fileName + ": " +
ie.getMessage());
}
return retVal;
} finally {
try {
if (!spoolFile.delete()) {
s_logger.error("Failed to delete CAStor spool file " + spoolFile.getAbsolutePath());
}
} catch (SecurityException e) {
s_logger.error("Unable to delete CAStor spool file " + spoolFile.getAbsolutePath(), e);
}
}
}
/**
* From a list of files (each being one part of the multipart upload), concatentate all files into a single
* object that can be accessed by normal S3 calls. This function could take a long time since a multipart is
* allowed to have upto 10,000 parts (each 5 gib long). Amazon defines that while this operation is in progress
* whitespace is sent back to the client inorder to keep the HTTP connection alive.
*
* @param mountedRoot - where both the source and dest buckets are located
* @param destBucket - resulting location of the concatenated objects
* @param fileName - resulting file name of the concatenated objects
* @param sourceBucket - special bucket used to save uploaded file parts
* @param parts - an array of file names in the sourceBucket
* @param client - if not null, then keep the servlet connection alive while this potentially long concatentation takes place
* @return OrderedPair with the first value the MD5 of the final object, and the second value the length of the final object
*/
@Override
public OrderedPair<String,Long> concatentateObjects(String mountedRoot, String destBucket, String fileName, String sourceBucket, S3MultipartPart[] parts, OutputStream client)
{
// TODO
throw new UnsupportedException("Multipart upload support not yet implemented in CAStor plugin");
/*
MessageDigest md5;
long totalLength = 0;
try {
md5 = MessageDigest.getInstance("MD5");
} catch (NoSuchAlgorithmException e) {
s_logger.error("Unexpected exception " + e.getMessage(), e);
throw new InternalErrorException("Unable to get MD5 MessageDigest", e);
}
File file = new File(getBucketFolderDir(mountedRoot, destBucket) + File.separatorChar + fileName);
try {
// -> when versioning is off we need to rewrite the file contents
file.delete();
file.createNewFile();
final FileOutputStream fos = new FileOutputStream(file);
byte[] buffer = new byte[4096];
// -> get the input stream for the next file part
for( int i=0; i < parts.length; i++ )
{
DataHandler nextPart = loadObject( mountedRoot, sourceBucket, parts[i].getPath());
InputStream is = nextPart.getInputStream();
int len = 0;
while( (len = is.read(buffer)) > 0) {
fos.write(buffer, 0, len);
md5.update(buffer, 0, len);
totalLength += len;
}
is.close();
// -> after each file write tell the client we are still here to keep connection alive
if (null != client) {
client.write( new String(" ").getBytes());
client.flush();
}
}
fos.close();
return new OrderedPair<String, Long>(StringHelper.toHexString(md5.digest()), new Long(totalLength));
//Create an ordered pair whose first element is the MD4 digest as a (lowercase) hex String
}
catch(IOException e) {
s_logger.error("concatentateObjects unexpected exception " + e.getMessage(), e);
throw new OutOfStorageException(e);
}
*/
}
@Override
public DataHandler loadObject(String mountedRoot, String bucket, String fileName) {
try {
return new DataHandler(new URL(castorURL(mountedRoot, bucket, fileName)));
} catch (MalformedURLException e) {
s_logger.error("Failed to loadObject from CAStor", e);
throw new FileNotExistException("Unable to load object from CAStor: " + e.getMessage());
}
}
@Override
public void deleteObject(String mountedRoot, String bucket, String fileName) {
String filePath = bucket + "/" + fileName;
try {
ScspResponse bwResponse = myClient(mountedRoot).delete("", filePath, domainQueryArg(), new ScspHeaders());
if (bwResponse.getHttpStatusCode() != HTTP_OK) {
s_logger.error("CAStor delete object responded with error " + bwResponse.getHttpStatusCode());
throw new OutOfStorageException("CAStor unable to delete object " + filePath + ": " +
bwResponse.getHttpStatusCode());
}
} catch (ScspExecutionException e) {
s_logger.error("CAStor unable to delete object " + filePath, e);
throw new OutOfStorageException("CAStor unable to delete object " + filePath + ": " + e.getMessage());
}
}
public class ScspDataSource implements DataSource {
GetMethod method;
public ScspDataSource(GetMethod m) {
method = m;
}
@Override
public String getContentType() {
Header h = method.getResponseHeader("Content-type");
return h==null ? null : h.getValue();
}
@Override
public InputStream getInputStream() throws IOException {
try {
return method.getResponseBodyAsStream();
} catch (Exception e) {
s_logger.error("CAStor loadObjectRange getInputStream error", e);
return null;
}
}
@Override
public String getName() {
assert(false);
return null;
}
@Override
public OutputStream getOutputStream() throws IOException {
assert(false);
return null;
}
}
@Override
public DataHandler loadObjectRange(String mountedRoot, String bucket, String fileName, long startPos, long endPos) {
try {
HttpClient httpClient = new HttpClient();
// Create a method instance.
GetMethod method = new GetMethod(castorURL(mountedRoot, bucket, fileName));
method.addRequestHeader("Range", "bytes=" + startPos + "-" + endPos);
int statusCode = httpClient.executeMethod(method);
if (statusCode < HTTP_OK || statusCode >= HTTP_UNSUCCESSFUL) {
s_logger.error("CAStor loadObjectRange response: "+ statusCode);
throw new FileNotExistException("CAStor loadObjectRange response: " + statusCode);
}
return new DataHandler(new ScspDataSource(method));
} catch (Exception e) {
s_logger.error("CAStor loadObjectRange failure", e);
throw new FileNotExistException("CAStor loadObjectRange failure: " + e);
}
}
@Override
public String getBucketFolderDir(String mountedRoot, String bucket) {
// This method shouldn't be needed and doesn't need to use
// mountedRoot (which is CAStor config values here), right?
String bucketFolder = getBucketFolderName(bucket);
return bucketFolder;
}
private String getBucketFolderName(String bucket) {
// temporary
String name = bucket.replace(' ', '_');
name = bucket.replace('\\', '-');
name = bucket.replace('/', '-');
return name;
}
}

4
awsapi/src/com/cloud/bridge/model/SHost.java
View File

@ -24,9 +24,11 @@ public interface SHost {
public static final int STORAGE_HOST_TYPE_LOCAL = 0;
public static final int STORAGE_HOST_TYPE_NFS = 1;
public static final int STORAGE_HOST_TYPE_CASTOR = 2;
public static enum StorageHostType {
STORAGE_HOST_TYPE_LOCAL, //0
STORAGE_HOST_TYPE_NFS //1
STORAGE_HOST_TYPE_NFS, //1
STORAGE_HOST_TYPE_CASTOR //2
}
/* private Long id;

23
awsapi/src/com/cloud/bridge/service/controller/s3/ServiceProvider.java
View File

@ -243,7 +243,13 @@ public class ServiceProvider {
//PersistContext.flush();
String localStorageRoot = properties.getProperty("storage.root");
if (localStorageRoot != null) setupLocalStorage(localStorageRoot);
if (localStorageRoot != null) {
if (localStorageRoot.toLowerCase().startsWith("castor")) {
setupCAStorStorage(localStorageRoot);
} else {
setupLocalStorage(localStorageRoot);
}
}
multipartDir = properties.getProperty("storage.multipartDir");
@ -318,7 +324,20 @@ public class ServiceProvider {
}
}
public void shutdown() {
private void setupCAStorStorage(String storageRoot) {
SHostVO shost = shostDao.getLocalStorageHost(mhost.getId(), storageRoot);
if(shost == null) {
shost = new SHostVO();
shost.setMhost(mhost);
shost.setMhostid(mhost.getId());
shost.setHostType(SHost.STORAGE_HOST_TYPE_CASTOR);
shost.setHost(NetHelper.getHostName());
shost.setExportRoot(storageRoot);
shostDao.persist(shost);
}
}
public void shutdown() {
timer.cancel();
if(logger.isInfoEnabled())

6
awsapi/src/com/cloud/bridge/service/core/s3/S3Engine.java
View File

@ -39,6 +39,7 @@ import org.apache.log4j.Logger;
import org.json.simple.parser.ParseException;
import com.cloud.bridge.io.S3FileSystemBucketAdapter;
import com.cloud.bridge.io.S3CAStorBucketAdapter;
import com.cloud.bridge.model.BucketPolicyVO;
import com.cloud.bridge.model.MHostMountVO;
import com.cloud.bridge.model.MHostVO;
@ -115,6 +116,7 @@ public class S3Engine {
public S3Engine() {
bucketAdapters.put(SHost.STORAGE_HOST_TYPE_LOCAL, new S3FileSystemBucketAdapter());
bucketAdapters.put(SHost.STORAGE_HOST_TYPE_CASTOR, new S3CAStorBucketAdapter());
}
@ -1398,6 +1400,10 @@ public class S3Engine {
return new OrderedPair<SHostVO, String>(shost, shost.getExportRoot());
}
if(shost.getHostType() == SHost.STORAGE_HOST_TYPE_CASTOR ) {
return new OrderedPair<SHostVO, String>(shost, shost.getExportRoot());
}
MHostMountVO mount = mountDao.getHostMount(ServiceProvider.getInstance().getManagementHostId(), shost.getId());
if(mount != null) {
return new OrderedPair<SHostVO, String>(shost, mount.getMountPath());

1
build/build-aws-api.xml
View File

@ -157,7 +157,6 @@
<jar jarfile="${dist.files.dir}/cloud-ec2.aar" basedir="${target.dir}/classes/cloud-awsapi.jar" excludes="**/*">
<metainf dir="${base.dir}/awsapi/resource/AmazonEC2">
<include name="services.xml" />
<include name="AmazonEC2.wsdl" />
</metainf>
</jar>
</target>

4
build/build-cloud-plugins.xml
View File

@ -194,8 +194,8 @@
<!-- ===================== Hypervisors ========================= -->
<target name="compile-hypervisors" depends="compile-ovm, compile-xen" description="Compile all hypervisors"/>
<target name="build-hypervisors" depends="build-ovm, build-xen " description="Builds all hypervisors"/>
<target name="compile-hypervisors" depends="compile-kvm, compile-ovm, compile-xen" description="Compile all hypervisors"/>
<target name="build-hypervisors" depends="build-kvm, build-ovm, build-xen " description="Builds all hypervisors"/>
<target name="compile-kvm" depends="-init, compile-core, compile-agent" description="Compile KVM">
<ant antfile="${base.dir}/plugins/hypervisors/kvm/build.xml" target="build"/>

5
client/pom.xml
View File

@ -73,6 +73,11 @@
<artifactId>cloud-plugin-hypervisor-ovm</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cloudstack</groupId>
<artifactId>cloud-plugin-hypervisor-kvm</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cloudstack</groupId>
<artifactId>cloud-plugin-storage-allocator-random</artifactId>

2
cloud.spec
View File

@ -538,6 +538,8 @@ fi
%files agent-libs
%defattr(0644,root,root,0755)
%{_javadir}/%{name}-agent.jar
%{_javadir}/%{name}-plugin-hypervisor-kvm.jar
%{_javadir}/libvirt-0.4.8.jar
%files agent
%defattr(0644,root,root,0755)

1
deps/install-non-oss.sh vendored
View File

@ -4,7 +4,6 @@ mvn install:install-file -Dfile=cloud-iControl.jar -DgroupId=com.cloud.com.
mvn install:install-file -Dfile=cloud-netscaler.jar -DgroupId=com.cloud.com.citrix -DartifactId=netscaler -Dversion=1.0 -Dpackaging=jar
mvn install:install-file -Dfile=cloud-netscaler-sdx.jar -DgroupId=com.cloud.com.citrix -DartifactId=netscaler-sdx -Dversion=1.0 -Dpackaging=jar
mvn install:install-file -Dfile=cloud-manageontap.jar -DgroupId=com.cloud.com.netapp -DartifactId=manageontap -Dversion=1.0 -Dpackaging=jar
mvn install:install-file -Dfile=libvirt-0.4.8.jar -DgroupId=org.libvirt -DartifactId=libvirt -Dversion=0.4.8 -Dpackaging=jar
mvn install:install-file -Dfile=vmware-vim.jar -DgroupId=com.cloud.com.vmware -DartifactId=vmware-vim -Dversion=1.0 -Dpackaging=jar
mvn install:install-file -Dfile=vmware-vim25.jar -DgroupId=com.cloud.com.vmware -DartifactId=vmware-vim25 -Dversion=1.0 -Dpackaging=jar
mvn install:install-file -Dfile=vmware-apputils.jar -DgroupId=com.cloud.com.vmware -DartifactId=vmware-apputils -Dversion=1.0 -Dpackaging=jar

5
deps/pom.xml vendored
View File

@ -73,6 +73,11 @@
<artifactId>cloud-plugin-hypervisor-ovm</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cloudstack</groupId>
<artifactId>cloud-plugin-hypervisor-kvm</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cloudstack</groupId>
<artifactId>cloud-plugin-storage-allocator-random</artifactId>

4
patches/systemvm/debian/buildsystemvm.sh
View File

@ -542,10 +542,10 @@ echo "*************CONFIGURING VPN********************"
vpn_config
echo "*************FIX DHCP ISSUE********************"
dhcp_fix
#dhcp_fix
echo "*************INSTALL XS TOOLS********************"
install_xs_tool
#install_xs_tool
echo "*************CLEANING UP********************"
cleanup

BIN
patches/systemvm/debian/iptables_1.4.8-3local1checksum1_i386.deb
View File

Binary file not shown.

BIN
patches/systemvm/debian/xe-guest-utilities_5.6.0-595_i386.deb
View File

Binary file not shown.

BIN
patches/systemvm/debian/xt_CHECKSUM.ko
View File

Binary file not shown.

6
plugins/hypervisors/kvm/pom.xml
View File

@ -27,6 +27,12 @@
<version>4.0.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>
<repositories>
<repository>
<id>libvirt-org</id>
<url>http://libvirt.org/maven2</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.apache.cloudstack</groupId>

1
plugins/pom.xml
View File

@ -37,6 +37,7 @@
<module>host-allocators/random</module>
<module>hypervisors/ovm</module>
<module>hypervisors/xen</module>
<module>hypervisors/kvm</module>
<module>network-elements/elastic-loadbalancer</module>
<module>network-elements/ovs</module>
<module>network-elements/nicira-nvp</module>

40
server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
View File

@ -444,37 +444,21 @@ public class VirtualNetworkApplianceManagerImpl implements VirtualNetworkApplian
}
@Override
public boolean savePasswordToRouter(Network network, NicProfile nic, VirtualMachineProfile<UserVm> profile, List<? extends VirtualRouter> routers) throws ResourceUnavailableException {
if (routers == null || routers.isEmpty()) {
s_logger.warn("Unable save password, router doesn't exist in network " + network.getId());
throw new CloudRuntimeException("Unable to save password to router");
}
public boolean savePasswordToRouter(Network network, final NicProfile nic, VirtualMachineProfile<UserVm> profile, List<? extends VirtualRouter> routers) throws ResourceUnavailableException {
_userVmDao.loadDetails((UserVmVO) profile.getVirtualMachine());
UserVm userVm = profile.getVirtualMachine();
String password = (String) profile.getParameter(Param.VmPassword);
String encodedPassword = PasswordGenerator.rot13(password);
DataCenter dc = _dcDao.findById(userVm.getDataCenterIdToDeployIn());
final VirtualMachineProfile<UserVm> updatedProfile = profile;
boolean result = true;
for (VirtualRouter router : routers) {
boolean sendPassword = true;
if (dc.getNetworkType() == NetworkType.Basic && userVm.getPodIdToDeployIn().longValue() != router.getPodIdToDeployIn().longValue()) {
sendPassword = false;
return applyRules(network, routers, "save password entry", false, null, false, new RuleApplier() {
@Override
public boolean execute(Network network, VirtualRouter router) throws ResourceUnavailableException {
// for basic zone, send vm data/password information only to the router in the same pod
Commands cmds = new Commands(OnError.Stop);
NicVO nicVo = _nicDao.findById(nic.getId());
createPasswordCommand(router, updatedProfile, nicVo, cmds);
return sendCommandsToRouter(router, cmds);
}
if (sendPassword) {
Commands cmds = new Commands(OnError.Continue);
SavePasswordCommand cmd = new SavePasswordCommand(encodedPassword, nic.getIp4Address(), userVm.getHostName());
cmd.setAccessDetail(NetworkElementCommand.ROUTER_IP, getRouterControlIp(router.getId()));
cmd.setAccessDetail(NetworkElementCommand.ROUTER_NAME, router.getInstanceName());
DataCenterVO dcVo = _dcDao.findById(router.getDataCenterIdToDeployIn());
cmd.setAccessDetail(NetworkElementCommand.ZONE_NETWORK_TYPE, dcVo.getNetworkType().toString());
cmds.addCommand("password", cmd);
result = result && sendCommandsToRouter(router, cmds);
}
}
return result;
});
}
@Override @ActionEvent(eventType = EventTypes.EVENT_ROUTER_STOP, eventDescription = "stopping router Vm", async = true)

32
tools/whisker/descriptor.xml
View File

@ -2400,6 +2400,10 @@ this distribution.
id='citrix.com'
name='Citrix Systems, Inc'
url='http://www.citrix.com/' />
<organisation
id='caringo.com'
name='Caringo, Inc.'
url='http://www.caringo.com/' />
</organisations>
<primary-license id='ApacheLicenseVersion2'>
<copyright-notice>Copyright (c) 2012 The Apache Software Foundation</copyright-notice>
@ -2516,6 +2520,7 @@ Updated November 2009 with contributions from: btburnett3, Anthony Aragues and X
</by-organisation>
</with-license>
<with-license id='MIT'>
<copyright-notice />
<by-organisation id='person:ole.laursen'>
<resource name="jquery.colorhelpers.js" notice='jquery.colorhelpers.notice' />
</by-organisation>
@ -2523,6 +2528,7 @@ Updated November 2009 with contributions from: btburnett3, Anthony Aragues and X
</within>
<within dir='ui/lib/jquery-ui'>
<with-license id='MIT'>
<copyright-notice />
<by-organisation id='jqueryui.com'>
<resource name="css/jquery-ui.css" />
<resource name="js/jquery-ui.js" />
@ -2531,10 +2537,10 @@ Updated November 2009 with contributions from: btburnett3, Anthony Aragues and X
</with-license>
</within>
<within dir='ui/lib/qunit'>
<copyright-notice>
Copyright © 2012 John Resig, Jörn Zaefferer
</copyright-notice>
<with-license id='MIT'>
<copyright-notice>
Copyright © 2012 John Resig, Jörn Zaefferer
</copyright-notice>
<by-organisation id='person:jorn.zaefferer'>
<resource name="qunit.css" source='http://docs.jquery.com/QUnit' />
<resource name="qunit.js" source='http://docs.jquery.com/QUnit' />
@ -2585,6 +2591,9 @@ Copyright © 2012 The Apache Software Foundation
</by-organisation>
</with-license>
<with-license id="ApacheLicenseVersion1.1">
<copyright-notice>
Copyright © 2012 The Apache Software Foundation
</copyright-notice>
<by-organisation id="apache.org">
<resource name="cloud-commons-discovery.jar" source="http://commons.apache.org/discovery/" />
</by-organisation>
@ -2640,6 +2649,14 @@ Copyright © 2012 The Eclipse Foundation.
</with-license>
<with-license id='BSD3ClauseGeneric'>
<copyright-notice>
Copyright © 2009, Caringo, Inc.
</copyright-notice>
<by-organisation id='caringo.com'>
<resource name='CAStorSDK.jar' source='http://www.castor.org/download.html' />
</by-organisation>
</with-license>
<with-license id='BSD3ClauseGeneric'>
<copyright-notice>
Copyright © 2002-2011 Atsuhiko Yamanaka, JCraft,Inc.
</copyright-notice>
<by-organisation id='jcraft.com'>
@ -2656,11 +2673,13 @@ Copyright © IBM Corp 2006
</by-organisation>
</with-license>
<with-license id='xstream.bsd'>
<copyright-notice />
<by-organisation id='xstream.codehaus.com'>
<resource name='cloud-xstream-1.3.1.jar' source='http://xstream.codehaus.org/repository.html' />
</by-organisation>
</with-license>
<with-license id='bouncy.mit'>
<copyright-notice />
<by-organisation id='bouncy.castle'>
<resource name='cloud-bcprov-jdk16-1.45.jar' source='http://repo1.maven.org/maven2/org/bouncycastle/bcprov-jdk16/1.45/bcprov-jdk16-1.45-sources.jar' />
</by-organisation>
@ -2675,12 +2694,14 @@ All rights reserved.
</by-organisation>
</with-license>
<with-license id='ApacheLicenseVersion2'>
<copyright-notice />
<by-organisation id='jetty.codehaus.com'>
<resource name='jetty-6.1.26.jar' source='http://repo1.maven.org/maven2/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26-sources.jar' />
<resource name='jetty-util-6.1.26.jar' source='http://repo1.maven.org/maven2/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26-sources.jar' />
</by-organisation>
</with-license>
<with-license id='CPL1'>
<copyright-notice />
<by-organisation id='junit.org'>
<resource name='cloud-junit.jar' source='http://kentbeck.github.com/junit/' />
</by-organisation>
@ -2731,6 +2752,7 @@ Copyright © 2004-2008 The Apache Software Foundation
</by-organisation>
</with-license>
<with-license id='antlr2'>
<copyright-notice />
<by-organisation id='antlr2.org'>
<resource name='antlr-2.7.6.jar' source='http://repo1.maven.org/maven2/antlr/antlr/2.7.6/antlr-2.7.6-sources.jar' />
</by-organisation>
@ -2788,6 +2810,7 @@ Copyright © 2009 Google Inc.
</by-organisation>
</with-license>
<with-license id='dom4j.license'>
<copyright-notice />
<by-organisation id='dom4j.sourceforge.net'>
<!-- TODO: Need to have a review of this license! -->
<resource name='dom4j-1.6.1.jar' source='http://dom4j.sourceforge.net/source-repository.html' />
@ -2820,6 +2843,7 @@ Copyright © 2002-2011 Atsuhiko Yamanaka, JCraft,Inc.
</by-organisation>
</with-license>
<with-license id='ApacheLicenseVersion2'>
<copyright-notice />
<by-organisation id='json-simple'>
<resource name='json_simple-1.1.jar' source='http://code.google.com/p/json-simple/source/checkout' />
</by-organisation>
@ -2833,11 +2857,13 @@ Copyright © 1997-2010 Oracle and/or its affiliates. All rights reserved.
</by-organisation>
</with-license>
<with-license id='CPL1'>
<copyright-notice />
<by-organisation id='junit.org'>
<resource name='junit-4.8.1.jar' source='http://kentbeck.github.com/junit/' />
</by-organisation>
</with-license>
<with-license id='MPL1'>
<copyright-notice />
<license-parameters>
<parameter><name>PROJECT</name><value>Javassist</value></parameter>
<parameter><name>INITIAL_DEVELOPER</name><value>Shigeru Chiba</value></parameter>

2
wscript_build
View File

@ -162,7 +162,7 @@ def build_dependences ():
start_path = bld.path.find_dir ("deps")
bld.install_files('${JAVADIR}',start_path.ant_glob(["commons-codec-1.6.jar", "ejb-api-3.0.jar", "xmlrpc-client-3.1.3.jar", "commons-dbcp-1.4.jar", "commons-pool-1.6.jar", "gson-1.7.1.jar",
bld.install_files('${JAVADIR}',start_path.ant_glob(["libvirt-0.4.8.jar", "commons-codec-1.6.jar", "ejb-api-3.0.jar", "xmlrpc-client-3.1.3.jar", "commons-dbcp-1.4.jar", "commons-pool-1.6.jar", "gson-1.7.1.jar",
"netscaler-1.0.jar", "netscaler-sdx-1.0.jar", "backport-util-concurrent-3.1.jar", "ehcache-1.5.0.jar", "httpcore-4.0.jar", "log4j-1.2.16.jar", "trilead-ssh2-build213-svnkit-1.3-patch.jar", "cglib-2.2.jar", "xmlrpc-common-3.*.jar",
"xmlrpc-client-3.*.jar", "axis-1.4.jar", "wsdl4j-1.6.2.jar", "bcprov-jdk16-1.46.jar", "jsch-0.1.42.jar", "jasypt-1.9.0.jar", "commons-configuration-1.8.jar", "commons-lang-2.6.jar", "mail-1.4.jar", "activation-1.1.jar", "mysql-connector-java-5.1.21.jar", "hibernate-jpa-2.0-api-1.0.0.Final.jar", "hibernate-entitymanager-3.5.1-Final.jar", "hibernate-core-3.5.1-Final.jar", "hibernate-commons-annotations-3.2.0.Final.jar", "hibernate-annotations-3.5.1-Final.jar", "asm-3.1.jar", "xapi-5.6.100-1-SNAPSHOT.jar"], excl = excludes), cwd=start_path)