Fix usage in ApiDispatcher. Add two kinds of helpers:
- One that gets list of exclude cmd whose fields are not be included
- One that loops till a base class is asssignable from superclass
Signed-off-by: Rohit Yadav <bhaisaab@apache.org>
The default data source assumption is that db server is on localhost, port 3306
and has user cloud and password cloud. The static variables rely on hardcoded
db.properties file only. We need to fix it
Signed-off-by: Rohit Yadav <bhaisaab@apache.org>
and CloudException in one place, and Introduced ApiErrorCode to handle CloudStack API error
code to standard Http code mapping.
Signed-off-by: Min Chen <min.chen@citrix.com>
Enhanced baremetal servers support on Cisco UCS
introduce an python etree like xml helper.
Ok, this is not a new wheel. Frankly speaking, all Java XML API just suc**.
there are two popular types of XML API in java, one class is for data binding, JAXB,
XStream fall into this category. Another class is tree based, like JDOM, XOM ...
for XML api call, data binding library is painful as you have to specify the schema
that how xml stream converts to java object, which means you have to pre-define all
schemas(xsd file for JAXB, java object for XStream ...). This is not productive, because you
must add new schema when XML document grows.
Tree based library shines in this case, for it's able to dynamically create an object tree
from xml stream without any knowledge of its structure. However, all tree based
XML API library fall into below convention:
Element e = root.getChildElement("child1").getChildElement("child2").getChildElement("child3")...getChildElement("childN")
anything wrong with it???
the sadness is if there is no "child2", you will get a NPE with above code, which means you have to judge
before getting.
And, why so verbose?? why not:
Element e = root.child1.child2.child3...childN ???
Ok I am joking, it's impossible in Java the world knows Java is a static language.
but you can actually do:
Element e = root.get("child1.child2.child3");
or
List<Element> e = root.getAsList("child1.child2.child3")
this is known as XPath style(though XPATH use '/'), python etree has supported it.
so I did this toy for my UCS xml api call, it's quite like etree which is easy to use, for example:
<components.xml>
<system-integrity-checker class="com.cloud.upgrade.DatabaseUpgradeChecker">
<checker name="ManagementServerNode" class="com.cloud.cluster.ManagementServerNode"/>
<checker name="EncryptionSecretKeyChecker" class="com.cloud.utils.crypt.EncryptionSecretKeyChecker"/>
<checker name="DatabaseIntegrityChecker" class="com.cloud.upgrade.DatabaseIntegrityChecker"/>
<checker name="DatabaseUpgradeChecker" class="com.cloud.upgrade.PremiumDatabaseUpgradeChecker"/>
</system-integrity-checker>
</components.xml>
XmlObject xo = XmlObjectParser.parseFromFile("~/components.xml.in");
List<XmlObject> checkers = xo.getAsList("system-integrity-checker.checker");
then you get a list of XmlObject which represent each 'checker' element:
XmlObject firstChecker = checkers.get(0);
// firstChecker.get("name") == "ManagementServerNode"
// firstChecker.get("class") == "com.cloud.cluster.ManagementServerNode"
// firstChecker.getTag() == "checker"
// firstChecker.getText() == "" if it's <checker/>xxx</checker>, then getText() == "xxx"
example 2:
<checker name="ManagementServerNode" class="com.cloud.cluster.ManagementServerNode"/>
<system-integrity-checker class="com.cloud.upgrade.DatabaseUpgradeChecker">
<checker name="ManagementServerNode" class="com.cloud.cluster.ManagementServerNode"/>
</system-integrity-checker>
</components.xml>
yout can do:
XmlObject xo = XmlObjectParser.parseFromFile("~/components.xml.in");
XmlObject checker = xo.get("system-integrity-checker.checker");
then it returns a single object as we only have one "checker" in xml stream,
or you still do
List<XmlObject> checkers = xo.getAsList("system-integrity-checker.checker");
it returns a list which only contains one element of "checker"
if you do:
XmlObject checker = xo.get("system-integrity-checker.checker.this_middle_element_doesnt_exist.some_element");
it returns a null without any exception, so you don't have to worry if a parent element is missing when getting a leaf element
again it's not a new wheel, I just hate JAVA xml api
Create OvsVifDriver to deal with openvswitch specifics for plugging
intefaces
Create a parameter to set the bridge type to use in
LibvirtComputingResource.
Create several functions to get bridge information from openvswitch
Add a check to detect the libvirt version and throw an exception when
the version is to low ( < 0.9.11 )
Fix classpath loading in Script.findScript to deal with missing path
separators at the end.
Add notification to the BridgeVifDriver that lswitch broadcast type is
not supported.
Due to generic programming, most classes declare Daos with ID as Long, so they
get the getUuid(Long) definition, it has to be getUuid(String), uuid is not
gonna be anything else. Fix GenericDaoBase and GenericDao.
Signed-off-by: Rohit Yadav <bhaisaab@apache.org>
Improve CglibThrowableRenderer test case
Log to a separate Logger instead of the default one to avoid spurious
stack traces in test run output.
Actually verify that registering CglibThrowableRenderer with the
alternative log hierarchy actually removes call trace lines that
contains the string <generated>
- Makes plugins self contained so they decide their properties file format
- PluggableService creates the contract that implementing entity will return a
properties map which is apiname:rolemask (both are strings)
Signed-off-by: Rohit Yadav <bhaisaab@apache.org>
If you update your build to build a version with a name not ending in -SNAPSHOT,
you are required to declare versions on all your depdendencies. There is already
a cs.mysql.version property, this patch makes sure it is used where appropriate.
Signed-off-by: Chip Childers <chip.childers@gmail.com>
- Fix interface to return array of strings, or filenames
- Fix StaticRoleBased ACL adapter to process config files by going through all pluggable services
- Refactor interface names
Signed-off-by: Rohit Yadav <bhaisaab@apache.org>
Multiple fixes:
1. changes to the mvn configuration
a. include simulator to client.war
b. activate simulator by profile
2. templates for simulator
3. developer prefill for simulator
a. Use deplydb-simulator to setup simulator db
4. Inherit components-simulator.xml from components.xml
5. ListVolumesCommand missed for MockStorageManager
6. Include simulator properties into utils/db.properties
TODO:
Secondary storage VMs don't come up because ComponentLocator doesn't
retain a unique set of adapaters by name. Fix this in subsequent
checkin.
- findByXId is read as find using some external Id (used by external network devices for ex.)
- what we really need is findByUuid that finds a unique VO using a uuid string
Signed-off-by: Rohit Yadav <bhaisaab@apache.org>
Declare and define findByXId() in GenericDao and EntityManager.
This method would return a VO by UUID and EntityManager's interface would
take in the VO Interface.
Signed-off-by: Rohit Yadav <bhaisaab@apache.org>
This was done for performance reasons.
I also refined the regex strings and added more test cases for different
string scenarios.
Signed-off-by: Chip Childers <chip.childers@gmail.com>
As noted in the bug, several of the API command in question
are async calls. I've added a simple regex-based string cleaning
function, and have the request and response strings running through
it prior to being appended to the audit log.
Unit tests added for the new cleaning function as well.
The call to skip logging the createSSHKeyPair response remains intact
for now, although it should probably be scrubbed similarly to the
password fields.
Signed-off-by: Chip Childers <chip.childers@gmail.com>
This is part 1 of list API refactoring. Commands covered:
listVmsCmd, listRoutersCmd Response covered:
UserVmResponse, DomainRouterResponse. DB views created:
user_vm_view, domain_router_view.
Signed-off-by: Rohit Yadav <bhaisaab@apache.org>
- Refactor VPN and VM APIs to admin and user pkgs
- Names space, org.apache.cloudstack
- Fix refactored apis in commands*.in
- Fix comments etc.
- Expand tabs, remove trailing whitespace
Signed-off-by: Rohit Yadav <bhaisaab@apache.org>
This change will allow the Script class to look for resources in the classpath of the webapp. This makes it possible to distribute the management server as a single prepackaged war. An added benefit is easier integration with IDE's that have the option to start webapps internally.
Also fixes a bug/feature in the URL handling were some components of the script path were translated to urlencoding. This change means that files are more often found in the first two steps of the findScript method which saves some filesystem calls.
ComponentLocator.
When using the extend option in the components-nonoss.xml the adapter
lsit was overwritten (putAll on the map), but it should be merged
instead. This prevented adapters from the parent to be loaded.
ability to do jetty:run in eclipse). This change breaks the ability to
upload scripts to XenServer (and possibly other places). Needs some
investigation before enabling this again.
Description:
Checking in fix for the exception seen due to the absence of
the AsyncCommandQueued exception in the map of CSExceptionErrorCode
which is used to look up the error code corresponding to a
cloudstack exception. Also printing the stack trace in case
an exception is not found in the map.
Description:
Checking in fix for the exception seen due to the absence of
the AsyncCommandQueued exception in the map of CSExceptionErrorCode
which is used to look up the error code corresponding to a
cloudstack exception. Also printing the stack trace in case
an exception is not found in the map.
Conflicts:
utils/src/com/cloud/utils/exception/CSExceptionErrorCode.java
policies. An error was getting reported during policy map creation that
config operation was in progress, Added synchronization to make sure
sending and receiving commands are seralized. Also removed the retry logic
as after this change it is not needed.
Reviewed-By: Vijay
Description:
Update create-schema.sql to create tables for
VSM and VSM-Cluster mapping.
Fixed an incorrect exception path in
CSExceptionErrorCode.
Conflicts:
utils/src/com/cloud/utils/exception/CSExceptionErrorCode.java
Description:
Adding a netconf helper class for adding and
deleting port profiles. These functions need
to be further parameterized and error handling
needs to be taken care of.
At this point, the mgmt server comes up, loading the
Nexus related modules without dying.
Description:
1) Added a new properties file for Cisco N1kv VSM commands:
cisconexusvsm_commands.properties.in
2) Added the CiscoNexusVSMElement to the components.xml file.
3) Modified CiscoNexusVSMElement to implement NetworkElement.
The NetworkElement interface functions are not
relevant to the N1KV VSM, so we override them
with noops.
4) Added an addDao() of CiscoNexusVSMDeviceDaoImpl in populateDaos(),
else we'd run into a failure to look up the VSM's dao when the
mgmt server is starting up:
com.cloud.utils.exception.CloudRuntimeException: Unable to find DAO com.cloud.network.dao.CiscoNexusVSMDeviceDao
5) Also added the CiscoNexusVSMElementService in populateServices(),
and modified CiscoNexusVSMElement to implement Manager as well.
6) populateServices() was running into an exception that indicated
that it was unable to find a commands.properties file for the
cisco n1kv vsm service. Fixed it by changing getProperties() in
CiscoNexusVSMElement to return the correct string
"cisconexusvsm_commands.properties", and putting in an @Override
for getProperties() in CiscoNexusVSMElement. Also fixed up all
the other functions in CiscoNexusVSMElement that needed to have
@Override. Also updated build/developers.xml with this file
location. And did other small cleanup.
7) More clean up in CiscoNexusVSMDeviceManagerImpl.
Conflicts:
server/src/com/cloud/configuration/DefaultComponentLibrary.java
Reviewed by: Sateesh Chodapuneedi, Devdeep Singh
Description:
This is the first in a series of commits for integrating the
Cloudstack Management Server with the Nexus 1000v Virtual
Supervisor Module.
These changes introduce the necessary API command interfaces
to work with a Cisco N1KV VSM. The backend logic is still to
be put in and will be incorporated in subsequent commits.
Please do not attempt to use these APIs until then. Also,
these are not yet filled in into commands.xml, so they are
not currently exposed.
Additional APIs would be added if required.
These changes will not break any current management server
functionality.
Given below is a description of the changes put in here:
Added Cisco N1KV commands to core/api:
These are the added commands -
AddCiscoNexusVSMCmd
DeleteCiscoNexusVSMCmd
ConfigureCiscoNexusVSMCmd
ListCiscoNexusVSMCmd
ListCiscoNexusVSMNetworksCmd
Added a Network Element service file for Cisco N1KV.
Declared the interface functions that we'll need for
the N1KV VSM.
Defined a DeviceVO file for the Cisco Nexus Element.
Created a response file for Cisco Nexus VSM.
Created new event types for external Switching Management devices.
Put in logic to call interface methods in ListCiscoNexusVSMNetworksCmd
and ListCiscoNexusVSMCmd
NOT VSM RELATED:
Fixed minor typo in some of the event types for external load balancers.
Added properties of a VSM in the VSM VO class.
Replaced the "url" input parameter by "ipaddress"
in the AddCiscoNexusVSMCmd API.
Added a new file - CiscoNexusVSMElement.java to
contain the implementation of the functions
declared in the VSMElementService interface, and
put in implementations of the functions for the
Nexus VSM API commands. These functions are
defined in the CiscoNexusVSMElement class.
Added a class for Port Profiles (PortProfile.java).
The fields in this class are still not correctly
declared as of now. We'll make the required changes
going forward.
Added CiscoNexusVSMDeviceManagerImpl class.
Added CiscoNexusVSMResource class.
Created a new class to provide a package to
connect to Cisco Nexus VSMs. This will be a
set of Java wrapper functions that allow us
to connect/disconnect and send commands and
receive the results of those commands via
XML-RPC. These functions are yet to be
implemented, and will be checked in in future
commits.
Added two new classes, VSMCommand and
VSMResponse, to encapsulate XML-RPCcommands
and responses to and from a Ciscon Nexus VSM.
Put in the following function stubs inside the
CiscoNexusVSMService class:
connectToVSM()
disconnectFromVSM()
executeVSMCommand()
Added new field in the Type enum of the "Host"
interface, for Cisco Nexus VSMs.
Added two parameters to AddCiscoNexusVSMCommand
vsmName
zoneId
Modified the CiscoNexusVSMDeviceVO constructor to
take in an zoneId as a parameter when creating
the VO object.
Added new interface and class for the DeviceDao
implementation for Cisco Nexus VSM devices:
CiscoNexusVSMDeviceDao
CiscoNexusVSMDeviceDaoImpl
Removed the vsmvCenterDomainId property, since it's
going to the same as vsmDomainId, which is the VSM's
switch Domain Id.
Have started putting in the following query functions
in the CiscoNexusVSMDeviceDao interface:
Put in DAO implementations of some of the above functions in the CiscoNexusVSMDeviceDaoImpl class.
Added a vsmName parameter to the CiscoNexusVSMDeviceVO class.
Atomatic addition of Apache license header broke lots of mysql files
fix DB upgrade cleanup script not get executed issue.
add "#" as comment signature in ScriptRunner.java
Description:
Adding overloaded addProxyObject() function to CloudException
and RuntimeCloudException classes and using this function
to stuff exceptions with IDs, to reduce code footprint.
Description:
Adding a new class AnnotationHelper, that provides routines
to read annotations from a VO class.
Cloudstack does the mapping between cloudstack java VO objects
and the database tables using cglib. cglib creates proxy objects
as the maps between VO objects and the database. A VO (value
object) class is populated after querying from the MYSQL database.
Ideally, a getAnnotation() issued on a cglib proxy object should
get a list of all the annotations in all classes in the inheritence
chain. However, this functionality seems to be currently broken
in cglib. Hence, when querying for annotations given the object
of a VO class, we need to get to the corresponding VO class of that
cglib proxy class and issue a getAnnotation() on that class. To get
the VO class, we simply need to get the super class of the proxy
object. Also, we need to recurse to the root VO class in case the
VO class extends another VO class.
Note that the cache used by CS is ehcache.
Bug 13127: API error text refer to database ids instead of uuids
Code-Reviewers: Ewan Mellor, Kelven Yang
Description:
1. A new class CSExceptionErrorCode has been added to utils.
It contains a list of error codes for each type of
Exception class. Use fully qualified package paths for
Exceptions in CSExceptionErrorCode. We log any exception
name not found in the list of error codes for exceptions.
2. Whenever we throw an exception exobj anywhere in the
CS code, the CSErrorCode is set in the base class
constructor.
3. We add a new field csErrorCode in classes CloudException,
RuntimeCloudException, ExecutionException and
ExceptionResponse.
4. Two places in ApiServer.java were wrongly modified when
putting in changes for bug 13127 to not throw an exception.
This has been corrected in this commit.
Description:
Modify Exception handling to enable addition of multiple
uuids in a single exception thrown by API functions. Both
XML and JSON outputs will store all uuids and Fieldnames.
This will make it easier to provide more information when
an exception occurs - for example, a zone id, a cluster id,
host id, and then a specific property id.
Description:
Added a field name for the db id in the IdentityProxy class, and
modified setProxyObject() to take an additional id name parameter.
This will let us know the name of the uuid that we are returning.
E.g.- domainId, zoneId, etc. The client can view this field in
the json/xml output. Modified the JSON/XML serialization routines
to append this new parameter to the serialized output for Exception
Responses.
Description:
1) Added a setProxyObject() method to CloudException and RuntimeCloudException
2) Modified a bunch of throw exceptions in NetworkManagerImpl.java to call setProxyObject() before throwing an exception.
3) Changed scope of ProxyIdentity attribute to protected.
4) Added routines to ServerApiException to get/set IdentityProxy object, and
routine in RuntimeCloudException to get the Idproxy object.
5) Modified the exception handling around the dispatcher and handlerequest()
to copy over the IdentityProxy information before rethrowing an exception
eventually back to handle().
6) Removed duplicate IdentityProxy object in ServerApiException.
It was extending RuntimeCloudException which already had an
IdentityProxy object.
Description:
1) Moved RuntimeCloudException from api/ to utils/.
Added simple constructor to RuntimeCloudException.
Modified all classes that extended RuntimeException
to extend RuntimeCloudException. These classes
are listed below:
ServerApiException
CloudAuthenticationException
CloudExecutionException
AsyncCommandQueued
HypervisorVersionChangedException
RuntimeCloudException
2) Added overloaded constructed to CloudException.
Modified all classes that extend Exception to extend CloudException instead.
These classes are listed below:
ConcurrentOperationException
ConflictingNetworkSettingsException
ConnectionException
DiscoveryException
InsufficientCapacityException
ManagementServerException
ResourceUnavailableException
VirtualMachineMigrationException
AgentControlChannelException
OperationTimedoutException.java
UnsupportedVersionException.java
UsageServerException.java
UnableDeleteHostException.java
AgentAuthnException.java
HttpCallException.java
ActiveFencingException.java
ClusterInvalidSessionException.java
GreTunnelException.java
OvsVlanExhaustedException.java
Description:
1) Adding two new classes, CloudException and RunTimeCloudException.
The former extends Exception and the latter RunTimeException.
These will be used by classes that formerly directly extended
Exception and RuntimeException. These two classes have an attribute
of type IdentityProxy to enable exceptions fill in db ids in separate
attribute fields rather than in a string. Doing so will allow the
serialization module (GSON for JSON and other for XML) to kick in
and convert this db id to a uuid in ApiServer.java just before the
JSON/XML responses are sent out.
2) Moving IdentityProxy.java from api/ to utils/ since
both CloudException and RuntimeCloudException refer to it.
3) Changing references to IdentityProxy class from api/ to utils/.
4) While rebasing to master, a new file was added, merging
api/src/com/cloud/api/response/IsoVmResponse.java to this diff.
using a flag instead of calculating timeout value. The monitor thread may have inaccurate execution time which
may cause script forever hang
status 13962: resolved fixed
reviewed-by: Alex, Edison
Changes for Script.java:
1. Even the script is non-timeout one, set default timeout to one hour. This avoid a wrongn script forever hang
2. When InterruptedException happens, check if timeout is really reached, if not, continue
status 13962: resolved fixed
reviewed-by: Alex, Edison
It’s due to an security fix of OpenJDK 1.6.0 added by Redhat. Here is excerpt
of [RHSA-2011:1380-01] Critical: java-1.6.0-openjdk security update(
https://www.redhat.com/archives/rhsa-announce/2011-October/msg00011.html)
A flaw was found in the way the SSL 3 and TLS 1.0 protocols used block
ciphers in cipher-block chaining (CBC) mode. An attacker able to perform a
chosen plain text attack against a connection mixing trusted and untrusted
data could use this flaw to recover portions of the trusted data sent over
the connection. (CVE-2011-3389)
Note: This update mitigates the CVE-2011-3389 issue by splitting the first
application data record byte to a separate SSL/TLS protocol record. This
mitigation may cause compatibility issues with some SSL/TLS implementations
and can be disabled using the jsse.enableCBCProtection boolean property.
This can be done on the command line by appending the flag
"-Djsse.enableCBCProtection=false" to the java command.
To our knowledge, there are two condition need to be met to trigger this bug:
1. Using old keystore generated by mgmt. server 2.2.8, which is signed with
SHA1withDSA. Any version later than 2.2.8 would generate keystore signed with
SHA1withRSA. RSA one seems fine with us so far.
2. Use OpenJDK >=1.6.0.
The reason is, due to the security fix above, the assumption that one packet
would contain only one SSL record is broken. The decrypted data maybe only
contained the first byte of original application data. Then result in buffer
underflow when mgmt server want to read more from it.
To workaround it, according to the message above, add
"-Djsse.enableCBCProtection=false" to tomcat6.conf JAVA_OPTS line would work.
Notice the parameter would only work with latest version of OpenJDK, so simply
add it to the all setup would not work.
This patch provided a fix for it.
status 11904: resolved fixed
Alex Huang
Kelven Yang
Rohit Yadav
Edison Su
Hugo Trippaers
frank
Min Chen
Noa Resare
Hugo Trippaers
Noa Resare
John Burwell
Prasanna Santhanam
Chip Childers
Laszlo Hornyak
Likitha Shetty
adefanti
Vijay Venkatachalam
Edison Su
Kelven Yang
Kishan Kavala
Alena Prokharchyk
Manikanta Kattamuri
Mice Xia
Rajesh Battala
Darren Shepherd
olivier lamy
Darren Shepherd
Darren Shepherd
Chiradeep Vittal
Mice Xia
Sheng Yang
David Nalley
Murali reddy
Vijayendra Bhamidipati
Devdeep Singh
Sateesh Chodapuneedi
U-CITRITE\vijayendra1
Kelven Yang
prachi
Sheng Yang
alena
Edison Su
Abhinandan Prateek